github/dokku
0
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Merge pull request #3241 from dokku/failed-deploy-logs

Browse Source 
feat: add a subcommand for retrieving failed app deploy logs
This commit is contained in:
Jose Diaz-Gonzalez
2018-07-29 20:28:15 -04:00
committed by GitHub
parent 8b932971df db29029653
commit f80b152c9b
13 changed files with 239 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docs/advanced-usage/schedulers/docker-local.md
View File

@@ -25,7 +25,8 @@ This plugin implements various functionality through `plugn` triggers to integra
- `pre-restore`
- `scheduler-deploy`
- `scheduler-docker-cleanup`
- `scheduler-logs-failed`
- `scheduler-run`
- `scheduler-stop`
- `scheduler-tags-create`
- `scheduler-tags-destroy`
- `scheduler-tags-destroy`

29
docs/deployment/logs.md
View File

@@ -1,19 +1,21 @@
# Log Management
```
logs <app> # display recent log output
logs <app> [-h] [-t] [-n num] [-q] [-p process] # Display recent log output
logs:failed [<app>] # Shows the last failed deploy logs
```
## Usage
### Application Logs
You can easily get logs of an application using the `logs` command:
### App Logs
You can easily get logs of an app using the `logs` command:
```shell
dokku logs node-js-app
```
## Behavioral modifiers
#### Behavioral modifiers
Dokku also supports certain command-line arguments that augment the `log` command's behavior.
@@ -26,8 +28,25 @@ Dokku also supports certain command-line arguments that augment the `log` comman
You can use these modifiers as follows:
```shell
dokku logs node-js-app -t -p web
```
will show logs continually from the web process.
### Failed Deploy Logs
> Warning: The default `docker-local` scheduler will "store" these until the next deploy or until the old containers are garbage collected - whichever runs first. If you require the logs beyond this point in time, please ship the logs to a centralized log server.
In some cases, it may be useful to retrieve the logs from a previously failed deploy.
You can retrieve these logs by using the `logs:failed` command
```shell
dokku logs:failed node-js-app
```
You can also retrieve the failed logs for each app:
```shell
dokku logs:failed node-js-app
```

49
docs/development/plugin-triggers.md
View File

@@ -1177,9 +1177,12 @@ APP="$1";
### `scheduler-deploy`
> Warning: The scheduler plugin trigger apis are under development and may change
> between minor releases until the 1.0 release.
- Description: Allows you to run scheduler commands when an app is deployed
- Invoked by: `dokku deploy`
- Arguments: `$DOKKU_SCHEDULER APP IMAGE_TAG`
- Arguments: `$DOKKU_SCHEDULER $APP $IMAGE_TAG`
- Example:
```shell
@@ -1193,9 +1196,12 @@ DOKKU_SCHEDULER="$1"; APP="$2"; IMAGE_TAG="$3";
### `scheduler-docker-cleanup`
> Warning: The scheduler plugin trigger apis are under development and may change
> between minor releases until the 1.0 release.
- Description: Allows you to run scheduler commands when dokku cleanup is invoked
- Invoked by: `dokku deploy, dokku cleanup`
- Arguments: `$DOKKU_SCHEDULER APP FORCE_CLEANUP`
- Arguments: `$DOKKU_SCHEDULER $APP $FORCE_CLEANUP`
- Example:
```shell
@@ -1207,11 +1213,33 @@ DOKKU_SCHEDULER="$1"; APP="$2"; FORCE_CLEANUP="$3";
# TODO
```
### `scheduler-logs-failed`
> Warning: The scheduler plugin trigger apis are under development and may change
> between minor releases until the 1.0 release.
- Description: Allows you to run scheduler commands when retrieving failed container logs
- Invoked by: `dokku logs:failed`
- Arguments: `$DOKKU_SCHEDULER $APP`
- Example:
```shell
#!/usr/bin/env bash
set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x
DOKKU_SCHEDULER="$1"; APP="$2"; ARGS="${@:3}";
# TODO
```
### `scheduler-run`
> Warning: The scheduler plugin trigger apis are under development and may change
> between minor releases until the 1.0 release.
- Description: Allows you to run scheduler commands when a command is executed for your app
- Invoked by: `dokku run`
- Arguments: `$DOKKU_SCHEDULER APP ...ARGS`
- Arguments: `$DOKKU_SCHEDULER $APP ...ARGS`
- Example:
```shell
@@ -1225,9 +1253,12 @@ DOKKU_SCHEDULER="$1"; APP="$2"; ARGS="${@:3}";
### `scheduler-stop`
> Warning: The scheduler plugin trigger apis are under development and may change
> between minor releases until the 1.0 release.
- Description: Allows you to run scheduler commands when a tag is destroyed
- Invoked by: `dokku apps:destroy, dokku ps:stop`
- Arguments: `$DOKKU_SCHEDULER APP REMOVE_CONTAINERS`
- Arguments: `$DOKKU_SCHEDULER $APP $REMOVE_CONTAINERS`
- Example:
```shell
@@ -1241,9 +1272,12 @@ DOKKU_SCHEDULER="$1"; APP="$2"; REMOVE_CONTAINERS="$3";
### `scheduler-tags-create`
> Warning: The scheduler plugin trigger apis are under development and may change
> between minor releases until the 1.0 release.
- Description: Allows you to run scheduler commands when a tag is created
- Invoked by: `dokku tags:create`
- Arguments: `$DOKKU_SCHEDULER APP SOURCE_IMAGE TARGET_IMAGE`
- Arguments: `$DOKKU_SCHEDULER $APP $SOURCE_IMAGE $TARGET_IMAGE`
- Example:
```shell
@@ -1257,9 +1291,12 @@ DOKKU_SCHEDULER="$1" APP="$2" SOURCE_IMAGE="$3" TARGET_IMAGE="$4"
### `scheduler-tags-destroy`
> Warning: The scheduler plugin trigger apis are under development and may change
> between minor releases until the 1.0 release.
- Description: Allows you to run scheduler commands when a tag is destroyed
- Invoked by: `dokku tags:destroy`
- Arguments: `$DOKKU_SCHEDULER APP IMAGE_REPO IMAGE_TAG`
- Arguments: `$DOKKU_SCHEDULER $APP $IMAGE_REPO $IMAGE_TAG`
- Example:
```shell

2
dokku
View File

@@ -42,6 +42,8 @@ export PLUGIN_ENABLED_PATH=${PLUGIN_ENABLED_PATH:="$PLUGIN_PATH/enabled"}
export PLUGIN_CORE_PATH=${PLUGIN_CORE_PATH:="$DOKKU_LIB_ROOT/core-plugins"}
export PLUGIN_CORE_AVAILABLE_PATH=${PLUGIN_CORE_AVAILABLE_PATH:="$PLUGIN_CORE_PATH/available"}
export PLUGIN_CORE_ENABLED_PATH=${PLUGIN_CORE_ENABLED_PATH:="$PLUGIN_CORE_PATH/enabled"}
export DOKKU_SYSTEM_GROUP=${DOKKU_SYSTEM_GROUP:="dokku"}
export DOKKU_SYSTEM_USER=${DOKKU_SYSTEM_USER:="dokku"}
export DOKKU_API_VERSION=1
export DOKKU_NOT_IMPLEMENTED_EXIT=10

11
plugins/common/property-functions
View File

@@ -67,15 +67,6 @@ fn-plugin-property-setup() {
mkdir -p "${DOKKU_LIB_ROOT}/config/${PLUGIN}"
chmod 0755 "${DOKKU_LIB_ROOT}/config/${PLUGIN}"
if [[ -z "$DOKKU_SYSTEM_GROUP" ]]; then
export DOKKU_SYSTEM_GROUP=dokku
fi
if [[ -z "$DOKKU_SYSTEM_USER" ]]; then
export DOKKU_SYSTEM_USER=dokku
fi
chown "${DOKKU_SYSTEM_USER}:${DOKKU_SYSTEM_GROUP}" "${DOKKU_LIB_ROOT}/config/${PLUGIN}"
}
@@ -86,7 +77,7 @@ fn-plugin-property-read() {
if [[ -z "$PLUGIN" ]] || [[ -z "$APP" ]] || [[ -z "$KEY" ]]; then
return 1
fi
if [[ ! -f "${DOKKU_LIB_ROOT:?}/config/${PLUGIN}/${APP}/${KEY}" ]]; then
return 1
fi

22
plugins/logs/commands
View File

@@ -1,27 +1,11 @@
#!/usr/bin/env bash
[[ " help logs:help " == *" $1 "* ]] || exit "$DOKKU_NOT_IMPLEMENTED_EXIT"
source "$PLUGIN_AVAILABLE_PATH/logs/internal-functions"
set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x
usage() {
declare desc="logs specific usage"
echo "Usage: dokku logs <app>"
echo " display recent log output"
echo ""
echo " -n, --num NUM # the number of lines to display"
echo " -p, --ps PS # only display logs from the given process"
echo " -t, --tail # continually stream logs"
echo " -q, --quiet # display raw logs without colors, time and names"
}
case "$1" in
logs:help)
usage
;;
help)
cat<<EOF
logs <app> [-h] [-t] [-n num] [-q] [-p process], Show the last logs for an application
EOF
help | logs:help)
cmd-logs-help "$@"
;;
*)

38
plugins/logs/internal-functions Executable file
View File

@@ -0,0 +1,38 @@
#!/usr/bin/env bash
set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x
fn-logs-usage() {
declare desc="logs specific usage"
echo "Usage: dokku logs <app>"
echo " display recent log output"
echo ""
echo " -n, --num NUM # the number of lines to display"
echo " -p, --ps PS # only display logs from the given process"
echo " -t, --tail # continually stream logs"
echo " -q, --quiet # display raw logs without colors, time and names"
}
fn-logs-help-content() {
declare desc="return logs plugin help content"
cat<<help_content
logs <app> [-h] [-t] [-n num] [-q] [-p process], Display recent log output
logs:failed <app>, Show the last logs for an application
help_content
}
cmd-logs-help() {
if [[ $1 = "logs:help" ]] ; then
echo -e 'Usage: dokku logs[:COMMAND]'
echo ''
echo 'Output app logs'
echo ''
echo 'Additional commands:'
fn-logs-help-content | sort | column -c2 -t -s,
elif [[ $(ps -o command= $PPID) == *"--all"* ]]; then
fn-logs-help-content
else
cat<<help_desc
logs, Output app logs
help_desc
fi
}

12
plugins/logs/subcommands/default
View File

@@ -1,24 +1,22 @@
#!/usr/bin/env bash
set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x
source "$PLUGIN_CORE_AVAILABLE_PATH/common/functions"
source "$PLUGIN_AVAILABLE_PATH/logs/internal-functions"
logs_default_cmd() {
declare desc="shows the last logs for an app via command line"
declare desc="display recent log output"
local cmd="logs"
[[ -z $2 ]] && dokku_log_fail "Please specify an app to run the command on"
local APP="$2"; verify_app_name "$2"
local APP_ROOT="$DOKKU_ROOT/$APP"
local COLORS=(36 33 32 35 31)
if ! (is_deployed "$APP"); then
echo "Application's container not found"
exit 1
fi
! (is_deployed "$APP") && dokku_log_fail "App $APP has not been deployed"
shift 2;
local TEMP=$(getopt -o htqn:p: --long help,tail,quiet,num:,ps: -n 'dokku logs' -- "$@")
local EXIT_CODE="$?"
if [[ "$EXIT_CODE" != 0 ]]; then usage >&2 ; exit 1 ; fi
if [[ "$EXIT_CODE" != 0 ]]; then fn-logs-usage >&2 ; exit 1 ; fi
eval set -- "$TEMP"
local DOKKU_LOGS_LINE_NUMBERS="100"
@@ -29,7 +27,7 @@ logs_default_cmd() {
-p|--ps) local DOKKU_LOGS_ONLY_PROCESS="$2"; shift 2 ;;
-q|--quiet) local DOKKU_LOGS_NO_PRETTY_PRINT="true"; shift ;;
--) shift; break ;;
*) echo "Internal error"; exit 1;;
*) dokku_log_fail "Internal error";;
esac
done

33
plugins/logs/subcommands/failed Executable file
View File

@@ -0,0 +1,33 @@
#!/usr/bin/env bash
source "$PLUGIN_CORE_AVAILABLE_PATH/common/functions"
source "$PLUGIN_AVAILABLE_PATH/apps/functions"
source "$PLUGIN_AVAILABLE_PATH/config/functions"
set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x
cmd-logs-failed() {
declare desc="shows the last failed deploy logs"
declare cmd="logs:failed" argv=("$@"); [[ ${argv[0]} == "$cmd" ]] && shift 1
declare APP="$1"
local DOKKU_SCHEDULER
if [[ -z "$APP" ]]; then
local INSTALLED_APPS=$(dokku_apps)
for app in $INSTALLED_APPS; do
fn-logs-failed-single "$app" | tee || true
done
else
fn-logs-failed-single "$APP"
fi
}
fn-logs-failed-single() {
declare desc="shows the last failed deploy logs for an app"
declare APP="$1"
local DOKKU_SCHEDULER
dokku_log_info2_quiet "${APP} failed deploy logs"
DOKKU_SCHEDULER=$(config_get "$APP" DOKKU_SCHEDULER || echo "docker-local")
plugn trigger scheduler-logs-failed "$DOKKU_SCHEDULER" "$APP"
}
cmd-logs-failed "$@"

12
plugins/scheduler-docker-local/install Executable file
View File

@@ -0,0 +1,12 @@
#!/usr/bin/env bash
set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x
scheduler-docker-local-install() {
declare desc="scheduler-docker-local install plugin trigger"
declare trigger="scheduler-docker-local-install"
mkdir -p "${DOKKU_LIB_ROOT}/data/scheduler-docker-local"
chown -R "${DOKKU_SYSTEM_USER}:${DOKKU_SYSTEM_GROUP}" "${DOKKU_LIB_ROOT}/data/scheduler-docker-local"
}
scheduler-docker-local-install "$@"

13
plugins/scheduler-docker-local/post-create Executable file
View File

@@ -0,0 +1,13 @@
#!/usr/bin/env bash
set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x
scheduler-docker-local-post-create() {
declare desc="scheduler-docker-local post-create plugin trigger"
declare trigger="scheduler-docker-local-post-create"
declare APP="$1"
mkdir -p "${DOKKU_LIB_ROOT}/data/scheduler-docker-local/${APP}"
chown -R "${DOKKU_SYSTEM_USER}:${DOKKU_SYSTEM_GROUP}" "${DOKKU_LIB_ROOT}/data/scheduler-docker-local/${APP}"
}
scheduler-docker-local-post-create "$@"

10
plugins/scheduler-docker-local/scheduler-deploy
View File

@@ -13,6 +13,8 @@ scheduler-docker-local-scheduler-deploy() {
return
fi
rm -f "${DOKKU_LIB_ROOT}/data/scheduler-docker-local/$APP/failed-containers"
local DOKKU_DOCKER_STOP_TIMEOUT DOKKU_HEROKUISH DOKKU_NETWORK_BIND_ALL IMAGE
DOKKU_HEROKUISH=false
IMAGE=$(get_deploying_app_image_name "$APP" "$IMAGE_TAG")
@@ -96,14 +98,16 @@ scheduler-docker-local-scheduler-deploy() {
kill_new() {
declare desc="wrapper function to kill newly started app container"
local cid="$1"
docker inspect "$cid" &> /dev/null && docker stop "$cid" > /dev/null && docker kill "$cid" &> /dev/null
declare CID="$1" PROC_TYPE="$2" CONTAINER_INDEX="$3"
docker inspect "$CID" &> /dev/null && docker stop "$CID" > /dev/null && docker kill "$CID" &> /dev/null
mkdir -p "${DOKKU_LIB_ROOT}/data/scheduler-docker-local/$APP"
echo "${CID} ${PROC_TYPE}.${CONTAINER_INDEX}" >> "${DOKKU_LIB_ROOT}/data/scheduler-docker-local/$APP/failed-containers"
trap - INT TERM EXIT
kill -9 $$
}
# run checks first, then post-deploy hooks, which switches proxy traffic
trap 'kill_new $cid' INT TERM EXIT
trap 'kill_new $cid $PROC_TYPE $CONTAINER_INDEX' INT TERM EXIT
if [[ "$(is_app_proctype_checks_disabled "$APP" "$PROC_TYPE")" == "false" ]]; then
dokku_log_info1 "Attempting pre-flight checks"
plugn trigger check-deploy "$APP" "$cid" "$PROC_TYPE" "$port" "$ipaddr"

56
plugins/scheduler-docker-local/scheduler-logs-failed Executable file
View File

@@ -0,0 +1,56 @@
#!/usr/bin/env bash
set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x
source "$PLUGIN_CORE_AVAILABLE_PATH/common/functions"
scheduler-docker-local-scheduler-logs-failed() {
declare desc="scheduler-docker-local scheduler-logs-failed plugin trigger"
declare trigger="scheduler-docker-local scheduler-logs-failed"
declare DOKKU_SCHEDULER="$1" APP="$2"
local FAILED_CONTAINERS_FILE="${DOKKU_LIB_ROOT}/data/scheduler-docker-local/$APP/failed-containers"
local DEAD_CONTAINERS=() DOKKU_LOGS_CMD=() RUNNING_CONTAINERS=()
local CID LINE PREFIX
shift 2
if [[ "$DOKKU_SCHEDULER" != "docker-local" ]]; then
return
fi
if [[ ! -f "$FAILED_CONTAINERS_FILE" ]]; then
dokku_log_warn "No failed containers found"
return
fi
while read -r LINE || [[ -n "$LINE" ]]; do
CID="$(echo "$LINE" | cut -d ' ' -f1)"
PREFIX="$(echo "$LINE" | cut -d ' ' -f2)"
if docker inspect "${CID}" > /dev/null 2>&1; then
RUNNING_CONTAINERS+=("$CID")
else
DEAD_CONTAINERS+=("$CID")
fi
done < "$FAILED_CONTAINERS_FILE"
for CID in "${DEAD_CONTAINERS[@]}"; do
dokku_log_warn "App container $CID no longer running"
sed -i "/${CID}/d" "$FAILED_CONTAINERS_FILE"
done
if [[ ${#RUNNING_CONTAINERS[@]} -eq 0 ]]; then
dokku_log_warn "No failed containers found"
return
fi
((MAX_INDEX=${#RUNNING_CONTAINERS[*]} - 1)) || true
for i in ${!RUNNING_CONTAINERS[*]}; do
local CID="${RUNNING_CONTAINERS[i]}"
DOKKU_LOGS_CMD+="(docker logs $DOKKU_LOGS_ARGS $CID 2>&1)"
if [[ $i != "$MAX_INDEX" ]]; then
local DOKKU_LOGS_CMD+="& "
else
local DOKKU_LOGS_CMD+="; "
fi
done
bash -c "($DOKKU_LOGS_CMD)"
}
scheduler-docker-local-scheduler-logs-failed "$@"