From 9f7a33ca3e22f8492a4824047d659ca2de7d2e06 Mon Sep 17 00:00:00 2001 From: "Ryan S. Northrup" Date: Wed, 22 Apr 2015 11:40:27 -0700 Subject: [PATCH 01/24] Safer installation method Piping the output of `curl` or `wget -O-` to a shell is a very bad idea for multiple reasons; security concerns aside, doing so provides no opportunity for the script's download to finish before execution starts, so if the download is interrupted for some reason (perhaps because of an intermittent network failure), the incompletely-downloaded script will still run with potentially disastrous results. See [here](http://www.seancassidy.me/dont-pipe-to-your-shell.html) for a more in-depth description. --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 8938b9578..9eb3c7eb5 100644 --- a/README.md +++ b/README.md @@ -8,9 +8,10 @@ Docker powered mini-Heroku. The smallest PaaS implementation you've ever seen. S ## Installing -To install the latest stable release, you can run the following command as a user that has access to `sudo`: +To install the latest stable release, you can run the following commands as a user that has access to `sudo`: - wget -qO- https://raw.github.com/progrium/dokku/v0.3.17/bootstrap.sh | sudo DOKKU_TAG=v0.3.17 bash + wget https://raw.github.com/progrium/dokku/v0.3.17/bootstrap.sh + sudo DOKKU_TAG=v0.3.17 bash bootstrap.sh ### Upgrading From 9242a5dd311162733e76283a8d9e3a7dd0fdcce8 Mon Sep 17 00:00:00 2001 From: christopher butcher Date: Wed, 22 Apr 2015 18:05:09 -0400 Subject: [PATCH 02/24] Include tip about using sshcommand acl-add --- docs/application-deployment.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/application-deployment.md b/docs/application-deployment.md index 08f934e0f..255a05b31 100644 --- a/docs/application-deployment.md +++ b/docs/application-deployment.md @@ -28,6 +28,14 @@ Dokku only supports deploying from its master branch, so if you'd like to deploy Right now Buildstep supports buildpacks for Node.js, Ruby, Python, [and more](https://github.com/progrium/buildstep#supported-buildpacks). It's not hard to add more, [go add more](https://github.com/progrium/buildstep#adding-buildpacks)! Please check the documentation for your particular build pack as you may need to include configuration files (such as a Procfile) in your project root. +## Deploying to server over SSH + +Pushing to the dokku remote may prompt you to input a password for the dokku user. It's preferable, however, to use key-based authentication, and you can add your public key to the dokku user's authorized_keys file with: + +``` +cat ~/.ssh/id_rsa.pub | ssh [sudouser]@[yourdomain].com "sudo sshcommand acl-add dokku [description]" +``` + ## Deploying with private git submodules Dokku uses git locally (i.e. not a docker image) to build its own copy of your app repo, including submodules. This is done as the `dokku` user. Therefore, in order to deploy private git submodules, you'll need to drop your deploy key in `~dokku/.ssh` and potentially add github.com (or your VCS host key) into `~dokku/.ssh/known_hosts`. A decent test like this should help confirm you've done it correctly. From 0fe3d92fd7cb00b9de9e4379d121e4fe81863049 Mon Sep 17 00:00:00 2001 From: Nigel Thorne Date: Thu, 23 Apr 2015 09:37:50 +1000 Subject: [PATCH 03/24] Replaced gitreceive with sshcommand as per #746 --- docs/advanced-installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/advanced-installation.md b/docs/advanced-installation.md index 9ee60240f..f9ec9c36a 100644 --- a/docs/advanced-installation.md +++ b/docs/advanced-installation.md @@ -19,7 +19,7 @@ sudo make install The `Makefile` allows source URLs to be overridden to include customizations from your own repositories. The `DOCKER_URL`, `PLUGINHOOK_URL`, `SSHCOMMAND_URL` and `STACK_URL` environment variables may be set to override the defaults (see the `Makefile` for how these apply). Example: ```shell -sudo SSHCOMMAND_URL=https://raw.github.com/yourusername/sshcommand/master/gitreceive make install +sudo SSHCOMMAND_URL=https://raw.github.com/yourusername/sshcommand/master/sshcommand make install ``` ## Bootstrap a server from your own repository From 3aa5231f7ee789899a33e2939f92f8fb003c3179 Mon Sep 17 00:00:00 2001 From: Andreas Litt Date: Thu, 23 Apr 2015 15:04:42 +0200 Subject: [PATCH 04/24] remove leading '/' from APP-Name. --- plugins/git/commands | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/plugins/git/commands b/plugins/git/commands index 47934f493..beb967289 100755 --- a/plugins/git/commands +++ b/plugins/git/commands @@ -72,7 +72,7 @@ case "$1" in ;; git-*) - APP="$(echo $2 | perl -pe 's/(? Date: Thu, 23 Apr 2015 15:17:25 +0200 Subject: [PATCH 05/24] exec git-receive-pack with correct path --- plugins/git/commands | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/plugins/git/commands b/plugins/git/commands index beb967289..993ab6e16 100755 --- a/plugins/git/commands +++ b/plugins/git/commands @@ -87,7 +87,11 @@ EOF chmod +x $PRERECEIVE_HOOK fi - args=$* + if [[ $1 == "git-receive-pack" ]]; then + args="$1 '$APP_PATH'" + else + args=$* + fi git-shell -c "$args" ;; From 3d2af158ff60534651a7dc08d5d90d2d2511b166 Mon Sep 17 00:00:00 2001 From: Michael Hobbs Date: Fri, 24 Apr 2015 12:09:58 -0700 Subject: [PATCH 06/24] allow docker-options over ssh. plus test. closes #1135 --- dokku | 2 +- tests/unit/docker-options.bats | 11 +++++++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/dokku b/dokku index c48397b98..1462d1619 100755 --- a/dokku +++ b/dokku @@ -37,7 +37,7 @@ fi if [[ -n "$SSH_ORIGINAL_COMMAND" ]]; then export -n SSH_ORIGINAL_COMMAND - if [[ $1 =~ config-* ]];then + if [[ $1 =~ config-* ]] || [[ $1 =~ docker-options* ]];then xargs $0 <<<$SSH_ORIGINAL_COMMAND exit $? else diff --git a/tests/unit/docker-options.bats b/tests/unit/docker-options.bats index c2b097eb2..d10f346e7 100644 --- a/tests/unit/docker-options.bats +++ b/tests/unit/docker-options.bats @@ -149,3 +149,14 @@ teardown() { assert_success deploy_app } + +@test "docker-options:add (all phases over SSH)" { + run ssh dokku@dokku.me docker-options:add $TEST_APP build,deploy,run \"-v /tmp\" + echo "output: "$output + echo "status: "$status + assert_success + run /bin/bash -c "dokku docker-options $TEST_APP | egrep '\-v /tmp' | wc -l | grep -q 3" + echo "output: "$output + echo "status: "$status + assert_success +} From 0d838d12480214b2aa321cce7d6982534475cbba Mon Sep 17 00:00:00 2001 From: Michael Hobbs Date: Fri, 24 Apr 2015 13:52:14 -0700 Subject: [PATCH 07/24] split up unit-tests and use 4 containers (they're free) --- tests.mk | 4 ++++ tests/ci/parallel_runner.sh | 11 ++++++++--- tests/ci/unit_test_runner.sh | 37 ++++++++++++++++++++++++++++++++++++ 3 files changed, 49 insertions(+), 3 deletions(-) create mode 100755 tests/ci/unit_test_runner.sh diff --git a/tests.mk b/tests.mk index 5c1931e54..1c2cb01a9 100644 --- a/tests.mk +++ b/tests.mk @@ -61,7 +61,11 @@ lint: unit-tests: @echo running unit tests... +ifndef UNIT_TEST_BATCH @$(QUIET) bats tests/unit +else + @$(QUIET) ./tests/ci/unit_test_runner.sh $$UNIT_TEST_BATCH +endif deploy-test-clojure: @echo deploying config app... diff --git a/tests/ci/parallel_runner.sh b/tests/ci/parallel_runner.sh index 68e1038ff..612ec0809 100755 --- a/tests/ci/parallel_runner.sh +++ b/tests/ci/parallel_runner.sh @@ -2,16 +2,21 @@ case "$CIRCLE_NODE_INDEX" in 0) - echo "=====> make unit-tests" - sudo -E make -e unit-tests + echo "=====> make unit-tests (1/2)" + sudo -E UNIT_TEST_BATCH=1 make -e unit-tests ;; 1) + echo "=====> make unit-tests (2/2)" + sudo -E UNIT_TEST_BATCH=2 make -e unit-tests + ;; + + 2) echo "=====> make deploy-tests (buildstep release)" sudo -E make -e deploy-tests ;; - 2) + 3) echo "=====> make deploy-tests (buildstep master)" docker rmi -f progrium/buildstep && \ sudo -E BUILD_STACK=true make -e stack && \ diff --git a/tests/ci/unit_test_runner.sh b/tests/ci/unit_test_runner.sh new file mode 100755 index 000000000..ebf5df679 --- /dev/null +++ b/tests/ci/unit_test_runner.sh @@ -0,0 +1,37 @@ +#!/usr/bin/env bash + +is_number() { + local NUMBER=$1; local NUM_RE='^[0-9]+$' + if [[ $NUMBER =~ $NUM_RE ]];then + return 0 + else + return 1 + fi +} + +usage() { + echo "usage: $0 1|2" + exit 0 +} + +BATCH_NUM="$1" +is_number $BATCH_NUM || usage + +TESTS=( $(find $(dirname $0)/../unit -maxdepth 1 -name "*.bats") ) +HALF_TESTS=$(( ${#TESTS[@]} / 2 )) +FIRST_HALF=${TESTS[@]:0:${HALF_TESTS}} +LAST_HALF=${TESTS[@]:${HALF_TESTS}:${#TESTS[@]}} + +case "$BATCH_NUM" in + 1) + bats $FIRST_HALF + ;; + + 2) + bats $LAST_HALF + ;; + + *) + usage + ;; +esac From a529f6fce8550e9e7ee86d8e05b297bf3b157d7d Mon Sep 17 00:00:00 2001 From: Michael Hobbs Date: Fri, 24 Apr 2015 14:02:44 -0700 Subject: [PATCH 08/24] lintball --- tests/ci/unit_test_runner.sh | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/ci/unit_test_runner.sh b/tests/ci/unit_test_runner.sh index ebf5df679..9f5e2ce39 100755 --- a/tests/ci/unit_test_runner.sh +++ b/tests/ci/unit_test_runner.sh @@ -17,18 +17,18 @@ usage() { BATCH_NUM="$1" is_number $BATCH_NUM || usage -TESTS=( $(find $(dirname $0)/../unit -maxdepth 1 -name "*.bats") ) +TESTS=($(find "$(dirname $0)"/../unit -maxdepth 1 -name "*.bats")) HALF_TESTS=$(( ${#TESTS[@]} / 2 )) -FIRST_HALF=${TESTS[@]:0:${HALF_TESTS}} -LAST_HALF=${TESTS[@]:${HALF_TESTS}:${#TESTS[@]}} +FIRST_HALF=("${TESTS[@]:0:${HALF_TESTS}}") +LAST_HALF=("${TESTS[@]:${HALF_TESTS}:${#TESTS[@]}}") case "$BATCH_NUM" in 1) - bats $FIRST_HALF + bats "${FIRST_HALF[@]}" ;; 2) - bats $LAST_HALF + bats "${LAST_HALF[@]}" ;; *) From 0c715d4b399457c1ad8b5d1f9d183abc8915ff95 Mon Sep 17 00:00:00 2001 From: Michael Hobbs Date: Fri, 24 Apr 2015 14:11:45 -0700 Subject: [PATCH 09/24] more test efficiencies and cleanup --- circle.yml | 14 ++------------ tests.mk | 1 - tests/ci/parallel_runner.sh | 33 ++++++++++++++++++++++++++------ tests/deploy/config.bats | 10 ---------- tests/deploy/gitsubmodules.bats | 10 ---------- tests/deploy/go.bats | 10 ---------- tests/deploy/java.bats | 10 ---------- tests/deploy/multi.bats | 10 ---------- tests/deploy/nodejs-express.bats | 10 ---------- tests/deploy/php.bats | 11 ----------- tests/deploy/python-flask.bats | 10 ---------- tests/deploy/static.bats | 11 ----------- 12 files changed, 29 insertions(+), 111 deletions(-) delete mode 100644 tests/deploy/config.bats delete mode 100644 tests/deploy/gitsubmodules.bats delete mode 100644 tests/deploy/go.bats delete mode 100644 tests/deploy/java.bats delete mode 100644 tests/deploy/multi.bats delete mode 100644 tests/deploy/nodejs-express.bats delete mode 100644 tests/deploy/php.bats delete mode 100644 tests/deploy/python-flask.bats delete mode 100644 tests/deploy/static.bats diff --git a/circle.yml b/circle.yml index 34ce1b34c..3ab50cccf 100644 --- a/circle.yml +++ b/circle.yml @@ -5,19 +5,9 @@ dependencies: cache_directories: - "~/docker" override: - # need to add the dokku user to the docker group - - sudo -E CI=true make -e sshcommand - - sudo usermod -G docker dokku - - sudo -E CI=true DOCKER_VERSION=1.5.0 make -e install: { timeout: 300 } - - make -e ci-dependencies - #### circle does some weird *expletive* with regards to root and gh auth (needed for gitsubmodules test) - - sudo rsync -a ~ubuntu/.ssh/ ~root/.ssh/ - - sudo chown -R root:root ~root/.ssh/ - - sudo sed -e 's:/home/ubuntu:/root:g' ~root/.ssh/config - #### - - sudo -E make -e setup-deploy-tests + - ./tests/ci/parallel_runner.sh setup: { timeout: 300 } post: - sudo -E make -e lint test: override: - - ./tests/ci/parallel_runner.sh: { parallel: true, timeout: 1800 } + - ./tests/ci/parallel_runner.sh testing: { parallel: true, timeout: 1800 } diff --git a/tests.mk b/tests.mk index 1c2cb01a9..c4d126bbb 100644 --- a/tests.mk +++ b/tests.mk @@ -129,7 +129,6 @@ deploy-test-static: deploy-tests: @echo running deploy tests... - # @$(QUIET) bats tests/deploy @$(QUIET) $(MAKE) deploy-test-config @$(QUIET) $(MAKE) deploy-test-clojure @$(QUIET) $(MAKE) deploy-test-dockerfile diff --git a/tests/ci/parallel_runner.sh b/tests/ci/parallel_runner.sh index 612ec0809..8a664b8b8 100755 --- a/tests/ci/parallel_runner.sh +++ b/tests/ci/parallel_runner.sh @@ -1,25 +1,46 @@ #!/usr/bin/env bash +MODE="$1"; MODE=${MODE:="testing"} + +setup_circle() { + MAKE_ENV="CI=true DOCKER_VERSION=1.5.0" + [[ "$1" == "buildstack" ]] && MAKE_ENV+=" BUILD_STACK=true " + echo "setting up with MAKE_ENV: $MAKE_ENV" + sudo -E CI=true make -e sshcommand + # need to add the dokku user to the docker group + sudo usermod -G docker dokku + #### circle does some weird *expletive* with regards to root and gh auth (needed for gitsubmodules test) + sudo rsync -a ~ubuntu/.ssh/ ~root/.ssh/ + sudo chown -R root:root ~root/.ssh/ + sudo sed --in-place 's:/home/ubuntu:/root:g' ~root/.ssh/config + #### + sudo -E $MAKE_ENV make -e install + sudo -E make -e setup-deploy-tests + make -e ci-dependencies +} + case "$CIRCLE_NODE_INDEX" in 0) - echo "=====> make unit-tests (1/2)" + echo "=====> make unit-tests (1/2) on CIRCLE_NODE_INDEX: $CIRCLE_NODE_INDEX" + [[ "$MODE" == "setup" ]] && setup_circle && exit 0 sudo -E UNIT_TEST_BATCH=1 make -e unit-tests ;; 1) - echo "=====> make unit-tests (2/2)" + echo "=====> make unit-tests (2/2) on CIRCLE_NODE_INDEX: $CIRCLE_NODE_INDEX" + [[ "$MODE" == "setup" ]] && setup_circle && exit 0 sudo -E UNIT_TEST_BATCH=2 make -e unit-tests ;; 2) - echo "=====> make deploy-tests (buildstep release)" + echo "=====> make deploy-tests (buildstep release) on CIRCLE_NODE_INDEX: $CIRCLE_NODE_INDEX" + [[ "$MODE" == "setup" ]] && setup_circle && exit 0 sudo -E make -e deploy-tests ;; 3) - echo "=====> make deploy-tests (buildstep master)" - docker rmi -f progrium/buildstep && \ - sudo -E BUILD_STACK=true make -e stack && \ + echo "=====> make deploy-tests (buildstep master) on CIRCLE_NODE_INDEX: $CIRCLE_NODE_INDEX" + [[ "$MODE" == "setup" ]] && setup_circle buildstack && exit 0 sudo -E make -e deploy-tests ;; esac diff --git a/tests/deploy/config.bats b/tests/deploy/config.bats deleted file mode 100644 index f8177e53e..000000000 --- a/tests/deploy/config.bats +++ /dev/null @@ -1,10 +0,0 @@ -#!/usr/bin/env bats - -load ../unit/test_helper - -@test "deploy config app" { - run bash -c "cd tests && ./test_deploy ./apps/config dokku.me" - echo "output: "$output - echo "status: "$status - assert_success -} diff --git a/tests/deploy/gitsubmodules.bats b/tests/deploy/gitsubmodules.bats deleted file mode 100644 index b5b66fdcf..000000000 --- a/tests/deploy/gitsubmodules.bats +++ /dev/null @@ -1,10 +0,0 @@ -#!/usr/bin/env bats - -load ../unit/test_helper - -@test "deploy gitsubmodules app" { - run bash -c "cd tests && ./test_deploy ./apps/gitsubmodules dokku.me" - echo "output: "$output - echo "status: "$status - assert_success -} diff --git a/tests/deploy/go.bats b/tests/deploy/go.bats deleted file mode 100644 index cdd75c2c2..000000000 --- a/tests/deploy/go.bats +++ /dev/null @@ -1,10 +0,0 @@ -#!/usr/bin/env bats - -load ../unit/test_helper - -@test "deploy go app" { - run bash -c "cd tests && ./test_deploy ./apps/go dokku.me" - echo "output: "$output - echo "status: "$status - assert_success -} diff --git a/tests/deploy/java.bats b/tests/deploy/java.bats deleted file mode 100644 index 560ad28e3..000000000 --- a/tests/deploy/java.bats +++ /dev/null @@ -1,10 +0,0 @@ -#!/usr/bin/env bats - -load ../unit/test_helper - -@test "deploy java app" { - run bash -c "cd tests && ./test_deploy ./apps/java dokku.me" - echo "output: "$output - echo "status: "$status - assert_success -} diff --git a/tests/deploy/multi.bats b/tests/deploy/multi.bats deleted file mode 100644 index e938e21fc..000000000 --- a/tests/deploy/multi.bats +++ /dev/null @@ -1,10 +0,0 @@ -#!/usr/bin/env bats - -load ../unit/test_helper - -@test "deploy multi app" { - run bash -c "cd tests && ./test_deploy ./apps/multi dokku.me" - echo "output: "$output - echo "status: "$status - assert_success -} diff --git a/tests/deploy/nodejs-express.bats b/tests/deploy/nodejs-express.bats deleted file mode 100644 index 2fcc82dc8..000000000 --- a/tests/deploy/nodejs-express.bats +++ /dev/null @@ -1,10 +0,0 @@ -#!/usr/bin/env bats - -load ../unit/test_helper - -@test "deploy nodejs-express app" { - run bash -c "cd tests && ./test_deploy ./apps/nodejs-express dokku.me" - echo "output: "$output - echo "status: "$status - assert_success -} diff --git a/tests/deploy/php.bats b/tests/deploy/php.bats deleted file mode 100644 index 06bfa081b..000000000 --- a/tests/deploy/php.bats +++ /dev/null @@ -1,11 +0,0 @@ -#!/usr/bin/env bats - -load ../unit/test_helper - -@test "deploy php app" { - skip "fails. ref: https://github.com/progrium/buildstep/issues/126" - run bash -c "cd tests && ./test_deploy ./apps/php dokku.me" - echo "output: "$output - echo "status: "$status - assert_success -} diff --git a/tests/deploy/python-flask.bats b/tests/deploy/python-flask.bats deleted file mode 100644 index 1834e8fc8..000000000 --- a/tests/deploy/python-flask.bats +++ /dev/null @@ -1,10 +0,0 @@ -#!/usr/bin/env bats - -load ../unit/test_helper - -@test "deploy python-flask app" { - run bash -c "cd tests && ./test_deploy ./apps/python-flask dokku.me" - echo "output: "$output - echo "status: "$status - assert_success -} diff --git a/tests/deploy/static.bats b/tests/deploy/static.bats deleted file mode 100644 index bef621f29..000000000 --- a/tests/deploy/static.bats +++ /dev/null @@ -1,11 +0,0 @@ -#!/usr/bin/env bats - -load ../unit/test_helper - -@test "deploy static app" { - skip "fails on apt-get update..." - run bash -c "cd tests && ./test_deploy ./apps/static dokku.me" - echo "output: "$output - echo "status: "$status - assert_success -} From dce031d38d0b0a486024ba5a0129688ba921980a Mon Sep 17 00:00:00 2001 From: Michael Hobbs Date: Fri, 24 Apr 2015 11:49:11 -0700 Subject: [PATCH 10/24] don't re-deploy on domains:add. allow multple domains on command line. closes #1142 --- plugins/domains/commands | 28 +++++++++++++++++++--------- tests/unit/domains.bats | 7 +++++++ 2 files changed, 26 insertions(+), 9 deletions(-) diff --git a/plugins/domains/commands b/plugins/domains/commands index 282d1eb65..5de0f426d 100755 --- a/plugins/domains/commands +++ b/plugins/domains/commands @@ -71,8 +71,8 @@ case "$1" in verify_app_name "$2" APP="$2" - if [[ -z $3 ]]; then - echo "Usage: dokku $1 $APP DOMAIN" + if [[ -z "${*:3}" ]]; then + echo "Usage: dokku $1 $APP DOMAIN [DOMAIN ...]" echo "Must specify DOMAIN." exit 1 fi @@ -82,12 +82,17 @@ case "$1" in exit 1 fi + shift 2 dokku domains:setup $APP - echo "$3" >> "$DOKKU_ROOT/$APP/VHOST" + for DOMAIN in "$@";do + echo "$DOMAIN" >> "$DOKKU_ROOT/$APP/VHOST" + done # we need to restart the app to make sure we're binding to the appropriate network interface - dokku ps:restart $APP + dokku nginx:build-config $APP pluginhook post-domains-update $APP - dokku_log_info1 "Added $3 to $APP" + for DOMAIN in "$@";do + dokku_log_info1 "Added $DOMAIN to $APP" + done ;; @@ -108,16 +113,21 @@ case "$1" in verify_app_name "$2" APP="$2" - if [[ -z $3 ]]; then - echo "Usage: dokku $1 $2 DOMAIN" + if [[ -z "${*:3}" ]]; then + echo "Usage: dokku $1 $APP DOMAIN [DOMAIN ...]" echo "Must specify DOMAIN." exit 1 fi + shift 2 dokku domains:setup $APP - sed -i "/^$3$/d" "$DOKKU_ROOT/$APP/VHOST" + for DOMAIN in "$@";do + sed -i "/^$DOMAIN$/d" "$DOKKU_ROOT/$APP/VHOST" + done pluginhook post-domains-update $APP - dokku_log_info1 "Removed $3 from $APP" + for DOMAIN in "$@";do + dokku_log_info1 "Removed $DOMAIN from $APP" + done ;; diff --git a/tests/unit/domains.bats b/tests/unit/domains.bats index bdc52994e..de07060e3 100644 --- a/tests/unit/domains.bats +++ b/tests/unit/domains.bats @@ -28,6 +28,13 @@ teardown() { assert_success } +@test "(domains) domains:add (multiple)" { + run dokku domains:add $TEST_APP www.test.app.dokku.me test.app.dokku.me + echo "output: "$output + echo "status: "$status + assert_success +} + @test "(domains) domains:remove" { run dokku domains:add $TEST_APP test.app.dokku.me echo "output: "$output From 2f0f2889db4d72bde9b1c37d823fff78cc1d1e77 Mon Sep 17 00:00:00 2001 From: Michael Hobbs Date: Fri, 24 Apr 2015 13:09:30 -0700 Subject: [PATCH 11/24] need to restart now --- tests/unit/core_ports.bats | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/unit/core_ports.bats b/tests/unit/core_ports.bats index f0076b381..0bee19df1 100644 --- a/tests/unit/core_ports.bats +++ b/tests/unit/core_ports.bats @@ -97,7 +97,12 @@ teardown() { echo "status: "$status assert_success - CONTAINER_ID=$(docker ps --no-trunc| grep dokku/$TEST_APP | grep "start web" | awk '{ print $1 }') + run dokku ps:restart $TEST_APP + echo "output: "$output + echo "status: "$status + assert_success + + CONTAINER_ID=$(docker ps --no-trunc| grep dokku/$TEST_APP | grep "start web" | awk '{ print $1 }' | head -1) run bash -c "docker port $CONTAINER_ID | sed 's/[0-9.]*://' | egrep '[0-9]*'" echo "output: "$output echo "status: "$status From 9fdff116872a7a128c4736e75146475f23019f37 Mon Sep 17 00:00:00 2001 From: Andreas Litt Date: Thu, 23 Apr 2015 15:04:42 +0200 Subject: [PATCH 12/24] remove leading '/' from APP-Name. --- plugins/git/commands | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/plugins/git/commands b/plugins/git/commands index 47934f493..beb967289 100755 --- a/plugins/git/commands +++ b/plugins/git/commands @@ -72,7 +72,7 @@ case "$1" in ;; git-*) - APP="$(echo $2 | perl -pe 's/(? Date: Thu, 23 Apr 2015 15:17:25 +0200 Subject: [PATCH 13/24] exec git-receive-pack with correct path --- plugins/git/commands | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/plugins/git/commands b/plugins/git/commands index beb967289..993ab6e16 100755 --- a/plugins/git/commands +++ b/plugins/git/commands @@ -87,7 +87,11 @@ EOF chmod +x $PRERECEIVE_HOOK fi - args=$* + if [[ $1 == "git-receive-pack" ]]; then + args="$1 '$APP_PATH'" + else + args=$* + fi git-shell -c "$args" ;; From c8c4a6a0c651b397cd5ae4d25248591f1a8d2504 Mon Sep 17 00:00:00 2001 From: Michael Hobbs Date: Sat, 25 Apr 2015 09:54:40 -0700 Subject: [PATCH 14/24] pass through return code when calling destroy_app. allow custom git remote --- tests/unit/test_helper.bash | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tests/unit/test_helper.bash b/tests/unit/test_helper.bash index 08d16f8ae..ec7f359ff 100644 --- a/tests/unit/test_helper.bash +++ b/tests/unit/test_helper.bash @@ -89,7 +89,9 @@ create_app() { } destroy_app() { + RC="$1"; RC=${RC:=0} echo $TEST_APP | dokku apps:destroy $TEST_APP + return $RC } add_domain() { @@ -98,18 +100,20 @@ add_domain() { deploy_app() { APP_TYPE="$1"; APP_TYPE=${APP_TYPE:="nodejs-express"} + GIT_REMOTE="$2"; GIT_REMOTE=${GIT_REMOTE:="dokku@dokku.me:$TEST_APP"} TMP=$(mktemp -d -t "$TARGET.XXXXX") rmdir $TMP && cp -r ./tests/apps/$APP_TYPE $TMP cd $TMP git init git config user.email "robot@example.com" git config user.name "Test Robot" - git remote add target dokku@dokku.me:$TEST_APP + echo "setting up remote: $GIT_REMOTE" + git remote add target $GIT_REMOTE [[ -f gitignore ]] && mv gitignore .gitignore git add . git commit -m 'initial commit' - git push target master || destroy_app + git push target master || destroy_app $? } setup_client_repo() { From 981e857ab162dab9164cb3d2169dcc473cf140f5 Mon Sep 17 00:00:00 2001 From: Michael Hobbs Date: Sat, 25 Apr 2015 09:54:49 -0700 Subject: [PATCH 15/24] add sshd config for off-port testing --- tests.mk | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tests.mk b/tests.mk index c4d126bbb..68f3e32fc 100644 --- a/tests.mk +++ b/tests.mk @@ -28,8 +28,15 @@ endif @echo "-----> Setting up ssh config..." ifneq ($(shell ls /root/.ssh/config > /dev/null 2>&1 ; echo $$?),0) echo "Host dokku.me \\r\\n RequestTTY yes \\r\\n IdentityFile /root/.ssh/dokku_test_rsa" >> /root/.ssh/config + echo "Host 127.0.0.1 \\r\\n Port 22333 \\r\\n RequestTTY yes \\r\\n IdentityFile /root/.ssh/dokku_test_rsa" >> /root/.ssh/config else ifeq ($(shell grep dokku.me /root/.ssh/config),) echo "Host dokku.me \\r\\n RequestTTY yes \\r\\n IdentityFile /root/.ssh/dokku_test_rsa" >> /root/.ssh/config + echo "Host 127.0.0.1 \\r\\n Port 22333 \\r\\n RequestTTY yes \\r\\n IdentityFile /root/.ssh/dokku_test_rsa" >> /root/.ssh/config +endif + +ifeq ($(shell grep 22333 /etc/ssh/sshd_config),) + sed --in-place "s:^Port 22:Port 22 \\nPort 22333:g" /etc/ssh/sshd_config + restart ssh endif @echo "-----> Installing SSH public key..." @@ -38,6 +45,7 @@ endif @echo "-----> Intitial SSH connection to populate known_hosts..." ssh -o StrictHostKeyChecking=no dokku@dokku.me help > /dev/null + ssh -o StrictHostKeyChecking=no dokku@127.0.0.1 help > /dev/null ifeq ($(shell grep dokku.me /home/dokku/VHOST 2>/dev/null),) @echo "-----> Setting default VHOST to dokku.me..." From a6c7cc50cef4637ba198cea1450a6b8ed34984bb Mon Sep 17 00:00:00 2001 From: Michael Hobbs Date: Sat, 25 Apr 2015 09:54:57 -0700 Subject: [PATCH 16/24] test off port git remote --- tests/unit/core.bats | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tests/unit/core.bats b/tests/unit/core.bats index 76d5702e2..99c545a99 100644 --- a/tests/unit/core.bats +++ b/tests/unit/core.bats @@ -115,3 +115,9 @@ build_nginx_config() { assert_urls "https://${TEST_APP}.dokku.me" "https://test.dokku.me" "http://dokku.example.com" } +@test "(core) git-remote (off-port)" { + run deploy_app nodejs-express ssh://dokku@127.0.0.1:22333/$TEST_APP + echo "output: "$output + echo "status: "$status + assert_success +} From e0adfd035fd96df1299cb1d114aeff9d6be1928e Mon Sep 17 00:00:00 2001 From: Mohammad El-Abid Date: Mon, 27 Apr 2015 10:00:48 -0400 Subject: [PATCH 17/24] Add information about 444 for nginx in default_sever. --- docs/nginx.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/nginx.md b/docs/nginx.md index 947e7d15f..d7589ca34 100644 --- a/docs/nginx.md +++ b/docs/nginx.md @@ -167,7 +167,7 @@ d6499edb0edb dokku/node-js-app:latest "/bin/bash -c '/star About a mi # Default site -By default, dokku will route any received request with an unknown HOST header value to the lexicographically first site in the nginx config stack. If this is not the desired behavior, you may want to add the following configuration to nginx. This will catch all unknown HOST header values and return a `410 Gone` response. +By default, dokku will route any received request with an unknown HOST header value to the lexicographically first site in the nginx config stack. If this is not the desired behavior, you may want to add the following configuration to nginx. This will catch all unknown HOST header values and return a `410 Gone` response. You can replace the `return 410;` with `return 444;` which will cause nginx to not respond to requests that do not match known domains (connection refused). ``` server { From 952ff31189c412b9b00b504ecee2a63a73b21ce6 Mon Sep 17 00:00:00 2001 From: cjblomqvist Date: Tue, 28 Apr 2015 15:53:32 +0200 Subject: [PATCH 18/24] Add monit plugin --- docs/plugins.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/plugins.md b/docs/plugins.md index 533b01121..98874d169 100644 --- a/docs/plugins.md +++ b/docs/plugins.md @@ -37,6 +37,7 @@ Note: The following plugins have been supplied by our community and may not have [cameron-martin]: https://github.com/cameron-martin [cedricziel]: https://github.com/cedricziel [cef]: https://github.com/cef +[cjblomqvist]: https://github.com/cjblomqvist [darkpixel]: https://github.com/darkpixel [dyson]: https://github.com/dyson [F4-Group]: https://github.com/F4-Group @@ -140,10 +141,13 @@ Note: The following plugins have been supplied by our community and may not have | [Supervisord](https://github.com/statianzo/dokku-supervisord) | [statianzo][] | Compatible with 0.2.0 | | [Logging Supervisord](https://github.com/sehrope/dokku-logging-supervisord) | [sehrope][] | Works with dokku @ [c77cbf1][] - no 0.2.0 compatibility | | [Forego](https://github.com/iskandar/dokku-forego) | [iskandar][] | Compatible with 0.2.x | +| [Monit](https://github.com/cjblomqvist/dokku-monit)1 | [cjblomqvist][] | | [c77cbf1]: https://github.com/progrium/dokku/commit/c77cbf1d3ae07f0eafb85082ed7edcae9e836147 [28de3ec]: https://github.com/progrium/dokku/commit/28de3ecaa3231a223f83fd8d03f373308673bc40 +1 This plugin is far from complete and a mere proof-of-concept of how Monit can be used to externally monitor dokku apps. Read the [README](https://github.com/cjblomqvist/dokku-monit) for more details or contact the author. + ### Dokku Features | Plugin | Author | Compatibility | From c1d901723d04b4cfa157acc73b9ce94e1b9f4d51 Mon Sep 17 00:00:00 2001 From: Felix Wolfsteller Date: Wed, 29 Apr 2015 10:40:41 +0200 Subject: [PATCH 19/24] add section about curl timeout issues when pulling buildpack stuff while deploying. --- docs/troubleshooting.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index f8a2e23f5..d46f464f4 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -141,3 +141,31 @@ When specifying your port you may want to use something similar to: Please see https://github.com/progrium/dokku/issues/282 +*** + +__Symptom:__ Deployment fails because of slow internet connection, messages shows `gzip: stdin: unexpected end of file` + +__Solution:__ + +If you see output similar this when deploying: + +``` + Command: 'set -o pipefail; curl --fail --retry 3 --retry-delay 1 --connect-timeout 3 --max-time 30 https://s3-external-1.amazonaws.com/heroku-buildpack-ruby/ruby-2.0.0-p451-default-cache.tgz -s -o - | tar zxf -' failed unexpectedly: + ! + ! gzip: stdin: unexpected end of file + ! tar: Unexpected EOF in archive + ! tar: Unexpected EOF in archive + ! tar: Error is not recoverable: exiting now +``` + +it might that the curl command that is supposed to fetch the buildpack (anything in the low megabyte file size range) takes too long to finish, due to slowish connection. To overwrite the default values (connection timeout: 3 seconds, total maximum time for operation: 30 seconds), edit `/home/dokku/ENV` like the following: + +``` +#/home/dokku/ENV +export CURL_TIMEOUT=600 +export CURL_CONNECT_TIMEOUT=30 +``` + +References +* https://github.com/progrium/dokku/issues/509 +* https://github.com/dokku-alt/dokku-alt/issues/169 From a36aaab686fe4ed288ab777a3a0b20547f8cc5c5 Mon Sep 17 00:00:00 2001 From: Felix Wolfsteller Date: Wed, 29 Apr 2015 10:52:38 +0200 Subject: [PATCH 20/24] add section about contribution to README --- README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/README.md b/README.md index 9eb3c7eb5..c74d1e020 100644 --- a/README.md +++ b/README.md @@ -25,6 +25,14 @@ Full documentation - including advanced installation docs - are available online You can use [Github Issues](https://github.com/progrium/dokku/issues), check [Troubleshooting](http://progrium.viewdocs.io/dokku/troubleshooting) in the documentation, or join us on [freenode in #dokku](https://webchat.freenode.net/?channels=%23dokku) +## Contribution + +After checking [Github Issues](https://github.com/progrium/dokku/issues), the [Troubleshooting Guide](http://progrium.viewdocs.io/dokku/troubleshooting) or having a chat with us on [freenode in #dokku](https://webchat.freenode.net/?channels=%23dokku), feel free to fork and create a Pull Request. +``` +<@savant> sometimes we dont merge because of discussion or maybe there is a better way +<@savant> but all prs welcome :) +``` + ## Sponsors Dokku is currently sponsored by the enterprise grade, multi-host PaaS project [Deis](http://deis.io/). From 4aa970ace821520d1df694e47d889e24bebb03bb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Piotr=20Nie=C5=82acny?= Date: Tue, 28 Apr 2015 17:41:48 +0200 Subject: [PATCH 21/24] Do not kill docker container with SIGKILL When we will use `kill` command on `docker` main process will receive `SIGKILL` signal. This will result in immediate termination of processes. We need to send a signal `SIGTERM` to main process and give the opportunity to close application in correct way. --- dokku | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/dokku b/dokku index 1462d1619..283e11d20 100755 --- a/dokku +++ b/dokku @@ -91,7 +91,7 @@ case "$1" in # if we can't post-deploy successfully, kill new container kill_new() { - docker inspect $id &> /dev/null && docker kill $id > /dev/null + docker inspect $id &> /dev/null && docker stop $id > /dev/null && docker kill $id > /dev/null trap - INT TERM EXIT kill -9 $$ } @@ -120,6 +120,7 @@ case "$1" in exec >/dev/null 2>/dev/null Date: Thu, 30 Apr 2015 08:35:25 +0200 Subject: [PATCH 22/24] improve contribution section in README as proposed by josegonzalez. --- README.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index c74d1e020..e7daafe53 100644 --- a/README.md +++ b/README.md @@ -28,10 +28,8 @@ You can use [Github Issues](https://github.com/progrium/dokku/issues), check [Tr ## Contribution After checking [Github Issues](https://github.com/progrium/dokku/issues), the [Troubleshooting Guide](http://progrium.viewdocs.io/dokku/troubleshooting) or having a chat with us on [freenode in #dokku](https://webchat.freenode.net/?channels=%23dokku), feel free to fork and create a Pull Request. -``` -<@savant> sometimes we dont merge because of discussion or maybe there is a better way -<@savant> but all prs welcome :) -``` + +While we may not merge your PR as is, they serve to start conversations and improve the general dokku experience for all users. ## Sponsors From 1a70dd1707cf9939e9574c17fe1aa8c1d8e36474 Mon Sep 17 00:00:00 2001 From: cjblomqvist Date: Thu, 30 Apr 2015 09:51:14 +0200 Subject: [PATCH 23/24] Update plugins.md --- docs/plugins.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/plugins.md b/docs/plugins.md index 98874d169..5db6a1829 100644 --- a/docs/plugins.md +++ b/docs/plugins.md @@ -141,12 +141,11 @@ Note: The following plugins have been supplied by our community and may not have | [Supervisord](https://github.com/statianzo/dokku-supervisord) | [statianzo][] | Compatible with 0.2.0 | | [Logging Supervisord](https://github.com/sehrope/dokku-logging-supervisord) | [sehrope][] | Works with dokku @ [c77cbf1][] - no 0.2.0 compatibility | | [Forego](https://github.com/iskandar/dokku-forego) | [iskandar][] | Compatible with 0.2.x | -| [Monit](https://github.com/cjblomqvist/dokku-monit)1 | [cjblomqvist][] | | +| [Monit](https://github.com/cjblomqvist/dokku-monit) | [cjblomqvist][] | | [c77cbf1]: https://github.com/progrium/dokku/commit/c77cbf1d3ae07f0eafb85082ed7edcae9e836147 [28de3ec]: https://github.com/progrium/dokku/commit/28de3ecaa3231a223f83fd8d03f373308673bc40 -1 This plugin is far from complete and a mere proof-of-concept of how Monit can be used to externally monitor dokku apps. Read the [README](https://github.com/cjblomqvist/dokku-monit) for more details or contact the author. ### Dokku Features From 56d0af02c4689219bc512537862d404ffb63ed3e Mon Sep 17 00:00:00 2001 From: cjblomqvist Date: Thu, 30 Apr 2015 09:51:30 +0200 Subject: [PATCH 24/24] Update plugins.md --- docs/plugins.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/plugins.md b/docs/plugins.md index 5db6a1829..b6ec82968 100644 --- a/docs/plugins.md +++ b/docs/plugins.md @@ -146,7 +146,6 @@ Note: The following plugins have been supplied by our community and may not have [c77cbf1]: https://github.com/progrium/dokku/commit/c77cbf1d3ae07f0eafb85082ed7edcae9e836147 [28de3ec]: https://github.com/progrium/dokku/commit/28de3ecaa3231a223f83fd8d03f373308673bc40 - ### Dokku Features | Plugin | Author | Compatibility |