github/dokku
0
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Merge pull request #4453 from dokku/3458-git-from-archive

Browse Source 
Simplify tar and zip deploys via git:from-archive
This commit is contained in:
Jose Diaz-Gonzalez
2021-03-01 08:33:33 -05:00
committed by GitHub
parent ade2cc8a7c 5c1a1baf08
commit 30e09b8a07
10 changed files with 210 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
debian/control vendored
View File

@@ -3,7 +3,7 @@ Version: 0.23.9
Section: web
Priority: optional
Architecture: amd64
Depends: locales, git, cpio, curl, man-db, netcat, sshcommand (>= 0.12.0), docker-engine-cs (>= 17.05.0) | docker-engine (>= 17.05.0) | docker-io (>= 17.05.0) | docker.io (>= 17.05.0) | docker-ce (>= 17.05.0) | docker-ee (>= 17.05.0) | moby-engine, docker-image-labeler (>= 0.2.2), net-tools, netrc, software-properties-common, procfile-util (>= 0.11.0), python-software-properties | python3-software-properties, rsyslog, dos2unix, jq
Depends: locales, git, cpio, curl, man-db, netcat, sshcommand (>= 0.12.0), docker-engine-cs (>= 17.05.0) | docker-engine (>= 17.05.0) | docker-io (>= 17.05.0) | docker.io (>= 17.05.0) | docker-ce (>= 17.05.0) | docker-ee (>= 17.05.0) | moby-engine, docker-image-labeler (>= 0.2.2), net-tools, netrc, software-properties-common, procfile-util (>= 0.11.0), python-software-properties | python3-software-properties, rsyslog, dos2unix, jq, unzip
Recommends: herokuish (>= 0.3.4), parallel, dokku-update, dokku-event-listener
Pre-Depends: gliderlabs-sigil, nginx (>= 1.8.0) | openresty, dnsutils, cgroupfs-mount | cgroup-lite, plugn (>= 0.3.0), sudo, python3, debconf
Maintainer: Jose Diaz-Gonzalez <dokku@josediazgonzalez.com>

3
docs/appendices/0.24.0-migration-guide.md
View File

@@ -8,4 +8,5 @@
## Deprecations
- The 1.0.0 release of Dokku will no longer select buildpack builders over dockerfile builders if both builders match. Instead, Dokku will choose the first builder that responds to the `builder-detect` trigger. Users that wish to use a specific builder may set a builder using the `builder:set` command, which will force Dokku to use the specified builder regardless of what might be auto-detected.
- The `tags` plugin is deprecated in favor of the [`git:from-image`](/docs/deployment/methods/git.md#initializing-an-app-repository-from-a-docker-image) command. It will be removed in a future release, and is considered unmaintained. Users are highly encouraged to switch their workflows to `git:from-image`.
- The `tags` plugin is deprecated in favor of the [`git:from-image`](/docs/deployment/methods/git.md#initializing-an-app-repository-from-a-docker-image) command. It will be removed in the next minor release, and is considered unmaintained. Users are highly encouraged to switch their workflows to `git:from-image`.
- The `tar` plugin is deprecated in favor of the [`git:from-archive`](/docs/deployment/methods/git.md#initializing-an-app-repository-from-an-archive-file) command. It will be removed in the next minor release, and is considered unmaintained. Users are highly encouraged to switch their workflows to `git:from-archive`.

37
docs/deployment/methods/git.md
View File

@@ -5,7 +5,8 @@
```
git:allow-host <host> # Adds a host to known_hosts
git:auth <host> [<username> <password>] # Configures netrc authentication for a given git server
git:from-image [--build-dir DIRECTORY] <app> <docker-image> [<git-username> <git-email>] # Updates a app's git repository with a given docker image
git:from-archive [--archive-type ARCHIVE_TYPE] <app> <archive-url> [<git-username> <git-email>] # Updates an app's git repository with a given archive file
git:from-image [--build-dir DIRECTORY] <app> <docker-image> [<git-username> <git-email>] # Updates an app's git repository with a given docker image
git:sync [--build] <app> <repository> [<git-ref>] # Clone or fetch an app from remote git repo
git:initialize <app> # Initialize a git repository for an app
git:public-key # Outputs the dokku public deploy key
@@ -131,7 +132,7 @@ In the above example, Dokku will build the app as if the repository contained _o
FROM dokku/node-js-getting-started:latest
```
Triggering a build with the same arguments multiple times will result in Dokku exiting `0` early as there is no build to perform.
Triggering a build with the same arguments multiple times will result in Dokku exiting `0` early as there will be no changes detected.
The `git:from-image` command can optionally take a git `user.name` and `user.email` argument (in that order) to customize the author. If the arguments are left empty, they will fallback to `Dokku` and `automated@dokku.sh`, respectively.
@@ -147,6 +148,38 @@ dokku git:from-image --build-dir path/to/build node-js-app dokku/node-js-getting
See the [dockerfile documentation](/docs/deployment/methods/dockerfiles.md) to learn about the different ways to configure Dockerfile-based deploys.
### Initializing an app repository from an archive file
> New as of 0.24.0
A Dokku app repository can be initialized or updated from the contents of an archive file via the `git:from-archive` command. This is an excellent way of tracking changes when deploying pre-built binary archives, such as java jars or go binaries. This can also be useful when deploying directly from a Github repository at a specific commit.
```shell
dokku git:from-archive node-js-app https://github.com/dokku/smoke-test-app/releases/download/2.0.0/smoke-test-app.tar
```
In the above example, Dokku will build the app as if the repository contained the extracted contents of the specified archive file.
Triggering a build with the same archive file multiple times will result in Dokku exiting `0` early as there will be no changes detected.
The `git:from-archive` command can optionally take a git `user.name` and `user.email` argument (in that order) to customize the author. If the arguments are left empty, they will fallback to `Dokku` and `automated@dokku.sh`, respectively.
```shell
dokku git:from-archive node-js-app https://github.com/dokku/smoke-test-app/releases/download/2.0.0/smoke-test-app.tar "Camila" "camila@example.com"
```
The default archive type is always set to `.tar`. To use a different archive type, specify the `--archive-type` flag. Failure to do so will result in a failure to extract the archive.
```shell
dokku git:from-archive --archive-type zip node-js-app https://github.com/dokku/smoke-test-app/archive/2.0.0.zip "Camila" "camila@example.com"
```
Finally, if the archive url is specified as `--`, the archive will be fetched from stdin.
```shell
curl -sSL https://github.com/dokku/smoke-test-app/releases/download/2.0.0/smoke-test-app.tar | dokku git:from-archive node-js-app --
```
### Initializing an app repository from a remote repository
> New as of 0.23.0

2
docs/deployment/methods/tar.md
View File

@@ -1,5 +1,7 @@
# Tarball Deployment
> Warning: As of 0.24.0, this functionality is deprecated in favor of the [`git:from-archive`](/docs/deployment/methods/git.md#initializing-an-app-repository-from-an-archive-file) command. It will be removed in a future release, and is considered unmaintained. Users are highly encouraged to switch their workflows to `git:from-arhive`.
>
> New as of 0.4.0
```

1
plugins/git/git-from-directory
View File

@@ -51,6 +51,7 @@ trigger-git-git-from-directory() {
fn-git-fetch "$APP" "$TMP_WORK_DIR" "$REV"
fi
rm -rf "$TMP_WORK_DIR" "$TMP_WORK_DIR_2" >/dev/null
git_receive_app "$APP" "$REV"
}

3
plugins/git/help-functions
View File

@@ -29,7 +29,8 @@ fn-help-content() {
cat <<help_content
git:allow-host <host>, Adds a host to known_hosts
git:auth <host> [<username> <password>], Configures netrc authentication for a given git server
git:from-image <app> <docker-image> [<git-username> <git-email>], Updates a app's git repository with a given docker image
git:from-archive <app> <archive-url> [<git-username> <git-email>], Updates an app's git repository with a given archive file
git:from-image <app> <docker-image> [<git-username> <git-email>], Updates an app's git repository with a given docker image
git:sync [--build] <app> <repository> [<git-ref>], Clone or fetch an app from remote git repo
git:initialize <app>, Initialize a git repository for an app
git:public-key, Outputs the dokku public deploy key

87
plugins/git/internal-functions
View File

@@ -15,6 +15,89 @@ cmd-git-allow-host() {
dokku_log_info1 "$HOST added to known hosts"
}
cmd-git-from-archive() {
declare desc="updates an app's git repository with a given archive file"
local cmd="git:from-archive"
[[ "$1" == "$cmd" ]] && shift 1
declare APP ARCHIVE_URL USER_NAME USER_EMAIL
local ARCHIVE_TYPE="tar"
ARGS=()
skip=false
for arg in "$@"; do
if [[ "$arg" == "--archive-type" ]]; then
skip=true
continue
fi
if [[ "$skip" == "true" ]]; then
ARCHIVE_TYPE="$arg"
skip=false
continue
fi
ARGS+=("$arg")
done
APP="${ARGS[0]}"
ARCHIVE_URL="${ARGS[1]}"
USER_NAME="${ARGS[2]}"
USER_EMAIL="${ARGS[3]}"
verify_app_name "$APP"
[[ -z "$ARCHIVE_URL" ]] && dokku_log_fail "Please specify an archive url or -- to fetch the archive from stdin"
local VALID_ARCHIVE_TYPES=("tar" "tar.gz" "zip")
if ! fn-in-array "$ARCHIVE_TYPE" "${ARCHIVE_TYPE[@]}"; then
dokku_log_fail "Invalid archive type specified, valid archive types include: tar, tar.gz, zip"
fi
local TMP_WORK_DIR=$(mktemp -d "/tmp/dokku-${DOKKU_PID}-${FUNCNAME[0]}.XXXXXX")
local TMP_WORK_DIR_2=$(mktemp -d "/tmp/dokku-${DOKKU_PID}-${FUNCNAME[0]}.XXXXXX")
local TMP_WORK_DIR_3=$(mktemp -d "/tmp/dokku-${DOKKU_PID}-${FUNCNAME[0]}.XXXXXX")
trap "rm -rf '$TMP_WORK_DIR' '$TMP_WORK_DIR_2' '$TMP_WORK_DIR_3' >/dev/null" RETURN INT TERM EXIT
if [[ "$ARCHIVE_URL" == "--" ]]; then
dokku_log_info1 "Fetching $ARCHIVE_TYPE file from stdin"
tee "$TMP_WORK_DIR_2/src.$ARCHIVE_TYPE" | wc -c
else
dokku_log_info1 "Downloading $ARCHIVE_TYPE file from $ARCHIVE_URL"
curl -# -L "$ARCHIVE_URL" -o "$TMP_WORK_DIR_2/src.$ARCHIVE_TYPE"
fi
dokku_log_info1 "Generating build context"
if [[ "$ARCHIVE_TYPE" == "tar" ]]; then
local COMMON_PREFIX=$(tar -tf "$TMP_WORK_DIR_2/src.tar" | sed -e 'N;s/^\(.*\).*\n\1.*$/\1\n\1/;D')
local BOGUS_PARTS=$(echo "$COMMON_PREFIX " | awk 'BEGIN{FS="/"} {print NF-1}')
dokku_log_verbose "Striping $BOGUS_PARTS worth of directories from tarball"
tar -x -C "$TMP_WORK_DIR_3" -f "$TMP_WORK_DIR_2/src.tar" --strip-components="$BOGUS_PARTS"
elif [[ "$ARCHIVE_TYPE" == "tar.gz" ]]; then
dokku_log_verbose "Extracting gzipped tarball"
tar -x -C "$TMP_WORK_DIR_3" -f "$TMP_WORK_DIR_2/src.tar.gz" -z
elif [[ "$ARCHIVE_TYPE" == "zip" ]]; then
dokku_log_verbose "Extracting zipball"
unzip -d "$TMP_WORK_DIR_3" "$TMP_WORK_DIR_2/src.zip"
fi
chmod -R u+r "$TMP_WORK_DIR_3"
# drop any top-level folder components that resulted from the folder extraction
if [[ "$(find "$TMP_WORK_DIR_3" -maxdepth 1 -printf %y)" == "dd" ]]; then
dokku_log_verbose "Stripping top-level archive folder components"
local subpath="$(find "$TMP_WORK_DIR_3" -mindepth 1 -maxdepth 1 -type d)"
pushd "$subpath" >/dev/null
find . -maxdepth 1 -exec mv {} "$TMP_WORK_DIR" \;
popd &>/dev/null || pushd "/tmp" >/dev/null
else
dokku_log_verbose "Moving unarchived files and folders into place"
pushd "$TMP_WORK_DIR_3" >/dev/null
find . -maxdepth 1 -exec mv {} "$TMP_WORK_DIR" \;
popd &>/dev/null || pushd "/tmp" >/dev/null
fi
plugn trigger git-from-directory "$APP" "$TMP_WORK_DIR" "$USER_NAME" "$USER_EMAIL"
}
cmd-git-auth() {
declare desc="configures netrc authentication for a given git server"
local cmd="git:auth"
@@ -37,7 +120,7 @@ cmd-git-auth() {
}
cmd-git-from-image() {
declare desc="updates a app's git repository with a given docker image"
declare desc="updates an app's git repository with a given docker image"
local cmd="git:from-image"
[[ "$1" == "$cmd" ]] && shift 1
declare APP DOCKER_IMAGE USER_NAME USER_EMAIL
@@ -69,7 +152,7 @@ cmd-git-from-image() {
[[ -z "$DOCKER_IMAGE" ]] && dokku_log_fail "Please specify a docker image"
local TMP_WORK_DIR=$(mktemp -d "/tmp/dokku-${DOKKU_PID}-${FUNCNAME[0]}.XXXXXX")
trap "rm -rf '$TMP_WORK_DIR' '$TMP_WORK_DIR_2' >/dev/null" RETURN INT TERM EXIT
trap "rm -rf '$TMP_WORK_DIR' >/dev/null" RETURN INT TERM EXIT
dokku_log_info1 "Generating build context"
if [[ -n "$BUILD_DIR" ]]; then

6
plugins/git/subcommands/from-archive Executable file
View File

@@ -0,0 +1,6 @@
#!/usr/bin/env bash
source "$PLUGIN_AVAILABLE_PATH/git/internal-functions"
set -eo pipefail
[[ $DOKKU_TRACE ]] && set -x
cmd-git-from-archive "$@"

1
rpm.mk
View File

@@ -47,6 +47,7 @@ endif
--depends '/usr/bin/python3' \
--depends 'sshcommand >= 0.11.0' \
--depends 'sudo' \
--depends 'unzip' \
--after-install rpm/dokku.postinst \
--url "https://github.com/$(DOKKU_REPO_NAME)" \
--description $(DOKKU_DESCRIPTION) \

75
tests/unit/git_5.bats Normal file
View File

@@ -0,0 +1,75 @@
#!/usr/bin/env bats
load test_helper
setup() {
global_setup
create_app
touch /home/dokku/.ssh/known_hosts
chown dokku:dokku /home/dokku/.ssh/known_hosts
}
teardown() {
rm -f /home/dokku/.ssh/id_rsa.pub || true
destroy_app
global_teardown
}
@test "(git) git:from-archive [missing]" {
run /bin/bash -c "dokku git:from-archive $TEST_APP"
echo "output: $output"
echo "status: $status"
assert_failure
}
@test "(git) git:from-archive [invalid archive type]" {
run /bin/bash -c "dokku git:from-archive --archive-type tarball $TEST_APP http://example.com/src.tar"
echo "output: $output"
echo "status: $status"
assert_failure
}
@test "(git) git:from-archive [tar]" {
run /bin/bash -c "dokku git:from-archive $TEST_APP https://github.com/dokku/smoke-test-app/releases/download/2.0.0/smoke-test-app.tar"
echo "output: $output"
echo "status: $status"
assert_success
assert_http_success http://${TEST_APP}.dokku.me
}
@test "(git) git:from-archive [tar.gz]" {
run /bin/bash -c "dokku git:from-archive --archive-type tar.gz $TEST_APP https://github.com/dokku/smoke-test-app/archive/2.0.0.tar.gz"
echo "output: $output"
echo "status: $status"
assert_success
assert_http_success http://${TEST_APP}.dokku.me
}
@test "(git) git:from-archive [zip]" {
run /bin/bash -c "dokku git:from-archive --archive-type zip $TEST_APP https://github.com/dokku/smoke-test-app/archive/2.0.0.zip"
echo "output: $output"
echo "status: $status"
assert_success
assert_http_success http://${TEST_APP}.dokku.me
}
@test "(git) git:from-archive [stdin-file]" {
run /bin/bash -c "curl -sL https://github.com/dokku/smoke-test-app/releases/download/2.0.0/smoke-test-app.tar -o /tmp/smoke-test-app.tar"
echo "output: $output"
echo "status: $status"
assert_success
run /bin/bash -c "cat /tmp/smoke-test-app.tar | dokku git:from-archive $TEST_APP --"
echo "output: $output"
echo "status: $status"
assert_success
assert_http_success http://${TEST_APP}.dokku.me
}
@test "(git) git:from-archive [stdin-curl]" {
run /bin/bash -c "curl -sL https://github.com/dokku/smoke-test-app/releases/download/2.0.0/smoke-test-app.tar | dokku git:from-archive $TEST_APP --"
echo "output: $output"
echo "status: $status"
assert_success
assert_http_success http://${TEST_APP}.dokku.me
}