Add core storage plugin to manage docker bind mounts [ci skip]

docs/dokku-storage.md

@@ -0,0 +1,45 @@
+# Dokku Core Storage Plugin
+
+> Spanking New as of 0.5.0
+
+As of Dokku 0.5.0, the preferred method to mount external containers to a dokku managed container, is to use the dokku storage plugin.
+
+
+```shell
+$ dokku storage:help
+    storage:list <app>, List bind mounts for app's container(s) (host:container)
+    storage:mount <app> <host-dir:container-dir>, Create a new bind mount
+    storage:unmount <app> <host-dir:container-dir>, Remove an existing bind mount
+```
+
+## Ideology and Background
+The storage plugin requires explicit paths on the host side. This is intentional to ensure that new users avoid running into unexpected results with implicit paths that may not exist (a feature deprecate in [Docker 1.9.0](https://github.com/docker/docker/releases/tag/v1.9.0])). The container directory is created for the mount point in the container. Any existing directory contents are not accessible after a mount is added to the container. Dokku creates a new directory `/var/lib/dokku/data/storage` during installation, it's the general consensus that new users should use this directory. Mounts are only available at run and deploy times, you must redeploy (restart) an app to mount or unmount to an existing app's container.
+
+## Usage
+This example demonstrates how to mount the recommended directory to `/storage` inside the container:
+```
+$ dokku storage:mount app-name /var/lib/dokku/data/storage:/storage
+```
+Dokku will then mount the shared contents of`/var/lib/dokku/storage` to `/storage` inside the container.
+
+## Use Cases
+
+##### Persistent storage
+
+Dokku is powered by Docker containers, which recommends in their [best practices](https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/#containers-should-be-ephemeral) that containers be treated as ephemeral. In order to manage persistent storage for web applications, like user uploads or large binary assets like images, a directory outside the container should be mounted.
+
+##### Shared storage between containers
+
+When scaling your app, you may require a common location to access shared assets between containers, a storage mount can be used in this situation.
+
+##### Shared storage across environments
+
+Your app may be used in a cluster that requires containers or resources not running on the same host access your data. Mounting a shared file service (like S3FS or EFS) inside your container will give you great flexibility.
+
+##### Backing up
+
+Your app may have services that are running in memory and need to be backed up locally (like a key store). Mount a non ephemeral storage mount will allow backups that are not lost when the app is shut down.
+
+## Docker-Options Note
+
+The storage plugins is compatible with storage mounts created with the docker-options. The storage plugin will only list mounts from the deploy phase.

docs/template.html

@@ -81,6 +81,7 @@
             <a href="/dokku/nginx/" class="list-group-item">Nginx Configuration</a>
             <a href="/dokku/deployment/ssl-configuration/" class="list-group-item">SSL Configuration</a>
             <a href="/dokku/docker-options/" class="list-group-item">Docker Container Options</a>
+            <a href="/dokku/dokku-storage/" class="list-group-item">Storage Plugin</a>
             <a href="/dokku/dokku-events-logs/" class="list-group-item">Dokku Event Logs</a>
 
             <a href="#" class="list-group-item disabled">Community Contributions</a>

plugins/storage/commands

@@ -0,0 +1,50 @@
+#!/usr/bin/env bash
+[[ " storage storage:list storage:mount storage:unmount storage:help help " == *" $1 "* ]] || exit "$DOKKU_NOT_IMPLEMENTED_EXIT"
+set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x
+source "$PLUGIN_CORE_AVAILABLE_PATH/common/functions"
+source "$PLUGIN_CORE_AVAILABLE_PATH/storage/functions"
+source "$PLUGIN_CORE_AVAILABLE_PATH/docker-options/functions"
+
+case "$1" in
+  storage:list)
+    # List all bound mounts
+    passed_phases="deploy"
+    [[ ! $2 ]] && dokku_log_fail "storage:list requires an app to list."
+    verify_app_name "$2" && APP="$2"
+    echo "$APP volume bind-mounts:"
+    get_bind_mounts "$(get_phase_file_path "$passed_phases")"
+    ;;
+
+  storage:mount)
+    # Add bind-mount, redeploy app if running
+    passed_phases=(deploy run)
+    [[ -z $3 || $4 ]] && dokku_log_fail "storage:mount requires an app and a two paths devided by colon."
+    verify_app_name "$2" && APP="$2"
+    verify_paths "$3" && MOUNT_PATH="$3"
+    check_if_path_exists "$MOUNT_PATH" "$(get_phase_file_path "${passed_phases[@]}")" && dokku_log_fail "Mount path already exists."
+    add_passed_docker_option passed_phases[@] "-v $MOUNT_PATH"
+    ;;
+
+  storage:unmount)
+    # Remove bind-mount, restart app if running
+    passed_phases=(deploy run)
+    [[ -z $3 || $4 ]] && dokku_log_fail "storage:unmount requires an app and a two paths devided by colon."
+    verify_app_name "$2" && APP="$2"
+    verify_paths "$3" && MOUNT_PATH="$3"
+    check_if_path_exists "$MOUNT_PATH" "$(get_phase_file_path "${passed_phases[@]}")" || dokku_log_fail "Mount path does not exist."
+    remove_passed_docker_option passed_phases[@] "-v $MOUNT_PATH"
+    ;;
+
+  help | storage:help)
+    cat<<EOF
+    storage:list <app>, List bind mounts for app's container(s) (host:container)
+    storage:mount <app> <host-dir:container-dir>, Create a new bind mount
+    storage:unmount <app> <host-dir:container-dir>, Remove an existing bind mount
+EOF
+    ;;
+
+  *)
+    exit "$DOKKU_NOT_IMPLEMENTED_EXIT"
+    ;;
+
+esac

plugins/storage/functions

@@ -0,0 +1,18 @@
+#!/usr/bin/env bash
+set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x
+source "$PLUGIN_CORE_AVAILABLE_PATH/common/functions"
+
+verify_paths() {
+  echo "$1" | grep -qe '^/.*\:/' || dokku_log_fail "Storage path must be two valid paths devided by colon."
+}
+
+check_if_path_exists() {
+  local -r passed_path=$1
+  local -r phase_file_path=$2
+  [[ -r "$phase_file_path" ]] && grep -qe "^-v $passed_path" "$phase_file_path"
+}
+
+get_bind_mounts() {
+  local -r phase_file_path=$1
+  [[ -r "$phase_file_path" ]] && sed -e '/^-v/!d' -e 's/^-v/    /' < "$phase_file_path"
+}

plugins/storage/install

@@ -0,0 +1,5 @@
+#!/usr/bin/env bash
+set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x
+
+mkdir -p /var/lib/dokku/data/storage
+chown dokku:dokku /var/lib/dokku/data/storage

plugins/storage/plugin.toml

@@ -0,0 +1,4 @@
+[plugin]
+description = "dokku core storage plugin"
+version = "0.5.0"
+[plugin.config]

tests/unit/20_storage.bats

@@ -0,0 +1,40 @@
+#!/usr/bin/env bats
+
+load test_helper
+
+setup() {
+  create_app
+}
+
+teardown() {
+  destroy_app
+}
+
+@test "(storage) storage:mount, storage:list, storage:umount" {
+  run /bin/bash -c "dokku storage:mount $TEST_APP /tmp/mount:/mount"
+  echo "output: "$output
+  echo "status: "$status
+  assert_success
+  run /bin/bash -c "dokku storage:list $TEST_APP | grep -qe '^\s*/tmp/mount:/mount'"
+  echo "output: "$output
+  echo "status: "$status
+  assert_success
+  run /bin/bash -c "dokku storage:mount $TEST_APP /tmp/mount:/mount"
+  echo "output: "$output
+  echo "status: "$status
+  assert_output "Mount path already exists."
+  assert_failure
+  run /bin/bash -c "dokku storage:unmount $TEST_APP /tmp/mount:/mount"
+  echo "output: "$output
+  echo "status: "$status
+  assert_success
+  run /bin/bash -c "dokku storage:list $TEST_APP | grep -vqe '^\s*/tmp/mount:/mount'"
+  echo "output: "$output
+  echo "status: "$status
+  assert_success
+  run /bin/bash -c "dokku storage:unmount $TEST_APP /tmp/mount:/mount"
+  echo "output: "$output
+  echo "status: "$status
+  assert_output "Mount path does not exist."
+  assert_failure
+}