Merge pull request #8538 from dokku/6814-k3s-plugin-add-pv-option

Add scheduler-aware named storage entries
This commit is contained in:
Jose Diaz-Gonzalez
2026-04-30 12:10:30 -04:00
committed by GitHub
38 changed files with 4011 additions and 106 deletions

View File

@@ -1,24 +1,37 @@
# Persistent Storage
> [!IMPORTANT]
> New as of 0.5.0
> New as of 0.5.0. Named storage entries new as of 0.38.0.
The preferred method to mount external containers to a Dokku managed container, is to use the Dokku storage plugin.
The preferred method to attach persistent storage to a Dokku-managed container is the Dokku storage plugin.
```
storage:ensure-directory [--chown option] <directory> # Creates a persistent storage directory in the recommended storage path
storage:list <app> [--format text|json] # List bind mounts for app's container(s) (host:container)
storage:mount <app> <host-dir:container-dir> # Create a new bind mount
storage:report [<app>] [<flag>] # Displays a checks report for one or more apps
storage:unmount <app> <host-dir:container-dir> # Remove an existing bind mount
storage:create <name> [<path>] [flags] # Register a named storage entry
storage:destroy <name> # Remove a named storage entry (must be unmounted from every app first)
storage:ensure-directory [--chown option] <directory> # [DEPRECATED] use storage:create instead
storage:exec <name> [-- <cmd>...] # Run a command (or shell) in a temporary container that mounts the entry
storage:info <name> [--format text|json] # Show details for one storage entry
storage:list <app> [--format text|json] # List bind mounts for an app's container(s) (legacy host:container view)
storage:list-entries [--scheduler s] [--format text|json] # List registered storage entries
storage:mount <app> <name> --container-dir <path> [flags] # Mount a named entry into an app
storage:mount <app> <host-dir:container-dir> # [LEGACY] colon-form mount, docker-local only
storage:report [<app>] [<flag>] # Display a storage report for one or more apps
storage:report --global # Display a cluster-wide entry inventory
storage:set <name> [flags] # Update a storage entry in place
storage:unmount <app> <name> [--container-dir <path>] # Remove an attachment
storage:wait <name> # Block until a k3s entry's PVC is bound
```
> The storage plugin is compatible with storage mounts created with the docker-options. The storage plugin will only list mounts from the deploy/run phase.
A storage entry is the source of truth for the underlying volume - a host directory on docker-local, or a PersistentVolumeClaim on k3s. Multiple apps can mount the same entry, and an attachment carries the per-app details (container path, phases, subpath, readonly, process type). Names are globally unique across the install and must be DNS-1123 labels (lowercase letters, digits, dashes) of 45 characters or less so they can be used verbatim as Helm release and PVC names.
The legacy `storage:mount <app> <host>:<container>` form continues to work on docker-local. On a k3s app it is rejected; create a named entry with `storage:create --scheduler k3s` and mount it instead. Existing colon-form mounts are migrated automatically the first time the new code runs - they show up as `legacy-<hash>` entries in `storage:list-entries`.
The storage plugin supports the following mount points:
- explicit paths that exist on the host
- docker volumes
- explicit paths that exist on the host (docker-local)
- docker volumes (docker-local)
- PersistentVolumeClaims provisioned via a StorageClass (k3s)
- hostPath-backed PVs (k3s)
## Usage

View File

@@ -22,6 +22,7 @@
- The `docker-local` scheduler no longer queues an image for retirement when another running container of the same app still uses it. This fixes the case where a `ps:rebuild` against an image-based deploy (`git:from-image`) produced an identical-SHA image and the `dokku-retire` cron timer would log `Image ... has running containers, skipping rm` on every run. Stuck entries from prior versions are pruned automatically on the next `ps:retire` run.
- All `:report` subcommands now accept the `--global` flag, which scopes the report to globally-configured properties. The flag composes with `--format json`, so a JSON report of global properties can be obtained via, for example, `dokku scheduler:report --global --format json`. Previously, combining `--global` with `--format json` was rejected with an "info flag" error, and `--global` on its own was treated as an unknown flag.
- The `scheduler-k3s` plugin now manages env config and the dokku-generated image pull Secret as their own helm releases with stable names (`config-{app}` and `pull-secret-{app}`) rather than bundling them into the app helm chart with a per-deploy timestamp suffix (`env-{app}.{ts}` / `ims-{app}.{ts}`). This fixes two bugs: a helm rollback of the app chart no longer deletes Secrets that older ReplicaSets still reference, and the Deployment's `imagePullSecrets` list no longer accumulates references to nonexistent Secrets across deploys. The next deploy of an app switches the Deployment's `envFrom` and `imagePullSecrets` references to the stable names and prunes any leaked entries; existing live Deployments do not need to be patched manually. App rename now also uninstalls the old `tls-{app}`, `config-{app}`, and `pull-secret-{app}` releases under the previous app name; the new name's releases are recreated on the next deploy or certs sync.
- The storage plugin now treats persistent volumes as named, scheduler-aware first-class resources via `storage:create`, `storage:mount`, `storage:set`, and `storage:destroy`. The legacy `storage:mount <app> <host>:<container>` colon form continues to work on docker-local apps but is deprecated; on k3s apps it is rejected. Existing colon-form mounts are migrated automatically the first time the new storage plugin runs (during the install trigger) - they appear as `legacy-<hash>` entries in `storage:list-entries`. The migration is idempotent and tied to a per-app flag file at `$DOKKU_LIB_ROOT/config/storage/.migrated/<app>`; deleting that file forces a re-scan on the next install. The `storage:ensure-directory` command keeps working but now emits a deprecation warning - prefer `storage:create <name> [<path>]` (the path defaults to the same `$DOKKU_LIB_ROOT/data/storage/<name>` location). Storage entry names must now be DNS-1123 labels of 45 characters or less so they can be used verbatim as Helm release and Kubernetes resource names; underscores and uppercase characters that the older `ensure-directory` validator accepted are rejected for new names. The migration synthesizer always uses lowercase hex hashes so existing data is never locked out.
### TLS handshake behavior change

View File

@@ -685,6 +685,27 @@ Alternatively, a comma separated list of chart names can be specified to only fo
dokku scheduler-k3s:ensure-charts --charts cert-manager
```
### Persistent storage
K3s apps mount persistent volumes through the Dokku storage plugin. A `storage:create --scheduler k3s` call provisions a PersistentVolumeClaim through a small per-entry helm release; `storage:mount <app> <name> --container-dir <path>` then attaches it to an app. Because storage entries are global, the same PVC can be mounted into multiple apps (for `ReadWriteMany` workloads) or into the same app at multiple subpaths.
```shell
# dynamically-provisioned PVC backed by Longhorn
dokku storage:create demo-data \
--scheduler k3s \
--size 2Gi \
--access-mode ReadWriteOnce \
--storage-class-name longhorn
dokku storage:mount demo demo-data --container-dir /data
dokku storage:wait demo-data
git push dokku master
```
For a hostPath-backed PV (no StorageClass), pass `<path>` as the second positional argument and omit `--storage-class-name`. The plugin renders both the PV and the PVC into the entry's helm release. The `--reclaim-policy` flag (`Retain` or `Delete`) controls whether the underlying PV survives `storage:destroy`. Annotations and labels on `storage:create` / `storage:set` propagate to both the PVC and the PV so backup tools (Velero, Longhorn snapshots) can find them.
The legacy `storage:mount <app> <host>:<container>` colon form is rejected on k3s apps; create a named entry instead. See [Persistent Storage](/docs/advanced-usage/persistent-storage.md) for the full command reference.
### Chart upgrade callbacks
Some chart version transitions require side-effects that a plain `helm upgrade` cannot perform. For example, the upgrade of `keda-add-ons-http` to `0.12.2` introduces breaking changes to deployment selectors, which are immutable in Kubernetes; the chart-managed deployments must be deleted before the upgrade can proceed.

View File

@@ -2904,21 +2904,52 @@ DOKKU_SCHEDULER="$1"; APP="$2"; REMOVE_CONTAINERS="$3";
# TODO
```
### `storage-list`
### `storage-list` (deprecated)
- Description: Returns a list of storage mounts
- Invoked by: `dokku storage:list` and `dokku deploy`
> [!WARNING]
> Deprecated as of 0.38.0. The trigger handler still functions for back-compat
> with external plugins but emits a deprecation warning on every invocation.
> In-process callers should use the `storage-app-mounts` trigger instead, or
> the `storage` Go package directly.
- Description: Returns a list of storage mounts for an app
- Invoked by: external plugins (the core `storage:list` subcommand no longer uses it)
- Arguments: `$APP $PHASE $FORMAT`
- Example:
```shell
#!/usr/bin/env bash
### `storage-app-mounts`
set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x
APP="$1"
- Description: Returns the (entry, attachment) pairs an app has for the given phase, in JSON. Schedulers consume this to render volume mounts.
- Invoked by: scheduler plugins during deploy
- Arguments: `$APP $PHASE`
- Output: a JSON array of objects shaped `{"entry": Entry, "attachment": Attachment}`
# TODO
```
### `storage-create`
- Description: Provisions or updates the underlying volume for a k3s storage entry. Called when `storage:create` or `storage:set` runs against a k3s entry. Receives the JSON-encoded entry on stdin.
- Invoked by: `dokku storage:create`, `dokku storage:set`
- Arguments: `$ENTRY_NAME`
- Stdin: JSON-encoded `Entry` payload
### `storage-destroy`
- Description: Releases the underlying volume for a k3s storage entry. Called when `storage:destroy` runs against a k3s entry. Receives the JSON-encoded entry on stdin.
- Invoked by: `dokku storage:destroy`
- Arguments: `$ENTRY_NAME`
- Stdin: JSON-encoded `Entry` payload
### `storage-status`
- Description: Returns the current status of a storage entry's underlying volume. Used by `storage:wait` to poll the cluster. The handler should print one line: `Bound`, `Pending`, or `Lost` for k3s entries.
- Invoked by: `dokku storage:wait`
- Arguments: `$ENTRY_NAME`
- Stdin: JSON-encoded `Entry` payload
### `scheduler-storage-exec`
- Description: Runs an interactive or non-interactive command against a storage entry. The storage plugin fires this with `<scheduler>` as the first arg; each scheduler plugin's handler matches against its own scheduler name and either handles or no-ops, mirroring `scheduler-deploy` / `scheduler-app-status`. Plugn forwards stdin/stdout/stderr to the handler subprocess so an interactive shell streams cleanly. The handler exits with the underlying tool's status code so `dokku storage:exec` propagates exit codes verbatim.
- Invoked by: `dokku storage:exec`
- Arguments: `$SCHEDULER $ENTRY_NAME $IMAGE [-- $cmd...]`
- Flags: `--interactive` (stdin is open), `--tty` (stdin is a terminal), `--as-user <uid>` (override `entry.Chown`).
### `traefik-template-source`

View File

@@ -21,4 +21,5 @@ use (
./plugins/scheduler
./plugins/scheduler-docker-local
./plugins/scheduler-k3s
./plugins/storage
)

View File

@@ -1,4 +1,4 @@
TRIGGERS = triggers/scheduler-cron-write
TRIGGERS = triggers/scheduler-cron-write triggers/scheduler-storage-exec
BUILD = triggers
PLUGIN_NAME = scheduler-docker-local

View File

@@ -5,13 +5,16 @@ go 1.26.2
require (
github.com/dokku/dokku/plugins/common v0.0.0-00010101000000-000000000000
github.com/dokku/dokku/plugins/cron v0.0.0-00010101000000-000000000000
github.com/dokku/dokku/plugins/storage v0.0.0-00010101000000-000000000000
github.com/multiformats/go-base36 v0.2.0
github.com/spf13/pflag v1.0.10
golang.org/x/sync v0.20.0
)
require (
github.com/alexellis/go-execute/v2 v2.2.1 // indirect
github.com/dokku/dokku/plugins/app-json v0.0.0-00010101000000-000000000000 // indirect
github.com/dokku/dokku/plugins/docker-options v0.0.0-00010101000000-000000000000 // indirect
github.com/fatih/color v1.19.0 // indirect
github.com/hashicorp/errwrap v1.0.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
@@ -38,3 +41,7 @@ replace github.com/dokku/dokku/plugins/app-json => ../app-json
replace github.com/dokku/dokku/plugins/common => ../common
replace github.com/dokku/dokku/plugins/cron => ../cron
replace github.com/dokku/dokku/plugins/storage => ../storage
replace github.com/dokku/dokku/plugins/docker-options => ../docker-options

View File

@@ -46,6 +46,8 @@ github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0t
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/ryanuber/columnize v2.1.2+incompatible h1:C89EOx/XBWwIXl8wm8OPJBd7kPF25UfsK2X7Ph/zCAk=
github.com/ryanuber/columnize v2.1.2+incompatible/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/tailscale/hujson v0.0.0-20250605163823-992244df8c5a h1:a6TNDN9CgG+cYjaeN8l2mc4kSz2iMiCDQxPEyltUV/I=

View File

@@ -1,11 +1,12 @@
package main
import (
"flag"
"fmt"
"os"
"strings"
flag "github.com/spf13/pflag"
"github.com/dokku/dokku/plugins/common"
schedulerdockerlocal "github.com/dokku/dokku/plugins/scheduler-docker-local"
)
@@ -14,6 +15,10 @@ import (
func main() {
parts := strings.Split(os.Args[0], "/")
trigger := parts[len(parts)-1]
interactive := flag.Bool("interactive", false, "--interactive: stdin is open")
tty := flag.Bool("tty", false, "--tty: stdin is a terminal")
asUser := flag.String("as-user", "", "--as-user: numeric uid override")
flag.Parse()
var err error
@@ -21,6 +26,27 @@ func main() {
case "scheduler-cron-write":
scheduler := flag.Arg(0)
err = schedulerdockerlocal.TriggerSchedulerCronWrite(scheduler)
case "scheduler-storage-exec":
args := flag.Args()
if len(args) < 3 {
err = fmt.Errorf("scheduler-storage-exec requires <scheduler> <entry-name> <image>")
break
}
scheduler := args[0]
entryName := args[1]
image := args[2]
var cmd []string
if len(args) > 3 {
cmd = args[3:]
}
err = schedulerdockerlocal.TriggerSchedulerStorageExec(scheduler, schedulerdockerlocal.StorageExecInput{
EntryName: entryName,
Image: image,
Interactive: *interactive,
Tty: *tty,
AsUser: *asUser,
Command: cmd,
})
default:
err = fmt.Errorf("Invalid plugin trigger call: %s", trigger)
}

View File

@@ -0,0 +1,164 @@
package schedulerdockerlocal
import (
"errors"
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
"github.com/dokku/dokku/plugins/common"
"github.com/dokku/dokku/plugins/storage"
)
// dockerNamedVolumeRegexp mirrors storage.dockerNamedVolumeRegexp and is
// used to decide whether a HostPath should be statted (filesystem) or
// looked up via `docker volume inspect`.
var dockerNamedVolumeRegexp = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9_.-]+$`)
// StorageExecInput captures the inputs forwarded by the storage plugin
// over the scheduler-storage-exec trigger.
type StorageExecInput struct {
EntryName string
Image string
Interactive bool
Tty bool
AsUser string
Command []string
}
// TriggerSchedulerStorageExec runs an interactive or non-interactive
// command in a throwaway docker container that mounts the storage
// entry's host path or named volume at /data. Returns the exit code of
// docker run via os.Exit so the caller's status mirrors the underlying
// tool.
func TriggerSchedulerStorageExec(scheduler string, input StorageExecInput) error {
if scheduler != "docker-local" {
// Not for us; let other handlers respond.
return nil
}
if !storage.EntryExists(input.EntryName) {
return fmt.Errorf("storage entry %q does not exist", input.EntryName)
}
entry, err := storage.LoadEntry(input.EntryName)
if err != nil {
return err
}
if entry.Scheduler != storage.SchedulerDockerLocal {
return fmt.Errorf("storage entry %q has scheduler %q, not docker-local", entry.Name, entry.Scheduler)
}
if err := preflightDockerLocalSource(entry.HostPath); err != nil {
return err
}
args, err := buildDockerExecArgs(entry, input)
if err != nil {
return err
}
result, err := common.CallExecCommand(common.ExecCommandInput{
Command: common.DockerBin(),
Args: args,
StreamStdio: true,
})
// CallExecCommand wraps non-zero exit as an error; for storage:exec
// the docker-run exit code is the signal we want to forward to the
// caller. Propagate it before falling through to the err return,
// which would otherwise be collapsed to exit 1 by the dispatcher.
if result.ExitCode != 0 {
os.Exit(result.ExitCode)
}
if err != nil {
return err
}
return nil
}
// preflightDockerLocalSource fails fast when the host path or named
// volume backing the entry doesn't exist, so the user gets a real error
// instead of a generic "docker: ..." line.
func preflightDockerLocalSource(hostPath string) error {
if hostPath == "" {
return errors.New("storage entry is missing a host path")
}
if filepath.IsAbs(hostPath) {
info, err := os.Stat(hostPath)
if err != nil {
if os.IsNotExist(err) {
return fmt.Errorf("host path %s does not exist", hostPath)
}
return fmt.Errorf("unable to stat host path %s: %w", hostPath, err)
}
if !info.IsDir() {
return fmt.Errorf("host path %s exists but is not a directory", hostPath)
}
return nil
}
if !dockerNamedVolumeRegexp.MatchString(hostPath) {
return fmt.Errorf("host path %q is neither absolute nor a valid docker volume token", hostPath)
}
result, err := common.CallExecCommand(common.ExecCommandInput{
Command: common.DockerBin(),
Args: []string{"volume", "inspect", hostPath},
})
if err != nil {
return err
}
if result.ExitCode != 0 {
return fmt.Errorf("docker named volume %q is not present: %s", hostPath, strings.TrimSpace(result.StderrContents()))
}
return nil
}
// buildDockerExecArgs assembles the `docker run` argv for a storage:exec
// invocation. Split out so the unit test can exercise it without
// actually invoking docker.
func buildDockerExecArgs(entry *storage.Entry, input StorageExecInput) ([]string, error) {
cmd := input.Command
if len(cmd) == 0 {
cmd = []string{"sh", "-c", "command -v bash >/dev/null 2>&1 && exec bash || exec sh"}
}
args := []string{"run", "--rm"}
if input.Tty {
args = append(args, "-it")
} else if input.Interactive {
args = append(args, "-i")
}
user, err := resolveUser(entry.Chown, input.AsUser)
if err != nil {
return nil, err
}
if user != "" {
args = append(args, "--user", user)
}
args = append(args, "-v", fmt.Sprintf("%s:/data", entry.HostPath))
args = append(args, input.Image)
args = append(args, cmd...)
return args, nil
}
// resolveUser turns the entry's Chown setting plus an optional --as-user
// override into a docker --user value. Returns an empty string when the
// container should run as the image's default user.
func resolveUser(chown string, asUser string) (string, error) {
if asUser != "" {
return fmt.Sprintf("%s:%s", asUser, asUser), nil
}
if chown == "" || chown == "false" {
return "", nil
}
uid, err := storage.ResolveChownID(chown)
if err != nil {
return "", err
}
if uid == "" || uid == "false" {
return "", nil
}
return fmt.Sprintf("%s:%s", uid, uid), nil
}

View File

@@ -0,0 +1,118 @@
package schedulerdockerlocal
import (
"strings"
"testing"
"github.com/dokku/dokku/plugins/storage"
)
func TestBuildDockerExecArgsTTY(t *testing.T) {
entry := &storage.Entry{
Name: "demo-data",
Scheduler: storage.SchedulerDockerLocal,
HostPath: "/var/lib/dokku/data/storage/demo-data",
}
cases := []struct {
name string
interactive bool
tty bool
want string
}{
{"non-interactive", false, false, ""},
{"piped stdin", true, false, "-i"},
{"interactive tty", true, true, "-it"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
args, err := buildDockerExecArgs(entry, StorageExecInput{
Image: "alpine:3",
Interactive: tc.interactive,
Tty: tc.tty,
Command: []string{"ls", "/data"},
})
if err != nil {
t.Fatalf("buildDockerExecArgs returned error: %v", err)
}
joined := strings.Join(args, " ")
if tc.want == "" {
if strings.Contains(joined, " -i ") || strings.Contains(joined, " -it ") {
t.Fatalf("expected no -i / -it flag, got: %s", joined)
}
} else if !strings.Contains(joined, " "+tc.want+" ") {
t.Fatalf("expected to find %q in args, got: %s", tc.want, joined)
}
})
}
}
func TestBuildDockerExecArgsUser(t *testing.T) {
entry := &storage.Entry{
Name: "demo-data",
Scheduler: storage.SchedulerDockerLocal,
HostPath: "/var/lib/dokku/data/storage/demo-data",
Chown: "false",
}
args, err := buildDockerExecArgs(entry, StorageExecInput{
Image: "alpine:3",
Command: []string{"ls"},
})
if err != nil {
t.Fatalf("err: %v", err)
}
if strings.Contains(strings.Join(args, " "), "--user") {
t.Fatalf("did not expect --user flag with chown=false, got: %v", args)
}
args, err = buildDockerExecArgs(entry, StorageExecInput{
Image: "alpine:3",
AsUser: "1234",
Command: []string{"ls"},
})
if err != nil {
t.Fatalf("err: %v", err)
}
if !strings.Contains(strings.Join(args, " "), "--user 1234:1234") {
t.Fatalf("expected --user 1234:1234, got: %v", args)
}
}
func TestBuildDockerExecArgsDefaultsToShellWhenNoCommand(t *testing.T) {
entry := &storage.Entry{
Name: "demo-data",
Scheduler: storage.SchedulerDockerLocal,
HostPath: "/var/lib/dokku/data/storage/demo-data",
}
args, err := buildDockerExecArgs(entry, StorageExecInput{
Image: "alpine:3",
})
if err != nil {
t.Fatalf("err: %v", err)
}
last := args[len(args)-3:]
if last[0] != "sh" || last[1] != "-c" {
t.Fatalf("expected the trailing command to be a shell fallback, got: %v", last)
}
}
func TestBuildDockerExecArgsMountsHostPath(t *testing.T) {
entry := &storage.Entry{
Name: "demo-data",
Scheduler: storage.SchedulerDockerLocal,
HostPath: "/srv/demo",
}
args, err := buildDockerExecArgs(entry, StorageExecInput{
Image: "alpine:3",
Command: []string{"ls"},
})
if err != nil {
t.Fatalf("err: %v", err)
}
joined := strings.Join(args, " ")
if !strings.Contains(joined, "-v /srv/demo:/data") {
t.Fatalf("expected -v /srv/demo:/data, got: %s", joined)
}
}

View File

@@ -1,5 +1,5 @@
SUBCOMMANDS = subcommands/annotations:set subcommands/autoscaling-auth:set subcommands/autoscaling-auth:report subcommands/cluster:add subcommands/cluster:list subcommands/cluster:remove subcommands/ensure-charts subcommands/initialize subcommands/labels:set subcommands/profiles:add subcommands/profiles:list subcommands/profiles:remove subcommands/report subcommands/set subcommands/show-kubeconfig subcommands/uninstall
TRIGGERS = triggers/core-post-deploy triggers/core-post-extract triggers/install triggers/post-app-clone-setup triggers/post-app-rename-setup triggers/post-certs-update triggers/post-certs-remove triggers/post-create triggers/post-delete triggers/report triggers/scheduler-app-status triggers/scheduler-deploy triggers/scheduler-enter triggers/scheduler-is-deployed triggers/scheduler-logs triggers/scheduler-proxy-config triggers/scheduler-proxy-logs triggers/scheduler-post-delete triggers/scheduler-run triggers/scheduler-run-list triggers/scheduler-stop triggers/scheduler-cron-write
TRIGGERS = triggers/core-post-deploy triggers/core-post-extract triggers/install triggers/post-app-clone-setup triggers/post-app-rename-setup triggers/post-certs-update triggers/post-certs-remove triggers/post-create triggers/post-delete triggers/report triggers/scheduler-app-status triggers/scheduler-deploy triggers/scheduler-enter triggers/scheduler-is-deployed triggers/scheduler-logs triggers/scheduler-proxy-config triggers/scheduler-proxy-logs triggers/scheduler-post-delete triggers/scheduler-run triggers/scheduler-run-list triggers/scheduler-stop triggers/scheduler-cron-write triggers/storage-create triggers/storage-destroy triggers/storage-status triggers/scheduler-storage-exec
BUILD = commands subcommands triggers
PLUGIN_NAME = scheduler-k3s

View File

@@ -13,6 +13,7 @@ require (
github.com/dokku/dokku/plugins/logs v0.0.0-00010101000000-000000000000
github.com/dokku/dokku/plugins/nginx-vhosts v0.0.0-00010101000000-000000000000
github.com/dokku/dokku/plugins/registry v0.0.0-00010101000000-000000000000
github.com/dokku/dokku/plugins/storage v0.0.0-00010101000000-000000000000
github.com/fatih/color v1.19.0
github.com/fluxcd/pkg/kustomize v1.31.0
github.com/go-openapi/jsonpointer v0.23.1
@@ -193,6 +194,8 @@ replace github.com/dokku/dokku/plugins/nginx-vhosts => ../nginx-vhosts
replace github.com/dokku/dokku/plugins/registry => ../registry
replace github.com/dokku/dokku/plugins/storage => ../storage
replace github.com/joho/godotenv => github.com/joho/godotenv v1.2.0
replace sigs.k8s.io/controller-runtime => sigs.k8s.io/controller-runtime v0.22.4

View File

@@ -1,6 +1,7 @@
package main
import (
"context"
"fmt"
"os"
"strconv"
@@ -18,6 +19,9 @@ func main() {
parts := strings.Split(os.Args[0], "/")
trigger := parts[len(parts)-1]
podIdentifier := flag.String("container-id", "", "--container-id: A pod identifier")
storageExecInteractive := flag.Bool("interactive", false, "--interactive: stdin is open (storage:exec)")
storageExecTty := flag.Bool("tty", false, "--tty: stdin is a terminal (storage:exec)")
storageExecAsUser := flag.String("as-user", "", "--as-user: numeric uid override (storage:exec)")
flag.Parse()
var err error
@@ -158,6 +162,36 @@ func main() {
appName := flag.Arg(1)
format := flag.Arg(2)
err = scheduler_k3s.TriggerSchedulerRunList(scheduler, appName, format)
case "storage-create":
entryName := flag.Arg(0)
err = scheduler_k3s.TriggerStorageCreate(context.Background(), entryName)
case "storage-destroy":
entryName := flag.Arg(0)
err = scheduler_k3s.TriggerStorageDestroy(context.Background(), entryName)
case "storage-status":
entryName := flag.Arg(0)
err = scheduler_k3s.TriggerStorageStatus(context.Background(), entryName)
case "scheduler-storage-exec":
positional := flag.Args()
if len(positional) < 3 {
err = fmt.Errorf("scheduler-storage-exec requires <scheduler> <entry-name> <image>")
break
}
schedulerName := positional[0]
entryName := positional[1]
image := positional[2]
var cmd []string
if len(positional) > 3 {
cmd = positional[3:]
}
err = scheduler_k3s.TriggerSchedulerStorageExec(context.Background(), schedulerName, scheduler_k3s.StorageExecInput{
EntryName: entryName,
Image: image,
Interactive: *storageExecInteractive,
Tty: *storageExecTty,
AsUser: *storageExecAsUser,
Command: cmd,
})
default:
err = fmt.Errorf("Invalid plugin trigger call: %s", trigger)
}

View File

@@ -0,0 +1,313 @@
package scheduler_k3s
import (
"context"
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"github.com/dokku/dokku/plugins/common"
storage "github.com/dokku/dokku/plugins/storage"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
// StorageChartValues are the helm chart inputs for the per-entry storage chart.
type StorageChartValues struct {
Global StorageChartGlobal `yaml:"global"`
}
// StorageChartGlobal contains global values for the storage chart.
type StorageChartGlobal struct {
EntryName string `yaml:"entry_name"`
Namespace string `yaml:"namespace"`
Size string `yaml:"size"`
AccessMode string `yaml:"access_mode,omitempty"`
StorageClass string `yaml:"storage_class,omitempty"`
HostPath string `yaml:"host_path,omitempty"`
ReclaimPolicy string `yaml:"reclaim_policy,omitempty"`
Annotations map[string]string `yaml:"annotations,omitempty"`
Labels map[string]string `yaml:"labels,omitempty"`
}
// GetStorageReleaseName returns the helm release name for a storage entry.
func GetStorageReleaseName(entryName string) string {
return fmt.Sprintf("storage-%s", entryName)
}
// readEntryFromStdin decodes a storage.Entry from stdin. The storage
// plugin pipes the JSON-encoded entry to scheduler triggers.
func readEntryFromStdin() (*storage.Entry, error) {
raw, err := io.ReadAll(os.Stdin)
if err != nil {
return nil, fmt.Errorf("unable to read storage entry from stdin: %w", err)
}
if len(raw) == 0 {
return nil, fmt.Errorf("no storage entry payload received on stdin")
}
entry := &storage.Entry{}
if err := json.Unmarshal(raw, entry); err != nil {
return nil, fmt.Errorf("unable to parse storage entry payload: %w", err)
}
return entry, nil
}
// TriggerStorageCreate provisions or updates the PVC/PV for a k3s storage
// entry. Idempotent: re-runs against the same entry are a helm upgrade.
func TriggerStorageCreate(ctx context.Context, entryName string) error {
if err := isKubernetesAvailable(); err != nil {
return fmt.Errorf("kubernetes not available: %w", err)
}
entry, err := readEntryFromStdin()
if err != nil {
return err
}
if entry.Name != entryName {
return fmt.Errorf("entry name mismatch: stdin=%q, arg=%q", entry.Name, entryName)
}
if entry.Scheduler != storage.SchedulerK3s {
return nil
}
if err := validateStorageClass(ctx, entry.StorageClass); err != nil {
return err
}
chartDir, err := os.MkdirTemp("", "dokku-storage-chart-")
if err != nil {
return fmt.Errorf("error creating chart directory: %w", err)
}
defer os.RemoveAll(chartDir)
releaseName := GetStorageReleaseName(entry.Name)
namespace := entry.Namespace
if namespace == "" {
namespace = "default"
}
if err := os.MkdirAll(filepath.Join(chartDir, "templates"), 0755); err != nil {
return fmt.Errorf("error creating chart templates directory: %w", err)
}
chart := &Chart{
ApiVersion: "v2",
AppVersion: "1.0.0",
Name: releaseName,
Icon: "https://dokku.com/assets/dokku-logo.svg",
Version: "0.0.1",
}
if err := writeYaml(WriteYamlInput{Object: chart, Path: filepath.Join(chartDir, "Chart.yaml")}); err != nil {
return fmt.Errorf("error writing chart: %w", err)
}
for _, name := range []string{"persistent-volume-claim.yaml", "persistent-volume.yaml"} {
b, err := templates.ReadFile(filepath.Join("templates", "storage-chart", "templates", name))
if err != nil {
return fmt.Errorf("error reading %s: %w", name, err)
}
if err := os.WriteFile(filepath.Join(chartDir, "templates", name), b, 0644); err != nil {
return fmt.Errorf("error writing %s: %w", name, err)
}
}
values := &StorageChartValues{
Global: StorageChartGlobal{
EntryName: entry.Name,
Namespace: namespace,
Size: entry.Size,
AccessMode: entry.AccessMode,
StorageClass: entry.StorageClass,
HostPath: entry.HostPath,
ReclaimPolicy: entry.ReclaimPolicy,
Annotations: entry.Annotations,
Labels: entry.Labels,
},
}
if err := writeYaml(WriteYamlInput{Object: values, Path: filepath.Join(chartDir, "values.yaml")}); err != nil {
return fmt.Errorf("error writing values: %w", err)
}
if err := createKubernetesNamespace(ctx, namespace); err != nil {
return fmt.Errorf("error creating namespace: %w", err)
}
helmAgent, err := NewHelmAgent(namespace, DeployLogPrinter)
if err != nil {
return fmt.Errorf("error creating helm agent: %w", err)
}
chartPath, err := filepath.Abs(chartDir)
if err != nil {
return fmt.Errorf("error getting chart path: %w", err)
}
common.LogVerboseQuiet(fmt.Sprintf("Installing storage chart for %s", entry.Name))
if err := helmAgent.InstallOrUpgradeChart(ctx, ChartInput{
ChartPath: chartPath,
Namespace: namespace,
ReleaseName: releaseName,
Wait: false,
}); err != nil {
return fmt.Errorf("error installing storage chart: %w", err)
}
return nil
}
// TriggerStorageDestroy removes the helm release backing a storage entry.
func TriggerStorageDestroy(ctx context.Context, entryName string) error {
if err := isKubernetesAvailable(); err != nil {
return fmt.Errorf("kubernetes not available: %w", err)
}
entry, err := readEntryFromStdin()
if err != nil {
return err
}
if entry.Scheduler != storage.SchedulerK3s {
return nil
}
namespace := entry.Namespace
if namespace == "" {
namespace = "default"
}
releaseName := GetStorageReleaseName(entryName)
helmAgent, err := NewHelmAgent(namespace, DeployLogPrinter)
if err != nil {
return fmt.Errorf("error creating helm agent: %w", err)
}
exists, err := helmAgent.ChartExists(releaseName)
if err != nil {
return fmt.Errorf("error checking if storage chart exists: %w", err)
}
if !exists {
return nil
}
common.LogVerboseQuiet(fmt.Sprintf("Removing storage chart for %s", entryName))
if err := helmAgent.UninstallChart(releaseName); err != nil {
return fmt.Errorf("error uninstalling storage chart: %w", err)
}
return nil
}
// TriggerStorageStatus prints "Bound" if the entry's PVC is bound,
// "Pending" or "Lost" otherwise. Used by the storage:wait command.
func TriggerStorageStatus(ctx context.Context, entryName string) error {
if err := isKubernetesAvailable(); err != nil {
return fmt.Errorf("kubernetes not available: %w", err)
}
entry, err := readEntryFromStdin()
if err != nil {
return err
}
namespace := entry.Namespace
if namespace == "" {
namespace = "default"
}
clientset, err := NewKubernetesClient()
if err != nil {
return err
}
pvc, err := clientset.Client.CoreV1().PersistentVolumeClaims(namespace).Get(ctx, entryName, metav1.GetOptions{})
if err != nil {
return fmt.Errorf("error fetching PVC %s/%s: %w", namespace, entryName, err)
}
fmt.Println(string(pvc.Status.Phase))
return nil
}
// validateStorageClass surfaces a clear error when the requested storage
// class doesn't exist in the cluster.
func validateStorageClass(ctx context.Context, name string) error {
if name == "" {
return nil
}
clientset, err := NewKubernetesClient()
if err != nil {
return err
}
classes, err := clientset.Client.StorageV1().StorageClasses().List(ctx, metav1.ListOptions{})
if err != nil {
return fmt.Errorf("error listing storage classes: %w", err)
}
available := []string{}
for _, cls := range classes.Items {
if cls.Name == name {
return nil
}
available = append(available, cls.Name)
}
return fmt.Errorf("storage class %q does not exist; available: %s", name, strings.Join(available, ", "))
}
// AppMountPair mirrors the storage plugin's wire format so we can decode
// the storage-app-mounts trigger output here.
type AppMountPair struct {
Entry *storage.Entry `json:"entry"`
Attachment *storage.Attachment `json:"attachment"`
}
// LoadAppMounts asks the storage plugin for an app's mount pairs in the
// given phase. Returns an empty slice when there are none.
func LoadAppMounts(appName string, phase string) ([]AppMountPair, error) {
results, err := common.CallPlugnTrigger(common.PlugnTriggerInput{
Trigger: "storage-app-mounts",
Args: []string{appName, phase},
})
if err != nil {
return nil, err
}
output := strings.TrimSpace(results.StdoutContents())
if output == "" {
return nil, nil
}
pairs := []AppMountPair{}
if err := json.Unmarshal([]byte(output), &pairs); err != nil {
return nil, fmt.Errorf("unable to parse storage-app-mounts output: %w", err)
}
return pairs, nil
}
// ToProcessVolumes converts each AppMountPair into a ProcessVolume. K3s
// app deployments reference the PVC by name; the PVC itself is owned by
// the storage entry's separate helm release. Any docker-local entries
// found here are an error - they cannot be mounted on a k3s app.
func ToProcessVolumes(pairs []AppMountPair) ([]ProcessVolume, error) {
volumes := []ProcessVolume{}
for _, pair := range pairs {
if pair.Entry == nil || pair.Attachment == nil {
continue
}
if pair.Entry.Scheduler == storage.SchedulerDockerLocal {
return nil, fmt.Errorf("storage entry %q is scheduler=docker-local but is mounted on a k3s app; recreate it with --scheduler k3s", pair.Entry.Name)
}
volumes = append(volumes, ProcessVolume{
Name: pair.Entry.Name,
MountPath: pair.Attachment.ContainerPath,
SubPath: pair.Attachment.Subpath,
ReadOnly: pair.Attachment.Readonly,
PersistentClaim: &ProcessVolumePersistentClaim{
ClaimName: pair.Entry.Name,
},
})
}
return volumes, nil
}
// asK8sVolumeMount is a small helper used by tests / other callers that
// want a corev1.VolumeMount from a ProcessVolume.
func asK8sVolumeMount(v ProcessVolume) corev1.VolumeMount {
return corev1.VolumeMount{
Name: v.Name,
MountPath: v.MountPath,
SubPath: v.SubPath,
ReadOnly: v.ReadOnly,
}
}

View File

@@ -0,0 +1,236 @@
package scheduler_k3s
import (
"context"
"errors"
"fmt"
"os"
"strconv"
"time"
"github.com/dokku/dokku/plugins/common"
"github.com/dokku/dokku/plugins/storage"
corev1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
utilexec "k8s.io/client-go/util/exec"
)
// StorageExecInput captures the inputs forwarded by the storage plugin
// over the scheduler-storage-exec trigger.
type StorageExecInput struct {
EntryName string
Image string
Interactive bool
Tty bool
AsUser string
Command []string
}
// TriggerSchedulerStorageExec runs an interactive or non-interactive
// command against a k3s storage entry by creating a throwaway Pod that
// mounts the PVC at /data, exec-ing the user's command via the
// kubernetes API, and deleting the Pod afterwards. Errors at every
// stage (PVC missing, image pull failure, etc.) come through the
// structured Kubernetes API objects rather than kubectl-shaped strings.
func TriggerSchedulerStorageExec(ctx context.Context, scheduler string, input StorageExecInput) error {
if scheduler != storage.SchedulerK3s {
return nil
}
if !storage.EntryExists(input.EntryName) {
return fmt.Errorf("storage entry %q does not exist", input.EntryName)
}
entry, err := storage.LoadEntry(input.EntryName)
if err != nil {
return err
}
if entry.Scheduler != storage.SchedulerK3s {
return fmt.Errorf("storage entry %q has scheduler %q, not k3s", entry.Name, entry.Scheduler)
}
if err := isKubernetesAvailable(); err != nil {
return fmt.Errorf("kubernetes not available: %w", err)
}
clientset, err := NewKubernetesClient()
if err != nil {
return err
}
namespace := entry.Namespace
if namespace == "" {
namespace = "default"
}
if _, err := clientset.Client.CoreV1().PersistentVolumeClaims(namespace).Get(ctx, entry.Name, metav1.GetOptions{}); err != nil {
if apierrors.IsNotFound(err) {
return fmt.Errorf("PVC %s not found in namespace %s", entry.Name, namespace)
}
return fmt.Errorf("error reading PVC %s in namespace %s: %w", entry.Name, namespace, err)
}
podName := fmt.Sprintf("dokku-storage-exec-%s-%d", entry.Name, time.Now().UnixNano()/int64(time.Millisecond)%1000000)
pod, err := buildStoragePodSpec(podName, namespace, entry, input)
if err != nil {
return err
}
common.LogVerboseQuiet(fmt.Sprintf("Creating exec pod %s/%s", namespace, podName))
if _, err := clientset.Client.CoreV1().Pods(namespace).Create(ctx, pod, metav1.CreateOptions{}); err != nil {
return fmt.Errorf("error creating exec pod %s/%s: %w", namespace, podName, err)
}
defer func() {
bg := context.Background()
gracePeriod := int64(0)
_ = clientset.Client.CoreV1().Pods(namespace).Delete(bg, podName, metav1.DeleteOptions{GracePeriodSeconds: &gracePeriod})
}()
if err := waitForPodRunning(ctx, clientset, namespace, podName, 60*time.Second); err != nil {
return err
}
command := input.Command
if len(command) == 0 {
command = []string{"sh", "-c", "command -v bash >/dev/null 2>&1 && exec bash || exec sh"}
}
execErr := clientset.ExecCommand(ctx, ExecCommandInput{
Name: podName,
Namespace: namespace,
ContainerName: "exec",
Command: command,
})
if execErr == nil {
return nil
}
var coded utilexec.CodeExitError
if errors.As(execErr, &coded) {
os.Exit(coded.Code)
}
return execErr
}
// buildStoragePodSpec assembles the throwaway Pod that mounts the PVC
// for storage:exec. Split out so the unit test can verify the spec
// without hitting a cluster.
func buildStoragePodSpec(name string, namespace string, entry *storage.Entry, input StorageExecInput) (*corev1.Pod, error) {
uid, err := resolveStorageExecUID(entry.Chown, input.AsUser)
if err != nil {
return nil, err
}
pod := &corev1.Pod{
ObjectMeta: metav1.ObjectMeta{
Name: name,
Namespace: namespace,
Labels: map[string]string{
"app.kubernetes.io/managed-by": "dokku",
"dokku.com/storage-entry": entry.Name,
"dokku.com/purpose": "storage-exec",
},
},
Spec: corev1.PodSpec{
RestartPolicy: corev1.RestartPolicyNever,
Volumes: []corev1.Volume{
{
Name: "data",
VolumeSource: corev1.VolumeSource{
PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{
ClaimName: entry.Name,
},
},
},
},
Containers: []corev1.Container{
{
Name: "exec",
Image: input.Image,
Command: []string{"sleep", "infinity"},
Stdin: input.Interactive,
TTY: input.Tty,
VolumeMounts: []corev1.VolumeMount{
{Name: "data", MountPath: "/data"},
},
},
},
},
}
if uid != nil {
pod.Spec.SecurityContext = &corev1.PodSecurityContext{
RunAsUser: uid,
RunAsGroup: uid,
FSGroup: uid,
}
}
return pod, nil
}
// resolveStorageExecUID picks the UID to run the exec Pod as. Returns
// nil to mean "leave the image's default user in place".
func resolveStorageExecUID(chown string, asUser string) (*int64, error) {
if asUser != "" {
n, err := strconv.ParseInt(asUser, 10, 64)
if err != nil {
return nil, fmt.Errorf("--as-user must be a numeric uid, got %q", asUser)
}
return &n, nil
}
if chown == "" || chown == "false" {
return nil, nil
}
uidStr, err := storage.ResolveChownID(chown)
if err != nil {
return nil, err
}
if uidStr == "" || uidStr == "false" {
return nil, nil
}
n, err := strconv.ParseInt(uidStr, 10, 64)
if err != nil {
return nil, fmt.Errorf("could not parse resolved chown UID %q: %w", uidStr, err)
}
return &n, nil
}
// waitForPodRunning polls the Pod until its phase is Running, surfacing
// the structured Kubernetes API state on failure (image pull failures,
// scheduling problems, etc.) rather than a generic timeout.
func waitForPodRunning(ctx context.Context, clientset KubernetesClient, namespace, name string, timeout time.Duration) error {
deadline := time.Now().Add(timeout)
for {
pod, err := clientset.Client.CoreV1().Pods(namespace).Get(ctx, name, metav1.GetOptions{})
if err != nil {
return fmt.Errorf("error reading exec pod %s/%s: %w", namespace, name, err)
}
switch pod.Status.Phase {
case corev1.PodRunning:
return nil
case corev1.PodFailed, corev1.PodSucceeded:
return fmt.Errorf("exec pod %s/%s reached terminal phase %s before exec could attach", namespace, name, pod.Status.Phase)
}
for _, cs := range pod.Status.ContainerStatuses {
if cs.State.Waiting == nil {
continue
}
reason := cs.State.Waiting.Reason
switch reason {
case "ImagePullBackOff", "ErrImagePull", "CreateContainerConfigError", "InvalidImageName":
return fmt.Errorf("exec pod %s/%s container %q failed to start: %s: %s", namespace, name, cs.Name, reason, cs.State.Waiting.Message)
}
}
if time.Now().After(deadline) {
return fmt.Errorf("exec pod %s/%s did not reach Running within %s (current phase: %s)", namespace, name, timeout, pod.Status.Phase)
}
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(500 * time.Millisecond):
}
}
}

View File

@@ -0,0 +1,103 @@
package scheduler_k3s
import (
"testing"
"github.com/dokku/dokku/plugins/storage"
)
func TestBuildStoragePodSpec(t *testing.T) {
entry := &storage.Entry{
Name: "demo-data",
Scheduler: storage.SchedulerK3s,
Namespace: "dokku",
}
pod, err := buildStoragePodSpec("dokku-storage-exec-demo-data-1", "dokku", entry, StorageExecInput{
Image: "alpine:3",
Interactive: true,
Tty: true,
Command: []string{"sh"},
})
if err != nil {
t.Fatalf("buildStoragePodSpec err: %v", err)
}
if pod.Spec.RestartPolicy != "Never" {
t.Fatalf("expected RestartPolicy=Never, got %s", pod.Spec.RestartPolicy)
}
if len(pod.Spec.Volumes) != 1 || pod.Spec.Volumes[0].PersistentVolumeClaim == nil {
t.Fatalf("expected one PVC volume, got %+v", pod.Spec.Volumes)
}
if pod.Spec.Volumes[0].PersistentVolumeClaim.ClaimName != "demo-data" {
t.Fatalf("expected PVC claim name demo-data, got %s", pod.Spec.Volumes[0].PersistentVolumeClaim.ClaimName)
}
if len(pod.Spec.Containers) != 1 {
t.Fatalf("expected one container, got %d", len(pod.Spec.Containers))
}
c := pod.Spec.Containers[0]
if c.Image != "alpine:3" {
t.Fatalf("expected image alpine:3, got %s", c.Image)
}
if !c.Stdin || !c.TTY {
t.Fatalf("expected stdin and tty true on the container, got stdin=%v tty=%v", c.Stdin, c.TTY)
}
if len(c.VolumeMounts) != 1 || c.VolumeMounts[0].MountPath != "/data" {
t.Fatalf("expected one volume mount at /data, got %+v", c.VolumeMounts)
}
}
func TestBuildStoragePodSpecChownToSecurityContext(t *testing.T) {
entry := &storage.Entry{
Name: "demo-data",
Scheduler: storage.SchedulerK3s,
Chown: "herokuish",
}
pod, err := buildStoragePodSpec("p", "default", entry, StorageExecInput{Image: "alpine:3"})
if err != nil {
t.Fatalf("err: %v", err)
}
if pod.Spec.SecurityContext == nil || pod.Spec.SecurityContext.RunAsUser == nil {
t.Fatalf("expected SecurityContext.RunAsUser to be populated for chown=herokuish")
}
// Don't assert exact UID because ResolveChownID consults docker for
// userns offset; only assert non-nil and non-zero.
if *pod.Spec.SecurityContext.RunAsUser == 0 {
t.Fatalf("expected non-zero UID for chown=herokuish, got 0")
}
}
func TestBuildStoragePodSpecAsUserOverride(t *testing.T) {
entry := &storage.Entry{
Name: "demo-data",
Scheduler: storage.SchedulerK3s,
Chown: "herokuish",
}
pod, err := buildStoragePodSpec("p", "default", entry, StorageExecInput{
Image: "alpine:3",
AsUser: "0",
})
if err != nil {
t.Fatalf("err: %v", err)
}
if pod.Spec.SecurityContext == nil || pod.Spec.SecurityContext.RunAsUser == nil {
t.Fatalf("expected RunAsUser to be set")
}
if *pod.Spec.SecurityContext.RunAsUser != 0 {
t.Fatalf("expected RunAsUser to be 0 (override), got %d", *pod.Spec.SecurityContext.RunAsUser)
}
}
func TestBuildStoragePodSpecChownFalseLeavesDefaultUser(t *testing.T) {
entry := &storage.Entry{
Name: "demo-data",
Scheduler: storage.SchedulerK3s,
Chown: "false",
}
pod, err := buildStoragePodSpec("p", "default", entry, StorageExecInput{Image: "alpine:3"})
if err != nil {
t.Fatalf("err: %v", err)
}
if pod.Spec.SecurityContext != nil {
t.Fatalf("expected no SecurityContext when chown=false, got %+v", pod.Spec.SecurityContext)
}
}

View File

@@ -117,9 +117,12 @@ type ProcessValues struct {
}
type ProcessVolume struct {
Name string `yaml:"name"`
MountPath string `yaml:"mount_path"`
EmptyDir *ProcessVolumeEmptyDir `yaml:"empty_dir,omitempty"`
Name string `yaml:"name"`
MountPath string `yaml:"mount_path"`
SubPath string `yaml:"sub_path,omitempty"`
ReadOnly bool `yaml:"read_only,omitempty"`
EmptyDir *ProcessVolumeEmptyDir `yaml:"empty_dir,omitempty"`
PersistentClaim *ProcessVolumePersistentClaim `yaml:"persistent_claim,omitempty"`
}
type ProcessVolumeEmptyDir struct {
@@ -127,6 +130,10 @@ type ProcessVolumeEmptyDir struct {
SizeLimit string `yaml:"size_limit"`
}
type ProcessVolumePersistentClaim struct {
ClaimName string `yaml:"claim_name"`
}
type ProcessAnnotations struct {
CertificateAnnotations map[string]string `yaml:"certificate,omitempty"`
CronJobAnnotations map[string]string `yaml:"cronjob,omitempty"`

View File

@@ -127,12 +127,44 @@ spec:
{{- if $.Values.global.image.working_dir }}
workingDir: {{ $.Values.global.image.working_dir }}
{{- end }}
{{- if $config.volumes }}
volumeMounts:
{{- range $volume := $config.volumes }}
- name: {{ $volume.name }}
mountPath: {{ $volume.mount_path }}
{{- if $volume.sub_path }}
subPath: {{ $volume.sub_path }}
{{- end }}
{{- if $volume.read_only }}
readOnly: true
{{- end }}
{{- end }}
{{- end }}
{{- if $.Values.global.image.image_pull_secrets }}
imagePullSecrets:
- name: {{ $.Values.global.image.image_pull_secrets }}
{{- end }}
restartPolicy: Never
serviceAccountName: {{ $.Values.global.app_name }}
{{- if $config.volumes }}
volumes:
{{- range $volume := $config.volumes }}
- name: {{ $volume.name }}
{{- if $volume.empty_dir }}
emptyDir:
{{- if $volume.empty_dir.medium }}
medium: {{ $volume.empty_dir.medium }}
{{- end }}
{{- if $volume.empty_dir.size_limit }}
sizeLimit: {{ $volume.empty_dir.size_limit }}
{{- end }}
{{- end }}
{{- if $volume.persistent_claim }}
persistentVolumeClaim:
claimName: {{ $volume.persistent_claim.claim_name }}
{{- end }}
{{- end }}
{{- end }}
schedule: {{ $config.cron.schedule }}
startingDeadlineSeconds: 60
successfulJobsHistoryLimit: 10

View File

@@ -152,6 +152,12 @@ spec:
{{- range $volume := $config.volumes }}
- name: {{ $volume.name }}
mountPath: {{ $volume.mount_path }}
{{- if $volume.sub_path }}
subPath: {{ $volume.sub_path }}
{{- end }}
{{- if $volume.read_only }}
readOnly: true
{{- end }}
{{- end }}
{{- end }}
{{- if $.Values.global.image.image_pull_secrets }}
@@ -172,6 +178,10 @@ spec:
sizeLimit: {{ $volume.empty_dir.size_limit }}
{{- end }}
{{- end }}
{{- if $volume.persistent_claim }}
persistentVolumeClaim:
claimName: {{ $volume.persistent_claim.claim_name }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}

View File

@@ -0,0 +1,29 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Values.global.entry_name }}
namespace: {{ .Values.global.namespace }}
labels:
app.kubernetes.io/name: {{ .Values.global.entry_name }}
app.kubernetes.io/managed-by: dokku
{{- range $k, $v := .Values.global.labels }}
{{ $k }}: {{ $v | quote }}
{{- end }}
annotations:
dokku.com/managed: "true"
{{- range $k, $v := .Values.global.annotations }}
{{ $k }}: {{ $v | quote }}
{{- end }}
spec:
accessModes:
- {{ .Values.global.access_mode | default "ReadWriteOnce" }}
resources:
requests:
storage: {{ .Values.global.size }}
{{- if .Values.global.storage_class }}
storageClassName: {{ .Values.global.storage_class }}
{{- end }}
{{- if .Values.global.host_path }}
volumeName: {{ .Values.global.entry_name }}
storageClassName: ""
{{- end }}

View File

@@ -0,0 +1,29 @@
{{- if .Values.global.host_path }}
apiVersion: v1
kind: PersistentVolume
metadata:
name: {{ .Values.global.entry_name }}
labels:
app.kubernetes.io/name: {{ .Values.global.entry_name }}
app.kubernetes.io/managed-by: dokku
{{- range $k, $v := .Values.global.labels }}
{{ $k }}: {{ $v | quote }}
{{- end }}
annotations:
dokku.com/managed: "true"
{{- range $k, $v := .Values.global.annotations }}
{{ $k }}: {{ $v | quote }}
{{- end }}
spec:
capacity:
storage: {{ .Values.global.size }}
accessModes:
- {{ .Values.global.access_mode | default "ReadWriteOnce" }}
persistentVolumeReclaimPolicy: {{ .Values.global.reclaim_policy | default "Retain" }}
hostPath:
path: {{ .Values.global.host_path }}
storageClassName: ""
claimRef:
namespace: {{ .Values.global.namespace }}
name: {{ .Values.global.entry_name }}
{{- end }}

View File

@@ -605,10 +605,19 @@ func TriggerSchedulerDeploy(scheduler string, appName string, imageTag string) e
})
}
deployMountPairs, err := LoadAppMounts(appName, "deploy")
if err != nil {
return fmt.Errorf("error loading storage app mounts: %w", err)
}
deployVolumes, err := ToProcessVolumes(deployMountPairs)
if err != nil {
return err
}
processVolumes = append(processVolumes, deployVolumes...)
for processType, processCount := range processes {
// todo: implement deployment annotations
// todo: implement pod annotations
// todo: implement volumes
healthchecks, ok := appJSON.Healthchecks[processType]
if !ok {
@@ -778,7 +787,6 @@ func TriggerSchedulerDeploy(scheduler string, appName string, imageTag string) e
for _, cronTask := range cronTasks {
// todo: implement deployment annotations
// todo: implement pod annotations
// todo: implement volumes
suffix := ""
for _, cronJob := range cronJobs {
if cronJob.Labels["dokku.com/cron-id"] == cronTask.ID {

View File

@@ -1,6 +1,6 @@
GOARCH ?= amd64
SUBCOMMANDS = subcommands/default subcommands/ensure-directory subcommands/list subcommands/mount subcommands/report subcommands/unmount
TRIGGERS = triggers/install triggers/storage-list
SUBCOMMANDS = subcommands/default subcommands/create subcommands/destroy subcommands/ensure-directory subcommands/exec subcommands/info subcommands/list subcommands/list-entries subcommands/migrate subcommands/mount subcommands/report subcommands/set subcommands/unmount subcommands/wait
TRIGGERS = triggers/install triggers/storage-list triggers/storage-app-mounts triggers/docker-args-deploy triggers/docker-args-run triggers/post-delete triggers/post-app-clone-setup triggers/post-app-rename-setup
BUILD = commands subcommands triggers
PLUGIN_NAME = storage

View File

@@ -0,0 +1,221 @@
package storage
import (
"encoding/json"
"errors"
"fmt"
"sort"
"strings"
"github.com/dokku/dokku/plugins/common"
)
// AttachmentsProperty is the property-list key used to store attachments
// for an app. All process types live under one list and carry their own
// ProcessType field; we filter on read.
const AttachmentsProperty = "mounts"
// PluginName is the name we register with the property system.
const PluginName = "storage"
// PhaseDeploy is the standard deploy phase identifier.
const PhaseDeploy = "deploy"
// PhaseRun is the standard run phase identifier.
const PhaseRun = "run"
// DefaultProcessType is the wildcard process type that applies to every
// process. Mirrors docker-options' DefaultProcessType.
const DefaultProcessType = "_default_"
// Attachment is the source of truth for *how* an app uses a storage entry.
// One Attachment binds one entry into one container path on one app.
type Attachment struct {
EntryName string `json:"entry_name"`
ContainerPath string `json:"container_path"`
Phases []string `json:"phases"`
ProcessType string `json:"process_type,omitempty"`
Subpath string `json:"subpath,omitempty"`
Readonly bool `json:"readonly,omitempty"`
VolumeOptions string `json:"volume_options,omitempty"`
VolumeChown string `json:"volume_chown,omitempty"`
}
// Validate checks an Attachment's fields against structural rules.
func (a *Attachment) Validate() error {
if a == nil {
return errors.New("attachment is nil")
}
if a.EntryName == "" {
return errors.New("attachment is missing entry name")
}
if a.ContainerPath == "" {
return errors.New("attachment is missing container path")
}
if !strings.HasPrefix(a.ContainerPath, "/") {
return fmt.Errorf("attachment container path %q must be absolute", a.ContainerPath)
}
if len(a.Phases) == 0 {
return errors.New("attachment must specify at least one phase")
}
for _, phase := range a.Phases {
if phase != PhaseDeploy && phase != PhaseRun {
return fmt.Errorf("attachment phase %q is not supported (must be %q or %q)", phase, PhaseDeploy, PhaseRun)
}
}
return nil
}
// LoadAttachments returns every attachment registered against an app.
func LoadAttachments(appName string) ([]*Attachment, error) {
lines, err := common.PropertyListGet(PluginName, appName, AttachmentsProperty)
if err != nil {
return nil, fmt.Errorf("unable to read storage attachments for %q: %w", appName, err)
}
attachments := []*Attachment{}
for _, line := range lines {
line = strings.TrimSpace(line)
if line == "" {
continue
}
attachment := &Attachment{}
if err := json.Unmarshal([]byte(line), attachment); err != nil {
return nil, fmt.Errorf("unable to parse storage attachment for %q: %w", appName, err)
}
attachments = append(attachments, attachment)
}
return attachments, nil
}
// SaveAttachments overwrites the entire attachment list for an app.
func SaveAttachments(appName string, attachments []*Attachment) error {
lines := make([]string, 0, len(attachments))
for _, attachment := range attachments {
data, err := json.Marshal(attachment)
if err != nil {
return fmt.Errorf("unable to encode storage attachment for %q: %w", appName, err)
}
lines = append(lines, string(data))
}
return common.PropertyListWrite(PluginName, appName, AttachmentsProperty, lines)
}
// AddAttachment appends an attachment, rejecting duplicates of the same
// (entry_name, container_path, process_type) tuple.
func AddAttachment(appName string, attachment *Attachment) error {
if err := attachment.Validate(); err != nil {
return err
}
attachments, err := LoadAttachments(appName)
if err != nil {
return err
}
for _, existing := range attachments {
if existing.EntryName == attachment.EntryName &&
existing.ContainerPath == attachment.ContainerPath &&
existing.ProcessType == attachment.ProcessType {
return fmt.Errorf("storage entry %q is already mounted at %q for process type %q on app %q",
attachment.EntryName, attachment.ContainerPath, attachment.ProcessType, appName)
}
}
attachments = append(attachments, attachment)
return SaveAttachments(appName, attachments)
}
// RemoveAttachment removes the attachment matching the given entry and
// optional container path. If containerPath is empty and there is more
// than one match, returns an error.
func RemoveAttachment(appName string, entryName string, containerPath string) error {
attachments, err := LoadAttachments(appName)
if err != nil {
return err
}
matches := []*Attachment{}
keep := []*Attachment{}
for _, attachment := range attachments {
if attachment.EntryName == entryName && (containerPath == "" || attachment.ContainerPath == containerPath) {
matches = append(matches, attachment)
continue
}
keep = append(keep, attachment)
}
if len(matches) == 0 {
if containerPath == "" {
return fmt.Errorf("storage entry %q is not mounted on app %q", entryName, appName)
}
return fmt.Errorf("storage entry %q is not mounted at %q on app %q", entryName, containerPath, appName)
}
if len(matches) > 1 && containerPath == "" {
paths := []string{}
for _, attachment := range matches {
paths = append(paths, attachment.ContainerPath)
}
sort.Strings(paths)
return fmt.Errorf("storage entry %q is mounted at multiple paths on app %q (%s); pass --container-dir to disambiguate",
entryName, appName, strings.Join(paths, ", "))
}
return SaveAttachments(appName, keep)
}
// AttachmentsForPhase returns the subset of an app's attachments that
// apply to the given phase, sorted for stable output.
func AttachmentsForPhase(appName string, phase string) ([]*Attachment, error) {
attachments, err := LoadAttachments(appName)
if err != nil {
return nil, err
}
filtered := []*Attachment{}
for _, attachment := range attachments {
for _, p := range attachment.Phases {
if p == phase {
filtered = append(filtered, attachment)
break
}
}
}
sort.Slice(filtered, func(i, j int) bool {
if filtered[i].EntryName != filtered[j].EntryName {
return filtered[i].EntryName < filtered[j].EntryName
}
return filtered[i].ContainerPath < filtered[j].ContainerPath
})
return filtered, nil
}
// AppsUsingEntry returns the list of app names that have at least one
// attachment referencing the given entry name. Used by storage:destroy
// to refuse removing an entry that's still mounted.
func AppsUsingEntry(entryName string) ([]string, error) {
apps, err := common.DokkuApps()
if err != nil {
if errors.Is(err, common.NoAppsExist) {
return nil, nil
}
return nil, err
}
using := []string{}
for _, app := range apps {
attachments, err := LoadAttachments(app)
if err != nil {
return nil, err
}
for _, attachment := range attachments {
if attachment.EntryName == entryName {
using = append(using, app)
break
}
}
}
sort.Strings(using)
return using, nil
}

View File

@@ -0,0 +1,161 @@
package storage
import (
"encoding/json"
"os"
"path/filepath"
"testing"
. "github.com/onsi/gomega"
)
// writeAttachmentsFile drops the JSON-encoded attachment list into the
// per-app property file directly, bypassing common.PropertyListWrite (which
// calls SetPermissions and chowns to dokku:dokku, which a developer machine
// or CI runner generally doesn't have).
func writeAttachmentsFile(t *testing.T, root string, appName string, attachments []*Attachment) {
t.Helper()
dir := filepath.Join(root, "config", "storage", appName)
if err := os.MkdirAll(dir, 0755); err != nil {
t.Fatalf("mkdir attachment dir: %v", err)
}
path := filepath.Join(dir, AttachmentsProperty)
f, err := os.Create(path)
if err != nil {
t.Fatalf("create attachment file: %v", err)
}
defer f.Close()
for _, a := range attachments {
data, err := json.Marshal(a)
if err != nil {
t.Fatalf("marshal attachment: %v", err)
}
f.Write(append(data, '\n'))
}
}
func TestAttachmentValidate(t *testing.T) {
RegisterTestingT(t)
good := &Attachment{
EntryName: "demo-data",
ContainerPath: "/data",
Phases: []string{PhaseDeploy, PhaseRun},
}
Expect(good.Validate()).To(Succeed())
noEntry := &Attachment{ContainerPath: "/data", Phases: []string{PhaseDeploy}}
Expect(noEntry.Validate()).To(HaveOccurred())
noPath := &Attachment{EntryName: "demo-data", Phases: []string{PhaseDeploy}}
Expect(noPath.Validate()).To(HaveOccurred())
relative := &Attachment{EntryName: "demo-data", ContainerPath: "data", Phases: []string{PhaseDeploy}}
Expect(relative.Validate()).To(HaveOccurred())
noPhases := &Attachment{EntryName: "demo-data", ContainerPath: "/data"}
Expect(noPhases.Validate()).To(HaveOccurred())
badPhase := &Attachment{EntryName: "demo-data", ContainerPath: "/data", Phases: []string{"build"}}
Expect(badPhase.Validate()).To(HaveOccurred())
}
func TestListAppMountEntriesDockerLocal(t *testing.T) {
RegisterTestingT(t)
root := withTempLibRoot(t)
Expect(SaveEntry(&Entry{
Name: "demo-data",
Scheduler: SchedulerDockerLocal,
HostPath: "/var/lib/dokku/data/storage/demo-data",
})).To(Succeed())
writeAttachmentsFile(t, root, "demo", []*Attachment{
{
EntryName: "demo-data",
ContainerPath: "/data",
Phases: []string{PhaseDeploy, PhaseRun},
Readonly: true,
},
})
rows, err := ListAppMountEntries("demo", PhaseDeploy)
Expect(err).NotTo(HaveOccurred())
Expect(rows).To(HaveLen(1))
Expect(rows[0].EntryName).To(Equal("demo-data"))
Expect(rows[0].HostPath).To(Equal("/var/lib/dokku/data/storage/demo-data"))
Expect(rows[0].ContainerPath).To(Equal("/data"))
Expect(rows[0].VolumeOptions).To(Equal("ro"))
// Run phase shows it too because the attachment includes both phases.
runRows, err := ListAppMountEntries("demo", PhaseRun)
Expect(err).NotTo(HaveOccurred())
Expect(runRows).To(HaveLen(1))
// formatStorageListEntry produces the legacy colon form including options.
Expect(formatStorageListEntry(rows[0])).To(Equal("/var/lib/dokku/data/storage/demo-data:/data:ro"))
}
func TestListAppMountEntriesK3sUsesEntryName(t *testing.T) {
RegisterTestingT(t)
root := withTempLibRoot(t)
// k3s entry has no host path because the cluster provisions it.
Expect(SaveEntry(&Entry{
Name: "demo-pvc",
Scheduler: SchedulerK3s,
Size: "2Gi",
StorageClass: "longhorn",
AccessMode: "ReadWriteOnce",
})).To(Succeed())
writeAttachmentsFile(t, root, "demo", []*Attachment{
{
EntryName: "demo-pvc",
ContainerPath: "/data",
Phases: []string{PhaseDeploy},
Subpath: "uploads",
},
})
rows, err := ListAppMountEntries("demo", PhaseDeploy)
Expect(err).NotTo(HaveOccurred())
Expect(rows).To(HaveLen(1))
// HostPath falls back to the entry name so the colon form remains
// well-formed for callers that only know about the legacy shape.
Expect(rows[0].HostPath).To(Equal("demo-pvc"))
Expect(rows[0].EntryName).To(Equal("demo-pvc"))
Expect(rows[0].ContainerPath).To(Equal("/data"))
Expect(formatStorageListEntry(rows[0])).To(Equal("demo-pvc:/data"))
}
func TestListAppMountEntriesPhaseFilter(t *testing.T) {
RegisterTestingT(t)
root := withTempLibRoot(t)
Expect(SaveEntry(&Entry{
Name: "demo-deploy-only",
Scheduler: SchedulerDockerLocal,
HostPath: "/srv/deploy",
})).To(Succeed())
Expect(SaveEntry(&Entry{
Name: "demo-run-only",
Scheduler: SchedulerDockerLocal,
HostPath: "/srv/run",
})).To(Succeed())
writeAttachmentsFile(t, root, "demo", []*Attachment{
{EntryName: "demo-deploy-only", ContainerPath: "/d", Phases: []string{PhaseDeploy}},
{EntryName: "demo-run-only", ContainerPath: "/r", Phases: []string{PhaseRun}},
})
deployRows, err := ListAppMountEntries("demo", PhaseDeploy)
Expect(err).NotTo(HaveOccurred())
Expect(deployRows).To(HaveLen(1))
Expect(deployRows[0].EntryName).To(Equal("demo-deploy-only"))
runRows, err := ListAppMountEntries("demo", PhaseRun)
Expect(err).NotTo(HaveOccurred())
Expect(runRows).To(HaveLen(1))
Expect(runRows[0].EntryName).To(Equal("demo-run-only"))
}

View File

@@ -0,0 +1,567 @@
package storage
import (
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/dokku/dokku/plugins/common"
)
// CommandCreateInput captures the flags accepted by storage:create.
type CommandCreateInput struct {
Name string
Path string
Scheduler string
Size string
AccessMode string
StorageClass string
Namespace string
Chown string
ReclaimPolicy string
Annotations map[string]string
Labels map[string]string
}
// CommandCreate registers a new storage entry.
func CommandCreate(input CommandCreateInput) error {
if err := ValidateEntryName(input.Name, false); err != nil {
return err
}
scheduler := input.Scheduler
if scheduler == "" {
scheduler = SchedulerDockerLocal
}
hostPath := input.Path
if scheduler == SchedulerDockerLocal && hostPath == "" {
hostPath = filepath.Join(GetStorageDirectory(), input.Name)
}
entry := &Entry{
Name: input.Name,
Scheduler: scheduler,
HostPath: hostPath,
Size: input.Size,
AccessMode: input.AccessMode,
StorageClass: input.StorageClass,
Namespace: input.Namespace,
Chown: input.Chown,
ReclaimPolicy: input.ReclaimPolicy,
Annotations: input.Annotations,
Labels: input.Labels,
SchemaVersion: SchemaVersion,
}
if err := entry.Validate(); err != nil {
return err
}
if EntryExists(entry.Name) {
existing, err := LoadEntry(entry.Name)
if err != nil {
return err
}
if existing.Scheduler != entry.Scheduler {
return fmt.Errorf("storage entry %q already exists with scheduler %q; refusing to redefine", entry.Name, existing.Scheduler)
}
common.LogInfo1Quiet(fmt.Sprintf("Storage entry %s already exists, leaving in place", entry.Name))
}
if scheduler == SchedulerDockerLocal {
if err := ensureDockerLocalPath(entry); err != nil {
return err
}
}
if err := SaveEntry(entry); err != nil {
return err
}
if scheduler == SchedulerK3s {
if err := callSchedulerCreateTrigger(entry); err != nil {
// Roll back the on-disk entry so the disk and the cluster stay in sync.
_ = DeleteEntry(entry.Name)
return fmt.Errorf("scheduler refused storage entry %q: %w", entry.Name, err)
}
}
common.LogInfo1(fmt.Sprintf("Storage entry %s created", entry.Name))
return nil
}
// CommandDestroy removes a registered storage entry. Refuses to remove
// an entry that any app still has attached.
func CommandDestroy(name string) error {
if name == "" {
return errors.New("storage entry name is required")
}
if !EntryExists(name) {
return fmt.Errorf("storage entry %q does not exist", name)
}
using, err := AppsUsingEntry(name)
if err != nil {
return err
}
if len(using) > 0 {
return fmt.Errorf("storage entry %q is still mounted by app(s): %s", name, strings.Join(using, ", "))
}
entry, err := LoadEntry(name)
if err != nil {
return err
}
if entry.Scheduler == SchedulerK3s {
if err := callSchedulerDestroyTrigger(entry); err != nil {
return fmt.Errorf("scheduler refused to remove storage entry %q: %w", name, err)
}
}
if err := DeleteEntry(name); err != nil {
return err
}
common.LogInfo1(fmt.Sprintf("Storage entry %s destroyed", name))
return nil
}
// CommandInfo prints a single entry's details, optionally as JSON.
func CommandInfo(name string, format string) error {
if name == "" {
return errors.New("storage entry name is required")
}
if !EntryExists(name) {
return fmt.Errorf("storage entry %q does not exist", name)
}
entry, err := LoadEntry(name)
if err != nil {
return err
}
if format == "json" {
data, err := json.MarshalIndent(entry, "", " ")
if err != nil {
return err
}
fmt.Println(string(data))
return nil
}
common.LogInfo1Quiet(fmt.Sprintf("Storage entry %s", entry.Name))
common.LogVerbose(fmt.Sprintf("Scheduler: %s", entry.Scheduler))
if entry.HostPath != "" {
common.LogVerbose(fmt.Sprintf("Host path: %s", entry.HostPath))
}
if entry.Size != "" {
common.LogVerbose(fmt.Sprintf("Size: %s", entry.Size))
}
if entry.AccessMode != "" {
common.LogVerbose(fmt.Sprintf("Access mode: %s", entry.AccessMode))
}
if entry.StorageClass != "" {
common.LogVerbose(fmt.Sprintf("Storage class: %s", entry.StorageClass))
}
if entry.Namespace != "" {
common.LogVerbose(fmt.Sprintf("Namespace: %s", entry.Namespace))
}
if entry.Chown != "" {
common.LogVerbose(fmt.Sprintf("Chown: %s", entry.Chown))
}
if entry.ReclaimPolicy != "" {
common.LogVerbose(fmt.Sprintf("Reclaim policy: %s", entry.ReclaimPolicy))
}
return nil
}
// CommandSetInput captures the flags accepted by storage:set.
type CommandSetInput struct {
Name string
Size string
AccessMode string
StorageClass string
Namespace string
Chown string
ReclaimPolicy string
Annotations map[string]string
Labels map[string]string
}
// CommandSet edits an existing entry's mutable fields and re-fires the
// scheduler-side helm release. Refuses changes Kubernetes can't apply
// in place (access-mode swap, storage-class swap, size shrink).
func CommandSet(input CommandSetInput) error {
if !EntryExists(input.Name) {
return fmt.Errorf("storage entry %q does not exist", input.Name)
}
entry, err := LoadEntry(input.Name)
if err != nil {
return err
}
if input.AccessMode != "" && input.AccessMode != entry.AccessMode {
return fmt.Errorf("storage:set cannot change access-mode in place; recreate the entry")
}
if input.StorageClass != "" && input.StorageClass != entry.StorageClass {
return fmt.Errorf("storage:set cannot change storage-class-name in place; recreate the entry")
}
if input.Size != "" {
entry.Size = input.Size
}
if input.Namespace != "" {
entry.Namespace = input.Namespace
}
if input.Chown != "" {
entry.Chown = input.Chown
}
if input.ReclaimPolicy != "" {
entry.ReclaimPolicy = input.ReclaimPolicy
}
if input.Annotations != nil {
entry.Annotations = input.Annotations
}
if input.Labels != nil {
entry.Labels = input.Labels
}
if err := entry.Validate(); err != nil {
return err
}
if err := SaveEntry(entry); err != nil {
return err
}
if entry.Scheduler == SchedulerK3s {
if err := callSchedulerCreateTrigger(entry); err != nil {
return fmt.Errorf("scheduler refused storage:set for %q: %w", entry.Name, err)
}
}
common.LogInfo1(fmt.Sprintf("Storage entry %s updated", entry.Name))
return nil
}
// CommandExecInput captures the storage:exec subcommand inputs.
type CommandExecInput struct {
Name string
Image string
AsUser string
Args []string
}
// CommandExec delegates the actual exec to the scheduler plugin that owns
// the storage entry, by firing the scheduler-storage-exec plugn trigger
// with stdin/stdout/stderr streamed through. The handler decides between
// docker run, the k8s exec SDK, etc. - storage stays scheduler-agnostic.
//
// Exit codes from the underlying tool are propagated verbatim via
// os.Exit so callers in scripts see the right status.
func CommandExec(input CommandExecInput) error {
if !EntryExists(input.Name) {
return fmt.Errorf("storage entry %q does not exist", input.Name)
}
entry, err := LoadEntry(input.Name)
if err != nil {
return err
}
image := input.Image
if image == "" {
image = "alpine:3"
}
interactive, tty := stdinModes()
triggerArgs := []string{
entry.Scheduler,
entry.Name,
image,
}
triggerArgs = append(triggerArgs, fmt.Sprintf("--interactive=%t", interactive))
triggerArgs = append(triggerArgs, fmt.Sprintf("--tty=%t", tty))
if input.AsUser != "" {
triggerArgs = append(triggerArgs, "--as-user", input.AsUser)
}
if len(input.Args) > 0 {
triggerArgs = append(triggerArgs, "--")
triggerArgs = append(triggerArgs, input.Args...)
}
results, err := common.CallPlugnTrigger(common.PlugnTriggerInput{
Trigger: "scheduler-storage-exec",
Args: triggerArgs,
StreamStdio: true,
})
// common.CallExecCommand wraps a non-zero exit code as an error, but
// for storage:exec the underlying tool's exit code IS the signal we
// want to surface - it's how `dokku storage:exec demo -- exit 42`
// reaches the user as exit 42. Check ExitCode first so that branch
// runs before err collapses to LogFailWithError → exit 1.
if results.ExitCode != 0 {
os.Exit(results.ExitCode)
}
if err != nil {
return err
}
return nil
}
// stdinModes inspects os.Stdin to decide whether docker / the k8s exec
// SDK should request an interactive session and a TTY.
func stdinModes() (interactive bool, tty bool) {
fi, err := os.Stdin.Stat()
if err != nil || fi == nil {
return false, false
}
mode := fi.Mode()
tty = mode&os.ModeCharDevice != 0
hasStdinData := mode&os.ModeNamedPipe != 0
interactive = tty || hasStdinData
return interactive, tty
}
// CommandMigrate re-runs the legacy `-v` → entry/attachment migration
// for a single app or every app on the install. Useful when an operator
// has restored an app from a backup that predates the install-time
// migration, or when they've added `-v` lines via docker-options:add
// after the per-app flag was set.
func CommandMigrate(appName string, all bool) error {
if all && appName != "" {
return errors.New("storage:migrate accepts either an app name or --all, not both")
}
if !all && appName == "" {
return errors.New("storage:migrate requires an app name (or --all)")
}
if all {
apps, err := common.DokkuApps()
if err != nil {
if errors.Is(err, common.NoAppsExist) {
return nil
}
return err
}
for _, app := range apps {
if err := MigrateApp(app); err != nil {
return err
}
}
common.LogInfo1(fmt.Sprintf("Re-migrated %d app(s)", len(apps)))
return nil
}
if err := common.VerifyAppName(appName); err != nil {
return err
}
if err := MigrateApp(appName); err != nil {
return err
}
common.LogInfo1(fmt.Sprintf("Re-migrated %s", appName))
return nil
}
// CommandWait blocks until a k3s storage entry's PVC is bound.
func CommandWait(name string) error {
if !EntryExists(name) {
return fmt.Errorf("storage entry %q does not exist", name)
}
entry, err := LoadEntry(name)
if err != nil {
return err
}
if entry.Scheduler != SchedulerK3s {
// Docker-local entries are immediately "ready" once the directory exists.
if entry.HostPath != "" {
info, statErr := os.Stat(entry.HostPath)
if statErr != nil || !info.IsDir() {
return fmt.Errorf("storage entry %q host path %s is not present", name, entry.HostPath)
}
}
return nil
}
data, err := json.Marshal(entry)
if err != nil {
return err
}
results, err := common.CallPlugnTrigger(common.PlugnTriggerInput{
Trigger: "storage-status",
Args: []string{entry.Name},
Stdin: strings.NewReader(string(data)),
})
if err != nil {
return err
}
status := strings.TrimSpace(results.StdoutContents())
if status != "Bound" {
return fmt.Errorf("storage entry %q is %s, not Bound", name, status)
}
common.LogInfo1(fmt.Sprintf("Storage entry %s is bound", name))
return nil
}
// CommandReportGlobal prints a global storage report listing every entry
// and the apps that mount it. Output is text by default; pass
// format="json" for machine-readable output.
func CommandReportGlobal(format string) error {
entries, err := ListEntries()
if err != nil {
return err
}
type entryWithUse struct {
Entry *Entry `json:"entry"`
MountedBy []string `json:"mounted_by"`
}
rows := []entryWithUse{}
for _, entry := range entries {
using, err := AppsUsingEntry(entry.Name)
if err != nil {
return err
}
rows = append(rows, entryWithUse{Entry: entry, MountedBy: using})
}
if format == "json" {
data, err := json.MarshalIndent(rows, "", " ")
if err != nil {
return err
}
fmt.Println(string(data))
return nil
}
if len(rows) == 0 {
common.LogInfo1Quiet("No storage entries registered")
return nil
}
common.LogInfo1Quiet("Storage report (global):")
for _, row := range rows {
mountedBy := "—"
if len(row.MountedBy) > 0 {
mountedBy = strings.Join(row.MountedBy, ", ")
}
common.LogVerbose(fmt.Sprintf("%s\t%s\tmounted by: %s", row.Entry.Name, row.Entry.Scheduler, mountedBy))
}
return nil
}
// CommandListEntries lists registered entries (distinct from the legacy
// per-app storage:list which lists colon-form mounts).
func CommandListEntries(scheduler string, format string) error {
entries, err := ListEntries()
if err != nil {
return err
}
filtered := []*Entry{}
for _, entry := range entries {
if scheduler != "" && entry.Scheduler != scheduler {
continue
}
filtered = append(filtered, entry)
}
if format == "json" {
data, err := json.MarshalIndent(filtered, "", " ")
if err != nil {
return err
}
fmt.Println(string(data))
return nil
}
if len(filtered) == 0 {
common.LogInfo1Quiet("No storage entries registered")
return nil
}
common.LogInfo1Quiet("Storage entries:")
for _, entry := range filtered {
common.LogVerbose(fmt.Sprintf("%s\t%s\t%s", entry.Name, entry.Scheduler, entry.HostPath))
}
return nil
}
// ensureDockerLocalPath creates the host directory referenced by a
// docker-local entry if it doesn't already exist. Idempotent: a
// pre-existing directory is left in place.
func ensureDockerLocalPath(entry *Entry) error {
info, err := os.Stat(entry.HostPath)
if err != nil && !os.IsNotExist(err) {
return fmt.Errorf("unable to stat %s: %w", entry.HostPath, err)
}
if err == nil && !info.IsDir() {
return fmt.Errorf("storage entry path %s exists but is not a directory", entry.HostPath)
}
if err != nil {
if mkdirErr := os.MkdirAll(entry.HostPath, 0755); mkdirErr != nil {
return fmt.Errorf("unable to create %s: %w", entry.HostPath, mkdirErr)
}
common.LogVerbose(fmt.Sprintf("Created %s", entry.HostPath))
}
if entry.Chown != "" && entry.Chown != "false" {
chownID, err := ResolveChownID(entry.Chown)
if err != nil {
return err
}
if chownID != "false" {
pluginPath := common.MustGetEnv("PLUGIN_AVAILABLE_PATH")
chownScript := filepath.Join(pluginPath, "storage", "bin", "chown-storage-dir")
result, err := common.CallExecCommand(common.ExecCommandInput{
Command: "sudo",
Args: []string{chownScript, entry.HostPath, chownID},
})
if err != nil {
return fmt.Errorf("unable to chown %s: %w", entry.HostPath, err)
}
if result.ExitCode != 0 {
return fmt.Errorf("unable to chown %s: %s", entry.HostPath, result.StderrContents())
}
}
}
return nil
}
// callSchedulerCreateTrigger asks the scheduler plugin (k3s) to provision
// the underlying PVC/PV. The scheduler is responsible for any
// cluster-level validation (storage class existence, etc.).
func callSchedulerCreateTrigger(entry *Entry) error {
data, err := json.Marshal(entry)
if err != nil {
return err
}
results, err := common.CallPlugnTrigger(common.PlugnTriggerInput{
Trigger: "storage-create",
Args: []string{entry.Name},
StreamStdio: true,
Stdin: strings.NewReader(string(data)),
})
if err != nil {
return err
}
if results.ExitCode != 0 {
return fmt.Errorf("storage-create trigger exited with %d: %s", results.ExitCode, results.StderrContents())
}
return nil
}
// callSchedulerDestroyTrigger asks the scheduler plugin to release the
// underlying PVC/PV.
func callSchedulerDestroyTrigger(entry *Entry) error {
data, err := json.Marshal(entry)
if err != nil {
return err
}
results, err := common.CallPlugnTrigger(common.PlugnTriggerInput{
Trigger: "storage-destroy",
Args: []string{entry.Name},
StreamStdio: true,
Stdin: strings.NewReader(string(data)),
})
if err != nil {
return err
}
if results.ExitCode != 0 {
return fmt.Errorf("storage-destroy trigger exited with %d: %s", results.ExitCode, results.StderrContents())
}
return nil
}

294
plugins/storage/entry.go Normal file
View File

@@ -0,0 +1,294 @@
package storage
import (
"crypto/sha1"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
"github.com/dokku/dokku/plugins/common"
)
// SchemaVersion is the on-disk schema version emitted by this plugin.
// Bumped when the format of Entry / Attachment changes incompatibly.
const SchemaVersion = 1
// SchedulerDockerLocal is the default scheduler value for storage entries.
const SchedulerDockerLocal = "docker-local"
// SchedulerK3s is the k3s scheduler value for storage entries.
const SchedulerK3s = "k3s"
// LegacyEntryPrefix is reserved for migration-synthesized entries; users
// cannot create entries whose name starts with this prefix.
const LegacyEntryPrefix = "legacy-"
// MaxEntryNameLength caps the storage entry name so that the helm release
// name "storage-<name>" stays within Helm's practical 53-char ceiling.
const MaxEntryNameLength = 45
// ReclaimPolicyRetain keeps the underlying PV after the entry is destroyed.
const ReclaimPolicyRetain = "Retain"
// ReclaimPolicyDelete deletes the underlying PV after the entry is destroyed.
const ReclaimPolicyDelete = "Delete"
var (
// dns1123LabelRegexp matches a Kubernetes DNS-1123 label, the format
// required for resource names like PVCs and Helm releases.
dns1123LabelRegexp = regexp.MustCompile(`^[a-z0-9]([-a-z0-9]*[a-z0-9])?$`)
// dockerNamedVolumeRegexp matches a Docker named-volume token. Same
// rule docker uses to disambiguate volume names from bind paths.
dockerNamedVolumeRegexp = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9_.-]+$`)
// supportedSchedulers lists the values accepted for --scheduler.
supportedSchedulers = map[string]bool{
SchedulerDockerLocal: true,
SchedulerK3s: true,
}
// supportedAccessModes lists the values accepted for --access-mode on k3s.
supportedAccessModes = map[string]bool{
"ReadWriteOnce": true,
"ReadOnlyMany": true,
"ReadWriteMany": true,
"ReadWriteOncePod": true,
}
)
// Entry is the source of truth for a storage volume. One file per entry
// lives at $DOKKU_LIB_ROOT/config/storage/entries/<name>.json.
type Entry struct {
Name string `json:"name"`
Scheduler string `json:"scheduler"`
HostPath string `json:"host_path,omitempty"`
Size string `json:"size,omitempty"`
AccessMode string `json:"access_mode,omitempty"`
StorageClass string `json:"storage_class,omitempty"`
Namespace string `json:"namespace,omitempty"`
Chown string `json:"chown,omitempty"`
ReclaimPolicy string `json:"reclaim_policy,omitempty"`
Annotations map[string]string `json:"annotations,omitempty"`
Labels map[string]string `json:"labels,omitempty"`
SchemaVersion int `json:"schema_version"`
}
// RegistryDirectory returns the parent directory for storage-plugin
// state that lives outside the property store. Kept under data/ rather
// than config/ so it doesn't collide with the property-list paths
// `config/storage/<appName>/<property>` that the rest of the plugin
// relies on for per-app attachments.
func RegistryDirectory() string {
root := common.GetenvWithDefault("DOKKU_LIB_ROOT", "/var/lib/dokku")
return filepath.Join(root, "data", "storage-registry")
}
// EntriesDirectory returns the directory holding per-entry JSON files.
func EntriesDirectory() string {
return filepath.Join(RegistryDirectory(), "entries")
}
// entryPath returns the on-disk path for a named entry.
func entryPath(name string) string {
return filepath.Join(EntriesDirectory(), name+".json")
}
// EnsureEntriesDirectory makes sure the entries directory exists with
// permissive enough mode that the dokku user can read it.
func EnsureEntriesDirectory() error {
dir := EntriesDirectory()
if err := os.MkdirAll(dir, 0755); err != nil {
return fmt.Errorf("unable to create storage entries directory: %w", err)
}
return nil
}
// EntryExists reports whether an entry with the given name is registered.
func EntryExists(name string) bool {
_, err := os.Stat(entryPath(name))
return err == nil
}
// LoadEntry reads a registered entry by name.
func LoadEntry(name string) (*Entry, error) {
data, err := os.ReadFile(entryPath(name))
if err != nil {
if os.IsNotExist(err) {
return nil, fmt.Errorf("storage entry %q does not exist", name)
}
return nil, fmt.Errorf("unable to read storage entry %q: %w", name, err)
}
entry := &Entry{}
if err := json.Unmarshal(data, entry); err != nil {
return nil, fmt.Errorf("unable to parse storage entry %q: %w", name, err)
}
return entry, nil
}
// SaveEntry persists an entry to its JSON file.
func SaveEntry(entry *Entry) error {
if err := EnsureEntriesDirectory(); err != nil {
return err
}
if entry.SchemaVersion == 0 {
entry.SchemaVersion = SchemaVersion
}
data, err := json.MarshalIndent(entry, "", " ")
if err != nil {
return fmt.Errorf("unable to encode storage entry %q: %w", entry.Name, err)
}
path := entryPath(entry.Name)
if err := os.WriteFile(path, data, 0600); err != nil {
return fmt.Errorf("unable to write storage entry %q: %w", entry.Name, err)
}
return nil
}
// DeleteEntry removes the on-disk record for an entry.
func DeleteEntry(name string) error {
if err := os.Remove(entryPath(name)); err != nil && !os.IsNotExist(err) {
return fmt.Errorf("unable to remove storage entry %q: %w", name, err)
}
return nil
}
// ListEntries returns every registered entry sorted by name.
func ListEntries() ([]*Entry, error) {
dir := EntriesDirectory()
dirEntries, err := os.ReadDir(dir)
if err != nil {
if os.IsNotExist(err) {
return nil, nil
}
return nil, fmt.Errorf("unable to list storage entries: %w", err)
}
entries := []*Entry{}
for _, dirEntry := range dirEntries {
if dirEntry.IsDir() || !strings.HasSuffix(dirEntry.Name(), ".json") {
continue
}
name := strings.TrimSuffix(dirEntry.Name(), ".json")
entry, err := LoadEntry(name)
if err != nil {
return nil, err
}
entries = append(entries, entry)
}
sort.Slice(entries, func(i, j int) bool { return entries[i].Name < entries[j].Name })
return entries, nil
}
// ValidateEntryName checks the user-supplied name against length and DNS
// label rules. Returns nil for legacy names if allowLegacyPrefix is true,
// which is only set during migration.
func ValidateEntryName(name string, allowLegacyPrefix bool) error {
if name == "" {
return errors.New("storage entry name is required")
}
if len(name) > MaxEntryNameLength {
return fmt.Errorf("storage entry name %q is too long (max %d characters); helm release names cap our budget", name, MaxEntryNameLength)
}
if !dns1123LabelRegexp.MatchString(name) {
return fmt.Errorf("storage entry name %q must be a DNS-1123 label (lowercase letters, digits, and dashes; cannot start or end with a dash)", name)
}
if !allowLegacyPrefix && strings.HasPrefix(name, LegacyEntryPrefix) {
return fmt.Errorf("storage entry name %q must not start with reserved prefix %q", name, LegacyEntryPrefix)
}
return nil
}
// Validate checks an Entry's fields against the cross-field rules for its
// scheduler. It is reused by both storage:create and the legacy migration.
func (e *Entry) Validate() error {
if e == nil {
return errors.New("entry is nil")
}
if err := ValidateEntryName(e.Name, strings.HasPrefix(e.Name, LegacyEntryPrefix)); err != nil {
return err
}
if !supportedSchedulers[e.Scheduler] {
return fmt.Errorf("storage entry %q has unsupported scheduler %q (supported: docker-local, k3s)", e.Name, e.Scheduler)
}
switch e.Scheduler {
case SchedulerDockerLocal:
if e.HostPath == "" {
return fmt.Errorf("storage entry %q (docker-local) is missing host_path", e.Name)
}
if !filepath.IsAbs(e.HostPath) && !dockerNamedVolumeRegexp.MatchString(e.HostPath) {
return fmt.Errorf("storage entry %q host_path must be an absolute path or a docker named volume token, got %q", e.Name, e.HostPath)
}
if e.Size != "" {
return fmt.Errorf("storage entry %q (docker-local) does not accept --size", e.Name)
}
if e.StorageClass != "" {
return fmt.Errorf("storage entry %q (docker-local) does not accept --storage-class-name", e.Name)
}
if e.AccessMode != "" {
return fmt.Errorf("storage entry %q (docker-local) does not accept --access-mode", e.Name)
}
case SchedulerK3s:
if e.Size == "" {
return fmt.Errorf("storage entry %q (k3s) requires --size", e.Name)
}
if e.StorageClass != "" && e.HostPath != "" {
return fmt.Errorf("storage entry %q (k3s) cannot set both --storage-class-name and a host path; the cluster provisions class-backed volumes", e.Name)
}
if e.AccessMode != "" && !supportedAccessModes[e.AccessMode] {
return fmt.Errorf("storage entry %q has unsupported access mode %q", e.Name, e.AccessMode)
}
if e.HostPath != "" && !filepath.IsAbs(e.HostPath) {
return fmt.Errorf("storage entry %q host_path must be absolute, got %q", e.Name, e.HostPath)
}
if e.ReclaimPolicy != "" && e.ReclaimPolicy != ReclaimPolicyRetain && e.ReclaimPolicy != ReclaimPolicyDelete {
return fmt.Errorf("storage entry %q has unsupported reclaim policy %q", e.Name, e.ReclaimPolicy)
}
}
return nil
}
// LegacyMountToEntry synthesizes a deterministic Entry for a legacy
// "host:container[:options]" mount string. The synthesized name is
// stable across runs so re-running the migration is idempotent and so
// two apps mounting the same host path converge on a single entry.
func LegacyMountToEntry(mountPath string) *Entry {
parsed := ParseMountPath(mountPath)
host := parsed.HostPath
hashInput := host
if !strings.HasPrefix(host, "/") {
hashInput = "vol:" + host
}
sum := sha1.Sum([]byte(hashInput))
name := LegacyEntryPrefix + hex.EncodeToString(sum[:])[:10]
hostPath := host
if !strings.HasPrefix(host, "/") {
// Named docker volumes pass through unchanged on docker-local;
// the docker engine resolves them. Store as-is so the args
// shim can emit the original token.
hostPath = host
}
return &Entry{
Name: name,
Scheduler: SchedulerDockerLocal,
HostPath: hostPath,
SchemaVersion: SchemaVersion,
}
}

View File

@@ -0,0 +1,205 @@
package storage
import (
"path/filepath"
"strings"
"testing"
. "github.com/onsi/gomega"
)
func withTempLibRoot(t *testing.T) string {
t.Helper()
dir := t.TempDir()
t.Setenv("DOKKU_LIB_ROOT", dir)
return dir
}
func TestValidateEntryName(t *testing.T) {
RegisterTestingT(t)
Expect(ValidateEntryName("foo", false)).To(Succeed())
Expect(ValidateEntryName("foo-bar", false)).To(Succeed())
Expect(ValidateEntryName("a", false)).To(Succeed())
Expect(ValidateEntryName("foo123", false)).To(Succeed())
err := ValidateEntryName("", false)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("required"))
err = ValidateEntryName("Foo", false)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("DNS-1123"))
err = ValidateEntryName("foo_bar", false)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("DNS-1123"))
err = ValidateEntryName("-foo", false)
Expect(err).To(HaveOccurred())
err = ValidateEntryName("foo-", false)
Expect(err).To(HaveOccurred())
long := strings.Repeat("a", MaxEntryNameLength+1)
err = ValidateEntryName(long, false)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("too long"))
maxOK := strings.Repeat("a", MaxEntryNameLength)
Expect(ValidateEntryName(maxOK, false)).To(Succeed())
err = ValidateEntryName("legacy-foo", false)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("reserved"))
Expect(ValidateEntryName("legacy-foo", true)).To(Succeed())
}
func TestEntryValidateDockerLocal(t *testing.T) {
RegisterTestingT(t)
good := &Entry{Name: "foo", Scheduler: SchedulerDockerLocal, HostPath: "/var/lib/dokku/data/storage/foo"}
Expect(good.Validate()).To(Succeed())
noPath := &Entry{Name: "foo", Scheduler: SchedulerDockerLocal}
Expect(noPath.Validate()).To(HaveOccurred())
// A leading-slash-less but DNS-1123-ish token is treated as a docker
// named volume and accepted.
namedVolume := &Entry{Name: "foo", Scheduler: SchedulerDockerLocal, HostPath: "myvolume"}
Expect(namedVolume.Validate()).To(Succeed())
// Slash-containing relative paths still fail, as do tokens with bad chars.
relativePath := &Entry{Name: "foo", Scheduler: SchedulerDockerLocal, HostPath: "relative/path"}
Expect(relativePath.Validate()).To(HaveOccurred())
badToken := &Entry{Name: "foo", Scheduler: SchedulerDockerLocal, HostPath: "with spaces"}
Expect(badToken.Validate()).To(HaveOccurred())
withSize := &Entry{Name: "foo", Scheduler: SchedulerDockerLocal, HostPath: "/data", Size: "2Gi"}
Expect(withSize.Validate()).To(HaveOccurred())
withClass := &Entry{Name: "foo", Scheduler: SchedulerDockerLocal, HostPath: "/data", StorageClass: "longhorn"}
Expect(withClass.Validate()).To(HaveOccurred())
}
func TestEntryValidateK3s(t *testing.T) {
RegisterTestingT(t)
dynamic := &Entry{Name: "foo", Scheduler: SchedulerK3s, Size: "2Gi", StorageClass: "longhorn", AccessMode: "ReadWriteOnce"}
Expect(dynamic.Validate()).To(Succeed())
hostPath := &Entry{Name: "foo", Scheduler: SchedulerK3s, Size: "2Gi", HostPath: "/data/foo"}
Expect(hostPath.Validate()).To(Succeed())
noSize := &Entry{Name: "foo", Scheduler: SchedulerK3s, StorageClass: "longhorn"}
Expect(noSize.Validate()).To(HaveOccurred())
bothPathAndClass := &Entry{Name: "foo", Scheduler: SchedulerK3s, Size: "2Gi", HostPath: "/data", StorageClass: "longhorn"}
err := bothPathAndClass.Validate()
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("storage-class-name"))
badAccessMode := &Entry{Name: "foo", Scheduler: SchedulerK3s, Size: "2Gi", StorageClass: "longhorn", AccessMode: "Bogus"}
Expect(badAccessMode.Validate()).To(HaveOccurred())
badReclaim := &Entry{Name: "foo", Scheduler: SchedulerK3s, Size: "2Gi", StorageClass: "longhorn", ReclaimPolicy: "Recycle"}
Expect(badReclaim.Validate()).To(HaveOccurred())
}
func TestEntryValidateScheduler(t *testing.T) {
RegisterTestingT(t)
bad := &Entry{Name: "foo", Scheduler: "nomad"}
err := bad.Validate()
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("scheduler"))
}
func TestEntryRoundTrip(t *testing.T) {
RegisterTestingT(t)
root := withTempLibRoot(t)
original := &Entry{
Name: "demo-data",
Scheduler: SchedulerK3s,
Size: "2Gi",
StorageClass: "longhorn",
AccessMode: "ReadWriteOnce",
Namespace: "dokku",
Annotations: map[string]string{"backup.velero.io/backup-volumes": "demo-data"},
Labels: map[string]string{"app.kubernetes.io/managed-by": "dokku"},
}
Expect(SaveEntry(original)).To(Succeed())
Expect(EntryExists("demo-data")).To(BeTrue())
loaded, err := LoadEntry("demo-data")
Expect(err).NotTo(HaveOccurred())
Expect(loaded.Name).To(Equal(original.Name))
Expect(loaded.Scheduler).To(Equal(original.Scheduler))
Expect(loaded.Size).To(Equal(original.Size))
Expect(loaded.StorageClass).To(Equal(original.StorageClass))
Expect(loaded.AccessMode).To(Equal(original.AccessMode))
Expect(loaded.Namespace).To(Equal(original.Namespace))
Expect(loaded.Annotations).To(Equal(original.Annotations))
Expect(loaded.Labels).To(Equal(original.Labels))
Expect(loaded.SchemaVersion).To(Equal(SchemaVersion))
expectedPath := filepath.Join(root, "data", "storage-registry", "entries", "demo-data.json")
Expect(expectedPath).To(BeARegularFile())
Expect(DeleteEntry("demo-data")).To(Succeed())
Expect(EntryExists("demo-data")).To(BeFalse())
}
func TestListEntries(t *testing.T) {
RegisterTestingT(t)
withTempLibRoot(t)
entries, err := ListEntries()
Expect(err).NotTo(HaveOccurred())
Expect(entries).To(BeEmpty())
Expect(SaveEntry(&Entry{Name: "b", Scheduler: SchedulerDockerLocal, HostPath: "/b"})).To(Succeed())
Expect(SaveEntry(&Entry{Name: "a", Scheduler: SchedulerDockerLocal, HostPath: "/a"})).To(Succeed())
entries, err = ListEntries()
Expect(err).NotTo(HaveOccurred())
Expect(entries).To(HaveLen(2))
Expect(entries[0].Name).To(Equal("a"))
Expect(entries[1].Name).To(Equal("b"))
}
func TestLegacyMountToEntry(t *testing.T) {
RegisterTestingT(t)
a := LegacyMountToEntry("/host/path:/container/path")
b := LegacyMountToEntry("/host/path:/container/different")
c := LegacyMountToEntry("/host/path:/container/path:ro")
// Same host path produces the same entry name regardless of container
// path or options - container path lives on the attachment, not the
// entry, so apps mounting the same source converge.
Expect(a.Name).To(Equal(b.Name))
Expect(a.Name).To(Equal(c.Name))
Expect(a.Name).To(HavePrefix(LegacyEntryPrefix))
Expect(len(a.Name)).To(Equal(len(LegacyEntryPrefix) + 10))
// Different host paths produce different names.
other := LegacyMountToEntry("/different/path:/container/path")
Expect(other.Name).NotTo(Equal(a.Name))
// Named docker volumes are distinguished from absolute paths.
named := LegacyMountToEntry("my_volume:/container/path")
Expect(named.Name).NotTo(Equal(a.Name))
Expect(named.HostPath).To(Equal("my_volume"))
// The synthesized entry is a valid docker-local entry that passes
// Validate (other than the legacy- prefix, which Validate accepts).
abs := LegacyMountToEntry("/host/path:/container/path")
Expect(abs.Scheduler).To(Equal(SchedulerDockerLocal))
Expect(abs.HostPath).To(Equal("/host/path"))
Expect(abs.Validate()).To(Succeed())
}

206
plugins/storage/migrate.go Normal file
View File

@@ -0,0 +1,206 @@
package storage
import (
"errors"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"github.com/dokku/dokku/plugins/common"
dockeroptions "github.com/dokku/dokku/plugins/docker-options"
)
// migrationFlagDir contains per-app flag files written after a successful
// per-app migration; presence means the app's legacy -v lines have been
// drained into the attachment store. Lives under the registry directory
// so it doesn't collide with the property-store path
// `config/storage/<appName>`.
func migrationFlagDir() string {
return filepath.Join(RegistryDirectory(), "migrations")
}
func migrationFlagFile(appName string) string {
return filepath.Join(migrationFlagDir(), appName)
}
// MigrateApp re-runs the legacy migration for a single app, ignoring
// the per-app flag file so an operator can force-rescan an app whose
// docker-options state changed after the bulk install-time pass.
// Used by `dokku storage:migrate <app>`.
func MigrateApp(appName string) error {
if err := os.MkdirAll(migrationFlagDir(), 0755); err != nil {
return fmt.Errorf("unable to create migration flag dir: %w", err)
}
_ = os.Remove(migrationFlagFile(appName))
if err := migrateApp(appName); err != nil {
return fmt.Errorf("storage migration failed for app %q: %w", appName, err)
}
return nil
}
// MigrateLegacyMounts walks every app and converts its legacy `-v`
// docker-options entries into named storage entries plus attachments.
// Idempotent: per-app flag files short-circuit re-runs.
func MigrateLegacyMounts() error {
if err := os.MkdirAll(migrationFlagDir(), 0755); err != nil {
return fmt.Errorf("unable to create migration flag dir: %w", err)
}
apps, err := common.DokkuApps()
if err != nil {
if errors.Is(err, common.NoAppsExist) {
return nil
}
return err
}
for _, app := range apps {
if _, err := os.Stat(migrationFlagFile(app)); err == nil {
continue
}
if err := migrateApp(app); err != nil {
return fmt.Errorf("storage migration failed for app %q: %w", app, err)
}
}
return nil
}
// migrateApp performs the per-app migration. Order is intentional: write
// the entry, write the attachment, drain the docker-options line. Any
// step failing aborts before drain, so the original behavior is
// preserved on partial failure.
func migrateApp(appName string) error {
deployLines, err := dockeroptions.GetDockerOptionsForPhase(appName, PhaseDeploy)
if err != nil {
return err
}
runLines, err := dockeroptions.GetDockerOptionsForPhase(appName, PhaseRun)
if err != nil {
return err
}
deployMounts := filterMountLines(deployLines)
runMounts := filterMountLines(runLines)
// Group identical mount strings across phases.
phaseMap := map[string][]string{}
for _, mount := range deployMounts {
phaseMap[mount] = appendUnique(phaseMap[mount], PhaseDeploy)
}
for _, mount := range runMounts {
phaseMap[mount] = appendUnique(phaseMap[mount], PhaseRun)
}
mounts := make([]string, 0, len(phaseMap))
for mount := range phaseMap {
mounts = append(mounts, mount)
}
sort.Strings(mounts)
for _, mount := range mounts {
if err := migrateMount(appName, mount, phaseMap[mount]); err != nil {
return err
}
}
return touchMigrationFlag(appName)
}
func migrateMount(appName string, mount string, phases []string) error {
parsed := ParseMountPath(mount)
entry := LegacyMountToEntry(mount)
if EntryExists(entry.Name) {
existing, err := LoadEntry(entry.Name)
if err != nil {
return err
}
if existing.HostPath != entry.HostPath || existing.Scheduler != entry.Scheduler {
return fmt.Errorf("legacy entry %q already exists with conflicting fields", entry.Name)
}
} else {
if err := SaveEntry(entry); err != nil {
return err
}
}
containerPath := parsed.ContainerPath
if containerPath == "" {
// Defensive: legacy mounts must have a container path; skip if not.
return nil
}
attachment := &Attachment{
EntryName: entry.Name,
ContainerPath: containerPath,
Phases: phases,
ProcessType: DefaultProcessType,
}
if parsed.VolumeOptions != "" {
if parsed.VolumeOptions == "ro" {
attachment.Readonly = true
} else {
attachment.VolumeOptions = parsed.VolumeOptions
}
}
existing, err := LoadAttachments(appName)
if err != nil {
return err
}
if !attachmentExists(existing, attachment) {
existing = append(existing, attachment)
if err := SaveAttachments(appName, existing); err != nil {
return err
}
}
mountedPhases := []string{}
for _, phase := range phases {
mountedPhases = append(mountedPhases, phase)
}
if err := dockeroptions.RemoveDockerOptionFromPhases(appName, mountedPhases, fmt.Sprintf("-v %s", mount)); err != nil {
return err
}
return nil
}
func touchMigrationFlag(appName string) error {
f, err := os.Create(migrationFlagFile(appName))
if err != nil {
return err
}
return f.Close()
}
func filterMountLines(lines []string) []string {
out := []string{}
for _, line := range lines {
if strings.HasPrefix(line, "-v ") {
out = append(out, strings.TrimPrefix(line, "-v "))
}
}
return out
}
func appendUnique(slice []string, value string) []string {
for _, existing := range slice {
if existing == value {
return slice
}
}
return append(slice, value)
}
func attachmentExists(attachments []*Attachment, candidate *Attachment) bool {
for _, existing := range attachments {
if existing.EntryName == candidate.EntryName &&
existing.ContainerPath == candidate.ContainerPath &&
existing.ProcessType == candidate.ProcessType {
return true
}
}
return false
}

View File

@@ -0,0 +1,317 @@
package storage
import (
"os"
"os/user"
"path/filepath"
"sort"
"testing"
"github.com/dokku/dokku/plugins/common"
dockeroptions "github.com/dokku/dokku/plugins/docker-options"
)
// setupMigrationEnv points DOKKU_LIB_ROOT and DOKKU_ROOT at temp dirs and
// tells the permission helpers to chown to the current user (a no-op),
// so the test works without root and without the dokku system user.
// Mirrors the helper in docker-options/migration_test.go.
func setupMigrationEnv(t *testing.T) (libRoot string, dokkuRoot string) {
t.Helper()
libRoot = t.TempDir()
dokkuRoot = t.TempDir()
t.Setenv("DOKKU_LIB_ROOT", libRoot)
t.Setenv("DOKKU_ROOT", dokkuRoot)
t.Setenv("PLUGIN_PATH", filepath.Join(libRoot, "plugins"))
current, err := user.Current()
if err != nil {
t.Fatalf("user.Current: %v", err)
}
group, err := user.LookupGroupId(current.Gid)
if err != nil {
t.Fatalf("user.LookupGroupId: %v", err)
}
t.Setenv("DOKKU_SYSTEM_USER", current.Username)
t.Setenv("DOKKU_SYSTEM_GROUP", group.Name)
// MigrateLegacyMounts creates the flag dir before iterating apps;
// migrateApp (the per-app helper) assumes it already exists. Pre-
// create it so tests that call migrateApp directly don't trip.
if err := os.MkdirAll(migrationFlagDir(), 0755); err != nil {
t.Fatalf("mkdir migration flag dir: %v", err)
}
return libRoot, dokkuRoot
}
// stageApp registers the named app under DOKKU_ROOT so DokkuApps can
// see it, and seeds its docker-options property files for the requested
// phases. Each map entry is a phase ("deploy", "run", "build") whose
// value is the full property-list contents (one option per line).
func stageApp(t *testing.T, dokkuRoot string, app string, options map[string][]string) {
t.Helper()
if err := os.MkdirAll(filepath.Join(dokkuRoot, app), 0755); err != nil {
t.Fatalf("mkdir app: %v", err)
}
for phase, lines := range options {
if err := dockeroptions.AddDockerOptionToPhases(app, []string{phase}, ""); err != nil {
t.Fatalf("seed property file: %v", err)
}
// Replace whatever AddDockerOptionToPhases left with the precise
// list the test wants, since it appends rather than overwriting.
key := "_default_." + phase
if err := common.PropertyListWrite("docker-options", app, key, lines); err != nil {
t.Fatalf("PropertyListWrite: %v", err)
}
}
}
func phaseOptions(t *testing.T, app string, phase string) []string {
t.Helper()
got, err := dockeroptions.GetDockerOptionsForPhase(app, phase)
if err != nil {
t.Fatalf("GetDockerOptionsForPhase %s/%s: %v", app, phase, err)
}
return got
}
func TestMigrateAppSinglePhase(t *testing.T) {
_, dokkuRoot := setupMigrationEnv(t)
stageApp(t, dokkuRoot, "alpha", map[string][]string{
"deploy": {"-v /var/log:/log", "--restart=on-failure:5"},
})
if err := migrateApp("alpha"); err != nil {
t.Fatalf("migrateApp: %v", err)
}
entry := LegacyMountToEntry("/var/log:/log")
if !EntryExists(entry.Name) {
t.Fatalf("expected entry %s on disk", entry.Name)
}
loaded, err := LoadEntry(entry.Name)
if err != nil {
t.Fatalf("LoadEntry: %v", err)
}
if loaded.HostPath != "/var/log" {
t.Errorf("entry HostPath = %s, want /var/log", loaded.HostPath)
}
if loaded.Scheduler != SchedulerDockerLocal {
t.Errorf("entry Scheduler = %s, want %s", loaded.Scheduler, SchedulerDockerLocal)
}
attachments, err := LoadAttachments("alpha")
if err != nil {
t.Fatalf("LoadAttachments: %v", err)
}
if len(attachments) != 1 {
t.Fatalf("expected 1 attachment, got %d", len(attachments))
}
att := attachments[0]
if att.EntryName != entry.Name || att.ContainerPath != "/log" {
t.Errorf("attachment = %+v, want entry=%s container=/log", att, entry.Name)
}
if len(att.Phases) != 1 || att.Phases[0] != PhaseDeploy {
t.Errorf("attachment phases = %v, want [deploy]", att.Phases)
}
// docker-options drained the -v line but kept everything else.
deployAfter := phaseOptions(t, "alpha", "deploy")
if !equalSorted(deployAfter, []string{"--restart=on-failure:5"}) {
t.Errorf("deploy options after migration = %v, want [--restart=on-failure:5]", deployAfter)
}
// Migration flag file is in place.
if _, err := os.Stat(migrationFlagFile("alpha")); err != nil {
t.Errorf("migration flag missing: %v", err)
}
}
func TestMigrateAppCrossPhaseGroupsIntoOneAttachment(t *testing.T) {
_, dokkuRoot := setupMigrationEnv(t)
// Same -v in both phases groups into one attachment with phases = [deploy, run].
stageApp(t, dokkuRoot, "alpha", map[string][]string{
"deploy": {"-v /data:/d"},
"run": {"-v /data:/d"},
})
if err := migrateApp("alpha"); err != nil {
t.Fatalf("migrateApp: %v", err)
}
attachments, err := LoadAttachments("alpha")
if err != nil {
t.Fatalf("LoadAttachments: %v", err)
}
if len(attachments) != 1 {
t.Fatalf("expected 1 attachment, got %d", len(attachments))
}
got := attachments[0].Phases
sort.Strings(got)
if !equalSorted(got, []string{PhaseDeploy, PhaseRun}) {
t.Errorf("phases = %v, want [deploy run]", got)
}
// Drain affected both phases.
if got := phaseOptions(t, "alpha", "deploy"); len(got) != 0 {
t.Errorf("deploy options after migration = %v, want []", got)
}
if got := phaseOptions(t, "alpha", "run"); len(got) != 0 {
t.Errorf("run options after migration = %v, want []", got)
}
}
func TestMigrateAppPreservesVolumeOptions(t *testing.T) {
_, dokkuRoot := setupMigrationEnv(t)
stageApp(t, dokkuRoot, "alpha", map[string][]string{
"deploy": {"-v /ro:/r:ro", "-v /label:/l:Z"},
})
if err := migrateApp("alpha"); err != nil {
t.Fatalf("migrateApp: %v", err)
}
atts, err := LoadAttachments("alpha")
if err != nil {
t.Fatalf("LoadAttachments: %v", err)
}
byPath := map[string]*Attachment{}
for _, a := range atts {
byPath[a.ContainerPath] = a
}
if a := byPath["/r"]; a == nil {
t.Fatalf("missing /r attachment")
} else if !a.Readonly || a.VolumeOptions != "" {
t.Errorf("/r attachment = readonly=%v opts=%q, want readonly=true opts=\"\"", a.Readonly, a.VolumeOptions)
}
if a := byPath["/l"]; a == nil {
t.Fatalf("missing /l attachment")
} else if a.Readonly || a.VolumeOptions != "Z" {
t.Errorf("/l attachment = readonly=%v opts=%q, want readonly=false opts=\"Z\"", a.Readonly, a.VolumeOptions)
}
}
func TestMigrateAppIdempotent(t *testing.T) {
_, dokkuRoot := setupMigrationEnv(t)
stageApp(t, dokkuRoot, "alpha", map[string][]string{
"deploy": {"-v /var/log:/log"},
})
if err := migrateApp("alpha"); err != nil {
t.Fatalf("first migrateApp: %v", err)
}
// Re-stage the same -v line by hand to simulate someone re-adding
// the legacy form via docker-options:add after migration. The
// migration is gated by the per-app flag file at the
// MigrateLegacyMounts level, but migrateApp itself should still be
// safe to call directly: a second run produces no duplicates.
if err := common.PropertyListWrite("docker-options", "alpha", "_default_.deploy", []string{"-v /var/log:/log"}); err != nil {
t.Fatalf("re-stage: %v", err)
}
if err := migrateApp("alpha"); err != nil {
t.Fatalf("second migrateApp: %v", err)
}
atts, err := LoadAttachments("alpha")
if err != nil {
t.Fatalf("LoadAttachments: %v", err)
}
if len(atts) != 1 {
t.Errorf("expected 1 attachment after rerun, got %d", len(atts))
}
}
func TestMigrateLegacyMountsBulkAndFlagFastPath(t *testing.T) {
_, dokkuRoot := setupMigrationEnv(t)
stageApp(t, dokkuRoot, "alpha", map[string][]string{
"deploy": {"-v /alpha-data:/data"},
})
stageApp(t, dokkuRoot, "beta", map[string][]string{
"deploy": {"-v /beta-data:/data"},
"run": {"-v /beta-data:/data"},
})
if err := MigrateLegacyMounts(); err != nil {
t.Fatalf("MigrateLegacyMounts: %v", err)
}
for _, app := range []string{"alpha", "beta"} {
if _, err := os.Stat(migrationFlagFile(app)); err != nil {
t.Errorf("flag file missing for %s: %v", app, err)
}
atts, err := LoadAttachments(app)
if err != nil {
t.Fatalf("LoadAttachments %s: %v", app, err)
}
if len(atts) != 1 {
t.Errorf("%s: expected 1 attachment, got %d", app, len(atts))
}
}
// Re-stage a -v on alpha and rerun MigrateLegacyMounts. The flag
// file short-circuits per-app migration, so the new line stays in
// docker-options.
if err := common.PropertyListWrite("docker-options", "alpha", "_default_.deploy", []string{"-v /sneaky:/x"}); err != nil {
t.Fatalf("re-stage alpha: %v", err)
}
if err := MigrateLegacyMounts(); err != nil {
t.Fatalf("MigrateLegacyMounts (rerun): %v", err)
}
got := phaseOptions(t, "alpha", "deploy")
if !equalSorted(got, []string{"-v /sneaky:/x"}) {
t.Errorf("expected sneaky -v line untouched after second pass, got %v", got)
}
atts, err := LoadAttachments("alpha")
if err != nil {
t.Fatalf("LoadAttachments alpha rerun: %v", err)
}
if len(atts) != 1 {
t.Errorf("expected 1 attachment after rerun, got %d", len(atts))
}
}
func TestMigrateAppRefusesEntryNameCollision(t *testing.T) {
libRoot, dokkuRoot := setupMigrationEnv(t)
_ = libRoot
// Plant a colliding entry at the legacy-<hash> name with conflicting
// fields so migration's "fields match" check trips.
collide := LegacyMountToEntry("/var/log:/log")
collide.HostPath = "/something/else"
if err := SaveEntry(collide); err != nil {
t.Fatalf("seed colliding entry: %v", err)
}
stageApp(t, dokkuRoot, "alpha", map[string][]string{
"deploy": {"-v /var/log:/log"},
})
if err := migrateApp("alpha"); err == nil {
t.Fatalf("expected migrateApp to error on entry name collision")
}
// Drain did not happen.
if got := phaseOptions(t, "alpha", "deploy"); len(got) != 1 || got[0] != "-v /var/log:/log" {
t.Errorf("expected legacy line preserved on collision, got %v", got)
}
}
func equalSorted(a, b []string) bool {
if len(a) != len(b) {
return false
}
ax := append([]string(nil), a...)
bx := append([]string(nil), b...)
sort.Strings(ax)
sort.Strings(bx)
for i := range ax {
if ax[i] != bx[i] {
return false
}
}
return true
}

View File

@@ -18,11 +18,19 @@ Manage mounted volumes
Additional commands:`
helpContent = `
storage:ensure-directory [--chown option] <directory>, Creates a persistent storage directory in the recommended storage path
storage:create <name> [<path>] [flags], Register a named storage entry
storage:destroy <name>, Remove a named storage entry (must be unmounted from every app first)
storage:ensure-directory [--chown option] <directory>, [DEPRECATED] use storage:create instead
storage:exec <name> [-- <cmd>...], Run a command (or shell) in a temporary container that mounts the entry
storage:info <name> [--format text|json], Show details for one storage entry
storage:list <app> [--format text|json], List bind mounts for app's container(s) (host:container)
storage:list-entries [--scheduler s] [--format text|json], List registered storage entries
storage:migrate [<app>|--all], Re-run the legacy -v to attachment migration for an app
storage:mount <app> <host-dir:container-dir>, Create a new bind mount
storage:report [<app>] [<flag>], Displays a storage report for one or more apps
storage:unmount <app> <host-dir:container-dir>, Remove an existing bind mount`
storage:set <name> [flags], Update a storage entry in place
storage:unmount <app> <host-dir:container-dir>, Remove an existing bind mount
storage:wait <name>, Wait for a storage entry's PVC to be bound (k3s)`
)
func main() {

View File

@@ -19,24 +19,152 @@ func main() {
switch subcommand {
case "default":
err = storage.CommandHelp()
case "create":
args := flag.NewFlagSet("storage:create", flag.ExitOnError)
scheduler := args.String("scheduler", storage.SchedulerDockerLocal, "--scheduler: target scheduler (docker-local, k3s)")
size := args.String("size", "", "--size: PVC size (k3s only, e.g. 2Gi)")
accessMode := args.String("access-mode", "", "--access-mode: PVC access mode (k3s only)")
storageClass := args.String("storage-class-name", "", "--storage-class-name: PVC storage class (k3s only)")
namespace := args.String("namespace", "", "--namespace: PVC namespace (k3s only)")
chown := args.String("chown", "", "--chown: chown option (docker-local only)")
reclaim := args.String("reclaim-policy", "", "--reclaim-policy: PV reclaim policy (Retain or Delete, k3s only)")
annotations := args.StringSlice("annotation", nil, "--annotation key=value: PVC annotation (repeatable)")
labels := args.StringSlice("label", nil, "--label key=value: PVC label (repeatable)")
args.Parse(os.Args[2:])
name := args.Arg(0)
path := args.Arg(1)
annotMap, parseErr := parseKVPairs(*annotations)
if parseErr != nil {
err = parseErr
break
}
labelMap, parseErr := parseKVPairs(*labels)
if parseErr != nil {
err = parseErr
break
}
err = storage.CommandCreate(storage.CommandCreateInput{
Name: name,
Path: path,
Scheduler: *scheduler,
Size: *size,
AccessMode: *accessMode,
StorageClass: *storageClass,
Namespace: *namespace,
Chown: *chown,
ReclaimPolicy: *reclaim,
Annotations: annotMap,
Labels: labelMap,
})
case "destroy":
args := flag.NewFlagSet("storage:destroy", flag.ExitOnError)
args.Parse(os.Args[2:])
err = storage.CommandDestroy(args.Arg(0))
case "ensure-directory":
args := flag.NewFlagSet("storage:ensure-directory", flag.ExitOnError)
chown := args.String("chown", "herokuish", "--chown: chown option (herokuish, heroku, paketo, root, false)")
args.Parse(os.Args[2:])
directory := args.Arg(0)
common.LogWarn("storage:ensure-directory is deprecated; use storage:create instead.")
err = storage.CommandEnsureDirectory(directory, *chown)
case "info":
args := flag.NewFlagSet("storage:info", flag.ExitOnError)
format := args.String("format", "text", "--format: output format (text, json)")
args.Parse(os.Args[2:])
err = storage.CommandInfo(args.Arg(0), *format)
case "list":
args := flag.NewFlagSet("storage:list", flag.ExitOnError)
format := args.String("format", "text", "--format: output format (text, json)")
args.Parse(os.Args[2:])
appName := args.Arg(0)
err = storage.CommandList(appName, *format)
case "set":
args := flag.NewFlagSet("storage:set", flag.ExitOnError)
size := args.String("size", "", "--size: new PVC size (k3s)")
accessMode := args.String("access-mode", "", "--access-mode: existing access mode (must match)")
storageClass := args.String("storage-class-name", "", "--storage-class-name: existing storage class (must match)")
namespace := args.String("namespace", "", "--namespace: new namespace")
chown := args.String("chown", "", "--chown: chown option")
reclaim := args.String("reclaim-policy", "", "--reclaim-policy: PV reclaim policy")
annotations := args.StringSlice("annotation", nil, "--annotation key=value: PVC annotation (repeatable, replaces all)")
labels := args.StringSlice("label", nil, "--label key=value: PVC label (repeatable, replaces all)")
args.Parse(os.Args[2:])
annotMap, parseErr := parseKVPairs(*annotations)
if parseErr != nil {
err = parseErr
break
}
labelMap, parseErr := parseKVPairs(*labels)
if parseErr != nil {
err = parseErr
break
}
err = storage.CommandSet(storage.CommandSetInput{
Name: args.Arg(0),
Size: *size,
AccessMode: *accessMode,
StorageClass: *storageClass,
Namespace: *namespace,
Chown: *chown,
ReclaimPolicy: *reclaim,
Annotations: annotMap,
Labels: labelMap,
})
case "exec":
args := flag.NewFlagSet("storage:exec", flag.ExitOnError)
image := args.String("image", "", "--image: container image to use (default alpine:3)")
asUser := args.String("as-user", "", "--as-user: numeric uid to run the exec container as (overrides the entry's chown)")
args.Parse(os.Args[2:])
positional := args.Args()
if len(positional) == 0 {
err = fmt.Errorf("storage:exec requires a storage entry name")
break
}
name := positional[0]
var cmd []string
if len(positional) > 1 {
cmd = positional[1:]
}
err = storage.CommandExec(storage.CommandExecInput{
Name: name,
Image: *image,
AsUser: *asUser,
Args: cmd,
})
case "migrate":
args := flag.NewFlagSet("storage:migrate", flag.ExitOnError)
all := args.Bool("all", false, "--all: re-migrate every app on this install")
args.Parse(os.Args[2:])
err = storage.CommandMigrate(args.Arg(0), *all)
case "wait":
args := flag.NewFlagSet("storage:wait", flag.ExitOnError)
args.Parse(os.Args[2:])
err = storage.CommandWait(args.Arg(0))
case "list-entries":
args := flag.NewFlagSet("storage:list-entries", flag.ExitOnError)
scheduler := args.String("scheduler", "", "--scheduler: filter to a single scheduler")
format := args.String("format", "text", "--format: output format (text, json)")
args.Parse(os.Args[2:])
err = storage.CommandListEntries(*scheduler, *format)
case "mount":
args := flag.NewFlagSet("storage:mount", flag.ExitOnError)
containerDir := args.String("container-dir", "", "--container-dir: container path (named-entry form)")
phases := args.StringSlice("phase", nil, "--phase: phase to mount in (deploy, run; default both)")
processType := args.String("process-type", "", "--process-type: process type to mount for")
subpath := args.String("volume-subpath", "", "--volume-subpath: subpath within the entry")
readonly := args.Bool("volume-readonly", false, "--volume-readonly: mount the volume read-only")
volumeChown := args.String("volume-chown", "", "--volume-chown: chown option applied at mount time")
args.Parse(os.Args[2:])
appName := args.Arg(0)
mountPath := args.Arg(1)
err = storage.CommandMount(appName, mountPath)
err = storage.CommandMount(storage.CommandMountInput{
AppName: args.Arg(0),
NameOrPath: args.Arg(1),
ContainerDir: *containerDir,
Phases: *phases,
ProcessType: *processType,
Subpath: *subpath,
Readonly: *readonly,
VolumeChown: *volumeChown,
})
case "report":
args := flag.NewFlagSet("storage:report", flag.ExitOnError)
format := args.String("format", "stdout", "format: [ stdout | json ]")
@@ -51,10 +179,13 @@ func main() {
}
case "unmount":
args := flag.NewFlagSet("storage:unmount", flag.ExitOnError)
containerDir := args.String("container-dir", "", "--container-dir: container path (named-entry form, disambiguates duplicates)")
args.Parse(os.Args[2:])
appName := args.Arg(0)
mountPath := args.Arg(1)
err = storage.CommandUnmount(appName, mountPath)
err = storage.CommandUnmount(storage.CommandUnmountInput{
AppName: args.Arg(0),
NameOrPath: args.Arg(1),
ContainerDir: *containerDir,
})
default:
err = fmt.Errorf("Invalid plugin subcommand call: %s", subcommand)
}
@@ -63,3 +194,18 @@ func main() {
common.LogFailWithError(err)
}
}
func parseKVPairs(pairs []string) (map[string]string, error) {
if len(pairs) == 0 {
return nil, nil
}
out := map[string]string{}
for _, pair := range pairs {
idx := strings.Index(pair, "=")
if idx <= 0 {
return nil, fmt.Errorf("expected key=value, got %q", pair)
}
out[pair[:idx]] = pair[idx+1:]
}
return out, nil
}

View File

@@ -24,6 +24,27 @@ func main() {
phase := flag.Arg(1)
format := flag.Arg(2)
err = storage.TriggerStorageList(appName, phase, format)
case "storage-app-mounts":
appName := flag.Arg(0)
phase := flag.Arg(1)
err = storage.TriggerStorageAppMounts(appName, phase)
case "docker-args-deploy":
appName := flag.Arg(0)
err = storage.TriggerDockerArgs(appName, storage.PhaseDeploy)
case "docker-args-run":
appName := flag.Arg(0)
err = storage.TriggerDockerArgs(appName, storage.PhaseRun)
case "post-delete":
appName := flag.Arg(0)
err = storage.TriggerPostDelete(appName)
case "post-app-clone-setup":
oldName := flag.Arg(0)
newName := flag.Arg(1)
err = storage.TriggerPostAppCloneSetup(oldName, newName)
case "post-app-rename-setup":
oldName := flag.Arg(0)
newName := flag.Arg(1)
err = storage.TriggerPostAppRenameSetup(oldName, newName)
default:
err = fmt.Errorf("Invalid plugin trigger call: %s", trigger)
}

View File

@@ -51,19 +51,18 @@ func CheckIfPathExists(appName string, mountPath string, phases []string) bool {
return false
}
// GetBindMounts returns the bind mounts for an app and phase
// GetBindMounts returns the bind mounts for an app and phase, synthesized
// from the attachment store. The returned strings use the legacy colon
// form (host:container[:options]) so existing display paths keep
// working unchanged.
func GetBindMounts(appName string, phase string) ([]string, error) {
mounts := []string{}
options, err := dockeroptions.GetDockerOptionsForPhase(appName, phase)
entries, err := ListAppMountEntries(appName, phase)
if err != nil {
return mounts, err
return nil, err
}
for _, option := range options {
if strings.HasPrefix(option, "-v ") {
mount := strings.TrimPrefix(option, "-v ")
mounts = append(mounts, mount)
}
mounts := make([]string, 0, len(entries))
for _, entry := range entries {
mounts = append(mounts, formatStorageListEntry(entry))
}
return mounts, nil
}
@@ -82,13 +81,71 @@ func GetBindMountsForDisplay(appName string, phase string) string {
return strings.Join(result, " ")
}
// StorageListEntry represents a storage mount entry for JSON output
// StorageListEntry represents a storage mount entry for JSON output.
type StorageListEntry struct {
EntryName string `json:"entry_name,omitempty"`
HostPath string `json:"host_path"`
ContainerPath string `json:"container_path"`
VolumeOptions string `json:"volume_options"`
}
// ListAppMountEntries returns one StorageListEntry per attachment on
// an app for the requested phase, joining each attachment with its
// referenced storage entry. Used by storage:list and the deprecated
// storage-list trigger.
func ListAppMountEntries(appName string, phase string) ([]StorageListEntry, error) {
if phase == "" {
phase = PhaseDeploy
}
attachments, err := AttachmentsForPhase(appName, phase)
if err != nil {
return nil, err
}
rows := make([]StorageListEntry, 0, len(attachments))
for _, attachment := range attachments {
entry, err := LoadEntry(attachment.EntryName)
if err != nil {
return nil, fmt.Errorf("attachment on %q references missing entry %q: %w", appName, attachment.EntryName, err)
}
host := entry.HostPath
if host == "" {
// k3s-only entries with no host path: surface the entry
// name as the host token so the colon-form output is
// well-formed and parseable.
host = entry.Name
}
options := ""
switch {
case attachment.Readonly && attachment.VolumeOptions != "":
options = "ro," + attachment.VolumeOptions
case attachment.Readonly:
options = "ro"
case attachment.VolumeOptions != "":
options = attachment.VolumeOptions
}
rows = append(rows, StorageListEntry{
EntryName: entry.Name,
HostPath: host,
ContainerPath: attachment.ContainerPath,
VolumeOptions: options,
})
}
return rows, nil
}
// formatStorageListEntry renders a StorageListEntry into the legacy
// host:container[:options] colon form for textual output.
func formatStorageListEntry(entry StorageListEntry) string {
if entry.VolumeOptions == "" {
return fmt.Sprintf("%s:%s", entry.HostPath, entry.ContainerPath)
}
return fmt.Sprintf("%s:%s:%s", entry.HostPath, entry.ContainerPath, entry.VolumeOptions)
}
// ParseMountPath parses a mount path into its components
func ParseMountPath(mountPath string) StorageListEntry {
parts := strings.SplitN(mountPath, ":", 3)

View File

@@ -1,6 +1,7 @@
package storage
import (
"encoding/json"
"errors"
"fmt"
"os"
@@ -8,7 +9,6 @@ import (
"strings"
"github.com/dokku/dokku/plugins/common"
dockeroptions "github.com/dokku/dokku/plugins/docker-options"
)
const (
@@ -19,11 +19,19 @@ Manage mounted volumes
Additional commands:`
helpContent = `
storage:ensure-directory [--chown option] <directory>, Creates a persistent storage directory in the recommended storage path
storage:create <name> [<path>] [flags], Register a named storage entry
storage:destroy <name>, Remove a named storage entry (must be unmounted from every app first)
storage:ensure-directory [--chown option] <directory>, [DEPRECATED] use storage:create instead
storage:exec <name> [-- <cmd>...], Run a command (or shell) in a temporary container that mounts the entry
storage:info <name> [--format text|json], Show details for one storage entry
storage:list <app> [--format text|json], List bind mounts for app's container(s) (host:container)
storage:list-entries [--scheduler s] [--format text|json], List registered storage entries
storage:migrate [<app>|--all], Re-run the legacy -v to attachment migration for an app
storage:mount <app> <host-dir:container-dir>, Create a new bind mount
storage:report [<app>] [<flag>], Displays a storage report for one or more apps
storage:unmount <app> <host-dir:container-dir>, Remove an existing bind mount`
storage:set <name> [flags], Update a storage entry in place
storage:unmount <app> <host-dir:container-dir>, Remove an existing bind mount
storage:wait <name>, Wait for a storage entry's PVC to be bound (k3s)`
)
// CommandHelp displays help for the storage plugin
@@ -38,7 +46,7 @@ func CommandEnsureDirectory(directory string, chownFlag string) error {
return err
}
chownID, err := resolveChownID(chownFlag)
chownID, err := ResolveChownID(chownFlag)
if err != nil {
return err
}
@@ -72,8 +80,8 @@ func CommandEnsureDirectory(directory string, chownFlag string) error {
return nil
}
// resolveChownID converts a chown flag value to a numeric UID
func resolveChownID(chownFlag string) (string, error) {
// ResolveChownID converts a chown flag value to a numeric UID
func ResolveChownID(chownFlag string) (string, error) {
var chownID string
switch chownFlag {
@@ -121,41 +129,167 @@ func isUserNamespacesEnabled() (bool, error) {
return strings.TrimSpace(result.StdoutContents()) == "true", nil
}
// CommandMount creates a new bind mount for an app
func CommandMount(appName string, mountPath string) error {
if err := common.VerifyAppName(appName); err != nil {
// CommandMountInput captures the optional flags accepted by storage:mount
// when the second argument is a named entry rather than a colon-form path.
type CommandMountInput struct {
AppName string
NameOrPath string
ContainerDir string
Phases []string
ProcessType string
Subpath string
Readonly bool
VolumeOptions string
VolumeChown string
}
// CommandMount creates a new bind mount for an app. The second positional
// argument may be either a legacy host:container[:options] string (kept
// for back-compat on docker-local) or a registered storage entry name.
func CommandMount(input CommandMountInput) error {
if err := common.VerifyAppName(input.AppName); err != nil {
return err
}
// Legacy colon form: synthesize a legacy-<hash> entry plus an
// attachment so storage:list (now attachment-only) sees the mount.
// The storage docker-args trigger emits the corresponding -v flag at
// deploy time, so behavior at the docker-run boundary is unchanged.
if strings.Contains(input.NameOrPath, ":") {
return mountLegacyColon(input.AppName, input.NameOrPath)
}
// Named-entry form: persist as an attachment.
if !EntryExists(input.NameOrPath) {
return fmt.Errorf("storage entry %q does not exist; create it first with `dokku storage:create`", input.NameOrPath)
}
if input.ContainerDir == "" {
return errors.New("--container-dir is required when mounting a named storage entry")
}
entry, err := LoadEntry(input.NameOrPath)
if err != nil {
return err
}
phases := input.Phases
if len(phases) == 0 {
phases = []string{PhaseDeploy, PhaseRun}
}
processType := input.ProcessType
if processType == "" {
processType = DefaultProcessType
}
attachment := &Attachment{
EntryName: entry.Name,
ContainerPath: input.ContainerDir,
Phases: phases,
ProcessType: processType,
Subpath: input.Subpath,
Readonly: input.Readonly,
VolumeOptions: input.VolumeOptions,
VolumeChown: input.VolumeChown,
}
return AddAttachment(input.AppName, attachment)
}
// CommandUnmountInput captures the flags accepted by storage:unmount when
// the second argument is a named entry.
type CommandUnmountInput struct {
AppName string
NameOrPath string
ContainerDir string
}
// CommandUnmount removes an existing bind mount from an app.
func CommandUnmount(input CommandUnmountInput) error {
if err := common.VerifyAppName(input.AppName); err != nil {
return err
}
if strings.Contains(input.NameOrPath, ":") {
return unmountLegacyColon(input.AppName, input.NameOrPath)
}
return RemoveAttachment(input.AppName, input.NameOrPath, input.ContainerDir)
}
// mountLegacyColon translates a `<host>:<container>[:options]` mount
// string into a synthesized legacy-<hash> entry plus an attachment.
// Idempotent: re-running the same mount errors with the existing
// "already mounted" message via AddAttachment's duplicate check.
func mountLegacyColon(appName string, mountPath string) error {
if err := VerifyPaths(mountPath); err != nil {
return err
}
if CheckIfPathExists(appName, mountPath, MountPhases) {
return errors.New("Mount path already exists.")
parsed := ParseMountPath(mountPath)
if parsed.ContainerPath == "" {
return errors.New("Storage path must be two valid paths divided by colon.")
}
return dockeroptions.AddDockerOptionToPhases(appName, MountPhases, fmt.Sprintf("-v %s", mountPath))
}
entry := LegacyMountToEntry(mountPath)
if !EntryExists(entry.Name) {
if err := SaveEntry(entry); err != nil {
return err
}
}
// CommandUnmount removes an existing bind mount from an app
func CommandUnmount(appName string, mountPath string) error {
if err := common.VerifyAppName(appName); err != nil {
attachment := &Attachment{
EntryName: entry.Name,
ContainerPath: parsed.ContainerPath,
Phases: []string{PhaseDeploy, PhaseRun},
ProcessType: DefaultProcessType,
}
switch parsed.VolumeOptions {
case "":
case "ro":
attachment.Readonly = true
default:
attachment.VolumeOptions = parsed.VolumeOptions
}
if err := AddAttachment(appName, attachment); err != nil {
// AddAttachment's duplicate error mentions the entry name, but
// the legacy form historically said "Mount path already
// exists." - preserve that exact wording so existing automation
// and the bats suite keep matching.
if strings.Contains(err.Error(), "is already mounted at") {
return errors.New("Mount path already exists.")
}
return err
}
return nil
}
// unmountLegacyColon is the inverse of mountLegacyColon. The legacy
// mount string identifies an entry+container-path tuple deterministically
// via LegacyMountToEntry, so we can route to RemoveAttachment.
func unmountLegacyColon(appName string, mountPath string) error {
if err := VerifyPaths(mountPath); err != nil {
return err
}
if !CheckIfPathExists(appName, mountPath, MountPhases) {
return errors.New("Mount path does not exist.")
parsed := ParseMountPath(mountPath)
if parsed.ContainerPath == "" {
return errors.New("Storage path must be two valid paths divided by colon.")
}
return dockeroptions.RemoveDockerOptionFromPhases(appName, MountPhases, fmt.Sprintf("-v %s", mountPath))
entry := LegacyMountToEntry(mountPath)
if err := RemoveAttachment(appName, entry.Name, parsed.ContainerPath); err != nil {
// Match the legacy wording for "not currently mounted".
if strings.Contains(err.Error(), "is not mounted") {
return errors.New("Mount path does not exist.")
}
return err
}
return nil
}
// CommandList lists all bind mounts for an app
// CommandList lists all bind mounts for an app. Reads attachments
// directly from the storage plugin's own state rather than going through
// the deprecated `storage-list` plugn trigger.
func CommandList(appName string, format string) error {
if err := common.VerifyAppName(appName); err != nil {
return err
@@ -165,39 +299,45 @@ func CommandList(appName string, format string) error {
return errors.New("Invalid --format value specified")
}
if format == "text" {
common.LogInfo1Quiet(fmt.Sprintf("%s volume bind-mounts:", appName))
}
results, err := common.CallPlugnTrigger(common.PlugnTriggerInput{
Trigger: "storage-list",
Args: []string{appName, "deploy", format},
})
rows, err := ListAppMountEntries(appName, PhaseDeploy)
if err != nil {
return err
}
output := results.StdoutContents()
if format == "text" && output != "" {
lines := strings.Split(strings.TrimSpace(output), "\n")
for _, line := range lines {
if line != "" {
if os.Getenv("DOKKU_QUIET_OUTPUT") != "" {
fmt.Println(line)
} else {
common.LogVerbose(line)
}
}
if format == "json" {
output, err := json.Marshal(rows)
if err != nil {
return err
}
} else if output != "" {
fmt.Println(output)
fmt.Println(string(output))
return nil
}
common.LogInfo1Quiet(fmt.Sprintf("%s volume bind-mounts:", appName))
for _, row := range rows {
line := formatStorageListEntry(row)
if os.Getenv("DOKKU_QUIET_OUTPUT") != "" {
fmt.Println(line)
} else {
common.LogVerbose(line)
}
}
return nil
}
// CommandReport displays a storage report for one or more apps
func CommandReport(appName string, format string, infoFlag string) error {
if appName == "--global" {
// Global storage report: list every registered entry and its
// attachment count; falls back to the legacy per-app loop when
// no entries exist so existing automation keeps working.
reportFormat := format
if reportFormat == "stdout" {
reportFormat = "text"
}
return CommandReportGlobal(reportFormat)
}
if appName == "" {
apps, err := common.DokkuApps()
if err != nil {

View File

@@ -6,12 +6,18 @@ import (
"os"
"path/filepath"
"runtime"
"strings"
"github.com/dokku/dokku/plugins/common"
)
// TriggerInstall sets up the storage plugin on installation
// TriggerInstall sets up the storage plugin on installation and runs
// the bulk legacy-mount migration once per upgrade.
func TriggerInstall() error {
if err := common.PropertySetup(PluginName); err != nil {
return fmt.Errorf("Unable to install the storage plugin: %s", err.Error())
}
storageDir := GetStorageDirectory()
if err := os.MkdirAll(storageDir, 0755); err != nil {
@@ -25,6 +31,29 @@ func TriggerInstall() error {
return fmt.Errorf("Unable to set storage directory permissions: %s", err.Error())
}
if err := EnsureEntriesDirectory(); err != nil {
return err
}
// The install trigger runs as root; the directories it just made are
// root-owned, but the dokku user is the one that runs storage:create
// and writes entry / migration-flag files into them.
for _, dir := range []string{RegistryDirectory(), EntriesDirectory(), migrationFlagDir()} {
if err := os.MkdirAll(dir, 0755); err != nil {
return fmt.Errorf("Unable to create %s: %s", dir, err.Error())
}
if err := common.SetPermissions(common.SetPermissionInput{
Filename: dir,
Mode: 0755,
}); err != nil {
return fmt.Errorf("Unable to set permissions on %s: %s", dir, err.Error())
}
}
if err := MigrateLegacyMounts(); err != nil {
return fmt.Errorf("storage migration failed: %w", err)
}
distro := detectDistro()
if distro == "" {
return nil
@@ -44,6 +73,52 @@ func TriggerInstall() error {
return nil
}
// TriggerPostDelete removes the attachment store for an app that's being
// destroyed. Entries are global and survive app deletion. The whole
// per-app property folder goes away rather than being rewritten as
// empty, matching what other plugins do on app deletion.
func TriggerPostDelete(appName string) error {
if appName == "" {
return nil
}
return common.PropertyDestroy(PluginName, appName)
}
// TriggerPostAppCloneSetup copies attachments from the source app to the
// cloned app. Entries are global so no entry-level work is needed.
func TriggerPostAppCloneSetup(oldName string, newName string) error {
if oldName == "" || newName == "" {
return nil
}
if !common.PropertyExists(PluginName, oldName, AttachmentsProperty) {
return nil
}
attachments, err := LoadAttachments(oldName)
if err != nil {
return err
}
return SaveAttachments(newName, attachments)
}
// TriggerPostAppRenameSetup moves attachments from the old name to the
// new name and removes the old per-app property folder.
func TriggerPostAppRenameSetup(oldName string, newName string) error {
if oldName == "" || newName == "" {
return nil
}
if !common.PropertyExists(PluginName, oldName, AttachmentsProperty) {
return nil
}
attachments, err := LoadAttachments(oldName)
if err != nil {
return err
}
if err := SaveAttachments(newName, attachments); err != nil {
return err
}
return common.PropertyDestroy(PluginName, oldName)
}
// detectDistro returns the Linux distribution name
func detectDistro() string {
if runtime.GOOS != "linux" {
@@ -65,29 +140,114 @@ func detectDistro() string {
return ""
}
// TriggerStorageList outputs storage mounts for an app
// TriggerStorageList outputs storage mounts for an app.
//
// Deprecated: the storage-list plugn trigger is retained for back-compat
// with external plugins that may still call it, but in-process callers
// should use storage.ListAppMountEntries directly. A deprecation warning
// is emitted on every invocation.
func TriggerStorageList(appName string, phase string, format string) error {
mounts, err := GetBindMounts(appName, phase)
common.LogWarn("the storage-list plugn trigger is deprecated; use the storage-app-mounts trigger or the storage Go package directly")
rows, err := ListAppMountEntries(appName, phase)
if err != nil {
return err
}
if format == "json" {
entries := []StorageListEntry{}
for _, mount := range mounts {
entries = append(entries, ParseMountPath(mount))
}
output, err := json.Marshal(entries)
output, err := json.Marshal(rows)
if err != nil {
return err
}
fmt.Println(string(output))
} else {
for _, mount := range mounts {
fmt.Println(mount)
}
return nil
}
for _, row := range rows {
fmt.Println(formatStorageListEntry(row))
}
return nil
}
// AppMountPair pairs an Attachment with the Entry it references; consumed
// by scheduler plugins via the storage-app-mounts trigger.
type AppMountPair struct {
Entry *Entry `json:"entry"`
Attachment *Attachment `json:"attachment"`
}
// TriggerStorageAppMounts emits the (entry, attachment) pairs an app has
// for the given phase, in JSON. Schedulers consume this at deploy time.
func TriggerStorageAppMounts(appName string, phase string) error {
if phase == "" {
phase = PhaseDeploy
}
attachments, err := AttachmentsForPhase(appName, phase)
if err != nil {
return err
}
pairs := []AppMountPair{}
for _, attachment := range attachments {
entry, err := LoadEntry(attachment.EntryName)
if err != nil {
return fmt.Errorf("attachment on %q references missing entry %q: %w", appName, attachment.EntryName, err)
}
pairs = append(pairs, AppMountPair{Entry: entry, Attachment: attachment})
}
output, err := json.Marshal(pairs)
if err != nil {
return err
}
fmt.Println(string(output))
return nil
}
// TriggerDockerArgs emits `-v` flags for each docker-local attachment in
// the requested phase. Plugn concatenates this with docker-options'
// equivalent trigger output, so docker-local apps continue to receive
// their bind mounts through the standard pipeline.
func TriggerDockerArgs(appName string, phase string) error {
attachments, err := AttachmentsForPhase(appName, phase)
if err != nil {
return err
}
for _, attachment := range attachments {
entry, err := LoadEntry(attachment.EntryName)
if err != nil {
return fmt.Errorf("attachment on %q references missing entry %q: %w", appName, attachment.EntryName, err)
}
if entry.Scheduler != SchedulerDockerLocal {
continue
}
flag := buildDockerVFlag(entry, attachment)
if flag == "" {
continue
}
fmt.Printf(" %s", flag)
}
return nil
}
// buildDockerVFlag formats the Docker -v argument for a docker-local
// attachment.
func buildDockerVFlag(entry *Entry, attachment *Attachment) string {
host := entry.HostPath
container := attachment.ContainerPath
if host == "" || container == "" {
return ""
}
options := []string{}
if attachment.Readonly {
options = append(options, "ro")
}
if attachment.VolumeOptions != "" {
options = append(options, attachment.VolumeOptions)
}
flag := fmt.Sprintf("-v %s:%s", host, container)
if len(options) > 0 {
flag = fmt.Sprintf("%s:%s", flag, strings.Join(options, ","))
}
return flag
}

View File

@@ -183,3 +183,217 @@ teardown() {
assert_success
assert_output "mount_volume:/mount"
}
@test "(storage) storage:create / storage:list-entries / storage:destroy" {
run /bin/bash -c "dokku storage:create rdmtest-entry"
echo "output: $output"
echo "status: $status"
assert_success
run /bin/bash -c "dokku storage:list-entries --format json | jq -r '.[].name' | grep '^rdmtest-entry$'"
echo "output: $output"
echo "status: $status"
assert_success
assert_output "rdmtest-entry"
run /bin/bash -c "dokku storage:info rdmtest-entry --format json | jq -r '.scheduler'"
echo "output: $output"
echo "status: $status"
assert_success
assert_output "docker-local"
run /bin/bash -c "dokku storage:destroy rdmtest-entry"
echo "output: $output"
echo "status: $status"
assert_success
}
@test "(storage) storage:create rejects invalid names" {
# underscore: rejected
run /bin/bash -c "dokku storage:create rdmtest_invalid"
echo "output: $output"
echo "status: $status"
assert_failure
# uppercase: rejected
run /bin/bash -c "dokku storage:create RdmTest"
echo "output: $output"
echo "status: $status"
assert_failure
# 46 chars: too long
long_name=$(printf 'a%.0s' {1..46})
run /bin/bash -c "dokku storage:create $long_name"
echo "output: $output"
echo "status: $status"
assert_failure
# legacy- prefix: reserved
run /bin/bash -c "dokku storage:create legacy-foo"
echo "output: $output"
echo "status: $status"
assert_failure
}
@test "(storage) storage:create + storage:mount with named entry attaches multiple entries to one app" {
run /bin/bash -c "dokku storage:create rdmtest-data"
assert_success
run /bin/bash -c "dokku storage:create rdmtest-cache"
assert_success
run /bin/bash -c "dokku storage:mount $TEST_APP rdmtest-data --container-dir /data"
echo "output: $output"
echo "status: $status"
assert_success
run /bin/bash -c "dokku storage:mount $TEST_APP rdmtest-cache --container-dir /cache"
echo "output: $output"
echo "status: $status"
assert_success
run /bin/bash -c "dokku storage:list $TEST_APP --format json | jq -r '. | length'"
echo "output: $output"
echo "status: $status"
assert_success
assert_output "2"
run /bin/bash -c "dokku storage:list $TEST_APP --format json | jq -r '.[].entry_name' | sort | xargs"
echo "output: $output"
echo "status: $status"
assert_success
assert_output "rdmtest-cache rdmtest-data"
# cleanup
run /bin/bash -c "dokku storage:unmount $TEST_APP rdmtest-data"
assert_success
run /bin/bash -c "dokku storage:unmount $TEST_APP rdmtest-cache"
assert_success
run /bin/bash -c "dokku storage:list $TEST_APP --format json | jq -r '. | length'"
echo "output: $output"
echo "status: $status"
assert_success
assert_output "0"
run /bin/bash -c "dokku storage:destroy rdmtest-data"
assert_success
run /bin/bash -c "dokku storage:destroy rdmtest-cache"
assert_success
}
@test "(storage) storage:destroy refuses to remove a still-mounted entry" {
run /bin/bash -c "dokku storage:create rdmtest-busy"
assert_success
run /bin/bash -c "dokku storage:mount $TEST_APP rdmtest-busy --container-dir /data"
assert_success
run /bin/bash -c "dokku storage:destroy rdmtest-busy"
echo "output: $output"
echo "status: $status"
assert_failure
assert_output_contains "still mounted"
run /bin/bash -c "dokku storage:unmount $TEST_APP rdmtest-busy"
assert_success
run /bin/bash -c "dokku storage:destroy rdmtest-busy"
assert_success
}
@test "(storage) storage:exec runs a non-interactive command and propagates exit code" {
run /bin/bash -c "dokku storage:create rdmtest-exec"
assert_success
# docker pull image up-front so the test isn't gated on registry latency.
run /bin/bash -c "docker pull alpine:3 >/dev/null"
assert_success
run /bin/bash -c "dokku storage:exec rdmtest-exec -- /bin/sh -c 'touch /data/marker && ls /data/marker'"
echo "output: $output"
echo "status: $status"
assert_success
assert_output_contains "/data/marker"
run /bin/bash -c "dokku storage:exec rdmtest-exec -- /bin/sh -c 'exit 42'"
echo "output: $output"
echo "status: $status"
assert_equal "$status" 42
run /bin/bash -c "dokku storage:destroy rdmtest-exec"
assert_success
}
@test "(storage) storage:migrate converts legacy docker-options -v lines into attachments" {
# Stage a legacy `-v` line directly via docker-options:add. This
# bypasses the new storage:mount path so the line lives in
# docker-options without a corresponding attachment, simulating an
# app upgraded from a pre-PR Dokku version.
run /bin/bash -c "dokku docker-options:add $TEST_APP deploy,run \"-v /tmp/legacy-mount:/legacy\""
echo "output: $output"
echo "status: $status"
assert_success
# Before migration, storage:list shows nothing (attachment-only) and
# docker-options has the -v line.
run /bin/bash -c "dokku storage:list $TEST_APP --format json | jq -r '. | length'"
assert_success
assert_output "0"
run /bin/bash -c "dokku docker-options:report $TEST_APP --docker-options-deploy"
assert_success
assert_output_contains "-v /tmp/legacy-mount:/legacy"
# Run the migration for this app.
run /bin/bash -c "dokku storage:migrate $TEST_APP"
echo "output: $output"
echo "status: $status"
assert_success
# storage:list now surfaces the synthesized colon form.
run /bin/bash -c "dokku --quiet storage:list $TEST_APP"
assert_success
assert_output "/tmp/legacy-mount:/legacy"
# The synthesized entry shows up under the legacy- prefix.
run /bin/bash -c "dokku storage:list-entries --format json | jq -r '.[].name' | grep '^legacy-' | head -1"
echo "output: $output"
echo "status: $status"
assert_success
# docker-options no longer holds the -v line on either phase.
run /bin/bash -c "dokku docker-options:report $TEST_APP --docker-options-deploy"
assert_success
assert_output_not_contains "-v /tmp/legacy-mount:/legacy"
run /bin/bash -c "dokku docker-options:report $TEST_APP --docker-options-run"
assert_success
assert_output_not_contains "-v /tmp/legacy-mount:/legacy"
# Idempotency: re-running storage:migrate is a no-op (still one
# attachment, no errors).
run /bin/bash -c "dokku storage:migrate $TEST_APP"
echo "output: $output"
echo "status: $status"
assert_success
run /bin/bash -c "dokku storage:list $TEST_APP --format json | jq -r '. | length'"
assert_success
assert_output "1"
# Cleanup: unmount + destroy the synthesized legacy entry. We need
# the entry name from list-entries since legacy-<hash> is content-derived.
legacy_name=$(dokku storage:list-entries --format json | jq -r '.[] | select(.name | startswith("legacy-")) | .name' | head -1)
if [[ -n "$legacy_name" ]]; then
run /bin/bash -c "dokku storage:unmount $TEST_APP $legacy_name --container-dir /legacy"
assert_success
run /bin/bash -c "dokku storage:destroy $legacy_name"
assert_success
fi
}
@test "(storage) storage:ensure-directory emits deprecation warning" {
run /bin/bash -c "dokku storage:ensure-directory $TEST_APP 2>&1"
echo "output: $output"
echo "status: $status"
assert_success
assert_output_contains "deprecated"
}