Merge pull request #8538 from dokku/6814-k3s-plugin-add-pv-option
Add scheduler-aware named storage entries
This commit is contained in:
@@ -1,24 +1,37 @@
|
||||
# Persistent Storage
|
||||
|
||||
> [!IMPORTANT]
|
||||
> New as of 0.5.0
|
||||
> New as of 0.5.0. Named storage entries new as of 0.38.0.
|
||||
|
||||
The preferred method to mount external containers to a Dokku managed container, is to use the Dokku storage plugin.
|
||||
The preferred method to attach persistent storage to a Dokku-managed container is the Dokku storage plugin.
|
||||
|
||||
```
|
||||
storage:ensure-directory [--chown option] <directory> # Creates a persistent storage directory in the recommended storage path
|
||||
storage:list <app> [--format text|json] # List bind mounts for app's container(s) (host:container)
|
||||
storage:mount <app> <host-dir:container-dir> # Create a new bind mount
|
||||
storage:report [<app>] [<flag>] # Displays a checks report for one or more apps
|
||||
storage:unmount <app> <host-dir:container-dir> # Remove an existing bind mount
|
||||
storage:create <name> [<path>] [flags] # Register a named storage entry
|
||||
storage:destroy <name> # Remove a named storage entry (must be unmounted from every app first)
|
||||
storage:ensure-directory [--chown option] <directory> # [DEPRECATED] use storage:create instead
|
||||
storage:exec <name> [-- <cmd>...] # Run a command (or shell) in a temporary container that mounts the entry
|
||||
storage:info <name> [--format text|json] # Show details for one storage entry
|
||||
storage:list <app> [--format text|json] # List bind mounts for an app's container(s) (legacy host:container view)
|
||||
storage:list-entries [--scheduler s] [--format text|json] # List registered storage entries
|
||||
storage:mount <app> <name> --container-dir <path> [flags] # Mount a named entry into an app
|
||||
storage:mount <app> <host-dir:container-dir> # [LEGACY] colon-form mount, docker-local only
|
||||
storage:report [<app>] [<flag>] # Display a storage report for one or more apps
|
||||
storage:report --global # Display a cluster-wide entry inventory
|
||||
storage:set <name> [flags] # Update a storage entry in place
|
||||
storage:unmount <app> <name> [--container-dir <path>] # Remove an attachment
|
||||
storage:wait <name> # Block until a k3s entry's PVC is bound
|
||||
```
|
||||
|
||||
> The storage plugin is compatible with storage mounts created with the docker-options. The storage plugin will only list mounts from the deploy/run phase.
|
||||
A storage entry is the source of truth for the underlying volume - a host directory on docker-local, or a PersistentVolumeClaim on k3s. Multiple apps can mount the same entry, and an attachment carries the per-app details (container path, phases, subpath, readonly, process type). Names are globally unique across the install and must be DNS-1123 labels (lowercase letters, digits, dashes) of 45 characters or less so they can be used verbatim as Helm release and PVC names.
|
||||
|
||||
The legacy `storage:mount <app> <host>:<container>` form continues to work on docker-local. On a k3s app it is rejected; create a named entry with `storage:create --scheduler k3s` and mount it instead. Existing colon-form mounts are migrated automatically the first time the new code runs - they show up as `legacy-<hash>` entries in `storage:list-entries`.
|
||||
|
||||
The storage plugin supports the following mount points:
|
||||
|
||||
- explicit paths that exist on the host
|
||||
- docker volumes
|
||||
- explicit paths that exist on the host (docker-local)
|
||||
- docker volumes (docker-local)
|
||||
- PersistentVolumeClaims provisioned via a StorageClass (k3s)
|
||||
- hostPath-backed PVs (k3s)
|
||||
|
||||
## Usage
|
||||
|
||||
|
||||
@@ -22,6 +22,7 @@
|
||||
- The `docker-local` scheduler no longer queues an image for retirement when another running container of the same app still uses it. This fixes the case where a `ps:rebuild` against an image-based deploy (`git:from-image`) produced an identical-SHA image and the `dokku-retire` cron timer would log `Image ... has running containers, skipping rm` on every run. Stuck entries from prior versions are pruned automatically on the next `ps:retire` run.
|
||||
- All `:report` subcommands now accept the `--global` flag, which scopes the report to globally-configured properties. The flag composes with `--format json`, so a JSON report of global properties can be obtained via, for example, `dokku scheduler:report --global --format json`. Previously, combining `--global` with `--format json` was rejected with an "info flag" error, and `--global` on its own was treated as an unknown flag.
|
||||
- The `scheduler-k3s` plugin now manages env config and the dokku-generated image pull Secret as their own helm releases with stable names (`config-{app}` and `pull-secret-{app}`) rather than bundling them into the app helm chart with a per-deploy timestamp suffix (`env-{app}.{ts}` / `ims-{app}.{ts}`). This fixes two bugs: a helm rollback of the app chart no longer deletes Secrets that older ReplicaSets still reference, and the Deployment's `imagePullSecrets` list no longer accumulates references to nonexistent Secrets across deploys. The next deploy of an app switches the Deployment's `envFrom` and `imagePullSecrets` references to the stable names and prunes any leaked entries; existing live Deployments do not need to be patched manually. App rename now also uninstalls the old `tls-{app}`, `config-{app}`, and `pull-secret-{app}` releases under the previous app name; the new name's releases are recreated on the next deploy or certs sync.
|
||||
- The storage plugin now treats persistent volumes as named, scheduler-aware first-class resources via `storage:create`, `storage:mount`, `storage:set`, and `storage:destroy`. The legacy `storage:mount <app> <host>:<container>` colon form continues to work on docker-local apps but is deprecated; on k3s apps it is rejected. Existing colon-form mounts are migrated automatically the first time the new storage plugin runs (during the install trigger) - they appear as `legacy-<hash>` entries in `storage:list-entries`. The migration is idempotent and tied to a per-app flag file at `$DOKKU_LIB_ROOT/config/storage/.migrated/<app>`; deleting that file forces a re-scan on the next install. The `storage:ensure-directory` command keeps working but now emits a deprecation warning - prefer `storage:create <name> [<path>]` (the path defaults to the same `$DOKKU_LIB_ROOT/data/storage/<name>` location). Storage entry names must now be DNS-1123 labels of 45 characters or less so they can be used verbatim as Helm release and Kubernetes resource names; underscores and uppercase characters that the older `ensure-directory` validator accepted are rejected for new names. The migration synthesizer always uses lowercase hex hashes so existing data is never locked out.
|
||||
|
||||
### TLS handshake behavior change
|
||||
|
||||
|
||||
@@ -685,6 +685,27 @@ Alternatively, a comma separated list of chart names can be specified to only fo
|
||||
dokku scheduler-k3s:ensure-charts --charts cert-manager
|
||||
```
|
||||
|
||||
### Persistent storage
|
||||
|
||||
K3s apps mount persistent volumes through the Dokku storage plugin. A `storage:create --scheduler k3s` call provisions a PersistentVolumeClaim through a small per-entry helm release; `storage:mount <app> <name> --container-dir <path>` then attaches it to an app. Because storage entries are global, the same PVC can be mounted into multiple apps (for `ReadWriteMany` workloads) or into the same app at multiple subpaths.
|
||||
|
||||
```shell
|
||||
# dynamically-provisioned PVC backed by Longhorn
|
||||
dokku storage:create demo-data \
|
||||
--scheduler k3s \
|
||||
--size 2Gi \
|
||||
--access-mode ReadWriteOnce \
|
||||
--storage-class-name longhorn
|
||||
|
||||
dokku storage:mount demo demo-data --container-dir /data
|
||||
dokku storage:wait demo-data
|
||||
git push dokku master
|
||||
```
|
||||
|
||||
For a hostPath-backed PV (no StorageClass), pass `<path>` as the second positional argument and omit `--storage-class-name`. The plugin renders both the PV and the PVC into the entry's helm release. The `--reclaim-policy` flag (`Retain` or `Delete`) controls whether the underlying PV survives `storage:destroy`. Annotations and labels on `storage:create` / `storage:set` propagate to both the PVC and the PV so backup tools (Velero, Longhorn snapshots) can find them.
|
||||
|
||||
The legacy `storage:mount <app> <host>:<container>` colon form is rejected on k3s apps; create a named entry instead. See [Persistent Storage](/docs/advanced-usage/persistent-storage.md) for the full command reference.
|
||||
|
||||
### Chart upgrade callbacks
|
||||
|
||||
Some chart version transitions require side-effects that a plain `helm upgrade` cannot perform. For example, the upgrade of `keda-add-ons-http` to `0.12.2` introduces breaking changes to deployment selectors, which are immutable in Kubernetes; the chart-managed deployments must be deleted before the upgrade can proceed.
|
||||
|
||||
@@ -2904,21 +2904,52 @@ DOKKU_SCHEDULER="$1"; APP="$2"; REMOVE_CONTAINERS="$3";
|
||||
# TODO
|
||||
```
|
||||
|
||||
### `storage-list`
|
||||
### `storage-list` (deprecated)
|
||||
|
||||
- Description: Returns a list of storage mounts
|
||||
- Invoked by: `dokku storage:list` and `dokku deploy`
|
||||
> [!WARNING]
|
||||
> Deprecated as of 0.38.0. The trigger handler still functions for back-compat
|
||||
> with external plugins but emits a deprecation warning on every invocation.
|
||||
> In-process callers should use the `storage-app-mounts` trigger instead, or
|
||||
> the `storage` Go package directly.
|
||||
|
||||
- Description: Returns a list of storage mounts for an app
|
||||
- Invoked by: external plugins (the core `storage:list` subcommand no longer uses it)
|
||||
- Arguments: `$APP $PHASE $FORMAT`
|
||||
- Example:
|
||||
|
||||
```shell
|
||||
#!/usr/bin/env bash
|
||||
### `storage-app-mounts`
|
||||
|
||||
set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x
|
||||
APP="$1"
|
||||
- Description: Returns the (entry, attachment) pairs an app has for the given phase, in JSON. Schedulers consume this to render volume mounts.
|
||||
- Invoked by: scheduler plugins during deploy
|
||||
- Arguments: `$APP $PHASE`
|
||||
- Output: a JSON array of objects shaped `{"entry": Entry, "attachment": Attachment}`
|
||||
|
||||
# TODO
|
||||
```
|
||||
### `storage-create`
|
||||
|
||||
- Description: Provisions or updates the underlying volume for a k3s storage entry. Called when `storage:create` or `storage:set` runs against a k3s entry. Receives the JSON-encoded entry on stdin.
|
||||
- Invoked by: `dokku storage:create`, `dokku storage:set`
|
||||
- Arguments: `$ENTRY_NAME`
|
||||
- Stdin: JSON-encoded `Entry` payload
|
||||
|
||||
### `storage-destroy`
|
||||
|
||||
- Description: Releases the underlying volume for a k3s storage entry. Called when `storage:destroy` runs against a k3s entry. Receives the JSON-encoded entry on stdin.
|
||||
- Invoked by: `dokku storage:destroy`
|
||||
- Arguments: `$ENTRY_NAME`
|
||||
- Stdin: JSON-encoded `Entry` payload
|
||||
|
||||
### `storage-status`
|
||||
|
||||
- Description: Returns the current status of a storage entry's underlying volume. Used by `storage:wait` to poll the cluster. The handler should print one line: `Bound`, `Pending`, or `Lost` for k3s entries.
|
||||
- Invoked by: `dokku storage:wait`
|
||||
- Arguments: `$ENTRY_NAME`
|
||||
- Stdin: JSON-encoded `Entry` payload
|
||||
|
||||
### `scheduler-storage-exec`
|
||||
|
||||
- Description: Runs an interactive or non-interactive command against a storage entry. The storage plugin fires this with `<scheduler>` as the first arg; each scheduler plugin's handler matches against its own scheduler name and either handles or no-ops, mirroring `scheduler-deploy` / `scheduler-app-status`. Plugn forwards stdin/stdout/stderr to the handler subprocess so an interactive shell streams cleanly. The handler exits with the underlying tool's status code so `dokku storage:exec` propagates exit codes verbatim.
|
||||
- Invoked by: `dokku storage:exec`
|
||||
- Arguments: `$SCHEDULER $ENTRY_NAME $IMAGE [-- $cmd...]`
|
||||
- Flags: `--interactive` (stdin is open), `--tty` (stdin is a terminal), `--as-user <uid>` (override `entry.Chown`).
|
||||
|
||||
### `traefik-template-source`
|
||||
|
||||
|
||||
1
go.work
1
go.work
@@ -21,4 +21,5 @@ use (
|
||||
./plugins/scheduler
|
||||
./plugins/scheduler-docker-local
|
||||
./plugins/scheduler-k3s
|
||||
./plugins/storage
|
||||
)
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
TRIGGERS = triggers/scheduler-cron-write
|
||||
TRIGGERS = triggers/scheduler-cron-write triggers/scheduler-storage-exec
|
||||
BUILD = triggers
|
||||
PLUGIN_NAME = scheduler-docker-local
|
||||
|
||||
|
||||
@@ -5,13 +5,16 @@ go 1.26.2
|
||||
require (
|
||||
github.com/dokku/dokku/plugins/common v0.0.0-00010101000000-000000000000
|
||||
github.com/dokku/dokku/plugins/cron v0.0.0-00010101000000-000000000000
|
||||
github.com/dokku/dokku/plugins/storage v0.0.0-00010101000000-000000000000
|
||||
github.com/multiformats/go-base36 v0.2.0
|
||||
github.com/spf13/pflag v1.0.10
|
||||
golang.org/x/sync v0.20.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/alexellis/go-execute/v2 v2.2.1 // indirect
|
||||
github.com/dokku/dokku/plugins/app-json v0.0.0-00010101000000-000000000000 // indirect
|
||||
github.com/dokku/dokku/plugins/docker-options v0.0.0-00010101000000-000000000000 // indirect
|
||||
github.com/fatih/color v1.19.0 // indirect
|
||||
github.com/hashicorp/errwrap v1.0.0 // indirect
|
||||
github.com/hashicorp/go-multierror v1.1.1 // indirect
|
||||
@@ -38,3 +41,7 @@ replace github.com/dokku/dokku/plugins/app-json => ../app-json
|
||||
replace github.com/dokku/dokku/plugins/common => ../common
|
||||
|
||||
replace github.com/dokku/dokku/plugins/cron => ../cron
|
||||
|
||||
replace github.com/dokku/dokku/plugins/storage => ../storage
|
||||
|
||||
replace github.com/dokku/dokku/plugins/docker-options => ../docker-options
|
||||
|
||||
@@ -46,6 +46,8 @@ github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0t
|
||||
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
|
||||
github.com/ryanuber/columnize v2.1.2+incompatible h1:C89EOx/XBWwIXl8wm8OPJBd7kPF25UfsK2X7Ph/zCAk=
|
||||
github.com/ryanuber/columnize v2.1.2+incompatible/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=
|
||||
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
|
||||
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
||||
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
|
||||
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||
github.com/tailscale/hujson v0.0.0-20250605163823-992244df8c5a h1:a6TNDN9CgG+cYjaeN8l2mc4kSz2iMiCDQxPEyltUV/I=
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
flag "github.com/spf13/pflag"
|
||||
|
||||
"github.com/dokku/dokku/plugins/common"
|
||||
schedulerdockerlocal "github.com/dokku/dokku/plugins/scheduler-docker-local"
|
||||
)
|
||||
@@ -14,6 +15,10 @@ import (
|
||||
func main() {
|
||||
parts := strings.Split(os.Args[0], "/")
|
||||
trigger := parts[len(parts)-1]
|
||||
|
||||
interactive := flag.Bool("interactive", false, "--interactive: stdin is open")
|
||||
tty := flag.Bool("tty", false, "--tty: stdin is a terminal")
|
||||
asUser := flag.String("as-user", "", "--as-user: numeric uid override")
|
||||
flag.Parse()
|
||||
|
||||
var err error
|
||||
@@ -21,6 +26,27 @@ func main() {
|
||||
case "scheduler-cron-write":
|
||||
scheduler := flag.Arg(0)
|
||||
err = schedulerdockerlocal.TriggerSchedulerCronWrite(scheduler)
|
||||
case "scheduler-storage-exec":
|
||||
args := flag.Args()
|
||||
if len(args) < 3 {
|
||||
err = fmt.Errorf("scheduler-storage-exec requires <scheduler> <entry-name> <image>")
|
||||
break
|
||||
}
|
||||
scheduler := args[0]
|
||||
entryName := args[1]
|
||||
image := args[2]
|
||||
var cmd []string
|
||||
if len(args) > 3 {
|
||||
cmd = args[3:]
|
||||
}
|
||||
err = schedulerdockerlocal.TriggerSchedulerStorageExec(scheduler, schedulerdockerlocal.StorageExecInput{
|
||||
EntryName: entryName,
|
||||
Image: image,
|
||||
Interactive: *interactive,
|
||||
Tty: *tty,
|
||||
AsUser: *asUser,
|
||||
Command: cmd,
|
||||
})
|
||||
default:
|
||||
err = fmt.Errorf("Invalid plugin trigger call: %s", trigger)
|
||||
}
|
||||
|
||||
164
plugins/scheduler-docker-local/storage_exec.go
Normal file
164
plugins/scheduler-docker-local/storage_exec.go
Normal file
@@ -0,0 +1,164 @@
|
||||
package schedulerdockerlocal
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"github.com/dokku/dokku/plugins/common"
|
||||
"github.com/dokku/dokku/plugins/storage"
|
||||
)
|
||||
|
||||
// dockerNamedVolumeRegexp mirrors storage.dockerNamedVolumeRegexp and is
|
||||
// used to decide whether a HostPath should be statted (filesystem) or
|
||||
// looked up via `docker volume inspect`.
|
||||
var dockerNamedVolumeRegexp = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9_.-]+$`)
|
||||
|
||||
// StorageExecInput captures the inputs forwarded by the storage plugin
|
||||
// over the scheduler-storage-exec trigger.
|
||||
type StorageExecInput struct {
|
||||
EntryName string
|
||||
Image string
|
||||
Interactive bool
|
||||
Tty bool
|
||||
AsUser string
|
||||
Command []string
|
||||
}
|
||||
|
||||
// TriggerSchedulerStorageExec runs an interactive or non-interactive
|
||||
// command in a throwaway docker container that mounts the storage
|
||||
// entry's host path or named volume at /data. Returns the exit code of
|
||||
// docker run via os.Exit so the caller's status mirrors the underlying
|
||||
// tool.
|
||||
func TriggerSchedulerStorageExec(scheduler string, input StorageExecInput) error {
|
||||
if scheduler != "docker-local" {
|
||||
// Not for us; let other handlers respond.
|
||||
return nil
|
||||
}
|
||||
|
||||
if !storage.EntryExists(input.EntryName) {
|
||||
return fmt.Errorf("storage entry %q does not exist", input.EntryName)
|
||||
}
|
||||
entry, err := storage.LoadEntry(input.EntryName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if entry.Scheduler != storage.SchedulerDockerLocal {
|
||||
return fmt.Errorf("storage entry %q has scheduler %q, not docker-local", entry.Name, entry.Scheduler)
|
||||
}
|
||||
|
||||
if err := preflightDockerLocalSource(entry.HostPath); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
args, err := buildDockerExecArgs(entry, input)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
result, err := common.CallExecCommand(common.ExecCommandInput{
|
||||
Command: common.DockerBin(),
|
||||
Args: args,
|
||||
StreamStdio: true,
|
||||
})
|
||||
// CallExecCommand wraps non-zero exit as an error; for storage:exec
|
||||
// the docker-run exit code is the signal we want to forward to the
|
||||
// caller. Propagate it before falling through to the err return,
|
||||
// which would otherwise be collapsed to exit 1 by the dispatcher.
|
||||
if result.ExitCode != 0 {
|
||||
os.Exit(result.ExitCode)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// preflightDockerLocalSource fails fast when the host path or named
|
||||
// volume backing the entry doesn't exist, so the user gets a real error
|
||||
// instead of a generic "docker: ..." line.
|
||||
func preflightDockerLocalSource(hostPath string) error {
|
||||
if hostPath == "" {
|
||||
return errors.New("storage entry is missing a host path")
|
||||
}
|
||||
if filepath.IsAbs(hostPath) {
|
||||
info, err := os.Stat(hostPath)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return fmt.Errorf("host path %s does not exist", hostPath)
|
||||
}
|
||||
return fmt.Errorf("unable to stat host path %s: %w", hostPath, err)
|
||||
}
|
||||
if !info.IsDir() {
|
||||
return fmt.Errorf("host path %s exists but is not a directory", hostPath)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if !dockerNamedVolumeRegexp.MatchString(hostPath) {
|
||||
return fmt.Errorf("host path %q is neither absolute nor a valid docker volume token", hostPath)
|
||||
}
|
||||
result, err := common.CallExecCommand(common.ExecCommandInput{
|
||||
Command: common.DockerBin(),
|
||||
Args: []string{"volume", "inspect", hostPath},
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if result.ExitCode != 0 {
|
||||
return fmt.Errorf("docker named volume %q is not present: %s", hostPath, strings.TrimSpace(result.StderrContents()))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// buildDockerExecArgs assembles the `docker run` argv for a storage:exec
|
||||
// invocation. Split out so the unit test can exercise it without
|
||||
// actually invoking docker.
|
||||
func buildDockerExecArgs(entry *storage.Entry, input StorageExecInput) ([]string, error) {
|
||||
cmd := input.Command
|
||||
if len(cmd) == 0 {
|
||||
cmd = []string{"sh", "-c", "command -v bash >/dev/null 2>&1 && exec bash || exec sh"}
|
||||
}
|
||||
|
||||
args := []string{"run", "--rm"}
|
||||
if input.Tty {
|
||||
args = append(args, "-it")
|
||||
} else if input.Interactive {
|
||||
args = append(args, "-i")
|
||||
}
|
||||
|
||||
user, err := resolveUser(entry.Chown, input.AsUser)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if user != "" {
|
||||
args = append(args, "--user", user)
|
||||
}
|
||||
|
||||
args = append(args, "-v", fmt.Sprintf("%s:/data", entry.HostPath))
|
||||
args = append(args, input.Image)
|
||||
args = append(args, cmd...)
|
||||
return args, nil
|
||||
}
|
||||
|
||||
// resolveUser turns the entry's Chown setting plus an optional --as-user
|
||||
// override into a docker --user value. Returns an empty string when the
|
||||
// container should run as the image's default user.
|
||||
func resolveUser(chown string, asUser string) (string, error) {
|
||||
if asUser != "" {
|
||||
return fmt.Sprintf("%s:%s", asUser, asUser), nil
|
||||
}
|
||||
if chown == "" || chown == "false" {
|
||||
return "", nil
|
||||
}
|
||||
uid, err := storage.ResolveChownID(chown)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if uid == "" || uid == "false" {
|
||||
return "", nil
|
||||
}
|
||||
return fmt.Sprintf("%s:%s", uid, uid), nil
|
||||
}
|
||||
118
plugins/scheduler-docker-local/storage_exec_test.go
Normal file
118
plugins/scheduler-docker-local/storage_exec_test.go
Normal file
@@ -0,0 +1,118 @@
|
||||
package schedulerdockerlocal
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/dokku/dokku/plugins/storage"
|
||||
)
|
||||
|
||||
func TestBuildDockerExecArgsTTY(t *testing.T) {
|
||||
entry := &storage.Entry{
|
||||
Name: "demo-data",
|
||||
Scheduler: storage.SchedulerDockerLocal,
|
||||
HostPath: "/var/lib/dokku/data/storage/demo-data",
|
||||
}
|
||||
|
||||
cases := []struct {
|
||||
name string
|
||||
interactive bool
|
||||
tty bool
|
||||
want string
|
||||
}{
|
||||
{"non-interactive", false, false, ""},
|
||||
{"piped stdin", true, false, "-i"},
|
||||
{"interactive tty", true, true, "-it"},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
args, err := buildDockerExecArgs(entry, StorageExecInput{
|
||||
Image: "alpine:3",
|
||||
Interactive: tc.interactive,
|
||||
Tty: tc.tty,
|
||||
Command: []string{"ls", "/data"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("buildDockerExecArgs returned error: %v", err)
|
||||
}
|
||||
joined := strings.Join(args, " ")
|
||||
if tc.want == "" {
|
||||
if strings.Contains(joined, " -i ") || strings.Contains(joined, " -it ") {
|
||||
t.Fatalf("expected no -i / -it flag, got: %s", joined)
|
||||
}
|
||||
} else if !strings.Contains(joined, " "+tc.want+" ") {
|
||||
t.Fatalf("expected to find %q in args, got: %s", tc.want, joined)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildDockerExecArgsUser(t *testing.T) {
|
||||
entry := &storage.Entry{
|
||||
Name: "demo-data",
|
||||
Scheduler: storage.SchedulerDockerLocal,
|
||||
HostPath: "/var/lib/dokku/data/storage/demo-data",
|
||||
Chown: "false",
|
||||
}
|
||||
|
||||
args, err := buildDockerExecArgs(entry, StorageExecInput{
|
||||
Image: "alpine:3",
|
||||
Command: []string{"ls"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("err: %v", err)
|
||||
}
|
||||
if strings.Contains(strings.Join(args, " "), "--user") {
|
||||
t.Fatalf("did not expect --user flag with chown=false, got: %v", args)
|
||||
}
|
||||
|
||||
args, err = buildDockerExecArgs(entry, StorageExecInput{
|
||||
Image: "alpine:3",
|
||||
AsUser: "1234",
|
||||
Command: []string{"ls"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("err: %v", err)
|
||||
}
|
||||
if !strings.Contains(strings.Join(args, " "), "--user 1234:1234") {
|
||||
t.Fatalf("expected --user 1234:1234, got: %v", args)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildDockerExecArgsDefaultsToShellWhenNoCommand(t *testing.T) {
|
||||
entry := &storage.Entry{
|
||||
Name: "demo-data",
|
||||
Scheduler: storage.SchedulerDockerLocal,
|
||||
HostPath: "/var/lib/dokku/data/storage/demo-data",
|
||||
}
|
||||
args, err := buildDockerExecArgs(entry, StorageExecInput{
|
||||
Image: "alpine:3",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("err: %v", err)
|
||||
}
|
||||
last := args[len(args)-3:]
|
||||
if last[0] != "sh" || last[1] != "-c" {
|
||||
t.Fatalf("expected the trailing command to be a shell fallback, got: %v", last)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildDockerExecArgsMountsHostPath(t *testing.T) {
|
||||
entry := &storage.Entry{
|
||||
Name: "demo-data",
|
||||
Scheduler: storage.SchedulerDockerLocal,
|
||||
HostPath: "/srv/demo",
|
||||
}
|
||||
args, err := buildDockerExecArgs(entry, StorageExecInput{
|
||||
Image: "alpine:3",
|
||||
Command: []string{"ls"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("err: %v", err)
|
||||
}
|
||||
joined := strings.Join(args, " ")
|
||||
if !strings.Contains(joined, "-v /srv/demo:/data") {
|
||||
t.Fatalf("expected -v /srv/demo:/data, got: %s", joined)
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
SUBCOMMANDS = subcommands/annotations:set subcommands/autoscaling-auth:set subcommands/autoscaling-auth:report subcommands/cluster:add subcommands/cluster:list subcommands/cluster:remove subcommands/ensure-charts subcommands/initialize subcommands/labels:set subcommands/profiles:add subcommands/profiles:list subcommands/profiles:remove subcommands/report subcommands/set subcommands/show-kubeconfig subcommands/uninstall
|
||||
TRIGGERS = triggers/core-post-deploy triggers/core-post-extract triggers/install triggers/post-app-clone-setup triggers/post-app-rename-setup triggers/post-certs-update triggers/post-certs-remove triggers/post-create triggers/post-delete triggers/report triggers/scheduler-app-status triggers/scheduler-deploy triggers/scheduler-enter triggers/scheduler-is-deployed triggers/scheduler-logs triggers/scheduler-proxy-config triggers/scheduler-proxy-logs triggers/scheduler-post-delete triggers/scheduler-run triggers/scheduler-run-list triggers/scheduler-stop triggers/scheduler-cron-write
|
||||
TRIGGERS = triggers/core-post-deploy triggers/core-post-extract triggers/install triggers/post-app-clone-setup triggers/post-app-rename-setup triggers/post-certs-update triggers/post-certs-remove triggers/post-create triggers/post-delete triggers/report triggers/scheduler-app-status triggers/scheduler-deploy triggers/scheduler-enter triggers/scheduler-is-deployed triggers/scheduler-logs triggers/scheduler-proxy-config triggers/scheduler-proxy-logs triggers/scheduler-post-delete triggers/scheduler-run triggers/scheduler-run-list triggers/scheduler-stop triggers/scheduler-cron-write triggers/storage-create triggers/storage-destroy triggers/storage-status triggers/scheduler-storage-exec
|
||||
BUILD = commands subcommands triggers
|
||||
PLUGIN_NAME = scheduler-k3s
|
||||
|
||||
|
||||
@@ -13,6 +13,7 @@ require (
|
||||
github.com/dokku/dokku/plugins/logs v0.0.0-00010101000000-000000000000
|
||||
github.com/dokku/dokku/plugins/nginx-vhosts v0.0.0-00010101000000-000000000000
|
||||
github.com/dokku/dokku/plugins/registry v0.0.0-00010101000000-000000000000
|
||||
github.com/dokku/dokku/plugins/storage v0.0.0-00010101000000-000000000000
|
||||
github.com/fatih/color v1.19.0
|
||||
github.com/fluxcd/pkg/kustomize v1.31.0
|
||||
github.com/go-openapi/jsonpointer v0.23.1
|
||||
@@ -193,6 +194,8 @@ replace github.com/dokku/dokku/plugins/nginx-vhosts => ../nginx-vhosts
|
||||
|
||||
replace github.com/dokku/dokku/plugins/registry => ../registry
|
||||
|
||||
replace github.com/dokku/dokku/plugins/storage => ../storage
|
||||
|
||||
replace github.com/joho/godotenv => github.com/joho/godotenv v1.2.0
|
||||
|
||||
replace sigs.k8s.io/controller-runtime => sigs.k8s.io/controller-runtime v0.22.4
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"strconv"
|
||||
@@ -18,6 +19,9 @@ func main() {
|
||||
parts := strings.Split(os.Args[0], "/")
|
||||
trigger := parts[len(parts)-1]
|
||||
podIdentifier := flag.String("container-id", "", "--container-id: A pod identifier")
|
||||
storageExecInteractive := flag.Bool("interactive", false, "--interactive: stdin is open (storage:exec)")
|
||||
storageExecTty := flag.Bool("tty", false, "--tty: stdin is a terminal (storage:exec)")
|
||||
storageExecAsUser := flag.String("as-user", "", "--as-user: numeric uid override (storage:exec)")
|
||||
flag.Parse()
|
||||
|
||||
var err error
|
||||
@@ -158,6 +162,36 @@ func main() {
|
||||
appName := flag.Arg(1)
|
||||
format := flag.Arg(2)
|
||||
err = scheduler_k3s.TriggerSchedulerRunList(scheduler, appName, format)
|
||||
case "storage-create":
|
||||
entryName := flag.Arg(0)
|
||||
err = scheduler_k3s.TriggerStorageCreate(context.Background(), entryName)
|
||||
case "storage-destroy":
|
||||
entryName := flag.Arg(0)
|
||||
err = scheduler_k3s.TriggerStorageDestroy(context.Background(), entryName)
|
||||
case "storage-status":
|
||||
entryName := flag.Arg(0)
|
||||
err = scheduler_k3s.TriggerStorageStatus(context.Background(), entryName)
|
||||
case "scheduler-storage-exec":
|
||||
positional := flag.Args()
|
||||
if len(positional) < 3 {
|
||||
err = fmt.Errorf("scheduler-storage-exec requires <scheduler> <entry-name> <image>")
|
||||
break
|
||||
}
|
||||
schedulerName := positional[0]
|
||||
entryName := positional[1]
|
||||
image := positional[2]
|
||||
var cmd []string
|
||||
if len(positional) > 3 {
|
||||
cmd = positional[3:]
|
||||
}
|
||||
err = scheduler_k3s.TriggerSchedulerStorageExec(context.Background(), schedulerName, scheduler_k3s.StorageExecInput{
|
||||
EntryName: entryName,
|
||||
Image: image,
|
||||
Interactive: *storageExecInteractive,
|
||||
Tty: *storageExecTty,
|
||||
AsUser: *storageExecAsUser,
|
||||
Command: cmd,
|
||||
})
|
||||
default:
|
||||
err = fmt.Errorf("Invalid plugin trigger call: %s", trigger)
|
||||
}
|
||||
|
||||
313
plugins/scheduler-k3s/storage.go
Normal file
313
plugins/scheduler-k3s/storage.go
Normal file
@@ -0,0 +1,313 @@
|
||||
package scheduler_k3s
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"github.com/dokku/dokku/plugins/common"
|
||||
storage "github.com/dokku/dokku/plugins/storage"
|
||||
corev1 "k8s.io/api/core/v1"
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
)
|
||||
|
||||
// StorageChartValues are the helm chart inputs for the per-entry storage chart.
|
||||
type StorageChartValues struct {
|
||||
Global StorageChartGlobal `yaml:"global"`
|
||||
}
|
||||
|
||||
// StorageChartGlobal contains global values for the storage chart.
|
||||
type StorageChartGlobal struct {
|
||||
EntryName string `yaml:"entry_name"`
|
||||
Namespace string `yaml:"namespace"`
|
||||
Size string `yaml:"size"`
|
||||
AccessMode string `yaml:"access_mode,omitempty"`
|
||||
StorageClass string `yaml:"storage_class,omitempty"`
|
||||
HostPath string `yaml:"host_path,omitempty"`
|
||||
ReclaimPolicy string `yaml:"reclaim_policy,omitempty"`
|
||||
Annotations map[string]string `yaml:"annotations,omitempty"`
|
||||
Labels map[string]string `yaml:"labels,omitempty"`
|
||||
}
|
||||
|
||||
// GetStorageReleaseName returns the helm release name for a storage entry.
|
||||
func GetStorageReleaseName(entryName string) string {
|
||||
return fmt.Sprintf("storage-%s", entryName)
|
||||
}
|
||||
|
||||
// readEntryFromStdin decodes a storage.Entry from stdin. The storage
|
||||
// plugin pipes the JSON-encoded entry to scheduler triggers.
|
||||
func readEntryFromStdin() (*storage.Entry, error) {
|
||||
raw, err := io.ReadAll(os.Stdin)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("unable to read storage entry from stdin: %w", err)
|
||||
}
|
||||
if len(raw) == 0 {
|
||||
return nil, fmt.Errorf("no storage entry payload received on stdin")
|
||||
}
|
||||
entry := &storage.Entry{}
|
||||
if err := json.Unmarshal(raw, entry); err != nil {
|
||||
return nil, fmt.Errorf("unable to parse storage entry payload: %w", err)
|
||||
}
|
||||
return entry, nil
|
||||
}
|
||||
|
||||
// TriggerStorageCreate provisions or updates the PVC/PV for a k3s storage
|
||||
// entry. Idempotent: re-runs against the same entry are a helm upgrade.
|
||||
func TriggerStorageCreate(ctx context.Context, entryName string) error {
|
||||
if err := isKubernetesAvailable(); err != nil {
|
||||
return fmt.Errorf("kubernetes not available: %w", err)
|
||||
}
|
||||
|
||||
entry, err := readEntryFromStdin()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if entry.Name != entryName {
|
||||
return fmt.Errorf("entry name mismatch: stdin=%q, arg=%q", entry.Name, entryName)
|
||||
}
|
||||
if entry.Scheduler != storage.SchedulerK3s {
|
||||
return nil
|
||||
}
|
||||
|
||||
if err := validateStorageClass(ctx, entry.StorageClass); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
chartDir, err := os.MkdirTemp("", "dokku-storage-chart-")
|
||||
if err != nil {
|
||||
return fmt.Errorf("error creating chart directory: %w", err)
|
||||
}
|
||||
defer os.RemoveAll(chartDir)
|
||||
|
||||
releaseName := GetStorageReleaseName(entry.Name)
|
||||
namespace := entry.Namespace
|
||||
if namespace == "" {
|
||||
namespace = "default"
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(filepath.Join(chartDir, "templates"), 0755); err != nil {
|
||||
return fmt.Errorf("error creating chart templates directory: %w", err)
|
||||
}
|
||||
|
||||
chart := &Chart{
|
||||
ApiVersion: "v2",
|
||||
AppVersion: "1.0.0",
|
||||
Name: releaseName,
|
||||
Icon: "https://dokku.com/assets/dokku-logo.svg",
|
||||
Version: "0.0.1",
|
||||
}
|
||||
if err := writeYaml(WriteYamlInput{Object: chart, Path: filepath.Join(chartDir, "Chart.yaml")}); err != nil {
|
||||
return fmt.Errorf("error writing chart: %w", err)
|
||||
}
|
||||
|
||||
for _, name := range []string{"persistent-volume-claim.yaml", "persistent-volume.yaml"} {
|
||||
b, err := templates.ReadFile(filepath.Join("templates", "storage-chart", "templates", name))
|
||||
if err != nil {
|
||||
return fmt.Errorf("error reading %s: %w", name, err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(chartDir, "templates", name), b, 0644); err != nil {
|
||||
return fmt.Errorf("error writing %s: %w", name, err)
|
||||
}
|
||||
}
|
||||
|
||||
values := &StorageChartValues{
|
||||
Global: StorageChartGlobal{
|
||||
EntryName: entry.Name,
|
||||
Namespace: namespace,
|
||||
Size: entry.Size,
|
||||
AccessMode: entry.AccessMode,
|
||||
StorageClass: entry.StorageClass,
|
||||
HostPath: entry.HostPath,
|
||||
ReclaimPolicy: entry.ReclaimPolicy,
|
||||
Annotations: entry.Annotations,
|
||||
Labels: entry.Labels,
|
||||
},
|
||||
}
|
||||
if err := writeYaml(WriteYamlInput{Object: values, Path: filepath.Join(chartDir, "values.yaml")}); err != nil {
|
||||
return fmt.Errorf("error writing values: %w", err)
|
||||
}
|
||||
|
||||
if err := createKubernetesNamespace(ctx, namespace); err != nil {
|
||||
return fmt.Errorf("error creating namespace: %w", err)
|
||||
}
|
||||
|
||||
helmAgent, err := NewHelmAgent(namespace, DeployLogPrinter)
|
||||
if err != nil {
|
||||
return fmt.Errorf("error creating helm agent: %w", err)
|
||||
}
|
||||
|
||||
chartPath, err := filepath.Abs(chartDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("error getting chart path: %w", err)
|
||||
}
|
||||
|
||||
common.LogVerboseQuiet(fmt.Sprintf("Installing storage chart for %s", entry.Name))
|
||||
if err := helmAgent.InstallOrUpgradeChart(ctx, ChartInput{
|
||||
ChartPath: chartPath,
|
||||
Namespace: namespace,
|
||||
ReleaseName: releaseName,
|
||||
Wait: false,
|
||||
}); err != nil {
|
||||
return fmt.Errorf("error installing storage chart: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// TriggerStorageDestroy removes the helm release backing a storage entry.
|
||||
func TriggerStorageDestroy(ctx context.Context, entryName string) error {
|
||||
if err := isKubernetesAvailable(); err != nil {
|
||||
return fmt.Errorf("kubernetes not available: %w", err)
|
||||
}
|
||||
|
||||
entry, err := readEntryFromStdin()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if entry.Scheduler != storage.SchedulerK3s {
|
||||
return nil
|
||||
}
|
||||
|
||||
namespace := entry.Namespace
|
||||
if namespace == "" {
|
||||
namespace = "default"
|
||||
}
|
||||
releaseName := GetStorageReleaseName(entryName)
|
||||
|
||||
helmAgent, err := NewHelmAgent(namespace, DeployLogPrinter)
|
||||
if err != nil {
|
||||
return fmt.Errorf("error creating helm agent: %w", err)
|
||||
}
|
||||
exists, err := helmAgent.ChartExists(releaseName)
|
||||
if err != nil {
|
||||
return fmt.Errorf("error checking if storage chart exists: %w", err)
|
||||
}
|
||||
if !exists {
|
||||
return nil
|
||||
}
|
||||
|
||||
common.LogVerboseQuiet(fmt.Sprintf("Removing storage chart for %s", entryName))
|
||||
if err := helmAgent.UninstallChart(releaseName); err != nil {
|
||||
return fmt.Errorf("error uninstalling storage chart: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// TriggerStorageStatus prints "Bound" if the entry's PVC is bound,
|
||||
// "Pending" or "Lost" otherwise. Used by the storage:wait command.
|
||||
func TriggerStorageStatus(ctx context.Context, entryName string) error {
|
||||
if err := isKubernetesAvailable(); err != nil {
|
||||
return fmt.Errorf("kubernetes not available: %w", err)
|
||||
}
|
||||
entry, err := readEntryFromStdin()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
namespace := entry.Namespace
|
||||
if namespace == "" {
|
||||
namespace = "default"
|
||||
}
|
||||
|
||||
clientset, err := NewKubernetesClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
pvc, err := clientset.Client.CoreV1().PersistentVolumeClaims(namespace).Get(ctx, entryName, metav1.GetOptions{})
|
||||
if err != nil {
|
||||
return fmt.Errorf("error fetching PVC %s/%s: %w", namespace, entryName, err)
|
||||
}
|
||||
fmt.Println(string(pvc.Status.Phase))
|
||||
return nil
|
||||
}
|
||||
|
||||
// validateStorageClass surfaces a clear error when the requested storage
|
||||
// class doesn't exist in the cluster.
|
||||
func validateStorageClass(ctx context.Context, name string) error {
|
||||
if name == "" {
|
||||
return nil
|
||||
}
|
||||
clientset, err := NewKubernetesClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
classes, err := clientset.Client.StorageV1().StorageClasses().List(ctx, metav1.ListOptions{})
|
||||
if err != nil {
|
||||
return fmt.Errorf("error listing storage classes: %w", err)
|
||||
}
|
||||
available := []string{}
|
||||
for _, cls := range classes.Items {
|
||||
if cls.Name == name {
|
||||
return nil
|
||||
}
|
||||
available = append(available, cls.Name)
|
||||
}
|
||||
return fmt.Errorf("storage class %q does not exist; available: %s", name, strings.Join(available, ", "))
|
||||
}
|
||||
|
||||
// AppMountPair mirrors the storage plugin's wire format so we can decode
|
||||
// the storage-app-mounts trigger output here.
|
||||
type AppMountPair struct {
|
||||
Entry *storage.Entry `json:"entry"`
|
||||
Attachment *storage.Attachment `json:"attachment"`
|
||||
}
|
||||
|
||||
// LoadAppMounts asks the storage plugin for an app's mount pairs in the
|
||||
// given phase. Returns an empty slice when there are none.
|
||||
func LoadAppMounts(appName string, phase string) ([]AppMountPair, error) {
|
||||
results, err := common.CallPlugnTrigger(common.PlugnTriggerInput{
|
||||
Trigger: "storage-app-mounts",
|
||||
Args: []string{appName, phase},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
output := strings.TrimSpace(results.StdoutContents())
|
||||
if output == "" {
|
||||
return nil, nil
|
||||
}
|
||||
pairs := []AppMountPair{}
|
||||
if err := json.Unmarshal([]byte(output), &pairs); err != nil {
|
||||
return nil, fmt.Errorf("unable to parse storage-app-mounts output: %w", err)
|
||||
}
|
||||
return pairs, nil
|
||||
}
|
||||
|
||||
// ToProcessVolumes converts each AppMountPair into a ProcessVolume. K3s
|
||||
// app deployments reference the PVC by name; the PVC itself is owned by
|
||||
// the storage entry's separate helm release. Any docker-local entries
|
||||
// found here are an error - they cannot be mounted on a k3s app.
|
||||
func ToProcessVolumes(pairs []AppMountPair) ([]ProcessVolume, error) {
|
||||
volumes := []ProcessVolume{}
|
||||
for _, pair := range pairs {
|
||||
if pair.Entry == nil || pair.Attachment == nil {
|
||||
continue
|
||||
}
|
||||
if pair.Entry.Scheduler == storage.SchedulerDockerLocal {
|
||||
return nil, fmt.Errorf("storage entry %q is scheduler=docker-local but is mounted on a k3s app; recreate it with --scheduler k3s", pair.Entry.Name)
|
||||
}
|
||||
volumes = append(volumes, ProcessVolume{
|
||||
Name: pair.Entry.Name,
|
||||
MountPath: pair.Attachment.ContainerPath,
|
||||
SubPath: pair.Attachment.Subpath,
|
||||
ReadOnly: pair.Attachment.Readonly,
|
||||
PersistentClaim: &ProcessVolumePersistentClaim{
|
||||
ClaimName: pair.Entry.Name,
|
||||
},
|
||||
})
|
||||
}
|
||||
return volumes, nil
|
||||
}
|
||||
|
||||
// asK8sVolumeMount is a small helper used by tests / other callers that
|
||||
// want a corev1.VolumeMount from a ProcessVolume.
|
||||
func asK8sVolumeMount(v ProcessVolume) corev1.VolumeMount {
|
||||
return corev1.VolumeMount{
|
||||
Name: v.Name,
|
||||
MountPath: v.MountPath,
|
||||
SubPath: v.SubPath,
|
||||
ReadOnly: v.ReadOnly,
|
||||
}
|
||||
}
|
||||
236
plugins/scheduler-k3s/storage_exec.go
Normal file
236
plugins/scheduler-k3s/storage_exec.go
Normal file
@@ -0,0 +1,236 @@
|
||||
package scheduler_k3s
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/dokku/dokku/plugins/common"
|
||||
"github.com/dokku/dokku/plugins/storage"
|
||||
corev1 "k8s.io/api/core/v1"
|
||||
apierrors "k8s.io/apimachinery/pkg/api/errors"
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
utilexec "k8s.io/client-go/util/exec"
|
||||
)
|
||||
|
||||
// StorageExecInput captures the inputs forwarded by the storage plugin
|
||||
// over the scheduler-storage-exec trigger.
|
||||
type StorageExecInput struct {
|
||||
EntryName string
|
||||
Image string
|
||||
Interactive bool
|
||||
Tty bool
|
||||
AsUser string
|
||||
Command []string
|
||||
}
|
||||
|
||||
// TriggerSchedulerStorageExec runs an interactive or non-interactive
|
||||
// command against a k3s storage entry by creating a throwaway Pod that
|
||||
// mounts the PVC at /data, exec-ing the user's command via the
|
||||
// kubernetes API, and deleting the Pod afterwards. Errors at every
|
||||
// stage (PVC missing, image pull failure, etc.) come through the
|
||||
// structured Kubernetes API objects rather than kubectl-shaped strings.
|
||||
func TriggerSchedulerStorageExec(ctx context.Context, scheduler string, input StorageExecInput) error {
|
||||
if scheduler != storage.SchedulerK3s {
|
||||
return nil
|
||||
}
|
||||
if !storage.EntryExists(input.EntryName) {
|
||||
return fmt.Errorf("storage entry %q does not exist", input.EntryName)
|
||||
}
|
||||
entry, err := storage.LoadEntry(input.EntryName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if entry.Scheduler != storage.SchedulerK3s {
|
||||
return fmt.Errorf("storage entry %q has scheduler %q, not k3s", entry.Name, entry.Scheduler)
|
||||
}
|
||||
|
||||
if err := isKubernetesAvailable(); err != nil {
|
||||
return fmt.Errorf("kubernetes not available: %w", err)
|
||||
}
|
||||
|
||||
clientset, err := NewKubernetesClient()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
namespace := entry.Namespace
|
||||
if namespace == "" {
|
||||
namespace = "default"
|
||||
}
|
||||
|
||||
if _, err := clientset.Client.CoreV1().PersistentVolumeClaims(namespace).Get(ctx, entry.Name, metav1.GetOptions{}); err != nil {
|
||||
if apierrors.IsNotFound(err) {
|
||||
return fmt.Errorf("PVC %s not found in namespace %s", entry.Name, namespace)
|
||||
}
|
||||
return fmt.Errorf("error reading PVC %s in namespace %s: %w", entry.Name, namespace, err)
|
||||
}
|
||||
|
||||
podName := fmt.Sprintf("dokku-storage-exec-%s-%d", entry.Name, time.Now().UnixNano()/int64(time.Millisecond)%1000000)
|
||||
pod, err := buildStoragePodSpec(podName, namespace, entry, input)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
common.LogVerboseQuiet(fmt.Sprintf("Creating exec pod %s/%s", namespace, podName))
|
||||
if _, err := clientset.Client.CoreV1().Pods(namespace).Create(ctx, pod, metav1.CreateOptions{}); err != nil {
|
||||
return fmt.Errorf("error creating exec pod %s/%s: %w", namespace, podName, err)
|
||||
}
|
||||
|
||||
defer func() {
|
||||
bg := context.Background()
|
||||
gracePeriod := int64(0)
|
||||
_ = clientset.Client.CoreV1().Pods(namespace).Delete(bg, podName, metav1.DeleteOptions{GracePeriodSeconds: &gracePeriod})
|
||||
}()
|
||||
|
||||
if err := waitForPodRunning(ctx, clientset, namespace, podName, 60*time.Second); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
command := input.Command
|
||||
if len(command) == 0 {
|
||||
command = []string{"sh", "-c", "command -v bash >/dev/null 2>&1 && exec bash || exec sh"}
|
||||
}
|
||||
|
||||
execErr := clientset.ExecCommand(ctx, ExecCommandInput{
|
||||
Name: podName,
|
||||
Namespace: namespace,
|
||||
ContainerName: "exec",
|
||||
Command: command,
|
||||
})
|
||||
if execErr == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
var coded utilexec.CodeExitError
|
||||
if errors.As(execErr, &coded) {
|
||||
os.Exit(coded.Code)
|
||||
}
|
||||
return execErr
|
||||
}
|
||||
|
||||
// buildStoragePodSpec assembles the throwaway Pod that mounts the PVC
|
||||
// for storage:exec. Split out so the unit test can verify the spec
|
||||
// without hitting a cluster.
|
||||
func buildStoragePodSpec(name string, namespace string, entry *storage.Entry, input StorageExecInput) (*corev1.Pod, error) {
|
||||
uid, err := resolveStorageExecUID(entry.Chown, input.AsUser)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
pod := &corev1.Pod{
|
||||
ObjectMeta: metav1.ObjectMeta{
|
||||
Name: name,
|
||||
Namespace: namespace,
|
||||
Labels: map[string]string{
|
||||
"app.kubernetes.io/managed-by": "dokku",
|
||||
"dokku.com/storage-entry": entry.Name,
|
||||
"dokku.com/purpose": "storage-exec",
|
||||
},
|
||||
},
|
||||
Spec: corev1.PodSpec{
|
||||
RestartPolicy: corev1.RestartPolicyNever,
|
||||
Volumes: []corev1.Volume{
|
||||
{
|
||||
Name: "data",
|
||||
VolumeSource: corev1.VolumeSource{
|
||||
PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{
|
||||
ClaimName: entry.Name,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
Containers: []corev1.Container{
|
||||
{
|
||||
Name: "exec",
|
||||
Image: input.Image,
|
||||
Command: []string{"sleep", "infinity"},
|
||||
Stdin: input.Interactive,
|
||||
TTY: input.Tty,
|
||||
VolumeMounts: []corev1.VolumeMount{
|
||||
{Name: "data", MountPath: "/data"},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
if uid != nil {
|
||||
pod.Spec.SecurityContext = &corev1.PodSecurityContext{
|
||||
RunAsUser: uid,
|
||||
RunAsGroup: uid,
|
||||
FSGroup: uid,
|
||||
}
|
||||
}
|
||||
|
||||
return pod, nil
|
||||
}
|
||||
|
||||
// resolveStorageExecUID picks the UID to run the exec Pod as. Returns
|
||||
// nil to mean "leave the image's default user in place".
|
||||
func resolveStorageExecUID(chown string, asUser string) (*int64, error) {
|
||||
if asUser != "" {
|
||||
n, err := strconv.ParseInt(asUser, 10, 64)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("--as-user must be a numeric uid, got %q", asUser)
|
||||
}
|
||||
return &n, nil
|
||||
}
|
||||
if chown == "" || chown == "false" {
|
||||
return nil, nil
|
||||
}
|
||||
uidStr, err := storage.ResolveChownID(chown)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if uidStr == "" || uidStr == "false" {
|
||||
return nil, nil
|
||||
}
|
||||
n, err := strconv.ParseInt(uidStr, 10, 64)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("could not parse resolved chown UID %q: %w", uidStr, err)
|
||||
}
|
||||
return &n, nil
|
||||
}
|
||||
|
||||
// waitForPodRunning polls the Pod until its phase is Running, surfacing
|
||||
// the structured Kubernetes API state on failure (image pull failures,
|
||||
// scheduling problems, etc.) rather than a generic timeout.
|
||||
func waitForPodRunning(ctx context.Context, clientset KubernetesClient, namespace, name string, timeout time.Duration) error {
|
||||
deadline := time.Now().Add(timeout)
|
||||
for {
|
||||
pod, err := clientset.Client.CoreV1().Pods(namespace).Get(ctx, name, metav1.GetOptions{})
|
||||
if err != nil {
|
||||
return fmt.Errorf("error reading exec pod %s/%s: %w", namespace, name, err)
|
||||
}
|
||||
switch pod.Status.Phase {
|
||||
case corev1.PodRunning:
|
||||
return nil
|
||||
case corev1.PodFailed, corev1.PodSucceeded:
|
||||
return fmt.Errorf("exec pod %s/%s reached terminal phase %s before exec could attach", namespace, name, pod.Status.Phase)
|
||||
}
|
||||
|
||||
for _, cs := range pod.Status.ContainerStatuses {
|
||||
if cs.State.Waiting == nil {
|
||||
continue
|
||||
}
|
||||
reason := cs.State.Waiting.Reason
|
||||
switch reason {
|
||||
case "ImagePullBackOff", "ErrImagePull", "CreateContainerConfigError", "InvalidImageName":
|
||||
return fmt.Errorf("exec pod %s/%s container %q failed to start: %s: %s", namespace, name, cs.Name, reason, cs.State.Waiting.Message)
|
||||
}
|
||||
}
|
||||
|
||||
if time.Now().After(deadline) {
|
||||
return fmt.Errorf("exec pod %s/%s did not reach Running within %s (current phase: %s)", namespace, name, timeout, pod.Status.Phase)
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-time.After(500 * time.Millisecond):
|
||||
}
|
||||
}
|
||||
}
|
||||
103
plugins/scheduler-k3s/storage_exec_test.go
Normal file
103
plugins/scheduler-k3s/storage_exec_test.go
Normal file
@@ -0,0 +1,103 @@
|
||||
package scheduler_k3s
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/dokku/dokku/plugins/storage"
|
||||
)
|
||||
|
||||
func TestBuildStoragePodSpec(t *testing.T) {
|
||||
entry := &storage.Entry{
|
||||
Name: "demo-data",
|
||||
Scheduler: storage.SchedulerK3s,
|
||||
Namespace: "dokku",
|
||||
}
|
||||
pod, err := buildStoragePodSpec("dokku-storage-exec-demo-data-1", "dokku", entry, StorageExecInput{
|
||||
Image: "alpine:3",
|
||||
Interactive: true,
|
||||
Tty: true,
|
||||
Command: []string{"sh"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("buildStoragePodSpec err: %v", err)
|
||||
}
|
||||
|
||||
if pod.Spec.RestartPolicy != "Never" {
|
||||
t.Fatalf("expected RestartPolicy=Never, got %s", pod.Spec.RestartPolicy)
|
||||
}
|
||||
if len(pod.Spec.Volumes) != 1 || pod.Spec.Volumes[0].PersistentVolumeClaim == nil {
|
||||
t.Fatalf("expected one PVC volume, got %+v", pod.Spec.Volumes)
|
||||
}
|
||||
if pod.Spec.Volumes[0].PersistentVolumeClaim.ClaimName != "demo-data" {
|
||||
t.Fatalf("expected PVC claim name demo-data, got %s", pod.Spec.Volumes[0].PersistentVolumeClaim.ClaimName)
|
||||
}
|
||||
if len(pod.Spec.Containers) != 1 {
|
||||
t.Fatalf("expected one container, got %d", len(pod.Spec.Containers))
|
||||
}
|
||||
c := pod.Spec.Containers[0]
|
||||
if c.Image != "alpine:3" {
|
||||
t.Fatalf("expected image alpine:3, got %s", c.Image)
|
||||
}
|
||||
if !c.Stdin || !c.TTY {
|
||||
t.Fatalf("expected stdin and tty true on the container, got stdin=%v tty=%v", c.Stdin, c.TTY)
|
||||
}
|
||||
if len(c.VolumeMounts) != 1 || c.VolumeMounts[0].MountPath != "/data" {
|
||||
t.Fatalf("expected one volume mount at /data, got %+v", c.VolumeMounts)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildStoragePodSpecChownToSecurityContext(t *testing.T) {
|
||||
entry := &storage.Entry{
|
||||
Name: "demo-data",
|
||||
Scheduler: storage.SchedulerK3s,
|
||||
Chown: "herokuish",
|
||||
}
|
||||
pod, err := buildStoragePodSpec("p", "default", entry, StorageExecInput{Image: "alpine:3"})
|
||||
if err != nil {
|
||||
t.Fatalf("err: %v", err)
|
||||
}
|
||||
if pod.Spec.SecurityContext == nil || pod.Spec.SecurityContext.RunAsUser == nil {
|
||||
t.Fatalf("expected SecurityContext.RunAsUser to be populated for chown=herokuish")
|
||||
}
|
||||
// Don't assert exact UID because ResolveChownID consults docker for
|
||||
// userns offset; only assert non-nil and non-zero.
|
||||
if *pod.Spec.SecurityContext.RunAsUser == 0 {
|
||||
t.Fatalf("expected non-zero UID for chown=herokuish, got 0")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildStoragePodSpecAsUserOverride(t *testing.T) {
|
||||
entry := &storage.Entry{
|
||||
Name: "demo-data",
|
||||
Scheduler: storage.SchedulerK3s,
|
||||
Chown: "herokuish",
|
||||
}
|
||||
pod, err := buildStoragePodSpec("p", "default", entry, StorageExecInput{
|
||||
Image: "alpine:3",
|
||||
AsUser: "0",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("err: %v", err)
|
||||
}
|
||||
if pod.Spec.SecurityContext == nil || pod.Spec.SecurityContext.RunAsUser == nil {
|
||||
t.Fatalf("expected RunAsUser to be set")
|
||||
}
|
||||
if *pod.Spec.SecurityContext.RunAsUser != 0 {
|
||||
t.Fatalf("expected RunAsUser to be 0 (override), got %d", *pod.Spec.SecurityContext.RunAsUser)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildStoragePodSpecChownFalseLeavesDefaultUser(t *testing.T) {
|
||||
entry := &storage.Entry{
|
||||
Name: "demo-data",
|
||||
Scheduler: storage.SchedulerK3s,
|
||||
Chown: "false",
|
||||
}
|
||||
pod, err := buildStoragePodSpec("p", "default", entry, StorageExecInput{Image: "alpine:3"})
|
||||
if err != nil {
|
||||
t.Fatalf("err: %v", err)
|
||||
}
|
||||
if pod.Spec.SecurityContext != nil {
|
||||
t.Fatalf("expected no SecurityContext when chown=false, got %+v", pod.Spec.SecurityContext)
|
||||
}
|
||||
}
|
||||
@@ -117,9 +117,12 @@ type ProcessValues struct {
|
||||
}
|
||||
|
||||
type ProcessVolume struct {
|
||||
Name string `yaml:"name"`
|
||||
MountPath string `yaml:"mount_path"`
|
||||
EmptyDir *ProcessVolumeEmptyDir `yaml:"empty_dir,omitempty"`
|
||||
Name string `yaml:"name"`
|
||||
MountPath string `yaml:"mount_path"`
|
||||
SubPath string `yaml:"sub_path,omitempty"`
|
||||
ReadOnly bool `yaml:"read_only,omitempty"`
|
||||
EmptyDir *ProcessVolumeEmptyDir `yaml:"empty_dir,omitempty"`
|
||||
PersistentClaim *ProcessVolumePersistentClaim `yaml:"persistent_claim,omitempty"`
|
||||
}
|
||||
|
||||
type ProcessVolumeEmptyDir struct {
|
||||
@@ -127,6 +130,10 @@ type ProcessVolumeEmptyDir struct {
|
||||
SizeLimit string `yaml:"size_limit"`
|
||||
}
|
||||
|
||||
type ProcessVolumePersistentClaim struct {
|
||||
ClaimName string `yaml:"claim_name"`
|
||||
}
|
||||
|
||||
type ProcessAnnotations struct {
|
||||
CertificateAnnotations map[string]string `yaml:"certificate,omitempty"`
|
||||
CronJobAnnotations map[string]string `yaml:"cronjob,omitempty"`
|
||||
|
||||
@@ -127,12 +127,44 @@ spec:
|
||||
{{- if $.Values.global.image.working_dir }}
|
||||
workingDir: {{ $.Values.global.image.working_dir }}
|
||||
{{- end }}
|
||||
{{- if $config.volumes }}
|
||||
volumeMounts:
|
||||
{{- range $volume := $config.volumes }}
|
||||
- name: {{ $volume.name }}
|
||||
mountPath: {{ $volume.mount_path }}
|
||||
{{- if $volume.sub_path }}
|
||||
subPath: {{ $volume.sub_path }}
|
||||
{{- end }}
|
||||
{{- if $volume.read_only }}
|
||||
readOnly: true
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if $.Values.global.image.image_pull_secrets }}
|
||||
imagePullSecrets:
|
||||
- name: {{ $.Values.global.image.image_pull_secrets }}
|
||||
{{- end }}
|
||||
restartPolicy: Never
|
||||
serviceAccountName: {{ $.Values.global.app_name }}
|
||||
{{- if $config.volumes }}
|
||||
volumes:
|
||||
{{- range $volume := $config.volumes }}
|
||||
- name: {{ $volume.name }}
|
||||
{{- if $volume.empty_dir }}
|
||||
emptyDir:
|
||||
{{- if $volume.empty_dir.medium }}
|
||||
medium: {{ $volume.empty_dir.medium }}
|
||||
{{- end }}
|
||||
{{- if $volume.empty_dir.size_limit }}
|
||||
sizeLimit: {{ $volume.empty_dir.size_limit }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if $volume.persistent_claim }}
|
||||
persistentVolumeClaim:
|
||||
claimName: {{ $volume.persistent_claim.claim_name }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
schedule: {{ $config.cron.schedule }}
|
||||
startingDeadlineSeconds: 60
|
||||
successfulJobsHistoryLimit: 10
|
||||
|
||||
@@ -152,6 +152,12 @@ spec:
|
||||
{{- range $volume := $config.volumes }}
|
||||
- name: {{ $volume.name }}
|
||||
mountPath: {{ $volume.mount_path }}
|
||||
{{- if $volume.sub_path }}
|
||||
subPath: {{ $volume.sub_path }}
|
||||
{{- end }}
|
||||
{{- if $volume.read_only }}
|
||||
readOnly: true
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if $.Values.global.image.image_pull_secrets }}
|
||||
@@ -172,6 +178,10 @@ spec:
|
||||
sizeLimit: {{ $volume.empty_dir.size_limit }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- if $volume.persistent_claim }}
|
||||
persistentVolumeClaim:
|
||||
claimName: {{ $volume.persistent_claim.claim_name }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: {{ .Values.global.entry_name }}
|
||||
namespace: {{ .Values.global.namespace }}
|
||||
labels:
|
||||
app.kubernetes.io/name: {{ .Values.global.entry_name }}
|
||||
app.kubernetes.io/managed-by: dokku
|
||||
{{- range $k, $v := .Values.global.labels }}
|
||||
{{ $k }}: {{ $v | quote }}
|
||||
{{- end }}
|
||||
annotations:
|
||||
dokku.com/managed: "true"
|
||||
{{- range $k, $v := .Values.global.annotations }}
|
||||
{{ $k }}: {{ $v | quote }}
|
||||
{{- end }}
|
||||
spec:
|
||||
accessModes:
|
||||
- {{ .Values.global.access_mode | default "ReadWriteOnce" }}
|
||||
resources:
|
||||
requests:
|
||||
storage: {{ .Values.global.size }}
|
||||
{{- if .Values.global.storage_class }}
|
||||
storageClassName: {{ .Values.global.storage_class }}
|
||||
{{- end }}
|
||||
{{- if .Values.global.host_path }}
|
||||
volumeName: {{ .Values.global.entry_name }}
|
||||
storageClassName: ""
|
||||
{{- end }}
|
||||
@@ -0,0 +1,29 @@
|
||||
{{- if .Values.global.host_path }}
|
||||
apiVersion: v1
|
||||
kind: PersistentVolume
|
||||
metadata:
|
||||
name: {{ .Values.global.entry_name }}
|
||||
labels:
|
||||
app.kubernetes.io/name: {{ .Values.global.entry_name }}
|
||||
app.kubernetes.io/managed-by: dokku
|
||||
{{- range $k, $v := .Values.global.labels }}
|
||||
{{ $k }}: {{ $v | quote }}
|
||||
{{- end }}
|
||||
annotations:
|
||||
dokku.com/managed: "true"
|
||||
{{- range $k, $v := .Values.global.annotations }}
|
||||
{{ $k }}: {{ $v | quote }}
|
||||
{{- end }}
|
||||
spec:
|
||||
capacity:
|
||||
storage: {{ .Values.global.size }}
|
||||
accessModes:
|
||||
- {{ .Values.global.access_mode | default "ReadWriteOnce" }}
|
||||
persistentVolumeReclaimPolicy: {{ .Values.global.reclaim_policy | default "Retain" }}
|
||||
hostPath:
|
||||
path: {{ .Values.global.host_path }}
|
||||
storageClassName: ""
|
||||
claimRef:
|
||||
namespace: {{ .Values.global.namespace }}
|
||||
name: {{ .Values.global.entry_name }}
|
||||
{{- end }}
|
||||
@@ -605,10 +605,19 @@ func TriggerSchedulerDeploy(scheduler string, appName string, imageTag string) e
|
||||
})
|
||||
}
|
||||
|
||||
deployMountPairs, err := LoadAppMounts(appName, "deploy")
|
||||
if err != nil {
|
||||
return fmt.Errorf("error loading storage app mounts: %w", err)
|
||||
}
|
||||
deployVolumes, err := ToProcessVolumes(deployMountPairs)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
processVolumes = append(processVolumes, deployVolumes...)
|
||||
|
||||
for processType, processCount := range processes {
|
||||
// todo: implement deployment annotations
|
||||
// todo: implement pod annotations
|
||||
// todo: implement volumes
|
||||
|
||||
healthchecks, ok := appJSON.Healthchecks[processType]
|
||||
if !ok {
|
||||
@@ -778,7 +787,6 @@ func TriggerSchedulerDeploy(scheduler string, appName string, imageTag string) e
|
||||
for _, cronTask := range cronTasks {
|
||||
// todo: implement deployment annotations
|
||||
// todo: implement pod annotations
|
||||
// todo: implement volumes
|
||||
suffix := ""
|
||||
for _, cronJob := range cronJobs {
|
||||
if cronJob.Labels["dokku.com/cron-id"] == cronTask.ID {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
GOARCH ?= amd64
|
||||
SUBCOMMANDS = subcommands/default subcommands/ensure-directory subcommands/list subcommands/mount subcommands/report subcommands/unmount
|
||||
TRIGGERS = triggers/install triggers/storage-list
|
||||
SUBCOMMANDS = subcommands/default subcommands/create subcommands/destroy subcommands/ensure-directory subcommands/exec subcommands/info subcommands/list subcommands/list-entries subcommands/migrate subcommands/mount subcommands/report subcommands/set subcommands/unmount subcommands/wait
|
||||
TRIGGERS = triggers/install triggers/storage-list triggers/storage-app-mounts triggers/docker-args-deploy triggers/docker-args-run triggers/post-delete triggers/post-app-clone-setup triggers/post-app-rename-setup
|
||||
BUILD = commands subcommands triggers
|
||||
PLUGIN_NAME = storage
|
||||
|
||||
|
||||
221
plugins/storage/attachment.go
Normal file
221
plugins/storage/attachment.go
Normal file
@@ -0,0 +1,221 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"github.com/dokku/dokku/plugins/common"
|
||||
)
|
||||
|
||||
// AttachmentsProperty is the property-list key used to store attachments
|
||||
// for an app. All process types live under one list and carry their own
|
||||
// ProcessType field; we filter on read.
|
||||
const AttachmentsProperty = "mounts"
|
||||
|
||||
// PluginName is the name we register with the property system.
|
||||
const PluginName = "storage"
|
||||
|
||||
// PhaseDeploy is the standard deploy phase identifier.
|
||||
const PhaseDeploy = "deploy"
|
||||
|
||||
// PhaseRun is the standard run phase identifier.
|
||||
const PhaseRun = "run"
|
||||
|
||||
// DefaultProcessType is the wildcard process type that applies to every
|
||||
// process. Mirrors docker-options' DefaultProcessType.
|
||||
const DefaultProcessType = "_default_"
|
||||
|
||||
// Attachment is the source of truth for *how* an app uses a storage entry.
|
||||
// One Attachment binds one entry into one container path on one app.
|
||||
type Attachment struct {
|
||||
EntryName string `json:"entry_name"`
|
||||
ContainerPath string `json:"container_path"`
|
||||
Phases []string `json:"phases"`
|
||||
ProcessType string `json:"process_type,omitempty"`
|
||||
Subpath string `json:"subpath,omitempty"`
|
||||
Readonly bool `json:"readonly,omitempty"`
|
||||
VolumeOptions string `json:"volume_options,omitempty"`
|
||||
VolumeChown string `json:"volume_chown,omitempty"`
|
||||
}
|
||||
|
||||
// Validate checks an Attachment's fields against structural rules.
|
||||
func (a *Attachment) Validate() error {
|
||||
if a == nil {
|
||||
return errors.New("attachment is nil")
|
||||
}
|
||||
if a.EntryName == "" {
|
||||
return errors.New("attachment is missing entry name")
|
||||
}
|
||||
if a.ContainerPath == "" {
|
||||
return errors.New("attachment is missing container path")
|
||||
}
|
||||
if !strings.HasPrefix(a.ContainerPath, "/") {
|
||||
return fmt.Errorf("attachment container path %q must be absolute", a.ContainerPath)
|
||||
}
|
||||
if len(a.Phases) == 0 {
|
||||
return errors.New("attachment must specify at least one phase")
|
||||
}
|
||||
for _, phase := range a.Phases {
|
||||
if phase != PhaseDeploy && phase != PhaseRun {
|
||||
return fmt.Errorf("attachment phase %q is not supported (must be %q or %q)", phase, PhaseDeploy, PhaseRun)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// LoadAttachments returns every attachment registered against an app.
|
||||
func LoadAttachments(appName string) ([]*Attachment, error) {
|
||||
lines, err := common.PropertyListGet(PluginName, appName, AttachmentsProperty)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("unable to read storage attachments for %q: %w", appName, err)
|
||||
}
|
||||
|
||||
attachments := []*Attachment{}
|
||||
for _, line := range lines {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" {
|
||||
continue
|
||||
}
|
||||
attachment := &Attachment{}
|
||||
if err := json.Unmarshal([]byte(line), attachment); err != nil {
|
||||
return nil, fmt.Errorf("unable to parse storage attachment for %q: %w", appName, err)
|
||||
}
|
||||
attachments = append(attachments, attachment)
|
||||
}
|
||||
return attachments, nil
|
||||
}
|
||||
|
||||
// SaveAttachments overwrites the entire attachment list for an app.
|
||||
func SaveAttachments(appName string, attachments []*Attachment) error {
|
||||
lines := make([]string, 0, len(attachments))
|
||||
for _, attachment := range attachments {
|
||||
data, err := json.Marshal(attachment)
|
||||
if err != nil {
|
||||
return fmt.Errorf("unable to encode storage attachment for %q: %w", appName, err)
|
||||
}
|
||||
lines = append(lines, string(data))
|
||||
}
|
||||
return common.PropertyListWrite(PluginName, appName, AttachmentsProperty, lines)
|
||||
}
|
||||
|
||||
// AddAttachment appends an attachment, rejecting duplicates of the same
|
||||
// (entry_name, container_path, process_type) tuple.
|
||||
func AddAttachment(appName string, attachment *Attachment) error {
|
||||
if err := attachment.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
attachments, err := LoadAttachments(appName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for _, existing := range attachments {
|
||||
if existing.EntryName == attachment.EntryName &&
|
||||
existing.ContainerPath == attachment.ContainerPath &&
|
||||
existing.ProcessType == attachment.ProcessType {
|
||||
return fmt.Errorf("storage entry %q is already mounted at %q for process type %q on app %q",
|
||||
attachment.EntryName, attachment.ContainerPath, attachment.ProcessType, appName)
|
||||
}
|
||||
}
|
||||
|
||||
attachments = append(attachments, attachment)
|
||||
return SaveAttachments(appName, attachments)
|
||||
}
|
||||
|
||||
// RemoveAttachment removes the attachment matching the given entry and
|
||||
// optional container path. If containerPath is empty and there is more
|
||||
// than one match, returns an error.
|
||||
func RemoveAttachment(appName string, entryName string, containerPath string) error {
|
||||
attachments, err := LoadAttachments(appName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
matches := []*Attachment{}
|
||||
keep := []*Attachment{}
|
||||
for _, attachment := range attachments {
|
||||
if attachment.EntryName == entryName && (containerPath == "" || attachment.ContainerPath == containerPath) {
|
||||
matches = append(matches, attachment)
|
||||
continue
|
||||
}
|
||||
keep = append(keep, attachment)
|
||||
}
|
||||
|
||||
if len(matches) == 0 {
|
||||
if containerPath == "" {
|
||||
return fmt.Errorf("storage entry %q is not mounted on app %q", entryName, appName)
|
||||
}
|
||||
return fmt.Errorf("storage entry %q is not mounted at %q on app %q", entryName, containerPath, appName)
|
||||
}
|
||||
if len(matches) > 1 && containerPath == "" {
|
||||
paths := []string{}
|
||||
for _, attachment := range matches {
|
||||
paths = append(paths, attachment.ContainerPath)
|
||||
}
|
||||
sort.Strings(paths)
|
||||
return fmt.Errorf("storage entry %q is mounted at multiple paths on app %q (%s); pass --container-dir to disambiguate",
|
||||
entryName, appName, strings.Join(paths, ", "))
|
||||
}
|
||||
|
||||
return SaveAttachments(appName, keep)
|
||||
}
|
||||
|
||||
// AttachmentsForPhase returns the subset of an app's attachments that
|
||||
// apply to the given phase, sorted for stable output.
|
||||
func AttachmentsForPhase(appName string, phase string) ([]*Attachment, error) {
|
||||
attachments, err := LoadAttachments(appName)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
filtered := []*Attachment{}
|
||||
for _, attachment := range attachments {
|
||||
for _, p := range attachment.Phases {
|
||||
if p == phase {
|
||||
filtered = append(filtered, attachment)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
sort.Slice(filtered, func(i, j int) bool {
|
||||
if filtered[i].EntryName != filtered[j].EntryName {
|
||||
return filtered[i].EntryName < filtered[j].EntryName
|
||||
}
|
||||
return filtered[i].ContainerPath < filtered[j].ContainerPath
|
||||
})
|
||||
return filtered, nil
|
||||
}
|
||||
|
||||
// AppsUsingEntry returns the list of app names that have at least one
|
||||
// attachment referencing the given entry name. Used by storage:destroy
|
||||
// to refuse removing an entry that's still mounted.
|
||||
func AppsUsingEntry(entryName string) ([]string, error) {
|
||||
apps, err := common.DokkuApps()
|
||||
if err != nil {
|
||||
if errors.Is(err, common.NoAppsExist) {
|
||||
return nil, nil
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
using := []string{}
|
||||
for _, app := range apps {
|
||||
attachments, err := LoadAttachments(app)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, attachment := range attachments {
|
||||
if attachment.EntryName == entryName {
|
||||
using = append(using, app)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
sort.Strings(using)
|
||||
return using, nil
|
||||
}
|
||||
161
plugins/storage/attachment_test.go
Normal file
161
plugins/storage/attachment_test.go
Normal file
@@ -0,0 +1,161 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
// writeAttachmentsFile drops the JSON-encoded attachment list into the
|
||||
// per-app property file directly, bypassing common.PropertyListWrite (which
|
||||
// calls SetPermissions and chowns to dokku:dokku, which a developer machine
|
||||
// or CI runner generally doesn't have).
|
||||
func writeAttachmentsFile(t *testing.T, root string, appName string, attachments []*Attachment) {
|
||||
t.Helper()
|
||||
dir := filepath.Join(root, "config", "storage", appName)
|
||||
if err := os.MkdirAll(dir, 0755); err != nil {
|
||||
t.Fatalf("mkdir attachment dir: %v", err)
|
||||
}
|
||||
path := filepath.Join(dir, AttachmentsProperty)
|
||||
f, err := os.Create(path)
|
||||
if err != nil {
|
||||
t.Fatalf("create attachment file: %v", err)
|
||||
}
|
||||
defer f.Close()
|
||||
for _, a := range attachments {
|
||||
data, err := json.Marshal(a)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal attachment: %v", err)
|
||||
}
|
||||
f.Write(append(data, '\n'))
|
||||
}
|
||||
}
|
||||
|
||||
func TestAttachmentValidate(t *testing.T) {
|
||||
RegisterTestingT(t)
|
||||
|
||||
good := &Attachment{
|
||||
EntryName: "demo-data",
|
||||
ContainerPath: "/data",
|
||||
Phases: []string{PhaseDeploy, PhaseRun},
|
||||
}
|
||||
Expect(good.Validate()).To(Succeed())
|
||||
|
||||
noEntry := &Attachment{ContainerPath: "/data", Phases: []string{PhaseDeploy}}
|
||||
Expect(noEntry.Validate()).To(HaveOccurred())
|
||||
|
||||
noPath := &Attachment{EntryName: "demo-data", Phases: []string{PhaseDeploy}}
|
||||
Expect(noPath.Validate()).To(HaveOccurred())
|
||||
|
||||
relative := &Attachment{EntryName: "demo-data", ContainerPath: "data", Phases: []string{PhaseDeploy}}
|
||||
Expect(relative.Validate()).To(HaveOccurred())
|
||||
|
||||
noPhases := &Attachment{EntryName: "demo-data", ContainerPath: "/data"}
|
||||
Expect(noPhases.Validate()).To(HaveOccurred())
|
||||
|
||||
badPhase := &Attachment{EntryName: "demo-data", ContainerPath: "/data", Phases: []string{"build"}}
|
||||
Expect(badPhase.Validate()).To(HaveOccurred())
|
||||
}
|
||||
|
||||
func TestListAppMountEntriesDockerLocal(t *testing.T) {
|
||||
RegisterTestingT(t)
|
||||
root := withTempLibRoot(t)
|
||||
|
||||
Expect(SaveEntry(&Entry{
|
||||
Name: "demo-data",
|
||||
Scheduler: SchedulerDockerLocal,
|
||||
HostPath: "/var/lib/dokku/data/storage/demo-data",
|
||||
})).To(Succeed())
|
||||
|
||||
writeAttachmentsFile(t, root, "demo", []*Attachment{
|
||||
{
|
||||
EntryName: "demo-data",
|
||||
ContainerPath: "/data",
|
||||
Phases: []string{PhaseDeploy, PhaseRun},
|
||||
Readonly: true,
|
||||
},
|
||||
})
|
||||
|
||||
rows, err := ListAppMountEntries("demo", PhaseDeploy)
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(rows).To(HaveLen(1))
|
||||
Expect(rows[0].EntryName).To(Equal("demo-data"))
|
||||
Expect(rows[0].HostPath).To(Equal("/var/lib/dokku/data/storage/demo-data"))
|
||||
Expect(rows[0].ContainerPath).To(Equal("/data"))
|
||||
Expect(rows[0].VolumeOptions).To(Equal("ro"))
|
||||
|
||||
// Run phase shows it too because the attachment includes both phases.
|
||||
runRows, err := ListAppMountEntries("demo", PhaseRun)
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(runRows).To(HaveLen(1))
|
||||
|
||||
// formatStorageListEntry produces the legacy colon form including options.
|
||||
Expect(formatStorageListEntry(rows[0])).To(Equal("/var/lib/dokku/data/storage/demo-data:/data:ro"))
|
||||
}
|
||||
|
||||
func TestListAppMountEntriesK3sUsesEntryName(t *testing.T) {
|
||||
RegisterTestingT(t)
|
||||
root := withTempLibRoot(t)
|
||||
|
||||
// k3s entry has no host path because the cluster provisions it.
|
||||
Expect(SaveEntry(&Entry{
|
||||
Name: "demo-pvc",
|
||||
Scheduler: SchedulerK3s,
|
||||
Size: "2Gi",
|
||||
StorageClass: "longhorn",
|
||||
AccessMode: "ReadWriteOnce",
|
||||
})).To(Succeed())
|
||||
|
||||
writeAttachmentsFile(t, root, "demo", []*Attachment{
|
||||
{
|
||||
EntryName: "demo-pvc",
|
||||
ContainerPath: "/data",
|
||||
Phases: []string{PhaseDeploy},
|
||||
Subpath: "uploads",
|
||||
},
|
||||
})
|
||||
|
||||
rows, err := ListAppMountEntries("demo", PhaseDeploy)
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(rows).To(HaveLen(1))
|
||||
// HostPath falls back to the entry name so the colon form remains
|
||||
// well-formed for callers that only know about the legacy shape.
|
||||
Expect(rows[0].HostPath).To(Equal("demo-pvc"))
|
||||
Expect(rows[0].EntryName).To(Equal("demo-pvc"))
|
||||
Expect(rows[0].ContainerPath).To(Equal("/data"))
|
||||
Expect(formatStorageListEntry(rows[0])).To(Equal("demo-pvc:/data"))
|
||||
}
|
||||
|
||||
func TestListAppMountEntriesPhaseFilter(t *testing.T) {
|
||||
RegisterTestingT(t)
|
||||
root := withTempLibRoot(t)
|
||||
|
||||
Expect(SaveEntry(&Entry{
|
||||
Name: "demo-deploy-only",
|
||||
Scheduler: SchedulerDockerLocal,
|
||||
HostPath: "/srv/deploy",
|
||||
})).To(Succeed())
|
||||
Expect(SaveEntry(&Entry{
|
||||
Name: "demo-run-only",
|
||||
Scheduler: SchedulerDockerLocal,
|
||||
HostPath: "/srv/run",
|
||||
})).To(Succeed())
|
||||
|
||||
writeAttachmentsFile(t, root, "demo", []*Attachment{
|
||||
{EntryName: "demo-deploy-only", ContainerPath: "/d", Phases: []string{PhaseDeploy}},
|
||||
{EntryName: "demo-run-only", ContainerPath: "/r", Phases: []string{PhaseRun}},
|
||||
})
|
||||
|
||||
deployRows, err := ListAppMountEntries("demo", PhaseDeploy)
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(deployRows).To(HaveLen(1))
|
||||
Expect(deployRows[0].EntryName).To(Equal("demo-deploy-only"))
|
||||
|
||||
runRows, err := ListAppMountEntries("demo", PhaseRun)
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(runRows).To(HaveLen(1))
|
||||
Expect(runRows[0].EntryName).To(Equal("demo-run-only"))
|
||||
}
|
||||
567
plugins/storage/commands_entries.go
Normal file
567
plugins/storage/commands_entries.go
Normal file
@@ -0,0 +1,567 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"github.com/dokku/dokku/plugins/common"
|
||||
)
|
||||
|
||||
// CommandCreateInput captures the flags accepted by storage:create.
|
||||
type CommandCreateInput struct {
|
||||
Name string
|
||||
Path string
|
||||
Scheduler string
|
||||
Size string
|
||||
AccessMode string
|
||||
StorageClass string
|
||||
Namespace string
|
||||
Chown string
|
||||
ReclaimPolicy string
|
||||
Annotations map[string]string
|
||||
Labels map[string]string
|
||||
}
|
||||
|
||||
// CommandCreate registers a new storage entry.
|
||||
func CommandCreate(input CommandCreateInput) error {
|
||||
if err := ValidateEntryName(input.Name, false); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
scheduler := input.Scheduler
|
||||
if scheduler == "" {
|
||||
scheduler = SchedulerDockerLocal
|
||||
}
|
||||
|
||||
hostPath := input.Path
|
||||
if scheduler == SchedulerDockerLocal && hostPath == "" {
|
||||
hostPath = filepath.Join(GetStorageDirectory(), input.Name)
|
||||
}
|
||||
|
||||
entry := &Entry{
|
||||
Name: input.Name,
|
||||
Scheduler: scheduler,
|
||||
HostPath: hostPath,
|
||||
Size: input.Size,
|
||||
AccessMode: input.AccessMode,
|
||||
StorageClass: input.StorageClass,
|
||||
Namespace: input.Namespace,
|
||||
Chown: input.Chown,
|
||||
ReclaimPolicy: input.ReclaimPolicy,
|
||||
Annotations: input.Annotations,
|
||||
Labels: input.Labels,
|
||||
SchemaVersion: SchemaVersion,
|
||||
}
|
||||
|
||||
if err := entry.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if EntryExists(entry.Name) {
|
||||
existing, err := LoadEntry(entry.Name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if existing.Scheduler != entry.Scheduler {
|
||||
return fmt.Errorf("storage entry %q already exists with scheduler %q; refusing to redefine", entry.Name, existing.Scheduler)
|
||||
}
|
||||
common.LogInfo1Quiet(fmt.Sprintf("Storage entry %s already exists, leaving in place", entry.Name))
|
||||
}
|
||||
|
||||
if scheduler == SchedulerDockerLocal {
|
||||
if err := ensureDockerLocalPath(entry); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
if err := SaveEntry(entry); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if scheduler == SchedulerK3s {
|
||||
if err := callSchedulerCreateTrigger(entry); err != nil {
|
||||
// Roll back the on-disk entry so the disk and the cluster stay in sync.
|
||||
_ = DeleteEntry(entry.Name)
|
||||
return fmt.Errorf("scheduler refused storage entry %q: %w", entry.Name, err)
|
||||
}
|
||||
}
|
||||
|
||||
common.LogInfo1(fmt.Sprintf("Storage entry %s created", entry.Name))
|
||||
return nil
|
||||
}
|
||||
|
||||
// CommandDestroy removes a registered storage entry. Refuses to remove
|
||||
// an entry that any app still has attached.
|
||||
func CommandDestroy(name string) error {
|
||||
if name == "" {
|
||||
return errors.New("storage entry name is required")
|
||||
}
|
||||
if !EntryExists(name) {
|
||||
return fmt.Errorf("storage entry %q does not exist", name)
|
||||
}
|
||||
|
||||
using, err := AppsUsingEntry(name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(using) > 0 {
|
||||
return fmt.Errorf("storage entry %q is still mounted by app(s): %s", name, strings.Join(using, ", "))
|
||||
}
|
||||
|
||||
entry, err := LoadEntry(name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if entry.Scheduler == SchedulerK3s {
|
||||
if err := callSchedulerDestroyTrigger(entry); err != nil {
|
||||
return fmt.Errorf("scheduler refused to remove storage entry %q: %w", name, err)
|
||||
}
|
||||
}
|
||||
|
||||
if err := DeleteEntry(name); err != nil {
|
||||
return err
|
||||
}
|
||||
common.LogInfo1(fmt.Sprintf("Storage entry %s destroyed", name))
|
||||
return nil
|
||||
}
|
||||
|
||||
// CommandInfo prints a single entry's details, optionally as JSON.
|
||||
func CommandInfo(name string, format string) error {
|
||||
if name == "" {
|
||||
return errors.New("storage entry name is required")
|
||||
}
|
||||
if !EntryExists(name) {
|
||||
return fmt.Errorf("storage entry %q does not exist", name)
|
||||
}
|
||||
|
||||
entry, err := LoadEntry(name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if format == "json" {
|
||||
data, err := json.MarshalIndent(entry, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Println(string(data))
|
||||
return nil
|
||||
}
|
||||
|
||||
common.LogInfo1Quiet(fmt.Sprintf("Storage entry %s", entry.Name))
|
||||
common.LogVerbose(fmt.Sprintf("Scheduler: %s", entry.Scheduler))
|
||||
if entry.HostPath != "" {
|
||||
common.LogVerbose(fmt.Sprintf("Host path: %s", entry.HostPath))
|
||||
}
|
||||
if entry.Size != "" {
|
||||
common.LogVerbose(fmt.Sprintf("Size: %s", entry.Size))
|
||||
}
|
||||
if entry.AccessMode != "" {
|
||||
common.LogVerbose(fmt.Sprintf("Access mode: %s", entry.AccessMode))
|
||||
}
|
||||
if entry.StorageClass != "" {
|
||||
common.LogVerbose(fmt.Sprintf("Storage class: %s", entry.StorageClass))
|
||||
}
|
||||
if entry.Namespace != "" {
|
||||
common.LogVerbose(fmt.Sprintf("Namespace: %s", entry.Namespace))
|
||||
}
|
||||
if entry.Chown != "" {
|
||||
common.LogVerbose(fmt.Sprintf("Chown: %s", entry.Chown))
|
||||
}
|
||||
if entry.ReclaimPolicy != "" {
|
||||
common.LogVerbose(fmt.Sprintf("Reclaim policy: %s", entry.ReclaimPolicy))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// CommandSetInput captures the flags accepted by storage:set.
|
||||
type CommandSetInput struct {
|
||||
Name string
|
||||
Size string
|
||||
AccessMode string
|
||||
StorageClass string
|
||||
Namespace string
|
||||
Chown string
|
||||
ReclaimPolicy string
|
||||
Annotations map[string]string
|
||||
Labels map[string]string
|
||||
}
|
||||
|
||||
// CommandSet edits an existing entry's mutable fields and re-fires the
|
||||
// scheduler-side helm release. Refuses changes Kubernetes can't apply
|
||||
// in place (access-mode swap, storage-class swap, size shrink).
|
||||
func CommandSet(input CommandSetInput) error {
|
||||
if !EntryExists(input.Name) {
|
||||
return fmt.Errorf("storage entry %q does not exist", input.Name)
|
||||
}
|
||||
entry, err := LoadEntry(input.Name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if input.AccessMode != "" && input.AccessMode != entry.AccessMode {
|
||||
return fmt.Errorf("storage:set cannot change access-mode in place; recreate the entry")
|
||||
}
|
||||
if input.StorageClass != "" && input.StorageClass != entry.StorageClass {
|
||||
return fmt.Errorf("storage:set cannot change storage-class-name in place; recreate the entry")
|
||||
}
|
||||
if input.Size != "" {
|
||||
entry.Size = input.Size
|
||||
}
|
||||
if input.Namespace != "" {
|
||||
entry.Namespace = input.Namespace
|
||||
}
|
||||
if input.Chown != "" {
|
||||
entry.Chown = input.Chown
|
||||
}
|
||||
if input.ReclaimPolicy != "" {
|
||||
entry.ReclaimPolicy = input.ReclaimPolicy
|
||||
}
|
||||
if input.Annotations != nil {
|
||||
entry.Annotations = input.Annotations
|
||||
}
|
||||
if input.Labels != nil {
|
||||
entry.Labels = input.Labels
|
||||
}
|
||||
|
||||
if err := entry.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := SaveEntry(entry); err != nil {
|
||||
return err
|
||||
}
|
||||
if entry.Scheduler == SchedulerK3s {
|
||||
if err := callSchedulerCreateTrigger(entry); err != nil {
|
||||
return fmt.Errorf("scheduler refused storage:set for %q: %w", entry.Name, err)
|
||||
}
|
||||
}
|
||||
common.LogInfo1(fmt.Sprintf("Storage entry %s updated", entry.Name))
|
||||
return nil
|
||||
}
|
||||
|
||||
// CommandExecInput captures the storage:exec subcommand inputs.
|
||||
type CommandExecInput struct {
|
||||
Name string
|
||||
Image string
|
||||
AsUser string
|
||||
Args []string
|
||||
}
|
||||
|
||||
// CommandExec delegates the actual exec to the scheduler plugin that owns
|
||||
// the storage entry, by firing the scheduler-storage-exec plugn trigger
|
||||
// with stdin/stdout/stderr streamed through. The handler decides between
|
||||
// docker run, the k8s exec SDK, etc. - storage stays scheduler-agnostic.
|
||||
//
|
||||
// Exit codes from the underlying tool are propagated verbatim via
|
||||
// os.Exit so callers in scripts see the right status.
|
||||
func CommandExec(input CommandExecInput) error {
|
||||
if !EntryExists(input.Name) {
|
||||
return fmt.Errorf("storage entry %q does not exist", input.Name)
|
||||
}
|
||||
entry, err := LoadEntry(input.Name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
image := input.Image
|
||||
if image == "" {
|
||||
image = "alpine:3"
|
||||
}
|
||||
|
||||
interactive, tty := stdinModes()
|
||||
|
||||
triggerArgs := []string{
|
||||
entry.Scheduler,
|
||||
entry.Name,
|
||||
image,
|
||||
}
|
||||
triggerArgs = append(triggerArgs, fmt.Sprintf("--interactive=%t", interactive))
|
||||
triggerArgs = append(triggerArgs, fmt.Sprintf("--tty=%t", tty))
|
||||
if input.AsUser != "" {
|
||||
triggerArgs = append(triggerArgs, "--as-user", input.AsUser)
|
||||
}
|
||||
if len(input.Args) > 0 {
|
||||
triggerArgs = append(triggerArgs, "--")
|
||||
triggerArgs = append(triggerArgs, input.Args...)
|
||||
}
|
||||
|
||||
results, err := common.CallPlugnTrigger(common.PlugnTriggerInput{
|
||||
Trigger: "scheduler-storage-exec",
|
||||
Args: triggerArgs,
|
||||
StreamStdio: true,
|
||||
})
|
||||
// common.CallExecCommand wraps a non-zero exit code as an error, but
|
||||
// for storage:exec the underlying tool's exit code IS the signal we
|
||||
// want to surface - it's how `dokku storage:exec demo -- exit 42`
|
||||
// reaches the user as exit 42. Check ExitCode first so that branch
|
||||
// runs before err collapses to LogFailWithError → exit 1.
|
||||
if results.ExitCode != 0 {
|
||||
os.Exit(results.ExitCode)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// stdinModes inspects os.Stdin to decide whether docker / the k8s exec
|
||||
// SDK should request an interactive session and a TTY.
|
||||
func stdinModes() (interactive bool, tty bool) {
|
||||
fi, err := os.Stdin.Stat()
|
||||
if err != nil || fi == nil {
|
||||
return false, false
|
||||
}
|
||||
mode := fi.Mode()
|
||||
tty = mode&os.ModeCharDevice != 0
|
||||
hasStdinData := mode&os.ModeNamedPipe != 0
|
||||
interactive = tty || hasStdinData
|
||||
return interactive, tty
|
||||
}
|
||||
|
||||
// CommandMigrate re-runs the legacy `-v` → entry/attachment migration
|
||||
// for a single app or every app on the install. Useful when an operator
|
||||
// has restored an app from a backup that predates the install-time
|
||||
// migration, or when they've added `-v` lines via docker-options:add
|
||||
// after the per-app flag was set.
|
||||
func CommandMigrate(appName string, all bool) error {
|
||||
if all && appName != "" {
|
||||
return errors.New("storage:migrate accepts either an app name or --all, not both")
|
||||
}
|
||||
if !all && appName == "" {
|
||||
return errors.New("storage:migrate requires an app name (or --all)")
|
||||
}
|
||||
if all {
|
||||
apps, err := common.DokkuApps()
|
||||
if err != nil {
|
||||
if errors.Is(err, common.NoAppsExist) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
for _, app := range apps {
|
||||
if err := MigrateApp(app); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
common.LogInfo1(fmt.Sprintf("Re-migrated %d app(s)", len(apps)))
|
||||
return nil
|
||||
}
|
||||
if err := common.VerifyAppName(appName); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := MigrateApp(appName); err != nil {
|
||||
return err
|
||||
}
|
||||
common.LogInfo1(fmt.Sprintf("Re-migrated %s", appName))
|
||||
return nil
|
||||
}
|
||||
|
||||
// CommandWait blocks until a k3s storage entry's PVC is bound.
|
||||
func CommandWait(name string) error {
|
||||
if !EntryExists(name) {
|
||||
return fmt.Errorf("storage entry %q does not exist", name)
|
||||
}
|
||||
entry, err := LoadEntry(name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if entry.Scheduler != SchedulerK3s {
|
||||
// Docker-local entries are immediately "ready" once the directory exists.
|
||||
if entry.HostPath != "" {
|
||||
info, statErr := os.Stat(entry.HostPath)
|
||||
if statErr != nil || !info.IsDir() {
|
||||
return fmt.Errorf("storage entry %q host path %s is not present", name, entry.HostPath)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
data, err := json.Marshal(entry)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
results, err := common.CallPlugnTrigger(common.PlugnTriggerInput{
|
||||
Trigger: "storage-status",
|
||||
Args: []string{entry.Name},
|
||||
Stdin: strings.NewReader(string(data)),
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
status := strings.TrimSpace(results.StdoutContents())
|
||||
if status != "Bound" {
|
||||
return fmt.Errorf("storage entry %q is %s, not Bound", name, status)
|
||||
}
|
||||
common.LogInfo1(fmt.Sprintf("Storage entry %s is bound", name))
|
||||
return nil
|
||||
}
|
||||
|
||||
// CommandReportGlobal prints a global storage report listing every entry
|
||||
// and the apps that mount it. Output is text by default; pass
|
||||
// format="json" for machine-readable output.
|
||||
func CommandReportGlobal(format string) error {
|
||||
entries, err := ListEntries()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
type entryWithUse struct {
|
||||
Entry *Entry `json:"entry"`
|
||||
MountedBy []string `json:"mounted_by"`
|
||||
}
|
||||
rows := []entryWithUse{}
|
||||
for _, entry := range entries {
|
||||
using, err := AppsUsingEntry(entry.Name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rows = append(rows, entryWithUse{Entry: entry, MountedBy: using})
|
||||
}
|
||||
if format == "json" {
|
||||
data, err := json.MarshalIndent(rows, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Println(string(data))
|
||||
return nil
|
||||
}
|
||||
if len(rows) == 0 {
|
||||
common.LogInfo1Quiet("No storage entries registered")
|
||||
return nil
|
||||
}
|
||||
common.LogInfo1Quiet("Storage report (global):")
|
||||
for _, row := range rows {
|
||||
mountedBy := "—"
|
||||
if len(row.MountedBy) > 0 {
|
||||
mountedBy = strings.Join(row.MountedBy, ", ")
|
||||
}
|
||||
common.LogVerbose(fmt.Sprintf("%s\t%s\tmounted by: %s", row.Entry.Name, row.Entry.Scheduler, mountedBy))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// CommandListEntries lists registered entries (distinct from the legacy
|
||||
// per-app storage:list which lists colon-form mounts).
|
||||
func CommandListEntries(scheduler string, format string) error {
|
||||
entries, err := ListEntries()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
filtered := []*Entry{}
|
||||
for _, entry := range entries {
|
||||
if scheduler != "" && entry.Scheduler != scheduler {
|
||||
continue
|
||||
}
|
||||
filtered = append(filtered, entry)
|
||||
}
|
||||
|
||||
if format == "json" {
|
||||
data, err := json.MarshalIndent(filtered, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Println(string(data))
|
||||
return nil
|
||||
}
|
||||
|
||||
if len(filtered) == 0 {
|
||||
common.LogInfo1Quiet("No storage entries registered")
|
||||
return nil
|
||||
}
|
||||
common.LogInfo1Quiet("Storage entries:")
|
||||
for _, entry := range filtered {
|
||||
common.LogVerbose(fmt.Sprintf("%s\t%s\t%s", entry.Name, entry.Scheduler, entry.HostPath))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ensureDockerLocalPath creates the host directory referenced by a
|
||||
// docker-local entry if it doesn't already exist. Idempotent: a
|
||||
// pre-existing directory is left in place.
|
||||
func ensureDockerLocalPath(entry *Entry) error {
|
||||
info, err := os.Stat(entry.HostPath)
|
||||
if err != nil && !os.IsNotExist(err) {
|
||||
return fmt.Errorf("unable to stat %s: %w", entry.HostPath, err)
|
||||
}
|
||||
if err == nil && !info.IsDir() {
|
||||
return fmt.Errorf("storage entry path %s exists but is not a directory", entry.HostPath)
|
||||
}
|
||||
if err != nil {
|
||||
if mkdirErr := os.MkdirAll(entry.HostPath, 0755); mkdirErr != nil {
|
||||
return fmt.Errorf("unable to create %s: %w", entry.HostPath, mkdirErr)
|
||||
}
|
||||
common.LogVerbose(fmt.Sprintf("Created %s", entry.HostPath))
|
||||
}
|
||||
|
||||
if entry.Chown != "" && entry.Chown != "false" {
|
||||
chownID, err := ResolveChownID(entry.Chown)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if chownID != "false" {
|
||||
pluginPath := common.MustGetEnv("PLUGIN_AVAILABLE_PATH")
|
||||
chownScript := filepath.Join(pluginPath, "storage", "bin", "chown-storage-dir")
|
||||
result, err := common.CallExecCommand(common.ExecCommandInput{
|
||||
Command: "sudo",
|
||||
Args: []string{chownScript, entry.HostPath, chownID},
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("unable to chown %s: %w", entry.HostPath, err)
|
||||
}
|
||||
if result.ExitCode != 0 {
|
||||
return fmt.Errorf("unable to chown %s: %s", entry.HostPath, result.StderrContents())
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// callSchedulerCreateTrigger asks the scheduler plugin (k3s) to provision
|
||||
// the underlying PVC/PV. The scheduler is responsible for any
|
||||
// cluster-level validation (storage class existence, etc.).
|
||||
func callSchedulerCreateTrigger(entry *Entry) error {
|
||||
data, err := json.Marshal(entry)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
results, err := common.CallPlugnTrigger(common.PlugnTriggerInput{
|
||||
Trigger: "storage-create",
|
||||
Args: []string{entry.Name},
|
||||
StreamStdio: true,
|
||||
Stdin: strings.NewReader(string(data)),
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if results.ExitCode != 0 {
|
||||
return fmt.Errorf("storage-create trigger exited with %d: %s", results.ExitCode, results.StderrContents())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// callSchedulerDestroyTrigger asks the scheduler plugin to release the
|
||||
// underlying PVC/PV.
|
||||
func callSchedulerDestroyTrigger(entry *Entry) error {
|
||||
data, err := json.Marshal(entry)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
results, err := common.CallPlugnTrigger(common.PlugnTriggerInput{
|
||||
Trigger: "storage-destroy",
|
||||
Args: []string{entry.Name},
|
||||
StreamStdio: true,
|
||||
Stdin: strings.NewReader(string(data)),
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if results.ExitCode != 0 {
|
||||
return fmt.Errorf("storage-destroy trigger exited with %d: %s", results.ExitCode, results.StderrContents())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
294
plugins/storage/entry.go
Normal file
294
plugins/storage/entry.go
Normal file
@@ -0,0 +1,294 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"crypto/sha1"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"github.com/dokku/dokku/plugins/common"
|
||||
)
|
||||
|
||||
// SchemaVersion is the on-disk schema version emitted by this plugin.
|
||||
// Bumped when the format of Entry / Attachment changes incompatibly.
|
||||
const SchemaVersion = 1
|
||||
|
||||
// SchedulerDockerLocal is the default scheduler value for storage entries.
|
||||
const SchedulerDockerLocal = "docker-local"
|
||||
|
||||
// SchedulerK3s is the k3s scheduler value for storage entries.
|
||||
const SchedulerK3s = "k3s"
|
||||
|
||||
// LegacyEntryPrefix is reserved for migration-synthesized entries; users
|
||||
// cannot create entries whose name starts with this prefix.
|
||||
const LegacyEntryPrefix = "legacy-"
|
||||
|
||||
// MaxEntryNameLength caps the storage entry name so that the helm release
|
||||
// name "storage-<name>" stays within Helm's practical 53-char ceiling.
|
||||
const MaxEntryNameLength = 45
|
||||
|
||||
// ReclaimPolicyRetain keeps the underlying PV after the entry is destroyed.
|
||||
const ReclaimPolicyRetain = "Retain"
|
||||
|
||||
// ReclaimPolicyDelete deletes the underlying PV after the entry is destroyed.
|
||||
const ReclaimPolicyDelete = "Delete"
|
||||
|
||||
var (
|
||||
// dns1123LabelRegexp matches a Kubernetes DNS-1123 label, the format
|
||||
// required for resource names like PVCs and Helm releases.
|
||||
dns1123LabelRegexp = regexp.MustCompile(`^[a-z0-9]([-a-z0-9]*[a-z0-9])?$`)
|
||||
|
||||
// dockerNamedVolumeRegexp matches a Docker named-volume token. Same
|
||||
// rule docker uses to disambiguate volume names from bind paths.
|
||||
dockerNamedVolumeRegexp = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9_.-]+$`)
|
||||
|
||||
// supportedSchedulers lists the values accepted for --scheduler.
|
||||
supportedSchedulers = map[string]bool{
|
||||
SchedulerDockerLocal: true,
|
||||
SchedulerK3s: true,
|
||||
}
|
||||
|
||||
// supportedAccessModes lists the values accepted for --access-mode on k3s.
|
||||
supportedAccessModes = map[string]bool{
|
||||
"ReadWriteOnce": true,
|
||||
"ReadOnlyMany": true,
|
||||
"ReadWriteMany": true,
|
||||
"ReadWriteOncePod": true,
|
||||
}
|
||||
)
|
||||
|
||||
// Entry is the source of truth for a storage volume. One file per entry
|
||||
// lives at $DOKKU_LIB_ROOT/config/storage/entries/<name>.json.
|
||||
type Entry struct {
|
||||
Name string `json:"name"`
|
||||
Scheduler string `json:"scheduler"`
|
||||
HostPath string `json:"host_path,omitempty"`
|
||||
Size string `json:"size,omitempty"`
|
||||
AccessMode string `json:"access_mode,omitempty"`
|
||||
StorageClass string `json:"storage_class,omitempty"`
|
||||
Namespace string `json:"namespace,omitempty"`
|
||||
Chown string `json:"chown,omitempty"`
|
||||
ReclaimPolicy string `json:"reclaim_policy,omitempty"`
|
||||
Annotations map[string]string `json:"annotations,omitempty"`
|
||||
Labels map[string]string `json:"labels,omitempty"`
|
||||
SchemaVersion int `json:"schema_version"`
|
||||
}
|
||||
|
||||
// RegistryDirectory returns the parent directory for storage-plugin
|
||||
// state that lives outside the property store. Kept under data/ rather
|
||||
// than config/ so it doesn't collide with the property-list paths
|
||||
// `config/storage/<appName>/<property>` that the rest of the plugin
|
||||
// relies on for per-app attachments.
|
||||
func RegistryDirectory() string {
|
||||
root := common.GetenvWithDefault("DOKKU_LIB_ROOT", "/var/lib/dokku")
|
||||
return filepath.Join(root, "data", "storage-registry")
|
||||
}
|
||||
|
||||
// EntriesDirectory returns the directory holding per-entry JSON files.
|
||||
func EntriesDirectory() string {
|
||||
return filepath.Join(RegistryDirectory(), "entries")
|
||||
}
|
||||
|
||||
// entryPath returns the on-disk path for a named entry.
|
||||
func entryPath(name string) string {
|
||||
return filepath.Join(EntriesDirectory(), name+".json")
|
||||
}
|
||||
|
||||
// EnsureEntriesDirectory makes sure the entries directory exists with
|
||||
// permissive enough mode that the dokku user can read it.
|
||||
func EnsureEntriesDirectory() error {
|
||||
dir := EntriesDirectory()
|
||||
if err := os.MkdirAll(dir, 0755); err != nil {
|
||||
return fmt.Errorf("unable to create storage entries directory: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// EntryExists reports whether an entry with the given name is registered.
|
||||
func EntryExists(name string) bool {
|
||||
_, err := os.Stat(entryPath(name))
|
||||
return err == nil
|
||||
}
|
||||
|
||||
// LoadEntry reads a registered entry by name.
|
||||
func LoadEntry(name string) (*Entry, error) {
|
||||
data, err := os.ReadFile(entryPath(name))
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil, fmt.Errorf("storage entry %q does not exist", name)
|
||||
}
|
||||
return nil, fmt.Errorf("unable to read storage entry %q: %w", name, err)
|
||||
}
|
||||
|
||||
entry := &Entry{}
|
||||
if err := json.Unmarshal(data, entry); err != nil {
|
||||
return nil, fmt.Errorf("unable to parse storage entry %q: %w", name, err)
|
||||
}
|
||||
return entry, nil
|
||||
}
|
||||
|
||||
// SaveEntry persists an entry to its JSON file.
|
||||
func SaveEntry(entry *Entry) error {
|
||||
if err := EnsureEntriesDirectory(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if entry.SchemaVersion == 0 {
|
||||
entry.SchemaVersion = SchemaVersion
|
||||
}
|
||||
|
||||
data, err := json.MarshalIndent(entry, "", " ")
|
||||
if err != nil {
|
||||
return fmt.Errorf("unable to encode storage entry %q: %w", entry.Name, err)
|
||||
}
|
||||
|
||||
path := entryPath(entry.Name)
|
||||
if err := os.WriteFile(path, data, 0600); err != nil {
|
||||
return fmt.Errorf("unable to write storage entry %q: %w", entry.Name, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// DeleteEntry removes the on-disk record for an entry.
|
||||
func DeleteEntry(name string) error {
|
||||
if err := os.Remove(entryPath(name)); err != nil && !os.IsNotExist(err) {
|
||||
return fmt.Errorf("unable to remove storage entry %q: %w", name, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ListEntries returns every registered entry sorted by name.
|
||||
func ListEntries() ([]*Entry, error) {
|
||||
dir := EntriesDirectory()
|
||||
dirEntries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil, nil
|
||||
}
|
||||
return nil, fmt.Errorf("unable to list storage entries: %w", err)
|
||||
}
|
||||
|
||||
entries := []*Entry{}
|
||||
for _, dirEntry := range dirEntries {
|
||||
if dirEntry.IsDir() || !strings.HasSuffix(dirEntry.Name(), ".json") {
|
||||
continue
|
||||
}
|
||||
name := strings.TrimSuffix(dirEntry.Name(), ".json")
|
||||
entry, err := LoadEntry(name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
entries = append(entries, entry)
|
||||
}
|
||||
|
||||
sort.Slice(entries, func(i, j int) bool { return entries[i].Name < entries[j].Name })
|
||||
return entries, nil
|
||||
}
|
||||
|
||||
// ValidateEntryName checks the user-supplied name against length and DNS
|
||||
// label rules. Returns nil for legacy names if allowLegacyPrefix is true,
|
||||
// which is only set during migration.
|
||||
func ValidateEntryName(name string, allowLegacyPrefix bool) error {
|
||||
if name == "" {
|
||||
return errors.New("storage entry name is required")
|
||||
}
|
||||
if len(name) > MaxEntryNameLength {
|
||||
return fmt.Errorf("storage entry name %q is too long (max %d characters); helm release names cap our budget", name, MaxEntryNameLength)
|
||||
}
|
||||
if !dns1123LabelRegexp.MatchString(name) {
|
||||
return fmt.Errorf("storage entry name %q must be a DNS-1123 label (lowercase letters, digits, and dashes; cannot start or end with a dash)", name)
|
||||
}
|
||||
if !allowLegacyPrefix && strings.HasPrefix(name, LegacyEntryPrefix) {
|
||||
return fmt.Errorf("storage entry name %q must not start with reserved prefix %q", name, LegacyEntryPrefix)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Validate checks an Entry's fields against the cross-field rules for its
|
||||
// scheduler. It is reused by both storage:create and the legacy migration.
|
||||
func (e *Entry) Validate() error {
|
||||
if e == nil {
|
||||
return errors.New("entry is nil")
|
||||
}
|
||||
if err := ValidateEntryName(e.Name, strings.HasPrefix(e.Name, LegacyEntryPrefix)); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if !supportedSchedulers[e.Scheduler] {
|
||||
return fmt.Errorf("storage entry %q has unsupported scheduler %q (supported: docker-local, k3s)", e.Name, e.Scheduler)
|
||||
}
|
||||
|
||||
switch e.Scheduler {
|
||||
case SchedulerDockerLocal:
|
||||
if e.HostPath == "" {
|
||||
return fmt.Errorf("storage entry %q (docker-local) is missing host_path", e.Name)
|
||||
}
|
||||
if !filepath.IsAbs(e.HostPath) && !dockerNamedVolumeRegexp.MatchString(e.HostPath) {
|
||||
return fmt.Errorf("storage entry %q host_path must be an absolute path or a docker named volume token, got %q", e.Name, e.HostPath)
|
||||
}
|
||||
if e.Size != "" {
|
||||
return fmt.Errorf("storage entry %q (docker-local) does not accept --size", e.Name)
|
||||
}
|
||||
if e.StorageClass != "" {
|
||||
return fmt.Errorf("storage entry %q (docker-local) does not accept --storage-class-name", e.Name)
|
||||
}
|
||||
if e.AccessMode != "" {
|
||||
return fmt.Errorf("storage entry %q (docker-local) does not accept --access-mode", e.Name)
|
||||
}
|
||||
case SchedulerK3s:
|
||||
if e.Size == "" {
|
||||
return fmt.Errorf("storage entry %q (k3s) requires --size", e.Name)
|
||||
}
|
||||
if e.StorageClass != "" && e.HostPath != "" {
|
||||
return fmt.Errorf("storage entry %q (k3s) cannot set both --storage-class-name and a host path; the cluster provisions class-backed volumes", e.Name)
|
||||
}
|
||||
if e.AccessMode != "" && !supportedAccessModes[e.AccessMode] {
|
||||
return fmt.Errorf("storage entry %q has unsupported access mode %q", e.Name, e.AccessMode)
|
||||
}
|
||||
if e.HostPath != "" && !filepath.IsAbs(e.HostPath) {
|
||||
return fmt.Errorf("storage entry %q host_path must be absolute, got %q", e.Name, e.HostPath)
|
||||
}
|
||||
if e.ReclaimPolicy != "" && e.ReclaimPolicy != ReclaimPolicyRetain && e.ReclaimPolicy != ReclaimPolicyDelete {
|
||||
return fmt.Errorf("storage entry %q has unsupported reclaim policy %q", e.Name, e.ReclaimPolicy)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// LegacyMountToEntry synthesizes a deterministic Entry for a legacy
|
||||
// "host:container[:options]" mount string. The synthesized name is
|
||||
// stable across runs so re-running the migration is idempotent and so
|
||||
// two apps mounting the same host path converge on a single entry.
|
||||
func LegacyMountToEntry(mountPath string) *Entry {
|
||||
parsed := ParseMountPath(mountPath)
|
||||
host := parsed.HostPath
|
||||
|
||||
hashInput := host
|
||||
if !strings.HasPrefix(host, "/") {
|
||||
hashInput = "vol:" + host
|
||||
}
|
||||
sum := sha1.Sum([]byte(hashInput))
|
||||
name := LegacyEntryPrefix + hex.EncodeToString(sum[:])[:10]
|
||||
|
||||
hostPath := host
|
||||
if !strings.HasPrefix(host, "/") {
|
||||
// Named docker volumes pass through unchanged on docker-local;
|
||||
// the docker engine resolves them. Store as-is so the args
|
||||
// shim can emit the original token.
|
||||
hostPath = host
|
||||
}
|
||||
|
||||
return &Entry{
|
||||
Name: name,
|
||||
Scheduler: SchedulerDockerLocal,
|
||||
HostPath: hostPath,
|
||||
SchemaVersion: SchemaVersion,
|
||||
}
|
||||
}
|
||||
205
plugins/storage/entry_test.go
Normal file
205
plugins/storage/entry_test.go
Normal file
@@ -0,0 +1,205 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
. "github.com/onsi/gomega"
|
||||
)
|
||||
|
||||
func withTempLibRoot(t *testing.T) string {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
t.Setenv("DOKKU_LIB_ROOT", dir)
|
||||
return dir
|
||||
}
|
||||
|
||||
func TestValidateEntryName(t *testing.T) {
|
||||
RegisterTestingT(t)
|
||||
|
||||
Expect(ValidateEntryName("foo", false)).To(Succeed())
|
||||
Expect(ValidateEntryName("foo-bar", false)).To(Succeed())
|
||||
Expect(ValidateEntryName("a", false)).To(Succeed())
|
||||
Expect(ValidateEntryName("foo123", false)).To(Succeed())
|
||||
|
||||
err := ValidateEntryName("", false)
|
||||
Expect(err).To(HaveOccurred())
|
||||
Expect(err.Error()).To(ContainSubstring("required"))
|
||||
|
||||
err = ValidateEntryName("Foo", false)
|
||||
Expect(err).To(HaveOccurred())
|
||||
Expect(err.Error()).To(ContainSubstring("DNS-1123"))
|
||||
|
||||
err = ValidateEntryName("foo_bar", false)
|
||||
Expect(err).To(HaveOccurred())
|
||||
Expect(err.Error()).To(ContainSubstring("DNS-1123"))
|
||||
|
||||
err = ValidateEntryName("-foo", false)
|
||||
Expect(err).To(HaveOccurred())
|
||||
|
||||
err = ValidateEntryName("foo-", false)
|
||||
Expect(err).To(HaveOccurred())
|
||||
|
||||
long := strings.Repeat("a", MaxEntryNameLength+1)
|
||||
err = ValidateEntryName(long, false)
|
||||
Expect(err).To(HaveOccurred())
|
||||
Expect(err.Error()).To(ContainSubstring("too long"))
|
||||
|
||||
maxOK := strings.Repeat("a", MaxEntryNameLength)
|
||||
Expect(ValidateEntryName(maxOK, false)).To(Succeed())
|
||||
|
||||
err = ValidateEntryName("legacy-foo", false)
|
||||
Expect(err).To(HaveOccurred())
|
||||
Expect(err.Error()).To(ContainSubstring("reserved"))
|
||||
|
||||
Expect(ValidateEntryName("legacy-foo", true)).To(Succeed())
|
||||
}
|
||||
|
||||
func TestEntryValidateDockerLocal(t *testing.T) {
|
||||
RegisterTestingT(t)
|
||||
|
||||
good := &Entry{Name: "foo", Scheduler: SchedulerDockerLocal, HostPath: "/var/lib/dokku/data/storage/foo"}
|
||||
Expect(good.Validate()).To(Succeed())
|
||||
|
||||
noPath := &Entry{Name: "foo", Scheduler: SchedulerDockerLocal}
|
||||
Expect(noPath.Validate()).To(HaveOccurred())
|
||||
|
||||
// A leading-slash-less but DNS-1123-ish token is treated as a docker
|
||||
// named volume and accepted.
|
||||
namedVolume := &Entry{Name: "foo", Scheduler: SchedulerDockerLocal, HostPath: "myvolume"}
|
||||
Expect(namedVolume.Validate()).To(Succeed())
|
||||
|
||||
// Slash-containing relative paths still fail, as do tokens with bad chars.
|
||||
relativePath := &Entry{Name: "foo", Scheduler: SchedulerDockerLocal, HostPath: "relative/path"}
|
||||
Expect(relativePath.Validate()).To(HaveOccurred())
|
||||
|
||||
badToken := &Entry{Name: "foo", Scheduler: SchedulerDockerLocal, HostPath: "with spaces"}
|
||||
Expect(badToken.Validate()).To(HaveOccurred())
|
||||
|
||||
withSize := &Entry{Name: "foo", Scheduler: SchedulerDockerLocal, HostPath: "/data", Size: "2Gi"}
|
||||
Expect(withSize.Validate()).To(HaveOccurred())
|
||||
|
||||
withClass := &Entry{Name: "foo", Scheduler: SchedulerDockerLocal, HostPath: "/data", StorageClass: "longhorn"}
|
||||
Expect(withClass.Validate()).To(HaveOccurred())
|
||||
}
|
||||
|
||||
func TestEntryValidateK3s(t *testing.T) {
|
||||
RegisterTestingT(t)
|
||||
|
||||
dynamic := &Entry{Name: "foo", Scheduler: SchedulerK3s, Size: "2Gi", StorageClass: "longhorn", AccessMode: "ReadWriteOnce"}
|
||||
Expect(dynamic.Validate()).To(Succeed())
|
||||
|
||||
hostPath := &Entry{Name: "foo", Scheduler: SchedulerK3s, Size: "2Gi", HostPath: "/data/foo"}
|
||||
Expect(hostPath.Validate()).To(Succeed())
|
||||
|
||||
noSize := &Entry{Name: "foo", Scheduler: SchedulerK3s, StorageClass: "longhorn"}
|
||||
Expect(noSize.Validate()).To(HaveOccurred())
|
||||
|
||||
bothPathAndClass := &Entry{Name: "foo", Scheduler: SchedulerK3s, Size: "2Gi", HostPath: "/data", StorageClass: "longhorn"}
|
||||
err := bothPathAndClass.Validate()
|
||||
Expect(err).To(HaveOccurred())
|
||||
Expect(err.Error()).To(ContainSubstring("storage-class-name"))
|
||||
|
||||
badAccessMode := &Entry{Name: "foo", Scheduler: SchedulerK3s, Size: "2Gi", StorageClass: "longhorn", AccessMode: "Bogus"}
|
||||
Expect(badAccessMode.Validate()).To(HaveOccurred())
|
||||
|
||||
badReclaim := &Entry{Name: "foo", Scheduler: SchedulerK3s, Size: "2Gi", StorageClass: "longhorn", ReclaimPolicy: "Recycle"}
|
||||
Expect(badReclaim.Validate()).To(HaveOccurred())
|
||||
}
|
||||
|
||||
func TestEntryValidateScheduler(t *testing.T) {
|
||||
RegisterTestingT(t)
|
||||
|
||||
bad := &Entry{Name: "foo", Scheduler: "nomad"}
|
||||
err := bad.Validate()
|
||||
Expect(err).To(HaveOccurred())
|
||||
Expect(err.Error()).To(ContainSubstring("scheduler"))
|
||||
}
|
||||
|
||||
func TestEntryRoundTrip(t *testing.T) {
|
||||
RegisterTestingT(t)
|
||||
root := withTempLibRoot(t)
|
||||
|
||||
original := &Entry{
|
||||
Name: "demo-data",
|
||||
Scheduler: SchedulerK3s,
|
||||
Size: "2Gi",
|
||||
StorageClass: "longhorn",
|
||||
AccessMode: "ReadWriteOnce",
|
||||
Namespace: "dokku",
|
||||
Annotations: map[string]string{"backup.velero.io/backup-volumes": "demo-data"},
|
||||
Labels: map[string]string{"app.kubernetes.io/managed-by": "dokku"},
|
||||
}
|
||||
|
||||
Expect(SaveEntry(original)).To(Succeed())
|
||||
Expect(EntryExists("demo-data")).To(BeTrue())
|
||||
|
||||
loaded, err := LoadEntry("demo-data")
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(loaded.Name).To(Equal(original.Name))
|
||||
Expect(loaded.Scheduler).To(Equal(original.Scheduler))
|
||||
Expect(loaded.Size).To(Equal(original.Size))
|
||||
Expect(loaded.StorageClass).To(Equal(original.StorageClass))
|
||||
Expect(loaded.AccessMode).To(Equal(original.AccessMode))
|
||||
Expect(loaded.Namespace).To(Equal(original.Namespace))
|
||||
Expect(loaded.Annotations).To(Equal(original.Annotations))
|
||||
Expect(loaded.Labels).To(Equal(original.Labels))
|
||||
Expect(loaded.SchemaVersion).To(Equal(SchemaVersion))
|
||||
|
||||
expectedPath := filepath.Join(root, "data", "storage-registry", "entries", "demo-data.json")
|
||||
Expect(expectedPath).To(BeARegularFile())
|
||||
|
||||
Expect(DeleteEntry("demo-data")).To(Succeed())
|
||||
Expect(EntryExists("demo-data")).To(BeFalse())
|
||||
}
|
||||
|
||||
func TestListEntries(t *testing.T) {
|
||||
RegisterTestingT(t)
|
||||
withTempLibRoot(t)
|
||||
|
||||
entries, err := ListEntries()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(entries).To(BeEmpty())
|
||||
|
||||
Expect(SaveEntry(&Entry{Name: "b", Scheduler: SchedulerDockerLocal, HostPath: "/b"})).To(Succeed())
|
||||
Expect(SaveEntry(&Entry{Name: "a", Scheduler: SchedulerDockerLocal, HostPath: "/a"})).To(Succeed())
|
||||
|
||||
entries, err = ListEntries()
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
Expect(entries).To(HaveLen(2))
|
||||
Expect(entries[0].Name).To(Equal("a"))
|
||||
Expect(entries[1].Name).To(Equal("b"))
|
||||
}
|
||||
|
||||
func TestLegacyMountToEntry(t *testing.T) {
|
||||
RegisterTestingT(t)
|
||||
|
||||
a := LegacyMountToEntry("/host/path:/container/path")
|
||||
b := LegacyMountToEntry("/host/path:/container/different")
|
||||
c := LegacyMountToEntry("/host/path:/container/path:ro")
|
||||
|
||||
// Same host path produces the same entry name regardless of container
|
||||
// path or options - container path lives on the attachment, not the
|
||||
// entry, so apps mounting the same source converge.
|
||||
Expect(a.Name).To(Equal(b.Name))
|
||||
Expect(a.Name).To(Equal(c.Name))
|
||||
Expect(a.Name).To(HavePrefix(LegacyEntryPrefix))
|
||||
Expect(len(a.Name)).To(Equal(len(LegacyEntryPrefix) + 10))
|
||||
|
||||
// Different host paths produce different names.
|
||||
other := LegacyMountToEntry("/different/path:/container/path")
|
||||
Expect(other.Name).NotTo(Equal(a.Name))
|
||||
|
||||
// Named docker volumes are distinguished from absolute paths.
|
||||
named := LegacyMountToEntry("my_volume:/container/path")
|
||||
Expect(named.Name).NotTo(Equal(a.Name))
|
||||
Expect(named.HostPath).To(Equal("my_volume"))
|
||||
|
||||
// The synthesized entry is a valid docker-local entry that passes
|
||||
// Validate (other than the legacy- prefix, which Validate accepts).
|
||||
abs := LegacyMountToEntry("/host/path:/container/path")
|
||||
Expect(abs.Scheduler).To(Equal(SchedulerDockerLocal))
|
||||
Expect(abs.HostPath).To(Equal("/host/path"))
|
||||
Expect(abs.Validate()).To(Succeed())
|
||||
}
|
||||
206
plugins/storage/migrate.go
Normal file
206
plugins/storage/migrate.go
Normal file
@@ -0,0 +1,206 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"github.com/dokku/dokku/plugins/common"
|
||||
dockeroptions "github.com/dokku/dokku/plugins/docker-options"
|
||||
)
|
||||
|
||||
// migrationFlagDir contains per-app flag files written after a successful
|
||||
// per-app migration; presence means the app's legacy -v lines have been
|
||||
// drained into the attachment store. Lives under the registry directory
|
||||
// so it doesn't collide with the property-store path
|
||||
// `config/storage/<appName>`.
|
||||
func migrationFlagDir() string {
|
||||
return filepath.Join(RegistryDirectory(), "migrations")
|
||||
}
|
||||
|
||||
func migrationFlagFile(appName string) string {
|
||||
return filepath.Join(migrationFlagDir(), appName)
|
||||
}
|
||||
|
||||
// MigrateApp re-runs the legacy migration for a single app, ignoring
|
||||
// the per-app flag file so an operator can force-rescan an app whose
|
||||
// docker-options state changed after the bulk install-time pass.
|
||||
// Used by `dokku storage:migrate <app>`.
|
||||
func MigrateApp(appName string) error {
|
||||
if err := os.MkdirAll(migrationFlagDir(), 0755); err != nil {
|
||||
return fmt.Errorf("unable to create migration flag dir: %w", err)
|
||||
}
|
||||
_ = os.Remove(migrationFlagFile(appName))
|
||||
if err := migrateApp(appName); err != nil {
|
||||
return fmt.Errorf("storage migration failed for app %q: %w", appName, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// MigrateLegacyMounts walks every app and converts its legacy `-v`
|
||||
// docker-options entries into named storage entries plus attachments.
|
||||
// Idempotent: per-app flag files short-circuit re-runs.
|
||||
func MigrateLegacyMounts() error {
|
||||
if err := os.MkdirAll(migrationFlagDir(), 0755); err != nil {
|
||||
return fmt.Errorf("unable to create migration flag dir: %w", err)
|
||||
}
|
||||
|
||||
apps, err := common.DokkuApps()
|
||||
if err != nil {
|
||||
if errors.Is(err, common.NoAppsExist) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
for _, app := range apps {
|
||||
if _, err := os.Stat(migrationFlagFile(app)); err == nil {
|
||||
continue
|
||||
}
|
||||
if err := migrateApp(app); err != nil {
|
||||
return fmt.Errorf("storage migration failed for app %q: %w", app, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// migrateApp performs the per-app migration. Order is intentional: write
|
||||
// the entry, write the attachment, drain the docker-options line. Any
|
||||
// step failing aborts before drain, so the original behavior is
|
||||
// preserved on partial failure.
|
||||
func migrateApp(appName string) error {
|
||||
deployLines, err := dockeroptions.GetDockerOptionsForPhase(appName, PhaseDeploy)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
runLines, err := dockeroptions.GetDockerOptionsForPhase(appName, PhaseRun)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
deployMounts := filterMountLines(deployLines)
|
||||
runMounts := filterMountLines(runLines)
|
||||
|
||||
// Group identical mount strings across phases.
|
||||
phaseMap := map[string][]string{}
|
||||
for _, mount := range deployMounts {
|
||||
phaseMap[mount] = appendUnique(phaseMap[mount], PhaseDeploy)
|
||||
}
|
||||
for _, mount := range runMounts {
|
||||
phaseMap[mount] = appendUnique(phaseMap[mount], PhaseRun)
|
||||
}
|
||||
|
||||
mounts := make([]string, 0, len(phaseMap))
|
||||
for mount := range phaseMap {
|
||||
mounts = append(mounts, mount)
|
||||
}
|
||||
sort.Strings(mounts)
|
||||
|
||||
for _, mount := range mounts {
|
||||
if err := migrateMount(appName, mount, phaseMap[mount]); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return touchMigrationFlag(appName)
|
||||
}
|
||||
|
||||
func migrateMount(appName string, mount string, phases []string) error {
|
||||
parsed := ParseMountPath(mount)
|
||||
entry := LegacyMountToEntry(mount)
|
||||
|
||||
if EntryExists(entry.Name) {
|
||||
existing, err := LoadEntry(entry.Name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if existing.HostPath != entry.HostPath || existing.Scheduler != entry.Scheduler {
|
||||
return fmt.Errorf("legacy entry %q already exists with conflicting fields", entry.Name)
|
||||
}
|
||||
} else {
|
||||
if err := SaveEntry(entry); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
containerPath := parsed.ContainerPath
|
||||
if containerPath == "" {
|
||||
// Defensive: legacy mounts must have a container path; skip if not.
|
||||
return nil
|
||||
}
|
||||
|
||||
attachment := &Attachment{
|
||||
EntryName: entry.Name,
|
||||
ContainerPath: containerPath,
|
||||
Phases: phases,
|
||||
ProcessType: DefaultProcessType,
|
||||
}
|
||||
if parsed.VolumeOptions != "" {
|
||||
if parsed.VolumeOptions == "ro" {
|
||||
attachment.Readonly = true
|
||||
} else {
|
||||
attachment.VolumeOptions = parsed.VolumeOptions
|
||||
}
|
||||
}
|
||||
|
||||
existing, err := LoadAttachments(appName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !attachmentExists(existing, attachment) {
|
||||
existing = append(existing, attachment)
|
||||
if err := SaveAttachments(appName, existing); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
mountedPhases := []string{}
|
||||
for _, phase := range phases {
|
||||
mountedPhases = append(mountedPhases, phase)
|
||||
}
|
||||
if err := dockeroptions.RemoveDockerOptionFromPhases(appName, mountedPhases, fmt.Sprintf("-v %s", mount)); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func touchMigrationFlag(appName string) error {
|
||||
f, err := os.Create(migrationFlagFile(appName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return f.Close()
|
||||
}
|
||||
|
||||
func filterMountLines(lines []string) []string {
|
||||
out := []string{}
|
||||
for _, line := range lines {
|
||||
if strings.HasPrefix(line, "-v ") {
|
||||
out = append(out, strings.TrimPrefix(line, "-v "))
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func appendUnique(slice []string, value string) []string {
|
||||
for _, existing := range slice {
|
||||
if existing == value {
|
||||
return slice
|
||||
}
|
||||
}
|
||||
return append(slice, value)
|
||||
}
|
||||
|
||||
func attachmentExists(attachments []*Attachment, candidate *Attachment) bool {
|
||||
for _, existing := range attachments {
|
||||
if existing.EntryName == candidate.EntryName &&
|
||||
existing.ContainerPath == candidate.ContainerPath &&
|
||||
existing.ProcessType == candidate.ProcessType {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
317
plugins/storage/migrate_test.go
Normal file
317
plugins/storage/migrate_test.go
Normal file
@@ -0,0 +1,317 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"os"
|
||||
"os/user"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"testing"
|
||||
|
||||
"github.com/dokku/dokku/plugins/common"
|
||||
dockeroptions "github.com/dokku/dokku/plugins/docker-options"
|
||||
)
|
||||
|
||||
// setupMigrationEnv points DOKKU_LIB_ROOT and DOKKU_ROOT at temp dirs and
|
||||
// tells the permission helpers to chown to the current user (a no-op),
|
||||
// so the test works without root and without the dokku system user.
|
||||
// Mirrors the helper in docker-options/migration_test.go.
|
||||
func setupMigrationEnv(t *testing.T) (libRoot string, dokkuRoot string) {
|
||||
t.Helper()
|
||||
libRoot = t.TempDir()
|
||||
dokkuRoot = t.TempDir()
|
||||
|
||||
t.Setenv("DOKKU_LIB_ROOT", libRoot)
|
||||
t.Setenv("DOKKU_ROOT", dokkuRoot)
|
||||
t.Setenv("PLUGIN_PATH", filepath.Join(libRoot, "plugins"))
|
||||
|
||||
current, err := user.Current()
|
||||
if err != nil {
|
||||
t.Fatalf("user.Current: %v", err)
|
||||
}
|
||||
group, err := user.LookupGroupId(current.Gid)
|
||||
if err != nil {
|
||||
t.Fatalf("user.LookupGroupId: %v", err)
|
||||
}
|
||||
t.Setenv("DOKKU_SYSTEM_USER", current.Username)
|
||||
t.Setenv("DOKKU_SYSTEM_GROUP", group.Name)
|
||||
|
||||
// MigrateLegacyMounts creates the flag dir before iterating apps;
|
||||
// migrateApp (the per-app helper) assumes it already exists. Pre-
|
||||
// create it so tests that call migrateApp directly don't trip.
|
||||
if err := os.MkdirAll(migrationFlagDir(), 0755); err != nil {
|
||||
t.Fatalf("mkdir migration flag dir: %v", err)
|
||||
}
|
||||
|
||||
return libRoot, dokkuRoot
|
||||
}
|
||||
|
||||
// stageApp registers the named app under DOKKU_ROOT so DokkuApps can
|
||||
// see it, and seeds its docker-options property files for the requested
|
||||
// phases. Each map entry is a phase ("deploy", "run", "build") whose
|
||||
// value is the full property-list contents (one option per line).
|
||||
func stageApp(t *testing.T, dokkuRoot string, app string, options map[string][]string) {
|
||||
t.Helper()
|
||||
if err := os.MkdirAll(filepath.Join(dokkuRoot, app), 0755); err != nil {
|
||||
t.Fatalf("mkdir app: %v", err)
|
||||
}
|
||||
for phase, lines := range options {
|
||||
if err := dockeroptions.AddDockerOptionToPhases(app, []string{phase}, ""); err != nil {
|
||||
t.Fatalf("seed property file: %v", err)
|
||||
}
|
||||
// Replace whatever AddDockerOptionToPhases left with the precise
|
||||
// list the test wants, since it appends rather than overwriting.
|
||||
key := "_default_." + phase
|
||||
if err := common.PropertyListWrite("docker-options", app, key, lines); err != nil {
|
||||
t.Fatalf("PropertyListWrite: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func phaseOptions(t *testing.T, app string, phase string) []string {
|
||||
t.Helper()
|
||||
got, err := dockeroptions.GetDockerOptionsForPhase(app, phase)
|
||||
if err != nil {
|
||||
t.Fatalf("GetDockerOptionsForPhase %s/%s: %v", app, phase, err)
|
||||
}
|
||||
return got
|
||||
}
|
||||
|
||||
func TestMigrateAppSinglePhase(t *testing.T) {
|
||||
_, dokkuRoot := setupMigrationEnv(t)
|
||||
stageApp(t, dokkuRoot, "alpha", map[string][]string{
|
||||
"deploy": {"-v /var/log:/log", "--restart=on-failure:5"},
|
||||
})
|
||||
|
||||
if err := migrateApp("alpha"); err != nil {
|
||||
t.Fatalf("migrateApp: %v", err)
|
||||
}
|
||||
|
||||
entry := LegacyMountToEntry("/var/log:/log")
|
||||
if !EntryExists(entry.Name) {
|
||||
t.Fatalf("expected entry %s on disk", entry.Name)
|
||||
}
|
||||
loaded, err := LoadEntry(entry.Name)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadEntry: %v", err)
|
||||
}
|
||||
if loaded.HostPath != "/var/log" {
|
||||
t.Errorf("entry HostPath = %s, want /var/log", loaded.HostPath)
|
||||
}
|
||||
if loaded.Scheduler != SchedulerDockerLocal {
|
||||
t.Errorf("entry Scheduler = %s, want %s", loaded.Scheduler, SchedulerDockerLocal)
|
||||
}
|
||||
|
||||
attachments, err := LoadAttachments("alpha")
|
||||
if err != nil {
|
||||
t.Fatalf("LoadAttachments: %v", err)
|
||||
}
|
||||
if len(attachments) != 1 {
|
||||
t.Fatalf("expected 1 attachment, got %d", len(attachments))
|
||||
}
|
||||
att := attachments[0]
|
||||
if att.EntryName != entry.Name || att.ContainerPath != "/log" {
|
||||
t.Errorf("attachment = %+v, want entry=%s container=/log", att, entry.Name)
|
||||
}
|
||||
if len(att.Phases) != 1 || att.Phases[0] != PhaseDeploy {
|
||||
t.Errorf("attachment phases = %v, want [deploy]", att.Phases)
|
||||
}
|
||||
|
||||
// docker-options drained the -v line but kept everything else.
|
||||
deployAfter := phaseOptions(t, "alpha", "deploy")
|
||||
if !equalSorted(deployAfter, []string{"--restart=on-failure:5"}) {
|
||||
t.Errorf("deploy options after migration = %v, want [--restart=on-failure:5]", deployAfter)
|
||||
}
|
||||
|
||||
// Migration flag file is in place.
|
||||
if _, err := os.Stat(migrationFlagFile("alpha")); err != nil {
|
||||
t.Errorf("migration flag missing: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrateAppCrossPhaseGroupsIntoOneAttachment(t *testing.T) {
|
||||
_, dokkuRoot := setupMigrationEnv(t)
|
||||
// Same -v in both phases groups into one attachment with phases = [deploy, run].
|
||||
stageApp(t, dokkuRoot, "alpha", map[string][]string{
|
||||
"deploy": {"-v /data:/d"},
|
||||
"run": {"-v /data:/d"},
|
||||
})
|
||||
|
||||
if err := migrateApp("alpha"); err != nil {
|
||||
t.Fatalf("migrateApp: %v", err)
|
||||
}
|
||||
|
||||
attachments, err := LoadAttachments("alpha")
|
||||
if err != nil {
|
||||
t.Fatalf("LoadAttachments: %v", err)
|
||||
}
|
||||
if len(attachments) != 1 {
|
||||
t.Fatalf("expected 1 attachment, got %d", len(attachments))
|
||||
}
|
||||
got := attachments[0].Phases
|
||||
sort.Strings(got)
|
||||
if !equalSorted(got, []string{PhaseDeploy, PhaseRun}) {
|
||||
t.Errorf("phases = %v, want [deploy run]", got)
|
||||
}
|
||||
|
||||
// Drain affected both phases.
|
||||
if got := phaseOptions(t, "alpha", "deploy"); len(got) != 0 {
|
||||
t.Errorf("deploy options after migration = %v, want []", got)
|
||||
}
|
||||
if got := phaseOptions(t, "alpha", "run"); len(got) != 0 {
|
||||
t.Errorf("run options after migration = %v, want []", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrateAppPreservesVolumeOptions(t *testing.T) {
|
||||
_, dokkuRoot := setupMigrationEnv(t)
|
||||
stageApp(t, dokkuRoot, "alpha", map[string][]string{
|
||||
"deploy": {"-v /ro:/r:ro", "-v /label:/l:Z"},
|
||||
})
|
||||
|
||||
if err := migrateApp("alpha"); err != nil {
|
||||
t.Fatalf("migrateApp: %v", err)
|
||||
}
|
||||
|
||||
atts, err := LoadAttachments("alpha")
|
||||
if err != nil {
|
||||
t.Fatalf("LoadAttachments: %v", err)
|
||||
}
|
||||
byPath := map[string]*Attachment{}
|
||||
for _, a := range atts {
|
||||
byPath[a.ContainerPath] = a
|
||||
}
|
||||
|
||||
if a := byPath["/r"]; a == nil {
|
||||
t.Fatalf("missing /r attachment")
|
||||
} else if !a.Readonly || a.VolumeOptions != "" {
|
||||
t.Errorf("/r attachment = readonly=%v opts=%q, want readonly=true opts=\"\"", a.Readonly, a.VolumeOptions)
|
||||
}
|
||||
|
||||
if a := byPath["/l"]; a == nil {
|
||||
t.Fatalf("missing /l attachment")
|
||||
} else if a.Readonly || a.VolumeOptions != "Z" {
|
||||
t.Errorf("/l attachment = readonly=%v opts=%q, want readonly=false opts=\"Z\"", a.Readonly, a.VolumeOptions)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrateAppIdempotent(t *testing.T) {
|
||||
_, dokkuRoot := setupMigrationEnv(t)
|
||||
stageApp(t, dokkuRoot, "alpha", map[string][]string{
|
||||
"deploy": {"-v /var/log:/log"},
|
||||
})
|
||||
|
||||
if err := migrateApp("alpha"); err != nil {
|
||||
t.Fatalf("first migrateApp: %v", err)
|
||||
}
|
||||
|
||||
// Re-stage the same -v line by hand to simulate someone re-adding
|
||||
// the legacy form via docker-options:add after migration. The
|
||||
// migration is gated by the per-app flag file at the
|
||||
// MigrateLegacyMounts level, but migrateApp itself should still be
|
||||
// safe to call directly: a second run produces no duplicates.
|
||||
if err := common.PropertyListWrite("docker-options", "alpha", "_default_.deploy", []string{"-v /var/log:/log"}); err != nil {
|
||||
t.Fatalf("re-stage: %v", err)
|
||||
}
|
||||
if err := migrateApp("alpha"); err != nil {
|
||||
t.Fatalf("second migrateApp: %v", err)
|
||||
}
|
||||
|
||||
atts, err := LoadAttachments("alpha")
|
||||
if err != nil {
|
||||
t.Fatalf("LoadAttachments: %v", err)
|
||||
}
|
||||
if len(atts) != 1 {
|
||||
t.Errorf("expected 1 attachment after rerun, got %d", len(atts))
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrateLegacyMountsBulkAndFlagFastPath(t *testing.T) {
|
||||
_, dokkuRoot := setupMigrationEnv(t)
|
||||
stageApp(t, dokkuRoot, "alpha", map[string][]string{
|
||||
"deploy": {"-v /alpha-data:/data"},
|
||||
})
|
||||
stageApp(t, dokkuRoot, "beta", map[string][]string{
|
||||
"deploy": {"-v /beta-data:/data"},
|
||||
"run": {"-v /beta-data:/data"},
|
||||
})
|
||||
|
||||
if err := MigrateLegacyMounts(); err != nil {
|
||||
t.Fatalf("MigrateLegacyMounts: %v", err)
|
||||
}
|
||||
|
||||
for _, app := range []string{"alpha", "beta"} {
|
||||
if _, err := os.Stat(migrationFlagFile(app)); err != nil {
|
||||
t.Errorf("flag file missing for %s: %v", app, err)
|
||||
}
|
||||
atts, err := LoadAttachments(app)
|
||||
if err != nil {
|
||||
t.Fatalf("LoadAttachments %s: %v", app, err)
|
||||
}
|
||||
if len(atts) != 1 {
|
||||
t.Errorf("%s: expected 1 attachment, got %d", app, len(atts))
|
||||
}
|
||||
}
|
||||
|
||||
// Re-stage a -v on alpha and rerun MigrateLegacyMounts. The flag
|
||||
// file short-circuits per-app migration, so the new line stays in
|
||||
// docker-options.
|
||||
if err := common.PropertyListWrite("docker-options", "alpha", "_default_.deploy", []string{"-v /sneaky:/x"}); err != nil {
|
||||
t.Fatalf("re-stage alpha: %v", err)
|
||||
}
|
||||
if err := MigrateLegacyMounts(); err != nil {
|
||||
t.Fatalf("MigrateLegacyMounts (rerun): %v", err)
|
||||
}
|
||||
got := phaseOptions(t, "alpha", "deploy")
|
||||
if !equalSorted(got, []string{"-v /sneaky:/x"}) {
|
||||
t.Errorf("expected sneaky -v line untouched after second pass, got %v", got)
|
||||
}
|
||||
atts, err := LoadAttachments("alpha")
|
||||
if err != nil {
|
||||
t.Fatalf("LoadAttachments alpha rerun: %v", err)
|
||||
}
|
||||
if len(atts) != 1 {
|
||||
t.Errorf("expected 1 attachment after rerun, got %d", len(atts))
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrateAppRefusesEntryNameCollision(t *testing.T) {
|
||||
libRoot, dokkuRoot := setupMigrationEnv(t)
|
||||
_ = libRoot
|
||||
|
||||
// Plant a colliding entry at the legacy-<hash> name with conflicting
|
||||
// fields so migration's "fields match" check trips.
|
||||
collide := LegacyMountToEntry("/var/log:/log")
|
||||
collide.HostPath = "/something/else"
|
||||
if err := SaveEntry(collide); err != nil {
|
||||
t.Fatalf("seed colliding entry: %v", err)
|
||||
}
|
||||
|
||||
stageApp(t, dokkuRoot, "alpha", map[string][]string{
|
||||
"deploy": {"-v /var/log:/log"},
|
||||
})
|
||||
|
||||
if err := migrateApp("alpha"); err == nil {
|
||||
t.Fatalf("expected migrateApp to error on entry name collision")
|
||||
}
|
||||
|
||||
// Drain did not happen.
|
||||
if got := phaseOptions(t, "alpha", "deploy"); len(got) != 1 || got[0] != "-v /var/log:/log" {
|
||||
t.Errorf("expected legacy line preserved on collision, got %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func equalSorted(a, b []string) bool {
|
||||
if len(a) != len(b) {
|
||||
return false
|
||||
}
|
||||
ax := append([]string(nil), a...)
|
||||
bx := append([]string(nil), b...)
|
||||
sort.Strings(ax)
|
||||
sort.Strings(bx)
|
||||
for i := range ax {
|
||||
if ax[i] != bx[i] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
@@ -18,11 +18,19 @@ Manage mounted volumes
|
||||
Additional commands:`
|
||||
|
||||
helpContent = `
|
||||
storage:ensure-directory [--chown option] <directory>, Creates a persistent storage directory in the recommended storage path
|
||||
storage:create <name> [<path>] [flags], Register a named storage entry
|
||||
storage:destroy <name>, Remove a named storage entry (must be unmounted from every app first)
|
||||
storage:ensure-directory [--chown option] <directory>, [DEPRECATED] use storage:create instead
|
||||
storage:exec <name> [-- <cmd>...], Run a command (or shell) in a temporary container that mounts the entry
|
||||
storage:info <name> [--format text|json], Show details for one storage entry
|
||||
storage:list <app> [--format text|json], List bind mounts for app's container(s) (host:container)
|
||||
storage:list-entries [--scheduler s] [--format text|json], List registered storage entries
|
||||
storage:migrate [<app>|--all], Re-run the legacy -v to attachment migration for an app
|
||||
storage:mount <app> <host-dir:container-dir>, Create a new bind mount
|
||||
storage:report [<app>] [<flag>], Displays a storage report for one or more apps
|
||||
storage:unmount <app> <host-dir:container-dir>, Remove an existing bind mount`
|
||||
storage:set <name> [flags], Update a storage entry in place
|
||||
storage:unmount <app> <host-dir:container-dir>, Remove an existing bind mount
|
||||
storage:wait <name>, Wait for a storage entry's PVC to be bound (k3s)`
|
||||
)
|
||||
|
||||
func main() {
|
||||
|
||||
@@ -19,24 +19,152 @@ func main() {
|
||||
switch subcommand {
|
||||
case "default":
|
||||
err = storage.CommandHelp()
|
||||
case "create":
|
||||
args := flag.NewFlagSet("storage:create", flag.ExitOnError)
|
||||
scheduler := args.String("scheduler", storage.SchedulerDockerLocal, "--scheduler: target scheduler (docker-local, k3s)")
|
||||
size := args.String("size", "", "--size: PVC size (k3s only, e.g. 2Gi)")
|
||||
accessMode := args.String("access-mode", "", "--access-mode: PVC access mode (k3s only)")
|
||||
storageClass := args.String("storage-class-name", "", "--storage-class-name: PVC storage class (k3s only)")
|
||||
namespace := args.String("namespace", "", "--namespace: PVC namespace (k3s only)")
|
||||
chown := args.String("chown", "", "--chown: chown option (docker-local only)")
|
||||
reclaim := args.String("reclaim-policy", "", "--reclaim-policy: PV reclaim policy (Retain or Delete, k3s only)")
|
||||
annotations := args.StringSlice("annotation", nil, "--annotation key=value: PVC annotation (repeatable)")
|
||||
labels := args.StringSlice("label", nil, "--label key=value: PVC label (repeatable)")
|
||||
args.Parse(os.Args[2:])
|
||||
name := args.Arg(0)
|
||||
path := args.Arg(1)
|
||||
annotMap, parseErr := parseKVPairs(*annotations)
|
||||
if parseErr != nil {
|
||||
err = parseErr
|
||||
break
|
||||
}
|
||||
labelMap, parseErr := parseKVPairs(*labels)
|
||||
if parseErr != nil {
|
||||
err = parseErr
|
||||
break
|
||||
}
|
||||
err = storage.CommandCreate(storage.CommandCreateInput{
|
||||
Name: name,
|
||||
Path: path,
|
||||
Scheduler: *scheduler,
|
||||
Size: *size,
|
||||
AccessMode: *accessMode,
|
||||
StorageClass: *storageClass,
|
||||
Namespace: *namespace,
|
||||
Chown: *chown,
|
||||
ReclaimPolicy: *reclaim,
|
||||
Annotations: annotMap,
|
||||
Labels: labelMap,
|
||||
})
|
||||
case "destroy":
|
||||
args := flag.NewFlagSet("storage:destroy", flag.ExitOnError)
|
||||
args.Parse(os.Args[2:])
|
||||
err = storage.CommandDestroy(args.Arg(0))
|
||||
case "ensure-directory":
|
||||
args := flag.NewFlagSet("storage:ensure-directory", flag.ExitOnError)
|
||||
chown := args.String("chown", "herokuish", "--chown: chown option (herokuish, heroku, paketo, root, false)")
|
||||
args.Parse(os.Args[2:])
|
||||
directory := args.Arg(0)
|
||||
common.LogWarn("storage:ensure-directory is deprecated; use storage:create instead.")
|
||||
err = storage.CommandEnsureDirectory(directory, *chown)
|
||||
case "info":
|
||||
args := flag.NewFlagSet("storage:info", flag.ExitOnError)
|
||||
format := args.String("format", "text", "--format: output format (text, json)")
|
||||
args.Parse(os.Args[2:])
|
||||
err = storage.CommandInfo(args.Arg(0), *format)
|
||||
case "list":
|
||||
args := flag.NewFlagSet("storage:list", flag.ExitOnError)
|
||||
format := args.String("format", "text", "--format: output format (text, json)")
|
||||
args.Parse(os.Args[2:])
|
||||
appName := args.Arg(0)
|
||||
err = storage.CommandList(appName, *format)
|
||||
case "set":
|
||||
args := flag.NewFlagSet("storage:set", flag.ExitOnError)
|
||||
size := args.String("size", "", "--size: new PVC size (k3s)")
|
||||
accessMode := args.String("access-mode", "", "--access-mode: existing access mode (must match)")
|
||||
storageClass := args.String("storage-class-name", "", "--storage-class-name: existing storage class (must match)")
|
||||
namespace := args.String("namespace", "", "--namespace: new namespace")
|
||||
chown := args.String("chown", "", "--chown: chown option")
|
||||
reclaim := args.String("reclaim-policy", "", "--reclaim-policy: PV reclaim policy")
|
||||
annotations := args.StringSlice("annotation", nil, "--annotation key=value: PVC annotation (repeatable, replaces all)")
|
||||
labels := args.StringSlice("label", nil, "--label key=value: PVC label (repeatable, replaces all)")
|
||||
args.Parse(os.Args[2:])
|
||||
annotMap, parseErr := parseKVPairs(*annotations)
|
||||
if parseErr != nil {
|
||||
err = parseErr
|
||||
break
|
||||
}
|
||||
labelMap, parseErr := parseKVPairs(*labels)
|
||||
if parseErr != nil {
|
||||
err = parseErr
|
||||
break
|
||||
}
|
||||
err = storage.CommandSet(storage.CommandSetInput{
|
||||
Name: args.Arg(0),
|
||||
Size: *size,
|
||||
AccessMode: *accessMode,
|
||||
StorageClass: *storageClass,
|
||||
Namespace: *namespace,
|
||||
Chown: *chown,
|
||||
ReclaimPolicy: *reclaim,
|
||||
Annotations: annotMap,
|
||||
Labels: labelMap,
|
||||
})
|
||||
case "exec":
|
||||
args := flag.NewFlagSet("storage:exec", flag.ExitOnError)
|
||||
image := args.String("image", "", "--image: container image to use (default alpine:3)")
|
||||
asUser := args.String("as-user", "", "--as-user: numeric uid to run the exec container as (overrides the entry's chown)")
|
||||
args.Parse(os.Args[2:])
|
||||
positional := args.Args()
|
||||
if len(positional) == 0 {
|
||||
err = fmt.Errorf("storage:exec requires a storage entry name")
|
||||
break
|
||||
}
|
||||
name := positional[0]
|
||||
var cmd []string
|
||||
if len(positional) > 1 {
|
||||
cmd = positional[1:]
|
||||
}
|
||||
err = storage.CommandExec(storage.CommandExecInput{
|
||||
Name: name,
|
||||
Image: *image,
|
||||
AsUser: *asUser,
|
||||
Args: cmd,
|
||||
})
|
||||
case "migrate":
|
||||
args := flag.NewFlagSet("storage:migrate", flag.ExitOnError)
|
||||
all := args.Bool("all", false, "--all: re-migrate every app on this install")
|
||||
args.Parse(os.Args[2:])
|
||||
err = storage.CommandMigrate(args.Arg(0), *all)
|
||||
case "wait":
|
||||
args := flag.NewFlagSet("storage:wait", flag.ExitOnError)
|
||||
args.Parse(os.Args[2:])
|
||||
err = storage.CommandWait(args.Arg(0))
|
||||
case "list-entries":
|
||||
args := flag.NewFlagSet("storage:list-entries", flag.ExitOnError)
|
||||
scheduler := args.String("scheduler", "", "--scheduler: filter to a single scheduler")
|
||||
format := args.String("format", "text", "--format: output format (text, json)")
|
||||
args.Parse(os.Args[2:])
|
||||
err = storage.CommandListEntries(*scheduler, *format)
|
||||
case "mount":
|
||||
args := flag.NewFlagSet("storage:mount", flag.ExitOnError)
|
||||
containerDir := args.String("container-dir", "", "--container-dir: container path (named-entry form)")
|
||||
phases := args.StringSlice("phase", nil, "--phase: phase to mount in (deploy, run; default both)")
|
||||
processType := args.String("process-type", "", "--process-type: process type to mount for")
|
||||
subpath := args.String("volume-subpath", "", "--volume-subpath: subpath within the entry")
|
||||
readonly := args.Bool("volume-readonly", false, "--volume-readonly: mount the volume read-only")
|
||||
volumeChown := args.String("volume-chown", "", "--volume-chown: chown option applied at mount time")
|
||||
args.Parse(os.Args[2:])
|
||||
appName := args.Arg(0)
|
||||
mountPath := args.Arg(1)
|
||||
err = storage.CommandMount(appName, mountPath)
|
||||
err = storage.CommandMount(storage.CommandMountInput{
|
||||
AppName: args.Arg(0),
|
||||
NameOrPath: args.Arg(1),
|
||||
ContainerDir: *containerDir,
|
||||
Phases: *phases,
|
||||
ProcessType: *processType,
|
||||
Subpath: *subpath,
|
||||
Readonly: *readonly,
|
||||
VolumeChown: *volumeChown,
|
||||
})
|
||||
case "report":
|
||||
args := flag.NewFlagSet("storage:report", flag.ExitOnError)
|
||||
format := args.String("format", "stdout", "format: [ stdout | json ]")
|
||||
@@ -51,10 +179,13 @@ func main() {
|
||||
}
|
||||
case "unmount":
|
||||
args := flag.NewFlagSet("storage:unmount", flag.ExitOnError)
|
||||
containerDir := args.String("container-dir", "", "--container-dir: container path (named-entry form, disambiguates duplicates)")
|
||||
args.Parse(os.Args[2:])
|
||||
appName := args.Arg(0)
|
||||
mountPath := args.Arg(1)
|
||||
err = storage.CommandUnmount(appName, mountPath)
|
||||
err = storage.CommandUnmount(storage.CommandUnmountInput{
|
||||
AppName: args.Arg(0),
|
||||
NameOrPath: args.Arg(1),
|
||||
ContainerDir: *containerDir,
|
||||
})
|
||||
default:
|
||||
err = fmt.Errorf("Invalid plugin subcommand call: %s", subcommand)
|
||||
}
|
||||
@@ -63,3 +194,18 @@ func main() {
|
||||
common.LogFailWithError(err)
|
||||
}
|
||||
}
|
||||
|
||||
func parseKVPairs(pairs []string) (map[string]string, error) {
|
||||
if len(pairs) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
out := map[string]string{}
|
||||
for _, pair := range pairs {
|
||||
idx := strings.Index(pair, "=")
|
||||
if idx <= 0 {
|
||||
return nil, fmt.Errorf("expected key=value, got %q", pair)
|
||||
}
|
||||
out[pair[:idx]] = pair[idx+1:]
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
@@ -24,6 +24,27 @@ func main() {
|
||||
phase := flag.Arg(1)
|
||||
format := flag.Arg(2)
|
||||
err = storage.TriggerStorageList(appName, phase, format)
|
||||
case "storage-app-mounts":
|
||||
appName := flag.Arg(0)
|
||||
phase := flag.Arg(1)
|
||||
err = storage.TriggerStorageAppMounts(appName, phase)
|
||||
case "docker-args-deploy":
|
||||
appName := flag.Arg(0)
|
||||
err = storage.TriggerDockerArgs(appName, storage.PhaseDeploy)
|
||||
case "docker-args-run":
|
||||
appName := flag.Arg(0)
|
||||
err = storage.TriggerDockerArgs(appName, storage.PhaseRun)
|
||||
case "post-delete":
|
||||
appName := flag.Arg(0)
|
||||
err = storage.TriggerPostDelete(appName)
|
||||
case "post-app-clone-setup":
|
||||
oldName := flag.Arg(0)
|
||||
newName := flag.Arg(1)
|
||||
err = storage.TriggerPostAppCloneSetup(oldName, newName)
|
||||
case "post-app-rename-setup":
|
||||
oldName := flag.Arg(0)
|
||||
newName := flag.Arg(1)
|
||||
err = storage.TriggerPostAppRenameSetup(oldName, newName)
|
||||
default:
|
||||
err = fmt.Errorf("Invalid plugin trigger call: %s", trigger)
|
||||
}
|
||||
|
||||
@@ -51,19 +51,18 @@ func CheckIfPathExists(appName string, mountPath string, phases []string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// GetBindMounts returns the bind mounts for an app and phase
|
||||
// GetBindMounts returns the bind mounts for an app and phase, synthesized
|
||||
// from the attachment store. The returned strings use the legacy colon
|
||||
// form (host:container[:options]) so existing display paths keep
|
||||
// working unchanged.
|
||||
func GetBindMounts(appName string, phase string) ([]string, error) {
|
||||
mounts := []string{}
|
||||
options, err := dockeroptions.GetDockerOptionsForPhase(appName, phase)
|
||||
entries, err := ListAppMountEntries(appName, phase)
|
||||
if err != nil {
|
||||
return mounts, err
|
||||
return nil, err
|
||||
}
|
||||
|
||||
for _, option := range options {
|
||||
if strings.HasPrefix(option, "-v ") {
|
||||
mount := strings.TrimPrefix(option, "-v ")
|
||||
mounts = append(mounts, mount)
|
||||
}
|
||||
mounts := make([]string, 0, len(entries))
|
||||
for _, entry := range entries {
|
||||
mounts = append(mounts, formatStorageListEntry(entry))
|
||||
}
|
||||
return mounts, nil
|
||||
}
|
||||
@@ -82,13 +81,71 @@ func GetBindMountsForDisplay(appName string, phase string) string {
|
||||
return strings.Join(result, " ")
|
||||
}
|
||||
|
||||
// StorageListEntry represents a storage mount entry for JSON output
|
||||
// StorageListEntry represents a storage mount entry for JSON output.
|
||||
type StorageListEntry struct {
|
||||
EntryName string `json:"entry_name,omitempty"`
|
||||
HostPath string `json:"host_path"`
|
||||
ContainerPath string `json:"container_path"`
|
||||
VolumeOptions string `json:"volume_options"`
|
||||
}
|
||||
|
||||
// ListAppMountEntries returns one StorageListEntry per attachment on
|
||||
// an app for the requested phase, joining each attachment with its
|
||||
// referenced storage entry. Used by storage:list and the deprecated
|
||||
// storage-list trigger.
|
||||
func ListAppMountEntries(appName string, phase string) ([]StorageListEntry, error) {
|
||||
if phase == "" {
|
||||
phase = PhaseDeploy
|
||||
}
|
||||
attachments, err := AttachmentsForPhase(appName, phase)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
rows := make([]StorageListEntry, 0, len(attachments))
|
||||
for _, attachment := range attachments {
|
||||
entry, err := LoadEntry(attachment.EntryName)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("attachment on %q references missing entry %q: %w", appName, attachment.EntryName, err)
|
||||
}
|
||||
|
||||
host := entry.HostPath
|
||||
if host == "" {
|
||||
// k3s-only entries with no host path: surface the entry
|
||||
// name as the host token so the colon-form output is
|
||||
// well-formed and parseable.
|
||||
host = entry.Name
|
||||
}
|
||||
|
||||
options := ""
|
||||
switch {
|
||||
case attachment.Readonly && attachment.VolumeOptions != "":
|
||||
options = "ro," + attachment.VolumeOptions
|
||||
case attachment.Readonly:
|
||||
options = "ro"
|
||||
case attachment.VolumeOptions != "":
|
||||
options = attachment.VolumeOptions
|
||||
}
|
||||
|
||||
rows = append(rows, StorageListEntry{
|
||||
EntryName: entry.Name,
|
||||
HostPath: host,
|
||||
ContainerPath: attachment.ContainerPath,
|
||||
VolumeOptions: options,
|
||||
})
|
||||
}
|
||||
return rows, nil
|
||||
}
|
||||
|
||||
// formatStorageListEntry renders a StorageListEntry into the legacy
|
||||
// host:container[:options] colon form for textual output.
|
||||
func formatStorageListEntry(entry StorageListEntry) string {
|
||||
if entry.VolumeOptions == "" {
|
||||
return fmt.Sprintf("%s:%s", entry.HostPath, entry.ContainerPath)
|
||||
}
|
||||
return fmt.Sprintf("%s:%s:%s", entry.HostPath, entry.ContainerPath, entry.VolumeOptions)
|
||||
}
|
||||
|
||||
// ParseMountPath parses a mount path into its components
|
||||
func ParseMountPath(mountPath string) StorageListEntry {
|
||||
parts := strings.SplitN(mountPath, ":", 3)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
@@ -8,7 +9,6 @@ import (
|
||||
"strings"
|
||||
|
||||
"github.com/dokku/dokku/plugins/common"
|
||||
dockeroptions "github.com/dokku/dokku/plugins/docker-options"
|
||||
)
|
||||
|
||||
const (
|
||||
@@ -19,11 +19,19 @@ Manage mounted volumes
|
||||
Additional commands:`
|
||||
|
||||
helpContent = `
|
||||
storage:ensure-directory [--chown option] <directory>, Creates a persistent storage directory in the recommended storage path
|
||||
storage:create <name> [<path>] [flags], Register a named storage entry
|
||||
storage:destroy <name>, Remove a named storage entry (must be unmounted from every app first)
|
||||
storage:ensure-directory [--chown option] <directory>, [DEPRECATED] use storage:create instead
|
||||
storage:exec <name> [-- <cmd>...], Run a command (or shell) in a temporary container that mounts the entry
|
||||
storage:info <name> [--format text|json], Show details for one storage entry
|
||||
storage:list <app> [--format text|json], List bind mounts for app's container(s) (host:container)
|
||||
storage:list-entries [--scheduler s] [--format text|json], List registered storage entries
|
||||
storage:migrate [<app>|--all], Re-run the legacy -v to attachment migration for an app
|
||||
storage:mount <app> <host-dir:container-dir>, Create a new bind mount
|
||||
storage:report [<app>] [<flag>], Displays a storage report for one or more apps
|
||||
storage:unmount <app> <host-dir:container-dir>, Remove an existing bind mount`
|
||||
storage:set <name> [flags], Update a storage entry in place
|
||||
storage:unmount <app> <host-dir:container-dir>, Remove an existing bind mount
|
||||
storage:wait <name>, Wait for a storage entry's PVC to be bound (k3s)`
|
||||
)
|
||||
|
||||
// CommandHelp displays help for the storage plugin
|
||||
@@ -38,7 +46,7 @@ func CommandEnsureDirectory(directory string, chownFlag string) error {
|
||||
return err
|
||||
}
|
||||
|
||||
chownID, err := resolveChownID(chownFlag)
|
||||
chownID, err := ResolveChownID(chownFlag)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -72,8 +80,8 @@ func CommandEnsureDirectory(directory string, chownFlag string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// resolveChownID converts a chown flag value to a numeric UID
|
||||
func resolveChownID(chownFlag string) (string, error) {
|
||||
// ResolveChownID converts a chown flag value to a numeric UID
|
||||
func ResolveChownID(chownFlag string) (string, error) {
|
||||
var chownID string
|
||||
|
||||
switch chownFlag {
|
||||
@@ -121,41 +129,167 @@ func isUserNamespacesEnabled() (bool, error) {
|
||||
return strings.TrimSpace(result.StdoutContents()) == "true", nil
|
||||
}
|
||||
|
||||
// CommandMount creates a new bind mount for an app
|
||||
func CommandMount(appName string, mountPath string) error {
|
||||
if err := common.VerifyAppName(appName); err != nil {
|
||||
// CommandMountInput captures the optional flags accepted by storage:mount
|
||||
// when the second argument is a named entry rather than a colon-form path.
|
||||
type CommandMountInput struct {
|
||||
AppName string
|
||||
NameOrPath string
|
||||
ContainerDir string
|
||||
Phases []string
|
||||
ProcessType string
|
||||
Subpath string
|
||||
Readonly bool
|
||||
VolumeOptions string
|
||||
VolumeChown string
|
||||
}
|
||||
|
||||
// CommandMount creates a new bind mount for an app. The second positional
|
||||
// argument may be either a legacy host:container[:options] string (kept
|
||||
// for back-compat on docker-local) or a registered storage entry name.
|
||||
func CommandMount(input CommandMountInput) error {
|
||||
if err := common.VerifyAppName(input.AppName); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Legacy colon form: synthesize a legacy-<hash> entry plus an
|
||||
// attachment so storage:list (now attachment-only) sees the mount.
|
||||
// The storage docker-args trigger emits the corresponding -v flag at
|
||||
// deploy time, so behavior at the docker-run boundary is unchanged.
|
||||
if strings.Contains(input.NameOrPath, ":") {
|
||||
return mountLegacyColon(input.AppName, input.NameOrPath)
|
||||
}
|
||||
|
||||
// Named-entry form: persist as an attachment.
|
||||
if !EntryExists(input.NameOrPath) {
|
||||
return fmt.Errorf("storage entry %q does not exist; create it first with `dokku storage:create`", input.NameOrPath)
|
||||
}
|
||||
if input.ContainerDir == "" {
|
||||
return errors.New("--container-dir is required when mounting a named storage entry")
|
||||
}
|
||||
|
||||
entry, err := LoadEntry(input.NameOrPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
phases := input.Phases
|
||||
if len(phases) == 0 {
|
||||
phases = []string{PhaseDeploy, PhaseRun}
|
||||
}
|
||||
|
||||
processType := input.ProcessType
|
||||
if processType == "" {
|
||||
processType = DefaultProcessType
|
||||
}
|
||||
|
||||
attachment := &Attachment{
|
||||
EntryName: entry.Name,
|
||||
ContainerPath: input.ContainerDir,
|
||||
Phases: phases,
|
||||
ProcessType: processType,
|
||||
Subpath: input.Subpath,
|
||||
Readonly: input.Readonly,
|
||||
VolumeOptions: input.VolumeOptions,
|
||||
VolumeChown: input.VolumeChown,
|
||||
}
|
||||
|
||||
return AddAttachment(input.AppName, attachment)
|
||||
}
|
||||
|
||||
// CommandUnmountInput captures the flags accepted by storage:unmount when
|
||||
// the second argument is a named entry.
|
||||
type CommandUnmountInput struct {
|
||||
AppName string
|
||||
NameOrPath string
|
||||
ContainerDir string
|
||||
}
|
||||
|
||||
// CommandUnmount removes an existing bind mount from an app.
|
||||
func CommandUnmount(input CommandUnmountInput) error {
|
||||
if err := common.VerifyAppName(input.AppName); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if strings.Contains(input.NameOrPath, ":") {
|
||||
return unmountLegacyColon(input.AppName, input.NameOrPath)
|
||||
}
|
||||
|
||||
return RemoveAttachment(input.AppName, input.NameOrPath, input.ContainerDir)
|
||||
}
|
||||
|
||||
// mountLegacyColon translates a `<host>:<container>[:options]` mount
|
||||
// string into a synthesized legacy-<hash> entry plus an attachment.
|
||||
// Idempotent: re-running the same mount errors with the existing
|
||||
// "already mounted" message via AddAttachment's duplicate check.
|
||||
func mountLegacyColon(appName string, mountPath string) error {
|
||||
if err := VerifyPaths(mountPath); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if CheckIfPathExists(appName, mountPath, MountPhases) {
|
||||
return errors.New("Mount path already exists.")
|
||||
parsed := ParseMountPath(mountPath)
|
||||
if parsed.ContainerPath == "" {
|
||||
return errors.New("Storage path must be two valid paths divided by colon.")
|
||||
}
|
||||
|
||||
return dockeroptions.AddDockerOptionToPhases(appName, MountPhases, fmt.Sprintf("-v %s", mountPath))
|
||||
}
|
||||
entry := LegacyMountToEntry(mountPath)
|
||||
if !EntryExists(entry.Name) {
|
||||
if err := SaveEntry(entry); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// CommandUnmount removes an existing bind mount from an app
|
||||
func CommandUnmount(appName string, mountPath string) error {
|
||||
if err := common.VerifyAppName(appName); err != nil {
|
||||
attachment := &Attachment{
|
||||
EntryName: entry.Name,
|
||||
ContainerPath: parsed.ContainerPath,
|
||||
Phases: []string{PhaseDeploy, PhaseRun},
|
||||
ProcessType: DefaultProcessType,
|
||||
}
|
||||
switch parsed.VolumeOptions {
|
||||
case "":
|
||||
case "ro":
|
||||
attachment.Readonly = true
|
||||
default:
|
||||
attachment.VolumeOptions = parsed.VolumeOptions
|
||||
}
|
||||
|
||||
if err := AddAttachment(appName, attachment); err != nil {
|
||||
// AddAttachment's duplicate error mentions the entry name, but
|
||||
// the legacy form historically said "Mount path already
|
||||
// exists." - preserve that exact wording so existing automation
|
||||
// and the bats suite keep matching.
|
||||
if strings.Contains(err.Error(), "is already mounted at") {
|
||||
return errors.New("Mount path already exists.")
|
||||
}
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// unmountLegacyColon is the inverse of mountLegacyColon. The legacy
|
||||
// mount string identifies an entry+container-path tuple deterministically
|
||||
// via LegacyMountToEntry, so we can route to RemoveAttachment.
|
||||
func unmountLegacyColon(appName string, mountPath string) error {
|
||||
if err := VerifyPaths(mountPath); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if !CheckIfPathExists(appName, mountPath, MountPhases) {
|
||||
return errors.New("Mount path does not exist.")
|
||||
parsed := ParseMountPath(mountPath)
|
||||
if parsed.ContainerPath == "" {
|
||||
return errors.New("Storage path must be two valid paths divided by colon.")
|
||||
}
|
||||
|
||||
return dockeroptions.RemoveDockerOptionFromPhases(appName, MountPhases, fmt.Sprintf("-v %s", mountPath))
|
||||
entry := LegacyMountToEntry(mountPath)
|
||||
if err := RemoveAttachment(appName, entry.Name, parsed.ContainerPath); err != nil {
|
||||
// Match the legacy wording for "not currently mounted".
|
||||
if strings.Contains(err.Error(), "is not mounted") {
|
||||
return errors.New("Mount path does not exist.")
|
||||
}
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// CommandList lists all bind mounts for an app
|
||||
// CommandList lists all bind mounts for an app. Reads attachments
|
||||
// directly from the storage plugin's own state rather than going through
|
||||
// the deprecated `storage-list` plugn trigger.
|
||||
func CommandList(appName string, format string) error {
|
||||
if err := common.VerifyAppName(appName); err != nil {
|
||||
return err
|
||||
@@ -165,39 +299,45 @@ func CommandList(appName string, format string) error {
|
||||
return errors.New("Invalid --format value specified")
|
||||
}
|
||||
|
||||
if format == "text" {
|
||||
common.LogInfo1Quiet(fmt.Sprintf("%s volume bind-mounts:", appName))
|
||||
}
|
||||
|
||||
results, err := common.CallPlugnTrigger(common.PlugnTriggerInput{
|
||||
Trigger: "storage-list",
|
||||
Args: []string{appName, "deploy", format},
|
||||
})
|
||||
rows, err := ListAppMountEntries(appName, PhaseDeploy)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
output := results.StdoutContents()
|
||||
if format == "text" && output != "" {
|
||||
lines := strings.Split(strings.TrimSpace(output), "\n")
|
||||
for _, line := range lines {
|
||||
if line != "" {
|
||||
if os.Getenv("DOKKU_QUIET_OUTPUT") != "" {
|
||||
fmt.Println(line)
|
||||
} else {
|
||||
common.LogVerbose(line)
|
||||
}
|
||||
}
|
||||
if format == "json" {
|
||||
output, err := json.Marshal(rows)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
} else if output != "" {
|
||||
fmt.Println(output)
|
||||
fmt.Println(string(output))
|
||||
return nil
|
||||
}
|
||||
|
||||
common.LogInfo1Quiet(fmt.Sprintf("%s volume bind-mounts:", appName))
|
||||
for _, row := range rows {
|
||||
line := formatStorageListEntry(row)
|
||||
if os.Getenv("DOKKU_QUIET_OUTPUT") != "" {
|
||||
fmt.Println(line)
|
||||
} else {
|
||||
common.LogVerbose(line)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// CommandReport displays a storage report for one or more apps
|
||||
func CommandReport(appName string, format string, infoFlag string) error {
|
||||
if appName == "--global" {
|
||||
// Global storage report: list every registered entry and its
|
||||
// attachment count; falls back to the legacy per-app loop when
|
||||
// no entries exist so existing automation keeps working.
|
||||
reportFormat := format
|
||||
if reportFormat == "stdout" {
|
||||
reportFormat = "text"
|
||||
}
|
||||
return CommandReportGlobal(reportFormat)
|
||||
}
|
||||
|
||||
if appName == "" {
|
||||
apps, err := common.DokkuApps()
|
||||
if err != nil {
|
||||
|
||||
@@ -6,12 +6,18 @@ import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
|
||||
"github.com/dokku/dokku/plugins/common"
|
||||
)
|
||||
|
||||
// TriggerInstall sets up the storage plugin on installation
|
||||
// TriggerInstall sets up the storage plugin on installation and runs
|
||||
// the bulk legacy-mount migration once per upgrade.
|
||||
func TriggerInstall() error {
|
||||
if err := common.PropertySetup(PluginName); err != nil {
|
||||
return fmt.Errorf("Unable to install the storage plugin: %s", err.Error())
|
||||
}
|
||||
|
||||
storageDir := GetStorageDirectory()
|
||||
|
||||
if err := os.MkdirAll(storageDir, 0755); err != nil {
|
||||
@@ -25,6 +31,29 @@ func TriggerInstall() error {
|
||||
return fmt.Errorf("Unable to set storage directory permissions: %s", err.Error())
|
||||
}
|
||||
|
||||
if err := EnsureEntriesDirectory(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// The install trigger runs as root; the directories it just made are
|
||||
// root-owned, but the dokku user is the one that runs storage:create
|
||||
// and writes entry / migration-flag files into them.
|
||||
for _, dir := range []string{RegistryDirectory(), EntriesDirectory(), migrationFlagDir()} {
|
||||
if err := os.MkdirAll(dir, 0755); err != nil {
|
||||
return fmt.Errorf("Unable to create %s: %s", dir, err.Error())
|
||||
}
|
||||
if err := common.SetPermissions(common.SetPermissionInput{
|
||||
Filename: dir,
|
||||
Mode: 0755,
|
||||
}); err != nil {
|
||||
return fmt.Errorf("Unable to set permissions on %s: %s", dir, err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
if err := MigrateLegacyMounts(); err != nil {
|
||||
return fmt.Errorf("storage migration failed: %w", err)
|
||||
}
|
||||
|
||||
distro := detectDistro()
|
||||
if distro == "" {
|
||||
return nil
|
||||
@@ -44,6 +73,52 @@ func TriggerInstall() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// TriggerPostDelete removes the attachment store for an app that's being
|
||||
// destroyed. Entries are global and survive app deletion. The whole
|
||||
// per-app property folder goes away rather than being rewritten as
|
||||
// empty, matching what other plugins do on app deletion.
|
||||
func TriggerPostDelete(appName string) error {
|
||||
if appName == "" {
|
||||
return nil
|
||||
}
|
||||
return common.PropertyDestroy(PluginName, appName)
|
||||
}
|
||||
|
||||
// TriggerPostAppCloneSetup copies attachments from the source app to the
|
||||
// cloned app. Entries are global so no entry-level work is needed.
|
||||
func TriggerPostAppCloneSetup(oldName string, newName string) error {
|
||||
if oldName == "" || newName == "" {
|
||||
return nil
|
||||
}
|
||||
if !common.PropertyExists(PluginName, oldName, AttachmentsProperty) {
|
||||
return nil
|
||||
}
|
||||
attachments, err := LoadAttachments(oldName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return SaveAttachments(newName, attachments)
|
||||
}
|
||||
|
||||
// TriggerPostAppRenameSetup moves attachments from the old name to the
|
||||
// new name and removes the old per-app property folder.
|
||||
func TriggerPostAppRenameSetup(oldName string, newName string) error {
|
||||
if oldName == "" || newName == "" {
|
||||
return nil
|
||||
}
|
||||
if !common.PropertyExists(PluginName, oldName, AttachmentsProperty) {
|
||||
return nil
|
||||
}
|
||||
attachments, err := LoadAttachments(oldName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := SaveAttachments(newName, attachments); err != nil {
|
||||
return err
|
||||
}
|
||||
return common.PropertyDestroy(PluginName, oldName)
|
||||
}
|
||||
|
||||
// detectDistro returns the Linux distribution name
|
||||
func detectDistro() string {
|
||||
if runtime.GOOS != "linux" {
|
||||
@@ -65,29 +140,114 @@ func detectDistro() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
// TriggerStorageList outputs storage mounts for an app
|
||||
// TriggerStorageList outputs storage mounts for an app.
|
||||
//
|
||||
// Deprecated: the storage-list plugn trigger is retained for back-compat
|
||||
// with external plugins that may still call it, but in-process callers
|
||||
// should use storage.ListAppMountEntries directly. A deprecation warning
|
||||
// is emitted on every invocation.
|
||||
func TriggerStorageList(appName string, phase string, format string) error {
|
||||
mounts, err := GetBindMounts(appName, phase)
|
||||
common.LogWarn("the storage-list plugn trigger is deprecated; use the storage-app-mounts trigger or the storage Go package directly")
|
||||
|
||||
rows, err := ListAppMountEntries(appName, phase)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if format == "json" {
|
||||
entries := []StorageListEntry{}
|
||||
for _, mount := range mounts {
|
||||
entries = append(entries, ParseMountPath(mount))
|
||||
}
|
||||
|
||||
output, err := json.Marshal(entries)
|
||||
output, err := json.Marshal(rows)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Println(string(output))
|
||||
} else {
|
||||
for _, mount := range mounts {
|
||||
fmt.Println(mount)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
for _, row := range rows {
|
||||
fmt.Println(formatStorageListEntry(row))
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// AppMountPair pairs an Attachment with the Entry it references; consumed
|
||||
// by scheduler plugins via the storage-app-mounts trigger.
|
||||
type AppMountPair struct {
|
||||
Entry *Entry `json:"entry"`
|
||||
Attachment *Attachment `json:"attachment"`
|
||||
}
|
||||
|
||||
// TriggerStorageAppMounts emits the (entry, attachment) pairs an app has
|
||||
// for the given phase, in JSON. Schedulers consume this at deploy time.
|
||||
func TriggerStorageAppMounts(appName string, phase string) error {
|
||||
if phase == "" {
|
||||
phase = PhaseDeploy
|
||||
}
|
||||
attachments, err := AttachmentsForPhase(appName, phase)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
pairs := []AppMountPair{}
|
||||
for _, attachment := range attachments {
|
||||
entry, err := LoadEntry(attachment.EntryName)
|
||||
if err != nil {
|
||||
return fmt.Errorf("attachment on %q references missing entry %q: %w", appName, attachment.EntryName, err)
|
||||
}
|
||||
pairs = append(pairs, AppMountPair{Entry: entry, Attachment: attachment})
|
||||
}
|
||||
|
||||
output, err := json.Marshal(pairs)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Println(string(output))
|
||||
return nil
|
||||
}
|
||||
|
||||
// TriggerDockerArgs emits `-v` flags for each docker-local attachment in
|
||||
// the requested phase. Plugn concatenates this with docker-options'
|
||||
// equivalent trigger output, so docker-local apps continue to receive
|
||||
// their bind mounts through the standard pipeline.
|
||||
func TriggerDockerArgs(appName string, phase string) error {
|
||||
attachments, err := AttachmentsForPhase(appName, phase)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for _, attachment := range attachments {
|
||||
entry, err := LoadEntry(attachment.EntryName)
|
||||
if err != nil {
|
||||
return fmt.Errorf("attachment on %q references missing entry %q: %w", appName, attachment.EntryName, err)
|
||||
}
|
||||
if entry.Scheduler != SchedulerDockerLocal {
|
||||
continue
|
||||
}
|
||||
flag := buildDockerVFlag(entry, attachment)
|
||||
if flag == "" {
|
||||
continue
|
||||
}
|
||||
fmt.Printf(" %s", flag)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// buildDockerVFlag formats the Docker -v argument for a docker-local
|
||||
// attachment.
|
||||
func buildDockerVFlag(entry *Entry, attachment *Attachment) string {
|
||||
host := entry.HostPath
|
||||
container := attachment.ContainerPath
|
||||
if host == "" || container == "" {
|
||||
return ""
|
||||
}
|
||||
options := []string{}
|
||||
if attachment.Readonly {
|
||||
options = append(options, "ro")
|
||||
}
|
||||
if attachment.VolumeOptions != "" {
|
||||
options = append(options, attachment.VolumeOptions)
|
||||
}
|
||||
flag := fmt.Sprintf("-v %s:%s", host, container)
|
||||
if len(options) > 0 {
|
||||
flag = fmt.Sprintf("%s:%s", flag, strings.Join(options, ","))
|
||||
}
|
||||
return flag
|
||||
}
|
||||
|
||||
@@ -183,3 +183,217 @@ teardown() {
|
||||
assert_success
|
||||
assert_output "mount_volume:/mount"
|
||||
}
|
||||
|
||||
@test "(storage) storage:create / storage:list-entries / storage:destroy" {
|
||||
run /bin/bash -c "dokku storage:create rdmtest-entry"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_success
|
||||
|
||||
run /bin/bash -c "dokku storage:list-entries --format json | jq -r '.[].name' | grep '^rdmtest-entry$'"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_success
|
||||
assert_output "rdmtest-entry"
|
||||
|
||||
run /bin/bash -c "dokku storage:info rdmtest-entry --format json | jq -r '.scheduler'"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_success
|
||||
assert_output "docker-local"
|
||||
|
||||
run /bin/bash -c "dokku storage:destroy rdmtest-entry"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_success
|
||||
}
|
||||
|
||||
@test "(storage) storage:create rejects invalid names" {
|
||||
# underscore: rejected
|
||||
run /bin/bash -c "dokku storage:create rdmtest_invalid"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_failure
|
||||
|
||||
# uppercase: rejected
|
||||
run /bin/bash -c "dokku storage:create RdmTest"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_failure
|
||||
|
||||
# 46 chars: too long
|
||||
long_name=$(printf 'a%.0s' {1..46})
|
||||
run /bin/bash -c "dokku storage:create $long_name"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_failure
|
||||
|
||||
# legacy- prefix: reserved
|
||||
run /bin/bash -c "dokku storage:create legacy-foo"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_failure
|
||||
}
|
||||
|
||||
@test "(storage) storage:create + storage:mount with named entry attaches multiple entries to one app" {
|
||||
run /bin/bash -c "dokku storage:create rdmtest-data"
|
||||
assert_success
|
||||
run /bin/bash -c "dokku storage:create rdmtest-cache"
|
||||
assert_success
|
||||
|
||||
run /bin/bash -c "dokku storage:mount $TEST_APP rdmtest-data --container-dir /data"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_success
|
||||
|
||||
run /bin/bash -c "dokku storage:mount $TEST_APP rdmtest-cache --container-dir /cache"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_success
|
||||
|
||||
run /bin/bash -c "dokku storage:list $TEST_APP --format json | jq -r '. | length'"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_success
|
||||
assert_output "2"
|
||||
|
||||
run /bin/bash -c "dokku storage:list $TEST_APP --format json | jq -r '.[].entry_name' | sort | xargs"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_success
|
||||
assert_output "rdmtest-cache rdmtest-data"
|
||||
|
||||
# cleanup
|
||||
run /bin/bash -c "dokku storage:unmount $TEST_APP rdmtest-data"
|
||||
assert_success
|
||||
run /bin/bash -c "dokku storage:unmount $TEST_APP rdmtest-cache"
|
||||
assert_success
|
||||
|
||||
run /bin/bash -c "dokku storage:list $TEST_APP --format json | jq -r '. | length'"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_success
|
||||
assert_output "0"
|
||||
|
||||
run /bin/bash -c "dokku storage:destroy rdmtest-data"
|
||||
assert_success
|
||||
run /bin/bash -c "dokku storage:destroy rdmtest-cache"
|
||||
assert_success
|
||||
}
|
||||
|
||||
@test "(storage) storage:destroy refuses to remove a still-mounted entry" {
|
||||
run /bin/bash -c "dokku storage:create rdmtest-busy"
|
||||
assert_success
|
||||
run /bin/bash -c "dokku storage:mount $TEST_APP rdmtest-busy --container-dir /data"
|
||||
assert_success
|
||||
|
||||
run /bin/bash -c "dokku storage:destroy rdmtest-busy"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_failure
|
||||
assert_output_contains "still mounted"
|
||||
|
||||
run /bin/bash -c "dokku storage:unmount $TEST_APP rdmtest-busy"
|
||||
assert_success
|
||||
run /bin/bash -c "dokku storage:destroy rdmtest-busy"
|
||||
assert_success
|
||||
}
|
||||
|
||||
@test "(storage) storage:exec runs a non-interactive command and propagates exit code" {
|
||||
run /bin/bash -c "dokku storage:create rdmtest-exec"
|
||||
assert_success
|
||||
|
||||
# docker pull image up-front so the test isn't gated on registry latency.
|
||||
run /bin/bash -c "docker pull alpine:3 >/dev/null"
|
||||
assert_success
|
||||
|
||||
run /bin/bash -c "dokku storage:exec rdmtest-exec -- /bin/sh -c 'touch /data/marker && ls /data/marker'"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_success
|
||||
assert_output_contains "/data/marker"
|
||||
|
||||
run /bin/bash -c "dokku storage:exec rdmtest-exec -- /bin/sh -c 'exit 42'"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_equal "$status" 42
|
||||
|
||||
run /bin/bash -c "dokku storage:destroy rdmtest-exec"
|
||||
assert_success
|
||||
}
|
||||
|
||||
@test "(storage) storage:migrate converts legacy docker-options -v lines into attachments" {
|
||||
# Stage a legacy `-v` line directly via docker-options:add. This
|
||||
# bypasses the new storage:mount path so the line lives in
|
||||
# docker-options without a corresponding attachment, simulating an
|
||||
# app upgraded from a pre-PR Dokku version.
|
||||
run /bin/bash -c "dokku docker-options:add $TEST_APP deploy,run \"-v /tmp/legacy-mount:/legacy\""
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_success
|
||||
|
||||
# Before migration, storage:list shows nothing (attachment-only) and
|
||||
# docker-options has the -v line.
|
||||
run /bin/bash -c "dokku storage:list $TEST_APP --format json | jq -r '. | length'"
|
||||
assert_success
|
||||
assert_output "0"
|
||||
|
||||
run /bin/bash -c "dokku docker-options:report $TEST_APP --docker-options-deploy"
|
||||
assert_success
|
||||
assert_output_contains "-v /tmp/legacy-mount:/legacy"
|
||||
|
||||
# Run the migration for this app.
|
||||
run /bin/bash -c "dokku storage:migrate $TEST_APP"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_success
|
||||
|
||||
# storage:list now surfaces the synthesized colon form.
|
||||
run /bin/bash -c "dokku --quiet storage:list $TEST_APP"
|
||||
assert_success
|
||||
assert_output "/tmp/legacy-mount:/legacy"
|
||||
|
||||
# The synthesized entry shows up under the legacy- prefix.
|
||||
run /bin/bash -c "dokku storage:list-entries --format json | jq -r '.[].name' | grep '^legacy-' | head -1"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_success
|
||||
|
||||
# docker-options no longer holds the -v line on either phase.
|
||||
run /bin/bash -c "dokku docker-options:report $TEST_APP --docker-options-deploy"
|
||||
assert_success
|
||||
assert_output_not_contains "-v /tmp/legacy-mount:/legacy"
|
||||
|
||||
run /bin/bash -c "dokku docker-options:report $TEST_APP --docker-options-run"
|
||||
assert_success
|
||||
assert_output_not_contains "-v /tmp/legacy-mount:/legacy"
|
||||
|
||||
# Idempotency: re-running storage:migrate is a no-op (still one
|
||||
# attachment, no errors).
|
||||
run /bin/bash -c "dokku storage:migrate $TEST_APP"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_success
|
||||
|
||||
run /bin/bash -c "dokku storage:list $TEST_APP --format json | jq -r '. | length'"
|
||||
assert_success
|
||||
assert_output "1"
|
||||
|
||||
# Cleanup: unmount + destroy the synthesized legacy entry. We need
|
||||
# the entry name from list-entries since legacy-<hash> is content-derived.
|
||||
legacy_name=$(dokku storage:list-entries --format json | jq -r '.[] | select(.name | startswith("legacy-")) | .name' | head -1)
|
||||
if [[ -n "$legacy_name" ]]; then
|
||||
run /bin/bash -c "dokku storage:unmount $TEST_APP $legacy_name --container-dir /legacy"
|
||||
assert_success
|
||||
run /bin/bash -c "dokku storage:destroy $legacy_name"
|
||||
assert_success
|
||||
fi
|
||||
}
|
||||
|
||||
@test "(storage) storage:ensure-directory emits deprecation warning" {
|
||||
run /bin/bash -c "dokku storage:ensure-directory $TEST_APP 2>&1"
|
||||
echo "output: $output"
|
||||
echo "status: $status"
|
||||
assert_success
|
||||
assert_output_contains "deprecated"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user