An attachment's process type was recorded, reported and used to scope `storage:mount --replace` and `storage:unmount --all`, but nothing read it when the volume was actually mounted, so `storage:mount <app> <entry> --container-dir <path> --process-type web` bound the volume into every process rather than only `web`. Both schedulers now scope a mount to the process type it was declared for, and containers that belong to no process type - a `dokku run` one-off, an `app.json` deploy task, a k3s cron job - receive default-scoped mounts only. A container path can now be claimed by a single mount per process type, and `storage:migrate` drains process-scoped `-v` lines out of `docker-options` rather than only the default-scoped ones.
The test used `DOKKU_IMAGE` as a variable that does not cross the privilege drop, which stopped being true once every environment-overridable variable was forwarded. It now reads the `--preserve-env` argument itself, so it tests the contract rather than one member of the list.
`DOKKU_INIT_SYSTEM` reaches the `dokku` user only through the environment while the Docker image is being built, so narrowing the forwarded set left `nginx:reload` taking the systemd branch during `debian/postinst` and blocking on a sudo password prompt until the build timed out. The forwarded set now covers every variable `dokku` honours from the environment, and `DOKKU_PRESERVE_ENV` still carries anything beyond it.
`sudo-rs`, the default sudo on Ubuntu 25.10 and later, does not support preserving the entire environment, so `--quiet`, `--trace`, `--force` and `--app` stopped taking effect once `dokku` re-executed itself as the `dokku` user, and a warning was printed on every invocation. Variables outside the set dokku forwards no longer cross that boundary and can be named in `DOKKU_PRESERVE_ENV` instead.
The datastore destroys were written with dokku's global `--force`, which works only because dokku translates it into `DOKKU_APPS_FORCE_DELETE` and the plugin reads that environment variable back. The flag belongs to the subcommand, which declares it, so it is passed there instead.
Only the datastore calls move. `network:destroy` and `storage:destroy` are dokku's own commands and keep dokku's own convention.
The ports and app-json tests passed the app name to `postgres:create` and `redis:create` a second time, as a positional the command has never taken. The bash datastore plugins parsed their arguments with getopts and simply stopped at it; those plugins now dispatch to a binary that reports the count instead, so every dokku pull request fails on `This command requires 2 arguments, 3 arguments given`.
The documented form takes the service alone, as `docs/deployment/application-deployment.md` shows. The `:link` and `:unlink` calls beside these are unchanged, since those do take two.
The `storage` plugin mounted and unmounted one attachment at a time, so matching an app's mounts to a declared set meant reading `storage:list`, computing the difference in both directions and issuing one `storage:mount` or `storage:unmount` per addition and removal, with a failure partway through leaving the app holding a mixture of the two sets.
A `--replace` flag on `storage:mount` now writes the declared set in a single call, taking `name:container-dir` pairs whose optional third field carries the same mount options the legacy colon form accepts. Every pair is parsed and validated before anything is written, so a rejected pair leaves the stored mounts untouched. The remaining mount-time flags scope the whole call the way `--process-type` already scoped a single mount, which means an omitted `--process-type` replaces the `_default_` scope and leaves mounts scoped to a named process type in place.
An empty pair list is rejected rather than treated as a request to unmount everything, so a generated list that expands to nothing cannot silently drop an app's storage. `storage:unmount --all` handles that case instead, taking an optional `--process-type` filter and treating a scope that matches nothing as a no-op so it stays idempotent.
`storage:unmount` also accepts more than one mount at a time, naming each by entry name or by the colon form, and resolves every argument against the app's attachments before removing any of them.
The help listing now carries one entry per subcommand with its flags inline rather than a row per variant, and the command listing at the top of the storage documentation matches it line for line. Folding those rows together also surfaced `--global` on `storage:report` and `storage:migrate`, neither of which both listings had mentioned.
Finally, `--volume-chown` is now validated on mount, where a typo previously exited zero and wrote a value that only the report ever read.
There was no way to ask whether Dokku was already logged in to a registry with a given credential, so tooling converging a declared state onto a server had to re-run `registry:login` on every pass purely to discover that nothing needed to change. The new `registry:auth-status` command compares a supplied credential against the stored one and reports the result through its exit code alone, distinguishing a server with no credential from one whose credential differs, and from one whose secret a docker credential helper or an identity token puts out of reach. An `auth-servers` triad on `registry:report` exposes which servers have credentials without emitting any part of one.
Docker stores Docker Hub under the index server address rather than the hostname the user logged in with, and `docker logout` only erases the exact key it is given, so `registry:logout docker.io` reported success while leaving the credential in place. The server is now resolved to the key docker actually stores before the logout runs.
The wrapper config used to run `nginx -t` against an app's rendered `nginx.conf` declared no `user` directive, so nginx fell back to its compile-time default and chowned every `proxy_cache_path` directory to that user, including directories that already existed. On Debian and Ubuntu that default is `nobody`, so deploying an app shipping a custom `nginx.conf.sigil` took the running server's live cache directory away for the length of the build and turned every request the cache would serve into an HTTP 500, healing only once the deploy's own reload restored the ownership. The wrapper now declares the same `user` as the host's main nginx config, which is also exposed to custom `validate-config` templates as `$.NGINX_USER`.
`scheduler-k3s:ensure-charts` announced the pinned chart version but installed whatever the chart repository had published most recently, and an affected cluster could not be brought back in line because a version mismatch was reported and then skipped. Charts now install at the version dokku pins, and a release that has drifted is reconciled back to it. Dokku also no longer registers helm repositories it never read, and the previously undocumented `ensure-charts` and `initialize` flags now appear in the help output.
The `scheduler-k3s` metadata commands were all per-key, so matching an app's stored annotations, labels or node sysctls to a declared set meant reading the report, computing the difference in both directions and issuing one command per added, changed and removed key, with a failure partway through leaving a mixture of the two sets. A `--replace` flag on `annotations:set`, `labels:set`, `node-sysctls:set` and `autoscaling-auth:set` now writes the declared map in a single call, taking trailing `key=value` pairs everywhere except `autoscaling-auth:set`, which keeps its repeatable `--metadata` flag. Every pair is parsed before anything is written, so a rejected pair leaves the stored map untouched, and a pair ending in `=` stores an empty value, which the per-key form cannot express. An empty pair list is rejected rather than treated as a request to remove everything, so a generated list that expands to nothing cannot silently drop an app's metadata; `annotations:clear`, `labels:clear` and `node-sysctls:clear` handle that case, taking the same optional filters as their report commands. The `--resource-type` flag and the app name are now validated on the set commands as well, where a typo previously exited zero and wrote a property that no report and no deploy ever read.
A process count below zero was accepted and stored verbatim, leaving the scheduler to reject the value long after the command reported success. Counts below zero are now rejected, as are process tuples that omit a process type, across `ps:scale`, the `ps-set-scale` trigger and the `formation` key of `app.json`. A negative count persisted by an earlier version stays readable so it can be corrected with `ps:scale`.
`config:set`, `config:unset` and `config:clear` discarded the result of writing the app's environment, so a write that never landed still printed `Setting config vars`, still fired the `post-config-update` trigger, still restarted the app and still exited 0. The `config-set` and `config-unset` plugin triggers behaved the same way, which meant an `app.json` `env` block that could not be persisted deployed anyway with the variables missing. The environment is now staged in a temporary file and renamed into place, so a write that fails partway leaves the previous environment intact, and that file is created at `0600` rather than being created at `0644` and chmodded afterwards. `apps:clone` and `apps:rename` never applied that chmod at all and left the new app's environment world-readable.
cmd-git-sync calls dokku_setup_build_capture before the clone/fetch,
because it cannot know whether there is anything to deploy until it has
fetched. Both paths that then decline to build - a plain `git:sync` with
no build flag, and a `--build-if-changes` poll that finds the remote
unchanged - returned without ever finalizing that record.
The record stayed at status=running forever, which has three
consequences on an app polled on a timer:
- PruneAppBuilds only runs from builds-record-finalize, so nothing
prunes the ticks while the app is idle; they accumulate without bound,
invisibly, because builds:list caps output at the retention count.
- The next real deploy's ReapAbandonedBuilds finalizes each of them as
status=failed, exit_code=-1, with finished_at stamped at the moment of
reaping - indistinguishable on disk from a build that really failed,
and with a duration that grew to the polling interval.
- That same deploy's prune then keeps the newest records by started_at,
which after an idle stretch are the ticks, deleting the last real
deploy's record and log.
Add a builds-record-discard trigger that removes a non-terminal record
and its log, and call it from both no-op paths in cmd-git-sync. The
trigger leaves missing and already-terminal records alone, so it can
never delete real history.
Fixes#9030
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The `git:auth-status` command returned `1` both when a host had no `netrc` entry and when it had one that differed, so tooling wanting to tell a credential that would be created apart from one that would be replaced had to issue a second call with no username purely to find out which of the two it was. A host with no entry now exits `1`, a host whose entry differs from what was passed exits `2`, and invalid arguments exit `3`, leaving `0` to mean the stored state already matches. Called without a username the requested state is that no entry exists, so an existing entry is reported as `2` and `1` is never returned. Callers that only test zero versus non-zero are unaffected.
The `ps:scale` command merges the specified process types into the existing formation, so replacing an entire formation from the CLI required reading the current scale, diffing it, and issuing a second call to zero out each unlisted process type. The `--replace` flag treats the specified process types as the whole formation and sets the process count for every unspecified process type to zero, while the `--clear` flag resets the formation to the scale a newly created app has. The two flags cannot be combined, `--replace` requires at least one process type, and `--clear` accepts none. Process types cleared this way are now scaled down as part of the same command rather than being left running with a stale formation.
The report described an installed certificate without carrying anything that identified it, so the only exact way to tell whether an app was serving a given certificate was to pull the full PEM back with `certs:show` and compare bytes. That moves certificate material off the server on every check, and the same command with `key` returns the private key. The new `--ssl-fingerprint` and `--ssl-serial` flags answer the same question in a single report read that discloses nothing sensitive. The sample output in the ssl documentation has also been corrected to match what the command actually prints.
The docker-args-process-deploy trigger initialized is_app_listening to
the string "false" and then tested it with [[ -n ... ]], which is always
true. Apps with domains, tls-internal, and no http/https port mapping
got a stray caddy.tls=internal label.
Fixes#8973
`scheduler-k3s:profiles:add` accepted profile names it could not later derive a valid helm release name from, so a profile was created successfully and only failed on commands with no obvious relationship to it: `profiles:remove` could not delete the profile it created, and a single bad name broke every subsequent node sysctls change, including the global scope and unrelated profiles. Names are now refused at add time against the release name they derive rather than against the profile name alone, and the paths that have to cope with a profile stored before this check keep working instead of erroring.
Cron containers were never reaped once they exceeded their active deadline, so a hung cron task ran indefinitely instead of being retired after 24 hours as documented. `cron:run` now also accepts a `--ttl-seconds` argument, matching the one `dokku run` already takes.
The dynamic `dns-provider-*` properties were reported as `--traefik-dns-provider-<env_var>`, outside the `global`/`computed` namespace every other traefik property uses, so a consumer reading the json report could not tell which of the two a key represented. They are now reported as `--traefik-global-dns-provider-<env_var>`. The `basic-auth-password` property is a credential as well and is now masked in the default report output on the same terms as the dns provider values, meaning the raw value is returned only for `--format json` or when the flag is queried by name.
The preserved copy holds everything the legacy file was not allowed to import, which includes keys that had been `config:unset` on purpose, so a revoked secret can outlive its revocation in a file `dokku config:*` no longer reads. Nothing said the copy was the operator's to remove once reviewed.
Releases 0.38.0 through 0.38.25 recorded the ENV file migration and deliberately left the file at the old path, so 0.38.26 treating a leftover file as a hand-edit replayed the environment as it stood at that upgrade over every `config:set` and `config:unset` made since, reinstating variables and secrets that had been deliberately unset. A file found at the old path once its migration is on record is no longer imported: it is removed when it agrees with the current config, and is otherwise moved aside to `ENV.migrated` with the keys it disagrees on named in a warning. That warning now reaches the operator during an upgrade rather than being swallowed, which is why the overwrite went unreported.
`storage:set` now takes `<name> <property> [<value>]` like every other `:set` command, where omitting the value unsets the property. Previously it took flags and could not distinguish an empty value from an omitted one, so nothing it set could ever be cleared. The flag form keeps working and emits a deprecation warning.
Annotations and labels move to `storage:annotations:set`, `storage:annotations:report`, `storage:labels:set`, and `storage:labels:report`, matching the `scheduler-k3s` equivalents. These operate on a single key, so clearing one leaves the rest in place rather than replacing the whole map as the `--annotation` and `--label` flags do.
`storage:create` and `storage:set` accept a `--mode` flag that sets the octal permissions of a docker-local host directory, and `storage:destroy` accepts a `--destroy-host-dir` flag that removes the directory along with its contents. A docker-local entry also honors `--reclaim-policy Delete` at destroy time now, matching how that policy governs a k3s PersistentVolume. Both are limited to the default `/var/lib/dokku/data/storage/<name>` location, the same restriction `--chown` already carries. `storage:set` applies `--chown` and `--mode` to the directory rather than only recording them.
The generated vector config is a snapshot of the app list and their sink properties, but it was only ever written by `logs:set` and `logs:vector-start`. Renaming an app left a source filtering on a label no container carries and gave the new name no source at all, so the app kept a sink with nothing feeding it. Cloning produced the same result for the clone, and destroying an app left its source and sink behind, the latter still pointing at an endpoint decommissioned along with the app. The global relabel transform embeds app names directly in generated VRL, so a rename also left behind a branch naming an app that no longer existed. Every case was silent, and the only repair was an operator running `logs:vector-start`. The `post-app-clone-setup`, `post-app-rename-setup` and `post-delete` triggers now rewrite the config, warning rather than failing so that a config write cannot abort the app operation whose state it is derived from.
Closes#8918.
The alias was only ever used to build the `include_labels` filter on the generated vector source, while dokku labels containers with `com.dokku.app-name` unconditionally. Setting the property therefore pointed the source at a label no container carries, and log collection stopped without any error. The source now always filters the label dokku applies, and a generated remap renames the field on its way to the sink, which is what the property was documented to do. An app whose own alias differs from the global one gets a branch in the global pipeline, so a per-app value is honored even when the app ships through the global sink.
Closes#8916.
The `app-label-alias` test left the global property set, so every later test in the file generated a vector source filtering on a label that dokku never applies to a container, silently collecting nothing. Clearing it in teardown restores log collection for the rest of the file. The cron routing test now asserts against console sinks rather than files, since the sink an event reached is identifiable from vector's own output without depending on a writable host mount, and its task sleeps either side of its output because a cron container that exits immediately is removed before vector can attach to it.
Scheduled cron task output previously reached only the `dokku` user's cron mail, and could not be redirected because `app.json` rejects bare shell operators in a cron `command`. Setting `vector-cron-sink` on an app or globally routes that output to a dedicated sink instead, on both the `docker-local` and `k3s` schedulers, which keeps log destinations under operator control rather than in a deployed repository. Cron events carry `dokku_app` and `dokku_cron_id` fields so a sink can give each task its own destination. This also fixes a `k3s` bug where configuring a global `vector-sink` silently removed the vector prometheus exporter sink.
Traefik matches hosts exactly, so an app serving a wildcard domain under the `traefik` ingress class had a valid certificate but silently 404d on every request. Wildcard domains now render as a `HostRegexp` rule that matches a single label, the same semantics as a Kubernetes wildcard host, so both ingress classes behave the same. Those routes carry an explicit low priority so an exact domain on any app still wins over another app's wildcard, mirroring ingress-nginx.
The `cert-issuer-name` and `cert-issuer-kind` properties point an app's generated `Certificate` at a cert-manager issuer created outside of Dokku, allowing certificates to be issued through solvers the built-in letsencrypt integration cannot use, such as `dns01` for wildcard certificates. Setting an issuer enables https on its own, as a manually managed issuer has no email for Dokku to configure. An imported certificate still takes precedence, and `letsencrypt-server false` remains the single off switch. Dokku warns before a build starts when the referenced issuer is absent from the cluster, without blocking the deploy. Wildcard domains no longer collide with their apex domain when generating ingress names, and `letsencrypt-server` values are now validated when set rather than at deploy time.
The usage strings for `scheduler-k3s:report` and `scheduler-k3s:set` omitted the `--global` option, which is the only way to reach the scheduler-wide report since a bare `scheduler-k3s:report` iterates every app, and `:report` also omitted `--format stdout|json`. The command listing in the k3s documentation is resynced with the help output, which additionally restores flags that had been dropped from `scheduler-k3s:cluster:add`, `scheduler-k3s:cluster:list`, and `scheduler-k3s:initialize`.
The hand-written marker file was created as root, which the config-migrate-env
trigger could not overwrite when it ran under a different user. Draining once up
front records the migration through the same code path the assertion exercises.
Install steps run in alphabetical order of the enabled plugin directory, so `apps`, `builder`, and `checks` read an app's environment before the `config` plugin had moved the `ENV` file to its new location. The read came back empty, so their deprecated `DOKKU_*` variables were never migrated to the matching plugin property and were never unset, with nothing reported either way: `dokku config:show` kept listing the variable while the plugin behaved as though it were unset. The relocation now runs before any deprecated variable is read, whatever the install order, and each old file is removed as soon as it has been drained rather than on a later install, which also covers the global file that was never removed at all. A file that reappears at the old path can only have been written by hand, so it is merged in with a warning naming its keys instead of being discarded.
Options drained out of the pre-0.38.0 `DOCKER_OPTIONS_<PHASE>` files were copied verbatim rather than re-serialized the way `docker-options:add` stores them, so `docker-options:remove` compared the canonical string it builds against a stored value that could never match it and exited successfully without removing anything. Removal now matches stored options by shell word, and stored options are rewritten into the canonical form once on upgrade, which additionally splits an entry that carried several flags on a single line into one entry per flag so a single flag can be removed and so the readers that match on a flag prefix see one value per entry. The leftover `.migrated` sentinel drain is restored to running ahead of the global short-circuit that had made it unreachable, and the plugin's Go tests are added to the test target that had never run them.
Sysctls the kernel does not namespace, such as `vm.max_map_count`, cannot be set from a pod spec and previously had no answer beyond editing `/etc/sysctl.d` on each host by hand. `scheduler-k3s:node-sysctls:set` now applies them through a privileged daemonset, which reaches nodes joined later and reapplies after a reboot. Sysctls may be scoped to a node profile, with a profile scope inheriting the global values and overriding them on conflict so that every node is covered by exactly one daemonset. Clearing a sysctl stops dokku managing it but does not restore the previous value, which persists until the node reboots.