Commit Graph
1228 Commits
Author SHA1 Message Date
Jose Diaz-Gonzalez dfd6e434a4 fix: bump openresty-docker-proxy to 0.12.1 for path-based routes
0.12.0 had a docker-gen template cache regression that suppressed
sidecar reloads when only the `strip-prefix` label was added or
removed on an existing route key. 0.12.1 fixes it, so the two
openresty bats tests previously marked `skip` against
dokku/openresty-docker-proxy#141 can run again.
2026-11-04 14:56:38 -05:00
Jose Diaz-Gonzalez 123fdc700c fix: drop k3s-traefik priority, wrap caddy web reverse_proxy in handle
The k3s-traefik IngressRoute was emitting priority: len(path) on the route rule but no priority on the bare-Host catch-all. Traefik defaults missing priority to rule length, so the catch-all rule got ~32 while the path rule got 7, and the catch-all always won. Same exact priority bug we fixed for docker-traefik. Dropping the priority line lets traefik use its rule-length default which gives longest-prefix-wins naturally because Host && PathPrefix is always longer than bare Host. The caddy web container's emission was a bare reverse_proxy directive that runs at caddy default order ~190, well before the handle directive at ~210 where my route emission lives. The bare reverse_proxy caught every request including paths intended for the route. When at least one route exists for the app, the web container now wraps its reverse_proxy in a catch-all handle block via caddy.9_handle.0_reverse_proxy=...; both the route's handle (at 0_) and web's catch-all (at 9_) live at the same directive order ~210 and caddy's mutually-exclusive handle logic picks the route handle for matched paths and the catch-all for everything else. The "9_" prefix orders web's catch-all after the route handles in the docker-proxy generated Caddyfile. Apps without routes keep the original bare reverse_proxy behaviour.
2026-11-04 14:56:38 -05:00
Jose Diaz-Gonzalez 3449c11a59 fix: diagnose caddy Caddyfile, skip openresty cache regression, rebuild k3s
Caddy T1 now dumps the generated Caddyfile and the adapted JSON from caddy-caddy-1 so the next CI run reveals how caddy-docker-proxy merged the web container's bare reverse_proxy with the api container's handle directive. Without the merged Caddyfile in front of us, we're guessing at caddy's directive evaluation order; with it we can pick the right structural fix. Openresty T2 and T3 are skipped with a reference to dokku/openresty-docker-proxy#141 - the 0.12.0 sidecar's docker-gen template cache observes the label set change but compares the rendered config against its previous output and concludes nothing changed when only the strip-prefix label is added, so proxy_pass never gets the trailing slash that strip-prefix is supposed to add. T1 keeps passing as the smoke that the path-prefix label vocabulary works at all. The k3s tests now run dokku ps:rebuild after every proxy:route:set/remove because scheduler-k3s has no proxy-build-config trigger handler today, so route mutations do not re-render the chart on their own; ps:rebuild forces a redeploy which renders the chart with current routes. The runtime gap (adding a proxy-build-config handler that calls into the chart-render path) is tracked as a follow-up.
2026-11-04 14:56:38 -05:00
Jose Diaz-Gonzalez 53066ec4ac fix: traefik priority loss, caddy directive order, openresty cache, k3s diagnostic
Traefik was setting an explicit priority equal to the path length, which is smaller than traefik's default priority (rule length) on the web container. Web's bare Host(...) rule has implicit priority ~40 while my explicit 7 lost the match, so /api/v0/Procfile always went to web. Dropping the explicit priority lets traefik use rule length, which already gives longest-prefix-wins because the route rule includes both Host and PathPrefix and so is always longer than web's bare Host(...). Caddy's route block was being preempted by web's bare reverse_proxy due to caddy's default directive order (reverse_proxy ~190, route ~210). Switching to handle, which terminates matched requests and isn't bypassed by reverse_proxy, makes /api/v0/* land on api instead of web's catch-all. Openresty T2 polls longer (60 retries) and dumps sidecar logs on failure so the upstream label-update cache delay can be reported. The k3s files dump kubectl get service/ingress/endpoints in T1 so the next CI run reveals whether the api Service has ports and whether the Ingress backend reference is correct.
2026-11-04 14:56:38 -05:00
Jose Diaz-Gonzalez 5dce07ca7e fix: drop shell-quoted body grep and add docker label diagnostic
The retry loop in T3 fed the response body through `bash -c "grep <<<\"$body\""`. When the body contains JSON env vars with embedded double quotes (PS1, paths with `$`, etc.), the inner `"$body"` terminates the outer `bash -c "..."` quoting prematurely and bash tries to execute the JSON content as commands. grep itself returns 0 but the subsequent error messages get appended to bats's `$output`, breaking `assert_output "0"`. Each T3 now curls into a file and greps the file directly, so the body never crosses a shell-quote boundary. Each label-based backend file additionally echoes `docker inspect <app>.api.1 --format '{{json .Config.Labels}}'` after `ps:rebuild`, surfacing the actual labels emitted onto the api container so the CI log shows whether the docker-args-process-deploy emission matches what the proxy daemon expects. A 5s sleep after rebuild gives the proxy daemon time to read the new labels before the HTTP retry loop starts.
2026-11-04 14:56:38 -05:00
Jose Diaz-Gonzalez 6dbd9b4d82 fix: make T3 route-removal assertions robust to reload race
T3 in every per-backend file polled the post-removal HTTP response using assert_http_localhost_response_contains, which retries on status code only. After a route is removed, both api and web return 200 for /api/v0/Procfile so the polling exits immediately; the body check then races with the proxy daemon picking up the new state and observes api's response (Procfile content) instead of web's catch-all. The proxy:route:remove output was also unechoed, hiding whether BuildConfig actually fired or was no-op suppressed. Each T3 now echoes the remove output and status, asserts via proxy:route:report that storage was cleared, then polls the body in a hand-rolled retry loop until the api marker is no longer present (timeout 30s). The k3s T3 files similarly retry the kubectl probe so the chart re-render has time to settle.
2026-11-04 14:56:38 -05:00
Jose Diaz-Gonzalez 40e13b5f13 fix: address PR review feedback on per-backend route tests
T1 of every per-backend test file asserted 200 with the api Procfile body for a request that lacks --strip-prefix, but without the strip the upstream sees /api/v0/Procfile and python's http.server returns 404; the retry loop polled on status only and accepted web's catch-all 200 first. The assertion now expects 404, which is the right signal that the route reaches the api process. T3 was rewritten to follow T2's two-step pattern (404 without strip, then 200 with strip) so the route is proven applied before checking content. T4 of proxy-route-nginx.bats relaxes the WebSocket header assertions to allow more than one occurrence because the headers correctly appear in both the route location block and the existing location / block. Backticks in fn-traefik-host-rule and fn-traefik-route-labels are now triple-escaped so they survive the docker-args eval path that turns the labels into a docker run invocation, matching the existing convention in docker-args-process-deploy. The label-based backend test files also echo the ps:rebuild output now so future failures of that step surface in the CI logs instead of silently exiting non-zero.
2026-11-04 14:56:38 -05:00
Jose Diaz-Gonzalez 4a550fe4b9 feat: render proxy routes under openresty and add per-backend tests
Bumps the openresty sidecar to 0.12.0, which introduces the openresty.route.<n>.* label family for path-based routing. The openresty-vhosts plugin now emits route labels for non-web containers targeted by a proxy:route:set, mirroring the traefik and caddy patterns. Adds integration tests for every rendering backend (proxy-route-{nginx,traefik,caddy,openresty}.bats covering basic routing, --strip-prefix, and removal) plus per-ingress-class scheduler-k3s tests (proxy-route-k3s-{nginx,traefik}.bats covering the same three cases against a real cluster). The nginx file additionally asserts the rendered nginx config carries the WebSocket upgrade headers in each route's location block.
2026-11-04 14:56:38 -05:00
Jose Diaz-Gonzalez a4d40becd8 refactor: drop proxy-supports-routes capability trigger
The trigger was added to refuse proxy:route:set under openresty and haproxy, but the hard refusal created an ordering dependency for declarative tools and blocked users from staging routes ahead of a backend switch. Routes are now stored regardless of the active backend and silently ignored by backends that do not render them; the docs call out the inert behavior under openresty and haproxy.
2026-11-04 14:56:38 -05:00
Jose Diaz-Gonzalez ffd5248cc3 feat: route paths to non-web processes via proxy:route
Adds proxy:route:set/remove/clear/report so users can declare path-prefix routes to non-web Procfile processes without hand-writing nginx.conf.sigil. Routes are stored on the proxy plugin as a PropertyMap of path to process:port:strip, with set semantics so each invocation produces a fully-determined state and re-application is idempotent. A proxy-supports-routes capability trigger gates the command per backend - nginx-vhosts, traefik-vhosts, caddy-vhosts, and scheduler-k3s render routes today; openresty-vhosts and haproxy-vhosts refuse until their sidecar images learn a path-prefix vocabulary (tracked in dokku/openresty-docker-proxy#137 and the haproxy sidecar). Closes #8725.
2026-11-04 14:56:38 -05:00
Jose Diaz-Gonzalez 59594937f9 feat: split stack property into per-builder properties
The `stack` property lived on the `buildpacks` plugin but was shared by both the herokuish and pack builders, so a value valid for one builder broke the other. Each builder now owns its own `stack` property managed via `builder-herokuish:set` and `builder-pack:set`, while `buildpacks:set-property stack` is deprecated and forwards the value to the builder detected from it; existing values migrate automatically on upgrade and changing a builder's stack clears that builder's cache.
2026-11-02 16:48:11 -05:00
Jose Diaz-Gonzalez 1086de5aae test: read caddy config without exec in the container 2026-11-02 00:33:35 -05:00
Jose Diaz-Gonzalez fba0681e5b fix: avoid glob expansion of app domains
A catch-all `*` domain or a wildcard ssl hostname could be expanded into file names from the current directory when generating app urls, healthcheck `Host` headers, or the nginx ssl server names.
2026-11-02 00:33:35 -05:00
Jose Diaz-Gonzalez 06f80b1a93 fix: map caddy healthcheck headers and apply tls-internal
Caddy healthchecks now honor the `attempts` and `httpHeaders` properties of the `app.json` readiness healthcheck, matching openresty. The `tls-internal` property was also never applied to app containers, and now takes effect for apps served over https.
2026-11-02 00:33:35 -05:00
Jose Diaz-Gonzalez 83c9c4d72e fix: add proxy health checks from app.json readiness
The caddy, haproxy, openresty, and traefik docs stated that requests are only routed to containers passing healthchecks, but these proxies route to a new container as soon as it starts. The first `web` readiness healthcheck in `app.json` is now also turned into health checks and retries for caddy and openresty, as it already was for traefik, so requests are routed away from containers that are not yet ready, and the docs now describe how each proxy actually behaves.
2026-11-02 00:33:35 -05:00
Jose Diaz-Gonzalez f29fdae409 fix: keep previous image restartable after failed deploy
When a deploy failed, the old containers kept serving while `dokku/<app>:latest` still pointed at the failed image, so `ps:restart`, `ps:start`, and `ps:restore` would redeploy the broken release or fail with `App image not found` once that image was retired. The running image is now kept available during a build and deploy, the `latest` tag is restored to it when the build, release, or deploy fails, and `ps:retire` no longer removes the image an app's `latest` tag points at. Failed containers are also now registered for retirement under the correct app so they and their images are cleaned up.
2026-10-30 18:00:05 -04:00
Jose Diaz-Gonzalez a3c31275a2 Merge pull request #9227 from dokku/datastore-plugin
Ship every datastore as a core datastore plugin
2026-10-30 17:56:11 -04:00
Jose Diaz-Gonzalez 8f11b7900a Merge pull request #9225 from dokku/9224-add-a-way-to-rebuild-an-app-without-build-cache-on-demand
Add --no-cache flag to ps:rebuild
2026-10-29 21:05:57 -04:00
Jose Diaz-Gonzalez 68e5757590 feat: ship every datastore as a core datastore plugin
Adds a `datastore` core plugin that serves every official datastore under one command, with the datastore type as the first argument, as in `dokku datastore:create postgres lollipop`. It is built on the dokku-datastore sdk, and leaves a datastore to its official plugin while that plugin is enabled.
2026-10-29 20:32:20 -04:00
Jose Diaz-Gonzalez d98761fe9a test: match pack cache-clear output 2026-10-29 18:53:08 -04:00
Jose Diaz-Gonzalez 32d8360ad3 test: assert uncached build steps rerun 2026-10-29 15:47:06 -04:00
Jose Diaz-Gonzalez f3b21babcc feat: add --no-cache flag to ps:rebuild
The `ps:rebuild` command now accepts `--no-cache` to perform a single uncached build regardless of builder, without persisting any setting for later deploys. The railpack builder now also honors `--no-cache` when set via `docker-options`, and build records are now correctly attributed when flags precede the app name.
2026-10-29 14:25:51 -04:00
Jose Diaz-Gonzalez fe0f024208 fix: attach every network in attach-post-create
Setting multiple networks for the `attach-post-create` network property only connected new app containers to the first network in the list. All specified networks are now attached, matching the existing `attach-post-deploy` behavior.
2026-10-29 12:25:14 -04:00
Jose Diaz-Gonzalez 00c3e48cf4 feat: add a whole-set form and clear for chart overrides
The `scheduler-k3s:charts:set` command only changed one chart override at a time, so matching a chart's stored overrides to a declared set meant reading the report, computing the difference in both directions and issuing one command per added, changed and removed property, with a failure partway through leaving a mixture of the two sets. A `--replace` flag on `scheduler-k3s:charts:set` now takes a chart name followed by `property=value` pairs and writes the declared map for that chart in a single call. Every pair is parsed before anything is written, so a rejected pair leaves the stored overrides untouched, and a pair ending in `=` stores an empty value, which the per-property form cannot express. An empty pair list is rejected rather than treated as a request to remove everything, and the new `scheduler-k3s:charts:clear` command handles that case by removing every override for a chart.
2026-10-24 19:34:13 -04:00
Jose Diaz-Gonzalez c4b164a270 fix: release deploy locks on exit and honor apps:lock
A deploy that exited before releasing its lock, such as after a failed build, left `.deploy.lock` behind, so `apps:locked` reported a lock nothing held and the build record stayed running. The lock is now released and the record finalized whenever the deploy exits. `apps:lock` never actually blocked deploys, as it only created the file deploys lock. It now creates its own lock that git pushes, `ps:rebuild`, `git:sync`, and the `git:from-*` commands refuse to deploy past, and `apps:locked` only reports a lock when the app was locked via `apps:lock` or a deploy is in progress. Locks created by `apps:lock` in older versions are migrated on install.
2026-10-24 13:58:47 -04:00
Jose Diaz-Gonzalez 57ae817dab Merge pull request #9128 from sean0407/codex/fix-cleanup-force-9127
Forward forced cleanup flag (#9127)
2026-10-24 12:03:17 -04:00
Jose Diaz-Gonzalez 58d8a52234 fix: redirection operator in docker-cleanup test 2026-10-24 12:03:06 -04:00
Jose Diaz-Gonzalez 3bd74e4eb5 test: pause the retire timer during source image test
The periodic `dokku-retire` timer could run `ps:retire` between the test's final deploy and its own `ps:retire` call, processing the retire list first and leaving the test with no output to assert on.
2026-10-23 20:21:28 -04:00
Jose Diaz-Gonzalez 2dd77d61e4 Merge pull request #9180 from dokku/9168-scheduler-k3s-deployment-retry-fails-cleaning-up-a-non-empty-kustomization-directory
Clean up stale extracted files on deploy
2026-10-23 19:23:44 -04:00
Jose Diaz-Gonzalez ad931ea817 Merge pull request #9176 from byjg/haproxy-easyhaproxy-6.5.1
Haproxy letsencrypt integration and upgrade EasyHAProxy to 6.5.1
2026-10-23 16:02:34 -04:00
Jose Diaz-Gonzalez 0690a9b655 fix: clean up stale extracted files on deploy
A failed deploy could leave a non-empty extraction directory behind, such as the scheduler-k3s `kustomization` directory, which caused every following deploy to fail. Leftovers from previous deploys are now fully removed, removing a directory such as `config/kustomize` from the repository now removes the stored copy, and openresty includes now persist across restarts.
2026-10-23 14:37:49 -04:00
Joao Gilberto Magalhaes 4887cf458e fix: haproxy letsencrypt integration and upgrade EasyHAProxy to 6.5.1
The letsencrypt integration set EASYHAPROXY_LETSENCRYPT_EMAIL (with a stray
quote) and a letsencrypt label, which EasyHAProxy has not read since before
4.6.0. With an email set, http redirected to https and nothing answered on 443.

- Pass EASYHAPROXY_CERTBOT_EMAIL and EASYHAPROXY_CERTBOT_SERVER.
- Label the app with certbot=true on its http definition: EasyHAProxy requests
  the certificate and serves https itself, so the separate 443 definition goes.
  https labels are only added when a letsencrypt email is set, as documented.
- Keep the certificates in /var/lib/dokku/data/haproxy.
- Turn the stats page off with HAPROXY_STATS_ENABLE=false.
- Upgrade EasyHAProxy from 6.1.1 to 6.5.1.
2026-10-23 09:29:01 -04:00
Jose Diaz-Gonzalez 97dd080441 fix: only clear the requested resource type
Running `resource:limit-clear` or `resource:reserve-clear` without `--process-type` deleted every resource property for the app, so clearing limits also removed reservations and vice versa. Both commands now clear only their own resource type across all process types.
2026-10-22 18:58:23 -04:00
Jose Diaz-Gonzalez 898924e541 Merge pull request #9152 from Schlepptop/make-nginx-proxy-http-version-configurable
Make nginx-proxy-http-version configurable per app or globally
2026-10-22 16:51:09 -04:00
Schlepptop 5b72061845 Merge branch 'master' into make-nginx-proxy-http-version-configurable 2026-10-22 11:37:55 +02:00
Jose Diaz-Gonzalez 9df45f6a6e fix: serialize concurrent property updates
Concurrent commands for the same app could each read a setting, change it, and write it back, so all of them exited successfully while silently dropping changes made by the others. Updates to app settings are now serialized so every change is kept, and readers never see a partially written value.
2026-10-21 21:21:57 -04:00
Jose Diaz-Gonzalez c8e28c2445 fix: accept all docker memory units in resource values
Memory values with uppercase or other docker-supported units such as `1G`, `512M` or `1GiB` are now passed to docker unchanged instead of having an extra `m` appended, and the k3s scheduler interprets memory values with the same binary units so a value means the same thing on both schedulers. Invalid `--memory` and `--memory-swap` values are now rejected by `resource:limit` and `resource:reserve`. Closes #9153.
2026-10-21 14:07:00 -04:00
Schlepptop 4badd065cc Make nginx-proxy-http-version configurable per app or globally 2026-10-19 00:44:59 +02:00
Jose Diaz-Gonzalez f768eb6d07 test: cover DOKKU_LIB_HOST_ROOT storage mounts in bats
Add a Dokku-in-Docker bats suite that writes under DOKKU_LIB_HOST_ROOT and
checks storage:exec, set those env vars in the docker CI container, and add
unit bats for the bare-metal (unset) and rewritten docker-args paths.
2026-10-16 11:52:01 -04:00
Jose Diaz-Gonzalez 94a82bd6ee Merge pull request #9130 from dokku/fix-ssh-arg-splitting
Preserve arguments such as --version and -n when running commands over SSH
2026-10-14 15:13:13 -04:00
Jose Diaz-Gonzalez 0e581f1a7f tests: move the shared source-image app off via git:from-archive
deploy_app cannot push onto an app that already has git history from git:from-image. git:from-archive deploys and clears source-image the same way git_4 already does.
2026-10-14 11:54:29 -04:00
Jose Diaz-Gonzalez e71dffb1a7 fix: do not retire an image that another app uses as its source image
Image retirement removes images by id, so a queued image could be deleted even though another app had since started using it as its git:from-image source image. Skip such images at retire time and drop them from the retire list.

The git_4 test images are also given a per-app label so they no longer share a cached image id, which let a retire entry left by one test delete the next test's source image.
2026-10-14 10:00:51 -04:00
Jose Diaz-Gonzalez a7539a54a0 tests: pass two option words to /bin/echo so GNU echo prints them literally 2026-10-12 13:13:31 -04:00
Jose Diaz-Gonzalez 316dafcb9f tests: add issue #9120 reproduction steps for run over ssh 2026-10-12 13:08:59 -04:00
Jose Diaz-Gonzalez c3a4b79e6e fix: preserve arguments such as --version and -n when running commands over ssh
xargs without a command runs /bin/echo per word, which interprets --version, --help, -n, -e and -E as its own options. Use printf with a NUL delimiter instead.

Closes #9120
2026-10-12 13:08:59 -04:00
Jose Diaz-Gonzalez 11c4e908d5 fix: hold the deploy lock across git-from-directory and harden the advisory lock
git-from-directory updated the deploy branch before taking the deploy
lock and rolled it back after releasing it. The lock is now held for the
whole update, build, and rollback, and the rollback only moves the branch
if it still points at the commit that was created.

A failed lock attempt and apps:lock both truncated the lock file,
dropping the build id of the in-progress deploy that builds:cancel and
builds:output read. Releasing the lock unlocked before removing the
file, so a waiter could end up holding a lock on a deleted file while
another deploy locked a new one. The file is now removed before
unlocking, and acquiring retries when the locked file is no longer the
one at the lock path.
2026-10-11 16:21:13 -04:00
Jose Diaz-Gonzalez 7955e8ee06 fix: keep the git source-image property in sync with failed deploys
The source-image property was written before the deploy lock was taken
and was not restored when a deploy failed, so after a rejected
git:from-image or git:load-image it named an image that was not
deployed, and later builds extracted app.json, Procfile, and CHECKS
from it. Deploys from other sources also built while the old image was
still set.

Entry points now pass the desired value via DOKKU_GIT_SOURCE_IMAGE and
git_receive_app applies it while holding the deploy lock, restoring the
previous value on failure. git:sync clears the property under the lock
when it fetches new code.

Closes #9117
2026-10-11 05:08:32 -04:00
Sean 28505a7ed4 fix: forward forced cleanup flag (#9127) 2026-10-11 15:50:54 +08:00
Jose Diaz-Gonzalez c776fbc7a4 test: expect the global MAILTO reset in cron-3 2026-09-30 21:40:31 -04:00
Jose Diaz-Gonzalez b158f482ef fix: reset MAILTO after a cron entry that sets its own
A task with its own `MAILTO` is now written in place, followed by a line resetting `MAILTO` to the global value, or to the `dokku` user when none is set, rather than grouped at the end of the crontab, so no other task is mailed to its recipient whatever order the tasks are written in.
2026-09-30 17:47:27 -04:00