feat: make coredns-docker upstream resolution opt-in

The bundled `nginx.conf.sigil` keeps rendering IP-based upstreams unless `dns-resolver` is set, for example with `dokku nginx:set --global dns-resolver 127.0.0.1:1053`. Unsetting the property reverts to IP-based upstreams, replacing the previous `off` sentinel.
This commit is contained in:
Jose Diaz-Gonzalez committed 2026-11-06 20:51:33 -05:00
1 parent b5315de49b
commit bf2795640c
10 files changed
+107 -73

No files matched your search

+11 -11
View File
@@ -8,22 +8,22 @@
- The internal `buildpack-stack-name` plugin trigger has been removed. Each builder now reads its own `stack` property directly during the build.
- Changing a builder's `stack` now clears that builder's build cache so the next deploy starts fresh. For herokuish this removes the `cache-<app>` Docker volume; for pack the next `pack build` runs with `--clear-cache`.
- The `dokku` Debian package now declares `coredns-docker (>= 0.6.0)` as a hard `Depends:`. `apt install dokku` and `apt upgrade dokku` pull it in automatically. The deb ships a default `/etc/coredns/Corefile` that binds `127.0.0.1:1053`, serves the `docker.` zone, and synthesizes `<app>.<process-type>.docker` and `<app>.docker` `A` records from the `com.dokku.app-name` and `com.dokku.process-type` labels Dokku already stamps on every deployed container. The Docker image of Dokku ships coredns-docker preinstalled and runs it under runit on the same port.
- The bundled `nginx.conf.sigil` now uses an nginx `resolver` directive and a variable-based `proxy_pass` against `<app>.<process-type>.docker` instead of pinning the upstream to a literal IP. This is what addresses the long-standing post-reboot and post-`apt-upgrade-docker` stale-vhost bug ([#8313](https://github.com/dokku/dokku/issues/8313)) - the rendered upstream re-resolves at request time via coredns-docker, so a host reboot or single-container restart no longer leaves nginx routing to a stale IP. See the [nginx documentation](/docs/networking/proxies/nginx.md#upstream-resolution-via-coredns-docker) for details.
- Three new global nginx properties control the resolver path:
- `dns-resolver` (default: `127.0.0.1:1053`) - the address nginx talks to.
- `dns-resolver-timeout` (default: `5s`) - how long nginx waits for a DNS response before giving up and returning 502. The short default keeps clients from waiting on nginx's built-in 30s timeout when coredns-docker is unreachable.
- `dns-zone` (default: `docker`) - the suffix appended to `<app>.<process-type>` when constructing the upstream hostname.
All three can be set via `dokku nginx:set --global <property> <value>`.
- The literal string `off` is the sentinel for disabling the resolver path. **Setting `dns-resolver` to the empty string does not disable it** - that deletes the property file and the new default (`127.0.0.1:1053`) re-engages. To revert to the legacy IP-based upstream block on hosts where coredns-docker cannot run (source/git installs, locally held coredns-docker package, postinst failures):
- The bundled `nginx.conf.sigil` can now use an nginx `resolver` directive and a variable-based `proxy_pass` against `<app>.<process-type>.docker` instead of pinning the upstream to a literal IP. This addresses the long-standing post-reboot and post-`apt-upgrade-docker` stale-vhost bug ([#8313](https://github.com/dokku/dokku/issues/8313)) - the rendered upstream re-resolves at request time via coredns-docker, so a host reboot or single-container restart no longer leaves nginx routing to a stale IP. This behavior is opt-in, and upstreams remain IP-based unless `dns-resolver` is set:
```shell
dokku nginx:set --global dns-resolver off
dokku nginx:set --global dns-resolver 127.0.0.1:1053
dokku proxy:build-config --all
```
- Vendored `nginx.conf.sigil` templates are unaffected by the change. The existing `DOKKU_APP_<PROC>_LISTENERS` variable continues to be populated with the IP-based listener list exactly as before, so templates that only reference it continue to render IP-based upstreams regardless of the new property defaults. To opt a vendored template into the resolver path, switch its `proxy_pass` line to use the new `DOKKU_APP_<PROC>_LISTENER_HOST`, `NGINX_DNS_RESOLVER`, `NGINX_DNS_RESOLVER_TIMEOUT`, and `NGINX_DNS_ZONE` sigil variables - see the [nginx.conf.sigil reference](/docs/appendices/file-formats/nginx-conf-sigil.md#upstream-rendering-ip-based-vs-dns-based) for the exact snippet.
See the [nginx documentation](/docs/networking/proxies/nginx.md#upstream-resolution-via-coredns-docker) for details.
- Three new nginx properties control the resolver path:
- `dns-resolver` (default: none) - the address nginx talks to. Resolver-based upstreams are only rendered when this is set.
- `dns-resolver-timeout` (default: `5s`) - how long nginx waits for a DNS response before giving up and returning 502. The short default keeps clients from waiting on nginx's built-in 30s timeout when coredns-docker is unreachable.
- `dns-zone` (default: `docker`) - the suffix appended to `<app>.<process-type>` when constructing the upstream hostname.
All three can be set per app or globally via `dokku nginx:set`.
- Vendored `nginx.conf.sigil` templates are unaffected by the change. The existing `DOKKU_APP_<PROC>_LISTENERS` variable continues to be populated with the IP-based listener list exactly as before, so templates that only reference it continue to render IP-based upstreams regardless of the new properties. To opt a vendored template into the resolver path, switch its `proxy_pass` line to use the new `DOKKU_APP_<PROC>_LISTENER_HOST`, `NGINX_DNS_RESOLVER`, `NGINX_DNS_RESOLVER_TIMEOUT`, and `NGINX_DNS_ZONE` sigil variables - see the [nginx.conf.sigil reference](/docs/appendices/file-formats/nginx-conf-sigil.md#upstream-rendering-ip-based-vs-dns-based) for the exact snippet.
### Stack property migrated to builder plugins
@@ -101,7 +101,7 @@ The new variables (`DOKKU_APP_<PROC>_LISTENER_HOST` + `NGINX_DNS_RESOLVER` + `NG
The bundled template emits this snippet:
```nginx
{{ if ne $.NGINX_DNS_RESOLVER "off" }}
{{ if $.NGINX_DNS_RESOLVER }}
resolver {{ $.NGINX_DNS_RESOLVER }} valid=10s ipv6=off;
resolver_timeout {{ $.NGINX_DNS_RESOLVER_TIMEOUT }};
set $dokku_upstream "{{ $.DOKKU_APP_WEB_LISTENER_HOST }}:{{ $upstream_port }}";
@@ -113,8 +113,8 @@ proxy_pass http://{{ $.APP }}-{{ $upstream_port }};
The `$dokku_upstream` variable name is intentionally namespaced (rather than `$upstream`) so vendored templates that already define `$upstream` will not collide if they adopt the resolver branch.
`NGINX_DNS_RESOLVER` defaults to `127.0.0.1:1053`, `NGINX_DNS_RESOLVER_TIMEOUT` defaults to `5s`, and `NGINX_DNS_ZONE` defaults to `docker`; all three are configurable via `dokku nginx:set --global dns-resolver ...`, `dokku nginx:set --global dns-resolver-timeout ...`, and `dokku nginx:set --global dns-zone ...`. The literal string `off` is the sentinel for disabling the resolver branch in the bundled template - empty strings are treated as "use the default" because the property is stored on disk and an empty value deletes the property file. The short default `dns-resolver-timeout` keeps clients from waiting on nginx's built-in 30s timeout when coredns-docker is unreachable; nginx returns 502 after the configured interval instead.
`NGINX_DNS_RESOLVER` is empty by default, `NGINX_DNS_RESOLVER_TIMEOUT` defaults to `5s`, and `NGINX_DNS_ZONE` defaults to `docker`; all three are configurable via `dokku nginx:set --global dns-resolver ...`, `dokku nginx:set --global dns-resolver-timeout ...`, and `dokku nginx:set --global dns-zone ...`. The bundled template only renders the resolver branch when `NGINX_DNS_RESOLVER` is set, for example to `127.0.0.1:1053`. Unsetting `dns-resolver` switches the bundled template back to the IP-based upstream. The short default `dns-resolver-timeout` keeps clients from waiting on nginx's built-in 30s timeout when coredns-docker is unreachable; nginx returns 502 after the configured interval instead.
### Choosing between the two
The DNS-based path is the default for the bundled template. Vendored templates that have not been updated continue to use `DOKKU_APP_<PROC>_LISTENERS` and are unaffected by the global default. The DNS-based variables are populated regardless of whether `dns-resolver` is `off` or not, so a vendored template can adopt the resolver path on its own schedule by switching its rendering snippet, even if other apps on the host are still using IP-based upstreams.
The IP-based path is the default for the bundled template, and the DNS-based path is used once `dns-resolver` is set. Vendored templates that have not been updated continue to use `DOKKU_APP_<PROC>_LISTENERS` and are unaffected by the `dns-resolver` setting. The `DOKKU_APP_<PROC>_LISTENER_HOST` and `NGINX_DNS_ZONE` variables are populated regardless of whether `dns-resolver` is set, so a vendored template can adopt the resolver path on its own schedule by switching its rendering snippet, even if other apps on the host are still using IP-based upstreams.
+1 -1
View File
@@ -88,7 +88,7 @@ Dokku is run in the following configuration:
- Image build cache is set to the data dir + `/home/dokku`.
- The docker socket is mounted into container.
- `DOKKU_HOST_ROOT` and `DOKKU_LIB_HOST_ROOT` tell Dokku how the host Docker daemon sees those same trees when creating bind mounts (app repos and plugin data such as named storage).
- `coredns-docker` (a CoreDNS plugin that serves DNS records for sibling Docker containers) is installed transitively via the dokku deb's `Depends:`, runs under runit on `127.0.0.1:1053` inside the container, and is what the `nginx-vhosts` `dns-resolver` default points at. This is what makes the `<app>.<process-type>.docker` upstream resolution work out of the box - see the [nginx proxy doc](/docs/networking/proxies/nginx.md#upstream-resolution-via-coredns-docker) for the operator-facing behavior.
- `coredns-docker` (a CoreDNS plugin that serves DNS records for sibling Docker containers) is installed transitively via the dokku deb's `Depends:`, runs under runit on `127.0.0.1:1053` inside the container, and is what the `nginx-vhosts` `dns-resolver` property can point at to enable `<app>.<process-type>.docker` upstream resolution - see the [nginx proxy doc](/docs/networking/proxies/nginx.md#upstream-resolution-via-coredns-docker) for the operator-facing behavior.
Application repositories, plugin config, as well as plugin data are persisted to disk within the specified host directory for `/var/lib/dokku`. `DOKKU_LIB_HOST_ROOT` must point at that host path's `var/lib/dokku` subtree so Docker `-v` mounts resolve under the data volume rather than under the host's own `/var/lib/dokku`.
+20 -10
View File
@@ -33,27 +33,34 @@ Nginx will proxy the requests in a [round-robin balancing fashion](http://nginx.
> [!IMPORTANT]
> New as of 0.39.0
Dokku resolves nginx upstreams through [coredns-docker](https://github.com/dokku/coredns-docker), a CoreDNS plugin that watches the Docker daemon and serves fresh `A` records keyed off container labels. The nginx `resolver` directive points at coredns-docker's local socket (`127.0.0.1:1053`) and the rendered upstream is a variable-based `proxy_pass`, which forces nginx to re-resolve at request time honoring DNS TTL. This is how nginx avoids serving the wrong vhost after a host reboot, after `apt upgrade docker`, or after a single container restarts mid-runtime: the on-disk nginx config no longer pins a stale IP.
Dokku can resolve nginx upstreams through [coredns-docker](https://github.com/dokku/coredns-docker), a CoreDNS plugin that watches the Docker daemon and serves fresh `A` records keyed off container labels. When enabled, the nginx `resolver` directive points at coredns-docker's local socket (`127.0.0.1:1053`) and the rendered upstream is a variable-based `proxy_pass`, which forces nginx to re-resolve at request time honoring DNS TTL. This is how nginx avoids serving the wrong vhost after a host reboot, after `apt upgrade docker`, or after a single container restarts mid-runtime: the on-disk nginx config no longer pins a stale IP. This behavior is opt-in, and nginx uses IP-based `upstream` blocks by default.
`coredns-docker` (>= `0.6.0`) is installed automatically as a transitive `Depends:` of the `dokku` Debian package, so `apt install dokku` and `apt upgrade dokku` pull it in without operator action. The shipped `/etc/coredns/Corefile` already binds `127.0.0.1:1053`, serves the `docker.` zone, and synthesizes `<app>.<process-type>.docker` and `<app>.docker` `A` records from Dokku's existing `com.dokku.app-name` and `com.dokku.process-type` container labels. Multiple containers for the same process type collapse onto one multi-A record set, so scaled apps load-balance across instances via DNS without any per-container configuration. The bundled Docker image of Dokku ships coredns-docker preinstalled and runs it under runit on the same port.
No `systemd-resolved` or `/etc/resolv.conf` change is required - nginx talks to `coredns-docker` directly via the `resolver` directive in each app's `nginx.conf`.
The behavior is controlled by three global nginx properties:
- `dns-resolver` (default: `127.0.0.1:1053`) - the resolver address nginx talks to.
- `dns-resolver-timeout` (default: `5s`) - how long nginx waits for a DNS response before giving up and returning 502. The default is short so requests fail fast when coredns-docker is unreachable rather than hanging on the nginx built-in default (30s).
- `dns-zone` (default: `docker`) - the suffix appended to `<app>.<process-type>` when constructing the upstream hostname.
To revert to the legacy IP-based upstream block (for instance on a host where `coredns-docker` cannot be installed):
To enable resolver-based upstreams:
```shell
dokku nginx:set --global dns-resolver off
dokku nginx:set --global dns-resolver 127.0.0.1:1053
dokku proxy:build-config --all
```
To revert to the IP-based upstream block, unset the property and rebuild the config:
```shell
dokku nginx:set --global dns-resolver
dokku proxy:build-config --all
```
The behavior is controlled by three nginx properties, each settable per app or globally:
- `dns-resolver` (default: none) - the resolver address nginx talks to. Resolver-based upstreams are only rendered when this is set.
- `dns-resolver-timeout` (default: `5s`) - how long nginx waits for a DNS response before giving up and returning 502. The default is short so requests fail fast when coredns-docker is unreachable rather than hanging on the nginx built-in default (30s).
- `dns-zone` (default: `docker`) - the suffix appended to `<app>.<process-type>` when constructing the upstream hostname.
> [!NOTE]
> The sentinel for disabling the feature is the literal string `off`. Setting `dns-resolver` to the empty string (`""`) does **not** disable it - that deletes the property and the default (`127.0.0.1:1053`) re-engages. Use `off` to disable, the default to re-enable.
> When `dns-resolver` is set, no `upstream` block is rendered for the `web` process, so `proxy-keepalive` has no effect on it. Upstreams for [proxy routes](/docs/networking/proxy-management.md#path-based-routing-to-non-web-processes) to other process types remain IP-based.
See [the `nginx.conf.sigil` reference](/docs/appendices/file-formats/nginx-conf-sigil.md) for the sigil variables that drive the resolver-based render, and how vendored templates can adopt or skip it.
@@ -672,6 +679,9 @@ All nginx-vhosts properties are settable at both the app and global scope.
| `client-header-timeout` | app + global | `60s` | `--nginx-client-header-timeout`, `--nginx-global-client-header-timeout`, `--nginx-computed-client-header-timeout` | Time allowed to read the request header from the client |
| `client-max-body-size` | app + global | `1m` | `--nginx-client-max-body-size`, `--nginx-global-client-max-body-size`, `--nginx-computed-client-max-body-size` | Maximum allowed request body size |
| `disable-custom-config` | app + global | `false` | `--nginx-disable-custom-config`, `--nginx-global-disable-custom-config`, `--nginx-computed-disable-custom-config` | When `true`, ignores app-supplied `nginx.conf.d/*.conf` snippets |
| `dns-resolver` | app + global | none | `--nginx-dns-resolver`, `--nginx-global-dns-resolver`, `--nginx-computed-dns-resolver` | Resolver address used for DNS-based upstreams. When unset, upstreams are IP-based |
| `dns-resolver-timeout` | app + global | `5s` | `--nginx-dns-resolver-timeout`, `--nginx-global-dns-resolver-timeout`, `--nginx-computed-dns-resolver-timeout` | Time nginx waits for a DNS response from the resolver |
| `dns-zone` | app + global | `docker` | `--nginx-dns-zone`, `--nginx-global-dns-zone`, `--nginx-computed-dns-zone` | Zone appended to `<app>.<process-type>` for DNS-based upstream hostnames |
| `error-log-path` | app + global | _`{nginx-log-root}/{app}-error.log`_ | `--nginx-error-log-path`, `--nginx-global-error-log-path`, `--nginx-computed-error-log-path` | Path inside the nginx container where error logs are written |
| `hsts` | app + global | `true` | `--nginx-hsts`, `--nginx-global-hsts`, `--nginx-computed-hsts` | When `true`, emits a `Strict-Transport-Security` header on HTTPS responses |
| `hsts-include-subdomains` | app + global | `true` | `--nginx-hsts-include-subdomains`, `--nginx-global-hsts-include-subdomains`, `--nginx-computed-hsts-include-subdomains` | Adds the `includeSubDomains` directive to the HSTS header |