feat: make coredns-docker upstream resolution opt-in
The bundled `nginx.conf.sigil` keeps rendering IP-based upstreams unless `dns-resolver` is set, for example with `dokku nginx:set --global dns-resolver 127.0.0.1:1053`. Unsetting the property reverts to IP-based upstreams, replacing the previous `off` sentinel.
This commit is contained in:
10 files changed
+107
-73
No files matched your search
@@ -8,22 +8,22 @@
|
||||
- The internal `buildpack-stack-name` plugin trigger has been removed. Each builder now reads its own `stack` property directly during the build.
|
||||
- Changing a builder's `stack` now clears that builder's build cache so the next deploy starts fresh. For herokuish this removes the `cache-<app>` Docker volume; for pack the next `pack build` runs with `--clear-cache`.
|
||||
- The `dokku` Debian package now declares `coredns-docker (>= 0.6.0)` as a hard `Depends:`. `apt install dokku` and `apt upgrade dokku` pull it in automatically. The deb ships a default `/etc/coredns/Corefile` that binds `127.0.0.1:1053`, serves the `docker.` zone, and synthesizes `<app>.<process-type>.docker` and `<app>.docker` `A` records from the `com.dokku.app-name` and `com.dokku.process-type` labels Dokku already stamps on every deployed container. The Docker image of Dokku ships coredns-docker preinstalled and runs it under runit on the same port.
|
||||
- The bundled `nginx.conf.sigil` now uses an nginx `resolver` directive and a variable-based `proxy_pass` against `<app>.<process-type>.docker` instead of pinning the upstream to a literal IP. This is what addresses the long-standing post-reboot and post-`apt-upgrade-docker` stale-vhost bug ([#8313](https://github.com/dokku/dokku/issues/8313)) - the rendered upstream re-resolves at request time via coredns-docker, so a host reboot or single-container restart no longer leaves nginx routing to a stale IP. See the [nginx documentation](/docs/networking/proxies/nginx.md#upstream-resolution-via-coredns-docker) for details.
|
||||
- Three new global nginx properties control the resolver path:
|
||||
|
||||
- `dns-resolver` (default: `127.0.0.1:1053`) - the address nginx talks to.
|
||||
- `dns-resolver-timeout` (default: `5s`) - how long nginx waits for a DNS response before giving up and returning 502. The short default keeps clients from waiting on nginx's built-in 30s timeout when coredns-docker is unreachable.
|
||||
- `dns-zone` (default: `docker`) - the suffix appended to `<app>.<process-type>` when constructing the upstream hostname.
|
||||
|
||||
All three can be set via `dokku nginx:set --global <property> <value>`.
|
||||
- The literal string `off` is the sentinel for disabling the resolver path. **Setting `dns-resolver` to the empty string does not disable it** - that deletes the property file and the new default (`127.0.0.1:1053`) re-engages. To revert to the legacy IP-based upstream block on hosts where coredns-docker cannot run (source/git installs, locally held coredns-docker package, postinst failures):
|
||||
- The bundled `nginx.conf.sigil` can now use an nginx `resolver` directive and a variable-based `proxy_pass` against `<app>.<process-type>.docker` instead of pinning the upstream to a literal IP. This addresses the long-standing post-reboot and post-`apt-upgrade-docker` stale-vhost bug ([#8313](https://github.com/dokku/dokku/issues/8313)) - the rendered upstream re-resolves at request time via coredns-docker, so a host reboot or single-container restart no longer leaves nginx routing to a stale IP. This behavior is opt-in, and upstreams remain IP-based unless `dns-resolver` is set:
|
||||
|
||||
```shell
|
||||
dokku nginx:set --global dns-resolver off
|
||||
dokku nginx:set --global dns-resolver 127.0.0.1:1053
|
||||
dokku proxy:build-config --all
|
||||
```
|
||||
|
||||
- Vendored `nginx.conf.sigil` templates are unaffected by the change. The existing `DOKKU_APP_<PROC>_LISTENERS` variable continues to be populated with the IP-based listener list exactly as before, so templates that only reference it continue to render IP-based upstreams regardless of the new property defaults. To opt a vendored template into the resolver path, switch its `proxy_pass` line to use the new `DOKKU_APP_<PROC>_LISTENER_HOST`, `NGINX_DNS_RESOLVER`, `NGINX_DNS_RESOLVER_TIMEOUT`, and `NGINX_DNS_ZONE` sigil variables - see the [nginx.conf.sigil reference](/docs/appendices/file-formats/nginx-conf-sigil.md#upstream-rendering-ip-based-vs-dns-based) for the exact snippet.
|
||||
See the [nginx documentation](/docs/networking/proxies/nginx.md#upstream-resolution-via-coredns-docker) for details.
|
||||
- Three new nginx properties control the resolver path:
|
||||
|
||||
- `dns-resolver` (default: none) - the address nginx talks to. Resolver-based upstreams are only rendered when this is set.
|
||||
- `dns-resolver-timeout` (default: `5s`) - how long nginx waits for a DNS response before giving up and returning 502. The short default keeps clients from waiting on nginx's built-in 30s timeout when coredns-docker is unreachable.
|
||||
- `dns-zone` (default: `docker`) - the suffix appended to `<app>.<process-type>` when constructing the upstream hostname.
|
||||
|
||||
All three can be set per app or globally via `dokku nginx:set`.
|
||||
- Vendored `nginx.conf.sigil` templates are unaffected by the change. The existing `DOKKU_APP_<PROC>_LISTENERS` variable continues to be populated with the IP-based listener list exactly as before, so templates that only reference it continue to render IP-based upstreams regardless of the new properties. To opt a vendored template into the resolver path, switch its `proxy_pass` line to use the new `DOKKU_APP_<PROC>_LISTENER_HOST`, `NGINX_DNS_RESOLVER`, `NGINX_DNS_RESOLVER_TIMEOUT`, and `NGINX_DNS_ZONE` sigil variables - see the [nginx.conf.sigil reference](/docs/appendices/file-formats/nginx-conf-sigil.md#upstream-rendering-ip-based-vs-dns-based) for the exact snippet.
|
||||
|
||||
### Stack property migrated to builder plugins
|
||||
|
||||
|
||||
@@ -101,7 +101,7 @@ The new variables (`DOKKU_APP_<PROC>_LISTENER_HOST` + `NGINX_DNS_RESOLVER` + `NG
|
||||
The bundled template emits this snippet:
|
||||
|
||||
```nginx
|
||||
{{ if ne $.NGINX_DNS_RESOLVER "off" }}
|
||||
{{ if $.NGINX_DNS_RESOLVER }}
|
||||
resolver {{ $.NGINX_DNS_RESOLVER }} valid=10s ipv6=off;
|
||||
resolver_timeout {{ $.NGINX_DNS_RESOLVER_TIMEOUT }};
|
||||
set $dokku_upstream "{{ $.DOKKU_APP_WEB_LISTENER_HOST }}:{{ $upstream_port }}";
|
||||
@@ -113,8 +113,8 @@ proxy_pass http://{{ $.APP }}-{{ $upstream_port }};
|
||||
|
||||
The `$dokku_upstream` variable name is intentionally namespaced (rather than `$upstream`) so vendored templates that already define `$upstream` will not collide if they adopt the resolver branch.
|
||||
|
||||
`NGINX_DNS_RESOLVER` defaults to `127.0.0.1:1053`, `NGINX_DNS_RESOLVER_TIMEOUT` defaults to `5s`, and `NGINX_DNS_ZONE` defaults to `docker`; all three are configurable via `dokku nginx:set --global dns-resolver ...`, `dokku nginx:set --global dns-resolver-timeout ...`, and `dokku nginx:set --global dns-zone ...`. The literal string `off` is the sentinel for disabling the resolver branch in the bundled template - empty strings are treated as "use the default" because the property is stored on disk and an empty value deletes the property file. The short default `dns-resolver-timeout` keeps clients from waiting on nginx's built-in 30s timeout when coredns-docker is unreachable; nginx returns 502 after the configured interval instead.
|
||||
`NGINX_DNS_RESOLVER` is empty by default, `NGINX_DNS_RESOLVER_TIMEOUT` defaults to `5s`, and `NGINX_DNS_ZONE` defaults to `docker`; all three are configurable via `dokku nginx:set --global dns-resolver ...`, `dokku nginx:set --global dns-resolver-timeout ...`, and `dokku nginx:set --global dns-zone ...`. The bundled template only renders the resolver branch when `NGINX_DNS_RESOLVER` is set, for example to `127.0.0.1:1053`. Unsetting `dns-resolver` switches the bundled template back to the IP-based upstream. The short default `dns-resolver-timeout` keeps clients from waiting on nginx's built-in 30s timeout when coredns-docker is unreachable; nginx returns 502 after the configured interval instead.
|
||||
|
||||
### Choosing between the two
|
||||
|
||||
The DNS-based path is the default for the bundled template. Vendored templates that have not been updated continue to use `DOKKU_APP_<PROC>_LISTENERS` and are unaffected by the global default. The DNS-based variables are populated regardless of whether `dns-resolver` is `off` or not, so a vendored template can adopt the resolver path on its own schedule by switching its rendering snippet, even if other apps on the host are still using IP-based upstreams.
|
||||
The IP-based path is the default for the bundled template, and the DNS-based path is used once `dns-resolver` is set. Vendored templates that have not been updated continue to use `DOKKU_APP_<PROC>_LISTENERS` and are unaffected by the `dns-resolver` setting. The `DOKKU_APP_<PROC>_LISTENER_HOST` and `NGINX_DNS_ZONE` variables are populated regardless of whether `dns-resolver` is set, so a vendored template can adopt the resolver path on its own schedule by switching its rendering snippet, even if other apps on the host are still using IP-based upstreams.
|
||||
@@ -88,7 +88,7 @@ Dokku is run in the following configuration:
|
||||
- Image build cache is set to the data dir + `/home/dokku`.
|
||||
- The docker socket is mounted into container.
|
||||
- `DOKKU_HOST_ROOT` and `DOKKU_LIB_HOST_ROOT` tell Dokku how the host Docker daemon sees those same trees when creating bind mounts (app repos and plugin data such as named storage).
|
||||
- `coredns-docker` (a CoreDNS plugin that serves DNS records for sibling Docker containers) is installed transitively via the dokku deb's `Depends:`, runs under runit on `127.0.0.1:1053` inside the container, and is what the `nginx-vhosts` `dns-resolver` default points at. This is what makes the `<app>.<process-type>.docker` upstream resolution work out of the box - see the [nginx proxy doc](/docs/networking/proxies/nginx.md#upstream-resolution-via-coredns-docker) for the operator-facing behavior.
|
||||
- `coredns-docker` (a CoreDNS plugin that serves DNS records for sibling Docker containers) is installed transitively via the dokku deb's `Depends:`, runs under runit on `127.0.0.1:1053` inside the container, and is what the `nginx-vhosts` `dns-resolver` property can point at to enable `<app>.<process-type>.docker` upstream resolution - see the [nginx proxy doc](/docs/networking/proxies/nginx.md#upstream-resolution-via-coredns-docker) for the operator-facing behavior.
|
||||
|
||||
Application repositories, plugin config, as well as plugin data are persisted to disk within the specified host directory for `/var/lib/dokku`. `DOKKU_LIB_HOST_ROOT` must point at that host path's `var/lib/dokku` subtree so Docker `-v` mounts resolve under the data volume rather than under the host's own `/var/lib/dokku`.
|
||||
|
||||
|
||||
@@ -33,27 +33,34 @@ Nginx will proxy the requests in a [round-robin balancing fashion](http://nginx.
|
||||
> [!IMPORTANT]
|
||||
> New as of 0.39.0
|
||||
|
||||
Dokku resolves nginx upstreams through [coredns-docker](https://github.com/dokku/coredns-docker), a CoreDNS plugin that watches the Docker daemon and serves fresh `A` records keyed off container labels. The nginx `resolver` directive points at coredns-docker's local socket (`127.0.0.1:1053`) and the rendered upstream is a variable-based `proxy_pass`, which forces nginx to re-resolve at request time honoring DNS TTL. This is how nginx avoids serving the wrong vhost after a host reboot, after `apt upgrade docker`, or after a single container restarts mid-runtime: the on-disk nginx config no longer pins a stale IP.
|
||||
Dokku can resolve nginx upstreams through [coredns-docker](https://github.com/dokku/coredns-docker), a CoreDNS plugin that watches the Docker daemon and serves fresh `A` records keyed off container labels. When enabled, the nginx `resolver` directive points at coredns-docker's local socket (`127.0.0.1:1053`) and the rendered upstream is a variable-based `proxy_pass`, which forces nginx to re-resolve at request time honoring DNS TTL. This is how nginx avoids serving the wrong vhost after a host reboot, after `apt upgrade docker`, or after a single container restarts mid-runtime: the on-disk nginx config no longer pins a stale IP. This behavior is opt-in, and nginx uses IP-based `upstream` blocks by default.
|
||||
|
||||
`coredns-docker` (>= `0.6.0`) is installed automatically as a transitive `Depends:` of the `dokku` Debian package, so `apt install dokku` and `apt upgrade dokku` pull it in without operator action. The shipped `/etc/coredns/Corefile` already binds `127.0.0.1:1053`, serves the `docker.` zone, and synthesizes `<app>.<process-type>.docker` and `<app>.docker` `A` records from Dokku's existing `com.dokku.app-name` and `com.dokku.process-type` container labels. Multiple containers for the same process type collapse onto one multi-A record set, so scaled apps load-balance across instances via DNS without any per-container configuration. The bundled Docker image of Dokku ships coredns-docker preinstalled and runs it under runit on the same port.
|
||||
|
||||
No `systemd-resolved` or `/etc/resolv.conf` change is required - nginx talks to `coredns-docker` directly via the `resolver` directive in each app's `nginx.conf`.
|
||||
|
||||
The behavior is controlled by three global nginx properties:
|
||||
|
||||
- `dns-resolver` (default: `127.0.0.1:1053`) - the resolver address nginx talks to.
|
||||
- `dns-resolver-timeout` (default: `5s`) - how long nginx waits for a DNS response before giving up and returning 502. The default is short so requests fail fast when coredns-docker is unreachable rather than hanging on the nginx built-in default (30s).
|
||||
- `dns-zone` (default: `docker`) - the suffix appended to `<app>.<process-type>` when constructing the upstream hostname.
|
||||
|
||||
To revert to the legacy IP-based upstream block (for instance on a host where `coredns-docker` cannot be installed):
|
||||
To enable resolver-based upstreams:
|
||||
|
||||
```shell
|
||||
dokku nginx:set --global dns-resolver off
|
||||
dokku nginx:set --global dns-resolver 127.0.0.1:1053
|
||||
dokku proxy:build-config --all
|
||||
```
|
||||
|
||||
To revert to the IP-based upstream block, unset the property and rebuild the config:
|
||||
|
||||
```shell
|
||||
dokku nginx:set --global dns-resolver
|
||||
dokku proxy:build-config --all
|
||||
```
|
||||
|
||||
The behavior is controlled by three nginx properties, each settable per app or globally:
|
||||
|
||||
- `dns-resolver` (default: none) - the resolver address nginx talks to. Resolver-based upstreams are only rendered when this is set.
|
||||
- `dns-resolver-timeout` (default: `5s`) - how long nginx waits for a DNS response before giving up and returning 502. The default is short so requests fail fast when coredns-docker is unreachable rather than hanging on the nginx built-in default (30s).
|
||||
- `dns-zone` (default: `docker`) - the suffix appended to `<app>.<process-type>` when constructing the upstream hostname.
|
||||
|
||||
> [!NOTE]
|
||||
> The sentinel for disabling the feature is the literal string `off`. Setting `dns-resolver` to the empty string (`""`) does **not** disable it - that deletes the property and the default (`127.0.0.1:1053`) re-engages. Use `off` to disable, the default to re-enable.
|
||||
> When `dns-resolver` is set, no `upstream` block is rendered for the `web` process, so `proxy-keepalive` has no effect on it. Upstreams for [proxy routes](/docs/networking/proxy-management.md#path-based-routing-to-non-web-processes) to other process types remain IP-based.
|
||||
|
||||
See [the `nginx.conf.sigil` reference](/docs/appendices/file-formats/nginx-conf-sigil.md) for the sigil variables that drive the resolver-based render, and how vendored templates can adopt or skip it.
|
||||
|
||||
@@ -672,6 +679,9 @@ All nginx-vhosts properties are settable at both the app and global scope.
|
||||
| `client-header-timeout` | app + global | `60s` | `--nginx-client-header-timeout`, `--nginx-global-client-header-timeout`, `--nginx-computed-client-header-timeout` | Time allowed to read the request header from the client |
|
||||
| `client-max-body-size` | app + global | `1m` | `--nginx-client-max-body-size`, `--nginx-global-client-max-body-size`, `--nginx-computed-client-max-body-size` | Maximum allowed request body size |
|
||||
| `disable-custom-config` | app + global | `false` | `--nginx-disable-custom-config`, `--nginx-global-disable-custom-config`, `--nginx-computed-disable-custom-config` | When `true`, ignores app-supplied `nginx.conf.d/*.conf` snippets |
|
||||
| `dns-resolver` | app + global | none | `--nginx-dns-resolver`, `--nginx-global-dns-resolver`, `--nginx-computed-dns-resolver` | Resolver address used for DNS-based upstreams. When unset, upstreams are IP-based |
|
||||
| `dns-resolver-timeout` | app + global | `5s` | `--nginx-dns-resolver-timeout`, `--nginx-global-dns-resolver-timeout`, `--nginx-computed-dns-resolver-timeout` | Time nginx waits for a DNS response from the resolver |
|
||||
| `dns-zone` | app + global | `docker` | `--nginx-dns-zone`, `--nginx-global-dns-zone`, `--nginx-computed-dns-zone` | Zone appended to `<app>.<process-type>` for DNS-based upstream hostnames |
|
||||
| `error-log-path` | app + global | _`{nginx-log-root}/{app}-error.log`_ | `--nginx-error-log-path`, `--nginx-global-error-log-path`, `--nginx-computed-error-log-path` | Path inside the nginx container where error logs are written |
|
||||
| `hsts` | app + global | `true` | `--nginx-hsts`, `--nginx-global-hsts`, `--nginx-computed-hsts` | When `true`, emits a `Strict-Transport-Security` header on HTTPS responses |
|
||||
| `hsts-include-subdomains` | app + global | `true` | `--nginx-hsts-include-subdomains`, `--nginx-global-hsts-include-subdomains`, `--nginx-computed-hsts-include-subdomains` | Adds the `includeSubDomains` directive to the HSTS header |
|
||||
|
||||
Reference in new issue
Block a user