fix: add proxy health checks from app.json readiness
The caddy, haproxy, openresty, and traefik docs stated that requests are only routed to containers passing healthchecks, but these proxies route to a new container as soon as it starts. The first `web` readiness healthcheck in `app.json` is now also turned into health checks and retries for caddy and openresty, as it already was for traefik, so requests are routed away from containers that are not yet ready, and the docs now describe how each proxy actually behaves.
This commit is contained in:
15 files changed
+694
-39
No files matched your search
@@ -32,7 +32,53 @@ The Caddy plugin has specific rules for routing requests:
|
||||
- If no `http:80` mapping is found, the first `http` port mapping is used for http requests.
|
||||
- If no `https:443` mapping is found, the first `https` port mapping is used for https requests.
|
||||
- If no `https` mapping is found, the container port from `http:80` will be used for https requests.
|
||||
- Requests are routed as soon as the container is running and passing healthchecks.
|
||||
- Requests are routed as soon as the container is running, before Dokku's healthchecks have passed. See the [healthchecks](#healthchecks) section for how Caddy avoids routing to containers that are not yet ready.
|
||||
|
||||
### Healthchecks
|
||||
|
||||
When an app has a readiness healthcheck defined in its `app.json` file with a `path` property, Dokku automatically generates Caddy healthcheck and retry labels. These configure Caddy to periodically check each container, stop routing traffic to containers that fail the check, and retry requests that fail to connect against another container.
|
||||
|
||||
The following `app.json` healthcheck properties are mapped to Caddy labels:
|
||||
|
||||
| app.json Property | Caddy Label | Description |
|
||||
|-------------------|-------------|-------------|
|
||||
| `path` | `caddy.reverse_proxy.health_uri` | The HTTP path to check (required) |
|
||||
| `port` | `caddy.reverse_proxy.health_port` | The port to check |
|
||||
| `timeout` | `caddy.reverse_proxy.health_timeout` | Timeout in seconds (formatted as `Xs`, defaults to `5s`) |
|
||||
| `wait` | `caddy.reverse_proxy.health_interval` | Interval between checks in seconds (formatted as `Xs`, defaults to `5s`) |
|
||||
|
||||
Additionally, the following labels are generated:
|
||||
|
||||
| Caddy Label | Value | Description |
|
||||
|-------------|-------|-------------|
|
||||
| `caddy.reverse_proxy.health_headers.Host` | the first app domain | The `Host` header sent with the healthcheck, matching the header Dokku sends during deploy checks |
|
||||
| `caddy.reverse_proxy.lb_try_duration` | `5s` | How long a request that fails to connect is retried against other containers |
|
||||
| `caddy.reverse_proxy.lb_try_interval` | `250ms` | How long to wait between retries |
|
||||
|
||||
Example `app.json` configuration:
|
||||
|
||||
```json
|
||||
{
|
||||
"healthchecks": {
|
||||
"web": [
|
||||
{
|
||||
"name": "web readiness check",
|
||||
"path": "/health",
|
||||
"timeout": 5,
|
||||
"type": "readiness",
|
||||
"wait": 10
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Caddy considers a new container healthy until its first check fails. The first check runs as soon as Caddy picks up the new container, and requests that reach the container before then and fail to connect are retried against the still-running old container.
|
||||
|
||||
The `scheme` property is not mapped, as Caddy uses the same transport for healthchecks as for proxied requests. Any of the generated labels may be overridden via the `caddy:labels:add` command.
|
||||
|
||||
> [!NOTE]
|
||||
> Only the first readiness healthcheck with a `path` property is used. Apps without one are routed as soon as the container is running. See the [zero downtime deploys documentation](/docs/deployment/zero-downtime-deploys.md#healthchecks-and-label-based-proxies) for more information.
|
||||
|
||||
### Switching to Caddy
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ The Haproxy plugin has specific rules for routing requests:
|
||||
- Haproxy integration is exposed via docker labels attached to containers. Changes in labels require either app deploys or rebuilds.
|
||||
- While Haproxy will respect labels associated with other containers, only `web` containers have Haproxy labels injected by the plugin.
|
||||
- Only `http:80` and `https:443` port mappings are supported at this time.
|
||||
- Requests are routed as soon as the container is running and passing healthchecks.
|
||||
- Requests are routed as soon as the container is running, before Dokku's healthchecks have passed. Haproxy does not support healthcheck configuration via container labels, so `app.json` healthchecks are not used for routing.
|
||||
|
||||
### Switching to Haproxy
|
||||
|
||||
|
||||
@@ -28,7 +28,44 @@ The OpenResty plugin has specific rules for routing requests:
|
||||
- OpenResty integration is exposed via docker labels attached to containers. Changes in labels require either app deploys or rebuilds.
|
||||
- While OpenResty will respect labels associated with other containers, only `web` containers have OpenResty labels injected by the plugin.
|
||||
- Only `http:80` and `https:443` port mappings are supported at this time.
|
||||
- Requests are routed as soon as the container is running and passing healthchecks.
|
||||
- Requests are routed as soon as the container is running, before Dokku's healthchecks have passed. See the [healthchecks](#healthchecks) section for how failed requests to a container that is not yet listening are handled.
|
||||
|
||||
### Healthchecks
|
||||
|
||||
When an app has a readiness healthcheck defined in its `app.json` file with a `path` property, Dokku automatically generates OpenResty healthcheck labels. These configure OpenResty to periodically check each container and stop routing traffic to containers that fail the check until they pass again.
|
||||
|
||||
The following `app.json` healthcheck properties are mapped to OpenResty labels:
|
||||
|
||||
| app.json Property | OpenResty Label | Description |
|
||||
|-------------------|-----------------|-------------|
|
||||
| `path` | `openresty.healthcheck-path` | The HTTP path to check (required) |
|
||||
| `port` | `openresty.healthcheck-port` | The port to check |
|
||||
| `scheme` | `openresty.healthcheck-scheme` | The scheme to use (`http` or `https`) |
|
||||
| `timeout` | `openresty.healthcheck-timeout` | Timeout in seconds (defaults to `5`) |
|
||||
| `wait` | `openresty.healthcheck-interval` | Interval between checks in seconds (defaults to `5`) |
|
||||
| `attempts` | `openresty.healthcheck-fall` | Number of successive failed checks before a container is taken out of rotation (defaults to `3`) |
|
||||
| `httpHeaders` | `openresty.healthcheck-header.<name>` | Headers to send with the check. A `Host` header is set via `openresty.healthcheck-host` instead |
|
||||
|
||||
If no `Host` header is specified, the first app domain is used, matching the header Dokku sends during deploy checks. Headers with invalid names or values containing quotes, backslashes, or control characters are skipped.
|
||||
|
||||
Additionally, the following labels are generated so that a request that fails to connect to a container is retried against another container, and the failing container is immediately taken out of rotation:
|
||||
|
||||
| OpenResty Label | Value | Description |
|
||||
|-----------------|-------|-------------|
|
||||
| `openresty.upstream-max-fails` | `1` | Number of failed requests before a container is taken out of rotation |
|
||||
| `openresty.upstream-fail-timeout` | the healthcheck `wait` value, or `5s` | How long a container is taken out of rotation after a failed request |
|
||||
| `openresty.proxy-next-upstream-timeout` | `5s` | How long a failed request may be retried against other containers |
|
||||
|
||||
OpenResty considers a new container healthy until it fails a check or a request to it fails to connect, and healthcheck state is reset whenever the OpenResty configuration is reloaded. Responses with a `2xx` or `3xx` status are considered healthy, and the `content` property is not supported.
|
||||
|
||||
The status of all checked upstreams can be viewed from within the OpenResty container:
|
||||
|
||||
```shell
|
||||
docker exec openresty-openresty-1 wget -qO- http://127.0.0.1:8999/upstream-healthcheck-status
|
||||
```
|
||||
|
||||
> [!NOTE]
|
||||
> Only the first readiness healthcheck with a `path` property is used. Apps without one are routed as soon as the container is running, though requests that fail to connect to a container are still retried against other containers. See the [zero downtime deploys documentation](/docs/deployment/zero-downtime-deploys.md#healthchecks-and-label-based-proxies) for more information.
|
||||
|
||||
### Switching to OpenResty
|
||||
|
||||
|
||||
@@ -31,12 +31,12 @@ The Traefik plugin has specific rules for routing requests:
|
||||
- If no `http:80` mapping is found, the first `http` port mapping is used for http requests.
|
||||
- If no `https:443` mapping is found, the first `https` port mapping is used for https requests.
|
||||
- If no `https` mapping is found, the container port from `http:80` will be used for https requests.
|
||||
- Requests are routed as soon as the container is running and passing healthchecks.
|
||||
- Requests are routed as soon as the container is running, before Dokku's healthchecks have passed.
|
||||
- Readiness healthchecks defined in `app.json` with a `path` property are automatically transformed into Traefik healthcheck labels.
|
||||
|
||||
### Healthchecks
|
||||
|
||||
When an app has a readiness healthcheck defined in its `app.json` file with a `path` property, Dokku automatically generates Traefik healthcheck labels. These labels configure Traefik to perform health checks on the container before routing traffic to it.
|
||||
When an app has a readiness healthcheck defined in its `app.json` file with a `path` property, Dokku automatically generates Traefik healthcheck labels. These labels configure Traefik to periodically check each container and stop routing traffic to containers that fail the check. Traefik considers a new container healthy until its first check fails, so requests may reach a container before its first check runs.
|
||||
|
||||
The following `app.json` healthcheck properties are mapped to Traefik labels:
|
||||
|
||||
@@ -67,7 +67,7 @@ Example `app.json` configuration:
|
||||
```
|
||||
|
||||
> [!NOTE]
|
||||
> Only the first readiness healthcheck with a `path` property is used. Liveness, startup, and command-based healthchecks are not transformed into Traefik labels.
|
||||
> Only the first readiness healthcheck with a `path` property is used. Liveness, startup, and command-based healthchecks are not transformed into Traefik labels. See the [zero downtime deploys documentation](/docs/deployment/zero-downtime-deploys.md#healthchecks-and-label-based-proxies) for more information.
|
||||
|
||||
### Switching to Traefik
|
||||
|
||||
|
||||
Reference in new issue
Block a user