The `scheduler-k3s` metadata commands were all per-key, so matching an app's stored annotations, labels or node sysctls to a declared set meant reading the report, computing the difference in both directions and issuing one command per added, changed and removed key, with a failure partway through leaving a mixture of the two sets. A `--replace` flag on `annotations:set`, `labels:set`, `node-sysctls:set` and `autoscaling-auth:set` now writes the declared map in a single call, taking trailing `key=value` pairs everywhere except `autoscaling-auth:set`, which keeps its repeatable `--metadata` flag. Every pair is parsed before anything is written, so a rejected pair leaves the stored map untouched, and a pair ending in `=` stores an empty value, which the per-key form cannot express. An empty pair list is rejected rather than treated as a request to remove everything, so a generated list that expands to nothing cannot silently drop an app's metadata; `annotations:clear`, `labels:clear` and `node-sysctls:clear` handle that case, taking the same optional filters as their report commands. The `--resource-type` flag and the app name are now validated on the set commands as well, where a typo previously exited zero and wrote a property that no report and no deploy ever read.
Sysctls the kernel does not namespace, such as `vm.max_map_count`, cannot be set from a pod spec and previously had no answer beyond editing `/etc/sysctl.d` on each host by hand. `scheduler-k3s:node-sysctls:set` now applies them through a privileged daemonset, which reaches nodes joined later and reapplies after a reboot. Sysctls may be scoped to a node profile, with a profile scope inheriting the global values and overriding them on conflict so that every node is covered by exactly one daemonset. Clearing a sysctl stops dokku managing it but does not restore the previous value, which persists until the node reboots.
The server node created by `scheduler-k3s:initialize` had no way to receive kubelet arguments, unlike nodes joined through `scheduler-k3s:cluster:add` or configured via `scheduler-k3s:profiles:add`. This meant settings such as `allowed-unsafe-sysctls` were unreachable on a single-node install.
Align autoscaling-auth:report with annotations and labels reporting so export tools can recover configured trigger auth via flat JSON keys and info flags, while stdout stays secret-safe unless --include-metadata is used.
Closes#8800
Users can override the default 3-line surrounding context per change. Pass
`--context -1` to render the full resource around every change, matching
upstream `helm diff upgrade` behavior.
Adds a `scheduler-k3s:preview <app>` command that renders a unified diff
between the manifests currently stored in the live Helm release for an app
and the manifests that the next deploy would roll out. The diff is produced
entirely via the Helm Go SDK that scheduler-k3s already uses, with no
external `helm` binary or `helm-diff` plugin install required. The
underlying diff and parse logic is vendored from `databus23/helm-diff`
(Apache 2.0) into a new internal package, trimmed of three-way-merge,
release-ownership tracking, and the non-default output formatters.
The chart-construction half of `TriggerSchedulerDeploy` has been extracted
into a reusable `BuildAppChart` helper so deploy and preview share the
same chart materialization without duplicating template logic. Secret
values are redacted by default in preview output; `--show-secrets` and
`--show-secrets-decoded` flags match the upstream `helm diff` UX. Only the
main app helm release is compared in this iteration; auxiliary releases
for config vars, image-pull secrets, and TLS certificates are out of scope
and documented as such.
The scheduler-k3s plugin exposes `annotations:set` and `labels:set` but no matching report subcommands, so scripts could not inspect the configured state for idempotent management without reading the property files directly. The new `scheduler-k3s:annotations:report` and `scheduler-k3s:labels:report` surface the configured entries with stdout, JSON, single-flag query, and `--process-type` / `--resource-type` filtering, mirroring the recently added `scheduler-k3s:charts:report`. The literal `GlobalProcessType` value `--global` is rendered as `global` in report keys to avoid leading dashes. The `scheduler-k3s:autoscaling-auth:report` command is updated alongside to loop over every app when no app and no `--global` flag is provided, matching the convention used by `scheduler-k3s:report`.
Helm chart overrides on the scheduler-k3s plugin were previously set
through the generic `scheduler-k3s:set --global chart.<chart>.<property>`
interface, which mixed chart-level configuration into the same command
that manages scheduler properties and offered no focused way to inspect
which overrides were configured. A dedicated `scheduler-k3s:charts:set`
sets and clears chart-specific helm values, and a complementary
`scheduler-k3s:charts:report` surfaces configured overrides per chart
with optional JSON output and single-field flag queries. The legacy form
on `scheduler-k3s:set` continues to work but now emits a deprecation
warning pointing users at the new subcommand.
Every `:report` subcommand now recognizes `--global` as a scope selector that limits the report to globally-configured properties, including in JSON form via `--global --format json`. Previously this combination was rejected because `--global` was treated as an info flag, conflicting with `--format`. The shared `common.ParseReportArgs` helper now returns a `ReportArgs` struct exposing the parsed scope; each Go and bash report selects a global-only flag map when scope is global, and skips per-app verification.
Node profiles can be used to specify repetitive options for clusters, making it easier to add specific types of instances quickly.
This PR additionally fixes kubelet-args specification - the same key can be specified multiple times, such as when specifying custom sysctl rules.
Dokku ships with default charts and customizes their helm values. In certain cases, it may be useful to further customize those values to better support specific installations. With this change, it is now possible to set custom values by providing the key in the format `chart.$CHART_NAME.$PROPERTY`. For instance, to increase timeouts for the keda http addon:
```shell
dokku scheduler-k3s:set --global chart.keda-add-ons-http.interceptor.tcpConnectTimeout 120s
dokku scheduler-k3s:set --global chart.keda-add-ons-http.interceptor.keepAlive 120s
dokku scheduler-k3s:set --global chart.keda-add-ons-http.interceptor.responseHeaderTimeout 120s
dokku scheduler-k3s:set --global chart.keda-add-ons-http.interceptor.idleConnTimeout 120s
dokku scheduler-k3s:set --global chart.keda-add-ons-http.interceptor.tlsHandshakeTimeout 120s
dokku scheduler-k3s:set --global chart.keda-add-ons-http.interceptor.expectContinueTimeout 120s
```
To reset a value to its default, simply set the value to an empty string.
```shell
dokku scheduler-k3s:set --global chart.keda-add-ons-http.interceptor.tcpConnectTimeout
```
Once the values are set, users will need to then run `scheduler-k3s:ensure-charts` with either the `--force` flag or specify the `--chart` flag with a list of charts to force apply.
```shell
dokku scheduler-k3s:ensure-charts --force
dokku scheduler-k3s:ensure-charts --chart keda-add-ons-http
```
Note that the existing dokku-provided values cannot be removed, but can be overridden.
Different options get applied to each depending on what is needed for the role.
This also updates how we initialize the cluster to better support how metrics exposure and cross-server networking.