If the scheduler is not docker-local, we will assume that a separate plugin trigger will pull access/error logs on our behalf. For k3s, the naive implementation is to pull logs from a single container (multiple ingress-nginx pods will be supported in a future release).
Closes#7268
While this will now more or less require a local docker login - which can be overridden by setting the image-pull-secrets property - this ensures every deploy has a valid context.
One annoying thing is that the secret is per-app vs global, meaning that if the secret must be rotated, all apps need to be redeployed. A future change can fix that by deploying a global secret, or if folks really desire, this can be done by setting the image-pull-secrets property on the app/globally.
Different options get applied to each depending on what is needed for the role.
This also updates how we initialize the cluster to better support how metrics exposure and cross-server networking.
This utilizes a Kubernetes Job to start a pod in order to take advantage of completed Job cleanup in Kubernetes. It also properly handles interactivity by ensuring the container has a TTY.
Note that there is currently no provision for parsing docker arguments and translating them into their Kubernetes equivalents.
The big feature missing from deployments today is healthchecks, which require extra parsing of the app.json format since the healthchecks there aren't immediately serializable to kubernetes deployment healthchecks.
Other smaller missing functionality from the existing OSS kubernetes scheduler are:
- The ability to set custom pod annotations
- The ability to set custom deployment annotations
- The ability to mount volumes
Exposing these pieces of functionality will come at a later date, if at all.
