diff --git a/docs/README.md b/docs/README.md index b4e6d5c81..00e8acd4a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -96,6 +96,7 @@ This documentation covers the installation, configuration, and usage of Dokku - - [Plugin Management](advanced-usage/plugin-management.md) - Install and manage plugins - [Registry Management](advanced-usage/registry-management.md) - Docker registry integration - [Repository Management](advanced-usage/repository-management.md) - Git repository settings +- [Build Tracking](advanced-usage/builds.md) - Inspect, cancel, and manage builds - [Deployment Tasks](advanced-usage/deployment-tasks.md) - Pre/post deploy hooks - [Event Logs](advanced-usage/event-logs.md) - Dokku event history - [Backup and Recovery](advanced-usage/backup-recovery.md) - Data backup strategies diff --git a/docs/advanced-usage/builds.md b/docs/advanced-usage/builds.md new file mode 100644 index 000000000..0a3fc16d9 --- /dev/null +++ b/docs/advanced-usage/builds.md @@ -0,0 +1,171 @@ +# Build Tracking + +> [!IMPORTANT] +> New as of 0.34.0 + +Every deploy that flows through Dokku - whether triggered by `git push`, `ps:rebuild`, `ps:restart`, `config:set`, or `git:from-archive` / `git:from-image` / `git:sync` / `git:load-image` - is recorded as a structured build record on disk. The `builds` plugin lets operators inspect what is currently deploying, look up the result of an old deploy, and stream the captured build log without depending on `journalctl`. + +``` +builds:cancel # Cancel a running build for an app +builds:info [--format json] # Show details for a single build +builds:list [] [--format json] [--kind ...] [--status ...] + # List builds (running across all apps, or running + history for one) +builds:output [|current] # Show build output (tail for live, cat for finished) +builds:prune [--all-apps] # Reap abandoned records and apply retention +builds:report [] [] # Display a build report +builds:set [--global|] [] # Set or clear a builds property +``` + +## Build records + +Every build is identified by a sortable base36 ULID-style id (`DOKKU_BUILD_ID`) generated at deploy start. For each build, Dokku writes: + +- `$DOKKU_LIB_ROOT/data/builds//.json` - the structured record +- `$DOKKU_LIB_ROOT/data/builds//.log` - the captured stdout/stderr of the deploy + +Output is also tagged into syslog as `dokku-` so `journalctl -t dokku-` continues to work. The on-disk log file is the durable source of truth and is read for `builds:output` even when journald has rotated old entries away. + +### Record schema + +```json +{ + "id": "01j8c4xv7bk5w3", + "app": "myapp", + "kind": "build", + "pid": 12345, + "started_at": "2026-04-30T13:50:00Z", + "finished_at": "2026-04-30T13:51:14Z", + "status": "succeeded", + "source": "git-hook", + "exit_code": 0 +} +``` + +- **kind** - `build` for paths that produce a new image (`git push`, `git:*`, `ps:rebuild`); `deploy` for paths that re-deploy an existing image (`ps:restart`, `ps:start`, `dokku deploy`, `config:set`). +- **status** - on-disk values are `running | succeeded | failed | canceled`. `abandoned` is a fifth display-only value computed at read time for `running` records whose PID is no longer alive; it is never persisted to the record. +- **source** - the user-typed command that originated the deploy (e.g. `git-hook`, `ps:restart`, `config-redeploy`, `git:sync`). + +## Listing builds + +Without an app argument, `builds:list` shows currently-running builds across every app on the host: + +```shell +dokku builds:list +``` + +``` +=====> Currently running builds +App Build ID Kind PID Source Started +myapp 01j8c4xv7bk5w3 build 12345 git-hook 2026-04-30T13:50:00Z +``` + +With an app, `builds:list` returns running builds plus the most recent finalized records up to the configured retention: + +```shell +dokku builds:list myapp +``` + +Filter by kind or status: + +```shell +dokku builds:list myapp --kind build +dokku builds:list myapp --status running +dokku builds:list --format json +``` + +## Inspecting a single build + +```shell +dokku builds:info myapp 01j8c4xv7bk5w3 +``` + +``` +=====> Build 01j8c4xv7bk5w3 + Build ID: 01j8c4xv7bk5w3 + App: myapp + Kind: build + Status: succeeded + PID: 12345 + Source: git-hook + Started: 2026-04-30T13:50:00Z + Finished: 2026-04-30T13:51:14Z + Duration: 1m14s + Exit Code: 0 + Log: /var/lib/dokku/data/builds/myapp/01j8c4xv7bk5w3.log +``` + +JSON output includes the same fields plus a computed `log_path`: + +```shell +dokku builds:info myapp 01j8c4xv7bk5w3 --format json | jq . +``` + +## Streaming build output + +```shell +# tail -f the live build, or cat the log for a finished one +dokku builds:output myapp 01j8c4xv7bk5w3 + +# resolve "current" from the in-flight deploy +dokku builds:output myapp current +``` + +If the on-disk log file is missing (for example, a build from before this plugin was installed), `builds:output` falls back to `journalctl -t dokku-`. + +## Cancelling a build + +`builds:cancel` reads the active `.deploy.lock` for an app, looks up the matching build record, and sends `SIGQUIT` to the deploy's process group. The record is finalized as `canceled` (or `failed` if the process had already exited without finalizing - in that case Dokku marks the record so it doesn't sit in `running` forever). + +```shell +dokku builds:cancel myapp +``` + +If the record is already finalized when cancel runs, no signal is sent and the record is left untouched. + +## Retention + +Build records are pruned by count, not by age. The default retention is **20** records per app. Live in-flight deploys are never pruned regardless of count. + +Set a per-app retention: + +```shell +dokku builds:set myapp retention 50 +``` + +Set the global default: + +```shell +dokku builds:set --global retention 10 +``` + +Clear an override (falls back to the global value, then the default): + +```shell +dokku builds:set myapp retention +``` + +## Manual pruning + +`builds:prune` invokes the same logic that runs at the end of every deploy - it reaps abandoned records (status=running with a dead PID, finalized as `failed`) and trims the directory to the configured retention. Useful after lowering retention via `builds:set` or to clean up after a host reboot: + +```shell +dokku builds:prune myapp +dokku builds:prune --all-apps +``` + +## Reports + +```shell +dokku builds:report +dokku builds:report myapp +dokku builds:report myapp --build-status +``` + +Available flags: + +- `--build-id`, `--build-kind`, `--build-status`, `--build-pid`, `--build-source`, `--build-started-at`, `--build-finished-at`, `--build-exit-code`: details of the most recent build for the app +- `--builds-retention`: per-app retention override (empty if none) +- `--builds-global-retention`: global retention override (empty if none) +- `--builds-computed-retention`: the resolved retention applied to this app + +`--build-status` returns the **display** status, so an abandoned in-flight build shows `abandoned` rather than `running`. The raw on-disk status is only visible by reading the JSON record directly. diff --git a/dokku b/dokku index 7e98c7174..495e7aa9f 100755 --- a/dokku +++ b/dokku @@ -188,9 +188,8 @@ execute_dokku_cmd() { set -- "$PLUGIN_CMD" "$@" fi - if [[ "$PLUGIN_NAME" == "git-hook" ]] || [[ "$PLUGIN_NAME" == "ps:rebuild" ]] || [[ "$PLUGIN_NAME" == "ps:restart" ]] || [[ "$PLUGIN_NAME" == "deploy" ]]; then - export DOKKU_REDIRECT_OUTPUT=true - exec &> >(tee >(tee | logger -i -t "dokku-${DOKKU_PID}")) + if [[ "$PLUGIN_NAME" == "git-hook" ]] || [[ "$PLUGIN_NAME" == "ps:rebuild" ]] || [[ "$PLUGIN_NAME" == "ps:restart" ]]; then + dokku_setup_build_capture "$2" "$PLUGIN_NAME" fi if [[ -x $PLUGIN_ENABLED_PATH/$PLUGIN_NAME/subcommands/default ]]; then diff --git a/plugins/builds/.gitignore b/plugins/builds/.gitignore new file mode 100644 index 000000000..ec242ac8a --- /dev/null +++ b/plugins/builds/.gitignore @@ -0,0 +1,10 @@ +/commands +/subcommands/* +/triggers/* +/triggers +/builds-generate-id +/builds-record-finalize +/builds-record-start +/install +/post-app-rename-setup +/post-delete diff --git a/plugins/builds/Makefile b/plugins/builds/Makefile new file mode 100644 index 000000000..f1ce6dc1f --- /dev/null +++ b/plugins/builds/Makefile @@ -0,0 +1,6 @@ +SUBCOMMANDS = subcommands/cancel subcommands/info subcommands/list subcommands/output subcommands/prune subcommands/report subcommands/set +TRIGGERS = triggers/builds-generate-id triggers/builds-record-finalize triggers/builds-record-start triggers/install triggers/post-app-rename-setup triggers/post-delete +BUILD = commands subcommands triggers +PLUGIN_NAME = builds + +include ../../common.mk diff --git a/plugins/builds/builds.go b/plugins/builds/builds.go new file mode 100644 index 000000000..80b0bed4b --- /dev/null +++ b/plugins/builds/builds.go @@ -0,0 +1,446 @@ +package builds + +import ( + "crypto/rand" + "encoding/json" + "fmt" + "os" + "path/filepath" + "sort" + "strconv" + "strings" + "syscall" + "time" + + "github.com/dokku/dokku/plugins/common" + + "github.com/multiformats/go-base36" +) + +const ( + DefaultRetention = 20 + MinimumRetention = 1 +) + +var ( + DefaultProperties = map[string]string{ + "retention": strconv.Itoa(DefaultRetention), + } + + GlobalProperties = map[string]bool{ + "retention": true, + } +) + +// BuildKind represents whether a record is a full build or a deploy-only run. +type BuildKind string + +const ( + BuildKindBuild BuildKind = "build" + BuildKindDeploy BuildKind = "deploy" +) + +// Valid reports whether the kind is one of the recognized values. +func (k BuildKind) Valid() bool { + return k == BuildKindBuild || k == BuildKindDeploy +} + +// BuildStatus is the on-disk lifecycle status of a build record. +// BuildStatusAbandoned is intentionally NOT considered Valid - it is a +// display-only value computed at read time and never persisted. +type BuildStatus string + +const ( + BuildStatusRunning BuildStatus = "running" + BuildStatusSucceeded BuildStatus = "succeeded" + BuildStatusFailed BuildStatus = "failed" + BuildStatusCanceled BuildStatus = "canceled" + BuildStatusAbandoned BuildStatus = "abandoned" +) + +// Valid reports whether the status is a persistable value. +func (s BuildStatus) Valid() bool { + switch s { + case BuildStatusRunning, BuildStatusSucceeded, BuildStatusFailed, BuildStatusCanceled: + return true + } + return false +} + +// IsTerminal reports whether the status represents a finalized build. +func (s BuildStatus) IsTerminal() bool { + switch s { + case BuildStatusSucceeded, BuildStatusFailed, BuildStatusCanceled: + return true + } + return false +} + +// BuildSource identifies the user-typed command (or internal trigger) that +// originated a build. +type BuildSource string + +const ( + BuildSourceGitHook BuildSource = "git-hook" + BuildSourcePsRebuild BuildSource = "ps:rebuild" + BuildSourcePsRestart BuildSource = "ps:restart" + BuildSourcePsStart BuildSource = "ps:start" + BuildSourceDeploy BuildSource = "deploy" + BuildSourceConfigRedeploy BuildSource = "config-redeploy" + BuildSourceGitSync BuildSource = "git:sync" + BuildSourceGitFromArchive BuildSource = "git:from-archive" + BuildSourceGitFromImage BuildSource = "git:from-image" + BuildSourceGitLoadImage BuildSource = "git:load-image" + BuildSourceUnknown BuildSource = "unknown" +) + +var allBuildSources = []BuildSource{ + BuildSourceGitHook, + BuildSourcePsRebuild, + BuildSourcePsRestart, + BuildSourcePsStart, + BuildSourceDeploy, + BuildSourceConfigRedeploy, + BuildSourceGitSync, + BuildSourceGitFromArchive, + BuildSourceGitFromImage, + BuildSourceGitLoadImage, + BuildSourceUnknown, +} + +// Valid reports whether the source is one of the recognized values. +func (s BuildSource) Valid() bool { + for _, candidate := range allBuildSources { + if s == candidate { + return true + } + } + return false +} + +// DefaultKind maps a source to the kind of work it performs. Sources that +// produce a new image map to BuildKindBuild; sources that re-deploy an existing +// image map to BuildKindDeploy. +func (s BuildSource) DefaultKind() BuildKind { + switch s { + case BuildSourceGitHook, + BuildSourcePsRebuild, + BuildSourceGitSync, + BuildSourceGitFromArchive, + BuildSourceGitFromImage, + BuildSourceGitLoadImage: + return BuildKindBuild + } + return BuildKindDeploy +} + +// Build is the persisted record for a single build/deploy. +type Build struct { + ID string `json:"id"` + App string `json:"app"` + Kind BuildKind `json:"kind"` + PID int `json:"pid"` + StartedAt time.Time `json:"started_at"` + FinishedAt *time.Time `json:"finished_at,omitempty"` + Status BuildStatus `json:"status"` + Source BuildSource `json:"source"` + ExitCode *int `json:"exit_code,omitempty"` +} + +// DisplayStatus returns the status the operator should see, computing +// BuildStatusAbandoned for running records whose PID is no longer alive. +func (b Build) DisplayStatus() BuildStatus { + if b.Status == BuildStatusRunning && !CheckPIDAlive(b.PID) { + return BuildStatusAbandoned + } + return b.Status +} + +// Duration returns the elapsed time between start and finish (or now, for +// in-flight builds). +func (b Build) Duration() time.Duration { + end := time.Now().UTC() + if b.FinishedAt != nil { + end = *b.FinishedAt + } + if end.Before(b.StartedAt) { + return 0 + } + return end.Sub(b.StartedAt).Round(time.Second) +} + +// LogPath returns the on-disk path of the per-build log file. +func (b Build) LogPath() string { + return filepath.Join(common.GetAppDataDirectory("builds", b.App), b.ID+".log") +} + +// GenerateBuildID produces a sortable base36 ULID-style id. +// +// Format: <8 base36 chars of ms timestamp><6 base36 chars of randomness>. +func GenerateBuildID() string { + const tsLen = 8 + const randLen = 6 + + ms := uint64(time.Now().UTC().UnixNano() / int64(time.Millisecond)) + tsBytes := make([]byte, 8) + for i := 7; i >= 0; i-- { + tsBytes[i] = byte(ms & 0xff) + ms >>= 8 + } + + tsEncoded := base36.EncodeToStringLc(tsBytes) + if len(tsEncoded) > tsLen { + tsEncoded = tsEncoded[len(tsEncoded)-tsLen:] + } + for len(tsEncoded) < tsLen { + tsEncoded = "0" + tsEncoded + } + + randBytes := make([]byte, 6) + if _, err := rand.Read(randBytes); err != nil { + // fall back to time-derived noise; uniqueness is degraded but never zero + now := time.Now().UTC().UnixNano() + for i := 0; i < len(randBytes); i++ { + randBytes[i] = byte(now >> (8 * i)) + } + } + randEncoded := base36.EncodeToStringLc(randBytes) + if len(randEncoded) > randLen { + randEncoded = randEncoded[len(randEncoded)-randLen:] + } + for len(randEncoded) < randLen { + randEncoded = "0" + randEncoded + } + + return tsEncoded + randEncoded +} + +// AppDataDir returns the directory that stores build records and logs for app. +func AppDataDir(appName string) string { + return common.GetAppDataDirectory("builds", appName) +} + +// RecordPath returns the on-disk path to the JSON record for a given build. +func RecordPath(appName, buildID string) string { + return filepath.Join(AppDataDir(appName), buildID+".json") +} + +// LogPathFor returns the on-disk path to the log file for a given build. +func LogPathFor(appName, buildID string) string { + return filepath.Join(AppDataDir(appName), buildID+".log") +} + +// WriteBuild writes a Build record to disk. It creates the data dir as needed. +func WriteBuild(b Build) error { + if err := os.MkdirAll(AppDataDir(b.App), 0755); err != nil { + return fmt.Errorf("create builds data dir: %w", err) + } + + body, err := json.MarshalIndent(b, "", " ") + if err != nil { + return fmt.Errorf("marshal build record: %w", err) + } + + tmp := RecordPath(b.App, b.ID) + ".tmp" + if err := os.WriteFile(tmp, body, 0644); err != nil { + return fmt.Errorf("write temp build record: %w", err) + } + if err := os.Rename(tmp, RecordPath(b.App, b.ID)); err != nil { + return fmt.Errorf("rename build record into place: %w", err) + } + return nil +} + +// ReadBuild loads the Build record from disk. Returns os.ErrNotExist when the +// record does not exist. +func ReadBuild(appName, buildID string) (Build, error) { + var b Build + body, err := os.ReadFile(RecordPath(appName, buildID)) + if err != nil { + return b, err + } + if err := json.Unmarshal(body, &b); err != nil { + return b, fmt.Errorf("parse build record %s: %w", buildID, err) + } + return b, nil +} + +// FetchBuilds returns every recorded build for an app, sorted newest-first by +// StartedAt. It is read-only; callers that want to reap abandoned records +// should call ReapAbandonedBuilds explicitly via PruneAppBuilds or the +// builds:prune subcommand. +func FetchBuilds(appName string) ([]Build, error) { + dir := AppDataDir(appName) + entries, err := os.ReadDir(dir) + if err != nil { + if os.IsNotExist(err) { + return nil, nil + } + return nil, fmt.Errorf("list builds dir for %s: %w", appName, err) + } + + builds := make([]Build, 0, len(entries)) + for _, entry := range entries { + if entry.IsDir() { + continue + } + name := entry.Name() + if !strings.HasSuffix(name, ".json") { + continue + } + buildID := strings.TrimSuffix(name, ".json") + b, err := ReadBuild(appName, buildID) + if err != nil { + common.LogWarn(fmt.Sprintf("Skipping unreadable build record %s/%s: %s", appName, name, err)) + continue + } + builds = append(builds, b) + } + + sort.Slice(builds, func(i, j int) bool { + return builds[i].StartedAt.After(builds[j].StartedAt) + }) + return builds, nil +} + +// FetchRunningBuilds returns the in-flight (display-status running) builds for +// an app. Records whose on-disk status is running but PID is dead are excluded. +func FetchRunningBuilds(appName string) ([]Build, error) { + all, err := FetchBuilds(appName) + if err != nil { + return nil, err + } + + running := make([]Build, 0, len(all)) + for _, b := range all { + if b.Status == BuildStatusRunning && CheckPIDAlive(b.PID) { + running = append(running, b) + } + } + return running, nil +} + +// CheckPIDAlive reports whether a process with the given pid currently exists. +// On unix, kill(pid, 0) returns nil if the process exists (or ESRCH if it does +// not). It treats invalid pids (<= 0) as dead. +func CheckPIDAlive(pid int) bool { + if pid <= 0 { + return false + } + if err := syscall.Kill(pid, 0); err != nil { + if err == syscall.ESRCH { + return false + } + // EPERM means the process exists but we don't have permission to + // signal it - still alive from the operator's perspective. + if err == syscall.EPERM { + return true + } + return false + } + return true +} + +// ResolveRetention returns the retention count for an app, cascading from +// per-app override → global override → DefaultRetention. Invalid (non-int or +// non-positive) property values fall through to the next level with a warning. +func ResolveRetention(appName string) int { + if appName != "" && appName != "--global" { + if n, ok := parseRetention(common.PropertyGet("builds", appName, "retention"), appName); ok { + return n + } + } + if n, ok := parseRetention(common.PropertyGet("builds", "--global", "retention"), "--global"); ok { + return n + } + return DefaultRetention +} + +func parseRetention(raw string, scope string) (int, bool) { + if raw == "" { + return 0, false + } + n, err := strconv.Atoi(raw) + if err != nil { + common.LogWarn(fmt.Sprintf("Ignoring non-integer builds retention %q for %s", raw, scope)) + return 0, false + } + if n < MinimumRetention { + common.LogWarn(fmt.Sprintf("Ignoring builds retention %d below minimum %d for %s", n, MinimumRetention, scope)) + return 0, false + } + return n, true +} + +// ReapAbandonedBuilds finalizes any record whose on-disk status is running but +// whose PID is no longer alive. Records are written with status=failed, +// exit_code=-1, finished_at=now. Returns the number of records reaped. +func ReapAbandonedBuilds(appName string) (int, error) { + builds, err := FetchBuilds(appName) + if err != nil { + return 0, err + } + + now := time.Now().UTC() + exitCode := -1 + reaped := 0 + for _, b := range builds { + if b.Status != BuildStatusRunning { + continue + } + if CheckPIDAlive(b.PID) { + continue + } + b.Status = BuildStatusFailed + b.FinishedAt = &now + b.ExitCode = &exitCode + if err := WriteBuild(b); err != nil { + common.LogWarn(fmt.Sprintf("Could not reap abandoned build %s/%s: %s", appName, b.ID, err)) + continue + } + reaped++ + } + return reaped, nil +} + +// PruneAppBuilds reaps abandoned records, then prunes finalized records beyond +// the retention cap. Live in-flight builds (status=running with alive PID) are +// always preserved. +func PruneAppBuilds(appName string) error { + if _, err := ReapAbandonedBuilds(appName); err != nil { + return err + } + + builds, err := FetchBuilds(appName) + if err != nil { + return err + } + + retention := ResolveRetention(appName) + + finalized := make([]Build, 0, len(builds)) + for _, b := range builds { + if b.Status == BuildStatusRunning && CheckPIDAlive(b.PID) { + continue + } + finalized = append(finalized, b) + } + + if len(finalized) <= retention { + return nil + } + + for _, b := range finalized[retention:] { + removeBuildFiles(appName, b.ID) + } + return nil +} + +func removeBuildFiles(appName, buildID string) { + if err := os.Remove(RecordPath(appName, buildID)); err != nil && !os.IsNotExist(err) { + common.LogWarn(fmt.Sprintf("Could not remove build record %s/%s: %s", appName, buildID, err)) + } + if err := os.Remove(LogPathFor(appName, buildID)); err != nil && !os.IsNotExist(err) { + common.LogWarn(fmt.Sprintf("Could not remove build log %s/%s.log: %s", appName, buildID, err)) + } +} diff --git a/plugins/builds/builds_test.go b/plugins/builds/builds_test.go new file mode 100644 index 000000000..235a1315c --- /dev/null +++ b/plugins/builds/builds_test.go @@ -0,0 +1,611 @@ +package builds + +import ( + "encoding/json" + "os" + "path/filepath" + "strconv" + "strings" + "testing" + "time" +) + +func setupTestRoot(t *testing.T) string { + t.Helper() + tmpDir, err := os.MkdirTemp("", "builds-test") + if err != nil { + t.Fatalf("mkdirtemp: %v", err) + } + t.Setenv("DOKKU_LIB_ROOT", tmpDir) + t.Setenv("PLUGIN_PATH", filepath.Join(tmpDir, "plugins")) + t.Setenv("PLUGIN_ENABLED_PATH", filepath.Join(tmpDir, "plugins", "enabled")) + t.Cleanup(func() { os.RemoveAll(tmpDir) }) + return tmpDir +} + +func TestBuildKindValid(t *testing.T) { + cases := map[BuildKind]bool{ + BuildKindBuild: true, + BuildKindDeploy: true, + BuildKind(""): false, + BuildKind("nope"): false, + } + for k, want := range cases { + if got := k.Valid(); got != want { + t.Errorf("BuildKind(%q).Valid() = %v, want %v", string(k), got, want) + } + } +} + +func TestBuildStatusValid(t *testing.T) { + cases := map[BuildStatus]bool{ + BuildStatusRunning: true, + BuildStatusSucceeded: true, + BuildStatusFailed: true, + BuildStatusCanceled: true, + BuildStatusAbandoned: false, // intentionally not persistable + BuildStatus(""): false, + BuildStatus("nope"): false, + } + for s, want := range cases { + if got := s.Valid(); got != want { + t.Errorf("BuildStatus(%q).Valid() = %v, want %v", string(s), got, want) + } + } +} + +func TestBuildStatusIsTerminal(t *testing.T) { + cases := map[BuildStatus]bool{ + BuildStatusRunning: false, + BuildStatusSucceeded: true, + BuildStatusFailed: true, + BuildStatusCanceled: true, + BuildStatusAbandoned: false, + } + for s, want := range cases { + if got := s.IsTerminal(); got != want { + t.Errorf("BuildStatus(%q).IsTerminal() = %v, want %v", string(s), got, want) + } + } +} + +func TestBuildSourceValid(t *testing.T) { + for _, src := range allBuildSources { + if !src.Valid() { + t.Errorf("expected %q to be valid", string(src)) + } + } + for _, bad := range []BuildSource{"", "garbage", "git:", "ps:something"} { + if bad.Valid() { + t.Errorf("expected %q to be invalid", string(bad)) + } + } +} + +func TestBuildSourceDefaultKind(t *testing.T) { + build := []BuildSource{ + BuildSourceGitHook, BuildSourcePsRebuild, BuildSourceGitSync, + BuildSourceGitFromArchive, BuildSourceGitFromImage, BuildSourceGitLoadImage, + } + deploy := []BuildSource{ + BuildSourcePsRestart, BuildSourcePsStart, BuildSourceDeploy, + BuildSourceConfigRedeploy, BuildSourceUnknown, + } + for _, s := range build { + if got := s.DefaultKind(); got != BuildKindBuild { + t.Errorf("%q.DefaultKind() = %v, want build", string(s), got) + } + } + for _, s := range deploy { + if got := s.DefaultKind(); got != BuildKindDeploy { + t.Errorf("%q.DefaultKind() = %v, want deploy", string(s), got) + } + } +} + +func TestGenerateBuildIDUniquenessAndShape(t *testing.T) { + const samples = 200 + seen := make(map[string]struct{}, samples) + for i := 0; i < samples; i++ { + id := GenerateBuildID() + if len(id) != 14 { + t.Fatalf("expected len 14, got %d for %q", len(id), id) + } + for _, r := range id { + if !((r >= '0' && r <= '9') || (r >= 'a' && r <= 'z')) { + t.Fatalf("non-base36 char %q in %q", r, id) + } + } + if _, dup := seen[id]; dup { + t.Fatalf("duplicate id %q in %d samples", id, samples) + } + seen[id] = struct{}{} + } +} + +func TestWriteAndReadBuildRoundTrip(t *testing.T) { + setupTestRoot(t) + + now := time.Now().UTC().Truncate(time.Second) + exit := 0 + finished := now.Add(time.Minute) + want := Build{ + ID: GenerateBuildID(), + App: "myapp", + Kind: BuildKindBuild, + PID: 12345, + StartedAt: now, + FinishedAt: &finished, + Status: BuildStatusSucceeded, + Source: BuildSourceGitHook, + ExitCode: &exit, + } + + if err := WriteBuild(want); err != nil { + t.Fatalf("WriteBuild: %v", err) + } + + got, err := ReadBuild(want.App, want.ID) + if err != nil { + t.Fatalf("ReadBuild: %v", err) + } + + if got.ID != want.ID || got.Status != want.Status || got.Kind != want.Kind || got.Source != want.Source { + t.Errorf("round-trip mismatch: got %+v want %+v", got, want) + } + if got.ExitCode == nil || *got.ExitCode != 0 { + t.Errorf("exit code mismatch: got %v want 0", got.ExitCode) + } + if got.FinishedAt == nil || !got.FinishedAt.Equal(finished) { + t.Errorf("finished_at mismatch: got %v want %v", got.FinishedAt, finished) + } +} + +func TestFetchBuildsSortsNewestFirst(t *testing.T) { + setupTestRoot(t) + + base := time.Now().UTC().Truncate(time.Second) + for i, offset := range []time.Duration{2 * time.Hour, 1 * time.Hour, 0} { + b := Build{ + ID: "build-" + strconv.Itoa(i), + App: "myapp", + Kind: BuildKindBuild, + PID: 100 + i, + StartedAt: base.Add(-offset), + Status: BuildStatusSucceeded, + Source: BuildSourceGitHook, + } + if err := WriteBuild(b); err != nil { + t.Fatalf("write %d: %v", i, err) + } + } + + builds, err := FetchBuilds("myapp") + if err != nil { + t.Fatalf("fetch: %v", err) + } + if len(builds) != 3 { + t.Fatalf("got %d builds, want 3", len(builds)) + } + for i := 0; i < len(builds)-1; i++ { + if !builds[i].StartedAt.After(builds[i+1].StartedAt) { + t.Errorf("not sorted descending at %d: %v vs %v", i, builds[i].StartedAt, builds[i+1].StartedAt) + } + } +} + +func TestFetchBuildsMissingDir(t *testing.T) { + setupTestRoot(t) + builds, err := FetchBuilds("never-deployed") + if err != nil { + t.Fatalf("expected nil err, got %v", err) + } + if len(builds) != 0 { + t.Errorf("expected empty slice, got %d entries", len(builds)) + } +} + +func TestCheckPIDAlive(t *testing.T) { + if CheckPIDAlive(os.Getpid()) != true { + t.Error("expected current pid to be alive") + } + if CheckPIDAlive(0) { + t.Error("expected pid 0 to be dead") + } + if CheckPIDAlive(-1) { + t.Error("expected pid -1 to be dead") + } + // PID 1 is init/launchd; it always exists. We don't have permission to + // signal it from a non-root user, but CheckPIDAlive should treat EPERM + // as alive. + if !CheckPIDAlive(1) { + t.Error("expected pid 1 (init) to be reported alive") + } +} + +func TestReapAbandonedBuilds(t *testing.T) { + setupTestRoot(t) + app := "reapapp" + + // 1: alive (this test process), should NOT be reaped. + alive := Build{ + ID: "alive", + App: app, + Kind: BuildKindBuild, + PID: os.Getpid(), + StartedAt: time.Now().UTC(), + Status: BuildStatusRunning, + Source: BuildSourceGitHook, + } + // 2: dead pid, status=running, should be reaped. + dead := Build{ + ID: "dead", + App: app, + Kind: BuildKindBuild, + PID: 1, // signal-eperm but we treat as alive; use a clearly-dead one + StartedAt: time.Now().UTC(), + Status: BuildStatusRunning, + Source: BuildSourceGitHook, + } + // pick a pid that shouldn't exist + dead.PID = pickDeadPID(t) + // 3: terminal, should not be touched. + finished := time.Now().UTC() + exit := 0 + terminal := Build{ + ID: "terminal", + App: app, + Kind: BuildKindBuild, + PID: 9999, + StartedAt: time.Now().UTC().Add(-time.Hour), + FinishedAt: &finished, + Status: BuildStatusSucceeded, + Source: BuildSourceGitHook, + ExitCode: &exit, + } + + for _, b := range []Build{alive, dead, terminal} { + if err := WriteBuild(b); err != nil { + t.Fatalf("write %s: %v", b.ID, err) + } + } + + reaped, err := ReapAbandonedBuilds(app) + if err != nil { + t.Fatalf("reap: %v", err) + } + if reaped != 1 { + t.Errorf("expected 1 reap, got %d", reaped) + } + + got, err := ReadBuild(app, "dead") + if err != nil { + t.Fatalf("read dead: %v", err) + } + if got.Status != BuildStatusFailed { + t.Errorf("dead.Status = %v, want failed", got.Status) + } + if got.ExitCode == nil || *got.ExitCode != -1 { + t.Errorf("dead.ExitCode = %v, want -1", got.ExitCode) + } + + stillAlive, err := ReadBuild(app, "alive") + if err != nil { + t.Fatalf("read alive: %v", err) + } + if stillAlive.Status != BuildStatusRunning { + t.Errorf("alive.Status = %v, want running", stillAlive.Status) + } + + stillTerminal, err := ReadBuild(app, "terminal") + if err != nil { + t.Fatalf("read terminal: %v", err) + } + if stillTerminal.Status != BuildStatusSucceeded { + t.Errorf("terminal.Status = %v, want succeeded", stillTerminal.Status) + } +} + +func pickDeadPID(t *testing.T) int { + t.Helper() + for candidate := 99999; candidate < 200000; candidate++ { + if !CheckPIDAlive(candidate) { + return candidate + } + } + t.Fatal("could not find a dead pid candidate") + return 0 +} + +func TestPruneAppBuildsRespectsRetentionAndProtectsLive(t *testing.T) { + setupTestRoot(t) + app := "pruneapp" + + now := time.Now().UTC() + + // 1 live in-flight build (status=running, alive PID). + live := Build{ + ID: "live", + App: app, + Kind: BuildKindBuild, + PID: os.Getpid(), + StartedAt: now, + Status: BuildStatusRunning, + Source: BuildSourceGitHook, + } + if err := WriteBuild(live); err != nil { + t.Fatalf("write live: %v", err) + } + + // Force retention down to 2 via property. + if err := os.MkdirAll(filepath.Join(os.Getenv("DOKKU_LIB_ROOT"), "config", "builds", app), 0755); err != nil { + t.Fatalf("setup property dir: %v", err) + } + if err := os.WriteFile(filepath.Join(os.Getenv("DOKKU_LIB_ROOT"), "config", "builds", app, "retention"), []byte("2"), 0644); err != nil { + t.Fatalf("write retention property: %v", err) + } + + // 5 finalized builds with descending start times. + exit := 0 + for i := 0; i < 5; i++ { + finished := now.Add(-time.Duration(i+1) * time.Minute) + started := finished.Add(-time.Second) + b := Build{ + ID: "old-" + strconv.Itoa(i), + App: app, + Kind: BuildKindBuild, + PID: 9000 + i, + StartedAt: started, + FinishedAt: &finished, + Status: BuildStatusSucceeded, + Source: BuildSourceGitHook, + ExitCode: &exit, + } + if err := WriteBuild(b); err != nil { + t.Fatalf("write old-%d: %v", i, err) + } + // Touch a log file so the prune deletes that too. + _ = os.WriteFile(b.LogPath(), []byte("log"), 0644) + } + + if err := PruneAppBuilds(app); err != nil { + t.Fatalf("prune: %v", err) + } + + remaining, err := FetchBuilds(app) + if err != nil { + t.Fatalf("fetch after prune: %v", err) + } + + // live + 2 retained = 3 + if len(remaining) != 3 { + t.Fatalf("expected 3 records after prune, got %d (%v)", len(remaining), remainingIDs(remaining)) + } + + // live record must survive + foundLive := false + for _, r := range remaining { + if r.ID == "live" { + foundLive = true + if r.Status != BuildStatusRunning { + t.Errorf("live.Status changed to %v", r.Status) + } + } + } + if !foundLive { + t.Errorf("live record was pruned (remaining=%v)", remainingIDs(remaining)) + } + + // the two newest old-* records (old-0 and old-1) should remain + expectedKept := map[string]bool{"old-0": true, "old-1": true} + for _, r := range remaining { + if r.ID == "live" { + continue + } + if !expectedKept[r.ID] { + t.Errorf("unexpected surviving record %s", r.ID) + } + } + + // log files for pruned old-* should be gone; for kept ones, still present. + for i := 2; i < 5; i++ { + path := filepath.Join(AppDataDir(app), "old-"+strconv.Itoa(i)+".log") + if _, err := os.Stat(path); !os.IsNotExist(err) { + t.Errorf("expected log %s removed, got err=%v", path, err) + } + } +} + +func remainingIDs(builds []Build) []string { + ids := make([]string, 0, len(builds)) + for _, b := range builds { + ids = append(ids, b.ID) + } + return ids +} + +func TestResolveRetentionCascade(t *testing.T) { + tmp := setupTestRoot(t) + app := "retapp" + + // no overrides: default + if got := ResolveRetention(app); got != DefaultRetention { + t.Errorf("no overrides: got %d want %d", got, DefaultRetention) + } + + // global override + globalDir := filepath.Join(tmp, "config", "builds", "--global") + if err := os.MkdirAll(globalDir, 0755); err != nil { + t.Fatalf("mkdir global: %v", err) + } + if err := os.WriteFile(filepath.Join(globalDir, "retention"), []byte("7"), 0644); err != nil { + t.Fatalf("write global retention: %v", err) + } + if got := ResolveRetention(app); got != 7 { + t.Errorf("global override: got %d want 7", got) + } + + // per-app override beats global + appDir := filepath.Join(tmp, "config", "builds", app) + if err := os.MkdirAll(appDir, 0755); err != nil { + t.Fatalf("mkdir app: %v", err) + } + if err := os.WriteFile(filepath.Join(appDir, "retention"), []byte("3"), 0644); err != nil { + t.Fatalf("write app retention: %v", err) + } + if got := ResolveRetention(app); got != 3 { + t.Errorf("per-app override: got %d want 3", got) + } + + // invalid app value falls back to global + if err := os.WriteFile(filepath.Join(appDir, "retention"), []byte("not-a-number"), 0644); err != nil { + t.Fatalf("write invalid: %v", err) + } + if got := ResolveRetention(app); got != 7 { + t.Errorf("invalid app value: got %d want 7 (global)", got) + } + + // negative value also falls through + if err := os.WriteFile(filepath.Join(appDir, "retention"), []byte("0"), 0644); err != nil { + t.Fatalf("write zero: %v", err) + } + if got := ResolveRetention(app); got != 7 { + t.Errorf("zero app value: got %d want 7 (global)", got) + } +} + +func TestDisplayStatusComputesAbandoned(t *testing.T) { + setupTestRoot(t) + abandoned := Build{ + ID: "x", + App: "myapp", + Kind: BuildKindBuild, + PID: pickDeadPID(t), + StartedAt: time.Now().UTC(), + Status: BuildStatusRunning, + Source: BuildSourceGitHook, + } + if got := abandoned.DisplayStatus(); got != BuildStatusAbandoned { + t.Errorf("DisplayStatus on dead-PID running record = %v, want abandoned", got) + } + + live := abandoned + live.PID = os.Getpid() + if got := live.DisplayStatus(); got != BuildStatusRunning { + t.Errorf("DisplayStatus on alive-PID running record = %v, want running", got) + } + + exit := 0 + finished := time.Now().UTC() + terminal := Build{ + ID: "y", + App: "myapp", + Kind: BuildKindBuild, + PID: os.Getpid(), + StartedAt: time.Now().UTC(), + FinishedAt: &finished, + Status: BuildStatusSucceeded, + Source: BuildSourceGitHook, + ExitCode: &exit, + } + if got := terminal.DisplayStatus(); got != BuildStatusSucceeded { + t.Errorf("terminal DisplayStatus = %v, want succeeded", got) + } +} + +func TestRecordStartFinalizeIdempotency(t *testing.T) { + setupTestRoot(t) + app := "idem" + id := GenerateBuildID() + + if err := TriggerBuildsRecordStart(app, id, strconv.Itoa(os.Getpid()), string(BuildSourceGitHook)); err != nil { + t.Fatalf("record-start: %v", err) + } + + // First finalize: succeeded + if err := TriggerBuildsRecordFinalize(app, id, "0"); err != nil { + t.Fatalf("first finalize: %v", err) + } + first, err := ReadBuild(app, id) + if err != nil { + t.Fatalf("read after first finalize: %v", err) + } + if first.Status != BuildStatusSucceeded { + t.Errorf("first finalize Status = %v, want succeeded", first.Status) + } + + // Manually mark canceled, then call finalize with non-zero exit; the + // canceled status should win because cancel finalized it first. + canceled := first + canceled.Status = BuildStatusCanceled + if err := WriteBuild(canceled); err != nil { + t.Fatalf("write canceled: %v", err) + } + if err := TriggerBuildsRecordFinalize(app, id, "1"); err != nil { + t.Fatalf("second finalize: %v", err) + } + final, err := ReadBuild(app, id) + if err != nil { + t.Fatalf("read after second finalize: %v", err) + } + if final.Status != BuildStatusCanceled { + t.Errorf("idempotent finalize Status = %v, want canceled", final.Status) + } +} + +func TestRecordStartUnknownSourceCoercedToUnknown(t *testing.T) { + setupTestRoot(t) + app := "src" + id := GenerateBuildID() + if err := TriggerBuildsRecordStart(app, id, strconv.Itoa(os.Getpid()), "made-up-source"); err != nil { + t.Fatalf("record-start: %v", err) + } + got, err := ReadBuild(app, id) + if err != nil { + t.Fatalf("read: %v", err) + } + if got.Source != BuildSourceUnknown { + t.Errorf("Source = %v, want unknown", got.Source) + } + if got.Kind != BuildKindDeploy { + t.Errorf("Kind = %v, want deploy (DefaultKind for unknown)", got.Kind) + } +} + +func TestBuildJSONShape(t *testing.T) { + setupTestRoot(t) + now := time.Now().UTC().Truncate(time.Second) + b := Build{ + ID: "abc", + App: "j", + Kind: BuildKindDeploy, + PID: 100, + StartedAt: now, + Status: BuildStatusRunning, + Source: BuildSourcePsRestart, + } + if err := WriteBuild(b); err != nil { + t.Fatalf("write: %v", err) + } + raw, err := os.ReadFile(RecordPath(b.App, b.ID)) + if err != nil { + t.Fatalf("read raw: %v", err) + } + var parsed map[string]any + if err := json.Unmarshal(raw, &parsed); err != nil { + t.Fatalf("unmarshal: %v", err) + } + for _, key := range []string{"id", "app", "kind", "pid", "started_at", "status", "source"} { + if _, ok := parsed[key]; !ok { + t.Errorf("expected key %q in JSON, got %s", key, string(raw)) + } + } + for _, omittable := range []string{"finished_at", "exit_code"} { + if _, ok := parsed[omittable]; ok { + t.Errorf("expected key %q to be omitted while running, got %s", omittable, string(raw)) + } + } + if !strings.Contains(string(raw), `"kind": "deploy"`) { + t.Errorf("kind not serialized as enum string: %s", string(raw)) + } +} diff --git a/plugins/builds/commands b/plugins/builds/commands deleted file mode 100755 index bbb121222..000000000 --- a/plugins/builds/commands +++ /dev/null @@ -1,16 +0,0 @@ -#!/usr/bin/env bash -[[ " help builds:help " == *" $1 "* ]] || exit "$DOKKU_NOT_IMPLEMENTED_EXIT" -source "$PLUGIN_AVAILABLE_PATH/builds/internal-functions" -set -eo pipefail -[[ $DOKKU_TRACE ]] && set -x - -case "$1" in - help | builds:help) - cmd-builds-help "$@" - ;; - - *) - exit "$DOKKU_NOT_IMPLEMENTED_EXIT" - ;; - -esac diff --git a/plugins/builds/go.mod b/plugins/builds/go.mod new file mode 100644 index 000000000..c8e11b445 --- /dev/null +++ b/plugins/builds/go.mod @@ -0,0 +1,30 @@ +module github.com/dokku/dokku/plugins/builds + +go 1.26.2 + +require ( + github.com/dokku/dokku/plugins/common v0.0.0-00010101000000-000000000000 + github.com/multiformats/go-base36 v0.2.0 + github.com/ryanuber/columnize v2.1.2+incompatible + github.com/spf13/pflag v1.0.10 +) + +require ( + github.com/alexellis/go-execute/v2 v2.2.1 // indirect + github.com/fatih/color v1.19.0 // indirect + github.com/hashicorp/errwrap v1.0.0 // indirect + github.com/hashicorp/go-multierror v1.1.1 // indirect + github.com/kr/fs v0.1.0 // indirect + github.com/mattn/go-colorable v0.1.14 // indirect + github.com/mattn/go-isatty v0.0.20 // indirect + github.com/melbahja/goph v1.5.0 // indirect + github.com/otiai10/copy v1.14.1 // indirect + github.com/otiai10/mint v1.6.3 // indirect + github.com/pkg/errors v0.9.1 // indirect + github.com/pkg/sftp v1.13.10 // indirect + golang.org/x/crypto v0.50.0 // indirect + golang.org/x/sync v0.20.0 // indirect + golang.org/x/sys v0.43.0 // indirect +) + +replace github.com/dokku/dokku/plugins/common => ../common diff --git a/plugins/builds/go.sum b/plugins/builds/go.sum new file mode 100644 index 000000000..37855810c --- /dev/null +++ b/plugins/builds/go.sum @@ -0,0 +1,57 @@ +github.com/alexellis/go-execute/v2 v2.2.1 h1:4Ye3jiCKQarstODOEmqDSRCqxMHLkC92Bhse743RdOI= +github.com/alexellis/go-execute/v2 v2.2.1/go.mod h1:FMdRnUTiFAmYXcv23txrp3VYZfLo24nMpiIneWgKHTQ= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/fatih/color v1.19.0 h1:Zp3PiM21/9Ld6FzSKyL5c/BULoe/ONr9KlbYVOfG8+w= +github.com/fatih/color v1.19.0/go.mod h1:zNk67I0ZUT1bEGsSGyCZYZNrHuTkJJB+r6Q9VuMi0LE= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA= +github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= +github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo= +github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM= +github.com/kr/fs v0.1.0 h1:Jskdu9ieNAYnjxsi0LbQp1ulIKZV1LAFgK1tWhpZgl8= +github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg= +github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE= +github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8= +github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= +github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= +github.com/melbahja/goph v1.5.0 h1:RQUBpLvfg3i7fjfG8rTcSWyMjVRfdhwrrfQhjYee4dQ= +github.com/melbahja/goph v1.5.0/go.mod h1:dDwo+44cmvfDLdiVpc6fJxexf5BA5yEDUeE5YgtuDO4= +github.com/multiformats/go-base36 v0.2.0 h1:lFsAbNOGeKtuKozrtBsAkSVhv1p9D0/qedU9rQyccr0= +github.com/multiformats/go-base36 v0.2.0/go.mod h1:qvnKE++v+2MWCfePClUEjE78Z7P2a1UV0xHgWc0hkp4= +github.com/onsi/gomega v1.40.0 h1:Vtol0e1MghCD2ZVIilPDIg44XSL9l2QAn8ZNaljWcJc= +github.com/onsi/gomega v1.40.0/go.mod h1:M/Uqpu/8qTjtzCLUA2zJHX9Iilrau25x1PdoSRbWh5A= +github.com/otiai10/copy v1.14.1 h1:5/7E6qsUMBaH5AnQ0sSLzzTg1oTECmcCmT6lvF45Na8= +github.com/otiai10/copy v1.14.1/go.mod h1:oQwrEDDOci3IM8dJF0d8+jnbfPDllW6vUjNc3DoZm9I= +github.com/otiai10/mint v1.6.3 h1:87qsV/aw1F5as1eH1zS/yqHY85ANKVMgkDrf9rcxbQs= +github.com/otiai10/mint v1.6.3/go.mod h1:MJm72SBthJjz8qhefc4z1PYEieWmy8Bku7CjcAqyUSM= +github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= +github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pkg/sftp v1.13.10 h1:+5FbKNTe5Z9aspU88DPIKJ9z2KZoaGCu6Sr6kKR/5mU= +github.com/pkg/sftp v1.13.10/go.mod h1:bJ1a7uDhrX/4OII+agvy28lzRvQrmIQuaHrcI1HbeGA= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/ryanuber/columnize v2.1.2+incompatible h1:C89EOx/XBWwIXl8wm8OPJBd7kPF25UfsK2X7Ph/zCAk= +github.com/ryanuber/columnize v2.1.2+incompatible/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts= +github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk= +github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA= +github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc= +go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= +golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI= +golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q= +golang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0= +golang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw= +golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4= +golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= +golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI= +golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= +golang.org/x/term v0.42.0 h1:UiKe+zDFmJobeJ5ggPwOshJIVt6/Ft0rcfrXZDLWAWY= +golang.org/x/term v0.42.0/go.mod h1:Dq/D+snpsbazcBG5+F9Q1n2rXV8Ma+71xEjTRufARgY= +golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg= +golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/plugins/builds/internal-functions b/plugins/builds/internal-functions deleted file mode 100755 index 1b56fad60..000000000 --- a/plugins/builds/internal-functions +++ /dev/null @@ -1,31 +0,0 @@ -#!/usr/bin/env bash -set -eo pipefail -[[ $DOKKU_TRACE ]] && set -x - -fn-builds-help-content() { - declare desc="return logs plugin help content" - cat <, Cancel a running build for an app - builds:list , List all running builds - builds:output , Shows build output - builds:report [] [], Displays a build report for one or more apps -help_content -} - -cmd-builds-help() { - if [[ $1 == "builds:help" ]]; then - echo -e 'Usage: dokku builds[:COMMAND]' - echo '' - echo 'Manage running builds' - echo '' - echo 'Additional commands:' - fn-builds-help-content | sort | column -c2 -t -s, - elif [[ $(ps -o command= $PPID) == *"--all"* ]]; then - fn-builds-help-content - else - cat <= %d", n, MinimumRetention) + } + } + return nil +} diff --git a/plugins/builds/src/commands/commands.go b/plugins/builds/src/commands/commands.go new file mode 100644 index 000000000..321d14a64 --- /dev/null +++ b/plugins/builds/src/commands/commands.go @@ -0,0 +1,60 @@ +package main + +import ( + "flag" + "fmt" + "os" + "strconv" + "strings" + + "github.com/dokku/dokku/plugins/common" +) + +const ( + helpHeader = `Usage: dokku builds[:COMMAND] + +Manage running and historical builds + +Additional commands:` + + helpContent = ` + builds:cancel , Cancel a running build for an app + builds:info [--format json|stdout], Show details for a single build + builds:list [] [--format json] [--kind build|deploy] [--status ], List builds + builds:output [|current], Show build output + builds:prune [--all-apps], Prune build records to retention + builds:report [] [], Display a build report for one or more apps + builds:set [--global|] [], Set or clear a builds property` +) + +func main() { + flag.Usage = usage + flag.Parse() + + cmd := flag.Arg(0) + switch cmd { + case "builds", "builds:help": + usage() + case "help": + result, err := common.CallExecCommand(common.ExecCommandInput{ + Command: "ps", + Args: []string{"-o", "command=", strconv.Itoa(os.Getppid())}, + }) + if err == nil && strings.Contains(result.StdoutContents(), "--all") { + fmt.Println(helpContent) + } else { + fmt.Print("\n builds, Manage running and historical builds\n") + } + default: + dokkuNotImplementExitCode, err := strconv.Atoi(os.Getenv("DOKKU_NOT_IMPLEMENTED_EXIT")) + if err != nil { + fmt.Println("failed to retrieve DOKKU_NOT_IMPLEMENTED_EXIT environment variable") + dokkuNotImplementExitCode = 10 + } + os.Exit(dokkuNotImplementExitCode) + } +} + +func usage() { + common.CommandUsage(helpHeader, helpContent) +} diff --git a/plugins/builds/src/subcommands/subcommands.go b/plugins/builds/src/subcommands/subcommands.go new file mode 100644 index 000000000..cb9f68a74 --- /dev/null +++ b/plugins/builds/src/subcommands/subcommands.go @@ -0,0 +1,77 @@ +package main + +import ( + "fmt" + "os" + "strings" + + "github.com/dokku/dokku/plugins/builds" + "github.com/dokku/dokku/plugins/common" + + flag "github.com/spf13/pflag" +) + +func main() { + parts := strings.Split(os.Args[0], "/") + subcommand := parts[len(parts)-1] + + var err error + switch subcommand { + case "cancel": + args := flag.NewFlagSet("builds:cancel", flag.ExitOnError) + args.Parse(os.Args[2:]) + err = builds.CommandCancel(args.Arg(0)) + case "info": + args := flag.NewFlagSet("builds:info", flag.ExitOnError) + format := args.String("format", "stdout", "format: [ stdout | json ]") + args.Parse(os.Args[2:]) + err = builds.CommandInfo(args.Arg(0), args.Arg(1), *format) + case "list": + args := flag.NewFlagSet("builds:list", flag.ExitOnError) + format := args.String("format", "stdout", "format: [ stdout | json ]") + kind := args.String("kind", "", "filter by kind: [ build | deploy ]") + status := args.String("status", "", "filter by status: [ running | succeeded | failed | canceled | abandoned ]") + args.Parse(os.Args[2:]) + err = builds.CommandList(args.Arg(0), *format, *kind, *status) + case "output": + args := flag.NewFlagSet("builds:output", flag.ExitOnError) + args.Parse(os.Args[2:]) + err = builds.CommandOutput(args.Arg(0), args.Arg(1)) + case "prune": + args := flag.NewFlagSet("builds:prune", flag.ExitOnError) + allApps := args.Bool("all-apps", false, "--all-apps: prune every app") + args.Parse(os.Args[2:]) + err = builds.CommandPrune(args.Arg(0), *allApps) + case "report": + args := flag.NewFlagSet("builds:report", flag.ExitOnError) + format := args.String("format", "stdout", "format: [ stdout | json ]") + reportArgs, flagErr := common.ParseReportArgs("builds", os.Args[2:]) + if flagErr == nil { + args.Parse(reportArgs.OSArgs) + appName := args.Arg(0) + if reportArgs.IsGlobal { + appName = "--global" + } + err = builds.CommandReport(appName, *format, reportArgs.InfoFlag) + } + case "set": + args := flag.NewFlagSet("builds:set", flag.ExitOnError) + global := args.Bool("global", false, "--global: set a global property") + args.Parse(os.Args[2:]) + appName := args.Arg(0) + property := args.Arg(1) + value := args.Arg(2) + if *global { + appName = "--global" + property = args.Arg(0) + value = args.Arg(1) + } + err = builds.CommandSet(appName, property, value) + default: + err = fmt.Errorf("Invalid plugin subcommand call: %s", subcommand) + } + + if err != nil { + common.LogFailWithError(err) + } +} diff --git a/plugins/builds/src/triggers/triggers.go b/plugins/builds/src/triggers/triggers.go new file mode 100644 index 000000000..9f283bd4e --- /dev/null +++ b/plugins/builds/src/triggers/triggers.go @@ -0,0 +1,49 @@ +package main + +import ( + "flag" + "fmt" + "os" + "strings" + + "github.com/dokku/dokku/plugins/builds" + "github.com/dokku/dokku/plugins/common" +) + +func main() { + parts := strings.Split(os.Args[0], "/") + trigger := parts[len(parts)-1] + flag.Parse() + + var err error + switch trigger { + case "builds-generate-id": + err = builds.TriggerBuildsGenerateID() + case "builds-record-finalize": + appName := flag.Arg(0) + buildID := flag.Arg(1) + exitStr := flag.Arg(2) + err = builds.TriggerBuildsRecordFinalize(appName, buildID, exitStr) + case "builds-record-start": + appName := flag.Arg(0) + buildID := flag.Arg(1) + pid := flag.Arg(2) + source := flag.Arg(3) + err = builds.TriggerBuildsRecordStart(appName, buildID, pid, source) + case "install": + err = builds.TriggerInstall() + case "post-app-rename-setup": + oldAppName := flag.Arg(0) + newAppName := flag.Arg(1) + err = builds.TriggerPostAppRenameSetup(oldAppName, newAppName) + case "post-delete": + appName := flag.Arg(0) + err = builds.TriggerPostDelete(appName) + default: + err = fmt.Errorf("Invalid plugin trigger call: %s", trigger) + } + + if err != nil { + common.LogFailWithError(err) + } +} diff --git a/plugins/builds/subcommands.go b/plugins/builds/subcommands.go new file mode 100644 index 000000000..390d0d97d --- /dev/null +++ b/plugins/builds/subcommands.go @@ -0,0 +1,458 @@ +package builds + +import ( + "encoding/json" + "errors" + "fmt" + "os" + "os/exec" + "path/filepath" + "sort" + "strings" + "syscall" + "time" + + "github.com/dokku/dokku/plugins/common" + + "github.com/ryanuber/columnize" +) + +// listView holds a Build plus its computed display status, used by +// builds:list / builds:info JSON output. +type listView struct { + Build + DisplayStatus BuildStatus `json:"display_status"` + Duration string `json:"duration"` + LogPath string `json:"log_path"` +} + +func toListView(b Build) listView { + return listView{ + Build: b, + DisplayStatus: b.DisplayStatus(), + Duration: b.Duration().String(), + LogPath: b.LogPath(), + } +} + +// CommandList lists running builds across apps (no app arg) or running + recent +// records for one app. +func CommandList(appName string, format string, kindFilter string, statusFilter string) error { + if format == "" { + format = "stdout" + } + if format != "stdout" && format != "json" { + return fmt.Errorf("Invalid format specified, supported formats: json, stdout") + } + + if kindFilter != "" { + k := BuildKind(kindFilter) + if !k.Valid() { + return fmt.Errorf("Invalid --kind value %q (allowed: build, deploy)", kindFilter) + } + } + if statusFilter != "" { + if !validStatusFilter(BuildStatus(statusFilter)) { + return fmt.Errorf("Invalid --status value %q (allowed: running, succeeded, failed, canceled, abandoned)", statusFilter) + } + } + + if appName == "" { + return commandListAllRunning(format, kindFilter, statusFilter) + } + + if err := common.VerifyAppName(appName); err != nil { + return err + } + + all, err := FetchBuilds(appName) + if err != nil { + return err + } + + retention := ResolveRetention(appName) + views := make([]listView, 0, len(all)) + for _, b := range all { + v := toListView(b) + if !matchesKind(b, kindFilter) { + continue + } + if !matchesStatus(v, statusFilter) { + continue + } + views = append(views, v) + } + + sort.SliceStable(views, func(i, j int) bool { + // Live-running first, then by start time descending. + iLive := views[i].Build.Status == BuildStatusRunning && views[i].DisplayStatus == BuildStatusRunning + jLive := views[j].Build.Status == BuildStatusRunning && views[j].DisplayStatus == BuildStatusRunning + if iLive != jLive { + return iLive + } + return views[i].StartedAt.After(views[j].StartedAt) + }) + + if statusFilter == "" && kindFilter == "" && len(views) > retention { + live := 0 + for _, v := range views { + if v.DisplayStatus == BuildStatusRunning { + live++ + } + } + cap := live + retention + if cap < len(views) { + views = views[:cap] + } + } + + if format == "json" { + body, err := json.Marshal(views) + if err != nil { + return err + } + common.Log(string(body)) + return nil + } + + if len(views) == 0 { + fmt.Println("No builds recorded for this app") + return nil + } + + rows := []string{"Build ID | Kind | Status | PID | Source | Started | Duration"} + for _, v := range views { + rows = append(rows, fmt.Sprintf("%s | %s | %s | %d | %s | %s | %s", + v.ID, + v.Kind, + v.DisplayStatus, + v.PID, + v.Source, + v.StartedAt.Format(time.RFC3339), + v.Duration, + )) + } + fmt.Println(columnize.SimpleFormat(rows)) + return nil +} + +func commandListAllRunning(format, kindFilter, statusFilter string) error { + apps, err := common.DokkuApps() + if err != nil && !errors.Is(err, common.NoAppsExist) { + return err + } + + views := []listView{} + for _, appName := range apps { + running, err := FetchBuilds(appName) + if err != nil { + common.LogWarn(fmt.Sprintf("Could not read builds for %s: %s", appName, err)) + continue + } + for _, b := range running { + if b.Status != BuildStatusRunning { + continue + } + v := toListView(b) + if !matchesKind(b, kindFilter) { + continue + } + if !matchesStatus(v, statusFilter) { + continue + } + views = append(views, v) + } + } + + sort.SliceStable(views, func(i, j int) bool { + return views[i].StartedAt.After(views[j].StartedAt) + }) + + if format == "json" { + body, err := json.Marshal(views) + if err != nil { + return err + } + common.Log(string(body)) + return nil + } + + if len(views) == 0 { + fmt.Println("No builds currently running") + return nil + } + + rows := []string{"App | Build ID | Kind | PID | Source | Started"} + for _, v := range views { + rows = append(rows, fmt.Sprintf("%s | %s | %s | %d | %s | %s", + v.App, + v.ID, + v.Kind, + v.PID, + v.Source, + v.StartedAt.Format(time.RFC3339), + )) + } + fmt.Println(columnize.SimpleFormat(rows)) + return nil +} + +func matchesKind(b Build, filter string) bool { + if filter == "" { + return true + } + return string(b.Kind) == filter +} + +func matchesStatus(v listView, filter string) bool { + if filter == "" { + return true + } + return string(v.DisplayStatus) == filter +} + +func validStatusFilter(s BuildStatus) bool { + switch s { + case BuildStatusRunning, BuildStatusSucceeded, BuildStatusFailed, BuildStatusCanceled, BuildStatusAbandoned: + return true + } + return false +} + +// CommandInfo prints details for a single build record. +func CommandInfo(appName, buildID, format string) error { + if format == "" { + format = "stdout" + } + if format != "stdout" && format != "json" { + return fmt.Errorf("Invalid format specified, supported formats: json, stdout") + } + if err := common.VerifyAppName(appName); err != nil { + return err + } + if buildID == "" { + return errors.New("Please specify a build id") + } + + b, err := ReadBuild(appName, buildID) + if err != nil { + if os.IsNotExist(err) { + return fmt.Errorf("No build record found for %s/%s", appName, buildID) + } + return err + } + v := toListView(b) + + if format == "json" { + body, err := json.Marshal(v) + if err != nil { + return err + } + common.Log(string(body)) + return nil + } + + rows := []string{ + fmt.Sprintf("Build ID:|%s", v.ID), + fmt.Sprintf("App:|%s", v.App), + fmt.Sprintf("Kind:|%s", v.Kind), + fmt.Sprintf("Status:|%s", v.DisplayStatus), + fmt.Sprintf("PID:|%d", v.PID), + fmt.Sprintf("Source:|%s", v.Source), + fmt.Sprintf("Started:|%s", v.StartedAt.Format(time.RFC3339)), + } + if v.FinishedAt != nil { + rows = append(rows, fmt.Sprintf("Finished:|%s", v.FinishedAt.Format(time.RFC3339))) + } + rows = append(rows, fmt.Sprintf("Duration:|%s", v.Duration)) + if v.ExitCode != nil { + rows = append(rows, fmt.Sprintf("Exit Code:|%d", *v.ExitCode)) + } + rows = append(rows, fmt.Sprintf("Log:|%s", v.LogPath)) + fmt.Println(columnize.Format(rows, &columnize.Config{Delim: "|"})) + return nil +} + +// CommandCancel cancels the in-flight build for an app. +func CommandCancel(appName string) error { + if err := common.VerifyAppName(appName); err != nil { + return err + } + + lockPath := deployLockPath(appName) + body, err := os.ReadFile(lockPath) + if err != nil { + if os.IsNotExist(err) { + common.LogInfo1("App not currently deploying") + return nil + } + return err + } + buildID := strings.TrimSpace(string(body)) + if buildID == "" { + common.LogInfo1("No matching app deploy found") + return nil + } + + b, err := ReadBuild(appName, buildID) + if err != nil { + if os.IsNotExist(err) { + common.LogInfo1(fmt.Sprintf("No build record for %s, removing stale lock file", buildID)) + _ = os.Remove(lockPath) + return nil + } + return err + } + + if b.Status != BuildStatusRunning { + common.LogInfo1(fmt.Sprintf("Build %s is no longer running (status=%s); leaving record untouched", buildID, b.Status)) + return nil + } + + if !CheckPIDAlive(b.PID) { + common.LogInfo1("Build was already terminated, marking failed") + now := time.Now().UTC() + exitCode := -1 + b.Status = BuildStatusFailed + b.FinishedAt = &now + b.ExitCode = &exitCode + if err := WriteBuild(b); err != nil { + return err + } + _ = os.Remove(lockPath) + return nil + } + + common.LogInfo1("Killing app deploy") + if err := killProcessGroup(b.PID); err != nil { + common.LogWarn(fmt.Sprintf("Failed to signal process group %d: %s", b.PID, err)) + } + + current, err := ReadBuild(appName, buildID) + if err == nil && current.Status == BuildStatusRunning { + now := time.Now().UTC() + exitCode := -1 + current.Status = BuildStatusCanceled + current.FinishedAt = &now + current.ExitCode = &exitCode + if err := WriteBuild(current); err != nil { + return err + } + } + + _ = os.Remove(lockPath) + return nil +} + +func killProcessGroup(pid int) error { + pgid, err := syscall.Getpgid(pid) + if err != nil { + return err + } + return syscall.Kill(-pgid, syscall.SIGQUIT) +} + +func deployLockPath(appName string) string { + return filepath.Join(common.GetAppDataDirectory("apps", appName), ".deploy.lock") +} + +// CommandOutput streams the build log for a given build (or the current one). +func CommandOutput(appName, buildID string) error { + if err := common.VerifyAppName(appName); err != nil { + return err + } + + if buildID == "" || buildID == "current" { + body, err := os.ReadFile(deployLockPath(appName)) + if err != nil { + if os.IsNotExist(err) { + common.LogInfo1("App not currently deploying") + return nil + } + return err + } + buildID = strings.TrimSpace(string(body)) + if buildID == "" { + common.LogInfo1("No matching app deploy found") + return nil + } + } + + logPath := LogPathFor(appName, buildID) + if _, err := os.Stat(logPath); err != nil { + if os.IsNotExist(err) { + return outputViaJournalctl(buildID) + } + return err + } + + b, err := ReadBuild(appName, buildID) + if err != nil && !os.IsNotExist(err) { + return err + } + + if err == nil && b.Status == BuildStatusRunning && CheckPIDAlive(b.PID) { + return execStreaming("tail", "-n", "1000", "-f", logPath) + } + return execStreaming("cat", logPath) +} + +func outputViaJournalctl(buildID string) error { + if _, err := exec.LookPath("journalctl"); err != nil { + return fmt.Errorf("log file missing and journalctl not available for build %s", buildID) + } + return execStreaming("journalctl", "-n", "1000", "-a", "-o", "cat", fmt.Sprintf("SYSLOG_IDENTIFIER=dokku-%s", buildID)) +} + +func execStreaming(name string, args ...string) error { + cmd := exec.Command(name, args...) + cmd.Stdout = os.Stdout + cmd.Stderr = os.Stderr + return cmd.Run() +} + +// CommandPrune manually invokes PruneAppBuilds for one or every app. +func CommandPrune(appName string, allApps bool) error { + if allApps { + apps, err := common.DokkuApps() + if err != nil { + if errors.Is(err, common.NoAppsExist) { + common.LogWarn(err.Error()) + return nil + } + return err + } + for _, app := range apps { + if err := PruneAppBuilds(app); err != nil { + common.LogWarn(fmt.Sprintf("Prune failed for %s: %s", app, err)) + } + } + return nil + } + + if err := common.VerifyAppName(appName); err != nil { + return err + } + return PruneAppBuilds(appName) +} + +// CommandReport displays a build report for one or more apps. +func CommandReport(appName, format, infoFlag string) error { + if appName == "" { + apps, err := common.DokkuApps() + if err != nil { + if errors.Is(err, common.NoAppsExist) { + common.LogWarn(err.Error()) + return nil + } + return err + } + for _, app := range apps { + if err := ReportSingleApp(app, format, infoFlag); err != nil { + return err + } + } + return nil + } + return ReportSingleApp(appName, format, infoFlag) +} diff --git a/plugins/builds/subcommands/cancel b/plugins/builds/subcommands/cancel deleted file mode 100755 index 8472feae6..000000000 --- a/plugins/builds/subcommands/cancel +++ /dev/null @@ -1,37 +0,0 @@ -#!/usr/bin/env bash -set -eo pipefail -[[ $DOKKU_TRACE ]] && set -x -source "$PLUGIN_CORE_AVAILABLE_PATH/common/functions" - -cmd-builds-cancel() { - declare desc="cancel a running build for an app" - declare cmd="builds:cancel" - [[ "$1" == "$cmd" ]] && shift 1 - declare APP="$1" - local APP_DEPLOY_LOCK_FILE PROCESS_ID PROCESS_GROUP_ID - verify_app_name "$APP" - - local APP_DEPLOY_LOCK_FILE PROCESS_ID PROCESS_GROUP_ID - APP_DEPLOY_LOCK_FILE="$DOKKU_LIB_ROOT/data/apps/$APP/.deploy.lock" - if [[ ! -f "$APP_DEPLOY_LOCK_FILE" ]]; then - dokku_log_info1 "App not currently deploying" - return - fi - - PROCESS_ID="$(cat "$APP_DEPLOY_LOCK_FILE")" - if [[ -z "$PROCESS_ID" ]]; then - dokku_log_info1 "No matching app deploy found" - return - fi - - PROCESS_GROUP_ID="$(ps -o pgid= "$PROCESS_ID" || true)" - if [[ -z "$PROCESS_ID" ]]; then - dokku_log_info1 "Unable to find process group id for app deploy" - return - fi - - dokku_log_info1 "Killing app deploy" - kill -quit -- "-${PROCESS_GROUP_ID}" && rm -f "$APP_DEPLOY_LOCK_FILE" -} - -cmd-builds-cancel "$@" diff --git a/plugins/builds/subcommands/default b/plugins/builds/subcommands/default deleted file mode 100755 index f0205d69f..000000000 --- a/plugins/builds/subcommands/default +++ /dev/null @@ -1,6 +0,0 @@ -#!/usr/bin/env bash -set -eo pipefail -[[ $DOKKU_TRACE ]] && set -x -source "$PLUGIN_AVAILABLE_PATH/builds/internal-functions" - -cmd-builds-help "builds:help" diff --git a/plugins/builds/subcommands/list b/plugins/builds/subcommands/list deleted file mode 100755 index b8bf8b624..000000000 --- a/plugins/builds/subcommands/list +++ /dev/null @@ -1,13 +0,0 @@ -#!/usr/bin/env bash -set -eo pipefail -[[ $DOKKU_TRACE ]] && set -x -source "$PLUGIN_CORE_AVAILABLE_PATH/common/functions" - -cmd-builds-list() { - declare desc="list all running builds" - declare cmd="builds:list" - [[ "$1" == "$cmd" ]] && shift 1 - -} - -cmd-builds-list "$@" diff --git a/plugins/builds/subcommands/output b/plugins/builds/subcommands/output deleted file mode 100755 index ab1453059..000000000 --- a/plugins/builds/subcommands/output +++ /dev/null @@ -1,40 +0,0 @@ -#!/usr/bin/env bash -set -eo pipefail -[[ $DOKKU_TRACE ]] && set -x -source "$PLUGIN_CORE_AVAILABLE_PATH/common/functions" - -cmd-builds-output() { - declare desc="shows build output" - declare cmd="builds:output" - [[ "$1" == "$cmd" ]] && shift 1 - declare APP="$1" DEPLOY_ID="$2" - verify_app_name "$APP" - - if [[ -z "$DEPLOY_ID" ]] || [[ "$DEPLOY_ID" == "current" ]]; then - local APP_DEPLOY_LOCK_FILE PROCESS_ID - APP_DEPLOY_LOCK_FILE="$DOKKU_ROOT/$APP/.deploy.lock" - if [[ ! -f "$APP_DEPLOY_LOCK_FILE" ]]; then - dokku_log_info1 "App not currently deploying" - return - fi - - DEPLOY_ID="$(cat "$APP_DEPLOY_LOCK_FILE")" - if [[ -z "$DEPLOY_ID" ]]; then - dokku_log_info1 "No matching app deploy found" - return - fi - fi - - if [[ -z "$DEPLOY_ID" ]]; then - dokku_log_fail "No deploy id specified" - return 1 - fi - - if [[ -x /usr/bin/systemctl ]] || [[ -x /usr/local/bin/systemctl ]]; then - journalctl -n1000 -f -b -a -o cat "SYSLOG_IDENTIFIER=dokku-${DEPLOY_ID}" - else - grep "dokku-${DEPLOY_ID}" /var/log/syslog - fi -} - -cmd-builds-output "$@" diff --git a/plugins/builds/subcommands/report b/plugins/builds/subcommands/report deleted file mode 100755 index f9a78f2f9..000000000 --- a/plugins/builds/subcommands/report +++ /dev/null @@ -1,15 +0,0 @@ -#!/usr/bin/env bash -set -eo pipefail -[[ $DOKKU_TRACE ]] && set -x -source "$PLUGIN_CORE_AVAILABLE_PATH/common/functions" - -cmd-builds-report() { - declare desc="displays a build report for one or more apps" - declare cmd="builds:report" - [[ "$1" == "$cmd" ]] && shift 1 - declare APP="$1" - verify_app_name "$APP" - -} - -cmd-builds-report "$@" diff --git a/plugins/builds/triggers.go b/plugins/builds/triggers.go new file mode 100644 index 000000000..2b44a16bf --- /dev/null +++ b/plugins/builds/triggers.go @@ -0,0 +1,145 @@ +package builds + +import ( + "errors" + "fmt" + "os" + "strconv" + "time" + + "github.com/dokku/dokku/plugins/common" +) + +// TriggerInstall sets up the builds plugin's property storage. +func TriggerInstall() error { + if err := common.PropertySetup("builds"); err != nil { + return fmt.Errorf("Unable to install the builds plugin: %s", err.Error()) + } + return nil +} + +// TriggerPostDelete removes all builds data and properties for the given app. +func TriggerPostDelete(appName string) error { + if err := os.RemoveAll(AppDataDir(appName)); err != nil { + common.LogWarn(fmt.Sprintf("Could not remove builds data for %s: %s", appName, err)) + } + if err := common.PropertyDestroy("builds", appName); err != nil { + return err + } + return nil +} + +// TriggerPostAppRenameSetup renames the per-app data directory and clones +// builds properties to the new app name. +func TriggerPostAppRenameSetup(oldAppName string, newAppName string) error { + oldDir := AppDataDir(oldAppName) + newDir := AppDataDir(newAppName) + if _, err := os.Stat(oldDir); err == nil { + if err := os.MkdirAll(common.GetAppDataDirectory("builds", ""), 0755); err == nil { + if err := os.Rename(oldDir, newDir); err != nil { + common.LogWarn(fmt.Sprintf("Could not rename builds data dir for %s: %s", oldAppName, err)) + } + } + } + + if err := common.PropertyClone("builds", oldAppName, newAppName); err != nil { + return err + } + if err := common.PropertyDestroy("builds", oldAppName); err != nil { + return err + } + return nil +} + +// TriggerBuildsGenerateID writes a fresh build-id to stdout. The bash callers +// of this trigger consume the value via $(...) capture, so this function MUST +// emit only the build-id and nothing else - no log output, no warnings. +func TriggerBuildsGenerateID() error { + fmt.Println(GenerateBuildID()) + return nil +} + +// TriggerBuildsRecordStart persists the initial build record on lock-acquire. +// +// Args: +// +// kind is derived from source via BuildSource.DefaultKind(); callers do not +// pass it. +func TriggerBuildsRecordStart(appName, buildID, pidStr, sourceStr string) error { + if appName == "" { + return errors.New("builds-record-start: missing app name") + } + if buildID == "" { + return errors.New("builds-record-start: missing build id") + } + + pid, err := strconv.Atoi(pidStr) + if err != nil { + return fmt.Errorf("builds-record-start: invalid pid %q: %w", pidStr, err) + } + + source := BuildSource(sourceStr) + if !source.Valid() { + common.LogWarn(fmt.Sprintf("builds-record-start: unknown source %q for app %s, recording as %q", sourceStr, appName, BuildSourceUnknown)) + source = BuildSourceUnknown + } + + b := Build{ + ID: buildID, + App: appName, + Kind: source.DefaultKind(), + PID: pid, + StartedAt: time.Now().UTC(), + Status: BuildStatusRunning, + Source: source, + } + return WriteBuild(b) +} + +// TriggerBuildsRecordFinalize writes the terminal status onto an existing +// build record. It is idempotent: records that are already terminal (or have +// been overwritten by an earlier finalize) are left untouched. +// +// Args: +func TriggerBuildsRecordFinalize(appName, buildID, exitStr string) error { + if appName == "" { + return errors.New("builds-record-finalize: missing app name") + } + if buildID == "" { + return errors.New("builds-record-finalize: missing build id") + } + + exitCode, err := strconv.Atoi(exitStr) + if err != nil { + return fmt.Errorf("builds-record-finalize: invalid exit code %q: %w", exitStr, err) + } + + b, err := ReadBuild(appName, buildID) + if err != nil { + if os.IsNotExist(err) { + common.LogWarn(fmt.Sprintf("builds-record-finalize: no record for %s/%s, skipping", appName, buildID)) + return nil + } + return err + } + + if b.Status.IsTerminal() { + // Idempotent path - typically hit when builds:cancel finalized the + // record before the dying process reached release_app_deploy_lock. + return PruneAppBuilds(appName) + } + + now := time.Now().UTC() + b.FinishedAt = &now + b.ExitCode = &exitCode + if exitCode == 0 { + b.Status = BuildStatusSucceeded + } else { + b.Status = BuildStatusFailed + } + if err := WriteBuild(b); err != nil { + return err + } + + return PruneAppBuilds(appName) +} diff --git a/plugins/common/functions b/plugins/common/functions index 27102fc27..4d9e2cb15 100755 --- a/plugins/common/functions +++ b/plugins/common/functions @@ -626,14 +626,14 @@ release_and_deploy() { declare desc="main function for releasing and deploying an app" source "$PLUGIN_AVAILABLE_PATH/config/functions" - if [[ "$DOKKU_REDIRECT_OUTPUT" != "true" ]]; then - export DOKKU_REDIRECT_OUTPUT=true - exec &> >(tee >(tee | logger -i -t "dokku-${DOKKU_PID}")) - fi - local APP="$1" local IMAGE_TAG="${2:-latest}" + local SOURCE="${3:-deploy}" + + dokku_setup_build_capture "$APP" "$SOURCE" + local IMAGE=$(get_app_image_name "$APP" "$IMAGE_TAG") + local exit_code=0 if verify_image "$IMAGE"; then IMAGE_SOURCE_TYPE="$(get_image_builder_type "$IMAGE" "$APP")" @@ -643,20 +643,28 @@ release_and_deploy() { local DOKKU_SKIP_DEPLOY=${DOKKU_APP_SKIP_DEPLOY:="$DOKKU_GLOBAL_SKIP_DEPLOY"} dokku_log_info1 "Releasing $APP..." - dokku_release "$APP" "$IMAGE_SOURCE_TYPE" "$IMAGE_TAG" + dokku_release "$APP" "$IMAGE_SOURCE_TYPE" "$IMAGE_TAG" || exit_code="$?" - if [[ "$DOKKU_SKIP_DEPLOY" != "true" ]]; then + if [[ "$exit_code" -eq 0 ]] && [[ "$DOKKU_SKIP_DEPLOY" != "true" ]]; then local DOKKU_SCHEDULER=$(get_app_scheduler "$APP") dokku_log_info1 "Deploying $APP via the $DOKKU_SCHEDULER scheduler..." - cmd-deploy "$APP" "$IMAGE_TAG" - dokku_log_info2 "Application deployed:" - plugn trigger domains-urls "$APP" urls | sed "s/^/ /" - else + cmd-deploy "$APP" "$IMAGE_TAG" || exit_code="$?" + if [[ "$exit_code" -eq 0 ]]; then + dokku_log_info2 "Application deployed:" + plugn trigger domains-urls "$APP" urls | sed "s/^/ /" + fi + elif [[ "$exit_code" -eq 0 ]]; then dokku_log_info1 "Skipping deployment" fi echo fi + + if [[ -n "$DOKKU_BUILD_ID" ]]; then + plugn trigger builds-record-finalize "$APP" "$DOKKU_BUILD_ID" "$exit_code" || true + fi + + return "$exit_code" } dokku_receive() { @@ -852,6 +860,30 @@ merge_dedupe_list() { echo "$merged_list" } +dokku_setup_build_capture() { + declare desc="generate a build-id, write a record, and redirect output to log + journald" + declare APP="$1" SOURCE="${2:-unknown}" + [[ "$DOKKU_REDIRECT_OUTPUT" == "true" ]] && return 0 + if [[ -z "$APP" ]]; then + return 0 + fi + export DOKKU_REDIRECT_OUTPUT=true + if [[ -z "$DOKKU_BUILD_ID" ]]; then + DOKKU_BUILD_ID="$(plugn trigger builds-generate-id 2>/dev/null)" + if [[ -z "$DOKKU_BUILD_ID" ]]; then + DOKKU_BUILD_ID="pid-${DOKKU_PID}" + fi + export DOKKU_BUILD_ID + fi + export DOKKU_BUILD_SOURCE="$SOURCE" + local DIR="$DOKKU_LIB_ROOT/data/builds/$APP" + local LOG="$DIR/${DOKKU_BUILD_ID}.log" + mkdir -p "$DIR" && : >"$LOG" + export DOKKU_BUILD_LOG_FILE="$LOG" + plugn trigger builds-record-start "$APP" "$DOKKU_BUILD_ID" "$$" "$SOURCE" || true + exec &> >(tee -a "$LOG" | logger -i -t "dokku-${DOKKU_BUILD_ID}") +} + acquire_app_deploy_lock() { declare desc="acquire advisory lock for use in deploys" local APP="$1" @@ -860,14 +892,27 @@ acquire_app_deploy_lock() { local LOCK_WAITING_MSG="$APP currently has a deploy lock in place. Waiting..." local LOCK_FAILED_MSG="$APP currently has a deploy lock in place. Exiting..." + if [[ -z "$DOKKU_BUILD_ID" ]]; then + DOKKU_BUILD_ID="$(plugn trigger builds-generate-id 2>/dev/null)" + if [[ -z "$DOKKU_BUILD_ID" ]]; then + DOKKU_BUILD_ID="pid-${DOKKU_PID}" + fi + export DOKKU_BUILD_ID + fi + acquire_advisory_lock "$APP_DEPLOY_LOCK_FILE" "$LOCK_TYPE" "$LOCK_WAITING_MSG" "$LOCK_FAILED_MSG" } release_app_deploy_lock() { declare desc="release advisory lock used in deploys" local APP="$1" + local EXIT_CODE="${2:-0}" local APP_DEPLOY_LOCK_FILE="$DOKKU_LIB_ROOT/data/apps/$APP/.deploy.lock" + if [[ -n "$DOKKU_BUILD_ID" ]]; then + plugn trigger builds-record-finalize "$APP" "$DOKKU_BUILD_ID" "$EXIT_CODE" || true + fi + release_advisory_lock "$APP_DEPLOY_LOCK_FILE" } @@ -895,7 +940,7 @@ acquire_advisory_lock() { dokku_log_fail "Run 'apps:unlock' to release the existing deploy lock" fi fi - echo "$DOKKU_PID" >"$LOCK_FILE" + echo "${DOKKU_BUILD_ID:-$DOKKU_PID}" >"$LOCK_FILE" } release_advisory_lock() { diff --git a/plugins/config/config.go b/plugins/config/config.go index 45cbe9b97..5b898370d 100644 --- a/plugins/config/config.go +++ b/plugins/config/config.go @@ -147,7 +147,7 @@ func triggerRestart(appName string) { common.LogInfo1(fmt.Sprintf("Restarting app %s", appName)) _, err := common.CallPlugnTrigger(common.PlugnTriggerInput{ Trigger: "release-and-deploy", - Args: []string{appName}, + Args: []string{appName, "", "config-redeploy"}, StreamStdio: true, }) if err != nil { diff --git a/plugins/git/functions b/plugins/git/functions index 9de24288d..764942437 100755 --- a/plugins/git/functions +++ b/plugins/git/functions @@ -142,7 +142,7 @@ git_receive_app() { local exit_code=0 acquire_app_deploy_lock "$APP" "exclusive" git_build "$APP" "$REV" || exit_code="$?" - release_app_deploy_lock "$APP" + release_app_deploy_lock "$APP" "$exit_code" return "$exit_code" fi } diff --git a/plugins/git/internal-functions b/plugins/git/internal-functions index 30634839b..eee315f7b 100755 --- a/plugins/git/internal-functions +++ b/plugins/git/internal-functions @@ -52,10 +52,7 @@ cmd-git-from-archive() { dokku_log_fail "Invalid archive type specified, valid archive types include: tar, tar.gz, zip" fi - if [[ "$DOKKU_REDIRECT_OUTPUT" != "true" ]]; then - export DOKKU_REDIRECT_OUTPUT=true - exec &> >(tee >(tee | logger -i -t "dokku-${DOKKU_PID}")) - fi + dokku_setup_build_capture "$APP" "git:from-archive" plugn trigger git-from-archive "$APP" "$ARCHIVE_URL" "$ARCHIVE_TYPE" "$USER_NAME" "$USER_EMAIL" plugn trigger deploy-source-set "$APP" "$ARCHIVE_TYPE" "$ARCHIVE_URL" } @@ -166,10 +163,7 @@ cmd-git-load-image() { [[ -z "$DOCKER_IMAGE" ]] && dokku_log_fail "Please specify a docker image" [[ ! -t 0 ]] || dokku_log_fail "Expecting tar archive containing docker image on STDIN" - if [[ "$DOKKU_REDIRECT_OUTPUT" != "true" ]]; then - export DOKKU_REDIRECT_OUTPUT=true - exec &> >(tee >(tee | logger -i -t "dokku-${DOKKU_PID}")) - fi + dokku_setup_build_capture "$APP" "git:load-image" cat | docker load if ! verify_image "$DOCKER_IMAGE"; then @@ -223,10 +217,7 @@ cmd-git-from-image() { export DOKKU_APPS_FORCE_DELETE=1 fi - if [[ "$DOKKU_REDIRECT_OUTPUT" != "true" ]]; then - export DOKKU_REDIRECT_OUTPUT=true - exec &> >(tee >(tee | logger -i -t "dokku-${DOKKU_PID}")) - fi + dokku_setup_build_capture "$APP" "git:from-image" if ! plugn trigger git-from-image "$APP" "$DOCKER_IMAGE" "$BUILD_DIR" "$USER_NAME" "$USER_EMAIL"; then return 1 @@ -276,10 +267,7 @@ cmd-git-sync() { DOKKU_DEPLOY_BRANCH="$(fn-git-deploy-branch "$APP")" CURRENT_REF="$(fn-git-cmd "$APP_ROOT" rev-parse "$DOKKU_DEPLOY_BRANCH" 2>/dev/null || true)" - if [[ "$DOKKU_REDIRECT_OUTPUT" != "true" ]]; then - export DOKKU_REDIRECT_OUTPUT=true - exec &> >(tee >(tee | logger -i -t "dokku-${DOKKU_PID}")) - fi + dokku_setup_build_capture "$APP" "git:sync" if ! fn-git-cmd "$APP_ROOT" rev-parse "$DOKKU_DEPLOY_BRANCH" &>/dev/null; then dokku_log_info1_quiet "Cloning $APP from $GIT_REMOTE#$GIT_REF" diff --git a/plugins/git/receive-app b/plugins/git/receive-app index 78821f835..74316595c 100755 --- a/plugins/git/receive-app +++ b/plugins/git/receive-app @@ -7,10 +7,7 @@ trigger-git-receive-app() { declare desc="builds the app from the local git repository" declare trigger="receive-app" - if [[ "$DOKKU_REDIRECT_OUTPUT" != "true" ]]; then - export DOKKU_REDIRECT_OUTPUT=true - exec &> >(tee >(tee | logger -i -t "dokku-${DOKKU_PID}")) - fi + dokku_setup_build_capture "$1" "git-hook" git_receive_app "$@" return $? diff --git a/tests/unit/builds.bats b/tests/unit/builds.bats new file mode 100644 index 000000000..f3bd1bbb0 --- /dev/null +++ b/tests/unit/builds.bats @@ -0,0 +1,328 @@ +#!/usr/bin/env bats + +load test_helper + +setup() { + global_setup + create_app +} + +teardown() { + rm -rf "$DOKKU_LIB_ROOT/data/builds/$TEST_APP" || true + destroy_app + global_teardown +} + +write_finished_record() { + local app="$1" id="$2" status="$3" source="${4:-git-hook}" pid="${5:-99999}" kind="${6:-build}" exit_code="${7:-0}" + local dir="$DOKKU_LIB_ROOT/data/builds/$app" + mkdir -p "$dir" + local now + now="$(date -u +%Y-%m-%dT%H:%M:%SZ)" + cat >"$dir/$id.json" <"$dir/$id.log" +} + +write_running_record() { + local app="$1" id="$2" pid="$3" source="${4:-git-hook}" kind="${5:-build}" + local dir="$DOKKU_LIB_ROOT/data/builds/$app" + mkdir -p "$dir" + local now + now="$(date -u +%Y-%m-%dT%H:%M:%SZ)" + cat >"$dir/$id.json" <"$dir/$id.log" +} + +@test "(builds) builds:help" { + run /bin/bash -c "dokku builds" + echo "output: $output" + echo "status: $status" + assert_output_contains "Manage running and historical builds" + + run /bin/bash -c "dokku builds:help" + echo "output: $output" + echo "status: $status" + assert_output_contains "Manage running and historical builds" +} + +@test "(builds:list) returns empty list when no builds have run" { + run /bin/bash -c "dokku builds:list $TEST_APP" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "No builds recorded" +} + +@test "(builds:list) shows a finished record after a successful deploy" { + run deploy_app python dokku@$DOKKU_DOMAIN:$TEST_APP + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku builds:list $TEST_APP" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "succeeded" + assert_output_contains "git-hook" +} + +@test "(builds:list) shows an abandoned record when the PID is gone but the record is still running" { + write_running_record "$TEST_APP" "ghost001" 99999 "git-hook" + + run /bin/bash -c "dokku builds:list $TEST_APP" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "abandoned" +} + +@test "(builds:list --kind=build) filters to build-kind records" { + write_finished_record "$TEST_APP" "b1" "succeeded" "git-hook" 1001 "build" + write_finished_record "$TEST_APP" "d1" "succeeded" "ps:restart" 1002 "deploy" + + run /bin/bash -c "dokku builds:list $TEST_APP --kind build" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "b1" + assert_output_not_contains "d1" +} + +@test "(builds:list --kind=deploy) filters to deploy-kind records" { + write_finished_record "$TEST_APP" "b1" "succeeded" "git-hook" 1001 "build" + write_finished_record "$TEST_APP" "d1" "succeeded" "ps:restart" 1002 "deploy" + + run /bin/bash -c "dokku builds:list $TEST_APP --kind deploy" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "d1" + assert_output_not_contains "b1" +} + +@test "(builds:list --kind=invalid) fails with a usage error" { + run /bin/bash -c "dokku builds:list $TEST_APP --kind=garbage" + echo "output: $output" + echo "status: $status" + assert_failure + assert_output_contains "Invalid --kind" +} + +@test "(builds:set) writes a per-app retention" { + run /bin/bash -c "dokku builds:set $TEST_APP retention 5" + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku builds:report $TEST_APP --builds-retention" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "5" +} + +@test "(builds:set --global) writes the global retention" { + run /bin/bash -c "dokku builds:set --global retention 7" + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku builds:report $TEST_APP --builds-computed-retention" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "7" + + dokku builds:set --global retention "" || true +} + +@test "(builds:set) clears the override when called with no value" { + dokku builds:set "$TEST_APP" retention 5 + + run /bin/bash -c "dokku builds:set $TEST_APP retention" + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku builds:report $TEST_APP --builds-retention" + echo "output: $output" + echo "status: $status" + assert_success + assert_output "" +} + +@test "(builds:set) rejects non-positive-integer values" { + run /bin/bash -c "dokku builds:set $TEST_APP retention 0" + echo "output: $output" + echo "status: $status" + assert_failure + + run /bin/bash -c "dokku builds:set $TEST_APP retention abc" + echo "output: $output" + echo "status: $status" + assert_failure +} + +@test "(builds:info) returns non-zero for a missing build-id" { + run /bin/bash -c "dokku builds:info $TEST_APP nonexistent" + echo "output: $output" + echo "status: $status" + assert_failure +} + +@test "(builds:info) returns the recorded fields including --format json" { + write_finished_record "$TEST_APP" "info001" "succeeded" "git-hook" 1234 "build" + + run /bin/bash -c "dokku builds:info $TEST_APP info001" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "info001" + assert_output_contains "succeeded" + assert_output_contains "Log:" + + run /bin/bash -c "dokku builds:info $TEST_APP info001 --format json" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "\"id\":\"info001\"" + assert_output_contains "\"log_path\":" +} + +@test "(builds:cancel) returns 0 with a friendly message when no lock is present" { + run /bin/bash -c "dokku builds:cancel $TEST_APP" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "App not currently deploying" +} + +@test "(builds:cancel) refuses to cancel a record whose status is not running" { + write_finished_record "$TEST_APP" "done01" "succeeded" "git-hook" 1234 "build" + mkdir -p "$DOKKU_LIB_ROOT/data/apps/$TEST_APP" + echo "done01" >"$DOKKU_LIB_ROOT/data/apps/$TEST_APP/.deploy.lock" + + run /bin/bash -c "dokku builds:cancel $TEST_APP" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "no longer running" + + rm -f "$DOKKU_LIB_ROOT/data/apps/$TEST_APP/.deploy.lock" +} + +@test "(builds:cancel) marks an abandoned record as failed instead of canceled" { + write_running_record "$TEST_APP" "abnd01" 99999 "git-hook" + mkdir -p "$DOKKU_LIB_ROOT/data/apps/$TEST_APP" + echo "abnd01" >"$DOKKU_LIB_ROOT/data/apps/$TEST_APP/.deploy.lock" + + run /bin/bash -c "dokku builds:cancel $TEST_APP" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "already terminated" + + run /bin/bash -c "dokku builds:info $TEST_APP abnd01" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "failed" +} + +@test "(builds:output) cats the log file for a finished build" { + write_finished_record "$TEST_APP" "out001" "succeeded" "git-hook" 1234 "build" + echo "build log line" >"$DOKKU_LIB_ROOT/data/builds/$TEST_APP/out001.log" + + run /bin/bash -c "dokku builds:output $TEST_APP out001" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "build log line" +} + +@test "(builds:prune) prunes an app's records to retention" { + dokku builds:set "$TEST_APP" retention 2 + + for i in 1 2 3 4 5; do + write_finished_record "$TEST_APP" "p$i" "succeeded" "git-hook" $((1000+i)) "build" + sleep 1 + done + + run /bin/bash -c "dokku builds:prune $TEST_APP" + echo "output: $output" + echo "status: $status" + assert_success + + count="$(ls $DOKKU_LIB_ROOT/data/builds/$TEST_APP/*.json 2>/dev/null | wc -l)" + echo "remaining: $count" + [[ "$count" -le 2 ]] || (echo "expected <=2 records, got $count" && false) +} + +@test "(builds:prune) reaps an abandoned record and finalizes it as failed" { + write_running_record "$TEST_APP" "ghost02" 99999 "git-hook" + + run /bin/bash -c "dokku builds:prune $TEST_APP" + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku builds:info $TEST_APP ghost02" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "failed" +} + +@test "(builds:prune) skips live-running records" { + local pid=$$ + write_running_record "$TEST_APP" "alive02" "$pid" "git-hook" + + run /bin/bash -c "dokku builds:prune $TEST_APP" + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku builds:info $TEST_APP alive02" + echo "output: $output" + echo "status: $status" + assert_success + assert_output_contains "running" +} + +@test "(builds) [storage] directory is removed on apps:destroy" { + write_finished_record "$TEST_APP" "del01" "succeeded" "git-hook" 1234 "build" + [[ -d "$DOKKU_LIB_ROOT/data/builds/$TEST_APP" ]] + + run /bin/bash -c "dokku --force apps:destroy $TEST_APP" + echo "output: $output" + echo "status: $status" + assert_success + + [[ ! -d "$DOKKU_LIB_ROOT/data/builds/$TEST_APP" ]] + + # Recreate so teardown can run cleanly. + create_app +}