diff --git a/docs/advanced-usage/persistent-storage.md b/docs/advanced-usage/persistent-storage.md index b11a50638..f1aa62ad6 100644 --- a/docs/advanced-usage/persistent-storage.md +++ b/docs/advanced-usage/persistent-storage.md @@ -147,4 +147,4 @@ By default, Dokku will execute your buildpack application processes as the `hero > NOTE: this user must exist in your herokuish image. -Additionally, the default `docker-local` scheduler that comes with Dokku will ensure your storage mounts are owned by either `herokuishuser` or the overridden value you have set in `DOKKU_APP_USER`. See the [scheduler-docker-local documentation](/docs/advanced-usage/schedulers/docker-local.md#disabling-chown-of-persistent-storage) docs for more information. +Additionally, the default `docker-local` scheduler that comes with Dokku will ensure your storage mounts are owned by either `herokuishuser` or the overridden value you have set in `DOKKU_APP_USER`. See the [docker-local scheduler documentation](/docs/advanced-usage/schedulers/docker-local.md#disabling-chown-of-persistent-storage) docs for more information. diff --git a/docs/deployment/logs.md b/docs/deployment/logs.md index 8c10208e8..712065f31 100644 --- a/docs/deployment/logs.md +++ b/docs/deployment/logs.md @@ -36,7 +36,7 @@ will show logs continually from the web process. ### Failed deploy logs -> Warning: The default `docker-local` scheduler will "store" these until the next deploy or until the old containers are garbage collected - whichever runs first. If you require the logs beyond this point in time, please ship the logs to a centralized log server. +> Warning: The default [docker-local scheduler](/docs/advanced-usage/schedulers/docker-local.md) will "store" these until the next deploy or until the old containers are garbage collected - whichever runs first. If you require the logs beyond this point in time, please ship the logs to a centralized log server. In some cases, it may be useful to retrieve the logs from a previously failed deploy. diff --git a/docs/development/plugin-triggers.md b/docs/development/plugin-triggers.md index 991b91af0..202c74a9d 100644 --- a/docs/development/plugin-triggers.md +++ b/docs/development/plugin-triggers.md @@ -821,6 +821,20 @@ set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x # TODO ``` +### `post-container-create` + +- Description: This trigger should be used to do stuff to containers after they are created but before they are started. They are explicitely for commands that may involve network traffic, and _not_ for commands that are self-contained, such as chown or tar. +- Invoked by: `dokku run`, `dokku ps:rebuild`, `dokku deploy` +- Arguments "app|service" "$CONTAINER_ID" "$APP|$SERVICE" "$PHASE" + +```shell +#!/usr/bin/env bash + +set -eo pipefail; [[ $DOKKU_TRACE ]] && set -x + +# TODO +``` + ### `post-create` - Description: Can be used to run commands after an app is created. diff --git a/docs/networking/network.md b/docs/networking/network.md index 584ef7fa4..0b2572584 100644 --- a/docs/networking/network.md +++ b/docs/networking/network.md @@ -1,8 +1,13 @@ # Network Management -> New as of 0.11.0 +> New as of 0.11.0, Enhanced in 0.20.0 ``` +network:create , # Creates an attachable docker network +network:destroy , # Destroys a docker network +network:exists , # Checks if a docker network exists +network:info , # Outputs information about a docker network +network:list # Lists all docker networks network:report [] [] # Displays a network report for one or more apps network:rebuild # Rebuilds network settings for an app network:rebuildall # Rebuild network settings for all apps @@ -13,6 +18,161 @@ The Network plugin allows developers to abstract the concept of container networ ## Usage +### Listing networks + +> New as of 0.20.0, Requires Docker 1.21+ + +You can easily list all available networks using the `network:list` command: + +```shell +dokku network:list +``` + +``` +=====> Networks +bridge +host +none +test-network +``` + +Note that you can easily hide extra output from Dokku commands by using the `--quiet` flag, which makes it easier to parse on the command line. + +```shell +dokku --quiet network:list +``` + +``` +bridge +host +none +test-network +``` + +### Creating a network + +> New as of 0.20.0, Requires Docker 1.21+ + +Docker networks can be created via the `network:create` command. Executing this command will create an attachable `bridge` network. This can be used to route requests between containers without going through any public network. + +```shell +dokku network:create test-network +``` + +``` +-----> Creating network test-network +``` + +Specifying other additional flags or other types of networks can be created directly via the `docker` command. + +### Destroying a network + +> New as of 0.20.0, Requires Docker 1.21+ + +A Docker network without any associated containers may be destroyed via the `network:destroy` command. Docker will refuse to destroy networks that have containers attached. + +```shell +dokku network:destroy test-network +``` + +```shell + ! WARNING: Potentially Destructive Action + ! This command will destroy network test. + ! To proceed, type "test" +> test +-----> Destroying network test +``` + +As the command is destructive, it will default to asking for confirmation before executing the removal of the network. This may be avoided by providing the `--force` flag: + +```shell +dokku --force network:destroy test-network +``` + +```shell +-----> Destroying network test +``` + +### Checking if a network exists + +> New as of 0.20.0, Requires Docker 1.21+ + +For CI/CD pipelines, it may be useful to see if an network exists before creating a new network. You can do so via the `network:exists` command: + +```shell +dokku network:exists nonexistent-network +``` + +``` +Network does not exist +``` + +The `network:exists` command will return non-zero if the network does not exist, and zero if it does. + +### Checking network info + +> New as of 0.20.0, Requires Docker 1.21+ + +Network information can be retrieved via the `network:info` command. This is a slightly different version of the `docker network` command. + +```shell +dokku network:info test-network +``` + +``` +// TODO +``` + +### Attaching an app to a network + +> New as of 0.20.0, Requires Docker 1.21+ + +Apps will default to being associated with the `bridge` network, but can be attached to `attachable` networks by changing the `attach-post-create` or `attach-post-deploy` network properties when using the [docker-local scheduler](/docs/advanced-usage/schedulers/docker-local.md). A change in these values will require an app deploy or rebuild. + +```shell +# associates the network after a container is created but before it is started +dokku network:set node-js-app attach-post-create test-network + +# associates the network after the deploy is successful but before the proxy is updated +dokku network:set node-js-app attach-post-create other-test-network +``` + +Setting the `attach` network property to an empty value will de-associate the container with the network. + +```shell +dokku network:set node-js-app attach-post-create +dokku network:set node-js-app attach-post-deploy +``` + +When a container created for a deployment is being attached to a network - regardless of which `attach` property was used - a network alias of the pattern `APP.PROC_TYPE` will be added to all containers. This can be used to load-balance requests between containers. + +#### When to attach containers to a network + +Containers can be attached to a network for a variety of reasons: + +- A background process in one app needs to communicate to a webservice in another app +- An app needs to talk to a container not managed by Dokku in a secure manner +- A custom network that allows transparent access to another host exists and is necessary for an app to run + +Whatever the reason, the semantics of the two network hooks are important and are outlined before. + +- `attach-post-create`: + - Phase it applies to: + - `build`: Intermediate containers created during the build process. + - `deploy`: Deployed app containers. + - `run`: Containers created by the `run` command. + - Container state on attach: `created` but not `running` + - Use case: When the container needs to access a resource on the network. + - Example: The app needs to talk to a database on the same network when it first boots. +- `attach-post-deploy` + - Phase it applies to: + - `deploy`: Deployed app containers. + - Container state on attach: `running` + - Use case: When another container on the network needs to access _this_ container. + - Example: A background process needs to communicate with the web process exposed by this container. + +> Warning: If the attachment fails at this stage, this may result in your application failing to respond to proxied requests once older containers are removed. + ### Rebuilding network settings There are cases where you may need to rebuild the network configuration for an app, such as on app boot or container restart. In these cases, you can use the `network:rebuild` command: @@ -37,7 +197,7 @@ dokku network:rebuildall > This functionality does not control the `--network` docker flag. Please use the [docker-options plugin](docs/advanced-usage/docker-options.md) to manage this flag. -By default, an application will only bind to the internal interface. This behavior can be modified per app by changing the `bind-all-interfaces` network property. +By default, an app will only bind to the internal interface. This behavior can be modified per app by changing the `bind-all-interfaces` network property. ```shell # bind to the default docker interface (`docker0`) with a random internal ip diff --git a/plugins/app-json/internal-functions b/plugins/app-json/internal-functions index 0be8030f5..c3abb5625 100755 --- a/plugins/app-json/internal-functions +++ b/plugins/app-json/internal-functions @@ -98,8 +98,13 @@ execute_script() { DOKKU_APP_SHELL="$(config_get "$APP" DOKKU_APP_SHELL || echo "$DOKKU_APP_SHELL")" [[ -z "$DOKKU_APP_SHELL" ]] && DOKKU_APP_SHELL="/bin/bash" - id=$("$DOCKER_BIN" container run "${DOCKER_RUN_LABEL_ARGS[@]}" $DOKKU_GLOBAL_RUN_ARGS --env DOKKU_TRACE="$DOKKU_TRACE" --label=dokku_phase_script="${PHASE_SCRIPT_KEY}" -d -v "$CACHE_HOST_DIR:/cache" "${ARG_ARRAY[@]}" "$IMAGE" "$DOKKU_APP_SHELL" -c "$COMMAND") - if test "$("$DOCKER_BIN" container wait "$id")" -ne 0; then + local DOKKU_CONTAINER_EXIT_CODE=0 + id=$("$DOCKER_BIN" container create "${DOCKER_RUN_LABEL_ARGS[@]}" $DOKKU_GLOBAL_RUN_ARGS --env DOKKU_TRACE="$DOKKU_TRACE" --label=dokku_phase_script="${PHASE_SCRIPT_KEY}" -v "$CACHE_HOST_DIR:/cache" "${ARG_ARRAY[@]}" "$IMAGE" "$DOKKU_APP_SHELL" -c "$COMMAND") + plugn trigger post-container-create "app" "$id" "$APP" "$PHASE_SCRIPT_KEY" + "$DOCKER_BIN" container start "$id" >/dev/null || DOKKU_CONTAINER_EXIT_CODE=$? + DOKKU_CONTAINER_EXIT_CODE="$("$DOCKER_BIN" container wait "$id" 2>/dev/null || echo "$DOKKU_CONTAINER_EXIT_CODE")" + + if [[ "$DOKKU_CONTAINER_EXIT_CODE" -ne 0 ]]; then dokku_container_log_verbose_quiet "$id" dokku_log_fail "execution of '$SCRIPT_CMD' failed!" fi diff --git a/plugins/builder-herokuish/builder-build b/plugins/builder-herokuish/builder-build index a81b5fd68..2627d74c5 100755 --- a/plugins/builder-herokuish/builder-build +++ b/plugins/builder-herokuish/builder-build @@ -44,10 +44,15 @@ trigger-builder-build-builder-herokuish() { declare -a ARG_ARRAY eval "ARG_ARRAY=($DOCKER_ARGS)" - # shellcheck disable=SC2086 - CID=$("$DOCKER_BIN" container run "${DOCKER_RUN_LABEL_ARGS[@]}" $DOKKU_GLOBAL_RUN_ARGS -d -v $DOKKU_APP_HOST_CACHE_DIR:/cache -e CACHE_PATH=/cache "${ARG_ARRAY[@]}" $IMAGE /build) + + local DOKKU_CONTAINER_EXIT_CODE=0 + local CID=$("$DOCKER_BIN" container create "${DOCKER_RUN_LABEL_ARGS[@]}" $DOKKU_GLOBAL_RUN_ARGS -v $DOKKU_APP_HOST_CACHE_DIR:/cache -e CACHE_PATH=/cache "${ARG_ARRAY[@]}" $IMAGE /build) + plugn trigger post-container-create "app" "$CID" "$APP" "build" + "$DOCKER_BIN" container start "$CID" >/dev/null || DOKKU_CONTAINER_EXIT_CODE=$? "$DOCKER_BIN" container attach "$CID" - if test "$("$DOCKER_BIN" container wait "$CID")" -ne 0; then + DOKKU_CONTAINER_EXIT_CODE="$("$DOCKER_BIN" container wait "$CID" 2>/dev/null || echo "$DOKKU_CONTAINER_EXIT_CODE")" + + if [[ "$DOKKU_CONTAINER_EXIT_CODE" -ne 0 ]]; then dokku_log_warn "Failure during app build" return 1 fi diff --git a/plugins/common/common.go b/plugins/common/common.go index 3ad404df2..f83190afa 100644 --- a/plugins/common/common.go +++ b/plugins/common/common.go @@ -6,6 +6,7 @@ import ( "io/ioutil" "os" "os/exec" + "path/filepath" "regexp" "strings" "unicode" @@ -115,6 +116,61 @@ func GetAppImageRepo(appName string) string { return strings.Join([]string{"dokku", appName}, "/") } +// GetAppContainerIDs returns a list of docker container ids for given app and optional container_type +func GetAppContainerIDs(appName string, containerType string) ([]string, error) { + var containerIDs []string + if err := VerifyAppName(appName); err != nil { + return containerIDs, err + } + + dokkuRoot := MustGetEnv("DOKKU_ROOT") + appRoot := fmt.Sprintf("%v/%v", dokkuRoot, appName) + containerFilePath := fmt.Sprintf("%v/CONTAINER", appRoot) + _, err := os.Stat(containerFilePath) + if !os.IsNotExist(err) { + containerIDs = append(containerIDs, ReadFirstLine(containerFilePath)) + } + + containerPattern := fmt.Sprintf("%v/CONTAINER.*", appRoot) + if containerType != "" { + containerPattern = fmt.Sprintf("%v/CONTAINER.%v.*", appRoot, containerType) + if strings.Contains(".", containerType) { + containerPattern = fmt.Sprintf("%v/CONTAINER.%v", appRoot, containerType) + } + } + + files, _ := filepath.Glob(containerPattern) + for _, containerFile := range files { + containerIDs = append(containerIDs, ReadFirstLine(containerFile)) + } + + return containerIDs, nil +} + +// GetAppRunningContainerIDs return a list of running docker container ids for given app and optional container_type +func GetAppRunningContainerIDs(appName string, containerType string) ([]string, error) { + var runningContainerIDs []string + if err := VerifyAppName(appName); err != nil { + return runningContainerIDs, err + } + + if !IsDeployed(appName) { + LogFail(fmt.Sprintf("App %v has not been deployed", appName)) + } + + containerIDs, err := GetAppContainerIDs(appName, containerType) + if err != nil { + return runningContainerIDs, nil + } + for _, containerID := range containerIDs { + if ContainerIsRunning(containerID) { + runningContainerIDs = append(runningContainerIDs, containerID) + } + } + + return runningContainerIDs, nil +} + // ContainerIsRunning checks to see if a container is running func ContainerIsRunning(containerID string) bool { b, err := DockerInspect(containerID, "'{{.State.Running}}'") diff --git a/plugins/network/.gitignore b/plugins/network/.gitignore index 6385eaf1e..dee017e8f 100644 --- a/plugins/network/.gitignore +++ b/plugins/network/.gitignore @@ -1,10 +1,9 @@ /commands +/core-post-deploy /subcommands/* /triggers/* /triggers /network-* /install -/post-app-clone-setup -/post-create -/post-delete +/post-* /report diff --git a/plugins/network/Makefile b/plugins/network/Makefile index 7dd901ccf..fe23e2813 100644 --- a/plugins/network/Makefile +++ b/plugins/network/Makefile @@ -1,5 +1,5 @@ -SUBCOMMANDS = subcommands/rebuild subcommands/rebuildall subcommands/report subcommands/set -TRIGGERS = triggers/install triggers/network-build-config triggers/network-compute-ports triggers/network-config-exists triggers/network-get-ipaddr triggers/network-get-listeners triggers/network-get-port triggers/network-get-property triggers/network-write-ipaddr triggers/network-write-port triggers/post-app-clone-setup triggers/post-create triggers/post-delete triggers/report +SUBCOMMANDS = subcommands/create subcommands/destroy subcommands/exists subcommands/info subcommands/list subcommands/rebuild subcommands/rebuildall subcommands/report subcommands/set +TRIGGERS = triggers/core-post-deploy triggers/install triggers/network-build-config triggers/network-compute-ports triggers/network-config-exists triggers/network-get-ipaddr triggers/network-get-listeners triggers/network-get-port triggers/network-get-property triggers/network-write-ipaddr triggers/network-write-port triggers/post-app-clone-setup triggers/post-container-create triggers/post-create triggers/post-delete triggers/report BUILD = commands subcommands triggers PLUGIN_NAME = network diff --git a/plugins/network/functions.go b/plugins/network/functions.go new file mode 100644 index 000000000..e07e02756 --- /dev/null +++ b/plugins/network/functions.go @@ -0,0 +1,110 @@ +package network + +import ( + "bytes" + "errors" + "fmt" + "strings" + + "github.com/dokku/dokku/plugins/common" + + sh "github.com/codeskyblue/go-sh" +) + +// attachAppToNetwork attaches a container to a network +func attachAppToNetwork(containerID string, networkName string, appName string, phase string, processType string) { + cmdParts := []string{ + common.DockerBin(), + "network", + "connect", + } + + if phase == "deploy" { + property := "tld" + defaultValue := GetDefaultValue(property) + tld := common.PropertyGetDefault("network", appName, property, defaultValue) + + networkAlias := fmt.Sprintf("%v.%v", appName, processType) + if tld != "" { + networkAlias = fmt.Sprintf("%v.%v", networkAlias, tld) + } + + cmdParts = append(cmdParts, "--alias") + cmdParts = append(cmdParts, networkAlias) + + hostname, err := common.DockerInspect(containerID, "{{ .Config.Hostname }}") + if err != nil { + common.LogWarn(err.Error()) + } else { + cmdParts = append(cmdParts, "--alias") + cmdParts = append(cmdParts, fmt.Sprintf("%v.%v", hostname, networkAlias)) + } + } + + cmdParts = append(cmdParts, networkName) + cmdParts = append(cmdParts, containerID) + attachCmd := common.NewShellCmd(strings.Join(cmdParts, " ")) + var stderr bytes.Buffer + attachCmd.ShowOutput = false + attachCmd.Command.Stderr = &stderr + _, err := attachCmd.Output() + if err != nil { + err = errors.New(strings.TrimSpace(stderr.String())) + common.LogFail(fmt.Sprintf("Unable to attach container to network: %v", err.Error())) + } +} + +// isConflictingPropertyValue returns true if the other attach property has a conflicting value +func isConflictingPropertyValue(appName string, property string, value string) bool { + if value == "" { + return false + } + + otherProperty := "attach-post-create" + if property == otherProperty { + otherProperty = "attach-post-deploy" + } + + defaultValue := GetDefaultValue(otherProperty) + otherValue := common.PropertyGetDefault("network", appName, otherProperty, defaultValue) + + return value == otherValue +} + +// networkExists checks to see if a network exists +func networkExists(networkName string) (bool, error) { + if networkName == "" { + return false, errors.New("No network name specified") + } + + exists := false + + networks, err := listNetworks() + if err != nil { + return false, err + } + + for _, n := range networks { + if networkName == n { + exists = true + break + } + } + + return exists, nil +} + +// listNetworks returns a list of docker networks +func listNetworks() ([]string, error) { + b, err := sh.Command(common.DockerBin(), "network", "list", "--format", "{{ .Name }}").Output() + output := strings.TrimSpace(string(b[:])) + + networks := []string{} + if err != nil { + common.LogVerboseQuiet(output) + return networks, err + } + + networks = strings.Split(output, "\n") + return networks, nil +} diff --git a/plugins/network/network.go b/plugins/network/network.go index d5b671fd1..5fbf2486b 100644 --- a/plugins/network/network.go +++ b/plugins/network/network.go @@ -19,6 +19,9 @@ var ( // DefaultProperties is a map of all valid network properties with corresponding default property values DefaultProperties = map[string]string{ "bind-all-interfaces": "false", + "attach-post-create": "", + "attach-post-deploy": "", + "tld": "", } ) @@ -217,6 +220,8 @@ func ReportSingleApp(appName, infoFlag string) { infoFlags := map[string]string{ "--network-bind-all-interfaces": common.PropertyGet("network", appName, "bind-all-interfaces"), + "--network-attach-post-create": common.PropertyGet("network", appName, "attach-post-create"), + "--network-attach-post-deploy": common.PropertyGet("network", appName, "attach-post-deploy"), "--network-listeners": strings.Join(GetListeners(appName), " "), } diff --git a/plugins/network/src/commands/commands.go b/plugins/network/src/commands/commands.go index b18de6202..fc270c213 100644 --- a/plugins/network/src/commands/commands.go +++ b/plugins/network/src/commands/commands.go @@ -19,6 +19,11 @@ Manages network settings for an app Additional commands:` helpContent = ` + network:create , Creates an attachable docker network + network:destroy , Destroys a docker network + network:exists , Checks if a docker network exists + network:info , Outputs information about a docker network + network:list, Lists all docker networks network:report [] [], Displays a network report for one or more apps network:rebuild , Rebuilds network settings for an app network:rebuildall, Rebuild network settings for all apps diff --git a/plugins/network/src/subcommands/subcommands.go b/plugins/network/src/subcommands/subcommands.go index 75e88ba89..b83d6aa5a 100644 --- a/plugins/network/src/subcommands/subcommands.go +++ b/plugins/network/src/subcommands/subcommands.go @@ -17,6 +17,38 @@ func main() { flag.Parse() switch subcommand { + case "create": + networkName := flag.Arg(1) + err := network.CommandCreate(networkName) + if err != nil { + common.LogFail(err.Error()) + } + case "destroy": + networkName := flag.Arg(1) + forceDestroy := false + if flag.Arg(2) == "force" { + forceDestroy = true + } + err := network.CommandDestroy(networkName, forceDestroy) + if err != nil { + common.LogFail(err.Error()) + } + case "exists": + networkName := flag.Arg(1) + err := network.CommandExists(networkName) + if err != nil { + common.LogFail(err.Error()) + } + case "info": + err := network.CommandInfo() + if err != nil { + common.LogFail(err.Error()) + } + case "list": + err := network.CommandList() + if err != nil { + common.LogFail(err.Error()) + } case "rebuild": appName := flag.Arg(1) network.BuildConfig(appName) diff --git a/plugins/network/src/triggers/triggers.go b/plugins/network/src/triggers/triggers.go index 0f34ca578..24bdd96d0 100644 --- a/plugins/network/src/triggers/triggers.go +++ b/plugins/network/src/triggers/triggers.go @@ -63,12 +63,22 @@ func main() { case "post-app-clone-setup": appName := flag.Arg(1) network.TriggerPostAppCloneSetup(appName) + case "post-container-create": + containerType := flag.Arg(0) + containerID := flag.Arg(1) + appName := flag.Arg(2) + phase := flag.Arg(3) + processType := flag.Arg(4) + network.TriggerPostContainerCreate(containerType, containerID, appName, phase, processType) case "post-create": appName := flag.Arg(0) network.TriggerPostCreate(appName) case "post-delete": appName := flag.Arg(0) network.TriggerPostDelete(appName) + case "core-post-deploy": + appName := flag.Arg(0) + network.TriggerCorePostDeploy(appName) case "report": appName := flag.Arg(0) network.ReportSingleApp(appName, "") diff --git a/plugins/network/subcommands.go b/plugins/network/subcommands.go index 3e5cdc7d2..81812431a 100644 --- a/plugins/network/subcommands.go +++ b/plugins/network/subcommands.go @@ -1,11 +1,124 @@ package network import ( + "bytes" + "errors" + "fmt" + "os" "strings" "github.com/dokku/dokku/plugins/common" ) +// CommandCreate is an alias for "docker network create" +func CommandCreate(networkName string) error { + common.LogInfo1Quiet(fmt.Sprintf("Creating network %v", networkName)) + createCmd := common.NewShellCmd(strings.Join([]string{ + common.DockerBin(), + "network", + "create", + "--attachable", + "--label", + "com.dokku.network=true", + "--label", + fmt.Sprintf("com.dokku.network-name=%v", networkName), + networkName, + }, " ")) + var stderr bytes.Buffer + createCmd.ShowOutput = false + createCmd.Command.Stderr = &stderr + _, err := createCmd.Output() + + if err != nil { + err = errors.New(strings.TrimSpace(stderr.String())) + } + + return err +} + +// CommandDestroy is an alias for "docker network rm" +func CommandDestroy(networkName string, forceDestroy bool) error { + if os.Getenv("DOKKU_APPS_FORCE_DELETE") == "1" { + forceDestroy = true + } + + if !forceDestroy { + common.LogWarn("WARNING: Potentially Destructive Action") + common.LogWarn(fmt.Sprintf("This command will destroy network %v.", networkName)) + common.LogWarn(fmt.Sprintf("To proceed, type \"%v\"", networkName)) + fmt.Print("> ") + var response string + _, err := fmt.Scanln(&response) + if err != nil { + return err + } + + if response != networkName { + common.LogStderr("Confirmation did not match test. Aborted.") + os.Exit(1) + return nil + } + } + + common.LogInfo1Quiet(fmt.Sprintf("Destroying network %v", networkName)) + destroyCmd := common.NewShellCmd(strings.Join([]string{ + common.DockerBin(), + "network", + "rm", + networkName, + }, " ")) + var stderr bytes.Buffer + destroyCmd.ShowOutput = false + destroyCmd.Command.Stderr = &stderr + _, err := destroyCmd.Output() + + if err != nil { + err = errors.New(strings.TrimSpace(stderr.String())) + } + + return err +} + +// CommandExists checks if a network exists +func CommandExists(networkName string) error { + if networkName == "" { + return errors.New("No network name specified") + } + + exists, err := networkExists(networkName) + if err != nil { + return err + } + + if exists { + common.LogInfo1Quiet("Network exists") + } else { + common.LogFail("Network does not exist") + } + + return nil +} + +// CommandInfo is an alias for "docker network inspect" +func CommandInfo() error { + return nil +} + +// CommandList is an alias for "docker network ls" +func CommandList() error { + networks, err := listNetworks() + if err != nil { + return err + } + + common.LogInfo2Quiet("Networks") + for _, networkName := range networks { + fmt.Println(networkName) + } + + return nil +} + // CommandRebuildall rebuilds network settings for all apps func CommandRebuildall() { apps, err := common.DokkuApps() @@ -44,5 +157,24 @@ func CommandSet(appName string, property string, value string) { value = "false" } + attachProperites := map[string]bool{ + "attach-post-create": true, + "attach-post-deploy": true, + } + + invalidNetworks := map[string]bool{ + "host": true, + "bridge": true, + } + if attachProperites[property] { + if invalidNetworks[value] { + common.LogFail("Invalid network name specified for attach") + } + + if isConflictingPropertyValue(appName, property, value) { + common.LogFail("Network name already associated with this app") + } + } + common.CommandPropertySet("network", appName, property, value, DefaultProperties) } diff --git a/plugins/network/triggers.go b/plugins/network/triggers.go index 22f8f3956..8fa11724a 100644 --- a/plugins/network/triggers.go +++ b/plugins/network/triggers.go @@ -161,6 +161,31 @@ func TriggerPostAppCloneSetup(appName string) { } } +// TriggerPostContainerCreate associates the container with a specified network +func TriggerPostContainerCreate(containerType string, containerID string, appName string, phase string, processType string) { + if containerType != "app" { + return + } + + property := "attach-post-create" + defaultValue := GetDefaultValue(property) + networkName := common.PropertyGetDefault("network", appName, property, defaultValue) + if networkName == "" { + return + } + + exists, err := networkExists(networkName) + if err != nil { + common.LogFail(err.Error()) + } + + if !exists { + common.LogFail(fmt.Sprintf("Network %v does not exist", networkName)) + } + + attachAppToNetwork(containerID, networkName, appName, phase, processType) +} + // TriggerPostCreate sets bind-all-interfaces to false by default func TriggerPostCreate(appName string) { err := common.PropertyWrite("network", appName, "bind-all-interfaces", "false") @@ -176,3 +201,36 @@ func TriggerPostDelete(appName string) { common.LogFail(err.Error()) } } + +// TriggerCorePostDeploy associates the container with a specified network +func TriggerCorePostDeploy(appName string) { + property := "attach-post-deploy" + defaultValue := GetDefaultValue(property) + networkName := common.PropertyGetDefault("network", appName, property, defaultValue) + if networkName == "" { + return + } + + common.LogInfo1Quiet(fmt.Sprintf("Associating app with network %s", networkName)) + containerIDs, err := common.GetAppRunningContainerIDs(appName, "") + if err != nil { + common.LogFail(err.Error()) + } + + exists, err := networkExists(networkName) + if err != nil { + common.LogFail(err.Error()) + } + + if !exists { + common.LogFail(fmt.Sprintf("Network %v does not exist", networkName)) + } + + for _, containerID := range containerIDs { + processType, err := common.DockerInspect(containerID, "{{ index .Config.Labels \"com.dokku.process-type\"}}") + if err != nil { + common.LogFail(err.Error()) + } + attachAppToNetwork(containerID, networkName, appName, "deploy", processType) + } +} diff --git a/plugins/scheduler-docker-local/scheduler-deploy b/plugins/scheduler-docker-local/scheduler-deploy index d87c46b41..6e3b6557d 100755 --- a/plugins/scheduler-docker-local/scheduler-deploy +++ b/plugins/scheduler-docker-local/scheduler-deploy @@ -98,18 +98,21 @@ scheduler-docker-local-scheduler-deploy() { START_CMD=$(fn-scheduler-docker-local-extract-start-cmd "$APP" "$PROC_TYPE" "$START_CMD" "$DOKKU_HEROKUISH" "$DOKKU_PORT") if [[ "$DOKKU_NETWORK_BIND_ALL" == "false" ]]; then # shellcheck disable=SC2086 - cid=$("$DOCKER_BIN" container run --label=com.dokku.container-type=deploy "${DOCKER_RUN_LABEL_ARGS[@]}" $DOKKU_GLOBAL_RUN_ARGS --init --detach --env PORT=$DOKKU_PORT "${ARG_ARRAY[@]}" $IMAGE $START_CMD) + cid=$("$DOCKER_BIN" container create --label=com.dokku.container-type=deploy --label=com.dokku.process-type=$PROC_TYPE "${DOCKER_RUN_LABEL_ARGS[@]}" $DOKKU_GLOBAL_RUN_ARGS --init --env PORT=$DOKKU_PORT "${ARG_ARRAY[@]}" $IMAGE $START_CMD) else # shellcheck disable=SC2086 - cid=$("$DOCKER_BIN" container run --label=com.dokku.container-type=deploy "${DOCKER_RUN_LABEL_ARGS[@]}" $DOKKU_GLOBAL_RUN_ARGS --init --detach $DOKKU_DOCKER_PORT_ARGS --env PORT=$DOKKU_PORT "${ARG_ARRAY[@]}" $IMAGE $START_CMD) + cid=$("$DOCKER_BIN" container create --label=com.dokku.container-type=deploy --label=com.dokku.process-type=$PROC_TYPE "${DOCKER_RUN_LABEL_ARGS[@]}" $DOKKU_GLOBAL_RUN_ARGS --init $DOKKU_DOCKER_PORT_ARGS --env PORT=$DOKKU_PORT "${ARG_ARRAY[@]}" $IMAGE $START_CMD) fi else START_CMD=$(fn-scheduler-docker-local-extract-start-cmd "$APP" "$PROC_TYPE" "$START_CMD" "$DOKKU_HEROKUISH") # shellcheck disable=SC2086 - cid=$("$DOCKER_BIN" container run --label=com.dokku.container-type=deploy "${DOCKER_RUN_LABEL_ARGS[@]}" $DOKKU_GLOBAL_RUN_ARGS --init --detach "${ARG_ARRAY[@]}" $IMAGE $START_CMD) + cid=$("$DOCKER_BIN" container create --label=com.dokku.container-type=deploy --label=com.dokku.process-type=$PROC_TYPE "${DOCKER_RUN_LABEL_ARGS[@]}" $DOKKU_GLOBAL_RUN_ARGS --init "${ARG_ARRAY[@]}" $IMAGE $START_CMD) fi + plugn trigger post-container-create "app" "$cid" "$APP" "deploy" "$PROC_TYPE" + "$DOCKER_BIN" container start "$cid" >/dev/null || true + ipaddr=$(plugn trigger network-get-ipaddr "$APP" "$PROC_TYPE" "$cid") port=$(plugn trigger network-get-port "$APP" "$PROC_TYPE" "$DOKKU_HEROKUISH" "$cid") diff --git a/plugins/scheduler-docker-local/scheduler-run b/plugins/scheduler-docker-local/scheduler-run index a91c518ab..cc3f347f7 100755 --- a/plugins/scheduler-docker-local/scheduler-run +++ b/plugins/scheduler-docker-local/scheduler-run @@ -76,6 +76,7 @@ scheduler-docker-local-scheduler-run() { # shellcheck disable=SC2086 CONTAINER_ID="$("$DOCKER_BIN" container create --label=com.dokku.container-type=run "${DOCKER_RUN_LABEL_ARGS[@]}" $DOKKU_GLOBAL_RUN_ARGS "${DOCKER_ARGS_ARRAY[@]}" $IMAGE $EXEC_CMD "$@")" + plugn trigger post-container-create "app" "$CONTAINER_ID" "$APP" "run" declare -a DOCKER_START_ARGS_ARRAY if [[ "$DOKKU_DETACH_CONTAINER" != "1" ]]; then @@ -84,17 +85,17 @@ scheduler-docker-local-scheduler-run() { has_tty && DOCKER_START_ARGS_ARRAY+=" --interactive" fi - local EARLY_EXIT_CODE=0 EXIT_CODE=0 + local EXIT_CODE=0 DOKKU_CONTAINER_EXIT_CODE=0 if [[ "$DOKKU_DETACH_CONTAINER" == "1" ]]; then - "$DOCKER_BIN" container start ${DOCKER_START_ARGS_ARRAY[@]} "$CONTAINER_ID" >/dev/null || CONTAINER_EXIT_CODE=$? + "$DOCKER_BIN" container start ${DOCKER_START_ARGS_ARRAY[@]} "$CONTAINER_ID" >/dev/null || DOKKU_CONTAINER_EXIT_CODE=$? echo "$CONTAINER_ID" else - "$DOCKER_BIN" container start ${DOCKER_START_ARGS_ARRAY[@]} "$CONTAINER_ID" || CONTAINER_EXIT_CODE=$? - EXIT_CODE="$("$DOCKER_BIN" container wait "$CONTAINER_ID" 2>/dev/null || echo "$CONTAINER_EXIT_CODE")" - [[ -z "$EXIT_CODE" ]] && EXIT_CODE=0 + "$DOCKER_BIN" container start ${DOCKER_START_ARGS_ARRAY[@]} "$CONTAINER_ID" || EXIT_CODE=$? + DOKKU_CONTAINER_EXIT_CODE="$("$DOCKER_BIN" container wait "$CONTAINER_ID" 2>/dev/null || echo "$EXIT_CODE")" + [[ -z "$DOKKU_CONTAINER_EXIT_CODE" ]] && DOKKU_CONTAINER_EXIT_CODE=0 fi - plugn trigger scheduler-post-run "$DOKKU_SCHEDULER" "$APP" "$CONTAINER_ID" "$EXIT_CODE" - return "$EXIT_CODE" + plugn trigger scheduler-post-run "$DOKKU_SCHEDULER" "$APP" "$CONTAINER_ID" "$DOKKU_CONTAINER_EXIT_CODE" + return "$DOKKU_CONTAINER_EXIT_CODE" } scheduler-docker-local-scheduler-run "$@" diff --git a/tests/unit/20_network.bats b/tests/unit/20_network.bats index 8b54dc08a..eb7278e56 100644 --- a/tests/unit/20_network.bats +++ b/tests/unit/20_network.bats @@ -13,6 +13,8 @@ teardown() { destroy_app 0 $TEST_APP [[ -f "$DOKKU_ROOT/VHOST.bak" ]] && mv "$DOKKU_ROOT/VHOST.bak" "$DOKKU_ROOT/VHOST" && chown dokku:dokku "$DOKKU_ROOT/VHOST" [[ -f "$DOKKU_ROOT/HOSTNAME.bak" ]] && mv "$DOKKU_ROOT/HOSTNAME.bak" "$DOKKU_ROOT/HOSTNAME" && chown dokku:dokku "$DOKKU_ROOT/HOSTNAME" + docker network rm create-network || true + docker network rm deploy-network || true global_teardown } @@ -41,11 +43,15 @@ assert_external_port() { fi } -@test "(proxy) network:set bind-all-interfaces" { +@test "(network) network:set bind-all-interfaces" { deploy_app assert_nonssl_domain "${TEST_APP}.dokku.me" run /bin/bash -c "dokku network:set $TEST_APP bind-all-interfaces true" + echo "output: $output" + echo "status: $status" + assert_success + run /bin/bash -c "dokku ps:rebuild $TEST_APP" echo "output: $output" echo "status: $status" @@ -57,6 +63,10 @@ assert_external_port() { done run /bin/bash -c "dokku network:set $TEST_APP bind-all-interfaces false" + echo "output: $output" + echo "status: $status" + assert_success + run /bin/bash -c "dokku ps:rebuild $TEST_APP" echo "output: $output" echo "status: $status" @@ -68,7 +78,7 @@ assert_external_port() { done } -@test "(proxy) network host-mode" { +@test "(network) network host-mode" { run /bin/bash -c "dokku docker-options:add $TEST_APP deploy \"--network=host\"" echo "output: $output" echo "status: $status" @@ -84,3 +94,142 @@ assert_external_port() { echo "status: $status" assert_success } + +@test "(network) network management" { + run /bin/bash -c "dokku network:list | grep test-network" + echo "output: $output" + echo "status: $status" + assert_failure + + run /bin/bash -c "dokku network:exists test-network" + echo "output: $output" + echo "status: $status" + assert_failure + + run /bin/bash -c "dokku network:create test-network" + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku network:list | grep test-network" + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku network:create test-network" + echo "output: $output" + echo "status: $status" + assert_failure + + run /bin/bash -c "dokku network:exists test-network" + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku network:exists nonexistent-network" + echo "output: $output" + echo "status: $status" + assert_failure + + run /bin/bash -c "dokku --force network:destroy nonexistent-network" + echo "output: $output" + echo "status: $status" + assert_failure + + run /bin/bash -c "dokku --force network:destroy test-network" + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku --force network:destroy test-network" + echo "output: $output" + echo "status: $status" + assert_failure + + run /bin/bash -c "dokku network:exists test-network" + echo "output: $output" + echo "status: $status" + assert_failure + + run /bin/bash -c "dokku network:list | grep test-network" + echo "output: $output" + echo "status: $status" + assert_failure +} + +@test "(network) network:set attach" { + run deploy_app + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku network:create create-network" + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku network:create deploy-network" + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku network:set $TEST_APP attach-post-create nonexistent-network" + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku network:set $TEST_APP attach-post-deploy nonexistent-network" + echo "output: $output" + echo "status: $status" + assert_failure + + run /bin/bash -c "dokku ps:rebuild $TEST_APP" + echo "output: $output" + echo "status: $status" + assert_failure + assert_http_success "${TEST_APP}.dokku.me" + + run /bin/bash -c "dokku network:set $TEST_APP attach-post-create create-network" + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku network:set $TEST_APP attach-post-deploy create-network" + echo "output: $output" + echo "status: $status" + assert_failure + + run /bin/bash -c "dokku network:set $TEST_APP attach-post-deploy deploy-network" + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku ps:rebuild $TEST_APP" + echo "output: $output" + echo "status: $status" + assert_success + assert_http_success "${TEST_APP}.dokku.me" + + run /bin/bash -c "dokku --force network:destroy create-network" + echo "output: $output" + echo "status: $status" + assert_failure + + run /bin/bash -c "dokku --force apps:destroy $TEST_APP" + echo "output: $output" + echo "status: $status" + assert_success + + # necessary in order to remove networks in use by "dead" containers + docker container prune --force + + run /bin/bash -c "dokku --force network:destroy create-network" + echo "output: $output" + echo "status: $status" + assert_success + + run /bin/bash -c "dokku --force network:destroy deploy-network" + echo "output: $output" + echo "status: $status" + assert_success +}