A very useful setup for debugging porter development

[richbai90]

I've recently undertaken several development projects relating to Porter, from working with the team to implement new core features, to developing custom Mixins around those features. Unit tests are of course extremely valuable, but sometimes it is nice to step through the execution of your bundle to debug exactly what is happening. With a simple docker file, a bash script, and some vscode configuration, a debug setup can be made to step through the code just as though it were executed via the Porter CLI. I hope this information helps you as much as it has me.

Setup steps

1. Place the Dockerfile at the root of your bundle directory along with the porter debug script.
2. Run the debug script and wait for the bash prompt to appear.
3. Launch visual studio code (install remote-containers extension if not already installed)
4. Launch the command palette (mod+shift+p) and search for "attach to running container"

5. Select the highlighted entry from the image above
6. Select your debug container from the list of running containers
7. Once inside the container open the command palette again (mod+shift+p)

8. Select the entry from the image above and populate the resulting file with the devcontainer.json file below
9. Stop the running container from step 2, and then rerun it by executing the debug script as before.
10. In the open workspace open the .vscode folder. (Create it if it doesn't exist)
11. Add the applicable tasks.json and launch.json config files from the files section below.12.
12. Finally, you should have a launch file debug option you can now use to step through your code just as it is executed using porter commands.

Files

Note: Any text in the files that follow, surrounded in {}, where the curly braces do not constitute appropriate syntax -- as in the case of the ${} notation in the Dockerfile -- should be replaced with the actual string referenced. If it appears in a comment, replacement is not necessary, but should be understood to mean the same.

Dockerfile

# The variant of debian to use when pulling the devcontainer image from MS
ARG VARIANT="1.18-bullseye"
# The porter image
ARG IMG
# Pull in the go devcontainer image from MS
FROM mcr.microsoft.com/vscode/devcontainers/go:0-${VARIANT}
# Pull in the porter image
FROM ${IMG}
# The group id of the docker group on the host system
ARG DOCKERID
# Set the GOPATH
ENV GOPATH /home/nonroot/go
# Add go to the system path
ENV PATH "$PATH:/usr/local/go/bin"
# MS installs go as root so these steps must be done as Root user
USER root
# Copy gopath from devcontainer root to porter nonroot user home directory 
COPY --from=0 /go /home/nonroot/go
# Copy go bin folder from devcontainer to porter
COPY --from=0 /usr/local/go /usr/local/go
# Correct goroot ownership and install git
RUN chown -R nonroot:0 /home/nonroot && \
    apt-get update && \
    apt-get install -y git && \
    # Create docker group with correct ID and add non root user to said group
    groupadd --gid ${DOCKERID} docker && \
    usermod -aG docker nonroot

# Set the user back to nonroot as expected by porter runtime
USER nonroot

porter-debug.sh

#!/bin/bash
# Build the runtime container and extract the image name and tag
img=$(porter build --debug 2>&1 | grep "naming to" | awk '{print $4}')
# Get the image name as its own var
imgname=$(docker images "$img" | awk '(NR>1) {print $1}')
# Create a friendly name equal to {bundle.name}-debug
friendlyname=$(echo "$img" | awk -F '/' '{print $2}' | awk -F ':' '{print $1}')-debug
# Get the ID of the group that owns the docker socket (so that we can access the socket via the nonroot user)
dockerid=$(stat -c '%g' /var/run/docker.sock)

# Build the debug image with the name/tag {bundle.repository}/{bundle.name}:debug
docker build -t "$imgname:debug" --build-arg IMG="$img" --build-arg DOCKERID="$dockerid" .
# Run the image interactively
docker run -it \
           --rm \
          # mount ssh service. May not be required but can be useful
           --mount type=bind,src=/run/host-services/ssh-auth.sock,target=/run/host-services/ssh-auth.sock \
          # container name is {bundle.name}-debug as defined above
           --name "$friendlyname" \
          # change these mounts to meet your needs
           -v {mixin directory}:/mixin \
           -v {local porter source dir}:/porter \
          # mounting docker socket may not be always required, but is common enough to include it always
           -v /var/run/docker.sock:/var/run/docker.sock \
          # use the debug image and run a bash shell
           "$imgname:debug" /bin/bash

devcontainer.json

{
    /* 
        Extensions to install at launch. Only golang.go is required, 
        but I end up writing some amount of shell scripts in almost every project I work on
        so I include the various bash extensions. I also include gitlens, as I find it to be a 
        very useful tool for understanding someone else's code.
    */
    "extensions": [
		"eamodio.gitlens",
		"foxundermoon.shell-format",
		"golang.go",
		"lizebang.bash-extension-pack",
		"mads-hartmann.bash-ide-vscode",
		"ms-vscode.powershell",
		"Remisa.shellman",
		"rogalmic.bash-debug",
		"rpinski.shebang-snippets",
		"timonwong.shellcheck"
	],
        // Set this to wherever your mixin source is mounted, or to where the porter source is mounted, or use a predefined workspace
       // See https://code.visualstudio.com/docs/remote/containers#_open-an-existing-workspace-in-a-container
	"workspaceFolder": "/mixin"
}

/mixin/.vscode/tasks.json

{
    // See https://go.microsoft.com/fwlink/?LinkId=733558
    // for the documentation about the tasks.json format
    "version": "2.0.0",
    "tasks": [
        {
            "label": "build-debug",
            "type": "shell",
            "command": "CGO_ENABLED=0 go build -gcflags 'all=-N -l' -o /cnab/app/mixins/{mixin.name}/runtimes/xfer-runtime $(ls ${workspaceFolder}/cmd/{mixin.name}/*.go | grep -v 'test' | xargs)"
        }
    ]
}

/mixin/.vscode/launch.json

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Launch file",
            "type": "go",
            "request": "launch",
            "mode": "exec",
            "cwd": "/cnab/app",
            "program": "/cnab/app/mixins/{mixin.name}/runtimes/{mixin.name}-runtime",
            // run a porter command (build, install, exec ...)
            "args": [
                "build",
                "--debug"
            ],
            "preLaunchTask": "build-debug",
            "env": {
                "PORTER_HOME": "/cnab/app",
                // Some actions expect to read from stdin. Rather than provide the config via the args section,
               // I use code that looks for this env var and creates a pipe of sample data to read from on the fly. 
              // See notes section
                "debugger": ""
            },
        },
        {
            "name": "Attach",
            "type": "go",
            "request": "attach",
            "mode": "remote",
            "remotePath": "",
            "port": 2345,
            "host": "127.0.0.1",
            "showLog": true,
            "trace": "log",
            "logOutput": "rpc"
        }
    ]
}

/porter/.vscode/tasks.json

{
    // See https://go.microsoft.com/fwlink/?LinkId=733558
    // for the documentation about the tasks.json format
    "version": "2.0.0",
    "tasks": [
        {
            "label": "build-debug",
            "type": "shell",
            "command": "CGO_ENABLED=0 go build -gcflags 'all=-N -l' -o /cnab/app/runtimes/porter-runtime $(ls ${workspaceFolder}/cmd/porter/*.go | grep -v 'test' | xargs)"
        }
    ]
}

/porter/.vscode/launch.json

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Launch file",
            "type": "go",
            "request": "launch",
            "mode": "exec",
            "cwd": "/cnab/app",
            "program": "/cnab/app/runtimes/porter-runtime",
            "args": [
                "build",
                "-f",
                "/cnab/app/porter.yaml",
            ],
            "preLaunchTask": "build-debug",
            "env": {
                "PORTER_HOME": "/cnab/app"
            }
        },
        {
            "name": "Launch Tests",
            "type": "go",
            "request": "launch",
            "mode": "test",
            "program": "${file}",
            "env": {
                "LOG_LEVEL": "debug"
            },
            "args": [
                "-test.run",
                "TestRebuild_InstallNewBundle"
            ],
            "showLog": true,
        },
        {
            "name": "Attach",
            "type": "go",
            "request": "attach",
            "mode": "remote",
            "remotePath": "",
            "port": 2346,
            "host": "127.0.0.1",
            "showLog": true,
            "trace": "log",
            "logOutput": "rpc"
        }
    ]
}

Final notes

Some mixin actions expect to read snippets of the yaml file from stdin. For these actions you can create a reader to use instead of stdin inside your mixin code. When you do this, you will want to protect this code from running during a production build. The porter debug flag can be run by any user in any environment so it is not a good idea to rely on this flag to supply testing data to the mixin. Instead, I opt to set an env variable in my launch config and check for it. This is a pattern similar to how these actions are tested in the built-in template tests. Here is an example from a mixin I'm developing:

func BuildInstallCommand(m *xfer.Mixin) *cobra.Command {
       // Create a pipe to read from. Could be a file or anything else that implements the io.Reader interface
      // Just be careful using a pipe because reading blocks the thread
	r, w, _ := os.Pipe()
	cmd := &cobra.Command{
		Use:   "install",
		Short: "Execute the install functionality of this mixin",
		PreRunE: func(cmd *cobra.Command, args []string) error {
			// Handle debug pipe
			if _, dbg := os.LookupEnv("debugger"); dbg {
                                // Write the snippet that action expects to the pipe
				w.WriteString("install:\n  - xfer:\n      description: File Transfer\n      destination: /Users/Rich/restore\n")
                               // Set the porter input stream to be the reader
				m.Context.In = r
			}
                       // Close the writer. If using a generic pipe, probably best not to defer
			defer w.Close()
			return nil
		},
		RunE: func(cmd *cobra.Command, args []string) error {
                       // Close the reader
			defer r.Close()
			return m.Execute()
		},
	}
	return cmd
}

[richbai90]

Made a few minor changes to the Dockerfile and debug script to correct some permissions and build errors. Errors were minor and could probably be fixed by anyone using the script, but it should all be correct now.