From 2551171e173df4cfaf37a6cd317be6ba5bcd11ec Mon Sep 17 00:00:00 2001 From: Halvard Skogsrud Date: Sat, 13 Nov 2021 07:03:24 +1100 Subject: [PATCH] docs(debug): Improve Go debugging documentation (#6852) Update explanation of how Go debugging works in Skaffold and add explanations for ko builder users. Also update links to the VS Code Go extension. Related: #6843 Tracking: #6041 --- .../en/docs/pipeline-stages/builders/ko.md | 16 ++- docs/content/en/docs/workflows/debug.md | 135 ++++++++++++------ integration/examples/ko/README.md | 5 +- integration/examples/ko/main.go | 3 +- integration/examples/ko/skaffold.yaml | 4 +- 5 files changed, 116 insertions(+), 47 deletions(-) diff --git a/docs/content/en/docs/pipeline-stages/builders/ko.md b/docs/content/en/docs/pipeline-stages/builders/ko.md index 16ee95523d4..c5969414676 100644 --- a/docs/content/en/docs/pipeline-stages/builders/ko.md +++ b/docs/content/en/docs/pipeline-stages/builders/ko.md @@ -327,7 +327,7 @@ To build images in parallel, consider setting the `SKAFFOLD_BUILD_CONCURRENCY` environment variable value to `0`: ```shell -SKAFFOLD_BUILD_CONCURRENCY=0 skaffold render [...] +SKAFFOLD_BUILD_CONCURRENCY=0 skaffold [...] ``` You can also set the concurrency value in your `skaffold.yaml`: @@ -350,6 +350,20 @@ Images built using `ko` are automatically identified as Go apps by the presence of the [`KO_DATA_PATH` environment variable](https://github.com/google/ko#static-assets). +Skaffold configures `ko` to build with compiler optimizations and inlining +disabled (`-gcflags='all=-N -l'`) when you run `skaffold debug` or use +Cloud Code to +[debug a Kubernetes application](https://cloud.google.com/code/docs/vscode/debug). + +If you debug using VS Code and need to configure a "remote path" or "path on +remote container", then this value should match your local path, typically +`${workspaceFolder}`. The reason is that "remote path" in this case means the +path to your Go source code where it was compiled. The `ko` builder currently +only supports `local` builds, so the remote path will be same as the local path. + +To learn more about how Skaffold debugs Go applications, read the +[Go section in the Debugging guide]({{< relref "/docs/workflows/debug#go-runtime-go-protocols-dlv" >}}). + ### File sync File `sync` is not supported while the ko builder feature is in Alpha. diff --git a/docs/content/en/docs/workflows/debug.md b/docs/content/en/docs/workflows/debug.md index deec2e8933b..effa2d3c10e 100644 --- a/docs/content/en/docs/workflows/debug.md +++ b/docs/content/en/docs/workflows/debug.md @@ -28,7 +28,7 @@ Debugging is currently supported for five language runtimes. - Java and JVM languages (runtime ID: `jvm`) using JDWP - Python 3.5+ (runtime ID: `python`) using `debugpy` (Debug Adapter Protocol) or `pydevd` - .NET Core (runtime ID: `netcore`) using `vsdbg` - + ## How It works @@ -38,9 +38,9 @@ Enabling debugging has two phases: attempts to recognize the underlying language runtime. Container images can be explicitly configured too. 3. **Monitoring:** Skaffold watches the cluster to detect when debuggable containers - start execution. + start execution. -### Configuring container images for debugging +### Configuring container images for debugging `skaffold debug` examines the *built artifacts* to determine the underlying language runtime technology. Kubernetes manifests that reference these artifacts are transformed on-the-fly to enable the @@ -73,7 +73,7 @@ an event that can be used by tools like IDEs to establish a debug session. - *Kubernetes Probes*: `debug` changes the timeouts on HTTP-based [liveness, readiness, and startup probes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) - to 600 seconds (10 minutes) from the default of 1 second. + to 600 seconds (10 minutes) from the default of 1 second. This change allows probes to be debugged, and avoids negative consequences from blocked probes when the app is already suspended during a debugging session. @@ -95,35 +95,45 @@ an event that can be used by tools like IDEs to establish a debug session. This section describes how `debug` recognizes the language runtime used in a container image, and how the container image is configured for debugging. - + Note that many debuggers may require additional information for the location of source files. We are looking for ways to identify this information and to pass it back if found. #### Go (runtime: `go`, protocols: `dlv`) -Go-based applications are configured to run under [Delve](https://github.com/go-delve/delve) in its headless-server mode. +Go-based applications are configured to run under +[Delve](https://github.com/go-delve/delve) in its headless-server mode. Go-based container images are recognized by: -- the presence of one of the [standard Go runtime environment variables](https://godoc.org/runtime): + +- the presence of one of the + [standard Go runtime environment variables](https://godoc.org/runtime): `GODEBUG`, `GOGC`, `GOMAXPROCS`, or `GOTRACEBACK`, or - the presence of the [`KO_DATA_PATH` environment variable](https://github.com/google/ko#static-assets) - in container images built by [`ko`]({{< relref "/docs/pipeline-stages/builders/ko" >}}), or + in container images built by + [`ko`]({{< relref "/docs/pipeline-stages/builders/ko" >}}), or - is launching using `dlv`. -Virtually all container images will need to set one of the Go environment variables. -`GOTRACEBACK=single` is the default setting for Go, and `GOTRACEBACK=all` is a -generally useful configuration. +Unless you built your container image using `ko`, set one of the standard Go +environment variables to allow Skaffold to detect your container as Go. +`GOTRACEBACK=single` is the default setting for Go, so you can add this +environment variable without changing behavior. Alternatively, `GOTRACEBACK=all` +is a useful setting in many situtions. On recognizing a Go-based container image, `debug` rewrites the container image's entrypoint to invoke your application using `dlv`: + ``` dlv exec --headless --continue --accept-multiclient --listen=:56268 --api-version=2 -- ... ``` -Your application should be built with the `-gcflags='all=-N -l'` options to disable optimizations and inlining. -Debugging can be confusing otherwise due to seemingly-random execution jumps from statement reordering and inlining. -Skaffold configures Docker builds with a `SKAFFOLD_GO_GCFLAGS` build argument flag with suitable values: +Your application should be built with the `-gcflags='all=-N -l'` flag to +disable optimizations and inlining. If you do not add this flag, debugging can +becoming confusing due to seemingly-random execution jumps that happen due to +statement reordering and inlining. Skaffold configures Docker builds with a +`SKAFFOLD_GO_GCFLAGS` build argument that you should use in your `Dockerfile`: + ``` FROM golang ENV GOTRACEBACK=all @@ -132,35 +142,78 @@ ARG SKAFFOLD_GO_GCFLAGS RUN go build -gcflags="${SKAFFOLD_GO_GCFLAGS}" -o /app . ``` -Note that the `golang:NN-alpine` container images do not include a C compiler which is required -for `-gcflags='all=-N -l'`. +Note that the Alpine Linux-based `golang` container images do not include a +C compiler which is required to build with `-gcflags='all=-N -l'`. + +The ko builder adds `-gcflags='all=-N -l'` automatically when you use +`skaffold debug`. -Note for users of [VS Code's debug adapter for Go](https://github.com/Microsoft/vscode-go): the debug adapter -may require configuring both the _local_ and _remote_ source path prefixes via the `cwd` and `remotePath` properties. -The `cwd` property should point to the top-level container of your source files and should generally match -the artifact's `context` directory in the `skaffold.yaml`. The `remotePath` path property should be set to the -remote source location _during compilation_. For example, the `golang` images, which are +We recommend that you do _not_ add the `-trimpath` flag when debugging, as this +flag removes information from the resulting binary that aids debugging. + +##### Skaffold debug using the VS Code Go extension + +If you use the debug functionality of the +[VS Code Go extension](https://github.com/golang/vscode-go) to connect to +`dlv`, you may need to configure _local_ and _remote_ source paths, via +either the `cwd` and `remotePath` properties, or the `substitutePath` array +property +([reference](https://github.com/golang/vscode-go/blob/master/docs/debugging-legacy.md#launch-configurations)). + +The `cwd` or `from` properties should be the absolute path of the top-level +directory of your source files and should generally match the artifact's +`context` directory in `skaffold.yaml`. You can represent this using the +`${workspaceFolder}` variable. + +The `remotePath` or `to` properties should be set to the absolute path of the +source location _during compilation_. For example, the `golang` images, which +are [often used in multi-stage builds](https://github.com/GoogleContainerTools/skaffold/tree/main/examples/getting-started/Dockerfile), -copy the source code to `/go`. The following -[remote launch configuration](https://github.com/Microsoft/vscode-go/wiki/Debugging-Go-code-using-VS-Code#remote-debugging) +have a default working directory of `/go`. The following +[configuration](https://github.com/golang/vscode-go/blob/master/docs/debugging-legacy.md#remote-debugging) works in this case: + +```json +{ + "name": "Skaffold Debug", + "type": "go", + "request": "attach", + "mode": "remote", + "host": "localhost", + "port": 56268, + "cwd": "${workspaceFolder}", + "remotePath": "/go" +} +``` + +You can use the `substitutePath` property instead of `cwd` and `remotePath`: + ```json { - "name": "Skaffold Debug", - "type": "go", - "request": "launch", - "mode": "remote", - "host": "localhost", - "port": 56268, - "cwd": "${workspaceFolder}", - "remotePath": "/go/" + "name": "Skaffold Debug", + "type": "go", + "request": "attach", + "mode": "remote", + "host": "localhost", + "port": 56268, + "substitutePath": [ + { + "from": "${workspaceFolder}", + "to": "/go", + }, + ], } ``` +If you use the `ko` local builder, the source path during compilation matches +the source path in your IDE. In this case, the value for `remotePath` will +match `cwd`. If the values of both `cwd` and `remotePath` are +`${workspaceFolder}`, you can omit these properties. + #### Java and Other JVM Languages (runtime: `jvm`, protocols: `jdwp`) Java/JVM applications are configured to expose the JDWP agent using the `JAVA_TOOL_OPTIONS` -environment variable. +environment variable. Note that the use of `JAVA_TOOL_OPTIONS` causes extra debugging output from the JVM on launch. JVM application are recognized by: @@ -184,7 +237,7 @@ NodeJS images are recognized by: On recognizing a NodeJS-based container image, `debug` rewrites the container image's entrypoint to invoke your application with `--inspect`: ``` -node --inspect=9229 +node --inspect=9229 ``` {{< alert title="Note" >}} @@ -196,14 +249,14 @@ and skips scripts located in node_modules. For more details see the associated PR. {{< /alert >}} -Note that a debugging client must first obtain [the inspector UUID](https://github.com/nodejs/node/issues/9185#issuecomment-254872466). +Note that a debugging client must first obtain [the inspector UUID](https://github.com/nodejs/node/issues/9185#issuecomment-254872466). #### Python (runtime: `python`, protocols: `dap` or `pydevd`) Python applications are configured to use either [`debugpy`](https://github.com/microsoft/debugpy/), or wrapper around [`pydevd`](https://github.com/fabioz/PyDev.Debugger). `debugpy` uses the -[_debug adapter protocol_ (DAP)](https://microsoft.github.io/debug-adapter-protocol/) which +[_debug adapter protocol_ (DAP)](https://microsoft.github.io/debug-adapter-protocol/) which is supported by Visual Studio Code, [Eclipse LSP4e](https://projects.eclipse.org/projects/technology.lsp4e), [and other editors and IDEs](https://microsoft.github.io/debug-adapter-protocol/implementors/tools/). @@ -230,12 +283,12 @@ python -m pydevd --server --port 5678 As many Python web frameworks use launcher scripts, like `gunicorn`, Skaffold now uses a debug launcher that examines the app command-line. {{< /alert >}} - + #### .NET Core (runtime: `dotnet`, protocols: `vsdbg`) .NET Core applications are configured to be deployed along with `vsdbg` -for VS Code. +for VS Code. .NET Core application are recognized by: @@ -264,7 +317,7 @@ your base image. (`//` comments must be stripped.) "name": "Skaffold Debug", "type": "coreclr", "request": "attach", - "processId" : 1, + "processId" : 1, "justMyCode": true, // set to `true` in debug configuration and `false` in release configuration "pipeTransport": { "pipeProgram": "kubectl", @@ -301,7 +354,7 @@ your base image. (`//` comments must be stripped.) **Was Skaffold able to recognize the image?** `debug` emits a warning when it is unable to configure an image for debugging: ``` -WARN[0005] Image "image-name" not configured for debugging: unable to determine runtime for "image-name" +WARN[0005] Image "image-name" not configured for debugging: unable to determine runtime for "image-name" ``` See the language runtime section details on how container images are recognized. @@ -360,7 +413,7 @@ appropriate configuration parameters. Each transformed workload object carries a `debug.cloud.google.com/config` annotation with a JSON object describing the debug configurations for the pod's containers (linebreaks for readability): -``` +``` debug.cloud.google.com/config={ "":{"runtime":"",...}, "":{"runtime":"",...}, @@ -387,7 +440,7 @@ debug.cloud.google.com/config={ ### API: Events Each debuggable container being started or stopped raises a _debug-container-event_ through -Skaffold's event mechanism ([gRPC](../references/api/grpc/#debuggingcontainerevent), +Skaffold's event mechanism ([gRPC](../references/api/grpc/#debuggingcontainerevent), [REST](../references/api/swagger/#/SkaffoldService/Events)).
diff --git a/integration/examples/ko/README.md b/integration/examples/ko/README.md index 6186fd98e06..66f49f96886 100644 --- a/integration/examples/ko/README.md +++ b/integration/examples/ko/README.md @@ -1,7 +1,8 @@ ### Example: ko builder -This is an example demonstrating building a Go app with the -[ko](https://github.com/google/ko) builder. +This example uses the +[`ko` builder](https://skaffold.dev/docs/pipeline-stages/builders/ko/) +to build a container image for a Go app. The included [Cloud Build](https://cloud.google.com/build/docs) configuration file shows how users can set up a simple pipeline using `skaffold build` and diff --git a/integration/examples/ko/main.go b/integration/examples/ko/main.go index ce9e997d532..58e43e2bef1 100644 --- a/integration/examples/ko/main.go +++ b/integration/examples/ko/main.go @@ -24,11 +24,10 @@ import ( func main() { http.HandleFunc("/", hello) - log.Println("Listening on port 8080") http.ListenAndServe(":8080", nil) } func hello(w http.ResponseWriter, r *http.Request) { - fmt.Fprintf(w, "Hello, World!") + fmt.Fprintln(w, "Hello, World!") } diff --git a/integration/examples/ko/skaffold.yaml b/integration/examples/ko/skaffold.yaml index f93e7cd91d9..b200cd56844 100644 --- a/integration/examples/ko/skaffold.yaml +++ b/integration/examples/ko/skaffold.yaml @@ -17,4 +17,6 @@ kind: Config build: artifacts: - image: skaffold-ko - ko: {} + ko: + flags: + - -v