Provide option to disable debugging for specific modules/artifacts

[gsquared94]

Skaffold should provide a way to disable debugging for specific modules/artifacts.

_Originally posted by @akostic-kostile in https://github.com//issues/2203#issuecomment-938752959_

One of the modules I'm working on is really a pod that consists of 2 containers. I'll post Dockerfiles just so you get the idea what we're working with here:

reporting-app (file cut for brevity, I just pasted parts that are important for local dev):

FROM mcr.microsoft.com/dotnet/sdk:5.0.401-buster-slim as base

WORKDIR /src/

## Install Sonar Scanner and Java (required for Sonar Scanner), libgdiplus (required for Simulsoft to work in local dev)
ENV JAVA_VERSION=11.0.12+7-2~deb10u1 \
        LIBGDIPLUS_VERSION=4.2-2 \
        SONAR_SCANNER_VERSION=5.3.1
RUN apt-get update \
        && apt-get install -y --no-install-recommends openjdk-11-jre="$JAVA_VERSION" libgdiplus="$LIBGDIPLUS_VERSION" \
        && dotnet tool install --global dotnet-sonarscanner --version "$SONAR_SCANNER_VERSION" \
        && rm -rf /var/lib/apt/lists/*
ENV PATH="$PATH:/root/.dotnet/tools"

ENV DOTNET_CONFIGURATION=Release \
    DOTNET_FRAMEWORK=net5.0 \
    DOTNET_RUNTIME=linux-x64
# cache dotnet restore in a separate layer to speed up builds
COPY *.sln */*.csproj nuget.config ./
RUN for file in $(ls *.csproj); do mkdir -p ${file%.*} && mv $file ${file%.*}/; done \
        && dotnet restore --runtime "$DOTNET_RUNTIME" ThirdEyes.sln

COPY . .


FROM base as local

ARG DOTNET_WATCH_ARGS
ENV DOTNET_WATCH_ARGS=$DOTNET_WATCH_ARGS

EXPOSE 5002
CMD [ "/bin/sh", "-exc", "dotnet watch --project Reporting.App -- $DOTNET_WATCH_ARGS " ]

Highcharts-export-server (only thing that gets installed with npm install is "highcharts-export-server": "^2.1.0"):

FROM node:12-slim

RUN apt-get update && apt-get install -y --no-install-recommends bzip2 libfontconfig1 && rm -rf /var/lib/apt/lists/*

WORKDIR /home/node
COPY package.json package-lock.json ./

ENV ACCEPT_HIGHCHARTS_LICENSE=1
RUN npm install

USER 1000:1000
EXPOSE 7801
CMD ["/home/node/node_modules/.bin/highcharts-export-server", "--enableServer", "1"]

And finally skaffold.yml:

apiVersion: skaffold/v2beta21
kind: Config
metadata:
  name: _reporting
profiles:
  - name: local
    build:
      artifacts:
        - image: registry.localhost:8082/reporting-app
          context: "."
          docker:
            dockerfile: Reporting.App/Dockerfile
            buildArgs:
              DOTNET_WATCH_ARGS: run
            # noCache: true
            target: local
          sync:
            manual:
              - dest: "."
                src: "**/*.cs"
        - image: registry.localhost:8082/highcharts-export-server
          context: Reporting.App/HighchartsExportServer
      insecureRegistries:
        - registry.localhost:8082
      local:
        concurrency: 0
        push: true
        tryImportMissing: true
        useBuildkit: true
      tagPolicy:
        ## for local dev only inputDigest policy makes sense as it will rebuild image if there are changes in workdir
        ## gitCommit policy only takes into consideration committed changes.
        inputDigest: {}
    deploy:
      kustomize:
        defaultNamespace: default
        paths:
          - ../kubernetes-manifests/local-cluster/default/reporting/
      statusCheck: true
      statusCheckDeadlineSeconds: 60
    portForward:
      - resourceType: Service
        resourceName: reporting-svc-local
        namespace: default
        port: 5002
        localPort: 5002
      - resourceType: Service
        resourceName: reporting-svc-local
        namespace: default
        port: 7801
        localPort: 7801

Now, what happens when I do skaffold debug --profile local --module _reporting ? Couple of things, almost all of them wrong. :)

Lets take a look at pod annotations:

metadata:
  annotations:
    debug.cloud.google.com/config: '{"highcharts":{"artifact":"registry.localhost:8082/highcharts-export-server","runtime":"nodejs","workingD
ir":"/home/node","ports":{"devtools":9229}},"reporting":{"artifact":"registry.localhost:8082/reporting-app","runtime":"jvm","workingDir":"/sr
c/","ports":{"jdwp":5005}}}'

For highcharts container runtime was correctly identified, but the problem is I don't care about debugging Highcharts export server. Is there a way to tell Skaffold not to debug that container? The only way I know of is to manually set pod level annotation to debug.cloud.google.com/config: {}, however this is far from ideal. It feels very awkward to pass configuration options to Skaffold through pod annotations, second problem is this will also stop me from debugging reporting container.

Leaving that aside for now, lets take a look at what Skaffold detected for a second container. Hm, runtime jvm?! Must be because I have a variable inside Dockerfile called JAVA_VERSION. I'm installing Java so SonarQube scanner would work correctly, and JAVA_VERSION is one of the special variables Skaffold uses to detect runtime. However in this case it completely misses the mark. Can I change the variable name to something else? Sure I can, but I created this Dockerfile before I started working with skaffold debug, I'm just trying to make a point here that current methods of detecting runtime are far from ideal

To conclude this long post there should definitely be some way to set these things from skaffold.yml, at the very least to enable/disable debugging on container level and a way to specify runtime so correct debug tools would be installed. At version 1.32 debug is very rough around the edges.

[akostic-kostile]

Your link is broken, here's the correct one:
#2203 (comment)

[MarlonGamez]

@gsquared94 mentioned that we were awaiting some comment on @briandealwis on this issue, as there was potentially some reason he had mentioned as to why we wouldn't be able to do this. Will ping Brian for input

[adawalli]

I am seeing a similar issue to this one. I am running a PYTHON based container, however, some environment variables are set (like JAVA_HOME) for some commands we must exec. This causes skaffold debug to assume this is a JAVA based artifact, when in fact the primary workload is python.