bug in _scan-job.tpl?

[corbosman]

Hi, I ran into an issue where I could not install datree using fluxcd. I got this error:

Helm install failed: error while running post render on files: map[string]interface {}(nil): yaml: unmarshal errors:
  line 42: mapping key "seccompProfile" already defined at line 40

This seems to be caused by scan-job.tpl which defines seccompProfile at line 67, but just before it loads values.yaml which also includes seccompProfile. It might be a bug in fluxcd helm-controller. I was able to workaround it by defining seccompProfile: null in my values config.

[noaabarki]

Hi @corbosman! Thank you for sharing this with us.

Could you please provide the steps you took before encountering the error? How did you install Datree on your cluster, and which version did you use? Any detail will be helpful for me to reproduce the bug on our systems. 🙏🏼

[corbosman]

Hi, I used fluxcd to configure a helm repository and a helm release. This then failed due to the error in the first message. If I add the securitycontext workaround it does install properly, but most people running fluxcd won't know this.

---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
  name: datree-webhook
  namespace: datree
spec:
  interval: 30m
  url: https://datreeio.github.io/admission-webhook-datree

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: datree
  namespace: flux-system
spec:
  releaseName: datree-webhook
  targetNamespace: datree
  chart:
    spec:
      chart: datree-admission-webhook
      sourceRef:
        kind: HelmRepository
        name: datree-webhook
        namespace: flux-system
  interval: 5m
  install:
    remediation:
      retries: 3
  values:
    datree:
      token: "xxx"
      clusterName: k3s
    securityContext:
      seccompProfile: null

[noaabarki]

Hi there @corbosman,

I noticed that you configured the HelmRepository in the datree namespace, but in the HelmRelease, you pulled it from the flux-system namespace. Additionally, you configured the HelmRelease in the flux-system namespace. Can you please explain why you did this? Is it possible that you accidentally configured some of the namespaces incorrectly?

I have updated the resources to use the datree namespace. Please try installing the chart using the following resources and let me know if it works.

kind: HelmRelease
metadata:
  name: admission-webhook-hr-datree
  namespace: datree
spec:
  releaseName: datree-webhook
  targetNamespace: datree
  chart:
    spec:
      chart: datree-admission-webhook
      sourceRef:
        kind: HelmRepository
        name: admission-webhook-datree
        namespace: datree
  interval: 5m
  install:
    remediation:
      retries: 3
  values:
    datree:
      token: "xxx"
      clusterName: "yyy"
---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
  name: admission-webhook-datree
  namespace: datree
spec:
  interval: 30m
  url: https://datreeio.github.io/admission-webhook-datree

[corbosman]

That makes no difference, I put all "flux" resources in the flux-system namespace. But I could also put it in datree namespace. It's kinda arbitrary. It's probably better to put the release in the datree namespace.

targetNamespace: datree

That makes sure the app itself gets installed in datree namespace. And this works fine, im running datree right now using the workaround. It could very well be that the actual bug is inside the FluxCD Helmcontroller, that it can't handle duplicate keys like you're currently creating with seccompProfile.

[noaabarki]

I tried to install datree via FluxCD, using the resources configured in the way you specified. However, the installation failed due to the following error: "failed to get source: HelmRepository.source.toolkit.fluxcd.io "datree-webhook" not found". This occurred both with and without setting the seccompProfile. However, after updating the resources with the correct namespaces, everything worked fine.

It's possible that FluxCD has an issue with duplicate keys in the helm chart, as discussed here. However, this shouldn't affect the datree helm chart. Can you explain why you believe the seccompProfile is duplicated?

[corbosman]

From _scan_job.tpl:

          securityContext:
            {{- with .Values.securityContext }}
            {{ toYaml . | nindent 12 }}
            {{- end }}
            seccompProfile:
              type: RuntimeDefault

Then from your default values.yaml (which is include above I guess?)

securityContext:
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: 25000
  capabilities:
    drop: [ "ALL" ]
  seccompProfile:
    type: RuntimeDefault

As you can see, seccompProfile is added twice.