HelmRelease Values from targetPath array

[robertpcontreras-ts]

Describe the bug

Hello, I cannot seem to be able to use the targetFrom field to access an existing array within the valuesFrom object in a helm release.

Steps to reproduce

In the main values object of a helm release I have the following:

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: release-name
  namespace: flux-system
spec:
  values:
    confd:
      file.yaml:
        init_config:
          service: kafka-consumers
        instances:
          - kafka_connect_str: xxx:9092
            monitor_unlisted_consumer_groups: true

I am then trying to add secret values to the first item of the instances array like so:

  valuesFrom:
    - kind: Secret
      name: secrets
      valuesKey: username
      targetPath: confd.file\.yaml.instances[0].sasl_plain_username
    - kind: Secret
      name: secrets
      valuesKey: password
      targetPath: confd.file\.yaml.instances[0].sasl_plain_password

But this is not working, and the new fields are not being applied to the helm release values after inspecting the output using

helm -n flux-system get values release-name

Expected behavior

I would expect the sasl_plain_username and sasl_plain_password values to be merged with the first item in the instances array.

Screenshots and recordings

No response

OS / Distro

GKE

Flux version

v0.24.1

Flux check

❯ flux check
► checking prerequisites
✔ Kubernetes 1.20.10-gke.1600 >=1.19.0-0
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v0.14.1
✔ image-automation-controller: deployment ready
► ghcr.io/fluxcd/image-automation-controller:v0.18.0
✔ image-reflector-controller: deployment ready
► ghcr.io/fluxcd/image-reflector-controller:v0.14.0
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v0.18.2
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v0.19.0
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v0.19.2
✔ all checks passed

Git provider

GitHub

Container Registry provider

gcr

Additional context

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[mchlumsky]

@robertpcontreras-ts I think you might want to open this issue against this repository as well: https://github.com/fluxcd/helm-controller . Looks like the code that handles HelmReleases lives in there.

I am also seeing this bug.

❯ flux check
► checking prerequisites
✔ Kubernetes 1.20.6 >=1.20.6-0
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v0.11.2
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v0.13.3
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v0.15.1
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v0.15.4
✔ all checks passed

[Kirgod]

Try this "hack":

1. Create configMap and put there all values from HelmChart:

apiVersion: v1
kind: ConfigMap
metadata:
  name: kafka-cm
data:
  values.yaml: |
      confd:
        file.yaml:
          init_config:
            service: kafka-consumers
          instances:
            - kafka_connect_str: xxx:9092
              monitor_unlisted_consumer_groups: true
              ...

2. Update Helm Release yaml file by removing values from HelmChart file and add configMap+secrets. Also make sure that first you are attaching configMap and only then secret-s.
   Example:

spec:
  valuesFrom:
    - kind: ConfigMap
      name: kafka-cm
      valuesKey: values.yaml
    - kind: Secret
      name: secrets
      valuesKey: username
      targetPath: confd.file\.yaml.instances[0].sasl_plain_username
    - kind: Secret
      name: secrets
      valuesKey: password
      targetPath: confd.file\.yaml.instances[0].sasl_plain_password

[stealthybox]

spec.values is documented to merge after valuesFrom
(maybe we made the wront behavior/API choice here seeing as there are two issues here with a misconception of this + Secrets/Configs typically have fewer, more specific, sensitive fields + the --set/valuesFrom style supports nested keys and array indexing)

In your example, your instances list is first being built by valuesFrom, and then being overriden by the instances in your spec.values.

The above workaround of using a ConfigMap to store your values file instead of using the incline .spec.values will solve your problem because it allows you to change the default ordering to meet your needs.