add init container for updating ca trust and shift getting ca cert from secret to config map

[mittal-ishaan]

What does this PR change?

• Create a init container to update ca trust which we earlier expected the user to do inside the container itself.
  Taken reference from https://jfrog.com/help/r/artifactory-how-to-add-custom-ssl-certificates-to-an-artifactory-pod-using-helm-charts/artifactory-how-to-add-custom-ssl-certificates-to-an-artifactory-pod-using-helm-charts

Does this PR rely on any other PRs?

#3760

How does this PR impact users? (This is the kind of thing that goes in release notes!)

Users will now not need to run update-ca-trust command inside the container. The init container will do it.

Links to Issues or tickets this PR addresses or fixes

What risks are associated with merging this PR? What is required to fully test this PR?

How was this PR tested?

Deployed the kubecost deployment with caCertsSecret helm value set and ca cert as Secret created.
reading through /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt we see it updated with the given ca cert in Secret.

helm values:

global:
  updateCaTrust:
    enabled: false  # Set to true to enable the init container for updating CA trust
    # Security context settings for the init container.
    securityContext:
      runAsUser: 0
      runAsGroup: 0
      runAsNonRoot: false
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      seccompProfile:
        type: RuntimeDefault
    caCertsSecret: ca-certs-secret  # The name of the Secret containing custom CA certificates to mount to the cost-model container.
    # caCertsConfig: ca-certs-config  # The name of the ConfigMap containing the CA trust configuration.
    resources: {}  # Resource requests and limits for the init container.
    caCertsMountPath: /etc/pki/ca-trust/source/anchors  # The path where the custom CA certificates will be mounted in the init container

Have you made an update to documentation? If so, please provide the corresponding PR.

[jessegoodier]

This is looking good to me. Tested with custom secret name.

Will ping customer and merge after we get feedback.

[chipzoller]

But why a secret? Root certificates aren't sensitive.

[mittal-ishaan]

@chipzoller The user requested it to be a secret as they store/retrieve certs as secrets internally.

[chipzoller]

Then maybe make this so it accepts a Secret or ConfigMap?

[mittal-ishaan]

sure, we should do that. let me get that in.

[thomasvn]

Future TODO. This updates the CA trust and certificates for the cost-analyzer pod. But it's likely that users would need to perform the same updates across other pods like the aggregator and cloudcost pod.

[jessegoodier]

@mittal-ishaan great work here. Thanks for all of the iterations while we figured out the best approach.

Agree with Thomas that we should add to aggregator and cloud cost when you have a moment.

I'm now questioning if we should have put this under a more general section instead of nesting under kubecostModel ?

@chipzoller and @thomasvn what are your thoughts?

[thomasvn]

Yes! I like the idea of doing .Values.updateCaTrust instead of .Values.kubecostModel.updateCaTrust.

[chipzoller]

Also agree that I'd try and pull this out and generalize it, but I'm not sure the root level is the best place.

You guys have any other suggestions for where these values may best go?

[thomasvn]

Only other place that seems suitable is under .Values.global. I'm open to other ideas though?

[jessegoodier]

Only other place that seems suitable is under .Values.global. I'm open to other ideas though?

I'm thinking global. is better than productConfigs

[thomasvn]

This looks good to me. These configs are now available under .Values.global.updateCaTrust.

Approving for immediate merge so fixes are available to customer. However we now need a follow-up PR to make similar changes to the Kubecost Aggregator and Kubecost CloudCost pods, then test those changes.

[jessegoodier]

Thanks Thomas. great work here team.

[jessegoodier]

/cherry-pick v2.5

[gcp-cherry-pick-bot]

Cherry-pick failed with Merge error 3eaddf172a83f530b881317b6945173e77751715 into temp-cherry-pick-debe5d-v2.5