Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

helm: templating prometheus url into cronjob heartbeat #11342

Closed
david972 opened this issue Sep 6, 2023 · 0 comments · Fixed by #11343 or #11409
Closed

helm: templating prometheus url into cronjob heartbeat #11342

david972 opened this issue Sep 6, 2023 · 0 comments · Fixed by #11343 or #11409
Labels

Comments

@david972
Copy link
Contributor

david972 commented Sep 6, 2023

What is the issue?

As the prometheus url is hard coded we have to edit it after the deployment if we use external instance or linkerd-viz installed in non-default namespace.

How can it be reproduced?

helm install linkerd-control-plane -n linkerd \
  --set-file identityTrustAnchorsPEM=ca.crt \
  --set-file identity.issuer.tls.crtPEM=issuer.crt \
  --set-file identity.issuer.tls.keyPEM=issuer.key \
  linkerd/linkerd-control-plane

Logs, error output, etc

kubectl get cronjob  

output:

apiVersion: batch/v1
kind: CronJob
metadata:
  ...
  name: linkerd-heartbeat
  namespace: linkerd
...
        spec:
          containers:
          - args:
            - heartbeat
            - -controller-namespace=linkerd
            - -log-level=info
            - -log-format=plain
            - -prometheus-url=http://prometheus.linkerd-viz.svc.cluster.local:9090

output of linkerd check -o short

inkerd-identity
----------------
‼ trust anchors are valid for at least 60 days
    Anchors expiring soon:
	* 498695134056946804576461668678438017343534005643 root.linkerd.cluster.local will expire on 2023-09-30T07:30:25Z
    see https://linkerd.io/2.13/checks/#l5d-identity-trustAnchors-not-expiring-soon for hints
‼ issuer cert is valid for at least 60 days
    issuer certificate will expire on 2023-09-07T20:43:28Z
    see https://linkerd.io/2.13/checks/#l5d-identity-issuer-cert-not-expiring-soon for hints

linkerd-version
---------------
‼ cli is up-to-date
    is running version 2.13.5 but the latest stable version is 2.14.0
    see https://linkerd.io/2.13/checks/#l5d-version-cli for hints

control-plane-version
---------------------
‼ control plane is up-to-date
    is running version 2.13.4 but the latest stable version is 2.14.0
    see https://linkerd.io/2.13/checks/#l5d-version-control for hints
‼ control plane and cli versions match
    control plane running stable-2.13.4 but cli running stable-2.13.5
    see https://linkerd.io/2.13/checks/#l5d-version-control for hints

linkerd-control-plane-proxy
---------------------------
‼ control plane proxies are up-to-date
    some proxies are not running the current version:
	* linkerd-destination-65c9bd846b-m6ctj (stable-2.13.4)
	* linkerd-identity-5c9cf7796c-gspsq (stable-2.13.4)
	* linkerd-proxy-injector-768454944f-k6trm (stable-2.13.4)
	* metrics-api-7cf574988f-jrr5q (stable-2.13.4)
	* tap-65f8b84587-9h9nh (stable-2.13.4)
	* tap-injector-8557f9fdbc-8dnnf (stable-2.13.4)
	* web-695f5ddbc4-9mkdd (stable-2.13.4)
    see https://linkerd.io/2.13/checks/#l5d-cp-proxy-version for hints
‼ control plane proxies and cli versions match
    linkerd-destination-65c9bd846b-m6ctj running stable-2.13.4 but cli running stable-2.13.5
    see https://linkerd.io/2.13/checks/#l5d-cp-proxy-cli-version for hints

linkerd-ha-checks
-----------------
‼ pod injection disabled on kube-system
    kube-system namespace needs to have the label config.linkerd.io/admission-webhooks: disabled if injector webhook failure policy is Fail
    see https://linkerd.io/2.13/checks/#l5d-injection-disabled for hints

linkerd-jaeger
--------------
‼ jaeger extension pods are injected
    could not find proxy container for collector-f4db58985-ks7cp pod
    see https://linkerd.io/2.13/checks/#l5d-jaeger-pods-injection for hints
‼ jaeger injector pods are running
    container "linkerd-proxy" in pod "collector-f4db58985-ks7cp" is not ready
    see https://linkerd.io/2.13/checks/#l5d-jaeger-pods-running for hints
‼ jaeger extension proxies are healthy
    no "linkerd-proxy" containers found in the "linkerd" namespace
    see https://linkerd.io/2.13/checks/#l5d-jaeger-proxy-healthy for hints

linkerd-viz
-----------
‼ linkerd-viz pods are injected
    could not find proxy container for trust-manager-55c864b669-9f4x6 pod
    see https://linkerd.io/2.13/checks/#l5d-viz-pods-injection for hints
‼ viz extension proxies are up-to-date
    some proxies are not running the current version:
	* linkerd-destination-65c9bd846b-m6ctj (stable-2.13.4)
	* linkerd-identity-5c9cf7796c-gspsq (stable-2.13.4)
	* linkerd-proxy-injector-768454944f-k6trm (stable-2.13.4)
	* metrics-api-7cf574988f-jrr5q (stable-2.13.4)
	* tap-65f8b84587-9h9nh (stable-2.13.4)
	* tap-injector-8557f9fdbc-8dnnf (stable-2.13.4)
	* web-695f5ddbc4-9mkdd (stable-2.13.4)
    see https://linkerd.io/2.13/checks/#l5d-viz-proxy-cp-version for hints
‼ viz extension proxies and cli versions match
    linkerd-destination-65c9bd846b-m6ctj running stable-2.13.4 but cli running stable-2.13.5
    see https://linkerd.io/2.13/checks/#l5d-viz-proxy-cli-version for hints
‼ prometheus is installed and configured correctly
    missing ClusterRoles: linkerd-linkerd-prometheus
    see https://linkerd.io/2.13/checks/#l5d-viz-prometheus for hints

Status check results are √

Environment

  • Kubernetes Version: v1.26.7-eks-2d98532
  • Cluster Environment: EKS
  • Host OS: Amazon Linux 2
  • Linkerd version: 2.13.4

Possible solution

Adding prometheusUrl key in the control-plane chart like viz chart can be fix the issue

Additional context

No response

Would you like to work on fixing this bug?

yes

@david972 david972 added the bug label Sep 6, 2023
david972 added a commit to w6d-io/linkerd2 that referenced this issue Sep 6, 2023
Can not use external prometheus with hearbeat

Added new variable `.prometheusUrl` in value and use it into heartbeat from linkerd-control-plane chart

Run `helm template` check the cronjob manifest

Fixes linkerd#11342

Signed-off-by: David ALEXANDRE <[email protected]>
david972 added a commit to w6d-io/linkerd2 that referenced this issue Sep 16, 2023
Can not use external prometheus with hearbeat

Added new variable `.prometheusUrl` in value and use it into heartbeat from linkerd-control-plane chart

Run `helm template` check the cronjob manifest

Fixes linkerd#11342

Signed-off-by: David ALEXANDRE <[email protected]>
mateiidavid pushed a commit that referenced this issue Sep 19, 2023
Can not use external prometheus with hearbeat. This change adds a new variable `.prometheusUrl` in value and use it into heartbeat from linkerd-control-plane chart.

Fixes #11342

Signed-off-by: David ALEXANDRE <[email protected]>
mateiidavid added a commit that referenced this issue Sep 22, 2023
This edge release updates the proxy's dependency on the `rustls` library to
patch security vulnerability [RUSTSEC-2023-0052]  (GHSA-8qv2-5vq6-g2g7), a
potential CPU usage denial-of-service attack when acceting a TLS handshake from
an untrusted peer with a maliciously-crafted certificate. Furthermore, this
edge release contains a few improvements to the control plane and jaeger
extension Helm charts.

* Addressed security vulnerability [RUSTSEC-2023-0052] in the proxy by updating
  its dependency on the `rustls` library
* Added a `prometheusUrl` field for the heartbeat job in the control plane Helm
  chart (thanks @david972!) ([#11343]; fixes [#11342])
* Introduced support for arbitrary labels in the `podMonitors` field in the
  control plane Helm chart (thanks @jseiser!) ([#11222]; fixes [#11175])
* Added support for config merge and Deployment environment to
  `opentelemetry-collector` in the jaeger extension (thanks @iAnomaly!)
  ([#11283])

[#11283]: #11283
[#11222]: #11222
[#11175]: #11175
[#11343]: #11343
[#11342]: #11342

Signed-off-by: Matei David <[email protected]>
mateiidavid added a commit that referenced this issue Sep 22, 2023
* edge-29.9.3

This edge release updates the proxy's dependency on the `rustls` library to
patch security vulnerability [RUSTSEC-2023-0052]  (GHSA-8qv2-5vq6-g2g7), a
potential CPU usage denial-of-service attack when acceting a TLS handshake from
an untrusted peer with a maliciously-crafted certificate. Furthermore, this
edge release contains a few improvements to the control plane and jaeger
extension Helm charts.

* Addressed security vulnerability [RUSTSEC-2023-0052] in the proxy by updating
  its dependency on the `rustls` library
* Added a `prometheusUrl` field for the heartbeat job in the control plane Helm
  chart (thanks @david972!) ([#11343]; fixes [#11342])
* Introduced support for arbitrary labels in the `podMonitors` field in the
  control plane Helm chart (thanks @jseiser!) ([#11222]; fixes [#11175])
* Added support for config merge and Deployment environment to
  `opentelemetry-collector` in the jaeger extension (thanks @iAnomaly!)
  ([#11283])

[#11283]: #11283
[#11222]: #11222
[#11175]: #11175
[#11343]: #11343
[#11342]: #11342

Signed-off-by: Matei David <[email protected]>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 20, 2023
adleong pushed a commit that referenced this issue Feb 17, 2024
Can not use external prometheus with hearbeat. This change adds a new variable `.prometheusUrl` in value and use it into heartbeat from linkerd-control-plane chart.

Fixes #11342

Signed-off-by: David ALEXANDRE <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant