jagger-operator 1.32.0 fails to start successfully

[ayaranga]

Describe the bug
Trying to install jagger-operator 1.32.0 using jaeger-operator-2.30.0 helm chart. Following error is seen in the jaeger-operator POD logs:

2022-04-11T01:33:17.406Z	ERROR	controller-runtime.manager.controller.jaeger	Could not wait for Cache to sync	{"reconciler group": "jaegertracing.io", "reconciler kind": "Jaeger", "error": "failed to wait for jaeger caches to sync: timed out waiting for cache to be synced"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:195
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:221
sigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).startRunnable.func1
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/manager/internal.go:696
2022-04-11T01:33:17.408Z	ERROR	controller-runtime.manager	error received after stop sequence was engaged	{"error": "failed to wait for deployment caches to sync: timed out waiting for cache to be synced"}
2022-04-11T01:33:17.408Z	ERROR	controller-runtime.manager	error received after stop sequence was engaged	{"error": "failed to wait for jaeger caches to sync: timed out waiting for cache to be synced"}
time="2022-04-11T01:33:17Z" level=fatal msg="open /tmp/k8s-webhook-server/serving-certs/tls.crt: no such file or directoryManager exited non-zero"

To Reproduce
Steps to reproduce the behavior:

1. Install jaeger-operator using helm

    git checkout jaeger-operator-2.30.0
    cd charts/jaeger-operator
    helm install jaeger-operator .  --set rbac.clusterRole=true
   

2. Check the jaeger-operator logs

  export POD=$(kubectl get pods -l app.kubernetes.io/instance=jaeger-operator -lapp.kubernetes.io/name=jaeger-operator --namespace default --output name)
  kubectl logs $POD --namespace=default

Expected behavior
Expected to see no errors in the jaeger-operator logs.

Screenshots
NA

Version (please complete the following information):

• OS: [e.g. Linux] Linux
• Jaeger version: [e.g. 1.8] NA
• Deployment: [e.g. bare metal, Docker, Kubernetes, ...] Kubernetes (kind version v0.11.1 go1.16.4 linux/amd64)

What troubleshooting steps did you try?
Not found any relevant troubleshooting steps in https://www.jaegertracing.io/docs/latest/troubleshooting/ for the error seen above.
Tried to install using jaeger-operator.yaml file specified at https://www.jaegertracing.io/docs/1.32/operator/#installing-the-operator-on-kubernetes. The operator comes up successfully. Issue is seen only when using the helm chart.

Additional context
The same steps work for installing jaeger-operator 1.30.0 using jaeger-operator-2.29.0 helm chart.

[ayaranga]

@LP0101, checking with you as you updated the helm chart for 1.32.0. Did you face the above issue? If so, can you please share the steps for resolution.

[LP0101]

Looks like there was an issue when I tested the upgrade, it appears to be broken for me as well.

I'll try and resolve the issues, see if I can spot what's actually going wrong. Will update the thread if I figure it out

[ayaranga]

Thanks @LP0101 . Hope the templates also would be updated to have certmanager and webhook changes required for 1.32.0

[ayaranga]

@LP0101 Any update on this?

[AlKapkone]

Hello, I've got the same issue. Any updates on this?

[stevehipwell]

The CRD in the chart is very old and the chart hasn't been updated to use cert-manager, as is required by the new CRD. I don't get who approved this release?

[NicolasKarnick]

Any news on this @LP0101 ?

[rafilkmp3]

+1

[AlKapkone]

Any updates?

[bhargavmg]

Still facing this issue. Any updates or anyone managed to resolve this ?

[daniltreshchev]

any update? I've got the same issue

[dimitarshenkov]

same, its broken

[JohnFrampton]

Same for me.

[scDisorder]

Same

[avishefi]

Same for me. Is there an expected update schedule? Looks like the kustomize manifests are up to date due to Kubebuilder but the Helm chart for jaeger-operator fell behind.

[djsly]

any maintainer for this chart ? have we reached a point that this chart just reached its end of life ?

[Oceanswave]

Any update? Is Jaeger dead?

[avishefi]

@Oceanswave work is under way:

• [jaeger-operator] update to 1.34.1 #367
• [jaeger-operator] Add cert machinery #368

[czomo]

This should be resolved now. Check 2.32.2 version of the helm chart. Before that look at prerequisites. Additional compability matrix here

[Oceanswave]

@czomo Thank you!

[ayaranga]

Thanks @czomo. It is working fine now. I have a question regarding the compatibility. Can we use the 2.32.2 helm chart for installing jaeger-operator 1.32.0 as well? I tried the following and it seemed to work fine. Looking for a confirmation for the same.

helm install jaeger-operator jaegertracing/jaeger-operator --set rbac.clusterRole=true --set image.tag=1.32.0

[czomo]

@ayaranga Despite fact it may work i don’t recommend it. It is untestested and you would need to match CRD to corresponding version of operator. If you want to do it or your own please adapt it to Crd whatever version you want.

[ayaranga]

@czomo Got it, thanks. So, there is no helm-chart version currently available that supports jaeger-operator 1.32.0. Are there any plans to patch 2.31.0 to make it work with 1.32.0?

[czomo]

@czomo Got it, thanks. So, there is no helm-chart version currently available that supports jaeger-operator 1.32.0. Are there any plans to patch 2.31.0 to make it work with 1.32.0?

This is question for maintainers tho, in my opinion there is no sense to patch it when 1.35 is out