Sidecar injection failures #1648

jpkrohling · 2021-12-02T10:31:18Z

When installing the simple-prod example in the monitoring namespace, the following is shown in the logs:

INFO[1123] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
ERRO[1123] failed to update deployment with sidecar      error="Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again" jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
2021-12-02T11:29:03.810+0100	ERROR	controller-runtime.manager.controller.deployment	Reconciler error	{"reconciler group": "apps", "reconciler kind": "Deployment", "name": "simple-prod-query", "namespace": "monitoring", "error": "Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214
INFO[1123] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
ERRO[1123] failed to update deployment with sidecar      error="Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again" jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
2021-12-02T11:29:03.868+0100	ERROR	controller-runtime.manager.controller.deployment	Reconciler error	{"reconciler group": "apps", "reconciler kind": "Deployment", "name": "simple-prod-query", "namespace": "monitoring", "error": "Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214
INFO[1123] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
INFO[1123] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
ERRO[1123] failed to update deployment with sidecar      error="Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again" jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
2021-12-02T11:29:03.900+0100	ERROR	controller-runtime.manager.controller.deployment	Reconciler error	{"reconciler group": "apps", "reconciler kind": "Deployment", "name": "simple-prod-query", "namespace": "monitoring", "error": "Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214
INFO[1123] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
ERRO[1123] failed to update deployment with sidecar      error="Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again" jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
2021-12-02T11:29:03.918+0100	ERROR	controller-runtime.manager.controller.deployment	Reconciler error	{"reconciler group": "apps", "reconciler kind": "Deployment", "name": "simple-prod-query", "namespace": "monitoring", "error": "Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214
INFO[1123] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
INFO[1123] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
INFO[1123] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
INFO[1123] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
ERRO[1123] failed to update deployment with sidecar      error="Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again" jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
2021-12-02T11:29:03.970+0100	ERROR	controller-runtime.manager.controller.deployment	Reconciler error	{"reconciler group": "apps", "reconciler kind": "Deployment", "name": "simple-prod-query", "namespace": "monitoring", "error": "Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214
INFO[1123] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
INFO[1123] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
ERRO[1123] failed to update deployment with sidecar      error="Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again" jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
2021-12-02T11:29:03.993+0100	ERROR	controller-runtime.manager.controller.deployment	Reconciler error	{"reconciler group": "apps", "reconciler kind": "Deployment", "name": "simple-prod-query", "namespace": "monitoring", "error": "Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214
INFO[1123] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
INFO[1123] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
ERRO[1123] failed to update deployment with sidecar      error="Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again" jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
2021-12-02T11:29:04.018+0100	ERROR	controller-runtime.manager.controller.deployment	Reconciler error	{"reconciler group": "apps", "reconciler kind": "Deployment", "name": "simple-prod-query", "namespace": "monitoring", "error": "Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214
INFO[1124] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
INFO[1124] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
INFO[1147] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
ERRO[1148] failed to update deployment with sidecar      error="Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again" jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
2021-12-02T11:29:28.058+0100	ERROR	controller-runtime.manager.controller.deployment	Reconciler error	{"reconciler group": "apps", "reconciler kind": "Deployment", "name": "simple-prod-query", "namespace": "monitoring", "error": "Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214
INFO[1148] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
ERRO[1148] failed to update deployment with sidecar      error="Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again" jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
2021-12-02T11:29:28.084+0100	ERROR	controller-runtime.manager.controller.deployment	Reconciler error	{"reconciler group": "apps", "reconciler kind": "Deployment", "name": "simple-prod-query", "namespace": "monitoring", "error": "Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214
INFO[1148] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
INFO[1148] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
ERRO[1148] failed to update deployment with sidecar      error="Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again" jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
2021-12-02T11:29:28.105+0100	ERROR	controller-runtime.manager.controller.deployment	Reconciler error	{"reconciler group": "apps", "reconciler kind": "Deployment", "name": "simple-prod-query", "namespace": "monitoring", "error": "Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214
INFO[1148] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
ERRO[1148] failed to update deployment with sidecar      error="Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again" jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
2021-12-02T11:29:28.120+0100	ERROR	controller-runtime.manager.controller.deployment	Reconciler error	{"reconciler group": "apps", "reconciler kind": "Deployment", "name": "simple-prod-query", "namespace": "monitoring", "error": "Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214
INFO[1148] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
INFO[1148] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
INFO[1148] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
INFO[1150] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
ERRO[1150] failed to update deployment with sidecar      error="Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again" jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
2021-12-02T11:29:30.589+0100	ERROR	controller-runtime.manager.controller.deployment	Reconciler error	{"reconciler group": "apps", "reconciler kind": "Deployment", "name": "simple-prod-query", "namespace": "monitoring", "error": "Operation cannot be fulfilled on deployments.apps \"simple-prod-query\": the object has been modified; please apply your changes to the latest version and try again"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/home/jpkroehling/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214
INFO[1150] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
INFO[1150] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
INFO[1157] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring
INFO[1157] injecting Jaeger Agent sidecar                jaeger=simple-prod jaeger-namespace=monitoring name=simple-prod-query namespace=monitoring

The text was updated successfully, but these errors were encountered:

rubenvp8510 · 2021-12-02T15:25:19Z

Which version are you using? 1.28.0? or is this with master?

jpkrohling · 2021-12-02T15:28:36Z

Latest master as of now (e0c53a5)

rubenvp8510 · 2021-12-02T21:58:30Z

I was debugging this issue, and it seems like this happens because we changed the way we auto instrumented the query deployment, when the deployment is starting it mutates its state, and when we try to update the deployment with the sidecar, it throws this kind of errors. I saw this before in other injection scenarios as well.

I can get the object one more time before the update to make sure I have the latest version, but I find this problematic and hard to solve. I think the only definitive solution will be to use webhook mutator.

QLeelulu · 2022-01-11T07:21:04Z

same error in 1.29.0

time="2022-01-11T07:08:37Z" level=info msg="injecting Jaeger Agent sidecar" jaeger=jaeger jaeger-namespace=observability name=jaeger-query namespace=observability

time="2022-01-11T07:08:37Z" level=error msg="failed to update deployment with sidecar" error="Operation cannot be fulfilled on deployments.apps \"jaeger-query\": the object has been modified; please apply your changes to the latest version and try again" jaeger=jaeger jaeger-namespace=observability name=jaeger-query namespace=observability

2022-01-11T07:08:37.655Z ERROR controller-runtime.manager.controller.deployment Reconciler error {"reconciler group": "apps", "reconciler kind": "Deployment", "name": "jaeger-query", "namespace": "observability", "error": "Operation cannot be fulfilled on deployments.apps \"jaeger-query\": the object has been modified; please apply your changes to the latest version and try again"}

sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem

/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253

sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2

/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214

time="2022-01-11T07:08:38Z" level=info msg="injecting Jaeger Agent sidecar" jaeger=jaeger jaeger-namespace=observability name=jaeger-query namespace=observability

time="2022-01-11T07:08:38Z" level=error msg="failed to update deployment with sidecar" error="Operation cannot be fulfilled on deployments.apps \"jaeger-query\": the object has been modified; please apply your changes to the latest version and try again" jaeger=jaeger jaeger-namespace=observability name=jaeger-query namespace=observability

2022-01-11T07:08:38.082Z ERROR controller-runtime.manager.controller.deployment Reconciler error {"reconciler group": "apps", "reconciler kind": "Deployment", "name": "jaeger-query", "namespace": "observability", "error": "Operation cannot be fulfilled on deployments.apps \"jaeger-query\": the object has been modified; please apply your changes to the latest version and try again"}

diogoserrano · 2022-03-22T10:25:57Z

after upgrading jaeger-operator from 1.29 to 1.32 i get the same error

time="2022-03-22T10:24:07Z" level=error msg="failed to update deployment with sidecar" error="Operation cannot be fulfilled on deployments.apps \"general-query\": the object has been modified; please apply your changes to the latest version and try again" jaeger=general jaeger-namespace=jaeger name=general-query namespace=jaeger 2022-03-22T10:24:07.740Z ERROR controller-runtime.manager.controller.deployment Reconciler error {"reconciler group": "apps", "reconciler kind": "Deployment", "name": "general-query", "namespace": "jaeger", "error": "Operation cannot be fulfilled on deployments.apps \"general-query\": the object has been modified; please apply your changes to the latest version and try again"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214 time="2022-03-22T10:24:07Z" level=info msg="injecting Jaeger Agent sidecar" jaeger=general jaeger-namespace=jaeger name=general-query namespace=jaeger time="2022-03-22T10:24:08Z" level=info msg="injecting Jaeger Agent sidecar" jaeger=general jaeger-namespace=jaeger name=general-query namespace=jaeger

arushi09-hub · 2022-05-23T10:13:53Z

I am also having the same error. I am using the very latest version of Jaeger-operator which is 1.34. Error which i am getting looks like

msg="Operation cannot be fulfilled on deployments.apps "myapp": the object has been modified; please apply your changes to the latest version and try again" component=jaeger-cr-sync
1.6532994235610714e+09 ERROR controller.jaeger Reconciler error {"reconciler group": "jaegertracing.io", "reconciler kind": "Jaeger", "name": "jaeger", "namespace": "observability", "error": "Operation cannot be fulfilled on deployments.apps "myapp": the object has been modified; please apply your changes to the latest version and try again"

Any suggestions what should i do?

arushi09-hub · 2022-05-24T10:12:01Z

I was debugging this issue, and it seems like this happens because we changed the way we auto instrumented the query deployment, when the deployment is starting it mutates its state, and when we try to update the deployment with the sidecar, it throws this kind of errors. I saw this before in other injection scenarios as well.

I can get the object one more time before the update to make sure I have the latest version, but I find this problematic and hard to solve. I think the only definitive solution will be to use webhook mutator.

How will a webhook mutator be used here? I think I am not aware of it, of how it will be used.

frzifus · 2022-05-24T10:13:22Z

Does the error message show an effect? My guess is, when a deployment is created, it passes through the provided webhook to modify the pod spec.
if now a Jaeger instance is created, updated or removed in parallel, the reconciler notices that this deployment needs an update. To realize this update the deployment specification is extended with an annotation. The error message appears because the webhook has already modified the deployment spec.
Since we return an error in the reconciler, the deployment is evaluated again shortly afterwards.

frzifus · 2022-05-24T10:22:32Z

Probably we can ignore this error and simply trigger the Jaeger reconciler again to be on the safe side.

arushi09-hub · 2022-05-24T10:34:05Z

Probably we can ignore this error and simply trigger the Jaeger reconciler again to be on the safe side.

What command can I use to trigger the reconciler again?

arushi09-hub · 2022-05-24T10:38:05Z

Does the error message show an effect? My guess is, when a deployment is created, it passes through the provided webhook to modify the pod spec. if now a Jaeger instance is created, updated or removed in parallel, the reconciler notices that this deployment needs an update. To realize this update the deployment specification is extended with an annotation. The error message appears because the webhook has already modified the deployment spec. Since we return an error in the reconciler, the deployment is evaluated again shortly afterwards.

I am unable to see this service on my Jaeger UI which is properly up and running and shows the jaegery-query service also. The pods were perfectly running still i could not see it on the UI. Went through the logs of jaeger-operator and saw this error over here. Assumed that it can be because this error.

frzifus · 2022-05-24T11:39:04Z

What command can I use to trigger the reconciler again?

Its done automatically. No action need.

I am unable to see this service on my Jaeger UI which is properly up and running and shows the jaegery-query service also. The pods were perfectly running still i could not see it on the UI. Went through the logs of jaeger-operator and saw this error over here. Assumed that it can be because this error.

I assume there is no correlation. But i will look into it.

violinorg · 2022-08-25T13:49:25Z

Having same problem - jaeger-operator v1.36.0. Yesterday was updated along with the cluster components. Previously on v1.35.0 there were no problems. Reverting the version didn't fix the problem

iblancasa · 2022-08-25T19:23:38Z

@violinorg, could you provide some log output?

frzifus · 2022-08-25T22:49:03Z

Probably the log output is similar to the other reports. Looks like the logging is causing confusion, maybe we should change the reported message and/or only log on debug level.

What do you think?

iblancasa · 2022-08-25T22:51:32Z

If it is not raising a real issue...

violinorg · 2022-08-26T12:59:03Z

@iblancasa yes of course

time="2022-08-26T12:40:04Z" level=error msg="Operation cannot be fulfilled on deployments.apps \"r882-dcc95-observability-tracing-demo\": the object has been modified; please apply your changes to the latest version an │
│ 1.6615176046172814e+09    ERROR    Reconciler error    {"controller": "jaeger", "controllerGroup": "jaegertracing.io", "controllerKind": "Jaeger", "jaeger": {"name":"observability-t1260-3f32c-jaeger-operator-jaeger","n │
│ sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem                                                                                                                                   │
│     /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:273

iblancasa · 2022-08-26T13:12:59Z

@frzifus I'm not sure but it seems this message is not printed by Jaeger Operator but by the inherit logger.

jpkrohling added the bug Something isn't working label Dec 2, 2021

purvaldur mentioned this issue Mar 23, 2022

Operation cannot be fulfilled on deployments.apps... #927

Closed

frzifus mentioned this issue May 24, 2022

ERROR controller-runtime.manager.controller.deployment Reconciler error #1895

Open

frzifus mentioned this issue Jul 8, 2022

Operation cannot be fulfilled on horizontalpodautoscalers.autoscaling \"jaeger-operator-jaeger-collector\" / failed to apply the changes #1982

Open

jpkrohling closed this as completed Jul 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sidecar injection failures #1648

Sidecar injection failures #1648

jpkrohling commented Dec 2, 2021

rubenvp8510 commented Dec 2, 2021

jpkrohling commented Dec 2, 2021

rubenvp8510 commented Dec 2, 2021

QLeelulu commented Jan 11, 2022

diogoserrano commented Mar 22, 2022

arushi09-hub commented May 23, 2022

arushi09-hub commented May 24, 2022

frzifus commented May 24, 2022

frzifus commented May 24, 2022

arushi09-hub commented May 24, 2022

arushi09-hub commented May 24, 2022

frzifus commented May 24, 2022

violinorg commented Aug 25, 2022

iblancasa commented Aug 25, 2022

frzifus commented Aug 25, 2022

iblancasa commented Aug 25, 2022

violinorg commented Aug 26, 2022

iblancasa commented Aug 26, 2022

Sidecar injection failures #1648

Sidecar injection failures #1648

Comments

jpkrohling commented Dec 2, 2021

rubenvp8510 commented Dec 2, 2021

jpkrohling commented Dec 2, 2021

rubenvp8510 commented Dec 2, 2021

QLeelulu commented Jan 11, 2022

diogoserrano commented Mar 22, 2022

arushi09-hub commented May 23, 2022

arushi09-hub commented May 24, 2022

frzifus commented May 24, 2022

frzifus commented May 24, 2022

arushi09-hub commented May 24, 2022

arushi09-hub commented May 24, 2022

frzifus commented May 24, 2022

violinorg commented Aug 25, 2022

iblancasa commented Aug 25, 2022

frzifus commented Aug 25, 2022

iblancasa commented Aug 25, 2022

violinorg commented Aug 26, 2022

iblancasa commented Aug 26, 2022