Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"cannot get controller" error log in tidb-controller-manager logs #1447

Closed
DanielZhangQD opened this issue Dec 30, 2019 · 3 comments · Fixed by #1448
Closed

"cannot get controller" error log in tidb-controller-manager logs #1447

DanielZhangQD opened this issue Dec 30, 2019 · 3 comments · Fixed by #1448
Labels
type/bug Something isn't working

Comments

@DanielZhangQD
Copy link
Contributor

Bug Report

What version of Kubernetes are you using?

1.12
What version of TiDB Operator are you using?

v3.0.5

What did you expect to see?
No such error log
What did you see instead?
The below logs are printed repeatedly:

E1230 03:44:54.263402       1 controller_utils.go:415] cannot get controller dan5/dan5-backup-s3-12191761
E1230 03:44:54.263497       1 controller_utils.go:415] cannot get controller br-test/demo1-backup-s3
E1230 03:44:54.264677       1 controller_utils.go:415] cannot get controller dan5/dan5-backup-schedule-s3-1345-2019-12-20t06-44-00
E1230 03:44:54.265805       1 controller_utils.go:415] cannot get controller dan5/dan5-backup-s3-12041551
@DanielZhangQD
Copy link
Contributor Author

Root Cause:
TiDB Initializer watches all the jobs in the k8s cluster and tries to get the owners of the jobs from the TidbInitializer lister, however, for the jobs not created by TidbInitializer CR, the get will fail and the above logs will be printed.
Solution:
Update the handlers for job events, only to handle the jobs that are created by TidbInitializer CR.

@aylei aylei added the type/bug Something isn't working label Dec 30, 2019
@aylei
Copy link
Contributor

aylei commented Dec 30, 2019

  1. IMO, no controller or the controller is not TidbInitializer are both normal cases, these objects could be simply ignored and no error log needed
  2. adjust release CI script for release-1.0 branch #1320 may help reduce the number of objects that not controlled by TidbInitializer we see by only watching objects with the managed-by: tidb-operator label

@Yisaer
Copy link
Contributor

Yisaer commented Dec 30, 2019

Besides, I think the error logs should be more detailed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants