Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logging from Kubernetes is not formatted correctly #17461

Closed
espadolini opened this issue Oct 17, 2022 · 0 comments · Fixed by #17462
Closed

Logging from Kubernetes is not formatted correctly #17461

espadolini opened this issue Oct 17, 2022 · 0 comments · Fixed by #17462
Labels
bug test-plan-problem Issues which have been surfaced by running the manual release test plan

Comments

@espadolini
Copy link
Contributor

Expected behavior:
Logs pertaining to Kubernetes (coming from the Proxy service and the Kubernetes service) should be formatted like other log lines.

Current behavior:
Logs pertaining to Kubernetes are formatted according to the standard logrus formatting:

Bug details:

  • Teleport version: 11.0.0-beta.2

Proxy logs (format json, output to terminal, INFO and trace.component are displayed in cyan)

{"caller":"cache/cache.go:823","component":"proxy:4:cache","level":"warning","message":"Re-init the cache on error: watcher is closed: error reading from server: EOF.","timestamp":"2022-10-17T11:50:50+02:00"}
INFO[1991] Round trip: GET https://kube.teleport.cluster.local/version?timeout=32s, code: 200, duration: 41.405ms tls:version: 304, tls:resume:false, tls:csuite:1301, tls:server:kube.main.wet-dry.world  trace.component="proxy:proxy:kube"
{"caller":"services/watcher.go:246","component":"proxy","level":"warning","message":"Restart watch on error: watcher is closed: watcher closed.","pid":"57930.4","resource-kind":"cert_authority","timestamp":"2022-10-17T11:50:53+02:00"}
INFO[1992] Round trip: GET https://kube.teleport.cluster.local/version?timeout=32s, code: 200, duration: 61.6ms tls:version: 304, tls:resume:false, tls:csuite:1301, tls:server:kube.main.wet-dry.world  trace.component="proxy:proxy:kube"
{"caller":"services/watcher.go:246","component":"proxy","level":"warning","message":"Restart watch on error: watcher is closed: error reading from server: EOF.","pid":"57930.4","resource-kind":"lock","timestamp":"2022-10-17T11:50:54+02:00"}
{"caller":"services/watcher.go:246","component":"proxy","level":"warning","message":"Restart watch on error: watcher is closed: watcher closed.","pid":"57930.4","resource-kind":"node","timestamp":"2022-10-17T11:50:54+02:00"}
{"caller":"services/watcher.go:246","component":"proxy:4","level":"warning","message":"Restart watch on error: watcher is closed: watcher closed.","pid":"57930.4","resource-kind":"proxy","timestamp":"2022-10-17T11:50:55+02:00"}
INFO[1994] Round trip: GET https://kube.teleport.cluster.local/version?timeout=32s, code: 200, duration: 52.49ms tls:version: 304, tls:resume:false, tls:csuite:1301, tls:server:kube.main.wet-dry.world  trace.component="proxy:proxy:kube"
INFO[1997] Round trip: GET https://kube.teleport.cluster.local/api?timeout=32s, code: 200, duration: 72.175ms tls:version: 304, tls:resume:false, tls:csuite:1301, tls:server:kube.main.wet-dry.world  trace.component="proxy:proxy:kube"
INFO[1997] Round trip: GET https://kube.teleport.cluster.local/apis?timeout=32s, code: 200, duration: 40.97ms tls:version: 304, tls:resume:false, tls:csuite:1301, tls:server:kube.main.wet-dry.world  trace.component="proxy:proxy:kube"

teleport-kube-agent pod logs (default text formatting):

2022-10-17T09:51:45Z INFO [UPLOAD:1]  Creating directory /var/lib/teleport/log/upload/streaming. pid:7.1 service/service.go:2501
2022-10-17T09:51:45Z INFO [UPLOAD:1]  Creating directory /var/lib/teleport/log/upload/streaming/default. pid:7.1 service/service.go:2501
2022-10-17T09:51:49Z WARN             Inventory control stream failed: inventory control stream closed: control streams do not support impersonation ("e342d190-62ea-4e75-8999-82bb4759cf39" -> "a7be082a-3a79-44da-850f-53a6f1a35ade") inventory/inventory.go:146
time="2022-10-17T09:51:50Z" level=info msg="Round trip: GET https://10.96.0.1:443/api/v1/namespaces/teleagent/pods?limit=500, code: 200, duration: 10.888083ms tls:version: 304, tls:resume:false, tls:csuite:1301, tls:server:kube.teleport.cluster.local" trace.component=kubernetes
time="2022-10-17T09:51:51Z" level=info msg="Round trip: GET https://10.96.0.1:443/api/v1/namespaces/teleagent/pods?limit=500, code: 200, duration: 8.212291ms tls:version: 304, tls:resume:false, tls:csuite:1301, tls:server:kube.teleport.cluster.local" trace.component=kubernetes
time="2022-10-17T09:51:52Z" level=info msg="Round trip: GET https://10.96.0.1:443/api/v1/namespaces/teleagent/pods?limit=500, code: 200, duration: 11.704542ms tls:version: 304, tls:resume:false, tls:csuite:1301, tls:server:kube.teleport.cluster.local" trace.component=kubernetes
time="2022-10-17T09:51:53Z" level=info msg="Round trip: GET https://10.96.0.1:443/api/v1/namespaces/teleagent/pods?limit=500, code: 200, duration: 11.222375ms tls:version: 304, tls:resume:false, tls:csuite:1301, tls:server:kube.teleport.cluster.local" trace.component=kubernetes
@espadolini espadolini added bug test-plan-problem Issues which have been surfaced by running the manual release test plan labels Oct 17, 2022
tigrato added a commit that referenced this issue Oct 17, 2022
This PR allows Kube proxy/service to use the global logger settings defined for the process.

Fixes #17461
tigrato added a commit that referenced this issue Oct 17, 2022
This PR allows Kube proxy/service to use the global logger settings defined for the process.

Fixes #17461
tigrato added a commit that referenced this issue Oct 17, 2022
This PR allows Kube proxy/service to use the global logger settings defined for the process.

Fixes #17461
tigrato added a commit that referenced this issue Oct 17, 2022
This PR allows Kube proxy/service to use the global logger settings defined for the process.

Fixes #17461
tigrato added a commit that referenced this issue Oct 17, 2022
This PR allows Kube proxy/service to use the global logger settings defined for the process.

Fixes #17461
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug test-plan-problem Issues which have been surfaced by running the manual release test plan
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant