Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

route kafka logs thru logger #2744

Merged
merged 1 commit into from
Mar 22, 2021
Merged

route kafka logs thru logger #2744

merged 1 commit into from
Mar 22, 2021

Conversation

maskarb
Copy link
Member

@maskarb maskarb commented Mar 22, 2021

Currently, rdkafka prints directly to stdout instead of using our log handler. This PR re-routes kafka log messages through our handler which will enable Kibana to pick up these logs.

Below shows a comparison of the output for the same log message:
Previous behavior:

koku_listener     | %4|1616432526.766|MAXPOLL|rdkafka#consumer-1| [thrd:main]: Application maximum poll interval (100000ms) exceeded by 215ms (adjust max.poll.interval.ms for long-running message processing): leaving group

New behavior:

koku_listener     | [2021-03-22 16:56:34,811] WARNING None MAXPOLL [rdkafka#consumer-1] [thrd:main]: Application maximum poll interval (100000ms) exceeded by 161ms (adjust max.poll.interval.ms for long-running message processing): leaving group

@maskarb maskarb requested a review from a team March 22, 2021 17:08
@maskarb maskarb self-assigned this Mar 22, 2021
@codecov
Copy link

codecov bot commented Mar 22, 2021

Codecov Report

Merging #2744 (e0ee8b9) into master (7246f95) will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master   #2744   +/-   ##
======================================
  Coverage    94.7%   94.7%           
======================================
  Files         289     289           
  Lines       22364   22364           
  Branches     2550    2550           
======================================
  Hits        21178   21178           
- Misses        714     716    +2     
+ Partials      472     470    -2     

@maskarb maskarb merged commit 3e253c6 into master Mar 22, 2021
@maskarb maskarb deleted the kafka-logging branch March 22, 2021 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants