Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lower log level in Kafka product tests #14645

Merged
merged 4 commits into from
Oct 28, 2022

Conversation

nevillelyh
Copy link
Member

Description

Lower log level in Kafka containers and test code used in Kafka product tests

Non-technical explanation

Release notes

(x) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
( ) Release notes are required, with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label Oct 14, 2022
@nevillelyh nevillelyh force-pushed the neville/kafka-logging branch 5 times, most recently from edeff82 to ba5d092 Compare October 17, 2022 01:46
Copy link
Member

@lukasz-walkiewicz lukasz-walkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

log4j.logger.{{logger}}={{loglevel}}
{% endfor %}

log4j.logger.kafka.server.KafkaServer=INFO
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turns out the environment depends on a started (kafka.server.KafkaServer) log message.

.waitingForAll(forSelectedPorts(9092), forLogMessage(".*started \\(kafka.server.KafkaServer\\).*", 1))

@nevillelyh nevillelyh force-pushed the neville/kafka-logging branch from 20a77cf to 1e0b94e Compare October 17, 2022 21:40
@nevillelyh
Copy link
Member Author

Two possible flakiness issues. Otherwise this is good to go. @lukasz-walkiewicz do we need another approval?
#3049
#6799

@nevillelyh nevillelyh force-pushed the neville/kafka-logging branch 2 times, most recently from 2d64fce to c9cce87 Compare October 19, 2022 14:10
@nevillelyh nevillelyh requested a review from kokosing October 20, 2022 13:49
@kokosing
Copy link
Member

Starting download for product tests and server tarball
Error: Unable to find an artifact with the name: product tests and server tarball

The ci / pt (default, suite-2, ) failed because of the above. I am retrying.

@nevillelyh nevillelyh force-pushed the neville/kafka-logging branch from 0d9c444 to 78fed64 Compare October 23, 2022 20:55
@nevillelyh
Copy link
Member Author

Also discovered that TestingKafka containers have the same problem. Added a commit to address that.

@kokosing
Copy link
Member

ci / pt (default, suite-2, ) fails again. So it does not look as flake.

@nevillelyh nevillelyh force-pushed the neville/kafka-logging branch 3 times, most recently from 9abab28 to 839f495 Compare October 26, 2022 13:07
@nevillelyh
Copy link
Member Author

nevillelyh commented Oct 26, 2022

Dead lock should be fixed now.

Also noticed noisy logging from io.confluent.kafka which I missed from #14149. Added a commit to address that.
https://github.com/trinodb/trino/actions/runs/3329251804/jobs/5507115101.

INFO	pool-3-thread-2	io.confluent.kafka.serializers.KafkaAvroSerializerConfig	KafkaAvroSerializerConfig values: 
...

@nevillelyh nevillelyh force-pushed the neville/kafka-logging branch from 529fe43 to 3f261be Compare October 27, 2022 04:55
@nevillelyh
Copy link
Member Author

Hit flaky test in Pinot #14239

@kokosing kokosing merged commit c18c976 into trinodb:master Oct 28, 2022
@kokosing
Copy link
Member

Thanks!

@nevillelyh nevillelyh deleted the neville/kafka-logging branch October 28, 2022 13:14
@github-actions github-actions bot added this to the 402 milestone Oct 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

3 participants