-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Don't reset max.poll.interval.ms limit on every poll #4176
Conversation
tests/0089-max_poll_interval.c
Outdated
|
||
int main_0089_max_poll_interval(int argc, char **argv) { | ||
do_test_with_optional_queue(rd_false); | ||
do_test_with_optional_queue(rd_true); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would suggest using different methods for different tests.
- It improves readability
- We are not touching the old test case just adding a new one
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done, also added a changelog entry
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think if we're adding some new checks to an old test case, it's ok to change it as long as we don't change existing asserts.
This is also to avoid having tests that are too slow, especially integration tests.
It can also be ok with to parameterize some tests to avoid code duplication, as long as the tests are very similar and the resulting code is readable.
What do you think @pranavrth?
That said @milindl you don't have to revert this change you can leave them separate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am okay with either option as per what we decide.
Both these tests are slow though and can't be run in make quick because of the timeout.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I felt that we are touching multiple places in the old test so asked to separate.
Though anything is fine as far as the old test is not changed.
99f2005
to
84b69e2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Just want to mention that this "fix" broke my binding where I use events and read consumer events from the consumer queue (and never calling |
Yep, we're aware of the bug already. We're looking into it with #4256 . Thanks. |
Earlier on, we changed the code to prevent max.poll.interval.ms from being triggered in case we were inside librdkafka in any sort of a
poll
call. Top achieve this, blocked the timer usingrd_kafka_app_poll_blocking
, and reset it at the end of the call.This doesn't work correctly in the cases where we're simply polling an unrelated queue, like the log queue. In that case, just by polling the log queue, the timer is reset, despite us not actually consuming anything (or doing any consume poll).
At the same time, it's a reasonable expectation that max.poll.interval.ms won't be triggered while we are doing any sort of consumer poll.
This PR takes care of both the cases. The methods (from public API) which block/reset the timer are:
All the other poll methods like queue_poll etc no longer reset or block the timer.
A failing test was added first, to test the change.
Ref #2365.