-
Notifications
You must be signed in to change notification settings - Fork 386
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: push based consumer group metadata #1739
Conversation
This improves draining on rebalance which would often if not always time out since the consumer actor is blocked by the rebalance drain.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like a viable solution to have the group metadata at hand when needed.
The unused new field makes me wonder if you had a thought that didn't make it into the code.
core/src/main/scala/akka/kafka/internal/TransactionalProducerStage.scala
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I notice the CommitTransaction
class in object TransactionalProducerStage
can now be removed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Getting rid of the extra callback in the committing logic makes it much better.
LGTM.
When rebalance triggers draining, it blocks the consumer actor so that it could not reply to the request for consumer group metadata, this would often if not always cause the stream to crash on rebalance (new consumer added, consumer leaving).