Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unclear Micronaut Kafka documentation about commit strategy for SYNC_PER_RECORD #1141

Open
jamesforwardnwboxed opened this issue Jan 30, 2025 · 1 comment

Comments

@jamesforwardnwboxed
Copy link

Issue description

Looking at: https://micronaut-projects.github.io/micronaut-kafka/latest/guide/#kafkaOffsets

We can see that SYNC says Commits offsets manually at the end of each poll() loop if no exceptions occur. Sets enable.auto.commit to false which explicitly implies that if an exception occurs offsets are not committed

However SYNC_PER_RECORD just says Commits offsets manually after each ConsumerRecord is processed. Sets enable.auto.commit to false without any clarification about exception handling.

2 questions:

  1. Can the docs be updated to be clearer?
  2. What happens in SYNC_PER_RECORD If an exception is thrown? And how does it relate to different ErrorStrategy values?
@graemerocher
Copy link
Contributor

looks like auto commit is only enabled if offset strategy is AUTO

properties.putIfAbsent(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, String.valueOf(offsetStrategy == OffsetStrategy.AUTO));

And it appears the same error strategy handling applies

try {
process(consumerRecord, consumerRecords);
} catch (Exception e) {
if (resolveWithErrorStrategy(consumerRecords, consumerRecord, e)) {
resetTheFollowingPartitions(consumerRecord, iterator);
failed = true;
return;
}
}
if (info.offsetStrategy == OffsetStrategy.SYNC_PER_RECORD) {
commitSync(consumerRecords, consumerRecord, currentOffsets);
} else if (info.offsetStrategy == OffsetStrategy.ASYNC_PER_RECORD) {
kafkaConsumer.commitAsync(currentOffsets, this::resolveCommitCallback);
}

Feel free to contribute all documentation clarifications that you feel are missing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants