You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This information seems to be currently missing in the data returned by kazoo-go (you can ask for the current offset of a consumergroup, but not for the per-partition oldest/newset offsets).
The text was updated successfully, but these errors were encountered:
I would also like to be able to get the partition offset details, as there seems to be no other way to compute consumer group lag with this library. I am happy to contribute but I also don't know where this information is stored in Zookeeper.
[Edit] it seems this information (first/last offset) is not stored in Zookeeper at all, and needs to be pulled from the Kafka broker. In Sarama this is done with sarama.Client.GetOffset(topic, partitionID, sarama.OffsetOldest) and sarama.Client.GetOffset(topic, partitionID, sarama.OffsetNewest).
The offset shell allows to dump the oldest and newest offsets currently available in a partition:
https://cwiki.apache.org/confluence/display/KAFKA/System+Tools#SystemTools-GetOffsetShell
The same information is also available from the SimpleConsumer: https://apache.googlesource.com/kafka/+/trunk/core/src/main/scala/kafka/consumer/SimpleConsumer.scala#184
This information seems to be currently missing in the data returned by kazoo-go (you can ask for the current offset of a consumergroup, but not for the per-partition oldest/newset offsets).
The text was updated successfully, but these errors were encountered: