client will export isr info of a topic/partition

[funkygao]

No description provided.

[wvanbergen]

Retrieving the ISR from Kafka is problematic, because the true value is stored in Zookeeper.
The Kafka broker you are talking to can give you an out of date, locally cached value that is out of sync with Zookeeper. This is a known bug/limitation/feature: https://issues.apache.org/jira/browse/KAFKA-1367?jql=text%20~%20%22ISR%20out%20of%20sync%22

When we were previously using this information, it was telling us that we had many out of sync replicas all the time, while the kafka-topics.sh tool that ships with Kafka was telling us that everything was fine. The difference was that the tool that ships with Kafka reads the information directly from Zookeeper. You can use Kazoo to find this information in Zookeeper.

[eapache]

Yes, the ISR is explicitly not exposed because it is known to be flaky. If that has changed in the kafka 0.9 release candidate then this may be worth revisiting, but it isn't clear to me one way or another if that is the case.

[funkygao]

Thanks.
I am writing a kafka clusters console CLI, so Isr being stale is not a problem for me because
it runs and exits quickly.
Anyway, Isr cache should really not be exported directly.
You are right!

[wvanbergen]

@funkygao the problem is not that the value is cached in arama, but on the kafka broker itself. So even if you just have a short running CLI app, you still see stale values because the values in the broker themselves are often stale. Again, talking to zookeeper is the best way to get Kafka metadata.

[wvanbergen]

A simple example app that talks to Zookeeper using kazoo: https://github.com/wvanbergen/kazoo-go/blob/master/tools/kafka-topics/kafka-topics.go

[funkygao]

thanks @wvanbergen

Yes, I read kazoo-go code. Cool!

BTW, my kafka clusters management tool is at https://github.com/funkygao/gafka