Skip to content

Commit

Permalink
kgo: occasionally use seed brokers when choosing "random" brokers
Browse files Browse the repository at this point in the history
See the embedded comments -- previously, we could get into a
pathological case where all discovered brokers are down. We would never
be able to refresh metadata because we would only try talking to
discovered brokers.
  • Loading branch information
twmb committed Oct 6, 2022
1 parent 3ebd775 commit 3e02574
Showing 1 changed file with 12 additions and 1 deletion.
13 changes: 12 additions & 1 deletion pkg/kgo/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -404,15 +404,26 @@ func (cl *Client) broker() *broker {
cl.brokersMu.Lock() // full lock needed for anyBrokerIdx below
defer cl.brokersMu.Unlock()

// Every time we loop through all discovered brokers, we issue one
// request to the next seed. This ensures that if all discovered
// brokers are down, we will *eventually* loop through seeds and
// hopefully have a reachable seed.
var b *broker
if len(cl.brokers) > 0 {
if len(cl.brokers) > 0 && int(cl.anyBrokerIdx) < len(cl.brokers) {
cl.anyBrokerIdx %= int32(len(cl.brokers))
b = cl.brokers[cl.anyBrokerIdx]
cl.anyBrokerIdx++
} else {
cl.anySeedIdx %= int32(len(cl.seeds))
b = cl.seeds[cl.anySeedIdx]
cl.anySeedIdx++

// If we have brokers, we ranged past discovered brokers.
// We now reset the anyBrokerIdx to begin ranging through
// discovered brokers again.
if len(cl.brokers) > 0 {
cl.anyBrokerIdx %= int32(len(cl.brokers))
}
}
return b
}
Expand Down

0 comments on commit 3e02574

Please sign in to comment.