add option to return error on non exiting topic for ConsumeTopics opt

[vtolstov]

sometimes someone =) specify invalid topic name (non exiting), but kgo does not fails on subscribe with ConsumeTopics, does it possible to add ability to return error if topic unknown or client does not have rights to fetch from it or produce?

[twmb]

So are you thinking an option to return all retryable errors in fetches, or just this one?

[vtolstov]

does it possible to add option to handle all retryable errors so inside this func we can decide to continue retry or not?

[vtolstov]

like
type RetryFunc func(ctx context.Context, retryCount int, err error) (bool, error)
so we know how much retries already done, error happened, and can return continue (via first arg), may be via second return arg return something if continue not needed like fmt.Errorf("retry count reached")

[twmb]

The client already internally retries following roughly that exact pattern. For fetching, failed partitions are always retried forever. I thought you were asking for an option like,

func KeepFetchRetryableErrors() Opt

such that that retryable errors here:

franz-go/pkg/kgo/source.go

Line 864 in 5379fb0

if kerr.IsRetriable(fp.Err) {

would not be stripped.

[vtolstov]

Internally, but if user wants to handle errors via own logic? One use-case - check that topic exists when subscribe to it.

[twmb]

In that case, existence checking needs to be done before subscribing to it. However, with the option, you could react to errors (such as a topic being deleted). Would the option above satisfy what you're looking for?