From 54f3092e2e5eab441afe71681344ca39bb3aa9a3 Mon Sep 17 00:00:00 2001 From: Travis Bischel Date: Sun, 17 Oct 2021 13:23:07 -0600 Subject: [PATCH] README&docs: mention kadm --- README.md | 1 + docs/admin-requests.md | 14 ++++++++++++++ docs/producing-and-consuming.md | 9 +++++++++ 3 files changed, 24 insertions(+) diff --git a/README.md b/README.md index c2dfe286..a11c3d28 100644 --- a/README.md +++ b/README.md @@ -25,6 +25,7 @@ This library attempts to provide an intuitive API while interacting with Kafka t - SSL/TLS provided through custom dialer options - All SASL mechanisms supported (GSSAPI/Kerberos, PLAIN, SCRAM, and OAUTHBEARER) - Low-level admin functionality supported through a simple `Request` function +- High-level admin package with many helper types to make cluster administration easy. - Utilizes modern & idiomatic Go (support for contexts, variadic configuration options, ...) - Highly performant by avoiding channels and goroutines where not necessary - Written in pure Go (no wrapper lib for a C library or other bindings) diff --git a/docs/admin-requests.md b/docs/admin-requests.md index 211edb7a..35e80f20 100644 --- a/docs/admin-requests.md +++ b/docs/admin-requests.md @@ -1,5 +1,19 @@ # Admin Requests +This repo provides two ways to issue admin requests: a high level [kadm][a] +package, and a low level [kmsg][b] package. The kadm package is an opinionated +package that attempts to abstract the low level details of administrating into +some intuitive methods and types that have many helper functions. If you see a +type that could have more helper functions, please create a pull request. + +[a]: https://pkg.go.dev/github.com/twmb/franz-go/pkg/kadm +[b]: https://pkg.go.dev/github.com/twmb/franz-go/pkg/kmsg + +Not all of the low level API can be encapsulated in a high level API, and the +kadm package must sacrifice some details to make things easier to use. If the +kadm package does not give you the control you need, the low level kadm package +allows you to construct requests to Kafka directly. + All Kafka requests and responses are supported through generated code in the kmsg package. The package aims to provide some relatively comprehensive documentation (at least more than Kafka itself provides), but may be lacking diff --git a/docs/producing-and-consuming.md b/docs/producing-and-consuming.md index 71a88f60..35ebe158 100644 --- a/docs/producing-and-consuming.md +++ b/docs/producing-and-consuming.md @@ -147,6 +147,15 @@ If you rely on the default options and do not commit yourself, all of this is automatically handled. My recommendation is to just set a custom `CommitCallback` if you need to and to rely on the default commit behavior. +##### Direct offset management outside of a group + +You can use the `ConsumePartitions` option to assign partitions manually and +consume outside of the context of a group. If you want to use Kafka to manage +group offsets even with direct partition assignment, this repo provides a +`kadm` package to easily manage offsets via an admin interface. Check the +[`manual_committing`](../examples/manual_committing) example to see some +example code for how to do this. + ##### Without transactions There are two easy patterns to success for offset management in a normal