autothrottle Kafka native API support

[jamiealquiza]

Changes

• Updates autothrottle to optionally perform all operations that currently read from ZooKeeper to use the Kafka Admin API (via the kafka-kit/kafkaadmin library):
  • KIP-455 compatible reassignment lookups
  • Topic states
  • Broker, topic throttle application
  • Broker, topic throttle clearing
• The feature is enabled via the --kafka-native-mode flag (defaults to false for now, which means autothrottle will function the same as it does before this PR)
• Deletes deprecated autothrottle API routes

Functional notes

• Internally (@ Datadog) we currently set static, broker level overrides on possibly not-yet-existing brokers. The Kafka APIs have seemingly unbounded waits when setting dynamic configs on non-existent resources. I need to validate whether we're exposed to this in these changes.
• The Kafka API is generally a bit more flakey to deal with than reading/writing znodes, which tends to succeed regardless of the Kafka cluster state. It's almost certain that we're going to see weird client behavior that will need more elaborate conditional behavior and timeout tuning.
• This entire change is a functional no-op unless you flag on --kafka-native-mode.

PR notes

• This is easiest to read by stepping through commits sequentially
• Much of the ZK based code has been organized and branded as "legacy" for eventual removal
• Autothrottle needs significant refactoring, but this PR introduces a considerable number of complicated changes. Accordingly, a minimal amount of restructuring was done in order to satisfy the introduction of this new functionality. A dedicated refactor will be a follow up task:
  • Added the autothrottle/internal/api and autothrottle/internal/throttlestore
  • Further refactoring will move a significant amount of the currently flat autothrottle codebase to better structured packages

TODOs

• Finish test coverage for newly introduced code
• Complete system testing