feat(kafka): Enable querier to optionally query partition ingesters

[benclive]

What this PR does / why we need it:

Implements logic to enable queriers to query the right ingesters based on active partitions.

• When the flag is active, old ingesters will no longer be queried at all.
• Includes shuffle sharding support
• Includes request minimization to only query 1 ingester instance per partition instead of all/both of them.

Which issue(s) this PR fixes:
Fixes https://github.com/grafana/loki-private/issues/1132

Special notes for your reviewer:

Checklist

• Reviewed the CONTRIBUTING.md guide (required)
• Documentation added
• Tests updated
• Title matches the required conventional commits format, see here
  • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
• Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
• If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

[benclive]

I couldn't find an answer to this: is the lookback period the same as the query-ingesters-within parameter or something else?

[cyriltovena]

Yes it is query-ingesters-within can we get confirmation from @pracucci ?

[pracucci]

The shuffle sharding lookback period in Mimir is configured here:
https://github.com/grafana/mimir/blob/ab3e627b1da134f33c06dd09ee4402a2eaad68f1/pkg/mimir/modules.go#L448-L450

It's set equal to TSDB retention, but that's a recent change. In the past it was set to the query-ingesters-within parameter. The reason why we changed to TSDB retention is because now we let to customize the query-ingesters-within parameter on a per-tenant basis (via the live-reloaded runtime config) and so it's not the same setting for the entire cluster. The worst case scenario we want to cover is the TSDB retention period in ingesters (which is also set to 13h).

I think for your use case setting it to query-ingesters-within parameter is fine.

[cyriltovena]

LGTM !

There's just a lint problem :)