Cherry-pick #16365 to 7.x: Refactor keystore interface into 3 different interfaces

[ChrsMark]

Cherry-pick of PR #16365 to 7.x branch. Original message:

What does this PR do?

This PR refactor keystore interface into 3 different interfaces.

This PR introduces a new Keystore backend which is ReadOnly and retrieves passwords from k8s secrets through k8s API. In order to implement this, the basic Keystore interface is split into 3 different interfaces, Keystore (readonly), WritableKeystore (Keystore + store/write/delete), ListingKeystore (WritableKeystore + list) as proposed on #5832.

The new Keystore that retrieves data from k8s secrets is of Keystore (readonly) type.

#### Autodiscover adoption
In order to support this k8s Keystore in Autodiscover, a keystore object is attached on every event. The keystore object is either created or retrieved if it already exists. We create only one keystore per namespace and we reuse it for events of the same namespace.

Why is it important?

This is important in order to support different keystore backends, like k8s keystore.

This is important in order to support autodiscover hints being able to refer to k8s secrets for password retrieval.

How to test this PR locally

Try all the set of available commands:

1. ./metricbeat keystore create
2. ./metricbeat keystore add ES_PWD
3. ./metricbeat keystore add ES_PWD --force
4. echo "passpass" | ./metricbeat keystore add ES_PWD --stdin --for
5. ./metricbeat keystore list
6. ./metricbeat keystore remove ES_PWD

Create a key and consume it (Elastic Cloud auth or any other ES setting can be used as an example):

1. ./metricbeat keystore add ES_PWD
2. ./metricbeat -e -d "*" -E "cloud.id=testkeys:xxxx" -E "cloud.auth=\${ES_PWD}"

or

1. ./metricbeat keystore add ES_HOST
2. ./metricbeat -e -d "*" -E output.elasticsearch.hosts=["\${ES_HOST}"]

Related issues

• Relates Allow the Keystore to be completely pluggable #5832, [Metricbeat] Support pulling secrets from Kubernetes/OpenShift for autodiscovery #8847

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[ycombinator]

Backport LGTM.