[Feature Request]: [ElasticsearchIO] Change bearer Token into ValueProvider or callback function

[EgbertW]

What would you like to happen?

When setting up a ElasticsearchIO source or sink, I can specify a bearer token to authenticate. However, for the project I'm working on I'm required to work with short-lived tokens that expire after 1 hour, so I need a way to dynamically update the token.

I see two options to do this: either add a variant of the org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO.ConnectionConfiguration#withBearerToken method that accepts a org.apache.beam.sdk.options.ValueProvider<String> instead, so that it can request the value on demand, or instead accept a callback function that can specify the bearer token after obtaining it from the authentication token endpoint.

Ideally, this value would be requested on every outgoing request to the Elasticsearch API, but it could also work if this is only requested whenever a request fails (because the token expired).

Any thoughts or comments? If you think this is an acceptable feature, I'd be willing to create a MR to add this feature, if that helps you.

Issue Priority

Priority: 3 (nice-to-have improvement)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[EgbertW]

This issue is resolved by PR #25024