[PROPOSAL] Support for multiple OpenSearch domains/workspaces/endpoints

[barthel]

What/Why

What are you proposing?

I would like to propose the possibility to configure more than one static OpenSearch domain at the same time with a terraform apply.

What users have asked for this feature?

There are a lot of users who, for example, want to further process a set of results from a terraform module where the static configuration of a provider prevents them from doing so.
However, this will not be possible in terraform for the foreseeable future.

What problems are you trying to solve?

We are building a complete platform with everthing-as-code and terraform in AWS.
This includes a lot of OpenSearch domains (in AWS managed OpenSearch).
The AWS terraform provider does not offer OpenSearch specific configurations.
Based on the created OpenSearch domains (list), we now face the challenge of having to perform user mapping for each individual domain, for example.
This does not work with statically initialized providers, because each domain has its own endpoint.

What is the developer experience going to be?

Are there any security considerations?

AWS Request Signing must be supported (already supported).

Are there any breaking changes to the API

Resources must be customized to support an optional endpoint (e.g., found via a DataSource).

What is the user experience going to be?

For example, as a user I want to be able to iterate over my list of OpenSearch domains with for_each in a resource and pass the endpoint in the resource.

Are there breaking changes to the User Experience?

This feature can be implemented as an optional feature in my opinion, so there should be no break in usage.

Why should it be built? Any reason not to?

The feature would significantly expand the provider's purpose and make it more flexible.
It saves (in this particular focus) the use of terragrunt or similar external tools.

What will it take to execute?

Any remaining open questions?

How can the AWS OpenSearch Provider be harmonized with the OpenSearch Provider so that there is no break in processing?

[barthel]

Cross reference: phillbaker/terraform-provider-elasticsearch#340

[barthel]

Interesting concept, used by https://registry.terraform.io/providers/elastic/elasticstack/latest/docs

[barthel]

Hi @opensearch-project/alerting-plugin , @opensearch-project/notifications Can you look into this issue?

[bbarani]

@lezzago @praveensameneni Can you please add your inputs here?

[lezzago]

@barthel and @bbarani, what kind of inputs/feedback is needed from the Alerting and Notifications team for this?