Determine OpenSearch Client SDK Strategy

[chelma]

Task

The UTF will need a way to "talk" to Elasticsearch/OpenSearch clusters of multiple versions simultaneously. The Elastic Python client has a good table explaining compatibility and publishes older client major versions as separate packages so they can be consumed in parallel without namespace issues [1]. When I look at the OpenSearch client docs [2], I don’t see anything about publishing old major versions separately, and I don’t see any discussion about compatibility with newer minor versions of the OpenSearch Core engine (newest mentioned is 2.0.1).

We need to investigate this to see what our options are.

[1] https://github.com/elastic/elasticsearch-py/blob/7.10/docs/sphinx/index.rst
[2] https://github.com/opensearch-project/opensearch-py/blob/main/COMPATIBILITY.md

Acceptance Criteria

At the end of this task, we should have answers to the following questions:

• Is Python a viable language choice for writing tests? Or must we use Java, which appears to have more support?
• If we are using Python, how are we going to handle situations like needing to talk to an OS 2.4 cluster, or talking to both an OS 1.0 and OS 2.1 cluster which ostensibly means we need multiple client versions?

[chelma]

Talking to the client owners. @nhtruong indicates that they are not currently offering a separate publication of the v1.x client. Cut the team a ticket [1] requesting an update to their compatibility guide, as it appears outdated.

[1] opensearch-project/opensearch-py#252

[chelma]

One option here is that it feels like we might be able to get away with making/running our own version of an OpenSearch Client SDK since the number of specific calls we need to make appears small. We'd wrap any third-party SDK in our own code anyways just as a best practice. We can then swap out our implementation to the official SDK when appropriate. This would prevent us from being blocked on the OS Python client's availability.

[chelma]

Also cut an issue [1] requesting the team publish older major versions as separate packages.

[1] opensearch-project/opensearch-py#253

[chelma]

Per my investigation and a discussion w/ @kartg on 2022-12-16, The answer here appears to be to not use any language client and instead build the tool off of the REST interface that Elasticsearch/OpenSearch offer. This is due to Kartik's excellent point that incompatibilities/bugs in the client, separate from the core engine, introduce an additional layer of complexity that must be accounted for and additionally that many users build their own clients anyways. The REST interface seems to be the most widely-applicable standard we can adhere to.

[chelma]

Task to implement strategy selected is here: #50

Resolving this task as it is complete