Added support for signing service name (partial support for OpenSearch Serverless)

CHANGELOG.md

@@ -2,12 +2,14 @@
 Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 
 ## [Unreleased]
+
 ### Added
-- Github workflow for changelog verification ([#239](https://github.com/opensearch-project/opensearch-java/pull/239))
-- Github workflow for dependabot PRs ([#247](https://github.com/opensearch-project/opensearch-java/pull/247))
+- Add GitHub workflow for changelog verification ([#239](https://github.com/opensearch-project/opensearch-java/pull/239))
+- Add GitHub workflow for dependabot PRs ([#247](https://github.com/opensearch-project/opensearch-java/pull/247))
 - Add javadoc link for the client ([#255](https://github.com/opensearch-project/opensearch-java/pull/255))
+- Add support for signing service name in AwsSdk2Transport ([#324](https://github.com/opensearch-project/opensearch-java/pull/324))
 - Add 1-click release workflows ([#321](https://github.com/opensearch-project/opensearch-java/pull/321))
-- Introduce new OpenSearchTransport based on Apache HttpClient 5 ([#281](https://github.com/opensearch-project/opensearch-java/pull/281))
+- Add new `OpenSearchTransport` based on Apache HttpClient 5 ([#281](https://github.com/opensearch-project/opensearch-java/pull/281))
 
 ### Dependencies
 - Bumps `classgraph` from 4.8.149 to 4.8.154
@@ -22,23 +24,22 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 - Update tests to use JUnit's Assert ([#244](https://github.com/opensearch-project/opensearch-java/pull/244))
 - Add support to parse sub-aggregations from filter/nested aggregations ([#234](https://github.com/opensearch-project/opensearch-java/pull/234))
 - Add timeout and throttle to the jenkins workflows ([#231](https://github.com/opensearch-project/opensearch-java/pull/231)) 
-- Updating maintainers, admins and documentation ([#248](https://github.com/opensearch-project/opensearch-java/pull/248))
+- Update maintainers, admins and documentation ([#248](https://github.com/opensearch-project/opensearch-java/pull/248))
 - Migrate client transports to Apache HttpClient / Core 5.x ([#246](https://github.com/opensearch-project/opensearch-java/pull/246))
 - Document how to use Java 8 Time (JSR 310) objects ([#251](https://github.com/opensearch-project/opensearch-java/pull/251))
-- Fixing version and build ([#254](https://github.com/opensearch-project/opensearch-java/pull/254))
+- Fix version and build ([#254](https://github.com/opensearch-project/opensearch-java/pull/254))
 - Update Gradle to 7.6 ([#309](https://github.com/opensearch-project/opensearch-java/pull/309))
 - Prevent SPI calls at runtime ([#293](https://github.com/opensearch-project/opensearch-java/pull/293))
 
 ### Deprecated
 
 ### Removed
-- Removed support for unsupported dynamic_templates in bulk ([#276](https://github.com/opensearch-project/opensearch-java/pull/276))
+- Remove support for unsupported dynamic_templates in bulk ([#276](https://github.com/opensearch-project/opensearch-java/pull/276))
 
 ### Fixed
 - Make ChildrenAggregate as a SingleBucketAggregate ([#306](https://github.com/opensearch-project/opensearch-java/pull/306))
 - Fix /_nodes/stats, /_nodes/info throwing serialization error ([#315](https://github.com/opensearch-project/opensearch-java/pull/315))
 
 ### Security
 
-
 [Unreleased]: https://github.com/opensearch-project/opensearch-java/compare/2.0...HEAD

DEVELOPER_GUIDE.md

@@ -7,6 +7,7 @@
     - [Run Tests](#run-tests)
       - [Unit Tests](#unit-tests)
       - [Integration Tests](#integration-tests)
+      - [AWS Transport Integration Tests](#aws-transport-integration-tests)
   - [Use an Editor](#use-an-editor)
     - [IntelliJ IDEA](#intellij-idea)
     - [Visual Studio Code](#visual-studio-code)
@@ -64,6 +65,20 @@ Run integration tests after starting OpenSearch cluster:
 ./gradlew clean integrationTest
 ```
 
+#### AWS Transport Integration Tests
+
+To run integration tests for the AWS transport client, ensure working AWS credentials and specify your OpenSearch domain and region as follows:
+
+```
+./gradlew integrationTest --tests "*AwsSdk2*" -Dtests.awsSdk2support.domainHost=search-...us-west-2.es.amazonaws.com -Dtests.awsSdk2support.domainRegion=us-west-2
+```
+
+For OpenSearch Serverless, change the signing service name.
+
+```
+./gradlew integrationTest --tests "*AwsSdk2*" -Dtests.awsSdk2support.domainHost=....us-west-2.aoss.amazonaws.com -Dtests.awsSdk2support.domainRegion=us-west-2 -Dtests.awsSdk2support.serviceName=aoss
+```
+
 ## Use an Editor
 
 ### IntelliJ IDEA

USER_GUIDE.md

@@ -2,15 +2,17 @@
 
 - [User Guide](#user-guide)
   - [Sample data](#sample-data)
+    - [IndexData class](#indexdata-class)
   - [Create a client](#create-a-client)
   - [Create an index](#create-an-index)
   - [Index data](#index-data)
-  - [Search for the document](#search-for-the-documents)
+  - [Search for the documents](#search-for-the-documents)
   - [Search documents using a match query](#search-documents-using-a-match-query)
   - [Aggregations](#aggregations)
   - [Delete the document](#delete-the-document)
   - [Delete the index](#delete-the-index)
-  - [Aggregations](#aggregations)
+- [Using different transport options](#using-different-transport-options)
+  - [Amazon Managed OpenSearch](#amazon-managed-opensearch)
 
 ## Sample data
 
@@ -150,3 +152,28 @@ client.delete(d -> d.index(index).id("1"));
 DeleteIndexRequest deleteIndexRequest = new DeleteRequest.Builder().index(index).build();
 DeleteIndexResponse deleteIndexResponse = client.indices().delete(deleteIndexRequest);
 ```
+
+# Using different transport options
+
+## Amazon Managed OpenSearch
+
+Use `AwsSdk2Transport` to make requests to Amazon Managed OpenSearch.
+
+```java
+SdkHttpClient httpClient = ApacheHttpClient.builder().build();
+
+OpenSearchClient client = new OpenSearchClient(
+    new AwsSdk2Transport(
+        httpClient,
+        "search-...us-west-2.es.amazonaws.com", // OpenSearch endpoint, without https://
+        "es" // signing service name
+        Region.US_WEST_2, // signing service region
+        AwsSdk2TransportOptions.builder().build()
+    )
+);
+
+InfoResponse info = client.info();
+System.out.println(info.version().distribution() + ": " + info.version().number());
+
+httpClient.close();
+```

java-client/build.gradle.kts

@@ -135,6 +135,8 @@ val integrationTest = task<Test>("integrationTest") {
     systemProperty("password", System.getProperty("password", "admin"))
     systemProperty("tests.awsSdk2support.domainHost",
             System.getProperty("tests.awsSdk2support.domainHost", null))
+    systemProperty("tests.awsSdk2support.serviceName",
+            System.getProperty("tests.awsSdk2support.serviceName", "es"))
     systemProperty("tests.awsSdk2support.domainRegion",
             System.getProperty("tests.awsSdk2support.domainRegion", "us-east-1"))
 }

java-client/src/main/java/org/opensearch/client/transport/aws/AwsSdk2Transport.java

@@ -60,34 +60,34 @@
 
 /**
  * Implementation of the OpenSearchTransport interface that sends signed requests using
- * the AWS v2 SDK HTTP clients, to connect to an AWS OpenSearch service using IAM authentication
+ * the AWS v2 SDK HTTP clients, to connect to an AWS OpenSearch service using IAM authentication.
  */
 public class AwsSdk2Transport implements OpenSearchTransport {
     /**
      * By default, requests that exceed this size will be automatically compressed.
-     * {@link AwsSdk2TransportOptions} can be used to override this setting or disable compresson.
+     * {@link AwsSdk2TransportOptions} can be used to override this setting or disable compression.
      */
     public static final Integer DEFAULT_REQUEST_COMPRESSION_SIZE = 8192;
 
     private static final byte[] NO_BYTES = new byte[0];
     private final SdkHttpClient httpClient;
     private final SdkAsyncHttpClient asyncHttpClient;
     private final String host;
+    private final String signingServiceName;
     private final Region signingRegion;
     private final JsonpMapper defaultMapper;
     private final AwsSdk2TransportOptions transportOptions;
 
     /**
-     * Create an {@link OpenSearchTransport} with a SYNCHRONOUS AWS Http client
+     * Create an {@link OpenSearchTransport} with a synchronous AWS HTTP client.
      * <p>
      * Note that asynchronous OpenSearch requests sent through this transport will be dispatched
      * *synchronously* on the calling thread.
      *
-     * @param httpClient  HTTP client to use for OpenSearch requests
-     * @param host        The fully qualified domain name to connect to
-     * @param signingRegion The AWS region for which requests will be signed.  This should typically match
-     *                      the region in `host`.
-     * @param options Options that apply to all requests.  Can be null.  Create with
+     * @param httpClient HTTP client to use for OpenSearch requests.
+     * @param host The fully qualified domain name to connect to.
+     * @param signingRegion The AWS region for which requests will be signed. This should typically match the region in `host`.
+     * @param options Options that apply to all requests. Can be null. Create with
      *                {@link AwsSdk2TransportOptions#builder()} and use these to specify non-default credentials,
      *                compression options, etc.
      */
@@ -96,20 +96,42 @@ public AwsSdk2Transport(
             @Nonnull String host,
             @Nonnull Region signingRegion,
             @CheckForNull AwsSdk2TransportOptions options) {
-        this(httpClient, null, host, signingRegion, options);
+        this(httpClient, null, host, "es", signingRegion, options);
     }
 
     /**
-     * Create an {@link OpenSearchTransport} with an ASYNCHRONOUS AWS Http client
+     * Create an {@link OpenSearchTransport} with a synchronous AWS HTTP client.
+     * <p>
+     * Note that asynchronous OpenSearch requests sent through this transport will be dispatched
+     * *synchronously* on the calling thread.
+     *
+     * @param httpClient HTTP client to use for OpenSearch requests.
+     * @param host The fully qualified domain name to connect to.
+     * @param signingServiceName The AWS signing service name, one of `es` (Amazon OpenSearch) or `aoss` (Amazon OpenSearch Serverless).
+     * @param signingRegion The AWS region for which requests will be signed. This should typically match the region in `host`.
+     * @param options Options that apply to all requests. Can be null. Create with
+     *                {@link AwsSdk2TransportOptions#builder()} and use these to specify non-default credentials,
+     *                compression options, etc.
+     */
+    public AwsSdk2Transport(
+            @Nonnull SdkHttpClient httpClient,
+            @Nonnull String host,
+            @Nonnull String signingServiceName,
+            @Nonnull Region signingRegion,
+            @CheckForNull AwsSdk2TransportOptions options) {
+        this(httpClient, null, host, signingServiceName, signingRegion, options);
+    }
+
+    /**
+     * Create an {@link OpenSearchTransport} with an asynchronous AWS HTTP client
      * <p>
      * Note that synchronous OpenSearch requests sent through this transport will be dispatched
      * using the asynchronous client, but the calling thread will block until they are complete.
      *
-     * @param asyncHttpClient HTTP client to use for OpenSearch requests
-     * @param host            The target host
-     * @param signingRegion The AWS region for which requests will be signed.  This should typically match
-     *                      the region in `host`.
-     * @param options Options that apply to all requests.  Can be null.  Create with
+     * @param asyncHttpClient HTTP client to use for OpenSearch requests.
+     * @param host The target host.
+     * @param signingRegion The AWS region for which requests will be signed. This should typically match region in `host`.
+     * @param options Options that apply to all requests. Can be null. Create with
      *                {@link AwsSdk2TransportOptions#builder()} and use these to specify non-default credentials,
      *                compression options, etc.
      */
@@ -118,7 +140,53 @@ public AwsSdk2Transport(
             @Nonnull String host,
             @Nonnull Region signingRegion,
             @CheckForNull AwsSdk2TransportOptions options) {
-        this(null, asyncHttpClient, host, signingRegion, options);
+        this(null, asyncHttpClient, host, "es", signingRegion, options);
+    }
+
+    /**
+     * Create an {@link OpenSearchTransport} with an asynchronous AWS HTTP client.
+     * <p>
+     * Note that synchronous OpenSearch requests sent through this transport will be dispatched
+     * using the asynchronous client, but the calling thread will block until they are complete.
+     *
+     * @param asyncHttpClient HTTP client to use for OpenSearch requests.
+     * @param host The target host.
+     * @param signingServiceName The AWS signing service name, one of `es` (Amazon OpenSearch) or `aoss` (Amazon OpenSearch Serverless).
+     * @param signingRegion The AWS region for which requests will be signed. This should typically match the region in `host`.
+     * @param options Options that apply to all requests. Can be null. Create with
+     *                {@link AwsSdk2TransportOptions#builder()} and use these to specify non-default credentials,
+     *                compression options, etc.
+     */
+    public AwsSdk2Transport(
+            @Nonnull SdkAsyncHttpClient asyncHttpClient,
+            @Nonnull String host,
+            @Nonnull String signingServiceName,
+            @Nonnull Region signingRegion,
+            @CheckForNull AwsSdk2TransportOptions options) {
+        this(null, asyncHttpClient, host, signingServiceName, signingRegion, options);
+    }
+
+    /**
+     * Create an {@link OpenSearchTransport} with both synchronous and asynchronous AWS HTTP clients.
+     * <p>
+     * The synchronous client will be used for synchronous OpenSearch requests, and the asynchronous client
+     * will be used for asynchronous HTTP requests.
+     *
+     * @param httpClient HTTP client to use for OpenSearch requests.
+     * @param asyncHttpClient HTTP client to use for synchronous OpenSearch requests.
+     * @param host The fully qualified domain name to connect to.
+     * @param signingRegion The AWS region for which requests will be signed. This should typically match the region in `host`.
+     * @param options Options that apply to all requests. Can be null. Create with
+     *                {@link AwsSdk2TransportOptions#builder()} and use these to specify non-default credentials,
+     *                compression options, etc.
+     */
+    public AwsSdk2Transport(
+            @CheckForNull SdkHttpClient httpClient,
+            @CheckForNull SdkAsyncHttpClient asyncHttpClient,
+            @Nonnull String host,
+            @Nonnull Region signingRegion,
+            @CheckForNull AwsSdk2TransportOptions options) {
+        this(httpClient, asyncHttpClient, host, "es", signingRegion, options);
     }
 
     /**
@@ -127,19 +195,20 @@ public AwsSdk2Transport(
      * The synchronous client will be used for synchronous OpenSearch requests, and the asynchronous client
      * will be used for asynchronous HTTP requests.
      *
-     * @param httpClient  HTTP client to use for OpenSearch requests
-     * @param asyncHttpClient HTTP client to use for synchronous OpenSearch requests
-     * @param host        The fully qualified domain name to connect to
-     * @param signingRegion The AWS region for which requests will be signed.  This should typically match
-     *                      the region in `host`.
-     * @param options Options that apply to all requests.  Can be null.  Create with
+     * @param httpClient HTTP client to use for OpenSearch requests.
+     * @param asyncHttpClient HTTP client to use for synchronous OpenSearch requests.
+     * @param host The fully qualified domain name to connect to.
+     * @param signingRegion The AWS region for which requests will be signed. This should typically match the region in `host`.
+     * @param signingServiceName The AWS signing service name, one of `es` (Amazon OpenSearch) or `aoss` (Amazon OpenSearch Serverless).
+     * @param options Options that apply to all requests. Can be null. Create with
      *                {@link AwsSdk2TransportOptions#builder()} and use these to specify non-default credentials,
      *                compression options, etc.
      */
     public AwsSdk2Transport(
             @CheckForNull SdkHttpClient httpClient,
             @CheckForNull SdkAsyncHttpClient asyncHttpClient,
             @Nonnull String host,
+            @Nonnull String signingServiceName,
             @Nonnull Region signingRegion,
             @CheckForNull AwsSdk2TransportOptions options) {
         if (httpClient == null && asyncHttpClient == null)
@@ -150,6 +219,7 @@ public AwsSdk2Transport(
         this.httpClient = httpClient;
         this.asyncHttpClient = asyncHttpClient;
         this.host = host;
+        this.signingServiceName = signingServiceName;
         this.signingRegion = signingRegion;
         this.transportOptions = options != null ? options : AwsSdk2TransportOptions.builder().build();
         this.defaultMapper = Optional.ofNullable(options)
@@ -314,7 +384,7 @@ private <RequestT> SdkHttpFullRequest prepareRequest(
 
         Aws4SignerParams signerParams = Aws4SignerParams.builder()
                 .awsCredentials(credentials.resolveCredentials())
-                .signingName("es")
+                .signingName(this.signingServiceName)
                 .signingRegion(signingRegion)
                 .build();
         return Aws4Signer.create().sign(req.build(), signerParams);

java-client/src/test/java/org/opensearch/client/opensearch/integTest/aws/AwsSdk2TransportTestCase.java

@@ -83,13 +83,15 @@ protected OpenSearchClient getClient(
             transport = new AwsSdk2Transport(
                     getAsyncHttpClient(),
                     getTestClusterHost(),
+                    getTestClusterServiceName(),
                     getTestClusterRegion(),
                     getTransportOptions().build()
             );
         } else {
             transport = new AwsSdk2Transport(
                     getHttpClient(),
                     getTestClusterHost(),
+                    getTestClusterServiceName(),
                     getTestClusterRegion(),
                     getTransportOptions().build()
             );
@@ -107,13 +109,15 @@ protected OpenSearchAsyncClient getAsyncClient(
             transport = new AwsSdk2Transport(
                     getAsyncHttpClient(),
                     getTestClusterHost(),
+                    getTestClusterServiceName(),
                     getTestClusterRegion(),
                     getTransportOptions().build()
             );
         } else {
             transport = new AwsSdk2Transport(
                     getHttpClient(),
                     getTestClusterHost(),
+                    getTestClusterServiceName(),
                     getTestClusterRegion(),
                     getTransportOptions().build()
             );
@@ -131,13 +135,15 @@ protected OpenSearchIndicesClient getIndexesClient(
             transport = new AwsSdk2Transport(
                     getAsyncHttpClient(),
                     getTestClusterHost(),
+                    getTestClusterServiceName(),
                     getTestClusterRegion(),
                     getTransportOptions().build()
             );
         } else {
             transport = new AwsSdk2Transport(
                     getHttpClient(),
                     getTestClusterHost(),
+                    getTestClusterServiceName(),
                     getTestClusterRegion(),
                     getTransportOptions().build()
             );
@@ -150,6 +156,11 @@ protected String getTestClusterHost() {
         return cluster;
     }
 
+    protected String getTestClusterServiceName() {
+        String cluster = System.getProperty("tests.awsSdk2support.serviceName");
+        return cluster;
+    }
+
     protected Region getTestClusterRegion() {
         String region = System.getProperty("tests.awsSdk2support.domainRegion");
         return region != null ? Region.of(region) : Region.US_EAST_1;