[BUG] Build fails due to transitive dependencies

[dbwiddis]

What is the bug?

Following merge of opensearch-project/OpenSearch#8434, transitive dependencies between OpenSearch and opensearch-java conflict for Apache Http dependencies.

Execution failed for task ':compileJava'.
> Could not resolve all dependencies for configuration ':compileClasspath'.
   > Conflict(s) found for the following module(s):
       - org.apache.httpcomponents.client5:httpclient5 between versions 5.2.1 and 5.1.4
       - org.apache.httpcomponents.core5:httpcore5 between versions 5.2.2, 5.1.5 and 5.2
       - org.apache.httpcomponents.core5:httpcore5-h2 between versions 5.2.2, 5.1.5 and 5.2

How can one reproduce the bug?

Build latest SDK against latest Opensearch and OpenSearch Client for Java

What is the expected behavior?

Code compiles

Do you have any additional context?

We need to either exclude the transitive dependency or force our own version.

[owaiskazi19]

This should be fixed by: opensearch-project/opensearch-java#554

[dbwiddis]

Yeah, and we'll break again next time one updates before the other.

[cwperks]

I'm still getting this issue even after the merge of the change in opensearch-java and publishing the latest changes to maven local.

> ./gradlew :dependencyInsight --configuration compileClasspath --dependency org.apache.httpcomponents.core5:httpcore5

org.apache.httpcomponents.core5:httpcore5:5.2.2
+--- org.apache.httpcomponents.core5:httpcore5-h2:5.2.2
|    +--- org.opensearch.client:opensearch-rest-client:3.0.0-SNAPSHOT
|    |    +--- compileClasspath
|    |    \--- org.opensearch.client:opensearch-rest-high-level-client:3.0.0-SNAPSHOT
|    |         \--- compileClasspath
|    +--- org.opensearch.client:opensearch-java:3.0.0-SNAPSHOT
|    |    \--- compileClasspath
|    \--- org.apache.httpcomponents.client5:httpclient5:5.2.1 (requested org.apache.httpcomponents.core5:httpcore5-h2:5.2)
|         +--- org.opensearch.client:opensearch-rest-client:3.0.0-SNAPSHOT (*)
|         \--- org.opensearch.client:opensearch-java:3.0.0-SNAPSHOT (*)
+--- org.opensearch.client:opensearch-java:3.0.0-SNAPSHOT (*)
\--- org.opensearch.client:opensearch-rest-client:3.0.0-SNAPSHOT (*)

I think the most pertinent line is org.apache.httpcomponents.client5:httpclient5:5.2.1 (requested org.apache.httpcomponents.core5:httpcore5-h2:5.2).

httpclient5:5.2.1 is requesting httpcore5-h2:5.2 which mismatches with 5.2.2 being requested elsewhere.

5.2.1 is the latest on maven central: https://mvnrepository.com/artifact/org.apache.httpcomponents.client5/httpclient5

[cwperks]

I had to downgrade httpcore5 to 5.2 in both core and opensearch-java to resolve this issue locally.

[reta]

@cwperks @dbwiddis I will submit the pull request shorlty

[dbwiddis]

<insert my recurring request for an OpenSearch version catalog here>

[reta]

Hehe .... I hear you, may be needs prioritization ,,,

[dbwiddis]

My background in dependency management is with Maven which seems a bit simpler but easier to resolve situations like this (e.g., order matters). Gradle has a lot of hidden switches and priorities that make these harder to diagnose!

I do think for known transitive dependencies that will have recurring conflicts on version updates, a better solution for SDK is to exclude the transitives from one option (e.g., the client) and thus always use the version in the other (e.g, OpenSearch).

Forcing our own version would work but for transitives seems to break some sort of encapsulation concept.