Quarkus jctools REF_ELEMENT_SHIFT substitution conflicts with DD substitution

[luneo7]

Describe the bug

Both Quarkus and DataDog agent are trying to do JCTools substitutions which renders the following error:

Error: Substition: org.jctools.util.UnsafeRefArrayAccess.REF_ELEMENT_SHIFT conflicts with previously registered: org.jctools.util.UnsafeRefArrayAccess.REF_ELEMENT_SHIFT
com.oracle.svm.core.util.UserError$UserException: Substition: org.jctools.util.UnsafeRefArrayAccess.REF_ELEMENT_SHIFT conflicts with previously registered: org.jctools.util.UnsafeRefArrayAccess.REF_ELEMENT_SHIFT
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.util.UserError.abort(UserError.java:73)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.util.UserError.guarantee(UserError.java:97)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.register(AnnotationSubstitutionProcessor.java:952)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleFieldInAliasClass(AnnotationSubstitutionProcessor.java:599)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleAliasClass(AnnotationSubstitutionProcessor.java:428)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleClass(AnnotationSubstitutionProcessor.java:395)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.init(AnnotationSubstitutionProcessor.java:351)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.createAnnotationSubstitutionProcessor(NativeImageGenerator.java:1029)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:907)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:590)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:550)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:539)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:721)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.start(NativeImageGeneratorRunner.java:143)
        at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:98)

Expected behavior

JCTools substitution works as expected

Actual behavior

Substitution fails

How to Reproduce?

Get any simple app, (https://github.com/luneo7/bazinga-repro this one will do it)...
Download DD agent (https://github.com/DataDog/dd-trace-java/releases/tag/v1.33.0)...
Change quarkus.native.additional-build-args and add the Java Agent... something like:

quarkus.native.additional-build-args= \
  -J-javaagent:/Downloads/dd-java-agent.jar,\
  -march=native

Run the native build mvn clean package -Dnative -DskipTests=true and you will get:

Output of uname -a or ver

Darwin C02C32WQMD6R 23.2.0 Darwin Kernel Version 23.2.0: Wed Nov 15 21:54:10 PST 2023; root:xnu-10002.61.3~2/RELEASE_X86_64 x86_64

Output of java -version

openjdk version "21.0.2" 2024-01-16 OpenJDK Runtime Environment GraalVM CE 21.0.2+13.1 (build 21.0.2+13-jvmci-23.1-b30) OpenJDK 64-Bit Server VM GraalVM CE 21.0.2+13.1 (build 21.0.2+13-jvmci-23.1-b30, mixed mode, sharing)

Mandrel or GraalVM version (if different from Java)

No response

Quarkus version or git rev

3.10.0

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546) Maven home: /usr/local/Cellar/maven/3.9.5/libexec Java version: 21.0.2, vendor: GraalVM Community, runtime: /Library/Java/JavaVirtualMachines/graalvm-community-openjdk-21.0.2+13.1/Contents/Home Default locale: en_CA, platform encoding: UTF-8 OS name: "mac os x", version: "14.2.1", arch: "x86_64", family: "mac"

Additional information

No response

[quarkus-bot]

/cc @Karm (mandrel), @galderz (mandrel), @zakkak (mandrel)

[luneo7]

Those are the existing substitutions in the DataDog agent:
1 - https://github.com/DataDog/dd-trace-java/blob/master/dd-java-agent/instrumentation/graal/native-image/src/main/java/datadog/trace/instrumentation/graal/nativeimage/Target_org_jctools_counters_FixedSizeStripedLongCounterFields.java
2 - https://github.com/DataDog/dd-trace-java/blob/master/dd-java-agent/instrumentation/graal/native-image/src/main/java/datadog/trace/instrumentation/graal/nativeimage/Target_org_jctools_util_UnsafeRefArrayAccess.java

[galderz]

Looking at the codebase, I see this has come up before. The trick that seems to have worked is for projects that depend on jctools to shade that dependency, e.g. io.netty.util.internal.shaded.org.jctools.util.UnsafeRefArrayAccess or io.opentelemetry.internal.shaded.jctools.util.UnsafeRefArrayAccess.

Interestingly, looking at the substitution for org.jctools.util.UnsafeRefArrayAccess in Quarkus, this is in mutiny and was added by @jponge and he references DataDog/dd-trace-java#6020. What's the story there?

[luneo7]

Yup @galderz, I've opened up a PR in the DD repo so it relocates the shaded version so there won't be no conflict anymore, just waiting for it to get a review there. (DataDog/dd-trace-java#7004)

Mutiny is going to change the queue eventually when there is new version of JCTools that does unpadded without unsafe... so that substitution there I'll be gone in the future.

[luneo7]

DataDog/dd-trace-java#7004 was merged in DD and the next release (1.35.0) will have the fix (relocation). Will close the issue here, if someone needs DD to work with Quarkus >= 3.9, you can just do like us and create a 1.34 build patched with the PR code and wait for the 1.35 (which will happen next month)

[gsmet]

/cc @jponge we still have to have this discussion about shading jctools or not. Let's try to organize something next week?

[jponge]

It's already being tracked in smallrye/smallrye-mutiny#1577 where we'll offer a JPMS-free + shaded dependencies.

Note: I don't have time this week, and I'm basically off until end of May