Support Test AOT processing with GraalVM tracing agent and Native Build Tools #30281

linghengqian · 2023-04-03T15:38:37Z

When using Spring's Gradle Plugin, the Tracing Agent of GraalVM Native Build Tools cannot be used normally. This issue is an extension of Add support for com.baomidou:dynamic-datasource-spring-boot-starter:3.6.1 oracle/graalvm-reachability-metadata#154, because it involves Spring's Gradle Plugin.
Related, the Spring Boot OSS version used for verification is 3.0.5. The minimal reproducible Example and corresponding reproduction steps are located at https://github.com/linghengqian/spring-plugin-graalvm-agent-test .

sudo apt install unzip zip curl sed -y
curl -s "https://get.sdkman.io" | bash
source "$HOME/.sdkman/bin/sdkman-init.sh"
sdk install java 22.3.1.r17-grl
sdk use java 22.3.1.r17-grl
gu install native-image
sudo apt-get install build-essential libz-dev zlib1g-dev -y

git clone [email protected]:linghengqian/spring-plugin-graalvm-agent-test.git
cd ./spring-plugin-graalvm-agent-test/
./gradlew -Pagent clean test
./gradlew metadataCopy --task test
./gradlew clean nativeTest

More early investigations are at Make ShardingSphere Proxy in GraalVM Native Image form available apache/shardingsphere#21347. This issue may also be related to the problem encountered at Add support for Spring Cloud Alibaba modules oracle/graalvm-reachability-metadata#260.
I noticed that when Spring Boot's Gradle Plugin is introduced, the following error will occur when executing ./gradlew -Pagent clean test.

> Task :processTestAot
[native-image-plugin] Instrumenting task with the native-image-agent: processTestAot
[0.214s][warning][jni,resolve] Re-registering of platform native method: jdk.internal.misc.Unsafe.allocateInstance(Ljava/lang/Class;)Ljava/lang/Object; from code in a different classloader
20:42:10.489 [main] INFO org.springframework.test.context.aot.TestClassScanner -- Scanning for Spring test classes in all packages in classpath roots [/home/linghengqian/TwinklingLiftWorks/git/public/spring-plugin-graalvm-agent-test/build/classes/java/test, /home/linghengqian/TwinklingLiftWorks/git/public/spring-plugin-graalvm-agent-test/build/resources/test]
Exception in thread "main" java.lang.IllegalStateException: Cannot perform AOT processing during AOT run-time execution
        at org.springframework.util.Assert.state(Assert.java:76)
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:126)
        at org.springframework.test.context.aot.TestAotProcessor.performAotProcessing(TestAotProcessor.java:91)
        at org.springframework.test.context.aot.TestAotProcessor.doProcess(TestAotProcessor.java:72)
        at org.springframework.test.context.aot.TestAotProcessor.doProcess(TestAotProcessor.java:39)
        at org.springframework.context.aot.AbstractAotProcessor.process(AbstractAotProcessor.java:82)
        at org.springframework.boot.test.context.SpringBootTestAotProcessor.main(SpringBootTestAotProcessor.java:63)

> Task :processTestAot FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':processTestAot'.
> Process 'command '/home/linghengqian/.sdkman/candidates/java/22.3.1.r17-grl/bin/java'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 11s
5 actionable tasks: 4 executed, 1 up-to-date

If I just execute ./gradlew clean test, everything works fine, but cannot use user-code-filter.json to generate GraalVM reachability metadata in Conditional Agent Mode.
Spring AOT overrides my configuration for Native Build Tools. I wonder if this is the expected behavior of Classpath-based filtering of the native configuration generation #28163 ? Or am I overlooking something?
As a follow-up to Add support for com.baomidou:dynamic-datasource-spring-boot-starter:3.6.1 oracle/graalvm-reachability-metadata#154, I opened this issue on the Spring Boot side.

The text was updated successfully, but these errors were encountered:

snicoll · 2023-04-04T14:06:41Z

@sbrannen process-test-aot is failing if NativeDetector.inNativeImage() is true. As far as I understand it, the agent is going to set that property. I do wonder why this check exists at the process AOT level...

sbrannen · 2023-05-10T15:27:34Z

@sbrannen process-test-aot is failing if NativeDetector.inNativeImage() is true. As far as I understand it, the agent is going to set that property.

That's correct. See my comments linked from this issue's description.

I do wonder why this check exists at the process AOT level...

It exists to ensure that test AOT processing does not run in an invalid state.

However, performing test AOT processing while running with the agent and Native Build Tools was not something that I originally thought about as a use case we would need to support.

That's why I asked @linghengqian to create this issue.

sbrannen · 2023-05-10T15:55:47Z

@linghengqian, the fix for this will be available in the next 6.0.9 snapshot.

We'd be grateful if you could try it out and let us know if it resolves the issue for you.

Thanks

oubidar-Abderrahim · 2023-05-16T08:51:22Z

Hi @sbrannen, is this fix also happening for SpringBoot? I see a similar issue with Spring Boot 3.0.6

sbrannen · 2023-05-16T09:02:43Z

Hi @sbrannen, is this fix also happening for SpringBoot? I see a similar issue with Spring Boot 3.0.6

I created #30511 so that the Boot team can investigate.

snicoll · 2023-05-21T08:28:56Z

Unfortunately, this turned to be an incomplete change in framework so I've moved that Spring Boot issue back here, see #30511

linghengqian · 2023-06-07T09:14:36Z

@sbrannen I'm sorry for my response time because I've been very busy for the last month. I am in https://github.com/linghengqian/spring-plugin-graalvm-agent-test 's README update the Error Log. Look org.springframework.test.context.aot.TestContextAotGenerator.** exist some content I don't know how to explain.

Log

$ ./gradlew -Pagent clean test

> Task :processTestAot
[native-image-plugin] Instrumenting task with the native-image-agent: processTestAot
[0.106s][warning][jni,resolve] Re-registering of platform native method: jdk.internal.misc.Unsafe.allocateInstance(Ljava/lang/Class;)Ljava/lang/Object; from code in a different classloader
17:05:04.089 [main] INFO org.springframework.test.context.aot.TestClassScanner -- Scanning for Spring test classes in all packages in classpath roots [/home/linghengqian/TwinklingLiftWorks/git/public/spring-plugin-graalvm-agent-test/build/classes/java/test, /home/linghengqian/TwinklingLiftWorks/git/public/spring-plugin-graalvm-agent-test/build/resources/test]

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v3.1.0)

2023-06-07T17:05:06.823+08:00 ERROR 11458 --- [           main] o.s.boot.SpringApplication               : Application run failed

java.lang.IllegalArgumentException: Could not find class [com.lingh.AddRemoveDatasourceTest__ApplicationContextInitializer]
        at org.springframework.util.ClassUtils.resolveClassName(ClassUtils.java:334) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.instantiateInitializer(AotApplicationContextInitializer.java:80) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.initialize(AotApplicationContextInitializer.java:71) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.lambda$forInitializerClasses$0(AotApplicationContextInitializer.java:61) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:606) ~[spring-boot-3.1.0.jar:3.1.0]
        at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:386) ~[spring-boot-3.1.0.jar:3.1.0]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:310) ~[spring-boot-3.1.0.jar:3.1.0]
        at org.springframework.boot.test.context.SpringBootContextLoader.lambda$loadContext$3(SpringBootContextLoader.java:137) ~[spring-boot-test-3.1.0.jar:3.1.0]
        at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:58) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:46) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.boot.SpringApplication.withHook(SpringApplication.java:1405) ~[spring-boot-3.1.0.jar:3.1.0]
        at org.springframework.boot.test.context.SpringBootContextLoader$ContextLoaderHook.run(SpringBootContextLoader.java:545) ~[spring-boot-test-3.1.0.jar:3.1.0]
        at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:137) ~[spring-boot-test-3.1.0.jar:3.1.0]
        at org.springframework.boot.test.context.SpringBootContextLoader.loadContextForAotProcessing(SpringBootContextLoader.java:113) ~[spring-boot-test-3.1.0.jar:3.1.0]
        at org.springframework.test.context.aot.TestContextAotGenerator.loadContextForAotProcessing(TestContextAotGenerator.java:263) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:232) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.lambda$processAheadOfTime$4(TestContextAotGenerator.java:204) ~[spring-test-6.0.9.jar:6.0.9]
        at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721) ~[na:na]
        at org.springframework.util.MultiValueMapAdapter.forEach(MultiValueMapAdapter.java:179) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:196) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:158) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.performAotProcessing(TestAotProcessor.java:91) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.doProcess(TestAotProcessor.java:72) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.doProcess(TestAotProcessor.java:39) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AbstractAotProcessor.process(AbstractAotProcessor.java:82) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.boot.test.context.SpringBootTestAotProcessor.main(SpringBootTestAotProcessor.java:63) ~[spring-boot-test-3.1.0.jar:3.1.0]
Caused by: java.lang.ClassNotFoundException: com.lingh.AddRemoveDatasourceTest__ApplicationContextInitializer
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) ~[na:na]
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) ~[na:na]
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[na:na]
        at java.base/java.lang.Class.forName0(Native Method) ~[na:na]
        at java.base/java.lang.Class.forName(Class.java:467) ~[na:na]
        at org.springframework.util.ClassUtils.forName(ClassUtils.java:284) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.util.ClassUtils.resolveClassName(ClassUtils.java:324) ~[spring-core-6.0.9.jar:6.0.9]
        ... 25 common frames omitted

2023-06-07T17:05:06.842+08:00  WARN 11458 --- [           main] o.s.t.c.aot.TestContextAotGenerator      : Failed to generate AOT artifacts for test classes [com.lingh.AddRemoveDatasourceTest]

org.springframework.test.context.aot.TestContextAotException: Failed to load ApplicationContext for AOT processing for test class [com.lingh.AddRemoveDatasourceTest]
        at org.springframework.test.context.aot.TestContextAotGenerator.loadContextForAotProcessing(TestContextAotGenerator.java:272) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:232) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.lambda$processAheadOfTime$4(TestContextAotGenerator.java:204) ~[spring-test-6.0.9.jar:6.0.9]
        at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721) ~[na:na]
        at org.springframework.util.MultiValueMapAdapter.forEach(MultiValueMapAdapter.java:179) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:196) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestContextAotGenerator.processAheadOfTime(TestContextAotGenerator.java:158) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.performAotProcessing(TestAotProcessor.java:91) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.doProcess(TestAotProcessor.java:72) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.test.context.aot.TestAotProcessor.doProcess(TestAotProcessor.java:39) ~[spring-test-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AbstractAotProcessor.process(AbstractAotProcessor.java:82) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.boot.test.context.SpringBootTestAotProcessor.main(SpringBootTestAotProcessor.java:63) ~[spring-boot-test-3.1.0.jar:3.1.0]
Caused by: java.lang.IllegalArgumentException: Could not find class [com.lingh.AddRemoveDatasourceTest__ApplicationContextInitializer]
        at org.springframework.util.ClassUtils.resolveClassName(ClassUtils.java:334) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.instantiateInitializer(AotApplicationContextInitializer.java:80) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.initialize(AotApplicationContextInitializer.java:71) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.context.aot.AotApplicationContextInitializer.lambda$forInitializerClasses$0(AotApplicationContextInitializer.java:61) ~[spring-context-6.0.9.jar:6.0.9]
        at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:606) ~[spring-boot-3.1.0.jar:3.1.0]
        at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:386) ~[spring-boot-3.1.0.jar:3.1.0]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:310) ~[spring-boot-3.1.0.jar:3.1.0]
        at org.springframework.boot.test.context.SpringBootContextLoader.lambda$loadContext$3(SpringBootContextLoader.java:137) ~[spring-boot-test-3.1.0.jar:3.1.0]
        at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:58) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:46) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.boot.SpringApplication.withHook(SpringApplication.java:1405) ~[spring-boot-3.1.0.jar:3.1.0]
        at org.springframework.boot.test.context.SpringBootContextLoader$ContextLoaderHook.run(SpringBootContextLoader.java:545) ~[spring-boot-test-3.1.0.jar:3.1.0]
        at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:137) ~[spring-boot-test-3.1.0.jar:3.1.0]
        at org.springframework.boot.test.context.SpringBootContextLoader.loadContextForAotProcessing(SpringBootContextLoader.java:113) ~[spring-boot-test-3.1.0.jar:3.1.0]
        at org.springframework.test.context.aot.TestContextAotGenerator.loadContextForAotProcessing(TestContextAotGenerator.java:263) ~[spring-test-6.0.9.jar:6.0.9]
        ... 11 common frames omitted
Caused by: java.lang.ClassNotFoundException: com.lingh.AddRemoveDatasourceTest__ApplicationContextInitializer
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) ~[na:na]
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) ~[na:na]
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[na:na]
        at java.base/java.lang.Class.forName0(Native Method) ~[na:na]
        at java.base/java.lang.Class.forName(Class.java:467) ~[na:na]
        at org.springframework.util.ClassUtils.forName(ClassUtils.java:284) ~[spring-core-6.0.9.jar:6.0.9]
        at org.springframework.util.ClassUtils.resolveClassName(ClassUtils.java:324) ~[spring-core-6.0.9.jar:6.0.9]
        ... 25 common frames omitted


> Task :test
[native-image-plugin] Instrumenting task with the native-image-agent: test
[0.098s][warning][jni,resolve] Re-registering of platform native method: jdk.internal.misc.Unsafe.allocateInstance(Ljava/lang/Class;)Ljava/lang/Object; from code in a different classloader

AddRemoveDatasourceTest > testAddAndRemoveDataSource() FAILED
    java.lang.IllegalStateException at DefaultCacheAwareContextLoaderDelegate.java:142
        Caused by: java.lang.IllegalArgumentException at ClassUtils.java:334
            Caused by: java.lang.ClassNotFoundException at BuiltinClassLoader.java:641

1 test completed, 1 failed

> Task :test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///home/linghengqian/TwinklingLiftWorks/git/public/spring-plugin-graalvm-agent-test/build/reports/tests/test/index.html

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/8.0/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 11s
7 actionable tasks: 7 executed

spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged or decided on label Apr 3, 2023

linghengqian mentioned this issue Apr 3, 2023

Add support for com.baomidou:dynamic-datasource-spring-boot-starter:3.6.1 oracle/graalvm-reachability-metadata#154

Closed

2 tasks

snicoll transferred this issue from spring-projects/spring-boot Apr 4, 2023

rstoyanchev added the theme: aot An issue related to Ahead-of-time processing label Apr 18, 2023

linghengqian mentioned this issue Apr 24, 2023

添加对spring native的支持 baomidou/dynamic-datasource#465

Closed

sbrannen self-assigned this May 10, 2023

sbrannen added the in: test Issues in the test module label May 10, 2023

sbrannen added type: enhancement A general enhancement and removed status: waiting-for-triage An issue we've not yet triaged or decided on labels May 10, 2023

sbrannen added this to the 6.0.x milestone May 10, 2023

sbrannen changed the title ~~When using Spring's Gradle Plugin, the Tracing Agent of GraalVM Native Build Tools cannot be used normally~~ Test AOT processing fails with GraalVM tracing agent and Native Build Tools May 10, 2023

sbrannen closed this as completed in 1113096 May 10, 2023

sbrannen modified the milestones: 6.0.x, 6.0.9 May 10, 2023

sbrannen changed the title ~~Test AOT processing fails with GraalVM tracing agent and Native Build Tools~~ Support Test AOT processing with GraalVM tracing agent and Native Build Tools May 11, 2023

sbrannen mentioned this issue May 21, 2023

Support for AOT processing with GraalVM tracing agent is not consistent #30511

Closed

yuluo-yx mentioned this issue Jun 8, 2023

The metadata generated by the maven plugin and the gradle plugin do not match! oracle/graal#6758

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support Test AOT processing with GraalVM tracing agent and Native Build Tools #30281

Support Test AOT processing with GraalVM tracing agent and Native Build Tools #30281

linghengqian commented Apr 3, 2023 •

edited by snicoll

Loading

snicoll commented Apr 4, 2023

sbrannen commented May 10, 2023

sbrannen commented May 10, 2023

oubidar-Abderrahim commented May 16, 2023

sbrannen commented May 16, 2023

snicoll commented May 21, 2023

linghengqian commented Jun 7, 2023

Support Test AOT processing with GraalVM tracing agent and Native Build Tools #30281

Support Test AOT processing with GraalVM tracing agent and Native Build Tools #30281

Comments

linghengqian commented Apr 3, 2023 • edited by snicoll Loading

snicoll commented Apr 4, 2023

sbrannen commented May 10, 2023

sbrannen commented May 10, 2023

oubidar-Abderrahim commented May 16, 2023

sbrannen commented May 16, 2023

snicoll commented May 21, 2023

linghengqian commented Jun 7, 2023

linghengqian commented Apr 3, 2023 •

edited by snicoll

Loading