Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adopt --strict-image-heap that will land in GraalVM for JDK 21 #35902

Merged
merged 1 commit into from
Sep 15, 2023

Conversation

zakkak
Copy link
Contributor

@zakkak zakkak commented Sep 13, 2023

oracle/graal#7393 adds a new --strict-image-heap option, that will eventually become the new default. Quarkus should start using the option with GraalVM for JDK 21 (23.1.0) to be able to test it before it becomes the new default.

Closes #35901

Draft till CI run https://github.com/graalvm/mandrel/actions/runs/6168707496 completes

@zakkak
Copy link
Contributor Author

zakkak commented Sep 14, 2023

The CI gives:

Error: Determining image-builder observable modules failed (Exit status 1). Process output: 

in some cases. Needs further investigation.

Update: Can't reproduce locally, I will reschedule a CI run https://github.com/graalvm/mandrel/actions/runs/6185102310.

Update2: the issue persists on the CI

@jerboaa
Copy link
Contributor

jerboaa commented Sep 14, 2023

@zakkak I suggest to debug this in CI with a patch like this (in order to see the -H:+ListModules command being used):
jerboaa/graal@0574fd7

@zakkak
Copy link
Contributor Author

zakkak commented Sep 14, 2023

CI run using mandrel built based on https://github.com/zakkak/mandrel/tree/2023-09-14-mandrel23.1-debug which includes the suggested debug patch: https://github.com/graalvm/mandrel/actions/runs/6189859027

Update 1

The output states that Module org.graalvm.nativeimage.librarysupport reads more than one module named org.graalvm.nativeimage, the full output is:

Error: Determining image-builder observable modules failed (Exit status 1). Process output: 

Process error: 
Fatal error: java.lang.module.ResolutionException: Module org.graalvm.nativeimage.librarysupport reads more than one module named org.graalvm.nativeimage
	at java.base/java.lang.module.Resolver.resolveFail(Resolver.java:900)
	at java.base/java.lang.module.Resolver.checkExportSuppliers(Resolver.java:721)
	at java.base/java.lang.module.Resolver.finish(Resolver.java:380)
	at java.base/java.lang.module.Configuration.<init>(Configuration.java:139)
	at java.base/java.lang.module.Configuration.resolve(Configuration.java:422)
	at java.base/java.lang.module.Configuration.resolve(Configuration.java:254)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageClassLoaderSupport.<init>(NativeImageClassLoaderSupport.java:173)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.installNativeImageClassLoader(NativeImageGeneratorRunner.java:274)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.start(NativeImageGeneratorRunner.java:132)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:97)
Command was: /home/runner/work/mandrel/mandrel/graalvm-home/bin/java -XX:+UseParallelGC -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI -Dtruffle.TrustAllTruffleRuntimeProviders=true -Dtruffle.TruffleRuntime=com.oracle.truffle.api.impl.DefaultTruffleRuntime -Dgraalvm.ForcePolyglotInvalid=true -Dgraalvm.locatorDisabled=true -Dsubstratevm.HostLibC=glibc -Dsubstratevm.IgnoreGraalVersionCheck=true --add-exports=java.base/com.sun.crypto.provider=org.graalvm.nativeimage.builder --add-exports=java.base/jdk.internal.access=org.graalvm.nativeimage.builder --add-exports=java.base/jdk.internal.event=org.graalvm.nativeimage.builder --add-exports=java.base/jdk.internal.loader=org.graalvm.nativeimage.builder --add-exports=java.base/jdk.internal.logger=org.graalvm.nativeimage.builder --add-exports=java.base/jdk.internal.misc=org.graalvm.nativeimage.builder,org.graalvm.nativeimage.objectfile,org.graalvm.nativeimage.pointsto --add-exports=java.base/jdk.internal.module=org.graalvm.nativeimage.base,org.graalvm.nativeimage.builder --add-exports=java.base/jdk.internal.org.objectweb.asm=org.graalvm.nativeimage.builder --add-exports=java.base/jdk.internal.perf=org.graalvm.nativeimage.builder --add-exports=java.base/jdk.internal.platform=org.graalvm.nativeimage.builder --add-exports=java.base/jdk.internal.ref=org.graalvm.nativeimage.builder,org.graalvm.nativeimage.objectfile --add-exports=java.base/jdk.internal.reflect=org.graalvm.nativeimage.builder --add-exports=java.base/jdk.internal.util=org.graalvm.nativeimage.builder --add-exports=java.base/jdk.internal.vm.annotation=org.graalvm.nativeimage.builder --add-exports=java.base/jdk.internal.vm=org.graalvm.nativeimage.builder --add-exports=java.base/jdk.internal=org.graalvm.nativeimage.builder --add-exports=java.base/sun.invoke.util=org.graalvm.nativeimage.builder --add-exports=java.base/sun.net.www=org.graalvm.nativeimage.builder --add-exports=java.base/sun.net=org.graalvm.nativeimage.builder --add-exports=java.base/sun.nio.ch=org.graalvm.nativeimage.builder,org.graalvm.nativeimage.objectfile --add-exports=java.base/sun.reflect.annotation=org.graalvm.nativeimage.builder --add-exports=java.base/sun.reflect.generics.factory=org.graalvm.nativeimage.builder --add-exports=java.base/sun.reflect.generics.reflectiveObjects=org.graalvm.nativeimage.builder --add-exports=java.base/sun.reflect.generics.repository=org.graalvm.nativeimage.builder --add-exports=java.base/sun.reflect.generics.scope=org.graalvm.nativeimage.builder --add-exports=java.base/sun.reflect.generics.tree=org.graalvm.nativeimage.builder --add-exports=java.base/sun.security.jca=org.graalvm.nativeimage.builder --add-exports=java.base/sun.security.provider=org.graalvm.nativeimage.builder --add-exports=java.base/sun.security.ssl=org.graalvm.nativeimage.builder --add-exports=java.base/sun.security.util=org.graalvm.nativeimage.builder --add-exports=java.base/sun.security.x509=org.graalvm.nativeimage.builder --add-exports=java.base/sun.text.spi=org.graalvm.nativeimage.builder --add-exports=java.base/sun.util.calendar=org.graalvm.nativeimage.builder --add-exports=java.base/sun.util.cldr=org.graalvm.nativeimage.builder --add-exports=java.base/sun.util.locale.provider=org.graalvm.nativeimage.builder --add-exports=java.base/sun.util.locale=org.graalvm.nativeimage.builder --add-exports=java.base/sun.util.resources=org.graalvm.nativeimage.builder --add-exports=java.base/sun.util=org.graalvm.nativeimage.builder --add-exports=java.management/com.sun.jmx.mbeanserver=org.graalvm.nativeimage.builder --add-exports=java.management/sun.management=org.graalvm.nativeimage.builder,org.graalvm.nativeimage.pointsto --add-exports=jdk.internal.vm.ci/jdk.vm.ci.aarch64=jdk.internal.vm.compiler,org.graalvm.nativeimage.builder,org.graalvm.nativeimage.objectfile --add-exports=jdk.internal.vm.ci/jdk.vm.ci.amd64=jdk.internal.vm.compiler,org.graalvm.nativeimage.builder,org.graalvm.nativeimage.objectfile --add-exports=jdk.internal.vm.ci/jdk.vm.ci.code.site=jdk.internal.vm.compiler,org.graalvm.nativeimage.builder --add-exports=jdk.internal.vm.ci/jdk.vm.ci.code.stack=jdk.internal.vm.compiler,org.graalvm.nativeimage.builder --add-exports=jdk.internal.vm.ci/jdk.vm.ci.code=jdk.internal.vm.compiler,org.graalvm.nativeimage.builder,org.graalvm.nativeimage.objectfile,org.graalvm.nativeimage.pointsto,org.graalvm.truffle.compiler --add-exports=jdk.internal.vm.ci/jdk.vm.ci.common=jdk.internal.vm.compiler,org.graalvm.nativeimage.builder,org.graalvm.nativeimage.pointsto --add-exports=jdk.internal.vm.ci/jdk.vm.ci.hotspot.aarch64=jdk.internal.vm.compiler --add-exports=jdk.internal.vm.ci/jdk.vm.ci.hotspot.amd64=jdk.internal.vm.compiler --add-exports=jdk.internal.vm.ci/jdk.vm.ci.hotspot=jdk.internal.vm.compiler,org.graalvm.nativeimage.builder --add-exports=jdk.internal.vm.ci/jdk.vm.ci.meta=jdk.internal.vm.compiler,org.graalvm.nativeimage.base,org.graalvm.nativeimage.builder,org.graalvm.nativeimage.objectfile,org.graalvm.nativeimage.pointsto,org.graalvm.truffle.compiler --add-exports=jdk.internal.vm.ci/jdk.vm.ci.runtime=jdk.internal.vm.compiler,org.graalvm.nativeimage.builder,org.graalvm.nativeimage.pointsto --add-exports=jdk.internal.vm.ci/jdk.vm.ci.services=jdk.internal.vm.compiler,org.graalvm.nativeimage.builder --add-exports=jdk.jfr/jdk.jfr.events=org.graalvm.nativeimage.builder --add-exports=jdk.jfr/jdk.jfr.internal.jfc=org.graalvm.nativeimage.builder --add-exports=jdk.jfr/jdk.jfr.internal=org.graalvm.nativeimage.builder --add-exports=jdk.management/com.sun.management.internal=org.graalvm.nativeimage.builder -XX:-UseJVMCICompiler -Xss10m -XX:MaxRAMPercentage=85.0 -XX:GCTimeRatio=9 -XX:+ExitOnOutOfMemoryError -Djava.awt.headless=true -Dorg.graalvm.vendor=GraalVM Community -Dorg.graalvm.vendorurl=https://github.com/graalvm/mandrel/issues -Dorg.graalvm.vendorversion=Mandrel-23.1.0.0-devde078ae3 -Dorg.graalvm.version=23.1.0.0-devde078ae3 -Dcom.oracle.graalvm.isaot=true -Djava.system.class.loader=com.oracle.svm.hosted.NativeImageSystemClassLoader -Xshare:off --upgrade-module-path /home/runner/work/mandrel/mandrel/graalvm-home/lib/jvmci/graal.jar -Djdk.internal.lambda.disableEagerInitialization=true -Djdk.internal.lambda.eagerlyInitialize=false -Djava.lang.invoke.InnerClassLambdaMetafactory.initializeLambdas=false --add-exports=jdk.internal.vm.ci/jdk.vm.ci.code=jdk.internal.vm.compiler -Dsun.nio.ch.maxUpdateArraySize=100 -Dcom.sun.xml.bind.v2.bytecode.ClassTailor.noOptimize=true -Dlogging.initial-configurator.min-level=500 -DCoordinatorEnvironmentBean.transactionStatusManagerEnable=false -Djava.util.logging.manager=org.jboss.logmanager.LogManager -Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -Dvertx.disableDnsResolver=true -Dio.netty.leakDetection.level=DISABLED -Dio.netty.allocator.maxOrder=3 -Duser.language=en -Duser.country=US -Dfile.encoding=UTF-8 --add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED -Djava.awt.headless=true -Xmx5g --add-exports=org.graalvm.nativeimage/org.graalvm.nativeimage.impl=ALL-UNNAMED --add-modules=ALL-DEFAULT --module-path /home/runner/work/mandrel/mandrel/graalvm-home/lib/jvmci/nativeimage.jar:/home/runner/work/mandrel/mandrel/graalvm-home/lib/jvmci/collections.jar:/home/runner/work/mandrel/mandrel/graalvm-home/lib/jvmci/word.jar:/home/runner/work/mandrel/mandrel/graalvm-home/lib/truffle/truffle-compiler.jar:/home/runner/work/mandrel/mandrel/graalvm-home/lib/svm/builder/objectfile.jar:/home/runner/work/mandrel/mandrel/graalvm-home/lib/svm/builder/svm.jar:/home/runner/work/mandrel/mandrel/graalvm-home/lib/svm/builder/native-image-base.jar:/home/runner/work/mandrel/mandrel/graalvm-home/lib/svm/builder/pointsto.jar --module org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner -imagemp /home/runner/work/mandrel/mandrel/quarkus/integration-tests/hibernate-orm-panache-kotlin/target/quarkus-integration-test-hibernate-orm-panache-kotlin-999-SNAPSHOT-native-image-source-jar/lib/org.graalvm.sdk.collections-23.1.0.0.SNAPSHOT.jar:/home/runner/work/mandrel/mandrel/quarkus/integration-tests/hibernate-orm-panache-kotlin/target/quarkus-integration-test-hibernate-orm-panache-kotlin-999-SNAPSHOT-native-image-source-jar/lib/org.graalvm.sdk.nativeimage-23.1.0.0.SNAPSHOT.jar:/home/runner/work/mandrel/mandrel/quarkus/integration-tests/hibernate-orm-panache-kotlin/target/quarkus-integration-test-hibernate-orm-panache-kotlin-999-SNAPSHOT-native-image-source-jar/lib/org.graalvm.sdk.word-23.1.0.0.SNAPSHOT.jar:/home/runner/work/mandrel/mandrel/graalvm-home/lib/svm/library-support.jar -H:+ListModules

Update 2

The issue is related to using local GraalVM artifacts. I am able to reproduce locally with:

./mvnw -pl integration-tests/hibernate-orm-panache-kotlin -Dnative -Dnative.surefire.skip -Dformat.skip -Dno-descriptor-tests install -DskipDocs -Dquarkus.native.container-build=false -Dgraal-sdk.version=23.1.0.0.SNAPSHOT

Specifically, the issue is caused by the following option:

-imagemp /home/zakkak/code/quarkus/integration-tests/hibernate-orm-panache-kotlin/target/quarkus-integration-test-hibernate-orm-panache-kotlin-999-SNAPSHOT-native-image-source-jar/lib/org.graalvm.sdk.collections-23.1.0.0.SNAPSHOT.jar:/home/zakkak/code/quarkus/integration-tests/hibernate-orm-panache-kotlin/target/quarkus-integration-test-hibernate-orm-panache-kotlin-999-SNAPSHOT-native-image-source-jar/lib/org.graalvm.sdk.nativeimage-23.1.0.0.SNAPSHOT.jar:/home/zakkak/code/quarkus/integration-tests/hibernate-orm-panache-kotlin/target/quarkus-integration-test-hibernate-orm-panache-kotlin-999-SNAPSHOT-native-image-source-jar/lib/org.graalvm.sdk.word-23.1.0.0.SNAPSHOT.jar

Update 3

The issue is actually fixed in main by #35873.
I will rebase...

oracle/graal#7393 adds a new --strict-image-heap option, that will
eventually become the new default. Quarkus should start using the option
with GraalVM for JDK 21 (23.1.0) to be able to test it before it becomes
the new default.

Closes quarkusio#35901
@zakkak zakkak force-pushed the 2023-09-13-fix-35901 branch from 90d57b2 to 59939ce Compare September 15, 2023 06:40
@zakkak zakkak marked this pull request as ready for review September 15, 2023 06:40
@zakkak zakkak added the triage/waiting-for-ci Ready to merge when CI successfully finishes label Sep 15, 2023
@zakkak zakkak added this to the 3.5 - main milestone Sep 15, 2023
@zakkak
Copy link
Contributor Author

zakkak commented Sep 15, 2023

New workflow testing with Mandrel 23.1 https://github.com/graalvm/mandrel/actions/runs/6194909863

CI is now happy.

@quarkus-bot
Copy link

quarkus-bot bot commented Sep 15, 2023

Failing Jobs - Building 59939ce

Status Name Step Failures Logs Raw logs Build scan
✔️ JVM Tests - JDK 11
✔️ JVM Tests - JDK 17
JVM Tests - JDK 17 Windows Build Failures Logs Raw logs
JVM Tests - JDK 20 Build Failures Logs Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ JVM Tests - JDK 17 Windows #

- Failing: extensions/vertx/deployment 
! Skipped: devtools/cli extensions/agroal/deployment extensions/amazon-lambda-http/deployment and 379 more

📦 extensions/vertx/deployment

io.quarkus.vertx.mdc.VertxMDCTest.mdc line 144 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: expected: <10> but was: <0>
	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)

io.quarkus.vertx.mdc.VertxMDCTest.mdcNonVertxThreadTest line 121 - More details - Source on GitHub

java.lang.AssertionError: 

Expected: a collection containing "Test MDC value ### Test 1"

⚙️ JVM Tests - JDK 20 #

- Failing: extensions/hibernate-orm/deployment 
! Skipped: extensions/flyway/deployment extensions/hibernate-envers/deployment extensions/hibernate-reactive/deployment and 93 more

📦 extensions/hibernate-orm/deployment

io.quarkus.hibernate.orm.interceptor.ApplicationScopedInterceptorTest. - More details - Source on GitHub

java.lang.RuntimeException: java.lang.RuntimeException: Failed to start quarkus
	at io.quarkus.test.QuarkusUnitTest.beforeAll(QuarkusUnitTest.java:705)
	at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeBeforeAllCallbacks$12(ClassBasedTestDescriptor.java:395)

@zakkak
Copy link
Contributor Author

zakkak commented Sep 15, 2023

Quarkus CI failures are irrelevant.

@zakkak zakkak requested a review from geoand September 15, 2023 13:55
@zakkak zakkak added area/native-image and removed triage/waiting-for-ci Ready to merge when CI successfully finishes labels Sep 15, 2023
@zakkak zakkak merged commit d5dfbcf into quarkusio:main Sep 15, 2023
@zakkak zakkak deleted the 2023-09-13-fix-35901 branch September 15, 2023 15:19
@quarkus-bot quarkus-bot bot added the kind/enhancement New feature or request label Sep 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Adopt new option --strict-image-heap that will land in GraalVM for JDK 21
3 participants