Kafka Snappy library fails to load when restarting Quarkus in the same JVM

[Empty-Glasss]

Describe the bug

I have a project where the tests run with multiple test profiles, which causes a new Quarkus instance for each group of tests (but I believe it's still the same JVM instance). However, the Snappy library that is being loaded cannot be loaded the second time since there is still a lock on the snappyjava.dll file (I think the OS keeps a lock since it's loaded in as a DLL)

Expected behavior

Snappy loads successfully each Quarkus startup

Actual behavior

The first time Quarkus starts up successfully. The second time Quarkus starts up, I get the following exception:

java.lang.RuntimeException: java.lang.RuntimeException: Failed to start quarkus
	at io.quarkus.test.junit.QuarkusTestExtension.throwBootFailureException(QuarkusTestExtension.java:638)
	at io.quarkus.test.junit.QuarkusTestExtension.interceptTestClassConstructor(QuarkusTestExtension.java:722)
	at java.base/java.util.Optional.orElseGet(Optional.java:364)
	at java.base/java.util.Optional.orElseGet(Optional.java:364)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1006)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: java.lang.RuntimeException: Failed to start quarkus
	at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
	at io.quarkus.runtime.Application.start(Application.java:101)
	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
	at io.quarkus.runner.bootstrap.StartupActionImpl.run(StartupActionImpl.java:285)
	at io.quarkus.test.junit.QuarkusTestExtension.doJavaStart(QuarkusTestExtension.java:252)
	at io.quarkus.test.junit.QuarkusTestExtension.ensureStarted(QuarkusTestExtension.java:605)
	at io.quarkus.test.junit.QuarkusTestExtension.beforeAll(QuarkusTestExtension.java:655)
	... 1 more
Caused by: java.io.UncheckedIOException: Unable to extract native library snappyjava.dll to C:\Users\<user>\AppData\Local\Temp\snappyjava.dll
	at io.quarkus.kafka.client.runtime.SnappyRecorder.extractLibraryFile(SnappyRecorder.java:54)
	at io.quarkus.kafka.client.runtime.SnappyRecorder.loadSnappy(SnappyRecorder.java:30)
	at io.quarkus.deployment.steps.KafkaProcessor$loadSnappyIfEnabled852439731.deploy_0(Unknown Source)
	at io.quarkus.deployment.steps.KafkaProcessor$loadSnappyIfEnabled852439731.deploy(Unknown Source)
	... 8 more
Caused by: java.io.FileNotFoundException: C:\Users\<user>\AppData\Local\Temp\snappyjava.dll (The process cannot access the file because it is being used by another process)
	at java.base/java.io.FileOutputStream.open0(Native Method)
	at java.base/java.io.FileOutputStream.open(FileOutputStream.java:295)
	at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:236)
	at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:185)
	at io.quarkus.kafka.client.runtime.SnappyRecorder.extractLibraryFile(SnappyRecorder.java:46)
	... 11 more

How to Reproduce?

1. Create a Quarkus 3.8.2 Maven project with a dependency to quarkus-kafka-client and quarkus.kafka.snappy.enabled=true
2. Create 2 tests, each with @QuarkusTest and a different @TestProfile
3. Run the tests, for example with mvn clean install

Output of uname -a or ver

Microsoft Windows [Version 10.0.22631.3296]

Output of java -version

openjdk 20.0.1 2023-04-18 OpenJDK Runtime Environment (build 20.0.1+9-29) OpenJDK 64-Bit Server VM (build 20.0.1+9-29, mixed mode, sharing)

Quarkus version or git rev

3.8.2

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.9.4 (dfbb324ad4a7c8fb0bf182e6d91b0ae20e3d2dd9)

Additional information

No response

[quarkus-bot]

/cc @alesj (kafka), @cescoffier (kafka), @ozangunalp (kafka)

[cescoffier]

Java has the restriction that a native library can only be loaded by a single classloader. Unfortunately, we cannot change that behavior. When using a test profile, you end up with 2 classloaders, which leads to the issue.

I've added a way to handle this situation in #40304. You would need to add the following property to your application.properties file:

quarkus.kafka.snappy.load-from-shared-classloader=true

[lorenzobenvenuti]

Java has the restriction that a native library can only be loaded by a single classloader. Unfortunately, we cannot change that behavior. When using a test profile, you end up with 2 classloaders, which leads to the issue.

I've added a way to handle this situation in #40304. You would need to add the following property to your application.properties file:

quarkus.kafka.snappy.load-from-shared-classloader=true

Hi, will this be backported to 3.8 (we want to stick with LTS versions)?

Thanks,

lorenzo

[cescoffier]

I would consider that as a feature so normally we do not backport them. But, it can also be seen as a bug fix.

It should be backportable - and let's see if it can be done.

Note that 3.15 is the next LTS, and will be released soonish (CR1 is already out)

[quarkus-bot]

@cescoffier triage/backport* labels may not be added to an issue. Please add them to the corresponding pull request.

_{This message is automatically generated by a bot.}