Support for project modules that produce multiple JARs (with classifiers)

[aloubyansky]

This is #22336 plus the fix for ApplicationArchiveImpl exposing paths from open JARs that are now switched to PathTree-based API as well.

[quarkus-bot]

/cc @Karm, @galderz, @zakkak

[quarkus-bot]

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building 8578ea9

Status	Name	Step	Failures	Logs	Raw logs
✔️	JVM Tests - JDK 11
✖	JVM Tests - JDK 11 Windows	Build	Failures	Logs	Raw logs
✔️	JVM Tests - JDK 17
✖	Native Tests - Misc4	Build	Failures	Logs	Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ JVM Tests - JDK 11 Windows #

- Failing: extensions/qute/deployment 
! Skipped: extensions/mailer/deployment extensions/resteasy-classic/resteasy-qute/deployment extensions/resteasy-reactive/quarkus-resteasy-reactive-qute/deployment and 3 more

📦 extensions/qute/deployment

✖ io.quarkus.qute.deployment.scanning.WrongTemplatesDirectoryTest. - More details - Source on GitHub

org.opentest4j.AssertionFailedError: The build was expected to fail
	at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:39)
	at org.junit.jupiter.api.Assertions.fail(Assertions.java:134)

---

⚙️ Native Tests - Misc4 #

- Failing: integration-tests/webjars-locator 

📦 integration-tests/webjars-locator

✖ io.quarkus.it.webjar.locator.WebJarResourceIT.testWebJar - More details - Source on GitHub

java.lang.AssertionError: 
1 expectation failed.
Expected status code <200> but was <404>.

[famod]

So the main thing to look at is ApplicationArchiveImpl and the rest is more or less the same as in the first PR?

[aloubyansky]

The Qute failure is clear how to fix but the webjars issue isn't. I am still investigating it.

[gsmet]

@aloubyansky I don't think the webjar locator issue is related to your PR. Rostislav added the native testing recently and it has been unstable since then. We will remove the native testing for now.

[aloubyansky]

The failure isn't even related to native apparently. We are using org.webjars.WebJarsAssetLocator to locate the resources. While the resources are clearly visible on the classpath, it fails to locate them with my PR applied. So even quarkus:build won't see them. I really want to know what the cause is.

[aloubyansky]

Ok, that's a pretty strong expectation from an internal API https://github.com/classgraph/classgraph/blob/latest/src/main/java/nonapi/io/github/classgraph/classloaderhandler/QuarkusClassLoaderHandler.java#L119-L132

[aloubyansky]

I am not sure we want to rely on that logic. Basically, the resources the webjars scanner is looking for are on the classpath but it is trying to use all sorts of tricks to not load them through the ClassLoader API and this PR breaks it. I'll probably try to find a workaround.

[aloubyansky]

I ended up removing the dependency on the webjars-locator-core replacing it with our own discovery of the webjars on the classpath.

[aloubyansky]

The LiquibaseFunctionalityNativeIT is reproducible locally. My change passed the CI in my working branch before I updated the PR though. I am wondering whether I rebased before updating and pulled in some other change.

[aloubyansky]

It's also reproducible on the main branch

[ERROR] Failures: 
[ERROR]   LiquibaseFunctionalityNativeIT>LiquibaseFunctionalityTest.testLiquibaseQuarkusFunctionality:21 1 expectation failed.
Response body doesn't match expectation.
Expected: is "create-tables-1,test-1,create-view-inline,create-view-file-abs,create-view-file-rel,json-create-tables-1,json-test-1,sql-create-tables-1,sql-test-1,yaml-create-tables-1,yaml-test-1,00000000000000,00000000000001,00000000000002,1613578374533-1,1613578374533-2,1613578374533-3"
  Actual: 

[INFO] 
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

[quarkus-bot]

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building e1a00a8

Status	Name	Step	Failures	Logs	Raw logs
✖	JVM Tests - JDK 11	Build	Failures	Logs	Raw logs
✔️	JVM Tests - JDK 17
✖	Native Tests - Data3	Build	Failures	Logs	Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ JVM Tests - JDK 11 #

- Failing: extensions/vertx-http/deployment 
! Skipped: core/test-extension/deployment extensions/agroal/deployment extensions/amazon-lambda-http/deployment and 290 more

📦 extensions/vertx-http/deployment

✖ io.quarkus.vertx.http.testrunner.includes.ExcludePatternTestCase.checkTestsAreRun line 41 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: Failed to wait for test run 1 State{lastRun=-1, running=true, inProgress=true, run=0, passed=0, failed=0, skipped=0, isBrokenOnly=false, isTestOutput=false, isInstrumentationBasedReload=false, isLiveReload=true}
	at io.quarkus.test.ContinuousTestingTestUtils.waitForNextCompletion(ContinuousTestingTestUtils.java:44)
	at io.quarkus.vertx.http.testrunner.includes.ExcludePatternTestCase.checkTestsAreRun(ExcludePatternTestCase.java:41)

---

⚙️ Native Tests - Data3 #

- Failing: integration-tests/liquibase 

📦 integration-tests/liquibase

✖ io.quarkus.it.liquibase.LiquibaseFunctionalityNativeIT.testLiquibaseQuarkusFunctionality - More details - Source on GitHub

java.lang.AssertionError: 
1 expectation failed.
Response body doesn't match expectation.

[aloubyansky]

Rebased just to get another CI run.

[aloubyansky]

I added a couple more fixes:

1. occasional failure originally reported in Introduce support for Uni return types in Micrometer annotations #22639 (comment). The issue is that I changed the bootstrap Maven resolver prefer locally built JARs if they exist over the classes dirs. It looks like this could be an issue in parallel builds when a JAR is created so Files.exists(jar) return true but it hasn't been written completely resulting in "end of ZIP missing" error trying to read it. Given that we know the workspace in extension-descriptor, I am checking whether it's a parallel build and if it is I am switching to the classes dir to read extension descriptors.
2. There is a trick in the current impl to allow re-reading JAR content if the thread was interrupted. I originally copied its impl but it's JarFile-based while this PR NIO-based. The thrown exceptions changed from InterruptedIOException to ClosedByInterruptionException and ClosedChannelException. I adjusted the code to handle those and I don't see misleading stacktraces logged for info purposes anymore.

I've been running CI multiple times with these changes in my working branches w/o getting any failure. So, this should be good to merge.

[famod]

@aloubyansky Sounds good! Shall I give it another try with my project?

[aloubyansky]

I won't say no to that :)

[famod]

Looking good!

[gsmet]

So this is a large patch but it fixes quite a lot of complex issues and after discussion with @geoand @aloubyansky @famod , we want to get it in for 2.7.

It has already been through a first round of reviews in the first PR and got through some important testing.