Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quarkus watches wrong protobuf directory #32092

Closed
micheljung opened this issue Mar 23, 2023 · 5 comments · Fixed by #36799
Closed

Quarkus watches wrong protobuf directory #32092

micheljung opened this issue Mar 23, 2023 · 5 comments · Fixed by #36799
Assignees
Labels
area/gradle Gradle area/grpc gRPC env/windows Impacts Windows machines kind/bug Something isn't working
Milestone

Comments

@micheljung
Copy link

Describe the bug

Using io.quarkus:quarkus-grpc.

When DEBUG log is enabled, this gets printed to the console a zillion times:

2023-03-23 22:32:00,132 DEBUG [io.qua.dep.uti.FSWatchUtil] (FSWatchUtil) Failed checking for code gen source modifications: java.nio.file.NoSuchFileException: C:\Users\foo\bar\build\classes\java\proto
	at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
	at java.base/sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
	at java.base/sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
	at java.base/sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:198)
	at java.base/java.nio.file.Files.readAttributes(Files.java:1764)
	at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
	at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
	at java.base/java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
	at java.base/java.nio.file.FileTreeIterator.<init>(FileTreeIterator.java:71)
	at java.base/java.nio.file.Files.walk(Files.java:3825)
	at java.base/java.nio.file.Files.walk(Files.java:3879)
	at io.quarkus.deployment.util.FSWatchUtil.doObserve(FSWatchUtil.java:55)
	at io.quarkus.deployment.util.FSWatchUtil.lambda$observe$1(FSWatchUtil.java:41)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

I'm not sure what directory it is looking for, but the ones that exist are:

  • src/main/proto
  • build/classes/java/quarkus-generated-sources/grpc

Expected behavior

No Exception. I understand it's just DEBUG but it seems like it's looking for the wrong directory. If not, I wonder why the directory doesn't exist.

Actual behavior

Exceptions are spamming the DEBUG log.

How to Reproduce?

I can create one if needed

Output of uname -a or ver

No response

Output of java -version

No response

GraalVM version (if different from Java)

No response

Quarkus version or git rev

3.0.0.Beta1

Build tool (ie. output of mvnw --version or gradlew --version)

Gradle 7.3.3

Additional information

No response

@micheljung micheljung added the kind/bug Something isn't working label Mar 23, 2023
@quarkus-bot quarkus-bot bot added env/windows Impacts Windows machines triage/needs-triage labels Mar 23, 2023
@quarkus-bot
Copy link

quarkus-bot bot commented Mar 24, 2023

/cc @alesj (grpc), @cescoffier (grpc)

@BrianSetz
Copy link

BrianSetz commented Jul 21, 2023

This issues appears to only happen when you use Gradle, even when you create a clean project (https://code.quarkus.io/?b=GRADLE&e=grpc&extension-search=origin:platform%20quarkus-grpc). You only have to set Quarkus log levels to debug for the issue to appear.

It is not only an issue on Windows, but also on Linux.

2023-07-21 14:23:29,685 DEBUG [io.qua.dep.uti.FSWatchUtil] (FSWatchUtil) Failed checking for code gen source modifications: java.nio.file.NoSuchFileException: /home/user/foo/bar/build/classes/java/quarkus-generated-sources/proto
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
	at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:148)
	at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
	at java.base/java.nio.file.Files.readAttributes(Files.java:1851)
	at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:220)
	at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:277)
	at java.base/java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:323)
	at java.base/java.nio.file.FileTreeIterator.<init>(FileTreeIterator.java:71)
	at java.base/java.nio.file.Files.walk(Files.java:3918)
	at java.base/java.nio.file.Files.walk(Files.java:3973)
	at io.quarkus.deployment.util.FSWatchUtil.doObserve(FSWatchUtil.java:55)
	at io.quarkus.deployment.util.FSWatchUtil.lambda$observe$1(FSWatchUtil.java:41)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

It seems to me that the codeGen.sourceDir is set incorrectly ( https://github.com/quarkusio/quarkus/blob/main/core/deployment/src/main/java/io/quarkus/deployment/dev/CodeGenWatcher.java#L42 ). But I am unsure where this sourceDir originates from.

@cescoffier
Copy link
Member

oh lovely, Thanks @BrianSetz.

@alesj .... I'm sure you are going to become a Gradle expert one day.

@cescoffier
Copy link
Member

It would be great if we could get a reproducer.

@cescoffier cescoffier added the triage/needs-reproducer We are waiting for a reproducer. label Jul 21, 2023
@BrianSetz
Copy link

In case it is useful, I have pushed an example of this issue to this repository: https://github.com/BrianSetz/quarkus-grpc-nosuchfileexception

You can also manually reproduce it using the following steps:

quarkus.log.level=DEBUG

You can also verify that it does not happen when using maven, using the following steps:

quarkus.log.level=DEBUG

@cescoffier cescoffier added the area/gradle Gradle label Aug 1, 2023
@geoand geoand removed the triage/needs-reproducer We are waiting for a reproducer. label Aug 30, 2023
@cescoffier cescoffier self-assigned this Oct 23, 2023
cescoffier added a commit to cescoffier/quarkus that referenced this issue Oct 31, 2023
Fix quarkusio#32092.
The issue was not specific to Gradle.
It was problematic with Gradle, as it was configuring the watcher on not existing directories.
@cescoffier cescoffier removed their assignment Oct 31, 2023
cescoffier added a commit to cescoffier/quarkus that referenced this issue Nov 2, 2023
Fix quarkusio#32092.
The issue was not specific to Gradle.
It was problematic with Gradle, as it was configuring the watcher on not existing directories.
@quarkus-bot quarkus-bot bot added this to the 3.6 - main milestone Nov 3, 2023
chetankokil added a commit to chetankokil/quarkus that referenced this issue Nov 3, 2023
* Use MongoDB 4.4 consistently

* Fix handling of HTTP/2 H2 empty frames in Resteasy Reactive

Closes quarkusio#36604

* Bump com.google.api.grpc:proto-google-common-protos

Bumps [com.google.api.grpc:proto-google-common-protos](https://github.com/googleapis/sdk-platform-java) from 2.23.0 to 2.27.0.
- [Release notes](https://github.com/googleapis/sdk-platform-java/releases)
- [Changelog](https://github.com/googleapis/sdk-platform-java/blob/main/CHANGELOG.md)
- [Commits](googleapis/sdk-platform-java@v2.23.0...v2.27.0)

---
updated-dependencies:
- dependency-name: com.google.api.grpc:proto-google-common-protos
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Add a custom OIDC identity provider test

* Updates Liquibase integrationtest to reproduce error: quarkusio#36446

* Fix zlib broken link

* Reverts changes made to workaround liquibase issue: liquibase/liquibase#4763 since we've updated to liquibase 4.24.0

* Fix version reporting and parsing

Quarkus currently only parses the feature and update version of JDK and
incorrectly prints message like:

```
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on MANDREL 22.3.3.1 JDK 17.8
```

while it should be

```
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on MANDREL 22.3.3.1 JDK 17.0.8
```

i.e. 17.0.8 vs 17.8

This change fixes that issue and relies on java.lang.Runtime.Version for
parsing and comparing the JDK version.

Follow up to quarkusio#36267

* Make sure OIDC/GraphQL client integration runs after client config is initialized, get rid of config merger bean

* Fix termination of the fallback virtual thread executor

* Stork observability integration

* Move HTTP CORS section to a new security-cors.adoc

* Add security-cors.adoc to the downstreamdoc.yaml

* QuarkusTest: handle beans declared on test profile specifically

- beans declared on a test profile implementation are only taken into
account if the test profile is used
- resolves quarkusio#36554

* Fix gRPC context propagation.

* Make the ZSTD Substitutions more robust

* Remove the pinned event example from the documentation

* Use container urls for connecting to services by default

It is better for the build cache and for local testing, nowadays, there
is a good chance people will use containers anyway.

* Update gRPC Protoc to version 3.24.4 and the generator to 1.59.0

- Add move architecture to cover ppcle64 and s390x
- Update code to handle deprecation and removed API

* Use GRADLE_JAVA_HOME in CI and test Java 21

* Remove more Docker images

* Indicate that Java 21 is supported

* Remove additional stuff before starting CI

* Remove --enable-preview from virtual-threads ITs

We are now using Java 21 so we don't need the flag anymore.

* Add a ci-disk-usage.sh script

* Skip Scala 2 tests on Java 21+

* Make RegisterForReflectionTestCase#testLambdaCapturing compatible with Java 21

* Higher Xmx for Java 21 testing

* Deprecate the scala extension

- https://github.com/quarkiverse/quarkus-scala3 should be preferred instead

* Codestarts: Add Entity annotation to Kotlin all-open

* QuarkusSecurityTestExtension afterEach call should not be made for tests without @testsecurity

* Introduce OidcClientRequestFilter

* Bump org.jboss.threads:jboss-threads from 3.5.0.Final to 3.5.1.Final

Bumps [org.jboss.threads:jboss-threads](https://github.com/jbossas/jboss-threads) from 3.5.0.Final to 3.5.1.Final.
- [Commits](jbossas/jboss-threads@3.5.0.Final...3.5.1.Final)

---
updated-dependencies:
- dependency-name: org.jboss.threads:jboss-threads
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump jacoco.version from 0.8.10 to 0.8.11

Bumps `jacoco.version` from 0.8.10 to 0.8.11.

Updates `org.jacoco:org.jacoco.core` from 0.8.10 to 0.8.11
- [Release notes](https://github.com/jacoco/jacoco/releases)
- [Commits](jacoco/jacoco@v0.8.10...v0.8.11)

Updates `org.jacoco:org.jacoco.report` from 0.8.10 to 0.8.11
- [Release notes](https://github.com/jacoco/jacoco/releases)
- [Commits](jacoco/jacoco@v0.8.10...v0.8.11)

Updates `org.jacoco:org.jacoco.agent` from 0.8.10 to 0.8.11
- [Release notes](https://github.com/jacoco/jacoco/releases)
- [Commits](jacoco/jacoco@v0.8.10...v0.8.11)

Updates `org.jacoco:jacoco-maven-plugin` from 0.8.10 to 0.8.11
- [Release notes](https://github.com/jacoco/jacoco/releases)
- [Commits](jacoco/jacoco@v0.8.10...v0.8.11)

---
updated-dependencies:
- dependency-name: org.jacoco:org.jacoco.core
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.jacoco:org.jacoco.report
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.jacoco:org.jacoco.agent
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.jacoco:jacoco-maven-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Dev UI: Fix height in Rest Client

Signed-off-by: Phillip Kruger <[email protected]>

* Disable CDI TCK ManagedBeanTypesTest for now

This test is not compatible with Java 21.

See jakartaee/cdi-tck#485

* Reinitialized the com.google.protobuf.UnsafeUtil class at runtime

Also fix the Unsafe accessor.
Fix quarkusio#30293.

This has been tested on GraalVM CE and EE.

* Revert "Bump io.strimzi:kafka-oauth-client from 0.12.0 to 0.14.0"

This reverts commit 09a13e6.

* Add a warning for how to report security vulnerability

* add JBang script to analyze our CI raw logs

* Small additional features to ModuleBuildDurationReport.java

* Some additional improvements and fixes

* More readable module names

* Add auth mechanism to the Liquibase MongoDB connection string

* Fix missing import probably due to asynchronous PRs

* Disable VertxMDCTest on Windows

It has been extremely unstable for weeks now. Let's disable it until we
have time to fix it.

* Take @ConstrainedTo into account for interceptors

Without this change interceptors meant to be
used on the client were being used on the
server as well

* Rename tooling codestarts

* ArC: introduce optimized contexts

- introduce the ContextInstances abstraction
- if optimization is enabled then generate the ContextInstances
  implementation for application and request context

* ⬆️ Update Gitpod configuration with Java 17

* CORS content review

Signed-off-by: Michal Maléř <[email protected]>

* ArC: optimize client proxy delegate access

- for normal scopes for which a single context is registered
- similar to how we optimize the client proxy delegate access for
application context

* Fix update on extensions

* Remove a duplicate dependency from Micrometer extension

* Avoid relying on Docker for Stork tests

We don't need the Redis Dev Services so let's not start them.

* Update the content for RHBQ docs.

* Bump io.smallrye.reactive:smallrye-mutiny-vertx-core from 3.6.0 to 3.7.2

Bumps io.smallrye.reactive:smallrye-mutiny-vertx-core from 3.6.0 to 3.7.2.

---
updated-dependencies:
- dependency-name: io.smallrye.reactive:smallrye-mutiny-vertx-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Document gRPC code generation

Also:

- Allows configuring the `proto` directory (quarkusio#12802)
- Provides Gradle snippets (quarkusio#33854)

* Bump asm.version from 9.5 to 9.6

Bumps `asm.version` from 9.5 to 9.6.

Updates `org.ow2.asm:asm` from 9.5 to 9.6

Updates `org.ow2.asm:asm-commons` from 9.5 to 9.6

Updates `org.ow2.asm:asm-tree` from 9.5 to 9.6

Updates `org.ow2.asm:asm-analysis` from 9.5 to 9.6

Updates `org.ow2.asm:asm-util` from 9.5 to 9.6

---
updated-dependencies:
- dependency-name: org.ow2.asm:asm
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.ow2.asm:asm-commons
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.ow2.asm:asm-tree
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.ow2.asm:asm-analysis
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.ow2.asm:asm-util
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Vert.x: fix NPE in ForwardedProxyHandler

The test `TrustedXForwarderProxiesUnknownHostnameFailureTest` sometimes fail
in CI due to a NPE in `ForwardedProxyHandler`. This shows an actual bug:
per the documentation, `io.vertx.core.dns.DnsClient.lookup()` may succeed
with a `null` value when no record was found. The `ForwardedProxyHandler`
ignores the possibility of a `null` result, which this commit fixes. We deal
with a `null` result just like with a failure, because it's equivalent
to a NXDOMAIN error.

* Arc - change hashing function to use base64 to shorten resulting String

* Try to stabilize PausedSchedulerTest

Let's give things a bit more time as CI is slow.

* HTTP and RestEasy-related content review

resteasy.adoc sanity check

Signed-off-by: Michal Maléř <[email protected]>

* Indicate that 2.x is not supported anymore

* Bump Keycloak version to 22.0.5

* Disable MySQL OIDC DB token manager test due to limited Github CI resources

* Bump org.yaml:snakeyaml from 2.1 to 2.2

Bumps [org.yaml:snakeyaml](https://bitbucket.org/snakeyaml/snakeyaml) from 2.1 to 2.2.
- [Commits](https://bitbucket.org/snakeyaml/snakeyaml/branches/compare/snakeyaml-2.2..snakeyaml-2.1)

---
updated-dependencies:
- dependency-name: org.yaml:snakeyaml
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Make Maven ITs less fragile

Enforcing versions of the plugins make sure we don't start downloading
them when running the tests which can lead to timeouts.
Also it makes the build more stable and actually tests the version we
are using when creating new projects.

* Not pushing compile in the command actually fixes this test

When running mvn compile quarkus:dev, compile is not executed when
restarting dev mode after the pom change. Not specifying compile makes
the compile goal to be run.

* Try to stabilize the tests on Windows

* Disable JarRunnerIT#testNonAsciiDir on Windows

With maven-compiler-plugin 2.11.0, this test is not working anymore
on Windows:
Error while storing the mojo status: Input length = 1

* Enable errors in Maven RunningInvoker

We need to see the details of the errors when something is going wrong.

* Disable DevMojoIT#testExternalReloadableArtifacts on Windows

Installing the lib jar a second time leads to an access denied exception
on Windows, probably because the jar is still used by the dev mode
instance.

* refactor BuildChainBuilder

* Move KeycloakContainer to keycloak-server test-framework

Create realm by posting to Keycloak admin API

* Bump io.strimzi:kafka-oauth-client from 0.12.0 to 0.14.0

* Fixes for Kafka and Pulsar schema discovery

* Bump elasticsearch-opensource-components.version from 8.10.2 to 8.10.4

Bumps `elasticsearch-opensource-components.version` from 8.10.2 to 8.10.4.

Updates `org.elasticsearch.client:elasticsearch-rest-client` from 8.10.2 to 8.10.4
- [Release notes](https://github.com/elastic/elasticsearch/releases)
- [Changelog](https://github.com/elastic/elasticsearch/blob/main/CHANGELOG.md)
- [Commits](elastic/elasticsearch@v8.10.2...v8.10.4)

Updates `co.elastic.clients:elasticsearch-java` from 8.10.2 to 8.10.4
- [Release notes](https://github.com/elastic/elasticsearch-java/releases)
- [Changelog](https://github.com/elastic/elasticsearch-java/blob/main/CHANGELOG.md)
- [Commits](elastic/elasticsearch-java@v8.10.2...v8.10.4)

Updates `org.elasticsearch.client:elasticsearch-rest-client-sniffer` from 8.10.2 to 8.10.4
- [Release notes](https://github.com/elastic/elasticsearch/releases)
- [Changelog](https://github.com/elastic/elasticsearch/blob/main/CHANGELOG.md)
- [Commits](elastic/elasticsearch@v8.10.2...v8.10.4)

---
updated-dependencies:
- dependency-name: org.elasticsearch.client:elasticsearch-rest-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: co.elastic.clients:elasticsearch-java
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.elasticsearch.client:elasticsearch-rest-client-sniffer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump flyway.version from 9.22.2 to 9.22.3

Bumps `flyway.version` from 9.22.2 to 9.22.3.

Updates `org.flywaydb:flyway-core` from 9.22.2 to 9.22.3
- [Release notes](https://github.com/flyway/flyway/releases)
- [Commits](flyway/flyway@flyway-9.22.2...flyway-9.22.3)

Updates `org.flywaydb:flyway-sqlserver` from 9.22.2 to 9.22.3
- [Release notes](https://github.com/flyway/flyway/releases)
- [Commits](flyway/flyway@flyway-9.22.2...flyway-9.22.3)

Updates `org.flywaydb:flyway-mysql` from 9.22.2 to 9.22.3
- [Release notes](https://github.com/flyway/flyway/releases)
- [Commits](flyway/flyway@flyway-9.22.2...flyway-9.22.3)

Updates `org.flywaydb:flyway-database-oracle` from 9.22.2 to 9.22.3
- [Release notes](https://github.com/flyway/flyway/releases)
- [Commits](flyway/flyway@flyway-9.22.2...flyway-9.22.3)

---
updated-dependencies:
- dependency-name: org.flywaydb:flyway-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.flywaydb:flyway-sqlserver
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.flywaydb:flyway-mysql
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.flywaydb:flyway-database-oracle
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump io.micrometer:micrometer-bom from 1.11.1 to 1.11.5

Bumps [io.micrometer:micrometer-bom](https://github.com/micrometer-metrics/micrometer) from 1.11.1 to 1.11.5.
- [Release notes](https://github.com/micrometer-metrics/micrometer/releases)
- [Commits](micrometer-metrics/micrometer@v1.11.1...v1.11.5)

---
updated-dependencies:
- dependency-name: io.micrometer:micrometer-bom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Make it possible to acquire OIDC SecurityIdentity after HTTP request has completed

* Clarify why QuarkusClassLoader doesn't pass its name to the super constructor

* Enrich classloader names with the name of the application/QuarkusUnitTest

* Fix a suspicious H2 URL in Hibernate ORM test config

It doesn't matter in practice since the only test using this config
reproduces a startup failure that happens before we start H2;
but still, let's fix this for correctness.

* Always reset the ForkJoinPool TCCL on startup

* Safer, slightly simpler register/release patterns for config

Some config was lazily created and bound to the dev class loader
through the configsForClassLoader map
io.smallrye.config.SmallRyeConfigProviderResolver#getConfig(java.lang.ClassLoader),
and the corresponding map entry was never cleared,
resulting in the corresponding classloader never being garbage
collected.

* Avoid referencing ClassLoaders in ProtectionDomains

This removes a whole class of metaspace leaks caused by
Thread.inheritedAccessControlContext referencing ProtectionDomains which
reference older classloaders.

Of course this may have impacts on how the SecurityManager behaves, but
as I understand it, absolutely no part of Quarkus is ready to run with a
SecurityManager enabled anyway.

* Improve error messages on KafkaDevServicesDevModeTestCase failures

* Plug leak in tests relying on lazily-initialized config

Since a previous commit, we now remember the lazily-created config in
QuarkusConfigFactory, which is good because in dev mode it allows us to
release it upon restart (when we call setConfig(null)).

However, in *tests* that rely on this lazily initialized config, just
calling releaseConfig(ConfigProvider.getConfig()) is no longer enough,
because of that remembered config in QuarkusConfigFactory: we need to
reset that reference too.

If we forget to reset it, then when KafkaDevServicesDevModeTestCase
will execute, TestHTTPResourceManager#getUri will retrieve the leaked
config from DefaultSerdeConfigTest or DefaultSchemaConfigTest,
the injected URI in KafkaDevServicesDevModeTestCase will be wrong,
and the test will fail (with very unhelpful error messages).

* Stronger safeguards against leaked config in internal Quarkus*Test extensions

If someone calls ConfigProvider.getConfig() out of the blue in a test
that doesn't use any Quarkus*Test extension, this will call
QuarkusConfigFactory and leak config in two ways:

1. In QuarkusConfigFactory#config
2. In SmallRyeConfigProviderResolver, registering config for the TCCL,
   which in such a case is most likely the system CL.

A well-behaved test would call QuarkusConfigFactory.setConfig(null) to
clean up all that, but it's easy to miss and there is potential for
ConfigProvider.getConfig() being called indirectly, so there's no way we
can guarantee all tests are well-behaved.

This should at least guarantee that after a badly behaving test
executes, the next test using a Quarkus*Test extension will clean up the
mess.

* More comprehensive registration/release of config in some tests

If someone calls ConfigProvider.getConfig() out of the blue in a test
that doesn't use any Quarkus*Test extension, this will indirectly call
QuarkusConfigFactory and leak config in two ways:

1. In QuarkusConfigFactory#config
2. In SmallRyeConfigProviderResolver, registering config for the TCCL,
   which in such a case is most likely the system CL.

Thus, a well-behaved test should call QuarkusConfigFactory.setConfig(null)
to clean up all that, no just SmallRyeConfigProviderResolver.releaseConfig().

Similarly, tests that register configuration explicitly can just call
QuarkusConfigFactory.setConfig(config) at the beginning and
QuarkusConfigFactory.setConfig(null) at the end,
which will properly simulate how a real Quarkus application behaves,
and should cover all edge cases involving multiple classloaders,
properly cleaning up everything at the end of the test.

* Merge test-related ConfigUtils into a single class

* Maven CLI: use recipes for platform extensions

* Raise FileSystemWatcherTestCase wait times

Consecutive to:
https://github.com/quarkusio/quarkus/runs/18068790284

* Improve error messages when an @Embedded type is not annotated with @embeddable

* Strimzi OAuth substitutions

* Changes to support jaeger remote sampler

* Ignore bridge methods when checking for @embeddable annotation on @Embedded/@EmbeddedId

* enable use layer 7 proxy for reactive pg client

* Remove type from ResolverMapKey

* Update OpenSearch container test configuration

- Limit OpenSearch memory usage
- Disable disk-based shard allocation thresholds

* Fix grpc-generation-reference.adoc

It was missing the header, not including the attributes and not using
the right syntax for subs=attributes+ leading to all sorts of rendering
issues.

* Fix a broken reference in grpc-getting-started.adoc

* docs: how to store secrets locally

Handy guide on how to deal with local secrets, SEO friendly.

* Add to Security JPA support for named persistence units

* Bump jaxb-runtime.version from 4.0.3 to 4.0.4

Bumps `jaxb-runtime.version` from 4.0.3 to 4.0.4.

Updates `org.glassfish.jaxb:jaxb-runtime` from 4.0.3 to 4.0.4

Updates `org.glassfish.jaxb:jaxb-core` from 4.0.3 to 4.0.4

Updates `org.glassfish.jaxb:txw2` from 4.0.3 to 4.0.4

---
updated-dependencies:
- dependency-name: org.glassfish.jaxb:jaxb-runtime
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.glassfish.jaxb:jaxb-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.glassfish.jaxb:txw2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Fix sockjs documentation

* ArC: improve how single-context normal scopes are found

* Disable Http2RSTFloodProtectionTest on Windows

* Fix typo in reactive-event-bus.adoc

* Validate Startup observers are not secured with RBAC annotations

* Cleanup codestarter tests

- Remove unnecessary modifiers
- Unify line-breaks
- Unify indentation

* Make contextResolverMap used for both reader and writer

* Update virtual thread documentation to Java 21

* Add a description to the Stork extension metadata

* Kc Admin Cl.:Fix typo that allows using customized ObjectMapper

* Bump apicurio-registry.version from 2.4.7.Final to 2.4.14.Final

* Kafka Confluent avro schema serializer doc update

Resolves quarkusio#36372

* Qute: dev mode - add config to skip restart for some templates

- resolves quarkusio#36692

* Maven CLI: add wildcard matching into recipes detection

* Recommend GraalVM CE

* Fix missing section in the gRPC getting started guide

* Security JPA: support Hibernate multitenancy

* Exclude resteasy-client from lra-proxy-api in narayana-lra extension

* Add jvmArgs option to Quarkus Gradle plugin task quarkusRun

* added discord as know oidc provider

added unittest for merging discord oidc properties

* Quarkus code-gen (Gradle): Fix behavior to filter unavailable services

Java Services (those in `META-INF/services/*` files) that are defined in the (Gradle) project that uses the Quarkus Gradle plugin, are not available when Quarkus code generation runs. This is simply a task-dependency requirement in Gradle, because Java compilation happens after code generation. If a Java service, for example a Smallrye config sources, is present and the (Smallrye) configuration is needed by a Quarkus extension, the build fails during Quarkus code generation with an error message like this:
```
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':quarkusGenerateCode'.
...
Caused by: org.gradle.workers.internal.DefaultWorkerExecutor$WorkExecutionException: A failure occurred while executing io.quarkus.gradle.tasks.worker.CodeGenWorker
...
Caused by: java.util.ServiceConfigurationError: io.smallrye.config.ConfigSourceFactory: Provider xyz not found
```

`io.quarkus.deployment.CodeGenerator` has a mechanism to filter out unavailable services via `getUnavailableConfigServices()`. However the callback passed to `io.quarkus.paths.PathTree.apply()` can stop before all roots/trees (`io.quarkus.paths.PathTree.getRoots()`) have been "asked" (`MultiRootPathTree`), because the callback can return a non-`null` value. This "early stop" happens before the root/tree containing the source with the `META-INF/services/*` has been processed.

The bug is only triggered, if a Java service is defined in the Gradle project using the Quarkus Gradle plugin and if a Quarkus extension using the configuration is a dependency of that project.

This change updates the callback implementation to collect all unavailable services from all `PathTree` roots/trees.

An integration test using the Gradle plugin is included as well.

Two logging/spelling mistakes have been fixed as well.

Fixes quarkusio#36716

* Fix assertions in Hibernate ORM 5.6 compatibility tests

Hibernate ORM 5.6 does not, in fact, preserve information for OffsetTime
columns. It just slaps the current JVM timezone on a stored LocalTime.

* Gradle plugin: use full URI for configuration source locations

`io.quarkus.gradle.tasks.EffectiveConfig.CombinedConfigSourceProvider` passes only the "file extension" (e.g. `application.properties`) down to `io.smallrye.config.AbstractLocationConfigSourceLoader#loadConfigSources(java.lang.String[], int, java.lang.ClassLoader)`, which may let that function behave wrong and try to for example access an `application.properties` in the wrong location. This can be reproduced by placing an `application.properties` file in the project directory of a Gradle project that uses the Quarkus Gradle plugin.

This change fixes this behavior by passing down the correct locations as the `String` representation of the resource URIs, instead of just the "file extensions".

Fixes quarkusio#36767

* Tests to check content type between Services and OpenAPI

Signed-off-by: Phillip Kruger <[email protected]>

* Kafka Streams Dev UI migration to v2

* Fix codestarts snapshots

* [quarkusio#36582] Fix bug: TransactionalUniAsserter never fails

TransactionalUniAsserterTestMethodInvoker is a subclass of
RunOnVertxContextTestMethodInvoker.

The problem is that there were two separate pointers keeping
track of the asserter in the superclass and in the subclass.
This lead to only one pointer being initialized and, if the wrong
pointer was null-checked, the asserter was ignored causing the test
to never fail.

This commit fixes the issue by keeping only one reference to the
asserter in the superclass.

* [quarkusio#36582] Minor clean up in TransactionalUniAsserterTest

Use method reference where possible

* [quarkusio#36582] Minor clean up around lambdas

* [quarkusio#36582] Add UnwrappableUniAsserter interface

The goal is to extract the `asUni()` method from `UniAsserter`

* Duplicate Authorization Bearer Header Fix

* Tiny Vale tweaks for Datasource and Logging guide

Signed-off-by: Michal Maléř <[email protected]>

* More reliable test setup in integration-tests/hibernate-orm-tenancy/datasource

The previous config was relying on FlyWay on a default datasource to create other databases
which named datasources pointed at.

Which works well, until a named datasource gets initialized before the default datasource;
then named datasource initialization fails because the corresponding database doesn't exist yet.

This happened on my local laptop, I don't know why it didn't happen on CI.

* Skip non-existing directory when watching for changes

Fix quarkusio#32092.
The issue was not specific to Gradle.
It was problematic with Gradle, as it was configuring the watcher on not existing directories.

* Remove erroneous preview status from cache doc

* Fix OIDC key resolver to accept SHA256 certificate thumbprints

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Phillip Kruger <[email protected]>
Signed-off-by: Michal Maléř <[email protected]>
Co-authored-by: Guillaume Smet <[email protected]>
Co-authored-by: Pedro Igor <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sergey Beryozkin <[email protected]>
Co-authored-by: asjervanasten <[email protected]>
Co-authored-by: Andrea Peruffo <[email protected]>
Co-authored-by: Clement Escoffier <[email protected]>
Co-authored-by: Clement Escoffier <[email protected]>
Co-authored-by: Foivos Zakkak <[email protected]>
Co-authored-by: Jan Martiska <[email protected]>
Co-authored-by: Auri Munoz <[email protected]>
Co-authored-by: Martin Kouba <[email protected]>
Co-authored-by: Ales Justin <[email protected]>
Co-authored-by: George Gastaldi <[email protected]>
Co-authored-by: Ozan Gunalp <[email protected]>
Co-authored-by: Eric Deandrea <[email protected]>
Co-authored-by: Phillip Kruger <[email protected]>
Co-authored-by: Rostislav Svoboda <[email protected]>
Co-authored-by: Rui Balau <[email protected]>
Co-authored-by: Loïc Mathieu <[email protected]>
Co-authored-by: Georgios Andrianakis <[email protected]>
Co-authored-by: Andy Damevin <[email protected]>
Co-authored-by: jeanphi.baconnais <[email protected]>
Co-authored-by: Michal Maléř <[email protected]>
Co-authored-by: Rolfe Dlugy-Hegwer <[email protected]>
Co-authored-by: Ladislav Thon <[email protected]>
Co-authored-by: Matej Novotny <[email protected]>
Co-authored-by: mariofusco <[email protected]>
Co-authored-by: Michal Vavřík <[email protected]>
Co-authored-by: Yoann Rodière <[email protected]>
Co-authored-by: JiriOndrusek <[email protected]>
Co-authored-by: hiteshkhatri97 <[email protected]>
Co-authored-by: Juan Diego López V <[email protected]>
Co-authored-by: Alex Martel <[email protected]>
Co-authored-by: Max Rydahl Andersen <[email protected]>
Co-authored-by: marko-bekhta <[email protected]>
Co-authored-by: Geoffrey De Smet <[email protected]>
Co-authored-by: Bruno Baptista <[email protected]>
Co-authored-by: kdnakt <[email protected]>
Co-authored-by: Marco Bungart <[email protected]>
Co-authored-by: xstefank <[email protected]>
Co-authored-by: Alexey Loubyansky <[email protected]>
Co-authored-by: Roman Ziske <[email protected]>
Co-authored-by: Robert Stupp <[email protected]>
Co-authored-by: David Cotton <[email protected]>
Co-authored-by: Roberto Cortez <[email protected]>
Co-authored-by: Davide D'Alto <[email protected]>
Co-authored-by: sahuefficy <[email protected]>
DavideD pushed a commit to DavideD/quarkus that referenced this issue Nov 27, 2023
Fix quarkusio#32092.
The issue was not specific to Gradle.
It was problematic with Gradle, as it was configuring the watcher on not existing directories.
holly-cummins pushed a commit to holly-cummins/quarkus that referenced this issue Feb 8, 2024
Fix quarkusio#32092.
The issue was not specific to Gradle.
It was problematic with Gradle, as it was configuring the watcher on not existing directories.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/gradle Gradle area/grpc gRPC env/windows Impacts Windows machines kind/bug Something isn't working
Projects
None yet
6 participants