Potential NPE in QuarkusClassLoader

[gastaldi]

I am trying to run a Quarkus app in a local container and use remote development mode and I'm facing the same error:

backend     | 2022-09-20 14:56:35,633 ERROR [io.qua.dep.dev.IsolatedDevModeMain] (vert.x-worker-thread-1) Failed to start quarkus: java.lang.RuntimeException: java.lang.NullPointerException: Cannot invoke "io.quarkus.bootstrap.classloading.ClassPathResource.getUrl()" because "res" is null
backend     | 	at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:330)
backend     | 	at io.quarkus.runner.bootstrap.AugmentActionImpl.reloadExistingApplication(AugmentActionImpl.java:265)
backend     | 	at io.quarkus.runner.bootstrap.AugmentActionImpl.reloadExistingApplication(AugmentActionImpl.java:60)
backend     | 	at io.quarkus.deployment.dev.IsolatedDevModeMain.restartApp(IsolatedDevModeMain.java:251)
backend     | 	at io.quarkus.deployment.dev.IsolatedDevModeMain.restartCallback(IsolatedDevModeMain.java:234)
backend     | 	at io.quarkus.deployment.dev.RuntimeUpdatesProcessor.doScan(RuntimeUpdatesProcessor.java:536)
backend     | 	at io.quarkus.deployment.dev.RuntimeUpdatesProcessor.doScan(RuntimeUpdatesProcessor.java:436)
backend     | 	at io.quarkus.vertx.http.runtime.devmode.VertxHttpHotReplacementSetup$4.handle(VertxHttpHotReplacementSetup.java:152)
backend     | 	at io.quarkus.vertx.http.runtime.devmode.VertxHttpHotReplacementSetup$4.handle(VertxHttpHotReplacementSetup.java:139)
backend     | 	at io.vertx.core.impl.ContextBase.lambda$null$0(ContextBase.java:137)
backend     | 	at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:264)
backend     | 	at io.vertx.core.impl.ContextBase.lambda$executeBlocking$1(ContextBase.java:135)
backend     | 	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
backend     | 	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
backend     | 	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1452)
backend     | 	at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
backend     | 	at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
backend     | 	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
backend     | 	at java.base/java.lang.Thread.run(Thread.java:833)
backend     | Caused by: java.lang.NullPointerException: Cannot invoke "io.quarkus.bootstrap.classloading.ClassPathResource.getUrl()" because "res" is null
backend     | 	at io.quarkus.bootstrap.classloading.QuarkusClassLoader.getResources(QuarkusClassLoader.java:250)
backend     | 	at io.quarkus.bootstrap.classloading.QuarkusClassLoader.getResources(QuarkusClassLoader.java:265)
backend     | 	at io.quarkus.bootstrap.classloading.QuarkusClassLoader.getResources(QuarkusClassLoader.java:197)
backend     | 	at io.smallrye.common.classloader.ClassPathUtils.consumeAsPaths(ClassPathUtils.java:84)
backend     | 	at io.smallrye.config.AbstractLocationConfigSourceLoader.tryClassPath(AbstractLocationConfigSourceLoader.java:128)
backend     | 	at io.smallrye.config.AbstractLocationConfigSourceLoader.loadConfigSources(AbstractLocationConfigSourceLoader.java:93)
backend     | 	at io.smallrye.config.AbstractLocationConfigSourceLoader.loadConfigSources(AbstractLocationConfigSourceLoader.java:76)
backend     | 	at io.quarkus.runtime.configuration.ApplicationPropertiesConfigSourceLoader$InClassPath.getConfigSources(ApplicationPropertiesConfigSourceLoader.java:30)
backend     | 	at io.quarkus.runtime.configuration.ApplicationPropertiesConfigSourceLoader$InClassPath.getConfigSources(ApplicationPropertiesConfigSourceLoader.java:27)
backend     | 	at io.smallrye.config.SmallRyeConfigBuilder.build(SmallRyeConfigBuilder.java:439)
backend     | 	at io.quarkus.deployment.ExtensionLoader.loadStepsFrom(ExtensionLoader.java:179)
backend     | 	at io.quarkus.deployment.QuarkusAugmentor.run(QuarkusAugmentor.java:105)
backend     | 	at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:328)
backend     | 	... 18 more

I am not using any database migration tool. Could it be related?

Originally posted by @y-luis in #25256 (comment)

[quarkus-bot]

/cc @Sanne, @aloubyansky, @geoand, @gsmet, @radcortez, @stuartwdouglas

[aloubyansky]

@y-luis could you please provide more info, such as after which actions/steps this error happens? Thanks.

[y-luis-rojo]

Sure. I am trying an app based on a Maven multi-module project here, with the following Dockerfile:

FROM azul/zulu-openjdk-alpine:17

ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en'

# We make four distinct layers so if there are application changes the library layers can be re-used
COPY --chown=185 bootstrap/target/quarkus-app/lib/ /deployments/lib/
COPY --chown=185 bootstrap/target/quarkus-app/*.jar /deployments/
COPY --chown=185 bootstrap/target/quarkus-app/app/ /deployments/app/
COPY --chown=185 bootstrap/target/quarkus-app/quarkus/ /deployments/quarkus/

EXPOSE 8080

USER 185
ENV JAVA_OPTS="-Dquarkus.http.host=0.0.0.0 -Djava.util.logging.manager=org.jboss.logmanager.LogManager"
ENV JAVA_APP_JAR="/deployments/quarkus-run.jar"

WORKDIR /deployments
ENTRYPOINT ["sh", "-c"]

CMD ["java -jar $JAVA_APP_JAR"]

and the following docker-compose.yml (in the project's parent folder):

version: "3.8"

networks:
  local-dev-env-network:

services:
  poc-process-db:
    image: postgres:14
    container_name: "poc-process-db-container"
    ports:
      - "5432:5432"
    networks:
      - local-dev-env-network
    restart: always
    environment:
      POSTGRES_USER: ${DB_USER}
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    volumes:
      - /var/lib/postgresql/data

  poc-process:
    image: "poc-process-image"
    container_name: "poc-process-container"
    ports:
      - "8080:8080"
    networks:
      - local-dev-env-network
    environment:
      QUARKUS_LAUNCH_DEVMODE: true
    build: ./poc-process
    volumes:
      - ~/.m2:/root/.m2
    depends_on:
      - poc-process-db

After ./mvnw clean package -DskipTests and docker-compose up --build, app seems running OK in a container, then I execute quarkus:remote-dev -Dquarkus.live-reload.url=http://localhost:8080 for remote development mode and so far so good, but when trying to hit app endpoint's URL (http://localhost:8080/processes), the referred error is thrown.

Environment

uname -a                                                                                                                                                                                                               
Darwin MacBook-Pro.local 21.6.0 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64 x86_64

java -version                                                                                                                                                                                                    
openjdk version "17.0.4.1" 2022-08-12 LTS
OpenJDK Runtime Environment Zulu17.36+17-CA (build 17.0.4.1+1-LTS)
OpenJDK 64-Bit Server VM Zulu17.36+17-CA (build 17.0.4.1+1-LTS, mixed mode, sharing)

./mvnw --version
Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
Maven home: /Users/<user>/.m2/wrapper/dists/apache-maven-3.8.6-bin/9fc57a49/apache-maven-3.8.6
Java version: 17.0.4.1, vendor: Azul Systems, Inc., runtime: /Library/Java/JavaVirtualMachines/zulu-17.jdk/Contents/Home
Default locale: en_PT, platform encoding: UTF-8
OS name: "mac os x", version: "12.6", arch: "x86_64", family: "mac"

Quarkus version: 2.11.2.Final

[aloubyansky]

Are you sure it's packaged as a mutable-jar?

[y-luis-rojo]

Yes, my bootstrap/application.properties file:

quarkus.datasource.reactive.url=postgresql://poc-process-db:5432/postgres
quarkus.datasource.username=docker
quarkus.datasource.password=pass

quarkus.datasource.db-kind=postgresql
quarkus.hibernate-orm.database.generation=drop-and-create
quarkus.devservices.enabled=true

quarkus.http.cors=true
quarkus.package.type=mutable-jar
quarkus.live-reload.password=changeit
quarkus.live-reload.url=http://localhost:8080

[aloubyansky]

@y-luis-rojo do you launch quarkus:remote-dev from the the application module or from the root of the project? If you are launching from the root, have all the project modules been packaged and installed in your local repo?
I see some issues but I would like to know what you were doing.

[y-luis-rojo]

I launch quarkus:remote-dev from the root of the project. The only I do is ./mvnw clean package -DskipTests, should I try running quarkus:remote-dev from the application module? Should I run ./mvnw clean install -DskipTests maybe?

[aloubyansky]

You could try launching it from the app module but you would have to actually install your modules in your local Maven repo, because otherwise Maven won't be able to resolve the other modules your app depends on. And I think live reload won't work for those other modules.

[y-luis-rojo]

OK but then for live reload to work, it is required to launch from project's root, does that confirms the error comes from a bug?

[aloubyansky]

The dev mode should work launching from the app module. Although I found the remote-dev doesn't for non-app modules when launched from the app module. I'll have a look into that.
And the exception you reported is also a bug, no question about it. I just want to clarify what causes it to fix it properly.

[aloubyansky]

@y-luis-rojo would you be able to test a custom branch of quarkus?

[aloubyansky]

It's not easy to create a proper test we could add to our testsuite for what you are seeing. I actually couldn't reproduce it but I saw other errors that could have a common cause with the one you reported.

[aloubyansky]

Before I open PRs it'd be good to know whether it helps in your situation.

[y-luis-rojo]

@y-luis-rojo would you be able to test a custom branch of quarkus?

I can try. Could you please share documentation on how to do it? Maybe you could publish the branch's artifact for me to use it?

[aloubyansky]

Excellent, here is the branch https://github.com/aloubyansky/quarkus/tree/remote-dev-delete-files-after-app-closed

• git clone [email protected]:aloubyansky/quarkus.git quarkus-remote-dev-fix
• cd quarkus-remote-dev-fix
• git checkout remote-dev-delete-files-after-app-closed
• mvn -Dquickly (this will install the 999-SNAPSHOT version of Quarkus into your local Maven repo)
• In your application replace the quarkus.version or quarkus.platform.version to 999-SNAPSHOT and make sure the groupId of the quarkus-bom imported in your project is io.quarkus and not io.quarkus.platform

Then build the app, the image and launch the remote-dev the way you were launching. Let us know whether it helped or changed the behavior in some way. Thanks a lot!

[y-luis-rojo]

I must be doing something wrong, I am stuck with the following error (means I am not using the most recent version of DevModeMediator but the most recent for RuntimeUpdatesProcessor?):

backend     | 2022-09-23 15:45:37,906 ERROR [io.qua.ver.cor.run.VertxCoreRecorder] (vert.x-eventloop-thread-2) Uncaught exception received by Vert.x: java.lang.NoSuchFieldError: removedFiles
backend     | 	at io.quarkus.deployment.dev.RuntimeUpdatesProcessor.syncState(RuntimeUpdatesProcessor.java:621)
backend     | 	at io.quarkus.vertx.http.runtime.devmode.RemoteSyncHandler$5.handle(RemoteSyncHandler.java:200)
backend     | 	at io.quarkus.vertx.http.runtime.devmode.RemoteSyncHandler$5.handle(RemoteSyncHandler.java:176)
backend     | 	at io.vertx.core.impl.future.FutureImpl$1.onSuccess(FutureImpl.java:91)
backend     | 	at io.vertx.core.impl.future.FutureBase.emitSuccess(FutureBase.java:60)
backend     | 	at io.vertx.core.impl.future.FutureImpl.tryComplete(FutureImpl.java:211)
backend     | 	at io.vertx.core.impl.future.PromiseImpl.tryComplete(PromiseImpl.java:23)
backend     | 	at io.vertx.core.http.impl.HttpEventHandler.handleEnd(HttpEventHandler.java:79)
backend     | 	at io.vertx.core.http.impl.Http1xServerRequest.onEnd(Http1xServerRequest.java:581)
backend     | 	at io.vertx.core.http.impl.Http1xServerRequest.lambda$pendingQueue$1(Http1xServerRequest.java:130)
backend     | 	at io.vertx.core.streams.impl.InboundBuffer.handleEvent(InboundBuffer.java:239)
backend     | 	at io.vertx.core.streams.impl.InboundBuffer.drain(InboundBuffer.java:226)
backend     | 	at io.vertx.core.streams.impl.InboundBuffer.lambda$fetch$0(InboundBuffer.java:279)
backend     | 	at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:264)
backend     | 	at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:246)
backend     | 	at io.vertx.core.impl.EventLoopContext.lambda$runOnContext$0(EventLoopContext.java:43)
backend     | 	at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
backend     | 	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
backend     | 	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
backend     | 	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569)
backend     | 	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
backend     | 	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
backend     | 	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
backend     | 	at java.base/java.lang.Thread.run(Thread.java:833)

[aloubyansky]

Have you done docker-compose down?

[y-luis-rojo]

Yes, I did. Not sure what it was last error, but it looks good now, I was able to work in remote development mode and use Live Coding feature! :)

[aloubyansky]

Thanks a lot for testing @y-luis-rojo

[y-luis-rojo]

Hi @aloubyansky, we've upgraded to version 2.13.2.Final and I confirm the issue is not happening but then it happens a different one I commented at #17747 (comment).