Bug! Spring bootJar: NoSuchFileException: /path/application.jar!/BOOT-INF/classes, /path/application.jar!/BOOT-INF/lib/somelib.jar #1101

magicprinc · 2024-02-01T12:22:43Z

Application is in spring boot application.jar, runs in Kubernetes

smallRyeConfigBuilder
  .withSources(JPropertiesConfigSourceProvider.of(115, "git.properties"))

where

JPropertiesConfigSourceProvider.of ≈ loadConfigSources(location, ordinal, classLoader);

App fails with

... irelevant 

Caused by: java.io.UncheckedIOException: Failed to read /opt/service/application.jar!/BOOT-INF/classes
	at io.smallrye.common.classloader.ClassPathUtils.processAsPath(ClassPathUtils.java:146) ~[smallrye-common-classloader-2.2.0.jar!/:2.2.0]
	at io.smallrye.common.classloader.ClassPathUtils.consumeAsPath(ClassPathUtils.java:102) ~[smallrye-common-classloader-2.2.0.jar!/:2.2.0]
	at io.smallrye.common.classloader.ClassPathUtils.consumeAsPaths(ClassPathUtils.java:86) ~[smallrye-common-classloader-2.2.0.jar!/:2.2.0]
	at io.smallrye.config.AbstractLocationConfigSourceLoader.tryClassPath(AbstractLocationConfigSourceLoader.java:141) ~[smallrye-config-core-3.5.2.jar!/:3.5.2]
	at io.smallrye.config.AbstractLocationConfigSourceLoader.loadConfigSources(AbstractLocationConfigSourceLoader.java:104) ~[smallrye-config-core-3.5.2.jar!/:3.5.2]
	at io.smallrye.config.AbstractLocationConfigSourceLoader.loadConfigSources(AbstractLocationConfigSourceLoader.java:87) ~[smallrye-config-core-3.5.2.jar!/:3.5.2]
	at org.jooq.lambda.conf.impl.JPropertiesConfigSourceProvider.<init>(JPropertiesConfigSourceProvider.java:38) ~[common-utils-1.141.jar!/:na]
	at org.jooq.lambda.conf.impl.JPropertiesConfigSourceProvider.of(JPropertiesConfigSourceProvider.java:50) ~[common-utils-1.141.jar!/:na]
	at org.jooq.lambda.conf.JProperties.smallRyeConfigBuilder(JProperties.java:712) ~[common-utils-1.141.jar!/:na]
	at org.jooq.lambda.conf.JProperties$SmallRyeConfigHolder.createConfig(JProperties.java:872) ~[common-utils-1.141.jar!/:na]
	at org.jooq.lambda.conf.JProperties$SmallRyeConfigHolder.<clinit>(JProperties.java:868) ~[common-utils-1.141.jar!/:na]	... 27 common frames omitted
Caused by: java.nio.file.NoSuchFileException: /opt/service/application.jar!/BOOT-INF/classes
	at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.<init>(ZipFileSystem.java:166) ~[jdk.zipfs:na]
	at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getZipFileSystem(ZipFileSystemProvider.java:125) ~[jdk.zipfs:na]
	at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.newFileSystem(ZipFileSystemProvider.java:120) ~[jdk.zipfs:na]
	at java.base/java.nio.file.FileSystems.newFileSystem(FileSystems.java:528) ~[na:na]
	at java.base/java.nio.file.FileSystems.newFileSystem(FileSystems.java:400) ~[na:na]
	at io.smallrye.common.classloader.ClassPathUtils.processAsPath(ClassPathUtils.java:139) ~[smallrye-common-classloader-2.2.0.jar!/:2.2.0]

The text was updated successfully, but these errors were encountered:

radcortez · 2024-02-01T12:28:16Z

Any chance you can provide a reproducer?

magicprinc · 2024-02-01T12:41:46Z

I can't reproduce locally: this is not even my project:
the project uses my library, that uses SmallRye Config.

I think, an easy fix is:

io.smallrye.common.classloader.ClassPathUtils#consumeAsPaths(ClassLoader, String, Consumer<Path>)

could logger.warn (Exception) • always or • if configuration flag is set

This strange path in Class Path comes from:

ClassPathUtils#consumeAsPaths
final Enumeration<URL> resources = cl.getResources(resource);

which returns such inaccessible path when running as Spring-boot in Kubernetes pod 🤷‍♀️

radcortez · 2024-02-01T14:39:29Z

Unfortunately without a way to reproduce the issue, we can't push random fixes. I'm happy to help you with this, but I don't have much to work with.

Can you reproduce it by deploying to a local k8s, like minikube?

magicprinc · 2024-02-01T21:16:58Z

I have managed to start app in minikube: no exception.
So I can't reproduce it. 🤷‍♀️

But now we know:
a resource from ClassLoader#getResources(name): Enumeration<URL> SOMETIMES can't be opened using
url.openStream()

I have checked: git.properties is in this class path directory (generated by gradle task: https://github.com/n0mer/gradle-git-properties)

processor:/workspace/BOOT-INF/classes:
-rw-r--r-- 1 1001 cnb 3932 Jan  1  1980 application.properties
drwxr-xr-x 3 1001 cnb 4096 Jan  1  1980 com                   
-rw-r--r-- 1 1001 cnb  675 Jan  1  1980 git.properties        
processor:/workspace/BOOT-INF:
drwxr-xr-x 3 1001 cnb  4096 Jan  1  1980 classes       
-rw-r--r-- 1 1001 cnb  5685 Jan  1  1980 classpath.idx 
-rw-r--r-- 1 1001 cnb   212 Jan  1  1980 layers.idx    
drwxr-xr-x 2 1001 cnb 12288 Jan  1  1980 lib           
processor:/workspace:
drwxr-xr-x 1 1001 cnb  4096 Jan  1  1980 BOOT-INF      
drwxr-xr-x 2 1001 cnb  4096 Jan  1  1980 META-INF      
drwxr-xr-x 3 1001 cnb  4096 Jan  1  1980 org

I have "solved" the problem by making copies of your classes and wrapping ClassPathUtils#consumeAsPath into try catch + warn(Exception)

radcortez · 2024-02-01T23:31:58Z

I have managed to start app in minikube: no exception. So I can't reproduce it. 🤷‍♀️

When someone reports an issue with Kubernetes, I can consistently reproduce it locally in Minikube. In cases where I could not do it, it was due to differences in the reproducer, configuration, or other deployment aspects.

But now we know: a resource from ClassLoader#getResources(name): Enumeration<URL> SOMETIMES can't be opened using url.openStream()

I don't see how that is possible, using the same binary in two different environments. It shouldn't make any difference, but we cannot just assume that without a proper reproducer.

magicprinc · 2024-03-22T11:42:10Z

Ok. The problem is not in a container, but in Spring Boot

If you run jar file created by (Spring) boorJar task, you receive

2024-03-22 14:39:04.092  WARN 20232 --- [           main] .j.l.c.i.JPropertiesConfigSourceProvider : Can't read properties: starter.properties

java.io.UncheckedIOException: Failed to read D:\REPO\PUSH.PUSH\PushWalletProcessorApplication.jar!\BOOT-INF\lib\common-starter-web-2.9.jar
        at io.smallrye.common.classloader.ClassPathUtils.processAsPath(ClassPathUtils.java:146) ~[smallrye-common-classloader-2.2.0.jar!/:2.2.0]
        at io.smallrye.common.classloader.ClassPathUtils.consumeAsPath(ClassPathUtils.java:102) ~[smallrye-common-classloader-2.2.0.jar!/:2.2.0]
        at io.smallrye.common.classloader.ClassPathUtils.consumeAsPaths(ClassPathUtils.java:86) ~[smallrye-common-classloader-2.2.0.jar!/:2.2.0]
        at io.smallrye.config.AbstractLocationConfigSourceLoader.tryClassPath(AbstractLocationConfigSourceLoader.java:141) ~[smallrye-config-core-3.7.1.jar!/:3.7.1]
        at io.smallrye.config.AbstractLocationConfigSourceLoader.loadConfigSources(AbstractLocationConfigSourceLoader.java:104) ~[smallrye-config-core-3.7.1.jar!/:3.7.1]
        at io.smallrye.config.AbstractLocationConfigSourceLoader.loadConfigSources(AbstractLocationConfigSourceLoader.java:87) ~[smallrye-config-core-3.7.1.jar!/:3.7.1]
        at org.jooq.lambda.conf.impl.JPropertiesConfigSourceProvider.<init>(JPropertiesConfigSourceProvider.java:42) ~[common-utils-1.155.jar!/:na]
        at org.jooq.lambda.conf.impl.JPropertiesConfigSourceProvider.of(JPropertiesConfigSourceProvider.java:55) ~[common-utils-1.155.jar!/:na]
        at org.jooq.lambda.conf.JProperties.smallRyeConfigBuilder(JProperties.java:716) ~[common-utils-1.155.jar!/:na]
        at org.jooq.lambda.conf.JProperties$SmallRyeConfigHolder.createConfig(JProperties.java:880) ~[common-utils-1.155.jar!/:na]
        at org.jooq.lambda.conf.JProperties$SmallRyeConfigHolder.<clinit>(JProperties.java:876) ~[common-utils-1.155.jar!/:na]
        at org.jooq.lambda.conf.JProperties.config(JProperties.java:872) ~[common-utils-1.155.jar!/:na]
        at org.jooq.lambda.conf.impl.spring.SpringSmallRyeConfigPropertySource.<init>(SpringSmallRyeConfigPropertySource.java:27) ~[common-utils-1.155.jar!/:na]
        at org.jooq.lambda.conf.impl.spring.JPropertiesSpringConf.smallRyeConfigPropertySourcesPlaceholderConfigurer(JPropertiesSpringConf.java:67) ~[common-utils-1.155.jar!/:na]
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[na:na]
        at java.base/java.lang.reflect.Method.invoke(Method.java:580) ~[na:na]
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~[spring-beans-5.3.31.jar!/:5.3.31]
        at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:648) ~[spring-beans-5.3.31.jar!/:5.3.31]
        at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:633) ~[spring-beans-5.3.31.jar!/:5.3.31]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1352) ~[spring-beans-5.3.31.jar!/:5.3.31]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1195) ~[spring-beans-5.3.31.jar!/:5.3.31]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.31.jar!/:5.3.31]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.31.jar!/:5.3.31]
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.31.jar!/:5.3.31]
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.31.jar!/:5.3.31]
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.31.jar!/:5.3.31]
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:213) ~[spring-beans-5.3.31.jar!/:5.3.31]
        at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:171) ~[spring-context-5.3.31.jar!/:5.3.31]
        at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:756) ~[spring-context-5.3.31.jar!/:5.3.31]
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:572) ~[spring-context-5.3.31.jar!/:5.3.31]
        at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:147) ~[spring-boot-2.7.18.jar!/:2.7.18]
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:732) ~[spring-boot-2.7.18.jar!/:2.7.18]
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:409) ~[spring-boot-2.7.18.jar!/:2.7.18]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:308) ~[spring-boot-2.7.18.jar!/:2.7.18]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1300) ~[spring-boot-2.7.18.jar!/:2.7.18]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1289) ~[spring-boot-2.7.18.jar!/:2.7.18]
        at com.devinotele.push.walletprocessor.PushWalletProcessor.main(PushWalletProcessor.java:15) ~[classes!/:na]
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[na:na]
        at java.base/java.lang.reflect.Method.invoke(Method.java:580) ~[na:na]
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[PushWalletProcessorApplication.jar:na]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) ~[PushWalletProcessorApplication.jar:na]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[PushWalletProcessorApplication.jar:na]
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) ~[PushWalletProcessorApplication.jar:na]
Caused by: java.nio.file.NoSuchFileException: D:\REPO\PUSH.PUSH\PushWalletProcessorApplication.jar!\BOOT-INF\lib\common-starter-web-2.9.jar
        at jdk.zipfs/jdk.nio.zipfs.ZipFileSystem.<init>(ZipFileSystem.java:166) ~[jdk.zipfs:na]
        at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getZipFileSystem(ZipFileSystemProvider.java:125) ~[jdk.zipfs:na]
        at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.newFileSystem(ZipFileSystemProvider.java:120) ~[jdk.zipfs:na]
        at java.base/java.nio.file.FileSystems.newFileSystem(FileSystems.java:528) ~[na:na]
        at java.base/java.nio.file.FileSystems.newFileSystem(FileSystems.java:400) ~[na:na]
        at io.smallrye.common.classloader.ClassPathUtils.processAsPath(ClassPathUtils.java:139) ~[smallrye-common-classloader-2.2.0.jar!/:2.2.0]
        ... 42 common frames omitted

openjdk version "21.0.2" 2024-01-16 LTS
OpenJDK Runtime Environment (build 21.0.2+14-LTS)
OpenJDK 64-Bit Server VM (build 21.0.2+14-LTS, mixed mode, sharing)

smallrye-config-common-3.7.1.jar

command line: java -jar PushWalletProcessorApplication.jar

magicprinc · 2024-03-22T15:11:02Z

quick fix:

io.smallrye.common.classloader.ClassPathUtils#processAsPath

➖ final int exclam = file.lastIndexOf('!');
➕ final int exclam = file.indexOf('!'); // lastIndexOf

magicprinc · 2024-03-22T18:10:49Z

The second problem is classpath: scheme. Spring uses it to specify classpath vs filesystem.

Could you add it to
io.smallrye.config.AbstractLocationConfigSourceLoader#loadConfigSources(java.lang.String[], int, java.lang.ClassLoader)
~ line 111

} else if (uri.getScheme().startsWith("classpath")) {
  configSources.addAll(tryClassPath(uri, ordinal, classLoader));
}

magicprinc · 2024-03-22T18:30:31Z

How I have temporarily solved the problem

I have made the above-mentioned fix and published it on github
https://github.com/magicprinc/smallrye-common-classloader-fix
built com.github.magicprinc:smallrye-common-classloader-fix:2.3.3 using jitpack.io:
https://jitpack.io/#magicprinc/smallrye-common-classloader-fix/2.3.3
and imported it according to instructions ^
removed original smallrye-common-classloader

configurations.configureEach {
  exclude group: "io.smallrye.common", module: "smallrye-common-classloader" // tmp fix: classes are in fix dependency
}

in my FallbackConfigSourceInterceptor (where I check
smallrye.config.locations and spring location) added replace "classpath:" with "", if request comes from SmallRye Config (and keep if from spring)

magicprinc · 2024-03-23T13:15:21Z

Fixed version:
https://github.com/magicprinc/smallrye-common/blob/main/classloader/src/main/java/io/smallrye/common/classloader/ClassPathUtils.java

magicprinc · 2024-03-23T14:36:41Z

Example application
Branch main shows the bug.
Branch fix shows the fix.
https://github.com/magicprinc/SmallRyeConfig-SpringBoot

git clone https://github.com/magicprinc/SmallRyeConfig-SpringBoot.git
cd SmallRyeConfig-SpringBoot
./gradlew runBootJar

radcortez added the bug Something isn't working label Feb 1, 2024

magicprinc mentioned this issue Mar 22, 2024

Doesn't work in Spring Boot jar smallrye/smallrye-common#289

Closed

magicprinc mentioned this issue Mar 22, 2024

Add classpath: scheme support (similar to Spring) #1141

Open

magicprinc changed the title ~~Bug? NoSuchFileException: /opt/service/application.jar!/BOOT-INF/classes~~ Bug! Spring bootJar: NoSuchFileException: /path/application.jar!/BOOT-INF/classes Mar 22, 2024

magicprinc changed the title ~~Bug! Spring bootJar: NoSuchFileException: /path/application.jar!/BOOT-INF/classes~~ Bug! Spring bootJar: NoSuchFileException: /path/application.jar!/BOOT-INF/classes, /path/application.jar!\BOOT-INF\lib\somelib.jar Mar 22, 2024

radcortez added this to the 3.7.2 milestone Apr 22, 2024

radcortez mentioned this issue Apr 22, 2024

Update SR Commons to 2.4.0 #1150

Merged

radcortez closed this as completed Apr 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug! Spring bootJar: NoSuchFileException: /path/application.jar!/BOOT-INF/classes, /path/application.jar!/BOOT-INF/lib/somelib.jar #1101

Bug! Spring bootJar: NoSuchFileException: /path/application.jar!/BOOT-INF/classes, /path/application.jar!/BOOT-INF/lib/somelib.jar #1101

magicprinc commented Feb 1, 2024

radcortez commented Feb 1, 2024

magicprinc commented Feb 1, 2024 •

edited

Loading

radcortez commented Feb 1, 2024

magicprinc commented Feb 1, 2024 •

edited

Loading

radcortez commented Feb 1, 2024

magicprinc commented Mar 22, 2024 •

edited

Loading

magicprinc commented Mar 22, 2024

magicprinc commented Mar 22, 2024 •

edited

Loading

magicprinc commented Mar 22, 2024 •

edited

Loading

magicprinc commented Mar 23, 2024

magicprinc commented Mar 23, 2024 •

edited

Loading

Bug! Spring bootJar: NoSuchFileException: /path/application.jar!/BOOT-INF/classes, /path/application.jar!/BOOT-INF/lib/somelib.jar #1101

Bug! Spring bootJar: NoSuchFileException: /path/application.jar!/BOOT-INF/classes, /path/application.jar!/BOOT-INF/lib/somelib.jar #1101

Comments

magicprinc commented Feb 1, 2024

radcortez commented Feb 1, 2024

magicprinc commented Feb 1, 2024 • edited Loading

radcortez commented Feb 1, 2024

magicprinc commented Feb 1, 2024 • edited Loading

radcortez commented Feb 1, 2024

magicprinc commented Mar 22, 2024 • edited Loading

magicprinc commented Mar 22, 2024

magicprinc commented Mar 22, 2024 • edited Loading

magicprinc commented Mar 22, 2024 • edited Loading

magicprinc commented Mar 23, 2024

magicprinc commented Mar 23, 2024 • edited Loading

magicprinc commented Feb 1, 2024 •

edited

Loading

magicprinc commented Feb 1, 2024 •

edited

Loading

magicprinc commented Mar 22, 2024 •

edited

Loading

magicprinc commented Mar 22, 2024 •

edited

Loading

magicprinc commented Mar 22, 2024 •

edited

Loading

magicprinc commented Mar 23, 2024 •

edited

Loading