Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jar loading order error causes spring-boot startup to fail #1871

Closed
xiaopihai7256 opened this issue Jul 28, 2019 · 11 comments · Fixed by #3280
Closed

Jar loading order error causes spring-boot startup to fail #1871

xiaopihai7256 opened this issue Jul 28, 2019 · 11 comments · Fixed by #3280

Comments

@xiaopihai7256
Copy link

Environment:

  • Jib version: 1.4.0
  • Build tool: maven 3.3.9
  • OS: windows 10 1903
  • *Server OS [use to run Docker imange]: Linux version 5.1.14-1.el7.elrepo.x86_64 *
  • Spring-boot: 2.0.6.RELEASE

Description of the issue:

Sorry, my English is not good, but I will try to explain the problems I have encountered.

In my project, there are two jars with different names, we call A.jar and B.jar. A rewrites and overwrites a Class with the same name in B. In order to work properly, I introduce A in pom.xml in order higher than B, which works fine in Spring-boot's Fatjar.

But when I use Jib, B always loads before A, causing the coverage of the Class to not take effect, so when the application starts, the JDK prompts "XXXX method not found in XXX.class". finally, application start failed

Expected behavior:

A.jar can be loaded earlier than B.jar, just like working in Spring-boot's Fatjar.

Steps to reproduce:

  1. add Jib maven plugin to pom.xml
  2. run command: mvn compile jib:build -Dregistry_url={my server ip}:4000 -Dregistry_username={username} -Dregistry_password={password} -Dimage_tag=v1.2.0-dev-2019072101 -DsendCredentialsOverHttp=true
  3. try to run docker image, spring-boot startup failed

jib-maven-plugin Configuration:

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
    <groupId>com.google.cloud.tools</groupId>
    <artifactId>jib-maven-plugin</artifactId>
    <version>1.4.0</version>
    <configuration>
        <container>
            <jvmFlags>
                <jvmFlag>-Xms512m</jvmFlag>
            </jvmFlags>
            <ports>
                <port>8080</port>
            </ports>
            <useCurrentTimestamp>true</useCurrentTimestamp>
        </container>
        <from>
            <image>registry.cn-hangzhou.aliyuncs.com/choerodon-tools/javabase:0.7.1</image>
        </from>
        <to>
            <image>${registry_url}/${docker.image.prefix}/${project.artifactId}:${maven.build.timestamp}</image>
            <auth>
                <username>${registry_username}</username>
                <password>${registry_password}</password>
            </auth>
        </to>
        <allowInsecureRegistries>true</allowInsecureRegistries>
        <jvmFlags>
            <jvmFlag>-Djava.security.edg=file:/dev/./urandom</jvmFlag>
        </jvmFlags>
        <
    </configuration>
</plugin>

Log output:

this is maven log:

[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building {my-cpplication-name} 0.9.0.RELEASE
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @ hcbm-iam ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 0 resource
[INFO] Copying 2 resources
[INFO] Copying 0 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ hcbm-iam ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- jib-maven-plugin:1.4.0:build (default-cli) @ hcbm-iam ---
[WARNING] Authentication over HTTP is enabled. It is strongly recommended that you do not enable this on a public network!
[WARNING] Setting image creation time to current time; your image may not be reproducible.
[INFO]
[INFO] Containerizing application to ?[{my server ip}:4000/contract/{my-application-name}:20190728?[0m...
[INFO] ?[1m[                              ] 0.0% complete?[0m
[INFO] ?[1m> building image to registry?[0m
[INFO] ?[1A?[1A?[1A?[0J
[INFO] ?[1A?[1mExecuting tasks:?[0m
[INFO] ?[1m[                              ] 0.0% complete?[0m
[INFO] ?[1m> pulling base image manifest?[0m
.......
[INFO] ?[1m> authenticating push to {my server ip}:4000?[0m
[INFO] ?[1m> preparing base image layer pullers?[0m
[INFO] ?[1A?[1A?[1A?[1A?[0J
[INFO] ?[1A?[1mExecuting tasks:?[0m

...// skip loading log , all success and 100%

 [INFO] ?[1A?[1mExecuting tasks:?[0m
[INFO] ?[1mExecuting tasks:?[0m
[INFO] ?[1m[==============================] 100.0% complete?[0m
[INFO] ?[1A?[1A?[0J
[INFO] ?[1m[==============================] 100.0% complete?[0m
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:21 min
[INFO] Finished at: 2019-07-29T00:31:05+08:00
[INFO] Final Memory: 50M/313M
[INFO] ------------------------------------------------------------------------

when application running, this is error log of spring-boot:

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled. 
2019-07-28 22:44:06.793   --- [           main]  : 
*************************** 
APPLICATION FAILED TO START 
*************************** 
 
Description: 
 
An attempt was made to call the method 

io.choerodon.mybatis.common.BaseMapper.selectOptional(Ljava/lang/Object;Lorg/hzero/mybatis/common/Criteria;)Ljava/util/List; 

but it does not exist. 

Its class, io.choerodon.mybatis.common.BaseMapper, is available from the following locations: 
 
    jar:file:/app/libs/choerodon-starter-mybatis-mapper-0.10.0.RELEASE.jar!/io/choerodon/mybatis/common/BaseMapper.class 
    jar:file:/app/libs/hzero-starter-mybatis-mapper-0.9.0.RELEASE.jar!/io/choerodon/mybatis/common/BaseMapper.class 
 
It was loaded from the following location: 
 
    file:/app/libs/choerodon-starter-mybatis-mapper-0.10.0.RELEASE.jar 
 
 
Action: 
 
Correct the classpath of your application so that it contains a single, compatible version of io.choerodon.mybatis.common.BaseMapper 
 

Additional Information:

If you need additional information, I can always provide the party.

@luxingxiao
Copy link

luxingxiao commented Jul 29, 2019

我建议你使用AOP来处理A.jar重写B.jar的问题,最好不要两个具备相同类的jar包

UPDATE by @chanseokoh: Google Translate says "I suggest you use AOP to handle the problem of A.jar rewriting B.jar. It is best not to have two jars with the same class."

@chanseokoh
Copy link
Member

chanseokoh commented Jul 29, 2019

I agree with what @luxingxiao said. I don't know about AOP, but I think it seems best to avoid having multiple JARs with same classes. This seems particularly true in this case where the class clash is not accidental but is to intentionally override the other class. Maybe you could consider shading (although I'm not saying shading is a good thing.)

As a last resort, there is an ugly workaround that you prepend /app/libs/hzero-starter-mybatis-mapper-0.9.0.RELEASE.jar in the classpath list using Jib's <extraClasspath> configuration, but I think you shouldn't really do this.

@luxingxiao
Copy link

I agree with what @luxingxiao said. I don't know about AOP, but I think it seems best to avoid having multiple JARs with same classes. This seems particularly true in this case where the class clash is not accidental but is to intentionally override the other class. Maybe you could consider shading (although I'm not saying shading is a good thing.)

As a last resort, there is an ugly workaround that you prepend /app/libs/hzero-starter-mybatis-mapper-0.9.0.RELEASE.jar in the classpath list using Jib's <extraClasspath> configuration, but I think you shouldn't really do this.

Thank you @chanseokoh . AOP means Aspect Oriented Programming .

@xiaopihai7256
Copy link
Author

我建议你使用AOP来处理A.jar重写B.jar的问题,最好不要两个具备相同类的jar包

UPDATE by @chanseokoh: Google Translate says "I suggest you use AOP to handle the problem of A.jar rewriting B.jar. It is best not to have two jars with the same class."

谢谢你的建议🙏。事实上,A和B两个jar都是项目中依赖的外部jar包,所以对于其中像覆盖这种不规范的操作我也无能为力。不过我也仔细看了下源码,覆写的A.jar不仅改变了原来B.jar中的代码逻辑,还变更了方法和类签名,所以我理解对于这种情况,是不是AOP也没有办法处理?

Thank you for your advice. In fact, both the A and B jars are external jars that depend on the project, so I can't do anything about covering such irregular operations. However, I also carefully looked at the source code. The overwritten A.jar not only changed the code logic in the original B.jar, but also changed the method and class signature, so I understand that for this situation, there is no way for AOP to handle it ?

@xiaopihai7256
Copy link
Author

I agree with what @luxingxiao said. I don't know about AOP, but I think it seems best to avoid having multiple JARs with same classes. This seems particularly true in this case where the class clash is not accidental but is to intentionally override the other class. Maybe you could consider shading (although I'm not saying shading is a good thing.)

As a last resort, there is an ugly workaround that you prepend /app/libs/hzero-starter-mybatis-mapper-0.9.0.RELEASE.jar in the classpath list using Jib's <extraClasspath> configuration, but I think you shouldn't really do this.

Thank you @chanseokoh. This is indeed a problem caused by an irregular operation.

@chanseokoh
Copy link
Member

Setting the milestone just not to forget discussing.

@TadCordle TadCordle modified the milestones: v1.6.0, v1.7.0 Sep 13, 2019
@TadCordle TadCordle modified the milestones: v1.7.0, v1.8.0 Oct 14, 2019
@chanseokoh chanseokoh added the question User inquiries label Oct 18, 2019
@chanseokoh chanseokoh modified the milestones: v1.8.0, v2.0.0 Nov 15, 2019
@loosebazooka loosebazooka modified the milestones: v2.0.0, v2.1.0 Jan 2, 2020
@loosebazooka loosebazooka modified the milestones: v2.1.0, v2.2.0 Feb 13, 2020
@loosebazooka loosebazooka modified the milestones: v2.2.0, v2.3.0 Apr 14, 2020
@loosebazooka loosebazooka modified the milestones: v2.3.0, v2.4.0 May 19, 2020
@chanseokoh chanseokoh modified the milestones: v2.4.0, v2.5.0 Jun 12, 2020
@chanseokoh chanseokoh removed this from the v2.5.0 milestone Aug 7, 2020
@loosebazooka
Copy link
Member

I think we should bump the priority on this. We can try to use the CLASSPATH env var, or maybe just fill the entrypoint with the correct values. We can allow users to preserveClasspathOrder as a config feature.

@chanseokoh
Copy link
Member

chanseokoh commented Sep 11, 2020

Another project setup (originally from https://github.com/rvaidya/jib-conflict-test1) affected by the order of JAR dependencies in the -cp classpath. The project has an issue that the order of loading JAR dependencies affects the application behavior, which is very risky.

  • project setup
├── pom.xml
└── src
    └── main
        └── java
            └── TestClass.java
  • pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>testpackage</groupId>
    <artifactId>test-project</artifactId>
    <version>0.0.1</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.tika</groupId>
            <artifactId>tika-parsers</artifactId>
            <version>1.24</version>
        </dependency>
        <dependency>
            <groupId>org.jscience</groupId>
            <artifactId>jscience</artifactId>
            <version>4.3.1</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>exec-maven-plugin</artifactId>
                <version>3.0.0</version>
                <configuration><mainClass>TestClass</mainClass></configuration>
            </plugin>
            <plugin>
                <groupId>com.google.cloud.tools</groupId>
                <artifactId>jib-maven-plugin</artifactId>
                <version>2.5.2</version>
            </plugin>
        </plugins>
    </build>
</project>
  • src/main/java/TestClass.java
public class TestClass {
    public static void main(String[] args) {
        javax.measure.unit.NonSI.getInstance();
        System.out.println("done");
    }
}

Running the project fails with exec:java:

$ mvn compile exec:java
...
[INFO] --- exec-maven-plugin:3.0.0:java (default-cli) @ test-project ---
[WARNING] 
java.lang.Error: java.lang.NoSuchFieldException: UNIT
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
...
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.0.0:java (default-cli) on project test-project: An exception occured while executing the Java class. java.lang.NoSuchFieldException: UNIT -> [Help 1]

Switching the order of jscience and tika-parsers in pom.xml resolves the issue.

$ mvn compile exec:java
...
[INFO] --- exec-maven-plugin:3.0.0:java (default-cli) @ test-project ---
done
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

Running locally with the -cp execution style (same as Jib) fails with the same error:

$ mvn compile dependency:copy-dependencies
$ java -cp target/classes:target/dependency/* TestClass
Exception in thread "main" java.lang.Error: java.lang.NoSuchFieldException: UNIT
        at javax.measure.unit.Unit.asType(Unit.java:172)
        at javax.measure.unit.NonSI.<clinit>(NonSI.java:667)
        at TestClass.main(TestClass.java:3)
Caused by: java.lang.NoSuchFieldException: UNIT
        at java.lang.Class.getField(Class.java:1703)
        at javax.measure.unit.Unit.asType(Unit.java:170)
        ... 2 more

Interestingly, the two dependencies don't have any common transitive dependencies.

Dependency tree (click to expand):
[INFO] testpackage:test-project:jar:0.0.1                                                                                                                                                                         
[INFO] +- org.jscience:jscience:jar:4.3.1:compile                                                                                                                                                                 
[INFO] |  \- org.javolution:javolution:jar:5.2.3:compile                                                                                                                                                          
[INFO] \- org.apache.tika:tika-parsers:jar:1.24:compile                                                                                                                                                           
[INFO]    +- org.apache.tika:tika-core:jar:1.24:compile                                                                                                                                                           
[INFO]    +- org.glassfish.jaxb:jaxb-runtime:jar:2.3.2:compile                                                                                                                                                    
[INFO]    |  +- jakarta.xml.bind:jakarta.xml.bind-api:jar:2.3.2:compile                                                                                                                                           
[INFO]    |  +- org.glassfish.jaxb:txw2:jar:2.3.2:compile           
[INFO]    |  +- com.sun.istack:istack-commons-runtime:jar:3.0.8:compile        
[INFO]    |  +- org.jvnet.staxex:stax-ex:jar:1.8.1:compile                                               
[INFO]    |  \- com.sun.xml.fastinfoset:FastInfoset:jar:1.2.16:compile 
[INFO]    +- com.sun.activation:jakarta.activation:jar:1.2.1:compile     
[INFO]    +- xerces:xercesImpl:jar:2.12.0:compile                                                        
[INFO]    |  \- xml-apis:xml-apis:jar:1.4.01:compile                                                     
[INFO]    +- org.apache.commons:commons-lang3:jar:3.9:compile         
[INFO]    +- javax.annotation:javax.annotation-api:jar:1.3.2:compile
[INFO]    +- org.gagravarr:vorbis-java-tika:jar:0.8:compile        
[INFO]    +- org.tallison:jmatio:jar:1.5:compile                                                         
[INFO]    +- org.apache.james:apache-mime4j-core:jar:0.8.3:compile      
[INFO]    +- org.apache.james:apache-mime4j-dom:jar:0.8.3:compile                                        
[INFO]    +- org.apache.commons:commons-compress:jar:1.20:compile    
[INFO]    +- org.tukaani:xz:jar:1.8:compile                                                              
[INFO]    +- com.epam:parso:jar:2.0.11:compile                                                           
[INFO]    +- org.brotli:dec:jar:0.1.2:compile                                                                                                                                                            [INFO]    +- commons-codec:commons-codec:jar:1.13:compile         
[INFO]    +- org.apache.pdfbox:pdfbox:jar:2.0.19:compile   
[INFO]    |  \- org.apache.pdfbox:fontbox:jar:2.0.19:compile
[INFO]    +- org.apache.pdfbox:pdfbox-tools:jar:2.0.19:compile                                           
[INFO]    +- org.apache.pdfbox:preflight:jar:2.0.19:compile
[INFO]    |  \- org.apache.pdfbox:xmpbox:jar:2.0.19:compile
[INFO]    |     \- commons-logging:commons-logging:jar:1.2:compile               
[INFO]    +- org.apache.pdfbox:jempbox:jar:1.8.16:compile                                              
[INFO]    +- org.bouncycastle:bcmail-jdk15on:jar:1.64:compile                                            
[INFO]    |  \- org.bouncycastle:bcpkix-jdk15on:jar:1.64:compile    
[INFO]    +- org.bouncycastle:bcprov-jdk15on:jar:1.64:compile                  
[INFO]    +- org.apache.poi:poi:jar:4.1.2:compile                                                        
[INFO]    |  +- org.apache.commons:commons-collections4:jar:4.4:compile
[INFO]    |  +- org.apache.commons:commons-math3:jar:3.6.1:compile       
[INFO]    |  \- com.zaxxer:SparseBitSet:jar:1.2:compile                
[INFO]    +- org.apache.poi:poi-scratchpad:jar:4.1.2:compile               
[INFO]    +- org.apache.poi:poi-ooxml:jar:4.1.2:compile               
[INFO]    |  +- org.apache.poi:poi-ooxml-schemas:jar:4.1.2:compile  
[INFO]    |  |  \- org.apache.xmlbeans:xmlbeans:jar:3.1.0:compile  
[INFO]    |  \- com.github.virtuald:curvesapi:jar:1.06:compile
[INFO]    +- com.healthmarketscience.jackcess:jackcess:jar:3.0.1:compile
[INFO]    +- com.healthmarketscience.jackcess:jackcess-encrypt:jar:3.0.0:compile                         
[INFO]    +- org.ccil.cowan.tagsoup:tagsoup:jar:1.2.1:compile        
[INFO]    +- org.ow2.asm:asm:jar:7.3.1:compile                                                           
[INFO]    +- com.googlecode.mp4parser:isoparser:jar:1.1.22:compile
[INFO]    +- org.tallison:metadata-extractor:jar:2.13.0:compile                                          
[INFO]    |  \- org.tallison.xmp:xmpcore-shaded:jar:6.1.10:compile
[INFO]    |     \- com.adobe.xmp:xmpcore:jar:6.1.10:compile         
[INFO]    +- de.l3s.boilerpipe:boilerpipe:jar:1.1.0:compile                                              
[INFO]    +- com.rometools:rome:jar:1.12.2:compile                                                       
[INFO]    |  \- com.rometools:rome-utils:jar:1.12.2:compile                                              
[INFO]    +- org.gagravarr:vorbis-java-core:jar:0.8:compile 
[INFO]    +- com.googlecode.juniversalchardet:juniversalchardet:jar:1.0.3:compile
[INFO]    +- org.codelibs:jhighlight:jar:1.0.3:compile                                                 
[INFO]    +- com.pff:java-libpst:jar:0.9.3:compile                                                       
[INFO]    +- com.github.junrar:junrar:jar:4.0.0:compile             
[INFO]    +- org.apache.cxf:cxf-rt-rs-client:jar:3.3.5:compile                 
[INFO]    |  +- org.apache.cxf:cxf-rt-transports-http:jar:3.3.5:compile
[INFO]    |  +- org.apache.cxf:cxf-core:jar:3.3.5:compile
[INFO]    |  |  +- com.fasterxml.woodstox:woodstox-core:jar:5.0.3:compile                                
[INFO]    |  |  |  \- org.codehaus.woodstox:stax2-api:jar:3.1.4:compile                                  
[INFO]    |  |  \- org.apache.ws.xmlschema:xmlschema-core:jar:2.2.5:compile                              
[INFO]    |  \- org.apache.cxf:cxf-rt-frontend-jaxrs:jar:3.3.5:compile                                   
[INFO]    |     +- jakarta.ws.rs:jakarta.ws.rs-api:jar:2.1.5:compile
[INFO]    |     \- org.apache.cxf:cxf-rt-security:jar:3.3.5:compile                                      
[INFO]    +- org.apache.commons:commons-exec:jar:1.3:compile                                             
[INFO]    +- org.apache.opennlp:opennlp-tools:jar:1.9.2:compile
[INFO]    +- commons-io:commons-io:jar:2.6:compile                                                       
[INFO]    +- com.googlecode.json-simple:json-simple:jar:1.1.1:compile
[INFO]    +- com.github.openjson:openjson:jar:1.0.11:compile
[INFO]    +- com.google.code.gson:gson:jar:2.8.6:compile                                                 
[INFO]    +- org.slf4j:slf4j-api:jar:1.7.28:compile
[INFO]    +- org.slf4j:jul-to-slf4j:jar:1.7.28:compile
[INFO]    +- org.slf4j:jcl-over-slf4j:jar:1.7.28:compile
[INFO]    +- edu.ucar:netcdf4:jar:4.5.5:compile
[INFO]    |  \- net.jcip:jcip-annotations:jar:1.0:compile
[INFO]    +- org.jdom:jdom2:jar:2.0.6:compile
[INFO]    +- com.google.guava:guava:jar:28.2-jre:compile
[INFO]    |  +- com.google.guava:failureaccess:jar:1.0.1:compile
[INFO]    |  +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile
[INFO]    |  +- com.google.code.findbugs:jsr305:jar:3.0.2:compile
[INFO]    |  +- org.checkerframework:checker-qual:jar:2.10.0:compile
[INFO]    |  +- com.google.errorprone:error_prone_annotations:jar:2.3.4:compile
[INFO]    |  \- com.google.j2objc:j2objc-annotations:jar:1.3:compile
[INFO]    +- edu.ucar:grib:jar:4.5.5:compile
[INFO]    |  \- org.itadaki:bzip2:jar:0.9.1:compile
[INFO]    +- com.beust:jcommander:jar:1.78:compile
[INFO]    +- net.java.dev.jna:jna:jar:5.5.0:compile
[INFO]    +- org.jsoup:jsoup:jar:1.13.1:compile
[INFO]    +- com.google.protobuf:protobuf-java:jar:3.11.4:compile
[INFO]    +- edu.ucar:cdm:jar:4.5.5:compile
[INFO]    |  +- edu.ucar:udunits:jar:4.5.5:compile
[INFO]    |  +- joda-time:joda-time:jar:2.2:compile
[INFO]    |  \- net.sf.ehcache:ehcache-core:jar:2.6.2:compile
[INFO]    +- org.quartz-scheduler:quartz:jar:2.3.2:compile
[INFO]    |  \- com.zaxxer:HikariCP-java7:jar:2.4.13:compile
[INFO]    +- com.mchange:c3p0:jar:0.9.5.5:compile
[INFO]    |  \- com.mchange:mchange-commons-java:jar:0.2.19:compile
[INFO]    +- edu.ucar:httpservices:jar:4.5.5:compile
[INFO]    +- org.apache.httpcomponents:httpclient:jar:4.5.12:compile
[INFO]    |  \- org.apache.httpcomponents:httpcore:jar:4.4.13:compile
[INFO]    +- org.apache.httpcomponents:httpmime:jar:4.5.12:compile
[INFO]    +- org.apache.commons:commons-csv:jar:1.8:compile
[INFO]    +- org.apache.sis.core:sis-utility:jar:1.0:compile
[INFO]    |  \- javax.measure:unit-api:jar:1.0:compile
[INFO]    +- org.apache.sis.storage:sis-netcdf:jar:1.0:compile
[INFO]    |  +- org.apache.sis.storage:sis-storage:jar:1.0:compile
[INFO]    |  |  \- org.apache.sis.core:sis-feature:jar:1.0:compile
[INFO]    |  \- org.apache.sis.core:sis-referencing:jar:1.0:compile
[INFO]    +- org.apache.sis.core:sis-metadata:jar:1.0:compile
[INFO]    +- org.opengis:geoapi:jar:3.0.1:compile
[INFO]    +- edu.usc.ir:sentiment-analysis-parser:jar:0.1:compile
[INFO]    +- com.fasterxml.jackson.core:jackson-core:jar:2.10.3:compile
[INFO]    +- com.fasterxml.jackson.core:jackson-databind:jar:2.10.3:compile
[INFO]    +- com.fasterxml.jackson.core:jackson-annotations:jar:2.10.3:compile
[INFO]    +- org.apache.pdfbox:jbig2-imageio:jar:3.0.3:compile
[INFO]    \- com.github.jai-imageio:jai-imageio-core:jar:1.4.0:compile

The issue is having different versions of the same classes in different JARs. For example, javax.measure.qunaity.Volume is in both jscience-4.3.1.jar and unit-api-1.0.jar.

  • jscience-4.3.1.jar
public interface Volume extends Quantity {
    public final static Unit<Volume> UNIT = SI.CUBIC_METRE;
}
  • unit-api-1.0.jar
public interface Volume extends Quantity<Volume> {}

Therefore, the application behavior becomes non-deterministic and depends on whether Quantity is first loaded from unit-api-1.0.jar or jscience-4.3.1.jar. A proper resolution is to eliminate the non-deterministic behavior by, for example,

    <dependencies>
        <dependency>
            <groupId>org.apache.tika</groupId>
            <artifactId>tika-parsers</artifactId>
            <version>1.24</version>
            <exclusions>
                <exclusion>
                    <groupId>javax.measure</groupId>
                    <artifactId>unit-api</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.jscience</groupId>
            <artifactId>jscience</artifactId>
            <version>4.3.1</version>
        </dependency>
    </dependencies>

@chanseokoh
Copy link
Member

@xiaopihai7256 @luxingxiao we've released Jib 2.7.0 which added a new configuration option (jib.container.expandClasspathDependencies (Gradle) / <container><expandClasspathDependencies> (Maven)) that enables expanding classpath dependencies in the default java command for an image ENTRYPOINT. Turning on the option (off by default) will enumerate all the dependencies, which will match the dependency loading order in Maven or Gradle builds. For example, the ENTRYPOINT becomes

java ... -cp /app/resources:/app/classes:/app/libs/spring-boot-starter-web-2.0.3.RELEASE.jar:/app/libs/shared-library-0.1.0.jar:/app/libs/spring-boot-starter-json-2.0.3.RELEASE.jar:... com.example.Main

instead of the default

java ... -cp /app/resources:/app/classes:/app/libs/* com.example.Main

Expanding the dependency list can be useful in AppCDS too.

Note that an expanded dependency list can become very long in practice, and we are not sure if there may be a potential issue due to a long command line ("argument list too long" or "command line is too long").

As with other Jib configurations, this option can also be set through the system property (-Djib.container.expandClasspathDependencies=true|false).

@chanseokoh
Copy link
Member

Will leave this issue open to add better support for Java 9+.

#2866 (comment)

But ideally, for Java 9+, I think we should really use the @-argument file feature; we can always preserve the classpath order while never hitting the system argument length limit. This is going to be a lot bigger change.

In sum, perhaps

  • Java 9+: always use a @-argument file. (expandClasspathDependencies is irrelevant.)
  • Java <=8: expand or not based on expandClasspathDependencies. Not decisive on if it should be true by default.

@chanseokoh
Copy link
Member

chanseokoh commented Jun 9, 2021

@xiaopihai7256 @luxingxiao Jib 3.1.1 is released, which creates two JVM argument files inside an image. One of them is the Java runtime classpath where all the dependencies are explicitly enumerated, which enables Jib to preseve the depending loading order by using this file for Java 9+. (For Java 8, you have to continue to set expandClasspathDependencies to true.)

For those interested, see here for more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment