-
Notifications
You must be signed in to change notification settings - Fork 118
1.3. Managing Dependencies
Prefer the use of plugin-management and dependency-management over plugins and dependencies as much as possible. When declaring dependencies and plugins, only declare groupId and artifactId and let the other elements be inferred from the management unless an override is explicitly required. In projects with multiple modules declare the management elements in the root module to provide better visibility and facilitate maintaining the project in the future.
When adding a plugin or a dependency to your build, make sure you really need to add it by checking if it is already defined in the management sections of the pentaho parent poms.
Always use webjar dependencies and favor npm ones over classic and bower. Webjar dependencies should be defined with scope runtime.
Third party dependencies may specify their versions directly. Example:
<dependency>
<groupId>org.eclipse</groupId>
<artifactId>jface</artifactId>
<version>3.3.0-I20070606-0010</version>
</dependency>
Alternatively, they may use a property of the form ${artifactId}.version. Example:
<properties>
<jetty-server.version>8.1.15.v20140411</jetty-server.version>
</properties>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-server</artifactId>
<version>${jetty-server.version}</version>
<scope>test</scope>
</dependency>
Third party dependencies with associated dependencies of the same version should use a property of the form ${shared-designator}.version. Example:
<properties>
<logback.version>1.1.2</logback.version>
</properties>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-core</artifactId>
<version>${logback.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>${logback.version}</version>
<scope>test</scope>
</dependency>
Pentaho modules that are declared as dependencies in other projects must use the property keys found in the global version.properties. This does NOT mean you will need a new global version property for every module in a multi-module project. In most cases, the version of the root pom of the project needs to be controlled in this fashion and the rest of the project tree will inherit from that. Adding a new module to an existing project will almost never require global version property control.
This file is kept in the jenkins-pipelines repository, see the branch for the version your mavenizing for.
WARNING! DO NOT use ${project.version} to specify the version of a dependency that is not a module of the current project. Use the appropriate version property like suggested above.
Example:
<properties>
<pdi.version>8.1-SNAPSHOT</pdi.version>
</properties>
<dependency>
<groupId>pentaho-kettle</groupId>
<artifactId>kettle-engine</artifactId>
<version>${pdi.version}</version>
<scope>compile</scope>
</dependency>
WARNING! Please make sure the properties used are a part of the version.properties file and if not contact the [email protected] about it.
By default, Maven will include all of the dependencies of any dependency you declare. It will continue on grabbing all the dependencies of dependencies until you have a nice big stack of jars.
- The contributors and developers of third party libraries know their code best. They also typically know what versions of dependencies work best with their code and help to prevent regressions. Regressions that we might end up owning ourselves if we specify a version for a dependency that those developers already know doesn't work.
- They can be a convenient and quick way to get an app compiling, testing, and running.
- Third party developers don't always keep abreast of important updates in their dependencies, including security concerns.
- Dependencies in applications, third-party or in-house, can change. Relying upon the dependencies of your dependency to compile, test, and run can create potential volatility.
- Probably the most damning of all ... there may be dozens of dependencies of your dependency that you don't even require at runtime, let alone compile time. This can greatly increase the size of your deliverable and potentially introduce unnecessary classloading conflicts.
When in doubt as to whether or not to explicitly declare a dependency in your project POM, follow these two simple rules:
-
#1) If it has an import statement, declare it!
- If a dependency is required to compile, specify it explicitly. Do not rely upon transitive inclusion.
-
#2) If it is not required at runtime, exclude it!
- This is where the challenging work is, and where the greatest reward is. Our artifacts are HUGE. Needless to say, every successful exclusion of an unused dependency is a BIG WIN! Let's be winners!
Assuming that you have met rule #1 above, then start by gathering a text representation of your dependency graph using the Dependency Plugin's "tree" or "analyze" goals. It might be a good idea to send it to a file for efficient review.
$ mvn dependency:analyze > dependencies.txt
The portion that you will want to focus on is the "Unused declared dependencies found" section of the analyze-report output.
$ mvn dependency:tree > dependencies.txt
The "usused declared" dependencies shown in the dependency:analyze report output can be quite helpful in identifying possible target exclusions in your dependency tree. Its a good idea to put your dependency:analysis and dependency:tree, for your project module, side-by-side and look for opportunities to pair down. Keep in mind though, that it is impossible for this type of analysis to infer runtime implementation binding or runtime dependency injection. So, be careful about excluding things like webjars and artifacts that are known to be API implementations, like XML parsers.
This strategy is a quite a bit more intensive than the relatively straightforward Dependency Analysis Report Strategy above, and should probably be considered a supplement to it.
-
Sep #1) Ignore all test scoped dependencies
- The test scope is ignored by projects that may declare this project as a dependency. It does not become a transitive for them. Therefore, you are having no affect on the size of any assembly that may attempt to gather all runtime scoped dependencies, which will include compile scoped as well.
-
Step #2) Decide if you can ignore provided scoped dependencies
- Like test, provided scope will also not become a transitive. It is used in circumstances where you need it to compile and test, but it will be provided by downstream projects or deployment classloaders. There are times, however, when the usage of the provided scope, in conjunction with the Maven Assembly Plugin, can be useful for assembly. In those cases, you should properly manage the exclusion of transitives as you would for compile and runtime scope.
-
Step #3) Iteratively isolate compile scoped dependencies for declaration
- Ultimately, our goal here is to abide by rule #1. Start by providing our artifact with the universal wildcard exclusion:
<dependency>
<groupId>pentaho-kettle</groupId>
<artifactId>kettle-core</artifactId>
<version>${pdi.version}</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<groupId>*</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
Then, attempt a simple compile
$ mvn compile
As you encounter build failures, define the exclusion as a new declared compile scope dependency. If the newly declared dependency also had next level transitives, then also define a universal wildcard exclusion for it as well. You will then regenerate your dependency tree for analysis. Attempt another compile, and iterate this process until you get a successful compile.
If you have doubt as to the jar that any particular third party class may have come from, you may find this resource helpful: http://www.findjar.com
-
Step #4) Iteratively un-exclude runtime scoped transitives a level at a time
- We are not after a universal wildcard exclusion when it comes to runtime dependencies. Some classnotfound exceptions might get caught running unit tests. Others might get caught during integration testing. Others still might not be discovered until QA. Keep in mind, it is possible that during STEP #3 that you've excluded a runtime dependency that was previously being brought in as a transitive compile scoped dependency. If you add a new declared dependency, you should regenerate your dependency tree for analysis.
The approach to this step is a bit different. Let's stick with our example artifact and pretend it is now runtime scoped. You'll begin by specifically excluding all 1st level transitives.
<dependency>
<groupId>pentaho-kettle</groupId>
<artifactId>kettle-core</artifactId>
<version>${pdi.version}</version>
<scope>runtime</scope>
<exclusions>
<exclusion>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils</artifactId>
</exclusion>
<exclusion>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
</exclusion>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
<exclusion>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
</exclusion>
<exclusion>
<groupId>commons-httpclient</groupId>
<artifactId>commons-httpclient</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.commons</groupId>
<artifactId>commons-vfs2</artifactId>
</exclusion>
...
</exclusions>
</dependency>
As you trip on classnotfound exception here, remove the exclusion for that artifact and replace it with any next level transitive exclusions as necessary. What you should end up with is a nice set of runtime exclusions for things that are never used and runtime transitives that are regression tested by the experts.
NOTE! Please make every attempt to avoid the blanket usage of universal wildcard exclusions with runtime scoped dependencies. Directly using a dependency in code is one thing, needlessly taking ownership of a third party transitive at runtime, bypassing expert advice, is something completely different. Only do it, if there is a REALLY good reason, like security concerns.