Dependency management for Derby is incomplete

[johnchurchill]

I'm on Java 17 and latest spring boot. spring-boot-project/spring-boot-dependencies/build.gradle has been giving me grief because of hard-coded overrides like this:

	library("Derby", "10.14.2.0") {
		prohibit("[10.15,)") {
			because "it requires Java 9"
		}
		group("org.apache.derby") {
			modules = [
				"derby",
				"derbyclient"
			]
		}
	}

Derby is 2 versions beyond the one in the bom. Shouldn't BomExtension.prohibit() at least check to see if I'm even targeting below Java 9 before overriding the version? prohibit also doesn't fail the build, it just limits the version number. In the case of Derby it still allows other new dependency jars that came after 10.14 (e.g. derbyshared), resulting in a mixed version build. Personally, I think Spring Boot should just allow the developer to use a later Derby version and they will realize their mistake when they run into problems with Java 8. This is a well-meaning guard rail that doesn't help. It's also an easy fix to remove them all.

[wilkinsona]

These guard rails are purely for our semi-automated dependency upgrade process. They have no affect at all on an application's dependency management when using either Maven or Gradle to build the application.

With a Java 8 baseline in 2.x, we've made the decision that, by default, all of Boot's dependencies should be compatible with Java 8. These guard rails help us with this goal as they stop us from accidentally updating spring-boot-dependencies to include a managed version that isn't compatible with Java 8. If you're using a more modern JVM, you're free to override Boot's dependency management to meet your needs.

[johnchurchill]

I get what you are saying, but to say "they have no effect at all" isn't true if using the dependency-management plugin. Try using these lines in a spring boot web application:

derbyVersion = "10.15.2.0"
apply plugin: 'io.spring.dependency-management'
implementation "org.apache.derby:derby:${derbyVersion}"
implementation "org.apache.derby:derbynet:${derbyVersion}"
implementation "org.apache.derby:derbyclient:${derbyVersion}"
implementation "org.apache.derby:derbyshared:${derbyVersion}"

What you get in the war:

derby-10.14.2.0.jar
derbyclient-10.14.2.0.jar
derbynet-10.15.2.0.jar
derbyshared-10.15.2.0.jar
derbytools-10.15.2.0.jar

There is no warning in the build log about the configured version being downgraded, the result is mismatched versions in the war. This is worse than failing the build. Supposing there is a CVE on Derby 14, so the junior dev upgrades the version in gradle and all seems to work. The application is still vulnerable.

[wilkinsona]

That problem isn't caused by a version being prohibited but by the dependency management for Derby not covering all of its modules. Removing this won't fix it:

prohibit("[10.15,)") {
    because "it requires Java 9"
}

Adding entries to modules will.

[wilkinsona]

One further thought, the behavior that you've described shouldn't happen. The dependency management plugin should honour the version in the dependency declaration, overriding the dependency management.

Here's a complete example:

plugins {
	id 'org.springframework.boot' version '2.7.1'
	id 'io.spring.dependency-management' version '1.0.11.RELEASE'
	id 'java'
}

group = 'com.example'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '17'

repositories {
	mavenCentral()
}

dependencies {
	def derbyVersion = "10.15.2.0"
	implementation "org.apache.derby:derby:${derbyVersion}"
	implementation "org.apache.derby:derbynet:${derbyVersion}"
	implementation "org.apache.derby:derbyclient:${derbyVersion}"
	implementation "org.apache.derby:derbyshared:${derbyVersion}"
}

The Derby dependencies are all 10.15.2.0:

./gradlew dependencies --configuration runtimeClasspath

> Task :dependencies

------------------------------------------------------------
Root project 'derby-version'
------------------------------------------------------------

runtimeClasspath - Runtime classpath of source set 'main'.
+--- org.apache.derby:derby:10.15.2.0
|    \--- org.apache.derby:derbyshared:10.15.2.0
+--- org.apache.derby:derbynet:10.15.2.0
|    +--- org.apache.derby:derby:10.15.2.0 (*)
|    +--- org.apache.derby:derbyshared:10.15.2.0
|    \--- org.apache.derby:derbytools:10.15.2.0
|         \--- org.apache.derby:derbyshared:10.15.2.0
+--- org.apache.derby:derbyclient:10.15.2.0
|    \--- org.apache.derby:derbyshared:10.15.2.0
\--- org.apache.derby:derbyshared:10.15.2.0

(*) - dependencies omitted (listed previously)

A web-based, searchable dependency report is available by adding the --scan option.

BUILD SUCCESSFUL in 806ms
1 actionable task: 1 executed

I'd like to understand exactly what's happening in your build. To that end, can you please provide a similarly minimal script that reproduces the behavior you have described?

[johnchurchill]

Hey Andy, I appreciate your time to respond. I tried your example build.gradle and I get your output (only 10.15), so there must be some other uncommon configuration coming into play in my build.gradle. Here's what I get on my project:

|    |    +--- org.apache.derby:derby:10.15.2.0 -> 10.14.2.0
|    |    +--- org.apache.derby:derbynet:10.15.2.0
|    |    |    +--- org.apache.derby:derby:10.15.2.0 -> 10.14.2.0
|    |    |    +--- org.apache.derby:derbyshared:10.15.2.0
|    |    |    \--- org.apache.derby:derbytools:10.15.2.0
|    |    |         \--- org.apache.derby:derbyshared:10.15.2.0
|    |    +--- org.apache.derby:derbyclient:10.15.2.0 -> 10.14.2.0
|    |    +--- org.apache.derby:derbyshared:10.15.2.0

I'll need some time to add things to your build.gradle until it does the same. I'll report back when I can.

[johnchurchill]

The downgrade happens if I put the derby dependency in a subproject. It doesn't matter if I have the Derby version in a variable at the top level build.gradle or not.

build.gradle:

plugins {
	id 'org.springframework.boot' version '2.7.1'
	id 'io.spring.dependency-management' version '1.0.11.RELEASE'
	id 'java'
}

group = 'com.example'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '17'

allprojects {
  repositories {
  	mavenCentral()
  }
}

dependencies {
  implementation project(":database")
}

settings.gradle:

include ':database'

database/build.gradle:

apply plugin: "java-library"
dependencies {
	implementation "org.apache.derby:derby:10.15.2.0"
}

dependency check:

 gradle dependencies --configuration runtimeClasspath

> Task :dependencies

------------------------------------------------------------
Root project 'gradletest'
------------------------------------------------------------

runtimeClasspath - Runtime classpath of source set 'main'.
\--- project :database
     \--- org.apache.derby:derby:10.15.2.0 -> 10.14.2.0

[wilkinsona]

That's due to how the dependency management plugin works. Spring Boot's dependency management is applied to the root project so it's overriding the transitive dependencies of your database project. To avoid problems like this, I'd encourage you to manage versions consistently across every project in your build. This advice holds whether you're using the dependency management plugin, Gradle's platform support, or your own dependency constraints.

Let's use this issue to fill the gaps in our dependency management for Derby.