Use Gradle Toolchains for JVM projects to test on unsupported JDK versions

[sbrannen]

Gradle 6.7 will introduce built-in support for Toolchains for JVM projects.

Once we upgrade to Gradle 6.7, we will should hopefully be able to replace the functionality in custom-java-home.gradle with the new built-in support.

Note from the Gradle team:

Toolchains and *Compatibility: the moment you use toolchain at the project level, sourceCompatibility and targetCompatibility from the project have a value derived from the toolchain. What you cannot do is overwrite that value. You can still overwrite it at the task level though.

[sbrannen]

Depends on #25918.

[sbrannen]

Beginning with Gradle 6.8, it will also be possible to track the vendor and implementation of the JVM for caching purposes. See gradle/gradle#9102 (comment) for details.

[bclozel]

Here's a proposal for introducing Gradle Toolchains in our build.

This changes two things:

• the Concourse build now relies on a single CI image shipping all JDKs we want to test
• we can now use the -PmainToolchain and -PtestToolchain project parameters to configure what to use

Gradle JVM Toolchain

This proposal is configuring Java, Kotlin and Groovy compilation with the following behavior:

Gradle Command	Main Sources Compiled	Tests Compiled + Run
./gradlew check	JDK8	JDK8
./gradlew check -PmainToolChain=11	JDK11	JDK11
./gradlew check -PmainToolChain=8 -PtestToolchain=15	JDK8	JDK15

You can use that locally. If Gradle detects the required JDKs it will use them, otherwise Gradle will download the AdoptOpenJDK one automcatically. You can check which JDKs are detected on your host with

$ ./gradlew -q javaToolchains

This is rather flexible and allows to:

1. Compile our artifacts as usual and test them using other JDK versions (basically, how developer experience Spring)
2. Compile and test our artifacts with recent JDK version to check for compatibility issues

Concourse build

The Concourse build now produces a single CI docker image for the entire pipeline; this image contains all the JDKs we need so far: JDK8, JDK11, JDK15. It is easy to add/remove JDKs from our setup. All JDKs locations are configured as environment variables, which makes them available to Gradle.

The pipeline now has:

• a main build, which does everything with JDK8
• a JDK11 build, which compiles main sources with JDK8 and compiles+run tests using JDK11
• a JDK15 build, which compiles main sources with JDK8 and compiles+run tests using JDK15

We can of course discuss options here. Let me know what you think!