[BUG] macOS version of JDK21.0.1+12 not able to be install with java global tools

[peterzhuamazon]

Error: https://build.ci.opensearch.org/blue/organizations/jenkins/publish-opensearch-min-snapshots/detail/publish-opensearch-min-snapshots/432/pipeline/

See here:

• api.adoptium.net broke with two list for 21.0.1+12 adoptium/api.adoptium.net#829

This should be change on the build side to use other means to install jdk on macOS.
Then sunset the jdk global tools on opensearch-ci.
So we can avoid relying on the global tools and adoptium api again just to install jdk.

Thanks.

[peterzhuamazon]

Once opensearch-project/opensearch-ci#376 is merged,

• Rebuild packer: https://build.ci.opensearch.org/job/packer-build/
• Make AMI public
• Update new ami id in opensearch-ci
• Deploy Jenkins
• Update min snapshot build jenkinsfile for opensearch so that macOS build can use jenv to dynamically switch jdk version based on values from dockerAgent lib on jenkins from input manifest.
• Remove/deprecate jenkins global java tool in opensearch-ci and deploy jenkins again (baseJenkins.yml) (https://github.com/opensearch-project/opensearch-ci/blob/main/resources/baseJenkins.yaml#L165-L202).

Thanks.

[jordarlu]

I am closing this issue as the implementaion has been made, and test on CI looks well with the new AMI https://build.ci.opensearch.org/job/publish-opensearch-min-snapshots/466/
We did not move forward to remove Jenkins global tool for Java at the moment with the consideration that there might be other jobs that needs it.

[jordarlu]

It appers the jenv did not set the JAVA_HOME and java version properly as the min snapshot fail on build 1.3.15 which has the Java 11 in manifest - https://build.ci.opensearch.org/blue/organizations/jenkins/publish-opensearch-min-snapshots/detail/publish-opensearch-min-snapshots/514/pipeline/62

* What went wrong:

Could not compile settings file '/private/var/folders/2g/jjj_bgln5qn8c6t1szz_fmmc0000gn/T/tmpa4byxpva/OpenSearch/settings.gradle'.

> startup failed:

  General error during semantic analysis: Unsupported class file major version 65

  

  java.lang.IllegalArgumentException: Unsupported class file major version 65

  	at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:196)

  	at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:177)

  	at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:163)

major version 65 is Java 21 (https://stackoverflow.com/questions/9170832/list-of-java-class-file-format-major-version-numbers) , which means it takes the latest installed JDK as the default and the jenv does not setup the needed JDK taken from the manifest properly

[peterzhuamazon]

The issue is due to jenv setup is using local instead of global.

opensearch-build/jenkins/opensearch/publish-min-snapshots.jenkinsfile

Line 129 in 11f9ea4

sh("jenv local ${dockerAgent.javaVersion}")

In Jenkins, each sh block is a new shell session and using local will only take effect on that specific session, and the next session of the build will still default back to the original jdk21.

Will PR.

[jordarlu]

Thanks, Peter , apart from that, in order to get JAVA_HOME set properly, we also need to enable plugin export as mentioned on this page ; this was another thing that I missed in packer process earlier ...

To make sure JAVA_HOME is set, make sure to enable the export plugin:

jenv enable-plugin export
exec $SHELL -l

[peterzhuamazon]

Thanks, Peter , apart from that, in order to get JAVA_HOME set properly, we also need to enable plugin export as mentioned on this page ; this was another thing that I missed in packer process earlier ...

To make sure JAVA_HOME is set, make sure to enable the export plugin:
jenv enable-plugin export
exec $SHELL -l

Got it, thanks!

[peterzhuamazon]

Correction:

We'll now set a jenv local VERSION local Java version for the current working directory. This will create a .java-version file we can check into Git for our projects, and jenv will load it correctly when a shell is started from this directory.

We will now only use jenv global to avoid confusion.

[peterzhuamazon]

There are a lot of issues related to the jenv setups, as it is actually eval a function and not switching versions properly without the export plugin.

Have a long session with @jordarlu and eventually I use update-alternatives to solve the issues:

1. Install dpkg with brew to get update-alternatives available on macOS.
2. Install intel macOS (https://aws.amazon.com/ec2/instance-types/mac/, mac1.metal is Intel Core i7 based CPU) temurin JDK tarballs in /opt/java, using the same naming convention as in linux hosts.
3. Setup update-alternatives with JDK11 having highest priority 100 and others with 1.

$ update-alternatives --install /usr/local/bin/java java /opt/java/openjdk-8/Contents/Home/bin/java 1
$ update-alternatives --install /usr/local/bin/java java /opt/java/openjdk-14/Contents/Home/bin/java 1
$ update-alternatives --install /usr/local/bin/java java /opt/java/openjdk-17/Contents/Home/bin/java 1
$ update-alternatives --install /usr/local/bin/java java /opt/java/openjdk-19/Contents/Home/bin/java 1
$ update-alternatives --install /usr/local/bin/java java /opt/java/openjdk-20/Contents/Home/bin/java 1
$ update-alternatives --install /usr/local/bin/java java /opt/java/openjdk-21/Contents/Home/bin/java 1
$ update-alternatives --install /usr/local/bin/java java /opt/java/openjdk-11/Contents/Home/bin/java 100

4. Reset jdk to 11 just be sure:

$ update-alternatives --set java `update-alternatives --list java | grep openjdk-11`

5. Update jenkinsfile to use above commands to switch versions:

sh("update-alternatives --set java `update-alternatives --list java | grep openjdk-${env.javaVersionNumber}`")

6. Update opensearch-ci AMI scripts on macOS, generate new AMI, test again.

Thanks.