Table driven versioning

[suzuki-shunsuke]

Discussed in https://github.com/orgs/aquaproj/discussions/1481

^{Originally posted by iilyak December 27, 2022}
I have a tricky problem with openjdk. I would like to fully replace asdf with aqua. Because asdf doesn't support M1 based mac and they do not have plans to do it. The asdf plugin for java uses a table driven approach (see here).

For example the entries for openjdk-18.2.0.1 look like:

version	file name	url	hash
openjdk-18.0.2.1	openjdk-18.0.2.1_linux-x64_bin.tar.gz	https://download.java.net/java/GA/jdk18.0.2.1/db379da656dc47308e138f21b33976fa/1/GPL/openjdk-18.0.2.1_linux-x64_bin.tar.gz	3bfdb59fc38884672677cebca9a216902d87fe867563182ae8bc3373a65a2ebd
openjdk-18.0.2.1	openjdk-18.0.2.1_macos-x64_bin.tar.gz	https://download.java.net/java/GA/jdk18.0.2.1/db379da656dc47308e138f21b33976fa/1/GPL/openjdk-18.0.2.1_macos-x64_bin.tar.gz	604ba4b3ccb594973a3a73779a367363c53dd91e5a9de743f4fbfae89798f93a
openjdk-18.0.2.1	openjdk-18.0.2.1_macos-aarch64_bin.tar.gz	https://download.java.net/java/GA/jdk18.0.2.1/db379da656dc47308e138f21b33976fa/1/GPL/openjdk-18.0.2.1_macos-aarch64_bin.tar.gz	c05aec589f55517b8bedd01463deeba80f666da3fb193be024490c9d293097a8
openjdk-18.0.2.1	openjdk-18.0.2.1_linux-aarch64_bin.tar.gz	https://download.java.net/java/GA/jdk18.0.2.1/db379da656dc47308e138f21b33976fa/1/GPL/openjdk-18.0.2.1_linux-aarch64_bin.tar.gz	79900237a5912045f8c9f1065b5204a474803cbbb4d075ab9620650fb75dfc1b

The complications are:

1. The url includes the hash of the commit
2. How to match version with integrity hash in aqua-checksums.json
3. How to specify available versions in registry.yaml

How would I define a registry entry for such complicated case?

[suzuki-shunsuke]

⚠️ I'm not familiar with Java and OpenJDK.

You can install java with the following Registry.

packages:
  - type: http
    repo_owner: openjdk
    repo_name: jdk
    description: JDK main-line development https://openjdk.org/projects/jdk
    replacements:
      amd64: x64
      arm64: aarch64
      darwin: macos
    files:
      - name: java
        src: jdk-{{.Version}}.jdk/Contents/Home/bin/java
    version_constraint: "false"
    version_overrides:
      - version_constraint: Version == "18.0.2.1" 
        url: https://download.java.net/java/GA/jdk{{.Version}}/db379da656dc47308e138f21b33976fa/1/GPL/openjdk-{{.Version}}_{{.OS}}-{{.Arch}}_bin.tar.gz

Then you have to add a version_constraint per version.

Generate Registry by programing language (Python, Ruby, Go, etc) or data templating language (Jsonnet, CUE, etc)

You can also generate Registry by programing language (Python, Ruby, Go, etc) or data templating language (Jsonnet, CUE, etc).
The good point is that you can separate the list of versions and commands from Registry setting.
But you may feel it is a bit complicated.

⚠️ Note that aqua doesn't support generating Registry dynamically. You have to generate Registry in advance.

e.g. Generate Registry from Jsonnet

registry.jsonnet

local versions = [
  {
    version: "18.0.2.1",
    commitHash: "db379da656dc47308e138f21b33976fa",
  },
];

local commands = [
  "jar",
  "jarsigner",
  "java",
  "javac",
  "javadoc",
  "javap",
  "jcmd",
  "jconsole",
  "jdb",
  "jdeprscan",
  "jdeps",
  "jfr",
  "jhsdb",
  "jimage",
  "jinfo",
  "jlink",
  "jmap",
  "jmod",
  "jpackage",
  "jps",
  "jrunscript",
  "jshell",
  "jstack",
  "jstat",
  "jstatd",
  "jwebserver",
  "keytool",
  "rmiregistry",
  "serialver",
];

{
  "packages": [
    {
      type: "http",
      repo_owner: "openjdk",
      repo_name: "jdk",
      description: "JDK main-line development https://openjdk.org/projects/jdk",
      replacements: {
        amd64: "x64",
        arm64: "aarch64",
        darwin: "macos",
      },
      files: [
        {
	  name: c,
          src: "jdk-{{.Version}}.jdk/Contents/Home/bin/%s" % c,
	},
	for c in commands
      ],
      version_constraint: "false",
      version_overrides: [
        {
          url: "https://download.java.net/java/GA/jdk{{.Version}}/%s/1/GPL/openjdk-{{.Version}}_{{.OS}}-{{.Arch}}_bin.tar.gz" % v.commitHash,
          version_constraint: "Version == \"%s\"" % v.version,
        }
        for v in versions
      ],
    }
  ]
}

Generate registry.json from registry.jsonnet.

jsonnet -o registry.json registry.jsonnet

registry.json

{
   "packages": [
      {
         "description": "JDK main-line development https://openjdk.org/projects/jdk",
         "files": [
            {
               "name": "jar",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jar"
            },
            {
               "name": "jarsigner",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jarsigner"
            },
            {
               "name": "java",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/java"
            },
            {
               "name": "javac",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/javac"
            },
            {
               "name": "javadoc",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/javadoc"
            },
            {
               "name": "javap",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/javap"
            },
            {
               "name": "jcmd",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jcmd"
            },
            {
               "name": "jconsole",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jconsole"
            },
            {
               "name": "jdb",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jdb"
            },
            {
               "name": "jdeprscan",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jdeprscan"
            },
            {
               "name": "jdeps",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jdeps"
            },
            {
               "name": "jfr",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jfr"
            },
            {
               "name": "jhsdb",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jhsdb"
            },
            {
               "name": "jimage",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jimage"
            },
            {
               "name": "jinfo",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jinfo"
            },
            {
               "name": "jlink",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jlink"
            },
            {
               "name": "jmap",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jmap"
            },
            {
               "name": "jmod",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jmod"
            },
            {
               "name": "jpackage",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jpackage"
            },
            {
               "name": "jps",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jps"
            },
            {
               "name": "jrunscript",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jrunscript"
            },
            {
               "name": "jshell",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jshell"
            },
            {
               "name": "jstack",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jstack"
            },
            {
               "name": "jstat",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jstat"
            },
            {
               "name": "jstatd",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jstatd"
            },
            {
               "name": "jwebserver",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/jwebserver"
            },
            {
               "name": "keytool",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/keytool"
            },
            {
               "name": "rmiregistry",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/rmiregistry"
            },
            {
               "name": "serialver",
               "src": "jdk-{{.Version}}.jdk/Contents/Home/bin/serialver"
            }
         ],
         "replacements": {
            "amd64": "x64",
            "arm64": "aarch64",
            "darwin": "macos"
         },
         "repo_name": "jdk",
         "repo_owner": "openjdk",
         "type": "http",
         "version_constraint": "false",
         "version_overrides": [
            {
               "url": "https://download.java.net/java/GA/jdk{{.Version}}/db379da656dc47308e138f21b33976fa/1/GPL/openjdk-{{.Version}}_{{.OS}}-{{.Arch}}_bin.tar.gz",
               "version_constraint": "Version == \"18.0.2.1\""
            }
         ]
      }
   ]
}

aqua-checksums.json

If OpenJDK doesn't provide checksum files (e.g. aqua_1.26.2_checksums.txt), aqua downloads assets and calculates checksums from them and adds them to aqua-checksums.json.

{
  "checksums": [
    {
      "id": "http/download.java.net/java/GA/jdk18.0.2.1/db379da656dc47308e138f21b33976fa/1/GPL/openjdk-18.0.2.1_macos-aarch64_bin.tar.gz",
      "checksum": "A9C4F224444440DDD3FE82D24687E5166F250D0C8E158BEA622795173742D4C70FA0F0876CA6623D85F0C5A984D05E6E0BB00547AB402A29C463A31A7A777A60",
      "algorithm": "sha512"
    }
  ]
}