[Enhancement] Re-release workflow should use corresponding tag in opensearch-build to do re-release instead of main branch

[peterzhuamazon]

[Enhancement] Re-release workflow should use corresponding tag in opensearch-build to do re-release instead of main branch.

The current changes in security plugin caused a lot of changes in main branch of opensearch-build: opensearch-project/security#3624. If we bound to re-release older versions of docker images, we would like to use the corresponding tag code to release, instead of introducing new changes from main back to older version, such as config and entrypoint changes.

Thanks.

[peterzhuamazon]

PRs:

• Add parameter for docker re-release to use corresponding opensearch-build tag reference opensearch-build-libraries#373
• Switch docker re-release to use 6.2.0 lib in jenkinsfile #4393

[peterzhuamazon]

2PR with @zelinh on 20240201 to re-release 1.3.14 and 2.11.1 version.

Confirmed that build is using the corresponding tag instead of main:

Checking out Revision 6444bd923de1057bb1c1d1e585181d2287dbb5b3 (1.3.14)
Commit message: "Replace refs in manifests with tags for 1.3.14 (#4282)"
 > git rev-parse --resolve-git-dir /var/jenkins/workspace/docker-build/.git # timeout=10
 > git config remote.origin.url https://github.com/opensearch-project/opensearch-build # timeout=10
Fetching upstream changes from https://github.com/opensearch-project/opensearch-build
 > git --version # timeout=10
 > git --version # 'git version 2.25.1'
 > git fetch --tags --force --progress -- https://github.com/opensearch-project/opensearch-build +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse origin/1.3.14^{commit} # timeout=10
 > git rev-parse 1.3.14^{commit} # timeout=10
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 6444bd923de1057bb1c1d1e585181d2287dbb5b3 # timeout=10
 > git rev-list --no-walk 6444bd923de1057bb1c1d1e585181d2287dbb5b3 # timeout=10

......

Checking out Revision dce080075c219010371c02e699e816dd4df7758f (2.11.1)
Commit message: "Update 2.11.1 tags in OS and OSD manifests (#4258)"
First time build. Skipping changelog.
[Pipeline] withCredentials
 > git rev-parse --resolve-git-dir /var/jenkins/workspace/docker-build/.git # timeout=10
 > git config remote.origin.url https://github.com/opensearch-project/opensearch-build # timeout=10
Fetching upstream changes from https://github.com/opensearch-project/opensearch-build
 > git --version # timeout=10
 > git --version # 'git version 2.25.1'
 > git fetch --tags --force --progress -- https://github.com/opensearch-project/opensearch-build +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse origin/2.11.1^{commit} # timeout=10
 > git rev-parse 2.11.1^{commit} # timeout=10
 > git config core.sparsecheckout # timeout=10
 > git checkout -f dce080075c219010371c02e699e816dd4df7758f # timeout=10

Without security admin:admin changes in here: opensearch-project/security#3622

% docker run -it -d -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" opensearchstaging/opensearch:1.3.14.8997.20240201
7f347291427ed0db5be0b7da341300056154659c3b2105bb2990cb60a1a980f9

% curl -XGET https://localhost:9200 -u admin:admin --insecure

{
  "name" : "7f347291427e",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "rLg_6PcTTI2tHVZ_RJPaMQ",
  "version" : {
    "distribution" : "opensearch",
    "number" : "1.3.14",
    "build_type" : "tar",
    "build_hash" : "21940d8239b50285ef7f98a1762ef281a5b1c7ee",
    "build_date" : "2023-12-08T22:13:08.793451Z",
    "build_snapshot" : false,
    "lucene_version" : "8.10.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}


% docker run -it -d -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" opensearchstaging/opensearch:2.11.1.8942.20240201
fcc05ec988405f82eb0b129e39b0e9b3abfcbe0bce4f3f1a28c822410b29cf4c

% curl -XGET https://localhost:9200 -u admin:admin --insecure

{
  "name" : "fcc05ec98840",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "pOa8aALvTsGqEYrH-XNv0g",
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.11.1",
    "build_type" : "tar",
    "build_hash" : "6b1986e964d440be9137eba1413015c31c5a7752",
    "build_date" : "2023-11-29T21:43:10.135035992Z",
    "build_snapshot" : false,
    "lucene_version" : "9.7.0",
    "minimum_wire_compatibility_version" : "7.10.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

[peterzhuamazon]

https://build.ci.opensearch.org/job/docker-re-release/10/console
https://build.ci.opensearch.org/job/docker-re-release/11/console
https://build.ci.opensearch.org/job/docker-re-release/12/console
https://build.ci.opensearch.org/job/docker-re-release/13/console

[peterzhuamazon]

https://build.ci.opensearch.org/job/docker-scan/2820/artifact/scan_docker_image.txt
https://build.ci.opensearch.org/job/docker-scan/2821/artifact/scan_docker_image.txt
https://build.ci.opensearch.org/job/docker-scan/2822/artifact/scan_docker_image.txt
https://build.ci.opensearch.org/job/docker-scan/2823/artifact/scan_docker_image.txt