Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GitHub action for publishing java artifacts #26430

Merged
merged 7 commits into from
May 12, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions .github/workflows/build_release_candidate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: build_release_candidate

# Workflow added after https://github.com/apache/beam/commit/4183e747becebd18becee5fff547af365910fc9c
# If help is needed debugging issues, you can view the release guide at that commit for guidance on how to do this manually.
# (https://github.com/apache/beam/blob/4183e747becebd18becee5fff547af365910fc9c/website/www/site/content/en/contribute/release-guide.md)
on:
workflow_dispatch:
inputs:
RELEASE:
description: Beam version of current release (e.g. 2.XX.0)
required: true
default: '2.XX.0'
RC:
description: Integer RC version for the release (e.g. 3 for RC3)
required: true
PUBLISH_JAVA_ARTIFACTS:
description: Whether to publish java artifacts to https://repository.apache.org/#stagingRepositories (yes/no)
required: true
default: 'no'

jobs:
publish_java_artifacts:
if: ${{github.event.inputs.PUBLISH_JAVA_ARTIFACTS == 'yes'}}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
with:
ref: "v${{ github.event.inputs.RELEASE }}-RC${{ github.event.inputs.RC }}"
repository: apache/beam
- name: Import GPG key
id: import_gpg
uses: crazy-max/ghaction-import-gpg@111c56156bcc6918c056dbef52164cfa583dc549
with:
gpg_private_key: ${{ secrets.GPG_PRIVATE_KEY }}
passphrase: ${{ secrets.PASSPHRASE }}
- name: Auth for nexus
run: |
mkdir -p ${HOME}/.m2
echo "<settings>
<servers>
<server>
<id>apache.releases.https</id>
<username>${{ secrets.APACHE_TOKEN_NAME }}</username>
<password>${{ secrets.APACHE_TOKEN_PASSWORD }}</password>
</server>
<server>
<id>apache.snapshots.https</id>
<username>${{ secrets.APACHE_TOKEN_NAME }}</username>
<password>${{ secrets.APACHE_TOKEN_PASSWORD }}</password>
</server>
</servers>
</settings>" > ${HOME}/.m2/settings.xml
- name: Build Java Artifacts with Gradle
run: git config credential.helper store
- name: Stage Java Artifacts into Maven
run: ./gradlew publish -Psigning.gnupg.keyName=FC383FCDE7D7E86699954EF2509872C8031C4DFB -PisRelease --no-daemon --no-parallel
39 changes: 4 additions & 35 deletions release/src/main/scripts/build_release_candidate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,10 @@
#

# This script will create a Release Candidate, includes:
# 1. Build and stage java artifacts
# 2. Stage source release on dist.apache.org
# 3. Stage python source distribution and wheels on dist.apache.org
# 4. Stage SDK docker images
# 5. Create a PR to update beam-site
# 1. Stage source release on dist.apache.org
# 2. Stage python source distribution and wheels on dist.apache.org
# 3. Stage SDK docker images
# 4. Create a PR to update beam-site

set -e

Expand Down Expand Up @@ -176,36 +175,6 @@ if [[ $confirmation != "y" ]]; then
exit
fi

echo "[Current Step]: Build and stage java artifacts"
echo "Do you want to proceed? [y|N]"
read confirmation
if [[ $confirmation = "y" ]]; then
echo "============Building and Staging Java Artifacts============="
echo "--------Cloning Beam Repo and Checkout Release Tag-------"
cd ~
wipe_local_clone_dir
mkdir -p ${LOCAL_CLONE_DIR}
cd ${LOCAL_CLONE_DIR}
git clone --depth 1 --branch "${RC_TAG}" ${GIT_REPO_URL} "${BEAM_ROOT_DIR}"
cd ${BEAM_ROOT_DIR}

echo "-------------Building Java Artifacts with Gradle-------------"
git config credential.helper store

echo "-------------Staging Java Artifacts into Maven---------------"
# Cache the key/passphrase in gpg-agent by signing an arbitrary file.
gpg --local-user ${SIGNING_KEY} --output /dev/null --sign ~/.bashrc
# Too many workers can overload (?) gpg-agent, causing gpg to prompt for a
# passphrase, and gradle doesn't play nice with pinentry.
# https://github.com/gradle/gradle/issues/11706
# --max-workers=6 works, but parallelism also seems to cause
# multiple Nexus repos to be created, so parallelism is disabled.
# https://issues.apache.org/jira/browse/BEAM-11813
./gradlew publish -Psigning.gnupg.keyName=${SIGNING_KEY} -PisRelease --no-daemon --no-parallel
echo "You need to close the staging repository manually on Apache Nexus. See the release guide for instructions."
wipe_local_clone_dir
fi

echo "[Current Step]: Stage source release on dist.apache.org"
echo "Do you want to proceed? [y|N]"
read confirmation
Expand Down
36 changes: 25 additions & 11 deletions website/www/site/content/en/contribute/release-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -548,6 +548,31 @@ is perfectly safe since the script does not depend on the current working tree.

See the source of the script for more details, or to run commands manually in case of a problem.

### Run build_release_candidate GitHub Action to create a release candidate

Note: This step is partially automated (in progress), so part of the rc creation is done by GitHub Actions and the rest is done by a script.
You don't need to wait for the action to complete to start running the script.

* **Action** [build_release_candidate](https://github.com/damccorm/beam/actions/workflows/build_release_candidate.yml) (click `run workflow`)

* **The script will:**
1. Clone the repo at the selected RC tag.
1. Run gradle publish to push java artifacts into Maven staging repo.

#### Tasks you need to do manually

1. Publish staging artifacts
1. Log in to the [Apache Nexus](https://repository.apache.org/#stagingRepositories) website.
1. Navigate to Build Promotion -> Staging Repositories (in the left sidebar).
1. Select repository `orgapachebeam-NNNN`.
1. Click the Close button.
1. When prompted for a description, enter “Apache Beam, version X, release candidate Y”.
1. Review all staged artifacts on `https://repository.apache.org/content/repositories/orgapachebeam-NNNN/`.
They should contain all relevant parts for each module, including `pom.xml`, jar, test jar, javadoc, etc.
Artifact names should follow [the existing format](https://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.beam%22) in which artifact name mirrors directory structure, e.g., `beam-sdks-java-io-kafka`.
Carefully review any new artifacts.
Some additional validation should be done during the rc validation step.

### Run build_release_candidate.sh to create a release candidate

* **Script:** [build_release_candidate.sh](https://github.com/apache/beam/blob/master/release/src/main/scripts/build_release_candidate.sh)
Expand All @@ -558,7 +583,6 @@ See the source of the script for more details, or to run commands manually in ca

* **The script will:**
1. Clone the repo at the selected RC tag.
1. Run gradle publish to push java artifacts into Maven staging repo.
1. Stage source release into dist.apache.org dev [repo](https://dist.apache.org/repos/dist/dev/beam/).
1. Stage, sign and hash python source distribution and wheels into dist.apache.org dev repo python dir
1. Stage SDK docker images to [docker hub Apache organization](https://hub.docker.com/search?q=apache%2Fbeam&type=image).
Expand Down Expand Up @@ -588,16 +612,6 @@ help with this step. Please email `dev@` and ask a member of the `beammaintainer
docker run --rm -it --entrypoint=/bin/bash apache/beam_java${ver}_sdk:${RELEASE_VERSION}_rc{RC_NUM}
ls -al /opt/apache/beam/third_party_licenses/ | wc -l
```
1. Publish staging artifacts
1. Log in to the [Apache Nexus](https://repository.apache.org/#stagingRepositories) website.
1. Navigate to Build Promotion -> Staging Repositories (in the left sidebar).
1. Select repository `orgapachebeam-NNNN`.
1. Click the Close button.
1. When prompted for a description, enter “Apache Beam, version X, release candidate Y”.
1. Review all staged artifacts on `https://repository.apache.org/content/repositories/orgapachebeam-NNNN/`.
They should contain all relevant parts for each module, including `pom.xml`, jar, test jar, javadoc, etc.
Artifact names should follow [the existing format](https://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.beam%22) in which artifact name mirrors directory structure, e.g., `beam-sdks-java-io-kafka`.
Carefully review any new artifacts.

### Upload release candidate to PyPi

Expand Down