Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deps: update library generation image tag #10758

Merged
merged 8 commits into from
May 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 18 additions & 34 deletions .github/workflows/generated_files_sync.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,34 +20,25 @@ on:
pull_request:
name: generation diff
env:
library_generation_image_tag: 2.39.0
repo_volumes: "-v repo-google-cloud-java:/workspace/google-cloud-java"
library_generation_image_tag: 2.40.0
jobs:
root-pom:
# root pom.xml does not have diff from generated one
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: setup docker environment
shell: bash
run: |
# we create a volume pointing to `pwd` (google-cloud-java) that will
# be referenced by the container and its children
if [[ $(docker volume inspect repo-google-cloud-java) != '[]' ]]; then
docker volume rm repo-google-cloud-java
fi
docker volume create --name "repo-google-cloud-java" --opt "type=none" --opt "device=$(pwd)" --opt "o=bind"
- name: Generate root pom.xml file
shell: bash
run: |
docker run --rm \
${repo_volumes} \
-v /tmp:/tmp \
-v /var/run/docker.sock:/var/run/docker.sock \
-e "RUNNING_IN_DOCKER=true" \
-e "REPO_BINDING_VOLUMES=${repo_volumes}" \
docker run \
--rm \
-u "$(id -u):$(id -g)" \
-v "$(pwd):/workspace" \
--entrypoint python \
gcr.io/cloud-devrel-public-resources/java-library-generation:"${library_generation_image_tag}" \
python /src/cli/generate_monorepo_root_pom.py generate --repository-path=/workspace/google-cloud-java
/src/cli/generate_monorepo_root_pom.py \
generate \
--repository-path=/workspace
- name: Fail if there's any difference
run: git --no-pager diff --exit-code

Expand All @@ -56,26 +47,19 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: setup docker environment
shell: bash
run: |
# we create a volume pointing to `pwd` (google-cloud-java) that will
# be referenced by the container and its children
if [[ $(docker volume inspect repo-google-cloud-java) != '[]' ]]; then
docker volume rm repo-google-cloud-java
fi
docker volume create --name "repo-google-cloud-java" --opt "type=none" --opt "device=$(pwd)" --opt "o=bind"
- name: Generate gapic-libraries-bom/pom.xml
shell: bash
run: |
docker run --rm \
${repo_volumes} \
-v /tmp:/tmp \
-v /var/run/docker.sock:/var/run/docker.sock \
-e "RUNNING_IN_DOCKER=true" \
-e "REPO_BINDING_VOLUMES=${repo_volumes}" \
docker run \
--rm \
-u "$(id -u):$(id -g)" \
-v "$(pwd):/workspace" \
--entrypoint python \
gcr.io/cloud-devrel-public-resources/java-library-generation:"${library_generation_image_tag}" \
python /src/cli/generate_monorepo_gapic_bom.py generate --repository-path=/workspace/google-cloud-java --versions-file=/workspace/google-cloud-java/versions.txt
/src/cli/generate_monorepo_gapic_bom.py \
generate \
--repository-path=/workspace \
--versions-file=/workspace/versions.txt
- name: Fail if there's any difference
run: git --no-pager diff --exit-code

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/hermetic_library_generation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
library_generation:
runs-on: ubuntu-latest
env:
library_generation_image_tag: 2.39.0
library_generation_image_tag: 2.40.0
steps:
- uses: actions/checkout@v4
with:
Expand Down
24 changes: 4 additions & 20 deletions generation/hermetic_library_generation.sh
Original file line number Diff line number Diff line change
Expand Up @@ -70,9 +70,7 @@ if [ -z "${generation_config}" ]; then
echo "Use default generation config: ${generation_config}"
fi

volume_name="repo"
workspace_name="/workspace/repo"
repo_volumes="${volume_name}:${workspace_name}"
workspace_name="/workspace"
baseline_generation_config="baseline_generation_config.yaml"
message="chore: generate libraries at $(date)"

Expand All @@ -89,28 +87,14 @@ fi
git show "${target_branch}":"${generation_config}" > "${baseline_generation_config}"
config_diff=$(diff "${generation_config}" "${baseline_generation_config}")

# bind docker volume to include the repository in docker running environment.
if [[ $(docker volume inspect ${volume_name}) != '[]' ]]; then
docker volume rm ${volume_name}
fi
docker volume create \
--name ${volume_name} \
--opt "type=none" \
--opt "device=$(pwd)" \
--opt "o=bind"
# run hermetic code generation docker image.
docker run \
--rm \
-v "${repo_volumes}" \
-v /tmp:/tmp \
-v /var/run/docker.sock:/var/run/docker.sock \
-e "RUNNING_IN_DOCKER=true" \
-e "REPO_BINDING_VOLUMES=-v ${repo_volumes}" \
-u "$(id -u):$(id -g)" \
-v "$(pwd):${workspace_name}" \
gcr.io/cloud-devrel-public-resources/java-library-generation:"${image_tag}" \
python /src/cli/entry_point.py generate \
--baseline-generation-config-path="${workspace_name}/${baseline_generation_config}" \
--current-generation-config-path="${workspace_name}/${generation_config}" \
--repository-path="${workspace_name}"
--current-generation-config-path="${workspace_name}/${generation_config}"
# commit the change to the pull request.
git add java-* pom.xml gapic-libraries-bom/pom.xml versions.txt
changed_files=$(git diff --cached --name-only)
Expand Down
44 changes: 10 additions & 34 deletions generation/new_client_hermetic_build/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# New client generation (GitHub Action)
This new generation workflow enables generation of new libraries by
1. appending a new library to our [generation_config.yaml](https://github.com/googleapis/google-cloud-java/blob/c7429c0eec419c01d4e2fe14d063b9335efb810b/generation_config.yaml).
2. running the hermetic build scripts docker image and
generate the newly added library.
3. create a PR with the changes.
1. Appending a new library to our [generation_config.yaml](/generation_config.yaml).
2. Creating a PR with the changes.

The new client will be generated by [Hermetic library generation workflow](/.github/workflows/hermetic_library_generation.yaml).


## Components
Expand All @@ -14,18 +14,15 @@ A new entry will be added to `libraries` with the necessary generation
configuration.

### `.github/workflows/generate_new_client_hermetic_build.yaml`
This workflow orchestrates the `add-new-client-config.py` script and also uses our docker
image
([gcr.io/cloud-devrel-public-resources/java-library-generation](https://pantheon.corp.google.com/gcr/images/cloud-devrel-public-resources/global/java-library-generation))
to generate a new library.
It also creates the pull request.
This workflow orchestrates the `add-new-client-config.py` script and creates
a pull request.


## Execute the GitHub Action

In order to run the
[GitHub Action](https://github.com/googleapis/google-cloud-java/actions/workflows/generate_new_client_hermetic_build.yaml)
, you need to specify a few parameters.
[GitHub Action](/.github/workflows/generate_new_client_hermetic_build.yaml),
you need to specify a few parameters.
These parameters will be available in the Cloud Drop link (a YAML file) included
in the Buganizer request.
The example in this README uses AlloyDB's [Cloud Drop](https://github.com/googleapis/googleapis/blob/master/google/cloud/alloydb/v1/alloydb_v1.yaml)
Expand Down Expand Up @@ -156,26 +153,5 @@ python3.9 generation/new_client_hermetic_build/add-new-client-config.py generate
After you run the script, you will see that the `generation_config.yaml` file
was modified (or the script exited because the library already existed).

The last step you need is to `cd` into the root of `google-cloud-java` and run
```
docker volume create --name "repo-google-cloud-java" --opt "type=none" --opt "device=$(pwd)" --opt "o=bind"
repo_volumes="-v repo-google-cloud-java:/workspace/google-cloud-java"
docker run --rm \
${repo_volumes} \
-v /tmp:/tmp \
-v /var/run/docker.sock:/var/run/docker.sock \
-e "RUNNING_IN_DOCKER=true" \
-e "REPO_BINDING_VOLUMES=${repo_volumes}" \
gcr.io/cloud-devrel-public-resources/java-library-generation:latest \
python /src/generate_repo.py generate \
--generation-config-yaml=/workspace/google-cloud-java/generation_config.yaml \
--repository-path=/workspace/google-cloud-java \
--target-library-names=<library_name if specified; otherwise api_shortname>

```

This docker container will run the generation scripts and generate the library
in your repo.
You can create a PR explaining what commands you used (the docker command is
not as informative as the `add-new-client-config.py` call, so make sure at least
the add-new-client-config.py arguments were listed).
Please create a PR explaining what commands you used and make sure the
add-new-client-config.py arguments were listed).
32 changes: 16 additions & 16 deletions generation_config.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
gapic_generator_version: 2.39.0
protobuf_version: '25.3'
protoc_version: '25.3'
googleapis_commitish: 5a0ae652df1ef39b7849bf2bd6ba417376ce3204
libraries_bom_version: 26.38.0
owlbot_cli_image: sha256:623647ee79ac605858d09e60c1382a716c125fb776f69301b72de1cd35d49409
Expand Down Expand Up @@ -40,21 +40,21 @@ libraries:
- proto_path: google/identity/accesscontextmanager/v1
- proto_path: google/identity/accesscontextmanager/type

#- api_shortname: admanager
# name_pretty: Google Ad Manager API
# product_documentation: https://developers.google.com/ad-manager/api/beta
# api_description: The Ad Manager API enables an app to integrate with Google Ad Manager.
# You can read Ad Manager data and run reports using the API.
# client_documentation: https://cloud.google.com/java/docs/reference/ad-manager/latest/overview
# release_level: preview
# distribution_name: com.google.api-ads:ad-manager
# api_id: admanager.googleapis.com
# library_type: GAPIC_AUTO
# group_id: com.google.api-ads
# cloud_api: false
# GAPICs:
# - proto_path: google/ads/admanager/v1
# requires_billing: true
- api_shortname: admanager
name_pretty: Google Ad Manager API
product_documentation: https://developers.google.com/ad-manager/api/beta
api_description: The Ad Manager API enables an app to integrate with Google Ad Manager.
You can read Ad Manager data and run reports using the API.
client_documentation: https://cloud.google.com/java/docs/reference/ad-manager/latest/overview
release_level: preview
distribution_name: com.google.api-ads:ad-manager
api_id: admanager.googleapis.com
library_type: GAPIC_AUTO
group_id: com.google.api-ads
cloud_api: false
GAPICs:
- proto_path: google/ads/admanager/v1
requires_billing: true

- api_shortname: advisorynotifications
name_pretty: Advisory Notifications API
Expand Down
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<version>3.1.1</version>
<version>3.1.2</version>
<configuration>
<skip>true</skip>
</configuration>
Expand Down
Loading