Skip to content

Commit

Permalink
chore: update new client generation workflow (#10621)
Browse files Browse the repository at this point in the history
* chore: update new client generation workflow

* update README.md
  • Loading branch information
JoeWang1127 authored Mar 26, 2024
1 parent 7f75319 commit c406cbf
Show file tree
Hide file tree
Showing 2 changed files with 48 additions and 38 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/generate_new_client_hermetic_build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ jobs:
python /src/generate_repo.py generate \
--generation-config-yaml=/workspace/google-cloud-java/generation_config.yaml \
--repository-path=/workspace/google-cloud-java \
--target-library-api-shortname=${API_SHORTNAME}
--target-library-names=${API_SHORTNAME}
env:
API_SHORTNAME: ${{ github.event.inputs.api_shortname }}
- name: Push to branch and create PR
Expand Down Expand Up @@ -144,7 +144,7 @@ jobs:
python /src/generate_repo.py generate \\
--generation-config-yaml=/workspace/google-cloud-java/generation_config.yaml \\
--repository-path=/workspace/google-cloud-java \\
--target-library-api-shortname=${API_SHORTNAME}
--target-library-names=${API_SHORTNAME}
\`\`\`"
gh pr create --title "${commit_message}" --label "owlbot:run" --head "${branch_name}" --body "${pr_body}"
Expand Down
82 changes: 46 additions & 36 deletions generation/new_client_hermetic_build/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,23 +9,27 @@ This new generation workflow enables generation of new libraries by
## Components
### `new_library_hermetic_build/add-new-client-config.py`
This script takes 10 arguments that map to items in the newly added library that
goes in `generation_config.yaml`. A new entry will be added to `libraries` with
the necessary generation configuration.
goes in `generation_config.yaml`.
A new entry will be added to `libraries` with the necessary generation
configuration.

### `.github/workflows/generate_new_client_hermetic_build.yaml`
This workflow orchestrates the `add-new-client-config.py` script and also uses our docker
image
([gcr.io/cloud-devrel-public-resources/java-library-generation](https://pantheon.corp.google.com/gcr/images/cloud-devrel-public-resources/global/java-library-generation))
to generate a new library. It also creates the pull request.
to generate a new library.
It also creates the pull request.


## Execute the Github Action
## Execute the GitHub Action

In order to run the
[Github Action](https://github.com/googleapis/google-cloud-java/actions/workflows/generate_new_client_hermetic_build.yaml)
[GitHub Action](https://github.com/googleapis/google-cloud-java/actions/workflows/generate_new_client_hermetic_build.yaml)
, you need to specify a few parameters.
These parameters will be available in the Cloud Drop link (a YAML file) included in the buganizer request.
The example in this README uses AlloyDB's [Cloud Drop](https://github.com/googleapis/googleapis/blob/master/google/cloud/alloydb/v1/alloydb_v1.yaml) file as an example.
These parameters will be available in the Cloud Drop link (a YAML file) included
in the Buganizer request.
The example in this README uses AlloyDB's [Cloud Drop](https://github.com/googleapis/googleapis/blob/master/google/cloud/alloydb/v1/alloydb_v1.yaml)
file as an example.

### API short name (`api_shortname`)

Expand All @@ -48,13 +52,14 @@ Example: `alloydb`
### Proto path (`proto_path`)

This is the path from the internal `google3/third_party/googleapis/stable` root to the
directory that contains the proto definitions for a specific version.
For example: `google/datastore/v2`. Root-level proto paths like
`google/datastore` are not supported.
Note that the internal `google3/third_party/googleapis/stable` directory is mirrored externally in https://github.com/googleapis/googleapis/blob/master/.
This is the path from the internal `google3/third_party/googleapis/stable` root
to the directory that contains the proto definitions for a specific version.
For example: `google/datastore/v2`.
Root-level proto paths like `google/datastore` are not supported.
Note that the internal `google3/third_party/googleapis/stable` directory is
mirrored externally in https://github.com/googleapis/googleapis/blob/master/.

For example, if the buganizer ticket includes:
For example, if the Buganizer ticket includes:

> Link to protos: `http://...(omit).../google/cloud/alloydb/v1alpha/alloydb_v1alpha.yaml`.
Expand All @@ -64,8 +69,8 @@ Therefore, the "proto path" value we supply to the command is
`google/cloud/alloydb/v1alpha`.

We will publish a single module for a service that includes the specified version
(in the example, `v1alpha`). Any future version must be manually added to
the configuration yaml (`google-cloud-java/generation_config.yaml`)
(in the example, `v1alpha`).
Any future version must be manually added to the configuration yaml (`google-cloud-java/generation_config.yaml`)

#### More than one `proto_path`

Expand Down Expand Up @@ -93,7 +98,7 @@ The value must starts with "https://".
Example: `https://cloud.google.com/alloydb/docs/reference/rest`

If the value exists, add it as a flag to the python command below (see [Advanced
Options](#advanced-options]):
Options](#advanced-options):
`--rest-docs="https://cloud.google.com/alloydb/docs/reference/rest" \`

### RPC Docs (`rpc_docs`)
Expand All @@ -104,13 +109,15 @@ The value must starts with "https://".
Example: `https://cloud.google.com/speech-to-text/docs/reference/rpc`

If the value exists, add it as a flag to the python command below (see [Advanced
Options](#advanced-options]):
Options](#advanced-options):
`--rpc-docs="https://cloud.google.com/speech-to-text/docs/reference/rpc" \`

### API description (`api_description`)

The corresponding value in the Cloud Drop page is `documentation.summary` or `documentation.overview`.
If both of those fields are missing, take the description from the product page above. Use the first sentence to keep it concise.
If both of those fields are missing, take the description from the product page
above.
Use the first sentence to keep it concise.

Example:
```
Expand All @@ -121,31 +128,33 @@ Example:

### Distribution Name (`distribution_name`)

This variable determines the Maven coordinates of the generated library. It
defaults to `com.google.cloud:google-cloud-{api_shortname}`. This mainly affect
the values in the generated `pom.xml` files.
This variable determines the Maven coordinates of the generated library.
It defaults to `com.google.cloud:google-cloud-{api_shortname}`.
This mainly affect the values in the generated `pom.xml` files.

### Library Name (`library_name`)

This variable indicates the output folder of the library. For example you can
have two libraries with `alloydb` (AlloyDB and AlloyDB Connectors)
as `api_shortname`. In order to avoid both
libraries going to the default `java-alloydb` folder, we can override this
behavior by specifying a value like `alloydb-connectors` so the AlloyDB
Connectors goes to `java-alloydb-connectors`.
This variable indicates the output folder of the library.
For example, you can have two libraries with `alloydb`
(AlloyDB and AlloyDB Connectors) as `api_shortname`.
In order to avoid both libraries going to the default `java-alloydb` folder,
we can override this behavior by specifying a value like `alloydb-connectors`
so the AlloyDB Connectors goes to `java-alloydb-connectors`.

## Advanced Options

In case the steps above don't show you how to specify the desired options, you can
run the `add-new-client-config.py` script in your local evironment. The advanced options
not shown in the section above **cannot be specified in the Github Action**,
hence the need for a local run (refer to the "Prerequisites
In case the steps above don't show you how to specify the desired options, you
can run the `add-new-client-config.py` script in your local environment.
The advanced options not shown in the section above **cannot be specified in
the GitHub Action**, hence the need for a local run (refer to the "Prerequisites
(for local environment)" section).
For the explanation of the available parameters, run:
`python3.9 generation/new_client_hermetic_build/add-new-client-config.py generate --help`.
```
python3.9 generation/new_client_hermetic_build/add-new-client-config.py generate --help
```

After you run the script, you will see that the `generation_config.yaml` file
was modified (or the script exited because the library already existed)
was modified (or the script exited because the library already existed).

The last step you need is to `cd` into the root of `google-cloud-java` and run
```
Expand All @@ -161,11 +170,12 @@ docker run --rm \
python /src/generate_repo.py generate \
--generation-config-yaml=/workspace/google-cloud-java/generation_config.yaml \
--repository-path=/workspace/google-cloud-java \
--target-library-api-shortname=<your api_shortname>
--target-library-names=<library_name if specified; otherwise api_shortname>
```

This docker container will run the generation scripts and generate the library
in your repo. You can create a PR explaining what commands you used (the docker
command is not as informative as the `add-new-client-config.py` call, so make sure at least
in your repo.
You can create a PR explaining what commands you used (the docker command is
not as informative as the `add-new-client-config.py` call, so make sure at least
the add-new-client-config.py arguments were listed).

0 comments on commit c406cbf

Please sign in to comment.