Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON Schema postgen native json fieldnames #178

Merged
merged 4 commits into from
Mar 10, 2021

Conversation

grant
Copy link
Contributor

@grant grant commented Mar 8, 2021

Fixes #176

  • Uses protoc plugin json_fieldnames config, which natively produces JSON fields in the JSON schema rather than a postgen step
    • Primarily notice field crc32C is now crc32c (which is correct)
  • Removes postgen code around changing fields to lowerCamelCase
  • id changes are unrelated/tangential (we should probably try to remove them in the future)

This will fix googleapis/google-cloudevents-java#58 when the Java library is re-generated.

@grant grant requested review from jskeet and grayside March 8, 2021 02:40
@grant grant requested a review from a team as a code owner March 8, 2021 02:40
@grant grant self-assigned this Mar 8, 2021
@product-auto-label product-auto-label bot added the api: eventarc Issues related to the googleapis/google-cloudevents API. label Mar 8, 2021
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Mar 8, 2021
@grant grant changed the title JSON Schema postgen JSON Schema postgen native json fieldnames Mar 8, 2021
@grant grant requested a review from jskeet March 8, 2021 16:30
Copy link
Collaborator

@jskeet jskeet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If naming collisions aren't a problem, I'm happy with this. (As noted in comments, I'm still not fully comfortable with the choices that lead to that situation, but that's separate and doesn't need to be addressed in this PR.)

@grant grant added the automerge Merge the pull request once unit tests and other checks pass. label Mar 10, 2021
@grant grant merged commit 28be01a into master Mar 10, 2021
@grant grant deleted the grant_jsonschema_postgen_json_cap branch March 10, 2021 17:37
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Mar 10, 2021
jay-shi added a commit that referenced this pull request Mar 23, 2021
* chore: run the generator, feb 26, 2021 (#173)

Runs the generator manually.

The latest change has improvements to metadata about base64 type strings.

Fixes: #174

* chore(deps): bump urijs from 1.19.5 to 1.19.6 in /tools/quicktype-wrapper (#175)

Bumps [urijs](https://github.com/medialize/URI.js) from 1.19.5 to 1.19.6.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/medialize/URI.js/releases">urijs's releases</a>.</em></p>
<blockquote>
<h2>1.19.6 (February 13th 2021)</h2>
<ul>
<li><strong>SECURITY</strong> fixing <a href="http://medialize.github.io/URI.js/docs.html#static-parse"><code>URI.parse()</code></a> to rewrite <code>\</code> in scheme delimiter to <code>/</code> as Node and Browsers do - disclosed privately by <a href="https://twitter.com/ynizry">Yaniv Nizry</a> from the CxSCA AppSec team at Checkmarx</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/medialize/URI.js/blob/gh-pages/CHANGELOG.md">urijs's changelog</a>.</em></p>
<blockquote>
<h3>1.19.6 (February 13th 2021)</h3>
<ul>
<li><strong>SECURITY</strong> fixing <a href="http://medialize.github.io/URI.js/docs.html#static-parse"><code>URI.parse()</code></a> to rewrite <code>\</code> in scheme delimiter to <code>/</code> as Node and Browsers do - disclosed privately by <a href="https://twitter.com/ynizry">Yaniv Nizry</a> from the CxSCA AppSec team at Checkmarx</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/medialize/URI.js/commit/46c8ac0c7e6997daad8ff6859784b39f8892fa97"><code>46c8ac0</code></a> chore(build): bumping to version 1.19.6</li>
<li><a href="https://github.com/medialize/URI.js/commit/a1ad8bcbc39a4d136d7e252e76e957f3ece70839"><code>a1ad8bc</code></a> fix(parse): treat backslash as forwardslash in scheme delimiter</li>
<li>See full diff in <a href="https://github.com/medialize/URI.js/compare/v1.19.5...v1.19.6">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=urijs&package-manager=npm_and_yarn&previous-version=1.19.5&new-version=1.19.6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/googleapis/google-cloudevents/network/alerts).

</details>

* JSON Schema postgen native json fieldnames (#178)

* fix: uses protoc plugin field for json schema field capitalization

Signed-off-by: Grant Timmerman <[email protected]>

* fix: fix replacement string

Signed-off-by: Grant Timmerman <[email protected]>

* fix: use full id name in jsonschema definitions

Signed-off-by: Grant Timmerman <[email protected]>

* Add action script to handle copybara flow (#181)

* Add a action script that auto-generates a pull request when there is a push to branch "copybara". The branch "copybara" only receives push from Copybara flow.

* Cloud Audit Log Catalog (#158)

* docs: add CAL catalog

Signed-off-by: Grant Timmerman <[email protected]>

* docs: updates docs with cal catalog

Signed-off-by: Grant Timmerman <[email protected]>

* docs: detailed test for CAL trigger docs

Signed-off-by: Grant Timmerman <[email protected]>

* fix: cal catalog script use json quotes not js quotes

Signed-off-by: Grant Timmerman <[email protected]>

* refactor: improve readability of audit log gen script

Signed-off-by: Grant Timmerman <[email protected]>

* chore: add licenses to files (#183)

Signed-off-by: Grant Timmerman <[email protected]>

* fix: Fix generation on Windows

Fixes #154

* ci: jsonschema validation (#186)

Validates that the JSON schemas are valid schemas according to the JSON schema schema.

I've ran a few CI runs to ensure that this check is correct.

We already have CI that checks for a 0 diff between proto and jsonschema, so naturally that check also breaks when if modifying the json schema manually.

## Example CI runs

Purposeful invalid JSON schema break:

![Screen Shot 2021-03-18 at 13 12 25](https://user-images.githubusercontent.com/744973/111676018-ba718c00-87eb-11eb-8b32-3ece124184f1.png)

Purposeful valid JSON schema fix:

![Screen Shot 2021-03-18 at 13 12 43](https://user-images.githubusercontent.com/744973/111676024-bcd3e600-87eb-11eb-82e5-00ba04dfbf9f.png)

## Notes

Local testing is pretty easy, just install Node and copy the npx script.

### Example CI output

```md
Run JSON_SCHEMAS=$(find ./jsonschema/google/events -name "*.json")
npx: installed 41 in 4.585s
./jsonschema/google/events/cloud/firestore/v1/DocumentEventData.json valid
npx: installed 41 in 2.189s
./jsonschema/google/events/cloud/storage/v1/StorageObjectData.json valid
npx: installed 41 in 2.085s
./jsonschema/google/events/cloud/cloudbuild/v1/BuildEventData.json valid
npx: installed 41 in 2.149s
./jsonschema/google/events/cloud/audit/v1/LogEntryData.json valid
npx: installed 41 in 2.085s
./jsonschema/google/events/cloud/scheduler/v1/SchedulerJobData.json valid
npx: installed 41 in 2.181s
./jsonschema/google/events/cloud/pubsub/v1/MessagePublishedData.json valid
npx: installed 41 in 2.166s
./jsonschema/google/events/firebase/database/v1/ReferenceEventData.json valid
npx: installed 41 in 2.094s
./jsonschema/google/events/firebase/analytics/v1/AnalyticsLogData.json valid
npx: installed 41 in 2.091s
./jsonschema/google/events/firebase/remoteconfig/v1/RemoteConfigEventData.json valid
npx: installed 41 in 2.161s
./jsonschema/google/events/firebase/auth/v1/AuthEventData.json valid
```

Co-authored-by: Grant Timmerman <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jon Skeet <[email protected]>
grant added a commit that referenced this pull request Mar 24, 2021
* chore: run the generator, feb 26, 2021 (#173)

Runs the generator manually.

The latest change has improvements to metadata about base64 type strings.

Fixes: #174

* chore(deps): bump urijs from 1.19.5 to 1.19.6 in /tools/quicktype-wrapper (#175)

Bumps [urijs](https://github.com/medialize/URI.js) from 1.19.5 to 1.19.6.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/medialize/URI.js/releases">urijs's releases</a>.</em></p>
<blockquote>
<h2>1.19.6 (February 13th 2021)</h2>
<ul>
<li><strong>SECURITY</strong> fixing <a href="http://medialize.github.io/URI.js/docs.html#static-parse"><code>URI.parse()</code></a> to rewrite <code>\</code> in scheme delimiter to <code>/</code> as Node and Browsers do - disclosed privately by <a href="https://twitter.com/ynizry">Yaniv Nizry</a> from the CxSCA AppSec team at Checkmarx</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/medialize/URI.js/blob/gh-pages/CHANGELOG.md">urijs's changelog</a>.</em></p>
<blockquote>
<h3>1.19.6 (February 13th 2021)</h3>
<ul>
<li><strong>SECURITY</strong> fixing <a href="http://medialize.github.io/URI.js/docs.html#static-parse"><code>URI.parse()</code></a> to rewrite <code>\</code> in scheme delimiter to <code>/</code> as Node and Browsers do - disclosed privately by <a href="https://twitter.com/ynizry">Yaniv Nizry</a> from the CxSCA AppSec team at Checkmarx</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/medialize/URI.js/commit/46c8ac0c7e6997daad8ff6859784b39f8892fa97"><code>46c8ac0</code></a> chore(build): bumping to version 1.19.6</li>
<li><a href="https://github.com/medialize/URI.js/commit/a1ad8bcbc39a4d136d7e252e76e957f3ece70839"><code>a1ad8bc</code></a> fix(parse): treat backslash as forwardslash in scheme delimiter</li>
<li>See full diff in <a href="https://github.com/medialize/URI.js/compare/v1.19.5...v1.19.6">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=urijs&package-manager=npm_and_yarn&previous-version=1.19.5&new-version=1.19.6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/googleapis/google-cloudevents/network/alerts).

</details>

* JSON Schema postgen native json fieldnames (#178)

* fix: uses protoc plugin field for json schema field capitalization

Signed-off-by: Grant Timmerman <[email protected]>

* fix: fix replacement string

Signed-off-by: Grant Timmerman <[email protected]>

* fix: use full id name in jsonschema definitions

Signed-off-by: Grant Timmerman <[email protected]>

* Add action script to handle copybara flow (#181)

* Add a action script that auto-generates a pull request when there is a push to branch "copybara". The branch "copybara" only receives push from Copybara flow.

* Cloud Audit Log Catalog (#158)

* docs: add CAL catalog

Signed-off-by: Grant Timmerman <[email protected]>

* docs: updates docs with cal catalog

Signed-off-by: Grant Timmerman <[email protected]>

* docs: detailed test for CAL trigger docs

Signed-off-by: Grant Timmerman <[email protected]>

* fix: cal catalog script use json quotes not js quotes

Signed-off-by: Grant Timmerman <[email protected]>

* refactor: improve readability of audit log gen script

Signed-off-by: Grant Timmerman <[email protected]>

* chore: add licenses to files (#183)

Signed-off-by: Grant Timmerman <[email protected]>

* fix: Fix generation on Windows

Fixes #154

* ci: jsonschema validation (#186)

Validates that the JSON schemas are valid schemas according to the JSON schema schema.

I've ran a few CI runs to ensure that this check is correct.

We already have CI that checks for a 0 diff between proto and jsonschema, so naturally that check also breaks when if modifying the json schema manually.

## Example CI runs

Purposeful invalid JSON schema break:

![Screen Shot 2021-03-18 at 13 12 25](https://user-images.githubusercontent.com/744973/111676018-ba718c00-87eb-11eb-8b32-3ece124184f1.png)

Purposeful valid JSON schema fix:

![Screen Shot 2021-03-18 at 13 12 43](https://user-images.githubusercontent.com/744973/111676024-bcd3e600-87eb-11eb-82e5-00ba04dfbf9f.png)

## Notes

Local testing is pretty easy, just install Node and copy the npx script.

### Example CI output

```md
Run JSON_SCHEMAS=$(find ./jsonschema/google/events -name "*.json")
npx: installed 41 in 4.585s
./jsonschema/google/events/cloud/firestore/v1/DocumentEventData.json valid
npx: installed 41 in 2.189s
./jsonschema/google/events/cloud/storage/v1/StorageObjectData.json valid
npx: installed 41 in 2.085s
./jsonschema/google/events/cloud/cloudbuild/v1/BuildEventData.json valid
npx: installed 41 in 2.149s
./jsonschema/google/events/cloud/audit/v1/LogEntryData.json valid
npx: installed 41 in 2.085s
./jsonschema/google/events/cloud/scheduler/v1/SchedulerJobData.json valid
npx: installed 41 in 2.181s
./jsonschema/google/events/cloud/pubsub/v1/MessagePublishedData.json valid
npx: installed 41 in 2.166s
./jsonschema/google/events/firebase/database/v1/ReferenceEventData.json valid
npx: installed 41 in 2.094s
./jsonschema/google/events/firebase/analytics/v1/AnalyticsLogData.json valid
npx: installed 41 in 2.091s
./jsonschema/google/events/firebase/remoteconfig/v1/RemoteConfigEventData.json valid
npx: installed 41 in 2.161s
./jsonschema/google/events/firebase/auth/v1/AuthEventData.json valid
```

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jay Shi <[email protected]>
Co-authored-by: Jon Skeet <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: eventarc Issues related to the googleapis/google-cloudevents API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
2 participants