-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add BlobWriteOption to support MD5 and CRC32C checks on create/write #271
Conversation
- Add MD5 and CRC32C computation to create and create from byte array - Change BlobTargetOption... to BlobWriteOption... in create from stream and writer - Change BlobTargetOption... to BlobWriteOption in Blob.writer - Update unit tests - Update and add integration tests
@@ -106,6 +108,15 @@ private static StorageException translate(GoogleJsonError exception) { | |||
return new StorageException(exception.getCode(), exception.getMessage(), retryable); | |||
} | |||
|
|||
private static void applyOptions(StorageObject storageObject, Map<Option, ?> options) { |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
9eceabc
to
4f24df3
Compare
- BlobWriteOption extends Serializable instead of Option - add BlobTargetObject.convert to convert BlobWriteOption and possibly mask md5 and crc32c fields - add private Storage.create(blobInfo, inputStream, blobTargetOption) - update Storage.create, Storage.writer and Blob.writer javadoc
4f24df3
to
3d13b99
Compare
Fixed the comments. I removed options from the RPC layer. |
BlobInfo updatedInfo = blobInfo.toBuilder() | ||
.md5(BaseEncoding.base64().encode(Hashing.md5().hashBytes(content).asBytes())) | ||
.crc32c(BaseEncoding.base64().encode( | ||
Ints.toByteArray(Hashing.crc32c().hashBytes(content).asInt()))) |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
Honestly, I wished |
@aozarov I don't like the idea of doing side effects on a builder either, lets keep it as is, we can possibly change it later. |
Removed the unnecessary |
Add BlobWriteOption to support MD5 and CRC32C checks on create/write
🤖 I have created a release *beep* *boop* --- ## [1.2.1](googleapis/java-artifact-registry@v1.2.0...v1.2.1) (2022-06-23) ### Dependencies * update dependency com.google.cloud:google-cloud-shared-dependencies to v2.13.0 ([#270](googleapis/java-artifact-registry#270)) ([43c1ae8](googleapis/java-artifact-registry@43c1ae8)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
This PR was generated using Autosynth. 🌈 <details><summary>Log from Synthtool</summary> ``` 2020-10-15 22:43:18,502 synthtool [DEBUG] > Executing /root/.cache/synthtool/java-resourcemanager/.github/readme/synth.py. On branch autosynth-readme nothing to commit, working directory clean 2020-10-15 22:43:19,459 synthtool [DEBUG] > Wrote metadata to .github/readme/synth.metadata/synth.metadata. ``` </details> Full log will be available here: https://source.cloud.google.com/results/invocations/128052fe-465f-41a7-88f9-a76876e9437e/targets - [ ] To automatically regenerate this PR, check this box.
Source-Link: googleapis/synthtool@7a220e2 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:6d4e3a15c62cfdcb823d60e16da7521e7c6fc00eba07c8ff12e4de9924a57d28
…s to v1 (#271) [![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [com.google.cloud:google-cloud-workflows](https://togithub.com/googleapis/java-workflows) | `0.4.2` -> `1.0.0` | [![age](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-workflows/1.0.0/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-workflows/1.0.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-workflows/1.0.0/compatibility-slim/0.4.2)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-workflows/1.0.0/confidence-slim/0.4.2)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>googleapis/java-workflows</summary> ### [`v1.0.0`](https://togithub.com/googleapis/java-workflows/blob/master/CHANGELOG.md#​100-httpswwwgithubcomgoogleapisjava-workflowscomparev042v100-2021-05-26) [Compare Source](https://togithub.com/googleapis/java-workflows/compare/v0.4.2...v1.0.0) ##### Features - promote to 1.0.0 ([#​267](https://www.github.com/googleapis/java-workflows/issues/267)) ([57f9b08](https://www.github.com/googleapis/java-workflows/commit/57f9b08ddda565f6d3209292df1a26a15daaebe8)) ##### [0.4.2](https://www.github.com/googleapis/java-workflows/compare/v0.4.1...v0.4.2) (2021-05-19) ##### Dependencies - update dependency com.google.cloud:google-cloud-shared-dependencies to v1.2.0 ([#​257](https://www.github.com/googleapis/java-workflows/issues/257)) ([98c0d36](https://www.github.com/googleapis/java-workflows/commit/98c0d36b8c531224dfea76c85eb2a35bc55e45d8)) ##### [0.4.1](https://www.github.com/googleapis/java-workflows/compare/v0.4.0...v0.4.1) (2021-05-14) ##### Dependencies - update dependency com.google.cloud:google-cloud-shared-dependencies to v1.1.0 ([#​249](https://www.github.com/googleapis/java-workflows/issues/249)) ([0326aa5](https://www.github.com/googleapis/java-workflows/commit/0326aa563bab44a93258fabe77e6d953a7c4e3c9)) </details> --- ### Configuration 📅 **Schedule**: At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻️ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box. --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-workflows).
🤖 I have created a release *beep* *boop* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
…ator_java versions (#271) - [ ] Regenerate this pull request now. PiperOrigin-RevId: 472750037 Source-Link: googleapis/googleapis@88f2ea3 Source-Link: https://github.com/googleapis/googleapis-gen/commit/230a5588306aae18fe8f2a57f14d4039ad72c901 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMjMwYTU1ODgzMDZhYWUxOGZlOGYyYTU3ZjE0ZDQwMzlhZDcyYzkwMSJ9
…1575) (#271) Source-Link: googleapis/synthtool@2e9ac19 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:8175681a918181d306d9c370d3262f16b4c724cc73d74111b7d42fc985ca7f93
🤖 I have created a release *beep* *boop* --- ## [2.3.4](googleapis/java-service-usage@v2.3.3...v2.3.4) (2022-09-15) ### Dependencies * Update dependency com.google.cloud:google-cloud-shared-dependencies to v3.0.3 ([#270](googleapis/java-service-usage#270)) ([54ce129](googleapis/java-service-usage@54ce129)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [2.2.6](googleapis/java-dms@v2.2.5...v2.2.6) (2022-09-29) ### Dependencies * Update dependency cachetools to v5 ([#288](googleapis/java-dms#288)) ([9370173](googleapis/java-dms@9370173)) * Update dependency certifi to v2022.9.24 ([#268](googleapis/java-dms#268)) ([1745004](googleapis/java-dms@1745004)) * Update dependency charset-normalizer to v2.1.1 ([#272](googleapis/java-dms#272)) ([367ffd9](googleapis/java-dms@367ffd9)) * Update dependency click to v8.1.3 ([#273](googleapis/java-dms#273)) ([328f00b](googleapis/java-dms@328f00b)) * Update dependency gcp-releasetool to v1.8.8 ([#269](googleapis/java-dms#269)) ([7d68646](googleapis/java-dms@7d68646)) * Update dependency google-api-core to v2.10.1 ([#274](googleapis/java-dms#274)) ([03bc3c5](googleapis/java-dms@03bc3c5)) * Update dependency google-auth to v2.12.0 ([#275](googleapis/java-dms#275)) ([bd96264](googleapis/java-dms@bd96264)) * Update dependency google-cloud-core to v2.3.2 ([#270](googleapis/java-dms#270)) ([fcd6bbd](googleapis/java-dms@fcd6bbd)) * Update dependency google-cloud-storage to v2.5.0 ([#276](googleapis/java-dms#276)) ([80b39f1](googleapis/java-dms@80b39f1)) * Update dependency google-crc32c to v1.5.0 ([#277](googleapis/java-dms#277)) ([a561425](googleapis/java-dms@a561425)) * Update dependency googleapis-common-protos to v1.56.4 ([#271](googleapis/java-dms#271)) ([296d07c](googleapis/java-dms@296d07c)) * Update dependency importlib-metadata to v4.12.0 ([#278](googleapis/java-dms#278)) ([b4509fa](googleapis/java-dms@b4509fa)) * Update dependency jeepney to v0.8.0 ([#279](googleapis/java-dms#279)) ([b2b6e8f](googleapis/java-dms@b2b6e8f)) * Update dependency jinja2 to v3.1.2 ([#280](googleapis/java-dms#280)) ([c7c2a10](googleapis/java-dms@c7c2a10)) * Update dependency keyring to v23.9.3 ([#281](googleapis/java-dms#281)) ([981c58c](googleapis/java-dms@981c58c)) * Update dependency markupsafe to v2.1.1 ([#282](googleapis/java-dms#282)) ([5a78aa3](googleapis/java-dms@5a78aa3)) * Update dependency protobuf to v3.20.2 ([#283](googleapis/java-dms#283)) ([2444bcd](googleapis/java-dms@2444bcd)) * Update dependency protobuf to v4 ([#289](googleapis/java-dms#289)) ([b9e330a](googleapis/java-dms@b9e330a)) * Update dependency pyjwt to v2.5.0 ([#284](googleapis/java-dms#284)) ([9fc617d](googleapis/java-dms@9fc617d)) * Update dependency requests to v2.28.1 ([#285](googleapis/java-dms#285)) ([51574b4](googleapis/java-dms@51574b4)) * Update dependency typing-extensions to v4.3.0 ([#286](googleapis/java-dms#286)) ([c36d416](googleapis/java-dms@c36d416)) * Update dependency zipp to v3.8.1 ([#287](googleapis/java-dms#287)) ([34e4986](googleapis/java-dms@34e4986)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [2.3.5](https://togithub.com/googleapis/java-essential-contacts/compare/v2.3.4...v2.3.5) (2022-10-03) ### Dependencies * Update dependency com.google.cloud:google-cloud-shared-dependencies to v3.0.4 ([#270](https://togithub.com/googleapis/java-essential-contacts/issues/270)) ([0836b2e](https://togithub.com/googleapis/java-essential-contacts/commit/0836b2e046202242d289c963393d4ab4cded4450)) --- This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ## [1.2.5](https://togithub.com/googleapis/java-policy-troubleshooter/compare/v1.2.4...v1.2.5) (2022-10-03) ### Dependencies * Update dependency cachetools to v5 ([#287](https://togithub.com/googleapis/java-policy-troubleshooter/issues/287)) ([2fea464](https://togithub.com/googleapis/java-policy-troubleshooter/commit/2fea464d4acf1cf78e01c58880053ff1d621e851)) * Update dependency certifi to v2022.9.24 ([#267](https://togithub.com/googleapis/java-policy-troubleshooter/issues/267)) ([9eeaf6c](https://togithub.com/googleapis/java-policy-troubleshooter/commit/9eeaf6c3d21e7c3eed8e50cd7bca445aba599313)) * Update dependency charset-normalizer to v2.1.1 ([#271](https://togithub.com/googleapis/java-policy-troubleshooter/issues/271)) ([9d3dfd8](https://togithub.com/googleapis/java-policy-troubleshooter/commit/9d3dfd8da1881a8d0c2dc3edd34ce21e4d8aa64b)) * Update dependency click to v8.1.3 ([#272](https://togithub.com/googleapis/java-policy-troubleshooter/issues/272)) ([5d92d7e](https://togithub.com/googleapis/java-policy-troubleshooter/commit/5d92d7e3551b2ca1b3ab6a91a8b6a589cb7c8028)) * Update dependency com.google.cloud:google-cloud-shared-dependencies to v3.0.4 ([#292](https://togithub.com/googleapis/java-policy-troubleshooter/issues/292)) ([d47d642](https://togithub.com/googleapis/java-policy-troubleshooter/commit/d47d64222138b45ed5375bf3e9bf9e76cf8971ad)) * Update dependency gcp-releasetool to v1.8.8 ([#268](https://togithub.com/googleapis/java-policy-troubleshooter/issues/268)) ([a75ee9b](https://togithub.com/googleapis/java-policy-troubleshooter/commit/a75ee9be6a2ac6b9fe573d9d086a94d325ddd814)) * Update dependency google-api-core to v2.10.1 ([#273](https://togithub.com/googleapis/java-policy-troubleshooter/issues/273)) ([edf53b7](https://togithub.com/googleapis/java-policy-troubleshooter/commit/edf53b7785d559cb1f0b35a83982d8b36c7b1c6a)) * Update dependency google-auth to v2.12.0 ([#274](https://togithub.com/googleapis/java-policy-troubleshooter/issues/274)) ([1b4d43f](https://togithub.com/googleapis/java-policy-troubleshooter/commit/1b4d43facb6739b12cf4c037333237af206f6524)) * Update dependency google-cloud-core to v2.3.2 ([#269](https://togithub.com/googleapis/java-policy-troubleshooter/issues/269)) ([9e9681a](https://togithub.com/googleapis/java-policy-troubleshooter/commit/9e9681a4c0da7aa7e402584a1ac2219558b5352e)) * Update dependency google-cloud-storage to v2.5.0 ([#275](https://togithub.com/googleapis/java-policy-troubleshooter/issues/275)) ([7280eaf](https://togithub.com/googleapis/java-policy-troubleshooter/commit/7280eafabd013c704cc63a042fab3255bee55111)) * Update dependency google-crc32c to v1.5.0 ([#276](https://togithub.com/googleapis/java-policy-troubleshooter/issues/276)) ([b144928](https://togithub.com/googleapis/java-policy-troubleshooter/commit/b1449280a8c1e8caef134aae7b62a1c4f7b5b5b6)) * Update dependency googleapis-common-protos to v1.56.4 ([#270](https://togithub.com/googleapis/java-policy-troubleshooter/issues/270)) ([942ea39](https://togithub.com/googleapis/java-policy-troubleshooter/commit/942ea39d9b5f655736b0ce0c7c532803dffa4484)) * Update dependency importlib-metadata to v4.12.0 ([#277](https://togithub.com/googleapis/java-policy-troubleshooter/issues/277)) ([bf03ef8](https://togithub.com/googleapis/java-policy-troubleshooter/commit/bf03ef8a75846e74b0b509a518caf85dcea2b901)) * Update dependency jeepney to v0.8.0 ([#278](https://togithub.com/googleapis/java-policy-troubleshooter/issues/278)) ([f51e9e7](https://togithub.com/googleapis/java-policy-troubleshooter/commit/f51e9e724f088a864dd70deeef16edf94922c246)) * Update dependency jinja2 to v3.1.2 ([#279](https://togithub.com/googleapis/java-policy-troubleshooter/issues/279)) ([95f9152](https://togithub.com/googleapis/java-policy-troubleshooter/commit/95f91525eb3ff0d57a3639b0f66cf2be6b1f0f8b)) * Update dependency keyring to v23.9.3 ([#280](https://togithub.com/googleapis/java-policy-troubleshooter/issues/280)) ([33d8606](https://togithub.com/googleapis/java-policy-troubleshooter/commit/33d8606ea8da63f7cffb9c38ad5e52a52276e5cc)) * Update dependency protobuf to v4 ([#288](https://togithub.com/googleapis/java-policy-troubleshooter/issues/288)) ([8a836ed](https://togithub.com/googleapis/java-policy-troubleshooter/commit/8a836edb54c0cd182dfb4ee4e29bbb910d452150)) * Update dependency pyjwt to v2.5.0 ([#283](https://togithub.com/googleapis/java-policy-troubleshooter/issues/283)) ([a4483cf](https://togithub.com/googleapis/java-policy-troubleshooter/commit/a4483cf44a22ba571a6378587270aff46c76e544)) * Update dependency typing-extensions to v4.3.0 ([#285](https://togithub.com/googleapis/java-policy-troubleshooter/issues/285)) ([2ccd257](https://togithub.com/googleapis/java-policy-troubleshooter/commit/2ccd257da7680d3952f7dcfc29633448fbc91c74)) * Update dependency zipp to v3.8.1 ([#286](https://togithub.com/googleapis/java-policy-troubleshooter/issues/286)) ([0270359](https://togithub.com/googleapis/java-policy-troubleshooter/commit/0270359b0316fe14ddd700120485929acb97fb8a)) --- This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
🤖 I have created a release *beep* *boop* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
…s#1687) (googleapis#271) * chore(java): add a note in README for migrated split repos Disable renovate bot and flaky bot for split repositories that have moved to the Java monorepo. The Java monorepo will pass the "monorepo=True" parameter to java.common_templates method in its owlbot.py files so that the migration note will not appear in the README in the monorepo. Co-authored-by: Jeff Ching <[email protected]> Source-Link: https://togithub.com/googleapis/synthtool/commit/d4b291604f148cde065838c498bc8aa79b8dc10e Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:edae91ccdd2dded2f572ec341a768ad180305a3e8fbfd93064b28e237d35920a
This PR fixes #267. Main changes are:
BlobWriteOption
with support formd5Match
andcrc32cMatch
create(blobInfo, content, options)
BlobTargetOption...
toBlobWriteOption...
in create from stream writerTo note that
md5Match
andcrc32cMatch
are boolean options always true. If not specified storageobject'scrc32c
andmd5
fields are set to null (so no validation occurs).Alternatives to boolean options were:
BlobInfo
: this would have required to modifystorage.optionMap
, that is now used to fetch implicit values only for (meta)generation matches. (meta)generation options are shared across all APIs, so I felt likemd5Match
andcrc32cMatch
did not fit here.BlobWriteOption.md5Match("7uyg8yg8==")
), still a valid option but a bit counter-intuitive: all values set inBlobInfo
are transmitted, why not to usemd5
andcrc32c
fields?Comments are appreciated!