Support withFormatRecordOnFailureFunction() for BigQuery STORAGE_WRITE_API and STORAGE_API_AT_LEAST_ONCE methods #31659
Conversation
Amar3tto
force-pushed
the
bigquery-storageapi-format-failure
branch
from
d9e87e0 to
e01f88c
Compare
|
R: @Abacn |
|
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control |
|
R: @sarinasij FYI one can follow https://github.com/apache/beam/blob/master/contributor-docs/code-change-guide.md#run-your-pipeline-with-modified-beam-code to test this PR with your use case |
|
R: @ahmedabu98 |
|
@Abacn Could you please help with the review process? |
|
Didn't get a chance to go through the change. I see the author of the FR @sarinasij is responsive. Would it be possible for you testing the change with your use case, to see if this unblocks you? |
|
@ahmedabu98 Could you please review? |
| .setPrimitiveBytes( | ||
| ByteString.copyFrom(Integer.toString(i).getBytes(StandardCharsets.UTF_8))) | ||
| .build(); | ||
| List<Proto3SchemaMessages.Primitive> goodRows = |
There was a problem hiding this comment.
nit: --> s/goodRows/inputRows
|
CC @johnjcasey @AnandInguva I believe this does what y'all wanted to do in #31106 |
|
@Amar3tto can we add a comment in CHANGES.md calling out this new feature? |
Amar3tto
force-pushed
the
bigquery-storageapi-format-failure
branch
from
e01f88c to
0739334
Compare
…E_API and STORAGE_API_AT_LEAST_ONCE methods
Amar3tto
force-pushed
the
bigquery-storageapi-format-failure
branch
from
0739334 to
0a492b1
Compare
Done |
|
Run Java_GCP_IO_Direct PreCommit |
…E_API and STORAGE_API_AT_LEAST_ONCE methods (apache#31659) * Support withFormatRecordOnFailureFunction() for BigQuery STORAGE_WRITE_API and STORAGE_API_AT_LEAST_ONCE methods * Update CHANGES.md
Fixes #31354
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.