[BEAM-13321] Fix exception with BigQuery StreamWriter TraceID. #16599
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When running a BigQuery IO transform with a query and query location set, was given the exception: "java.lang.IllegalArgumentException: TraceId must follow the format of A:B. Actual:Dataflow". Documentation shows an example of using it with the format "Dataflow:job_id" which is what I changed this to: https://cloud.google.com/java/docs/reference/google-cloud-bigquerystorage/latest/com.google.cloud.bigquery.storage.v1.StreamWriter.Builder#com_google_cloud_bigquery_storage_v1_StreamWriter_Builder_setTraceId_java_lang_String_
|
R: @chamikaramj @ihji (Whoever gets to it first) |
|
cc: @reuvenlax |
|
LGTM, it seems we should accept both A and A:B in our client lib though...
…On Tue, Jan 25, 2022 at 5:13 PM Chamikara Jayalath ***@***.***> wrote:
cc: @reuvenlax <https://github.com/reuvenlax>
—
Reply to this email directly, view it on GitHub
<#16599 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHGYVEXQ5UX3WRKZGET62J3UX5DETANCNFSM5MWMUIZQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Thanks.
Yiru
|
yirutang
approved these changes
Jan 26, 2022
|
Is this a general breakage in the STORAGE_WRITE_API mode when using BQ sink ? In that case we should cherry-pick to the ongoing release. |
|
And it needs to be cherry picked.
…On Tue, Jan 25, 2022 at 5:21 PM Yiru Tang ***@***.***> wrote:
LGTM, it seems we should accept both A and A:B in our client lib though...
On Tue, Jan 25, 2022 at 5:13 PM Chamikara Jayalath <
***@***.***> wrote:
> cc: @reuvenlax <https://github.com/reuvenlax>
>
> —
> Reply to this email directly, view it on GitHub
> <#16599 (comment)>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AHGYVEXQ5UX3WRKZGET62J3UX5DETANCNFSM5MWMUIZQ>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
--
Thanks.
Yiru
--
Thanks.
Yiru
|
|
Thanks. cc: @emilymye |
4 tasks
|
Created cherry-pick #16616 |
|
@yirutang also noting that we don't have integration testing for STORAGE_WRITE_API write mode which should have caught this early. Also mentioned in https://issues.apache.org/jira/browse/BEAM-11648. |
|
Yes, could you give me instructions on how to add an integration test for
dataflow connector? Or it can be added by your team?
…On Tue, Jan 25, 2022 at 5:35 PM Chamikara Jayalath ***@***.***> wrote:
@yirutang <https://github.com/yirutang> also noting that we don't have
integration testing for STORAGE_WRITE_API write mode which should have
caught this early. Also mentioned in
https://issues.apache.org/jira/browse/BEAM-11648.
—
Reply to this email directly, view it on GitHub
<#16599 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHGYVEW3MOOIWJGYTP5M3MLUX5FUZANCNFSM5MWMUIZQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Thanks.
Yiru
|
|
Created https://issues.apache.org/jira/browse/BEAM-13747 with some info. |
|
Thanks Chamikara,
Our team can look into picking this up.
…On Wed, Jan 26, 2022 at 8:58 AM Chamikara Jayalath ***@***.***> wrote:
Created https://issues.apache.org/jira/browse/BEAM-13747 with some info.
Our team may not have time to work on this immediately so appreciate if
you or Reuven can add this.
—
Reply to this email directly, view it on GitHub
<#16599 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHGYVEW7GB2EVJCKPJ6LPIDUYAR3VANCNFSM5MWMUIZQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Thanks.
Yiru
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When running a BigQuery IO transform with a query and query location set, was given the exception: "java.lang.IllegalArgumentException: TraceId must follow the format of A:B. Actual:Dataflow". Documentation shows an example of using it with the format "Dataflow:job_id" which is what I changed this to: https://cloud.google.com/java/docs/reference/google-cloud-bigquerystorage/latest/com.google.cloud.bigquery.storage.v1.StreamWriter.Builder#com_google_cloud_bigquery_storage_v1_StreamWriter_Builder_setTraceId_java_lang_String_
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
R: @username).[BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replaceBEAM-XXXwith the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
ValidatesRunnercompliance status (on master branch)Examples testing status on various runners
Post-Commit SDK/Transform Integration Tests Status (on master branch)
Pre-Commit Tests Status (on master branch)
See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI.