Skip to content

Update bigquery.py (#31620) #16309

Update bigquery.py (#31620)

Update bigquery.py (#31620) #16309

GitHub Actions / Test Results failed Jun 18, 2024 in 0s

1 fail, 65 pass in 35m 15s

66 tests   65 ✅  35m 15s ⏱️
11 suites   0 💤
11 files     1 ❌

Results for commit 3d05eec.

Annotations

Check warning on line 0 in org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkRowUpdateIT

See this annotation in the file changed.

@github-actions github-actions / Test Results

testCdcUsingHexSequenceNum (org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkRowUpdateIT) failed

sdks/java/io/google-cloud-platform/build/test-results/bigQueryEarlyRolloutIntegrationTest/TEST-org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkRowUpdateIT.xml [took 2s]
Raw output
org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.lang.RuntimeException: Append to stream projects/apache-beam-testing/datasets/storage_api_sink_rows_update109932829101572/tables/table109933662863807/streams/_default failed with Status Code FAILED_PRECONDITION. The stream may not exist.
	at org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:377)
	at org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:345)
	at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:218)
	at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:67)
	at org.apache.beam.sdk.Pipeline.run(Pipeline.java:325)
	at org.apache.beam.sdk.Pipeline.run(Pipeline.java:310)
	at org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkRowUpdateIT.testCdcUsingHexSequenceNum(StorageApiSinkRowUpdateIT.java:184)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:112)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:40)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:60)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:52)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Caused by: java.lang.RuntimeException: Append to stream projects/apache-beam-testing/datasets/storage_api_sink_rows_update109932829101572/tables/table109933662863807/streams/_default failed with Status Code FAILED_PRECONDITION. The stream may not exist.
	at org.apache.beam.sdk.io.gcp.bigquery.StorageApiWriteUnshardedRecords$WriteRecordsDoFn$DestinationState.lambda$flush$8(StorageApiWriteUnshardedRecords.java:789)
	at org.apache.beam.sdk.io.gcp.bigquery.RetryManager.await(RetryManager.java:311)
	at org.apache.beam.sdk.io.gcp.bigquery.StorageApiWriteUnshardedRecords$WriteRecordsDoFn.flushAll(StorageApiWriteUnshardedRecords.java:977)
	at org.apache.beam.sdk.io.gcp.bigquery.StorageApiWriteUnshardedRecords$WriteRecordsDoFn.finishBundle(StorageApiWriteUnshardedRecords.java:1125)
Caused by: com.google.cloud.bigquery.storage.v1.Exceptions$StreamWriterClosedException: FAILED_PRECONDITION: Connection is closed due to com.google.api.gax.rpc.PermissionDeniedException: io.grpc.StatusRuntimeException: PERMISSION_DENIED: Permission 'TABLES_UPDATE_DATA' denied on resource 'projects/apache-beam-testing/datasets/storage_api_sink_rows_update109932829101572/tables/table109933662863807' (or it may not exist).
	at com.google.cloud.bigquery.storage.v1.ConnectionWorker.appendInternal(ConnectionWorker.java:536)
	at com.google.cloud.bigquery.storage.v1.ConnectionWorker.append(ConnectionWorker.java:479)
	at com.google.cloud.bigquery.storage.v1.StreamWriter$SingleConnectionOrConnectionPool.append(StreamWriter.java:169)
	at com.google.cloud.bigquery.storage.v1.StreamWriter.append(StreamWriter.java:459)
	at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$WriteStreamServiceImpl$1.appendRows(BigQueryServicesImpl.java:1480)
	at org.apache.beam.sdk.io.gcp.bigquery.StorageApiWriteUnshardedRecords$WriteRecordsDoFn$DestinationState.lambda$flush$7(StorageApiWriteUnshardedRecords.java:648)
	at org.apache.beam.sdk.io.gcp.bigquery.RetryManager$Operation.run(RetryManager.java:166)
	at org.apache.beam.sdk.io.gcp.bigquery.RetryManager.run(RetryManager.java:292)
	at org.apache.beam.sdk.io.gcp.bigquery.RetryManager.await(RetryManager.java:324)
	at org.apache.beam.sdk.io.gcp.bigquery.StorageApiWriteUnshardedRecords$WriteRecordsDoFn.flushAll(StorageApiWriteUnshardedRecords.java:977)
	at org.apache.beam.sdk.io.gcp.bigquery.StorageApiWriteUnshardedRecords$WriteRecordsDoFn.finishBundle(StorageApiWriteUnshardedRecords.java:1125)
	at org.apache.beam.sdk.io.gcp.bigquery.StorageApiWriteUnshardedRecords$WriteRecordsDoFn$DoFnInvoker.invokeFinishBundle(Unknown Source)
	at org.apache.beam.runners.core.SimpleDoFnRunner.finishBundle(SimpleDoFnRunner.java:222)
	at org.apache.beam.runners.core.SimplePushbackSideInputDoFnRunner.finishBundle(SimplePushbackSideInputDoFnRunner.java:130)
	at org.apache.beam.runners.direct.ParDoEvaluator.finishBundle(ParDoEvaluator.java:269)
	at org.apache.beam.runners.direct.DoFnLifecycleManagerRemovingTransformEvaluator.finishBundle(DoFnLifecycleManagerRemovingTransformEvaluator.java:73)
	at org.apache.beam.runners.direct.DirectTransformExecutor.finishBundle(DirectTransformExecutor.java:193)
	at org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:131)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)

Check notice on line 0 in .github

See this annotation in the file changed.

@github-actions github-actions / Test Results

66 tests found

There are 66 tests, see "Raw output" for the full list of tests.
Raw output
org.apache.beam.sdk.io.gcp.bigquery.BigQueryIOJsonIT ‑ testFileLoadWriteExportRead
org.apache.beam.sdk.io.gcp.bigquery.BigQueryIOJsonIT ‑ testLegacyStreamingWriteDefaultRead
org.apache.beam.sdk.io.gcp.bigquery.BigQueryIOJsonIT ‑ testQueryRead
org.apache.beam.sdk.io.gcp.bigquery.BigQueryIOJsonIT ‑ testStorageWriteRead
org.apache.beam.sdk.io.gcp.bigquery.BigQueryIOStorageReadTableRowIT ‑ testBigQueryStorageReadTableRow100
org.apache.beam.sdk.io.gcp.bigquery.BigQueryIOStorageReadTableRowIT ‑ testBigQueryStorageReadTableRow10k
org.apache.beam.sdk.io.gcp.bigquery.BigQueryIOStorageReadTableRowIT ‑ testBigQueryStorageReadTableRow1k
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySchemaUpdateOptionsIT ‑ runWriteTestTempTableAndDynamicDestination
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySchemaUpdateOptionsIT ‑ testAllowFieldAddition
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySchemaUpdateOptionsIT ‑ testAllowFieldRelaxation
org.apache.beam.sdk.io.gcp.bigquery.BigQueryTimePartitioningClusteringIT ‑ testE2EBigQueryClustering
org.apache.beam.sdk.io.gcp.bigquery.BigQueryTimePartitioningClusteringIT ‑ testE2EBigQueryClusteringDynamicDestinations
org.apache.beam.sdk.io.gcp.bigquery.BigQueryTimePartitioningClusteringIT ‑ testE2EBigQueryClusteringTableFunction
org.apache.beam.sdk.io.gcp.bigquery.BigQueryTimePartitioningClusteringIT ‑ testE2EBigQueryTimePartitioning
org.apache.beam.sdk.io.gcp.bigquery.BigQueryToTableIT ‑ testLegacyQueryWithoutReshuffle
org.apache.beam.sdk.io.gcp.bigquery.BigQueryToTableIT ‑ testNewTypesQueryWithReshuffle
org.apache.beam.sdk.io.gcp.bigquery.BigQueryToTableIT ‑ testNewTypesQueryWithoutReshuffle
org.apache.beam.sdk.io.gcp.bigquery.BigQueryToTableIT ‑ testStandardQueryWithoutCustom
org.apache.beam.sdk.io.gcp.bigquery.FileLoadsStreamingIT ‑ testDynamicDestinationsWithAutoShardingAndCopyJobs[0]
org.apache.beam.sdk.io.gcp.bigquery.FileLoadsStreamingIT ‑ testDynamicDestinationsWithAutoShardingAndCopyJobs[1]
org.apache.beam.sdk.io.gcp.bigquery.FileLoadsStreamingIT ‑ testDynamicDestinationsWithFixedShards[0]
org.apache.beam.sdk.io.gcp.bigquery.FileLoadsStreamingIT ‑ testDynamicDestinationsWithFixedShards[1]
org.apache.beam.sdk.io.gcp.bigquery.FileLoadsStreamingIT ‑ testLoadWithAutoShardingAndCopyJobs[0]
org.apache.beam.sdk.io.gcp.bigquery.FileLoadsStreamingIT ‑ testLoadWithAutoShardingAndCopyJobs[1]
org.apache.beam.sdk.io.gcp.bigquery.FileLoadsStreamingIT ‑ testLoadWithFixedShards[0]
org.apache.beam.sdk.io.gcp.bigquery.FileLoadsStreamingIT ‑ testLoadWithFixedShards[1]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiDirectWriteProtosIT ‑ testDirectWriteProtos[0]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiDirectWriteProtosIT ‑ testDirectWriteProtos[1]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiDirectWriteProtosIT ‑ testDirectWriteProtos[2]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiDirectWriteProtosIT ‑ testDirectWriteProtos[3]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkFailedRowsIT ‑ testInvalidRowCaughtByBigquery[0]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkFailedRowsIT ‑ testInvalidRowCaughtByBigquery[1]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkFailedRowsIT ‑ testInvalidRowCaughtByBigquery[2]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkFailedRowsIT ‑ testInvalidRowCaughtByBigquery[3]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkFailedRowsIT ‑ testSchemaMismatchCaughtByBeam[0]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkFailedRowsIT ‑ testSchemaMismatchCaughtByBeam[1]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkFailedRowsIT ‑ testSchemaMismatchCaughtByBeam[2]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkFailedRowsIT ‑ testSchemaMismatchCaughtByBeam[3]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkRowUpdateIT ‑ testCdcUsingHexSequenceNum
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkRowUpdateIT ‑ testCdcUsingLongSeqNum
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testAtLeastOnceWithAutoSchemaUpdate[0]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testAtLeastOnceWithAutoSchemaUpdate[1]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testAtLeastOnceWithAutoSchemaUpdate[2]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testAtLeastOnceWithAutoSchemaUpdate[3]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testAtLeastOnceWithIgnoreUnknownValues[0]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testAtLeastOnceWithIgnoreUnknownValues[1]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testAtLeastOnceWithIgnoreUnknownValues[2]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testAtLeastOnceWithIgnoreUnknownValues[3]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testAtLeastOnce[0]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testAtLeastOnce[1]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testAtLeastOnce[2]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testAtLeastOnce[3]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testExactlyOnceWithAutoSchemaUpdate[0]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testExactlyOnceWithAutoSchemaUpdate[1]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testExactlyOnceWithAutoSchemaUpdate[2]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testExactlyOnceWithAutoSchemaUpdate[3]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testExactlyOnceWithIgnoreUnknownValues[0]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testExactlyOnceWithIgnoreUnknownValues[1]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testExactlyOnceWithIgnoreUnknownValues[2]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testExactlyOnceWithIgnoreUnknownValues[3]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testExactlyOnce[0]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testExactlyOnce[1]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testExactlyOnce[2]
org.apache.beam.sdk.io.gcp.bigquery.StorageApiSinkSchemaUpdateIT ‑ testExactlyOnce[3]
org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProtoIT ‑ testBaseTableRow
org.apache.beam.sdk.io.gcp.bigquery.TableRowToStorageApiProtoIT ‑ testNestedRichTypesAndNull