Making the default BigQuery write disposition to be WRITE_APPEND (apa…

…che#25434) * Added element counter and error counter for BQ write schema transform * Fixied styling issues with naming * Combined two trivial counter class for brevity. Used finishbundle annotation to reduce the number of calls to counter.inc() for better performance. * fix formatting * change the default write disposition to write append instead of write empty --------- Co-authored-by: Nick Li <[email protected]>
akvelon · Mar 10, 2023 · 6d8ccce · 6d8ccce
1 parent cd5863d
commit 6d8ccce
Showing 1 changed file with 2 additions and 1 deletion.
diff --git a/...he/beam/sdk/io/gcp/bigquery/providers/BigQueryStorageWriteApiSchemaTransformProvider.java b/...he/beam/sdk/io/gcp/bigquery/providers/BigQueryStorageWriteApiSchemaTransformProvider.java
@@ -316,7 +316,8 @@ BigQueryIO.Write<Row> createStorageWriteApiTransform() {
               .to(configuration.getTable())
               .withMethod(writeMethod)
               .useBeamSchema()
-              .withFormatFunction(BigQueryUtils.toTableRow());
+              .withFormatFunction(BigQueryUtils.toTableRow())
+              .withWriteDisposition(WriteDisposition.WRITE_APPEND);
 
       if (!Strings.isNullOrEmpty(configuration.getCreateDisposition())) {
         CreateDisposition createDisposition =