Modify batch IT to use count instead of hash #26327

johnjcasey · 2023-04-18T14:31:00Z

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

johnjcasey · 2023-04-18T14:31:07Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-18T15:24:11Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-18T15:33:51Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-18T16:07:51Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-18T17:54:55Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-18T21:05:08Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-19T14:21:33Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-19T17:13:18Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-19T19:12:57Z

run seed job

johnjcasey · 2023-04-19T19:27:24Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-19T20:31:20Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-20T15:00:14Z

Run Java KafkaIO Performance Test

…cords

johnjcasey · 2023-04-20T18:10:25Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-20T18:18:58Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-20T18:48:15Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-20T19:45:30Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-20T20:11:18Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-20T20:16:45Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-26T19:10:30Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-26T21:19:41Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-27T13:41:22Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-27T15:31:20Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-27T15:35:36Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-27T17:58:27Z

Run Java KafkaIO Performance Test

johnjcasey · 2023-04-27T20:09:52Z

@Abacn for review

github-actions · 2023-04-27T21:06:01Z

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

github-actions · 2023-04-28T14:35:11Z

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @Abacn for label java.
R: @Abacn for label io.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

This reverts commit 9903b2f.

#26466)" This reverts commit 5c67668.

johnjcasey · 2023-04-28T14:53:06Z

I accidentally merged this, and quickly reverted it. This is still the right place for review, and I'll open a revert of revert pr: #26467

#26466)" (#26467) This reverts commit 5c67668.

* Modify batch IT to use count instead of hash * remove unused varaiable * run spotless, update pipeline state checking * Update timeout to 45 minutes * revert timeout, add additional counter to try and pinpoint missing records * add a log to notify ranges used when workers restart * change counts from metrics to combiners * add a window to streaming test * move the passert to the correct place * Remove extra counter, apply spotless * add additional metric to KafkaWriter * Remove debugging metrics * verify pipeline is not failed * remove extra newline

…pache#26466) This reverts commit 9903b2f.

…26327)" (apache#26466)" (apache#26467) This reverts commit 5c67668.

* Add channel split separations. * Use channel leases for parallel processing. * move data wait to chan + atomic * Progress and split on execute goroutine. * Convert metrics tests to short ids. * Add Provision handler w/ capabilities. * Add artifact validation to run_rc_validation (#26407) * Add artifact validation to run_rc_validation * config file for artifacts * use java bom contents for validation * add pip_pre = True * Update tox.ini * Remove out of date experimental warning on SDFs (#26450) * [ToBF] Refinement 26.04.23 (#26428) * content tree rebuilds on sdk change * expandable parent node widget * comment fix * comment fix --------- Co-authored-by: darkhan.nausharipov <[email protected]> * Allow implicit flattening for yaml inputs. (#26423) * Add field annotations for high-priority Syndeo schema transforms (#26384) add changes * [Go SDK] Timers with new datalayer (#26101) * added timer package * add timer changes and merged with rebo's pr * timer fired in stateful * error setting new timer in ontimer * looping timers work * send fv instead of bytes * changes to coder/pardo * works for all cases, only cleanup left * remove comments and validate onTimer * generic coder for user key * fixes coder end to end * remove logs * add unit test and refactor * add docs * new example * fix static lint * support emitters * allow input col of CoGBK as well * unit tests, periodic impulse, minor refactor * update PipelineTimer interface, minor refactor, doc comment for example * add warn message * single edge timer coder, rm kv coder check, cache encoder,decoder * Update chromedriver-binary requirement in /sdks/python Updates the requirements on [chromedriver-binary](https://github.com/danielkaiser/python-chromedriver-binary) to permit the latest version. - [Release notes](https://github.com/danielkaiser/python-chromedriver-binary/releases) - [Commits](danielkaiser/python-chromedriver-binary@v100.0.4896.20.0...v113.0.5672.24.0) --- updated-dependencies: - dependency-name: chromedriver-binary dependency-type: direct:development ... Signed-off-by: dependabot[bot] <[email protected]> * Automation: Tour of Beam infrastructure deployment (#25793) * Added Terraform scripts for TOB infra * ToB Frontend related updates * Update settings.gradle.kts * Deleted redundant file and minor README change * Addressing comments in the PR * Added newline at the end of variables.tf file * Update README.md * Updates related to Tour of Beam infrastructure * Update locals.tf * Output.tf updates * Update output.tf * Updates * Update main.tf * Updates to cloudfunctions_bucket variable * service_account_id changes * Update main.tf * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Bulk update of terraform scripts * Update README.md * Update README.md * Datastore_namespace updates * Update README.md * Update README.md * Update README.md * Update README.md * Update main.tf * Update README.md * Update README.md * Update README.md * Some minor TF updates * Update README.md * Modify batch IT to use count instead of hash (#26327) * Modify batch IT to use count instead of hash * remove unused varaiable * run spotless, update pipeline state checking * Update timeout to 45 minutes * revert timeout, add additional counter to try and pinpoint missing records * add a log to notify ranges used when workers restart * change counts from metrics to combiners * add a window to streaming test * move the passert to the correct place * Remove extra counter, apply spotless * add additional metric to KafkaWriter * Remove debugging metrics * verify pipeline is not failed * remove extra newline * Revert "Modify batch IT to use count instead of hash (#26327)" (#26466) This reverts commit 9903b2f. * Bump Java Dataflow container images (#26459) * keep retrying mass_comment until it has started all jobs (#26457) * keep retrying mass_comment until it has started all jobs * fix lookups * Add driverJars parameter to JdbcIO. (#25824) This change allows users to use driver jars saved in GCS. With this change, Dataflow templates will be able to migrate to JdbcIO instead of DynamicJdbcIO. * [Roll Fwd PR] Rename _namespace to _get_display_data_namespace"" (#26470) * Update parquetio and textio to work with -beam_strict (#26469) * use wheel sdk location for PostCommit_Py_Examples (#26473) * Move back the timeout of Python PostCommit to 4h * Minor fix on Python PostCommit description strings * Add recent postcommits to jenkins README * More user-friendly providers. * Add yaml preprocessing phases. * Add flexible windowing syntax to yaml. * Implement flatten in terms of preprocessor phase. This composes better with windowing. * Reword SQL note. * Make linter happy. * Survive errors in size estimation in MongoDbIO * Fix jdbc xlang schema type mismatch (#26480) * Fix jdbc xlang schema type mismatch * Also fix fetch_size type mismatch * Add new fields in the end * [Python] Add saved_weights example to tf notebook (#26472) * add saved_weights example to tf notebook * add description * updated text blocks * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Rebecca Szper <[email protected]> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Rebecca Szper <[email protected]> --------- Co-authored-by: Rebecca Szper <[email protected]> * Revert "Revert "Modify batch IT to use count instead of hash (#26327)" (#26466)" (#26467) This reverts commit 5c67668. * Bump github.com/tetratelabs/wazero from 1.0.3 to 1.1.0 in /sdks (#26486) Bumps [github.com/tetratelabs/wazero](https://github.com/tetratelabs/wazero) from 1.0.3 to 1.1.0. - [Release notes](https://github.com/tetratelabs/wazero/releases) - [Commits](tetratelabs/wazero@v1.0.3...v1.1.0) --- updated-dependencies: - dependency-name: github.com/tetratelabs/wazero dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * tour of beam integration tests (#25925) * Integration test to load the default example of the default SDK and change the example (#24730) (#24729) * Fix formatting and README (#24730) * Support collection v1.17.0 (#24730) * LoadingIndicator on chaning examples, remove duplicating licenses (#24730) * Add a missing license header (#24730) * Integration test for changing SDK and running code (#24779) (#382) * Integration test for changing SDK and running code (#24779) * Rename an integration test (#24779) * Use enum to switch SDK in integration test (#24779) * Find SDK in a dropdown by key (#24779) * Add a TODO (#24779) * Fix exports (#24779) * Issue24779 integration changing sdk from 24370 (#387) * Integration test for changing SDK and running code (#24779) * Rename an integration test (#24779) * Use enum to switch SDK in integration test (#24779) * Find SDK in a dropdown by key (#24779) * Add a TODO (#24779) * Fix exports (#24779) * Integration tests miscellaneous UI (#383) * miscellaneous ui integration tests * reverted pubspec.lock * gradle tasks ordered alhpabetically * integration tests refactoring * clean code * integration tests miscellaneous ui fix pr * rename method * added layout adaptivity * A minor cleanup (#24779) Co-authored-by: Dmitry Repin <[email protected]> * integration tests run and editing * example selector test * minor fixes * rat * fix pr * minor * minor * rat * integration test finder written * integration test minor fixes * minor fixes * removed comment * minor fixes * playground integration tests minor fixes * integration test pumpAnSettleNoException * integration test shortcut refactor * integration test another changing shortcuts running * upgrade to flutter 3.7.1 * workaround comment * playground frontend updated major versions * issues 25329 25331 25336 * 25329 extract connectivity extension to separate file * Upgrade Flutter to 3.7.3 in integration tests (#24730) * Fix integration test (#24730) * fix cors issue and added mouse scroll to tags * Upgrade Flutter in Dockerfile (#24720) * sorting moved to model * sorting moved to model * sorting moved to model * bugs fix * issue 25278 * fix pr * quites fix in en.yaml * Fix not loading default example (#25528) * fix compile error * Refactor output tabs, test embedded playground (#25136) (#439) * Refactor output tabs, test embedded playground (#25136) * Clean up (#25136) * Change example paths to IDs in integration tests * issue25640 tob ci * fix tob ci * rename ci process * test add new line to main * test add new line to main * commented unit test run * issue25640 changed server path * issue25640 tests on welcome page * deleted config.g.dart * issue25640 pr fixes * Update .github/workflows/tour_of_beam_frontend_test.yml Co-authored-by: alexeyinkin <[email protected]> * Update learning/tour-of-beam/frontend/integration_test/welcome_page_test.dart Co-authored-by: alexeyinkin <[email protected]> * Improve tests (#25640) * issue25640 tour page tests * pr fix * removed import * pr fix * fix test * 25640 fixed pubspec.lock * issue25640 fix readme * updated readme * issue25640 fixed after master merge * issue25483 ToB pipeline options * removed unnecesary variable * pr fix * Update learning/tour-of-beam/frontend/assets/translations/en.yaml Co-authored-by: alexeyinkin <[email protected]> * playground hides when snippet does not exists * pipeline options extracted to playground components * issue25483 pipeline options * added errors handling, fix pr * refactoring * Revert "refactoring" This reverts commit 1540961. * removed unnecessary constants * playground controller in tour notifier becomes nullable * playground controller returned to non nullable in tour notifier * playground controller actions * removed unnecessary code * tob scaffold wrapped with animated builder * minor fixes * partially fixed tests * Upgrade flutter_code_editor to v0.2.19 (#25640) * Replace output SelectableText with a CodeField instance (#25640) * Trigger ToB integration tests (#25640) * Clean up (#25640) * Enable manual workflow runs for Playground and ToB integration tests (#25640) --------- Co-authored-by: Alexey Inkin <[email protected]> Co-authored-by: alexeyinkin <[email protected]> * Eliminate nullness errors from MongoDbIO * Move provisioned options outside of harness.Main (#26476) Co-authored-by: lostluck <[email protected]> * Add SDF adjacent element test * Consolidate residual processing * fix race condition on split boundaries * deflake fixed window combine tests * Add comments to job functions. * Make int64check per window. * Update sdks/go/pkg/beam/runners/prism/internal/worker/bundle.go Co-authored-by: Ritesh Ghorse <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: lostluck <[email protected]> Co-authored-by: Danny McCormick <[email protected]> Co-authored-by: Anand Inguva <[email protected]> Co-authored-by: Anand Inguva <[email protected]> Co-authored-by: Darkhan Nausharipov <[email protected]> Co-authored-by: darkhan.nausharipov <[email protected]> Co-authored-by: Robert Bradshaw <[email protected]> Co-authored-by: Andrei Gurau <[email protected]> Co-authored-by: Ritesh Ghorse <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: ruslan-ikhsan <[email protected]> Co-authored-by: johnjcasey <[email protected]> Co-authored-by: Yi Hu <[email protected]> Co-authored-by: Pranav Bhandari <[email protected]> Co-authored-by: Svetak Sundhar <[email protected]> Co-authored-by: Jeremy Edwards <[email protected]> Co-authored-by: Kenneth Knowles <[email protected]> Co-authored-by: Rebecca Szper <[email protected]> Co-authored-by: Dmitry Repin <[email protected]> Co-authored-by: Alexey Inkin <[email protected]> Co-authored-by: alexeyinkin <[email protected]>

* Add channel split separations. * Use channel leases for parallel processing. * move data wait to chan + atomic * Progress and split on execute goroutine. * Convert metrics tests to short ids. * Add Provision handler w/ capabilities. * Add artifact validation to run_rc_validation (apache#26407) * Add artifact validation to run_rc_validation * config file for artifacts * use java bom contents for validation * add pip_pre = True * Update tox.ini * Remove out of date experimental warning on SDFs (apache#26450) * [ToBF] Refinement 26.04.23 (apache#26428) * content tree rebuilds on sdk change * expandable parent node widget * comment fix * comment fix --------- Co-authored-by: darkhan.nausharipov <[email protected]> * Allow implicit flattening for yaml inputs. (apache#26423) * Add field annotations for high-priority Syndeo schema transforms (apache#26384) add changes * [Go SDK] Timers with new datalayer (apache#26101) * added timer package * add timer changes and merged with rebo's pr * timer fired in stateful * error setting new timer in ontimer * looping timers work * send fv instead of bytes * changes to coder/pardo * works for all cases, only cleanup left * remove comments and validate onTimer * generic coder for user key * fixes coder end to end * remove logs * add unit test and refactor * add docs * new example * fix static lint * support emitters * allow input col of CoGBK as well * unit tests, periodic impulse, minor refactor * update PipelineTimer interface, minor refactor, doc comment for example * add warn message * single edge timer coder, rm kv coder check, cache encoder,decoder * Update chromedriver-binary requirement in /sdks/python Updates the requirements on [chromedriver-binary](https://github.com/danielkaiser/python-chromedriver-binary) to permit the latest version. - [Release notes](https://github.com/danielkaiser/python-chromedriver-binary/releases) - [Commits](danielkaiser/python-chromedriver-binary@v100.0.4896.20.0...v113.0.5672.24.0) --- updated-dependencies: - dependency-name: chromedriver-binary dependency-type: direct:development ... Signed-off-by: dependabot[bot] <[email protected]> * Automation: Tour of Beam infrastructure deployment (apache#25793) * Added Terraform scripts for TOB infra * ToB Frontend related updates * Update settings.gradle.kts * Deleted redundant file and minor README change * Addressing comments in the PR * Added newline at the end of variables.tf file * Update README.md * Updates related to Tour of Beam infrastructure * Update locals.tf * Output.tf updates * Update output.tf * Updates * Update main.tf * Updates to cloudfunctions_bucket variable * service_account_id changes * Update main.tf * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Bulk update of terraform scripts * Update README.md * Update README.md * Datastore_namespace updates * Update README.md * Update README.md * Update README.md * Update README.md * Update main.tf * Update README.md * Update README.md * Update README.md * Some minor TF updates * Update README.md * Modify batch IT to use count instead of hash (apache#26327) * Modify batch IT to use count instead of hash * remove unused varaiable * run spotless, update pipeline state checking * Update timeout to 45 minutes * revert timeout, add additional counter to try and pinpoint missing records * add a log to notify ranges used when workers restart * change counts from metrics to combiners * add a window to streaming test * move the passert to the correct place * Remove extra counter, apply spotless * add additional metric to KafkaWriter * Remove debugging metrics * verify pipeline is not failed * remove extra newline * Revert "Modify batch IT to use count instead of hash (apache#26327)" (apache#26466) This reverts commit 9903b2f. * Bump Java Dataflow container images (apache#26459) * keep retrying mass_comment until it has started all jobs (apache#26457) * keep retrying mass_comment until it has started all jobs * fix lookups * Add driverJars parameter to JdbcIO. (apache#25824) This change allows users to use driver jars saved in GCS. With this change, Dataflow templates will be able to migrate to JdbcIO instead of DynamicJdbcIO. * [Roll Fwd PR] Rename _namespace to _get_display_data_namespace"" (apache#26470) * Update parquetio and textio to work with -beam_strict (apache#26469) * use wheel sdk location for PostCommit_Py_Examples (apache#26473) * Move back the timeout of Python PostCommit to 4h * Minor fix on Python PostCommit description strings * Add recent postcommits to jenkins README * More user-friendly providers. * Add yaml preprocessing phases. * Add flexible windowing syntax to yaml. * Implement flatten in terms of preprocessor phase. This composes better with windowing. * Reword SQL note. * Make linter happy. * Survive errors in size estimation in MongoDbIO * Fix jdbc xlang schema type mismatch (apache#26480) * Fix jdbc xlang schema type mismatch * Also fix fetch_size type mismatch * Add new fields in the end * [Python] Add saved_weights example to tf notebook (apache#26472) * add saved_weights example to tf notebook * add description * updated text blocks * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Rebecca Szper <[email protected]> * Update examples/notebooks/beam-ml/run_inference_tensorflow.ipynb Co-authored-by: Rebecca Szper <[email protected]> --------- Co-authored-by: Rebecca Szper <[email protected]> * Revert "Revert "Modify batch IT to use count instead of hash (apache#26327)" (apache#26466)" (apache#26467) This reverts commit 5c67668. * Bump github.com/tetratelabs/wazero from 1.0.3 to 1.1.0 in /sdks (apache#26486) Bumps [github.com/tetratelabs/wazero](https://github.com/tetratelabs/wazero) from 1.0.3 to 1.1.0. - [Release notes](https://github.com/tetratelabs/wazero/releases) - [Commits](tetratelabs/wazero@v1.0.3...v1.1.0) --- updated-dependencies: - dependency-name: github.com/tetratelabs/wazero dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * tour of beam integration tests (apache#25925) * Integration test to load the default example of the default SDK and change the example (apache#24730) (apache#24729) * Fix formatting and README (apache#24730) * Support collection v1.17.0 (apache#24730) * LoadingIndicator on chaning examples, remove duplicating licenses (apache#24730) * Add a missing license header (apache#24730) * Integration test for changing SDK and running code (apache#24779) (apache#382) * Integration test for changing SDK and running code (apache#24779) * Rename an integration test (apache#24779) * Use enum to switch SDK in integration test (apache#24779) * Find SDK in a dropdown by key (apache#24779) * Add a TODO (apache#24779) * Fix exports (apache#24779) * Issue24779 integration changing sdk from 24370 (apache#387) * Integration test for changing SDK and running code (apache#24779) * Rename an integration test (apache#24779) * Use enum to switch SDK in integration test (apache#24779) * Find SDK in a dropdown by key (apache#24779) * Add a TODO (apache#24779) * Fix exports (apache#24779) * Integration tests miscellaneous UI (apache#383) * miscellaneous ui integration tests * reverted pubspec.lock * gradle tasks ordered alhpabetically * integration tests refactoring * clean code * integration tests miscellaneous ui fix pr * rename method * added layout adaptivity * A minor cleanup (apache#24779) Co-authored-by: Dmitry Repin <[email protected]> * integration tests run and editing * example selector test * minor fixes * rat * fix pr * minor * minor * rat * integration test finder written * integration test minor fixes * minor fixes * removed comment * minor fixes * playground integration tests minor fixes * integration test pumpAnSettleNoException * integration test shortcut refactor * integration test another changing shortcuts running * upgrade to flutter 3.7.1 * workaround comment * playground frontend updated major versions * issues 25329 25331 25336 * 25329 extract connectivity extension to separate file * Upgrade Flutter to 3.7.3 in integration tests (apache#24730) * Fix integration test (apache#24730) * fix cors issue and added mouse scroll to tags * Upgrade Flutter in Dockerfile (apache#24720) * sorting moved to model * sorting moved to model * sorting moved to model * bugs fix * issue 25278 * fix pr * quites fix in en.yaml * Fix not loading default example (apache#25528) * fix compile error * Refactor output tabs, test embedded playground (apache#25136) (apache#439) * Refactor output tabs, test embedded playground (apache#25136) * Clean up (apache#25136) * Change example paths to IDs in integration tests * issue25640 tob ci * fix tob ci * rename ci process * test add new line to main * test add new line to main * commented unit test run * issue25640 changed server path * issue25640 tests on welcome page * deleted config.g.dart * issue25640 pr fixes * Update .github/workflows/tour_of_beam_frontend_test.yml Co-authored-by: alexeyinkin <[email protected]> * Update learning/tour-of-beam/frontend/integration_test/welcome_page_test.dart Co-authored-by: alexeyinkin <[email protected]> * Improve tests (apache#25640) * issue25640 tour page tests * pr fix * removed import * pr fix * fix test * 25640 fixed pubspec.lock * issue25640 fix readme * updated readme * issue25640 fixed after master merge * issue25483 ToB pipeline options * removed unnecesary variable * pr fix * Update learning/tour-of-beam/frontend/assets/translations/en.yaml Co-authored-by: alexeyinkin <[email protected]> * playground hides when snippet does not exists * pipeline options extracted to playground components * issue25483 pipeline options * added errors handling, fix pr * refactoring * Revert "refactoring" This reverts commit 1540961. * removed unnecessary constants * playground controller in tour notifier becomes nullable * playground controller returned to non nullable in tour notifier * playground controller actions * removed unnecessary code * tob scaffold wrapped with animated builder * minor fixes * partially fixed tests * Upgrade flutter_code_editor to v0.2.19 (apache#25640) * Replace output SelectableText with a CodeField instance (apache#25640) * Trigger ToB integration tests (apache#25640) * Clean up (apache#25640) * Enable manual workflow runs for Playground and ToB integration tests (apache#25640) --------- Co-authored-by: Alexey Inkin <[email protected]> Co-authored-by: alexeyinkin <[email protected]> * Eliminate nullness errors from MongoDbIO * Move provisioned options outside of harness.Main (apache#26476) Co-authored-by: lostluck <[email protected]> * Add SDF adjacent element test * Consolidate residual processing * fix race condition on split boundaries * deflake fixed window combine tests * Add comments to job functions. * Make int64check per window. * Update sdks/go/pkg/beam/runners/prism/internal/worker/bundle.go Co-authored-by: Ritesh Ghorse <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: lostluck <[email protected]> Co-authored-by: Danny McCormick <[email protected]> Co-authored-by: Anand Inguva <[email protected]> Co-authored-by: Anand Inguva <[email protected]> Co-authored-by: Darkhan Nausharipov <[email protected]> Co-authored-by: darkhan.nausharipov <[email protected]> Co-authored-by: Robert Bradshaw <[email protected]> Co-authored-by: Andrei Gurau <[email protected]> Co-authored-by: Ritesh Ghorse <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: ruslan-ikhsan <[email protected]> Co-authored-by: johnjcasey <[email protected]> Co-authored-by: Yi Hu <[email protected]> Co-authored-by: Pranav Bhandari <[email protected]> Co-authored-by: Svetak Sundhar <[email protected]> Co-authored-by: Jeremy Edwards <[email protected]> Co-authored-by: Kenneth Knowles <[email protected]> Co-authored-by: Rebecca Szper <[email protected]> Co-authored-by: Dmitry Repin <[email protected]> Co-authored-by: Alexey Inkin <[email protected]> Co-authored-by: alexeyinkin <[email protected]>

Modify batch IT to use count instead of hash

5fbb44a

github-actions bot added io java kafka labels Apr 18, 2023

johnjcasey force-pushed the testing/fix-batch-performance-test branch from 6797301 to 5fbb44a Compare April 18, 2023 14:39

remove unused varaiable

bad87d7

run spotless, update pipeline state checking

8285795

Update timeout to 45 minutes

dd29c43

github-actions bot added the infra label Apr 19, 2023

revert timeout, add additional counter to try and pinpoint missing re…

9e877b0

…cords

github-actions bot removed the infra label Apr 20, 2023

add a log to notify ranges used when workers restart

cb9ead1

verify pipeline is not failed

80166bb

johnjcasey marked this pull request as ready for review April 27, 2023 20:09

remove extra newline

acbd3c0

Abacn self-requested a review April 27, 2023 20:32

github-actions bot added the Next Action: Reviewers label Apr 28, 2023

johnjcasey merged commit 9903b2f into apache:master Apr 28, 2023

johnjcasey added a commit that referenced this pull request Apr 28, 2023

Revert "Modify batch IT to use count instead of hash (#26327)"

54bc4a7

This reverts commit 9903b2f.

johnjcasey mentioned this pull request Apr 28, 2023

Revert "Modify batch IT to use count instead of hash" #26466

Merged

johnjcasey added a commit that referenced this pull request Apr 28, 2023

Revert "Modify batch IT to use count instead of hash (#26327)" (#26466)

5c67668

This reverts commit 9903b2f.

johnjcasey added a commit that referenced this pull request Apr 28, 2023

Revert "Revert "Modify batch IT to use count instead of hash (#26327)" (

7371fc5

#26466)" This reverts commit 5c67668.

johnjcasey added a commit that referenced this pull request May 1, 2023

Revert "Revert "Modify batch IT to use count instead of hash (#26327)" (

90d6666

#26466)" (#26467) This reverts commit 5c67668.

lostluck pushed a commit to lostluck/beam that referenced this pull request May 1, 2023

Revert "Modify batch IT to use count instead of hash (apache#26327)" (a…

8d0ff00

…pache#26466) This reverts commit 9903b2f.

lostluck pushed a commit to lostluck/beam that referenced this pull request May 1, 2023

Revert "Revert "Modify batch IT to use count instead of hash (apache#…

ce9ed5f

…26327)" (apache#26466)" (apache#26467) This reverts commit 5c67668.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Modify batch IT to use count instead of hash #26327

Modify batch IT to use count instead of hash #26327

johnjcasey commented Apr 18, 2023

johnjcasey commented Apr 18, 2023

johnjcasey commented Apr 18, 2023

johnjcasey commented Apr 18, 2023

johnjcasey commented Apr 18, 2023

johnjcasey commented Apr 18, 2023

johnjcasey commented Apr 18, 2023

johnjcasey commented Apr 19, 2023

johnjcasey commented Apr 19, 2023

johnjcasey commented Apr 19, 2023

johnjcasey commented Apr 19, 2023

johnjcasey commented Apr 19, 2023

johnjcasey commented Apr 20, 2023

johnjcasey commented Apr 20, 2023

johnjcasey commented Apr 20, 2023

johnjcasey commented Apr 20, 2023

johnjcasey commented Apr 20, 2023

johnjcasey commented Apr 20, 2023

johnjcasey commented Apr 20, 2023

johnjcasey commented Apr 26, 2023

johnjcasey commented Apr 26, 2023

johnjcasey commented Apr 27, 2023

johnjcasey commented Apr 27, 2023

johnjcasey commented Apr 27, 2023

johnjcasey commented Apr 27, 2023

johnjcasey commented Apr 27, 2023

github-actions bot commented Apr 27, 2023

github-actions bot commented Apr 28, 2023

johnjcasey commented Apr 28, 2023

Modify batch IT to use count instead of hash #26327

Modify batch IT to use count instead of hash #26327

Conversation

johnjcasey commented Apr 18, 2023

GitHub Actions Tests Status (on master branch)

johnjcasey commented Apr 18, 2023

johnjcasey commented Apr 18, 2023

johnjcasey commented Apr 18, 2023

johnjcasey commented Apr 18, 2023

johnjcasey commented Apr 18, 2023

johnjcasey commented Apr 18, 2023

johnjcasey commented Apr 19, 2023

johnjcasey commented Apr 19, 2023

johnjcasey commented Apr 19, 2023

johnjcasey commented Apr 19, 2023

johnjcasey commented Apr 19, 2023

johnjcasey commented Apr 20, 2023

johnjcasey commented Apr 20, 2023

johnjcasey commented Apr 20, 2023

johnjcasey commented Apr 20, 2023

johnjcasey commented Apr 20, 2023

johnjcasey commented Apr 20, 2023

johnjcasey commented Apr 20, 2023

johnjcasey commented Apr 26, 2023

johnjcasey commented Apr 26, 2023

johnjcasey commented Apr 27, 2023

johnjcasey commented Apr 27, 2023

johnjcasey commented Apr 27, 2023

johnjcasey commented Apr 27, 2023

johnjcasey commented Apr 27, 2023

github-actions bot commented Apr 27, 2023

github-actions bot commented Apr 28, 2023

johnjcasey commented Apr 28, 2023