Code

Optimize Datastream for batch#31950

Code

Closed

jto wants to merge 38 commits intoapache:release-2.56.0apache/beam:release-2.56.0from jto:julient/patched-2.56jto/beam:julient/patched-2.56Copy head branch name to clipboard

+1,665-854

Commits

Commits on Jul 23, 2024

• Limit max split size for bounded sources in Datastream API

  jto

  committed
  44d1271

  Copy full SHA for 44d1271

• [Flink] Set return type of bounded sources

  jto

  committed
  82e4a78

  Copy full SHA for 82e4a78

Commits on Jul 24, 2024

• [Flink] Use a lazy split enumerator for bounded sources

  jto

  committed
  9641b2a

  Copy full SHA for 9641b2a

Commits on Jul 26, 2024

• spotless

  jto

  committed
  4c71518

  Copy full SHA for 4c71518

• Revert "Limit max split size for bounded sources in Datastream API"

  Show description for 362b649
  jto

  committed
  362b649

  Copy full SHA for 362b649

Commits on Aug 6, 2024

• [Flink] make ToKeyedWorkItem part of the DoFnOperator

  jto

  committed
  57736fe

  Copy full SHA for 57736fe

Commits on Aug 8, 2024

• [Flink] refactor CombinePerKeyTranslator

  jto

  committed
  d1dbd5a

  Copy full SHA for d1dbd5a

Commits on Aug 9, 2024

• [Flink] Combine before Reduce (no side-input only)

  jto

  committed
  8560709

  Copy full SHA for 8560709

Commits on Aug 13, 2024

• [WIP] combine before reduce

  jto

  committed
  90485c7

  Copy full SHA for 90485c7

• Revert "[WIP] combine before reduce"

  Show description for 0786acf
  jto

  committed
  0786acf

  Copy full SHA for 0786acf

• [WIP] combine before reduce (again)

  jto

  committed
  78da517

  Copy full SHA for 78da517

• [WIP] Also fire timer on finished bundle

  jto

  committed
  5e7b170

  Copy full SHA for 5e7b170

Commits on Aug 14, 2024

• Revert "[WIP] Also fire timer on finished bundle"

  Show description for 7e82123
  jto

  committed
  7e82123

  Copy full SHA for 7e82123

• Revert "[WIP] combine before reduce (again)"

  Show description for 6be49e0
  jto

  committed
  6be49e0

  Copy full SHA for 6be49e0

Commits on Aug 19, 2024

• [FLink] Set default maxParallelism to parallelism in batch to avoid skew

  jto

  committed
  7da29a8

  Copy full SHA for 7da29a8

• [Flink] Implement partial reduce

  jto

  committed
  4faa6d0

  Copy full SHA for 4faa6d0

• [Flink] remove toBinaryKV

  jto

  committed
  4cbd239

  Copy full SHA for 4cbd239

Commits on Aug 20, 2024

• [Flink] fastercopy everywhere

  jto

  committed
  7e7b300

  Copy full SHA for 7e7b300

• [Flink] Avoid re-evaluating options every time a new state is stored

  jto

  committed
  0a634bf

  Copy full SHA for 0a634bf

• Avoid re-serializing trigger on every element in GroupAlsoByWindowViaWindowSetNewDoFn

  jto

  committed
  03a1996

  Copy full SHA for 03a1996

• Cache stringKey in StateNamespaces.WindowNamespace

  jto

  committed
  e885cb2

  Copy full SHA for e885cb2

• Revert "[Flink] fastercopy everywhere"

  Show description for 164973c
  jto

  committed
  164973c

  Copy full SHA for 164973c

• Revert "Cache stringKey in StateNamespaces.WindowNamespace"

  Show description for b37aa7d
  jto

  committed
  b37aa7d

  Copy full SHA for b37aa7d

• [Flink] dead code cleanup

  jto

  committed
  7e423e7

  Copy full SHA for 7e423e7

• [Flink] spotless

  jto

  committed
  7f0983d

  Copy full SHA for 7f0983d

Commits on Aug 21, 2024

• [Flink] Only serialize namespace keys if necessary

  jto

  committed
  8751150

  Copy full SHA for 8751150

Commits on Aug 23, 2024

• [Flink] pre-combine before GBK

  jto

  committed
  dec75f6

  Copy full SHA for dec75f6

• [Flink] persistent PartialReduceBundleOperator operator state

  jto

  committed
  b3b7c2c

  Copy full SHA for b3b7c2c

• [Flink] lower default max bundle size in batch mode

  jto

  committed
  d0e7cb3

  Copy full SHA for d0e7cb3

Commits on Aug 26, 2024

• [Flink] disable bundles in batch mode

  jto

  committed
  fb9ef12

  Copy full SHA for fb9ef12

Commits on Aug 27, 2024

• [Flink] force slot sharing group for source in batch mode

  jto

  committed
  7fc687f

  Copy full SHA for 7fc687f

Commits on Aug 28, 2024

• [Flink] fix lazy enumerator package

  jto

  committed
  0b07e4a

  Copy full SHA for 0b07e4a

• [Flink] implement combine for reduce with side input

  jto

  committed
  686a100

  Copy full SHA for 686a100

• [Flink] further reduce default bundle size in batch

  jto

  committed
  772f4af

  Copy full SHA for 772f4af

• [Flink] fix licence

  jto

  committed
  06dd507

  Copy full SHA for 06dd507

• [Flink] spotless

  jto

  committed
  3b1c6e9

  Copy full SHA for 3b1c6e9

• [Flink] checkstyle

  jto

  committed
  d7d3a7f

  Copy full SHA for d7d3a7f

Commits on Aug 29, 2024

• [Flink] spotless

  jto

  committed
  2c7f737

  Copy full SHA for 2c7f737