Code

Apache Beam Executor#225

Code

Merged

rabernat merged 19 commits intopangeo-forge:masterpangeo-forge/pangeo-forge-recipes:masterfrom alxmrs:beamalxmrs/pangeo-forge-recipes:beamCopy head branch name to clipboardNov 16, 2021

+125-7

Commits

Commits on Nov 7, 2021

• Naive implementation of Beam executor.

  alxmrs

  committed
  b63f440

  Copy full SHA for b63f440

• An improvement on the Beam executor; includes parallel mappable stages.

  alxmrs

  committed
  81aa14e

  Copy full SHA for 81aa14e

• Adding beam dependencies to CI conda environment files.

  alxmrs

  committed
  c562e0f

  Copy full SHA for c562e0f

Commits on Nov 8, 2021

• Updated change to pass precommit checks. Includes some refactoring for readability.

  alxmrs

  committed
  c8ffecc

  Copy full SHA for c8ffecc

• Update ci/py3.8.yml

  rabernat

  authored
  7d5a7b3

  Copy full SHA for 7d5a7b3

• Chose better variable name that didn't break Beam's type-checking. Apparently, func is also an internal variable name.

  alxmrs

  committed
  3d693eb

  Copy full SHA for 3d693eb

• Merge remote-tracking branch 'origin/beam' into beam

  alxmrs

  committed
  bbdb8b4

  Copy full SHA for bbdb8b4

• Update ci/py3.9.yml

  Show description for c7198b8
  

  alxmrs

  and

  rabernat

  authored
  c7198b8

  Copy full SHA for c7198b8

Commits on Nov 11, 2021

• Merge branch 'master' of github.com:pangeo-forge/pangeo-forge-recipes into beam

  alxmrs

  committed
  0026701

  Copy full SHA for 0026701

• Merge branch 'beam' of github.com:alxmrs/pangeo-forge-recipes into beam

  alxmrs

  committed
  a4dafc5

  Copy full SHA for a4dafc5

• Remove apache-beam from py3.9 env (not supported).

  alxmrs

  committed
  d9c5b2b

  Copy full SHA for d9c5b2b

• Skip Beam pipeline tests if apache_beam is not installed in the test environment.

  alxmrs

  committed
  e3423db

  Copy full SHA for e3423db

• Merge branch 'master' of github.com:pangeo-forge/pangeo-forge-recipes into beam

  alxmrs

  committed
  e9f5077

  Copy full SHA for e9f5077

• Added reshuffle step to prevent operator fusion.

  alxmrs

  committed
  97a60a5

  Copy full SHA for 97a60a5

• Pipeline tests restructured to skip tests for Executors that have import problems (now, using pytest fixtures).

  alxmrs

  committed
  c0c3d5a

  Copy full SHA for c0c3d5a

Commits on Nov 13, 2021

• Added to_beam() to base recipe.

  alxmrs

  committed
  a871809

  Copy full SHA for a871809

Commits on Nov 14, 2021

• Documented usage of to_beam().

  alxmrs

  committed
  ca8afbb

  Copy full SHA for ca8afbb

Commits on Nov 16, 2021

• Better Executor fixture in tests: using import_module instead of importskip.

  alxmrs

  committed
  27467d1

  Copy full SHA for 27467d1

• Update release_notes.md

  rabernat

  authored
  f52fece

  Copy full SHA for f52fece