Code

Create Dask resource#2811

Code

Merged

alangenfeld merged 27 commits intodagster-io:masterdagster-io/dagster:masterfrom kinghuang:dask-resourcekinghuang/dagster:dask-resourceCopy head branch name to clipboardSep 18, 2020

+217-6

Commits

Commits on Sep 16, 2020

• Create Dask resource

  Show description for 86bb0a2
  kinghuang

  committed
  86bb0a2

• Remove compute fields from materialization options

  Show description for a420fb0
  kinghuang

  committed
  a420fb0

• Compute materializations on Dask resource

  Show description for 74b7e49
  kinghuang

  committed
  74b7e49

• Import dask_resource to the base module

  kinghuang

  committed
  74b9694

• Create passthrough solid for testing

  Show description for f01040b
  kinghuang

  committed
  f01040b

• Create pipeline for Dask testing

  Show description for 468cb03
  kinghuang

  committed
  468cb03

• Test local cluster configuration

  Show description for 749596b
  kinghuang

  committed
  749596b

• Test multiple local cluster configurations

  kinghuang

  committed
  5333085

• Change scheduler config to client config

  Show description for cfd464d
  kinghuang

  committed
  cfd464d

• Reduce number of threads per worker in tests

  Show description for 800ec5b
  kinghuang

  committed
  800ec5b

• Reformat code according to Black

  kinghuang

  committed
  12b3d2d

• Use config ypes instead of inferring

  Show description for 5febe27
  kinghuang

  committed
  5febe27

• Refine some code comments

  kinghuang

  committed
  fb045d8

• Set and test the current client

  Show description for d6e052b
  kinghuang

  committed
  d6e052b

• Remove unused imports

  kinghuang

  committed
  a604aeb

• Refine import of Dask Client

  Show description for 02aa1a4
  kinghuang

  committed
  02aa1a4

• Use Selector for cluster config config options

  Show description for d7f2591
  kinghuang

  committed
  d7f2591

• Improve code formatting when getting results

  kinghuang

  committed
  872b596

• Set _cluster to None if no config

  Show description for 2779bc9
  kinghuang

  committed
  2779bc9

• Use cluster property to get cluster

  Show description for ef924e7
  kinghuang

  committed
  ef924e7

• Remove extraneous spaces

  kinghuang

  committed
  6b8d6fd

• Add changelog entry for removal of compute option

  kinghuang

  committed
  7fb0bb5

• Remove unused import of DataFrame

  kinghuang

  committed
  e888530

• Note required module for each cluster type

  kinghuang

  committed
  f66a1cd

• Apply changes from black and isort

  kinghuang

  committed
  cc46023

• More changes from black

  kinghuang

  committed
  6b3f5f8

Commits on Sep 18, 2020

• Merge branch 'master' into dask-resource

  alangenfeld

  authored
  0c388f9