Code

[tune] Cluster Fault Tolerance#3309

Code

Merged

richardliaw merged 213 commits intoray-project:masterray-project/ray:masterfrom richardliaw:tune_cluster-2richardliaw/ray:tune_cluster-2Copy head branch name to clipboardDec 29, 2018

+805-127

Commits

Commits on Nov 5, 2018

• [tune] Throw on overstepping

  richardliaw

  committed
  e557b6a

  Copy full SHA for e557b6a

• Add Tune Multi-Node Tests

  richardliaw

  committed
  b755785

  Copy full SHA for b755785

• Add cluster bookkeeping code

  richardliaw

  committed
  32d1242

  Copy full SHA for 32d1242

• add test for adding node

  richardliaw

  committed
  9ec3a60

  Copy full SHA for 9ec3a60

• multinode test fixes

  richardliaw

  committed
  44fe1e2

  Copy full SHA for 44fe1e2

Commits on Nov 6, 2018

• First pass at allowing updatable values

  richardliaw

  committed
  d9c9e3b

  Copy full SHA for d9c9e3b

• Fix compilation issues

  richardliaw

  committed
  d6cade1

  Copy full SHA for d6cade1

• Merge branch 'config_updating' into global_state_multinode

  richardliaw

  committed
  ac74520

  Copy full SHA for ac74520

• Add config file parsing

  richardliaw

  committed
  a95c718

  Copy full SHA for a95c718

• Full initialization

  richardliaw

  committed
  5814655

  Copy full SHA for 5814655

• Merge branch 'config_updating' into global_state_multinode

  richardliaw

  committed
  f63df3f

  Copy full SHA for f63df3f

• Wrote a good test

  richardliaw

  committed
  2824836

  Copy full SHA for 2824836

• Merge branch 'config_updating' into tune_cluster

  richardliaw

  committed
  6e7bd6a

  Copy full SHA for 6e7bd6a

Commits on Nov 7, 2018

• configuration parsing and stuff

  richardliaw

  committed
  4842481

  Copy full SHA for 4842481

• docs

  richardliaw

  committed
  8e52103

  Copy full SHA for 8e52103

• write some tests, make it good

  richardliaw

  committed
  83d6947

  Copy full SHA for 83d6947

• Merge branch 'master' into config_updating

  richardliaw

  committed
  4349adf

  Copy full SHA for 4349adf

• fixed init

  richardliaw

  committed
  8078967

  Copy full SHA for 8078967

• Add all config options and bring back stress tests.

  

  Robert Nishihara

  authored and

  Robert Nishihara

  committed
  2db9f18

  Copy full SHA for 2db9f18

• Merge branch 'config_updating' into tune_cluster

  richardliaw

  committed
  cc8fca2

  Copy full SHA for cc8fca2

• Update python/ray/worker.py

  richardliaw

  authored
  59480dc

  Copy full SHA for 59480dc

• Update python/ray/worker.py

  richardliaw

  authored
  6fa9d7c

  Copy full SHA for 6fa9d7c

• TEMP

  richardliaw

  committed
  856547c

  Copy full SHA for 856547c

• Fix internalization

  richardliaw

  committed
  25e45cd

  Copy full SHA for 25e45cd

• Merge branch 'config_updating' of github.com:richardliaw/ray into config_updating

  richardliaw

  committed
  2e2b8b0

  Copy full SHA for 2e2b8b0

• some last changes

  richardliaw

  committed
  d3fa8f0

  Copy full SHA for d3fa8f0

• Merge branch 'config_updating' into tune_cluster

  richardliaw

  committed
  233f3ee

  Copy full SHA for 233f3ee

• skip for now

  richardliaw

  committed
  c3c1c9c

  Copy full SHA for c3c1c9c

• Linting and Java fix

  

  Robert Nishihara

  committed
  3e96ec9

  Copy full SHA for 3e96ec9

• add docstring

  richardliaw

  committed
  4081c60

  Copy full SHA for 4081c60

• Merge branch 'config_updating' into global_state_multinode

  richardliaw

  committed
  a916257

  Copy full SHA for a916257

Commits on Nov 8, 2018

• Merge branch 'config_updating' into tune_cluster

  richardliaw

  committed
  5646982

  Copy full SHA for 5646982

• Merge branch 'master' into tune_cluster

  richardliaw

  committed
  38eda57

  Copy full SHA for 38eda57

• Merge branch 'master' into global_state_multinode

  richardliaw

  committed
  7d19c9f

  Copy full SHA for 7d19c9f

• Fix test, add assertions

  richardliaw

  committed
  2f5861c

  Copy full SHA for 2f5861c

• Merge branch 'master' into tune_cluster

  richardliaw

  committed
  90bc3fb

  Copy full SHA for 90bc3fb

• Merge branch 'global_state_multinode' into tune_cluster

  richardliaw

  committed
  d2dccae

  Copy full SHA for d2dccae

• fix up tests

  richardliaw

  committed
  7f675f7

  Copy full SHA for 7f675f7

• pytest ext

  richardliaw

  committed
  af0fe9c

  Copy full SHA for af0fe9c

• Merge branch 'global_state_multinode' into tune_cluster

  richardliaw

  committed
  8f942fc

  Copy full SHA for 8f942fc

Commits on Nov 11, 2018

• Merge branch 'master' into tune_cluster

  richardliaw

  committed
  6194744

  Copy full SHA for 6194744

Commits on Nov 12, 2018

• code to make requeueing work

  richardliaw

  committed
  d01d80c

  Copy full SHA for d01d80c

• yapf

  richardliaw

  committed
  1e32227

  Copy full SHA for 1e32227

• lint

  richardliaw

  committed
  9db9d16

  Copy full SHA for 9db9d16

• comments

  richardliaw

  committed
  2639a98

  Copy full SHA for 2639a98

• lint

  richardliaw

  committed
  d45c74b

  Copy full SHA for d45c74b

• Update multi_node_test_2.py

  richardliaw

  authored
  ee6a800

  Copy full SHA for ee6a800

• lit

  richardliaw

  committed
  3f13bfa

  Copy full SHA for 3f13bfa

• re-enable

  richardliaw

  committed
  5f0d75e

  Copy full SHA for 5f0d75e

• lint

  richardliaw

  committed
  b1793bd

  Copy full SHA for b1793bd

• initial nuke test

  richardliaw

  committed
  0763004

  Copy full SHA for 0763004

• Track last result

  richardliaw

  committed
  a1a05f0

  Copy full SHA for a1a05f0

• Merge branch 'global_state_multinode' into tune_cluster

  richardliaw

  committed
  5abf9d1

  Copy full SHA for 5abf9d1

• note

  richardliaw

  committed
  6ecc2bb

  Copy full SHA for 6ecc2bb

• Merge branch 'tune_cluster' into tune_cluster-2

  richardliaw

  committed
  eabb28b

  Copy full SHA for eabb28b

• add checkpointing to trial_runner

  richardliaw

  committed
  09995fa

  Copy full SHA for 09995fa

• trialrunners

  richardliaw

  committed
  24d6e12

  Copy full SHA for 24d6e12

• logging

  richardliaw

  committed
  dbd1bbc

  Copy full SHA for dbd1bbc

• Redo checkpointing from trial runner

  richardliaw

  committed
  e2b8380

  Copy full SHA for e2b8380

Commits on Nov 14, 2018

• Merge branch 'master' into tune_cluster

  richardliaw

  committed
  1239c1a

  Copy full SHA for 1239c1a

• fix up tests and checkpointing

  richardliaw

  committed
  7aab84f

  Copy full SHA for 7aab84f

• Merge branch 'tune_cluster' into tune_cluster-2

  richardliaw

  committed
  6c3d53e

  Copy full SHA for 6c3d53e

• import error

  richardliaw

  committed
  1e8a33d

  Copy full SHA for 1e8a33d

• Merge branch 'tune_cluster' into tune_cluster-2

  richardliaw

  committed
  026192b

  Copy full SHA for 026192b

Commits on Nov 15, 2018

• timeout?

  richardliaw

  committed
  637e707

  Copy full SHA for 637e707

• lint

  richardliaw

  committed
  162b308

  Copy full SHA for 162b308

Commits on Nov 16, 2018

• Merge branch 'master' into tune_cluster

  richardliaw

  committed
  a65fc45

  Copy full SHA for a65fc45

Commits on Nov 17, 2018

• Merge branch 'tune_cluster' into tune_cluster-2

  richardliaw

  committed
  b0d5997

  Copy full SHA for b0d5997

Commits on Nov 19, 2018

• Checkpoint and tests

  richardliaw

  committed
  e683224

  Copy full SHA for e683224

• one full cluster failure

  richardliaw

  committed
  ef67acf

  Copy full SHA for ef67acf

Commits on Nov 21, 2018

• lint

  richardliaw

  committed
  0541f92

  Copy full SHA for 0541f92

• Merge branch 'tune_cluster' into tune_cluster-2

  richardliaw

  committed
  267150d

  Copy full SHA for 267150d

• Add better test

  richardliaw

  committed
  b884fed

  Copy full SHA for b884fed

• Merge branch 'master' into tune_cluster-2

  richardliaw

  committed
  81ff30f

  Copy full SHA for 81ff30f

Commits on Nov 22, 2018

• error test

  richardliaw

  committed
  8617f2d

  Copy full SHA for 8617f2d

Commits on Nov 24, 2018

• some docs

  richardliaw

  committed
  f7e31bd

  Copy full SHA for f7e31bd

Commits on Nov 25, 2018

• Tests and better recovery handling

  richardliaw

  committed
  a2355f8

  Copy full SHA for a2355f8

• Add unit test for restoring (but currently failing

  richardliaw

  committed
  5513099

  Copy full SHA for 5513099

• pickle if needed when you set status

  richardliaw

  committed
  782f194

  Copy full SHA for 782f194

• yapf

  richardliaw

  committed
  d1d5a56

  Copy full SHA for d1d5a56

• docs and small test for nosaving

  richardliaw

  committed
  be445e8

  Copy full SHA for be445e8

• doc

  richardliaw

  committed
  c13270b

  Copy full SHA for c13270b

• more docs

  richardliaw

  committed
  8a6ed91

  Copy full SHA for 8a6ed91

• test docs

  richardliaw

  committed
  b1e3bf0

  Copy full SHA for b1e3bf0

Commits on Nov 26, 2018

• py2mock

  richardliaw

  committed
  40248aa

  Copy full SHA for 40248aa

• dirpath from tmpdir

  richardliaw

  committed
  22930c8

  Copy full SHA for 22930c8

• fix tsts?

  richardliaw

  committed
  82ff45e

  Copy full SHA for 82ff45e

• yapf

  richardliaw

  committed
  bde644c

  Copy full SHA for bde644c

• Fix up tests

  richardliaw

  committed
  79197b8

  Copy full SHA for 79197b8

• nits

  richardliaw

  committed
  66be742

  Copy full SHA for 66be742

• nit

  richardliaw

  committed
  25be843

  Copy full SHA for 25be843

• test fixup

  richardliaw

  committed
  ff7b114

  Copy full SHA for ff7b114

• yapf

  richardliaw

  committed
  defe524

  Copy full SHA for defe524

• no skip

  richardliaw

  committed
  02a9cf8

  Copy full SHA for 02a9cf8

• cluster tests

  richardliaw

  committed
  f80e318

  Copy full SHA for f80e318

• nit

  richardliaw

  committed
  07df20b

  Copy full SHA for 07df20b

Commits on Nov 27, 2018

• Fix counting resources test

  richardliaw

  committed
  1ff31cb

  Copy full SHA for 1ff31cb

• better test and error msg

  richardliaw

  committed
  6998a01

  Copy full SHA for 6998a01

• Merge branch 'master' into tune_cluster-2

  richardliaw

  committed
  8ea0f70

  Copy full SHA for 8ea0f70

• Tests and better recovery handling

  richardliaw

  committed
  fcbc6de

  Copy full SHA for fcbc6de

• Merge branch 'tune_cluster-2a' into tune_cluster-2_copy

  richardliaw

  committed
  fc5d407

  Copy full SHA for fc5d407

• py2mock

  richardliaw

  committed
  5d8e414

  Copy full SHA for 5d8e414

• nit

  richardliaw

  committed
  9137de0

  Copy full SHA for 9137de0

• Fix counting resources test

  richardliaw

  committed
  4453724

  Copy full SHA for 4453724

• Remove extraneous changes

  richardliaw

  committed
  5a24499

  Copy full SHA for 5a24499

• Merge branch 'tune_cluster-2a' into tune_cluster-2_copy

  richardliaw

  committed
  2dcba23

  Copy full SHA for 2dcba23

• docs

  richardliaw

  committed
  b750d4e

  Copy full SHA for b750d4e

• yapf

  richardliaw

  committed
  14da6ec

  Copy full SHA for 14da6ec

• Lint and small changes to tests

  richardliaw

  committed
  394c0e9

  Copy full SHA for 394c0e9

• lint

  richardliaw

  committed
  48fd3c3

  Copy full SHA for 48fd3c3

• nit

  richardliaw

  committed
  bcf4051

  Copy full SHA for bcf4051

• small extraneous removals

  richardliaw

  committed
  0f67265

  Copy full SHA for 0f67265

• fix some merge?

  richardliaw

  committed
  74b6a93

  Copy full SHA for 74b6a93

• Merge branch 'tune_cluster-2a' into tune_cluster-2_copy

  richardliaw

  committed
  1d6c185

  Copy full SHA for 1d6c185

Commits on Dec 4, 2018

• Merge branch 'master' into tune_cluster-2_copy

  richardliaw

  committed
  4bd54e3

  Copy full SHA for 4bd54e3

• try recover

  richardliaw

  committed
  3d0a2e3

  Copy full SHA for 3d0a2e3

• merge

  richardliaw

  committed
  4bb938f

  Copy full SHA for 4bb938f

• Removed error raising

  richardliaw

  committed
  ac5d8c0

  Copy full SHA for ac5d8c0

• Rename checkpoint_mode

  richardliaw

  committed
  83e1a26

  Copy full SHA for 83e1a26

• note for pickling

  richardliaw

  committed
  d77b934

  Copy full SHA for d77b934

Commits on Dec 5, 2018

• Better UI

  richardliaw

  committed
  f3071eb

  Copy full SHA for f3071eb

• Fix lint

  richardliaw

  committed
  0263e93

  Copy full SHA for 0263e93

• nit

  richardliaw

  committed
  71db5df

  Copy full SHA for 71db5df

Commits on Dec 7, 2018

• fix up tests

  richardliaw

  committed
  9b9e771

  Copy full SHA for 9b9e771

• note

  richardliaw

  committed
  d669463

  Copy full SHA for d669463

• nit

  richardliaw

  committed
  55562a5

  Copy full SHA for 55562a5

• text

  richardliaw

  committed
  6dd5e59

  Copy full SHA for 6dd5e59

• nit

  richardliaw

  committed
  0c3ade9

  Copy full SHA for 0c3ade9

• Merge branch 'master' into tune_cluster-2

  richardliaw

  committed
  dd1bb6b

  Copy full SHA for dd1bb6b

Commits on Dec 8, 2018

• Merge branch 'master' into tune_cluster-2

  richardliaw

  committed
  afcf2b9

  Copy full SHA for afcf2b9

• fix

  richardliaw

  committed
  e1e7b4e

  Copy full SHA for e1e7b4e

Commits on Dec 11, 2018

• fix usability

  richardliaw

  committed
  8961de1

  Copy full SHA for 8961de1

Commits on Dec 15, 2018

• Atomic Movement

  richardliaw

  committed
  adaaf43

  Copy full SHA for adaaf43

• removed checkpoint freq

  richardliaw

  committed
  1976873

  Copy full SHA for 1976873

• Merge branch 'master' into tune_cluster-2

  richardliaw

  committed
  84434f7

  Copy full SHA for 84434f7

• Merge branch 'master' into tune_cluster-2

  richardliaw

  committed
  1d63eea

  Copy full SHA for 1d63eea

• tweaks to update

  richardliaw

  committed
  c2a578b

  Copy full SHA for c2a578b

• move sync function

  richardliaw

  committed
  60653c1

  Copy full SHA for 60653c1

• fix

  richardliaw

  committed
  c0d6db5

  Copy full SHA for c0d6db5

• fixup

  richardliaw

  committed
  96c9f70

  Copy full SHA for 96c9f70

• small modifications

  richardliaw

  committed
  2afe291

  Copy full SHA for 2afe291

• fix

  richardliaw

  committed
  a5647db

  Copy full SHA for a5647db

• fix

  richardliaw

  committed
  f17a648

  Copy full SHA for f17a648

Commits on Dec 18, 2018

• fix

  richardliaw

  committed
  52ac11b

  Copy full SHA for 52ac11b

• fix assumption

  richardliaw

  committed
  8a57a5b

  Copy full SHA for 8a57a5b

• better error message

  richardliaw

  committed
  530d5af

  Copy full SHA for 530d5af

• fix registration

  richardliaw

  committed
  758ed30

  Copy full SHA for 758ed30

• Merge branch 'tune_cluster-2' of github.com:richardliaw/ray into tune_cluster-2

  richardliaw

  committed
  ac8ba8e

  Copy full SHA for ac8ba8e

• ok fix

  richardliaw

  committed
  1ff4814

  Copy full SHA for 1ff4814

• fix add test

  richardliaw

  committed
  1052d3b

  Copy full SHA for 1052d3b

• cloudpickle

  richardliaw

  committed
  e0963e5

  Copy full SHA for e0963e5

• Merge branch 'tune_cluster-2' of github.com:richardliaw/ray into tune_cluster-2

  richardliaw

  committed
  de38782

  Copy full SHA for de38782

• classmethod

  richardliaw

  committed
  5358f0c

  Copy full SHA for 5358f0c

• Merge branch 'tune_cluster-2' of github.com:richardliaw/ray into tune_cluster-2

  richardliaw

  committed
  3c1f5bc

  Copy full SHA for 3c1f5bc

• fix

  richardliaw

  committed
  61034c9

  Copy full SHA for 61034c9

• Merge branch 'master' into tune_cluster-2

  richardliaw

  committed
  cc336f6

  Copy full SHA for cc336f6

• lint...

  richardliaw

  committed
  b301963

  Copy full SHA for b301963

• Revert "better error message"

  Show description for 8207846
  richardliaw

  committed
  8207846

  Copy full SHA for 8207846

• comment

  richardliaw

  committed
  c6ec396

  Copy full SHA for c6ec396

• better handling

  richardliaw

  committed
  0db58a5

  Copy full SHA for 0db58a5

• nit

  richardliaw

  committed
  4d75a38

  Copy full SHA for 4d75a38

• a little confusing

  richardliaw

  committed
  da21686

  Copy full SHA for da21686

• lint

  richardliaw

  committed
  386a9fa

  Copy full SHA for 386a9fa

• fixtest

  richardliaw

  committed
  9c1cdb9

  Copy full SHA for 9c1cdb9

Commits on Dec 20, 2018

• fix mess

  richardliaw

  committed
  5b69662

  Copy full SHA for 5b69662

• docs

  richardliaw

  committed
  355d760

  Copy full SHA for 355d760

• checkpointmode

  richardliaw

  committed
  f412130

  Copy full SHA for f412130

Commits on Dec 24, 2018

• some reversions

  richardliaw

  committed
  64e4a09

  Copy full SHA for 64e4a09

Commits on Dec 25, 2018

• Resolve into experiment dir

  richardliaw

  committed
  0e78363

  Copy full SHA for 0e78363

• add example

  richardliaw

  committed
  9c9cd9d

  Copy full SHA for 9c9cd9d

• Trial is pickled

  richardliaw

  committed
  3e8641e

  Copy full SHA for 3e8641e

• small path fix

  richardliaw

  committed
  c9ca3e2

  Copy full SHA for c9ca3e2

• Fix trial serialization

  richardliaw

  committed
  4e7d438

  Copy full SHA for 4e7d438

• JSONify state

  richardliaw

  committed
  2318f6a

  Copy full SHA for 2318f6a

• make sure trials even without checkpointing are not duplicated

  richardliaw

  committed
  72d027a

  Copy full SHA for 72d027a

• small tweaks

  richardliaw

  committed
  b63f705

  Copy full SHA for b63f705

• Merge branch 'master' into tune_cluster-2

  richardliaw

  committed
  31c99df

  Copy full SHA for 31c99df

• Update doc/source/tune-usage.rst

  Show description for b764ab7
  

  ericl

  and

  richardliaw

  authored
  b764ab7

  Copy full SHA for b764ab7

• fix up some components

  richardliaw

  committed
  07a36fc

  Copy full SHA for 07a36fc

Commits on Dec 26, 2018

• remove accidental merge

  richardliaw

  committed
  c2eddef

  Copy full SHA for c2eddef

• Fix test for uncheckpointables

  richardliaw

  committed
  27fecb1

  Copy full SHA for 27fecb1

• back to dict

  richardliaw

  committed
  bc8e67a

  Copy full SHA for bc8e67a

• __getstate__, remove checkpointmode, trial name

  richardliaw

  committed
  de1c14b

  Copy full SHA for de1c14b

• resources_to_json

  richardliaw

  committed
  33dac8f

  Copy full SHA for 33dac8f

• Fix experiments and final comments.

  richardliaw

  committed
  763e6d4

  Copy full SHA for 763e6d4

• Fix more changes

  richardliaw

  committed
  1a15237

  Copy full SHA for 1a15237

• resume to prompt on None

  richardliaw

  committed
  677e09d

  Copy full SHA for 677e09d

• turn off prompt for tests

  richardliaw

  committed
  83b1523

  Copy full SHA for 83b1523

• Merge

  richardliaw

  committed
  d1180c3

  Copy full SHA for d1180c3

• fix for tests

  richardliaw

  committed
  45d5d90

  Copy full SHA for 45d5d90

• Merge branch 'tune_cluster-2' of github.com:richardliaw/ray into tune_cluster-2

  richardliaw

  committed
  21481f6

  Copy full SHA for 21481f6

• Accidentally removed a file

  richardliaw

  committed
  b89b910

  Copy full SHA for b89b910

• example not needed

  richardliaw

  committed
  6995a59

  Copy full SHA for 6995a59

• no need to change this

  richardliaw

  committed
  86b37f6

  Copy full SHA for 86b37f6

• doc changes and small guard

  richardliaw

  committed
  f92198e

  Copy full SHA for f92198e

• renames, remove checkpoint_mode

  richardliaw

  committed
  a096772

  Copy full SHA for a096772

• Update python/ray/tune/trial_runner.py

  Show description for ec726d2
  

  ericl

  and

  richardliaw

  authored
  ec726d2

  Copy full SHA for ec726d2

• Simpify, fix tests, address comments, make expmt mandatory

  richardliaw

  committed
  bf45f42

  Copy full SHA for bf45f42

• Merge branch 'tune_cluster-2' of github.com:richardliaw/ray into tune_cluster-2

  richardliaw

  committed
  0fe066a

  Copy full SHA for 0fe066a

• typo

  richardliaw

  committed
  3f2e1ae

  Copy full SHA for 3f2e1ae

• some more tests

  richardliaw

  committed
  e7b4f20

  Copy full SHA for e7b4f20

Commits on Dec 27, 2018

• lint

  richardliaw

  committed
  80ace1d

  Copy full SHA for 80ace1d

• test fix and move env logic

  richardliaw

  committed
  0348eb4

  Copy full SHA for 0348eb4

• fix

  richardliaw

  committed
  fc6802b

  Copy full SHA for fc6802b

• fix tests

  richardliaw

  committed
  d1f1c0b

  Copy full SHA for d1f1c0b

• Update train.py

  ericl

  authored
  7650e9f

  Copy full SHA for 7650e9f

• Update python/ray/tune/tune.py

  Show description for 54af15c
  

  ericl

  and

  richardliaw

  authored
  54af15c

  Copy full SHA for 54af15c

• Update python/ray/tune/tune.py

  Show description for 8503e17
  

  ericl

  and

  richardliaw

  authored
  8503e17

  Copy full SHA for 8503e17

Commits on Dec 28, 2018

• fix py2 test

  richardliaw

  committed
  4294f42

  Copy full SHA for 4294f42

• Merge branch 'tune_cluster-2' of github.com:richardliaw/ray into tune_cluster-2

  richardliaw

  committed
  b6083b3

  Copy full SHA for b6083b3

• grammar

  richardliaw

  committed
  b8da076

  Copy full SHA for b8da076

• fix

  richardliaw

  committed
  c92313f

  Copy full SHA for c92313f

• fix travis

  richardliaw

  committed
  577da9b

  Copy full SHA for 577da9b