Code

Batched mcts#147

Code

Open

Whojo wants to merge 167 commits intojonathan-laurent:masterjonathan-laurent/AlphaZero.jl:masterfrom Whojo:2_batched_mcts_PRWhojo/AlphaZero.jl:2_batched_mcts_PRCopy head branch name to clipboard

+1,856-252

Commits

Commits on Jun 29, 2022

• Add private/ in .gitignore

  Whojo

  committed
  c81b3ba

• Add tests on oracle of SimpleMctsTests.jl

  Whojo

  committed
  7147ca5

• Add tests on completed_qvalues() in SimpleMctsTests.jl

  Whojo

  committed
  390c40e

• Add test for explore() in SimpleMctsTests.jl

  Whojo

  committed
  28113e4

• fixup! Add test for explore() in SimpleMctsTests.jl

  Whojo

  committed
  5912561

• Add test on gumbel_explore() in SimpleMctsTests.jl

  Whojo

  committed
  18e6b93

• fixup! Add test on gumbel_explore() in SimpleMctsTests.jl

  Whojo

  committed
  83950e5

• fixup! fixup! Add test on gumbel_explore() in SimpleMctsTests.jl

  Whojo

  committed
  2387f16

• Homogeneise interface between SimpleMcts and BatchedMctsAos

  Whojo

  committed
  115df4c

• Add inference test in BatchedMctsAosTests.jl

  Whojo

  committed
  6321f56

• fixup! Add inference test in BatchedMctsAosTests.jl

  Whojo

  committed
  e9d33bb

• fixup! fixup! Add inference test in BatchedMctsAosTests.jl

  Whojo

  committed
  cbb0e7d

• Add test on explore in BatchedMctsAosTests.jl

  Whojo

  committed
  a2a1a77

• Few renamings for convenience

  Whojo

  committed
  8f70518

Commits on Jul 3, 2022

• Add test on exploration part of gumbel_explore in SimpleMcts

  Whojo

  committed
  5db9817

• Fix issue with terminal nodes on valid_actions list

  Whojo

  committed
  7fb8041

• Remove useless test

  Whojo

  committed
  865f7fa

Commits on Jul 4, 2022

• Correctly save children and bacpropagate in BatchedMctsAos

  Whojo

  committed
  4434011

• Add tests in BatchedMctsAosTests

  Show description for aa97bee
  Whojo

  committed
  aa97bee

Commits on Jul 5, 2022

• Add oracle for BatchedMctsAos

  Whojo

  committed
  1a15149

• Add num_simulations arguments to uniform_mcts_tic_tac_toe

  Whojo

  committed
  8eb1b41

Commits on Jul 7, 2022

• Add validity check on prior move evaluation by oracles in BatchedMctsAos

  Whojo

  committed
  c66772c

Commits on Jul 8, 2022

• Fix completed_qvalues calculation

  Whojo

  committed
  c3cef3d

• Removed RolloutOracle from BatchedMctsAos

  Whojo

  committed
  21960d7

• Remove valid_actions

  Whojo

  committed
  8a0ddd5

Commits on Jul 9, 2022

• fixup! Removed RolloutOracle from BatchedMctsAos

  Whojo

  committed
  0cb8dad

Commits on Jul 11, 2022

• Return, in completed_qvalues, -Inf for invalid actions

  Whojo

  committed
  f5218de

• Merge remote-tracking branch 'Jonathan-laurent/AlphaZero.jl/master' into Add-Mcts-Tests

  Whojo

  committed
  a3cd59d

Commits on Jul 13, 2022

• Remove deadcode in BatchedMctsAosTests

  Whojo

  committed
  457400b

Commits on Jul 14, 2022

• Add assert for action validity in BatchedMctsAos

  Whojo

  committed
  c2da628

Commits on Jul 16, 2022

• Fix type stability in BatchedMctsAos

  Show description for 4abf7bd
  Whojo

  committed
  4abf7bd

• Fix Blue Style issue

  

  Guillaume THOMAS

  committed
  42229fa

Commits on Jul 17, 2022

• SimpleMcts now stops exploration at the first unexplored node

  Show description for 240f80d
  Whojo

  committed
  240f80d

• Homogenize number of simulations between SimpleMcts and BatchedMctsAos

  Whojo

  committed
  6313cc1

• Correct total_rewards on leaf node of BatchedMctsAos

  Whojo

  committed
  ce649a9

• Add equivalence test on BatchedMctsAosTests

  Whojo

  committed
  1bf890b

• Add Gumbel_explore on BatchedMctsAos

  Show description for e4d6498
  Whojo

  committed
  e4d6498

Commits on Jul 18, 2022

• Fix in Gumbel on BatchMctsAos + add tests

  Whojo

  committed
  36f12bd

• Add compilation test for Gumbel on BatchMctsAos

  Whojo

  committed
  501eba3

• Refacto Gumbel on BatchMctsAos

  Show description for c8bc4fd
  Whojo

  committed
  c8bc4fd

• Refacto sorting in Gumbel on BatchMctsAos

  Whojo

  committed
  0f9acc9

• Refacto considered matrix in Gumbel on BatchMctsAos

  Whojo

  committed
  2278bfe

Commits on Jul 21, 2022

• Add code squeleton for BatchedMcts

  Show description for 387ef11
  Whojo

  committed
  387ef11

Commits on Jul 25, 2022

• Add a few tests + fix

  Whojo

  committed
  95e4d15

• Removed terminal from the returned named-tuple of init_fn

  Whojo

  committed
  7ab1d5c

• Merge branch 'Add-Mcts-Tests' into BatchedMcts

  Whojo

  committed
  78ee0a0

• Add exploration test for BatchedMcts

  Whojo

  committed
  704f281

• Add equivalence tests in BatchedMcts

  Whojo

  committed
  8ba50c5

Commits on Jul 27, 2022

• Improve GPU support on BatchedMcts.jl

  

  Guillaume THOMAS

  committed
  05c0f78

• WIP: Gumbel on BatchedMcts

  Whojo

  committed
  813c54f

• Add action to the tuple returned in select! of BatchedMcts

  Whojo

  committed
  d798353

• Add tests for Gumbel explore

  Whojo

  committed
  2b4374c

Commits on Jul 28, 2022

• Fix gumbel_explore of BatchedMcts

  Whojo

  committed
  79a5c2d

Commits on Jul 29, 2022

• Replace typeof(...[1]) by eltype(...)

  

  Guillaume THOMAS

  committed
  dfa00b2

• Remove aids argument from check_oracles

  Whojo

  committed
  c6cc425

• Check action size coherence in check_oracle

  Whojo

  committed
  b169f8c

• Use global variable to improve readibility

  Whojo

  committed
  ae33493

• fixup! Check action size coherence in check_oracle

  Whojo

  committed
  fc70563

• fixup! fixup! Check action size coherence in check_oracle

  Whojo

  committed
  1f1a59e

• Experiment with MCTX formulas & remove useless comments

  Whojo

  committed
  69322f2

Commits on Aug 2, 2022

• Adapt BatchedMCts's explore to compile on GPU

  

  Guillaume THOMAS

  committed
  0b5141e

• Adapt BatchedMcts's gumbel_explore to compile on GPU

  

  Guillaume THOMAS

  committed
  e017a92

• Merge branch 'BatchedMcts' of github.com:Whojo/AlphaZero.jl into BatchedMcts

  Whojo

  committed
  6abc086

• Merge branch 'Add-Mcts-Tests' into Gumbel_on_BatchMctsAos

  Whojo

  committed
  03cffda

• Removed GPU test for gumbel_explore of BatchedAos

  Whojo

  committed
  0082e65

• Merge branch 'Gumbel_on_BatchMctsAos' into benchmark_mcts

  Whojo

  committed
  bbe37fd

Commits on Aug 3, 2022

• Removed useless dependencies

  Whojo

  committed
  cc2f217

• Fixed typos in documentation of BatchedMcts

  Whojo

  committed
  e2da1e7

• Formated BatchedMcts + refacto valid_actions in UniformTicTacToeEnvOracle

  Whojo

  committed
  7d32e35

• Add a small documentation for Tree in BatchedMcts

  Whojo

  committed
  db72f97

• Replace a loop with eachindex

  Whojo

  committed
  f32bdde

• WIP: convert eval! to GPU

  Whojo

  committed
  3d8e0f5

• Merge branch 'BatchedMcts' of github.com:Whojo/AlphaZero.jl into BatchedMcts

  Whojo

  committed
  9a0d0e6

Commits on Aug 4, 2022

• Adapt transition_fn for GPU

  Whojo

  committed
  f5926cb

• Merge branch 'BatchedMcts' into benchmark_mcts

  Whojo

  committed
  715149e

• Copy directly gumbel and table_of_considered_visits to GPU in BatchedMcts

  Whojo

  committed
  cfffb3a

Commits on Aug 5, 2022

• Included the dimensions of internal_states in tree.state

  Show description for f0a448f
  Whojo

  committed
  f0a448f

Commits on Aug 6, 2022

• Add usage documentation

  Whojo

  committed
  40f8749

• fixup! Add usage documentation

  Whojo

  committed
  ac59571

• fixup! Add usage documentation

  Whojo

  committed
  132895c

• Fix tree.state access on GPU

  Whojo

  committed
  bda7125

Commits on Aug 7, 2022

• Add comments to EnvOracle

  Whojo

  committed
  f716eac

• Add comments for UniformTicTacToeEnvOracle

  Whojo

  committed
  37887f1

• Add comments to Policy of BatchedMcts

  Whojo

  committed
  4ec1f43

• Add comments for Tree in BatchedMcts

  Whojo

  committed
  f56e682

Commits on Aug 8, 2022

• Add docstring for utility functions of MCTS

  Whojo

  committed
  0d496a9

• fixup! Add docstring for utility functions of MCTS

  Whojo

  committed
  006e97b

Commits on Aug 9, 2022

• Add MCTS documentation

  Whojo

  committed
  cbdfd81

• Add a comment about doubly-linked tree

  Whojo

  committed
  f16bd14

• Add Gumbel documentation

  Whojo

  committed
  c71a61b

• Add a part on namming conventions in the documentation

  Whojo

  committed
  ecf7028

• Correct typos and grammar error.

  Whojo

  committed
  b294a8e

• fixup! Correct typos and grammar error.

  Whojo

  committed
  7dd0ecc

Commits on Aug 13, 2022

• transition_fn on GPU at 100%

  Whojo

  committed
  30b4979

• Change parent_frontier to Matrix

  Whojo

  committed
  f2a6270

• Replace boolean indexing by direct int indexing

  Show description for a8fa4a3
  Whojo

  committed
  a8fa4a3

• Add @inbounds in eval!

  Whojo

  committed
  124e229

• Improved Data preparation performance in eval! of BatchedMcs

  Whojo

  committed
  458f2bd

Commits on Aug 20, 2022

• Minor typos fixes

  Whojo

  committed
  fc41318

• Add utility completed_qvalues

  Whojo

  committed
  7f97caa

Commits on Aug 21, 2022

• Add GameHistory

  Whojo

  committed
  36f0aae

• Add ReplayBuffer

  Whojo

  committed
  f7500f2

• Add TrainableEnvOracle

  Whojo

  committed
  eaf26ce

• Add Train.jl

  Whojo

  committed
  ca5923b

Commits on Aug 22, 2022

• Add make_image to BitwiseTicTacToe

  Whojo

  committed
  e755014

• fixup! Add TrainableEnvOracle

  Whojo

  committed
  d2ad581

• Rename train to train_network

  Whojo

  committed
  f394ec2

• fixup! Add make_image to BitwiseTicTacToe

  Whojo

  committed
  2895810

• fixup! Add make_image to BitwiseTicTacToe

  Whojo

  committed
  ce0e09d

Commits on Aug 23, 2022

• First working MuZero (without DL)

  Whojo

  committed
  1f85b5f

Commits on Aug 25, 2022

• Refacto validate_prior with l1_normalise

  Whojo

  committed
  4fc69c2

Commits on Aug 26, 2022

• Fixed "Usage" documentation of BatchedMcts

  Whojo

  committed
  e362624

• fixup! Refacto validate_prior with l1_normalise

  Whojo

  committed
  bd25e88

• fixup! Add make_image to BitwiseTicTacToe

  Whojo

  committed
  d171929

• Group batches in Train.jl

  Whojo

  committed
  c368ab3

• Validate prior after transition_fn

  Whojo

  committed
  adef141

Commits on Aug 28, 2022

• Removed terminated environment from self-play

  Whojo

  committed
  ec90bf6

• WIP: DL part of update_weights

  Show description for 351f581
  Whojo

  committed
  351f581

• Update BatchedMcts Documentation on value_prior & policy_prior

  Whojo

  committed
  19d5347

• fixup! Refacto validate_prior with l1_normalise

  Whojo

  committed
  4bc385e

• Invalid actions in completed_qvalues now returns -1

  Whojo

  committed
  589c937

• Rescale completed_qvalues to [0; 1]

  Whojo

  committed
  0fcccea

Commits on Aug 29, 2022

• Revert "Validate prior after transition_fn"

  Show description for c1ca400
  Whojo

  committed
  c1ca400

• Rename a few variable for clarity

  Whojo

  committed
  1bf70cd

Commits on Aug 30, 2022

• MCTS's explore manipulates states instead of envs

  Whojo

  committed
  5bbb4f7

• Correct issue with multiple dimensions state in BatchedMcts.jl

  Whojo

  committed
  4cd729b

• WIP: inference in get_env_oracle

  Whojo

  committed
  ff862e3

Commits on Aug 31, 2022

• Explore in Train.jl is based on environments (not states)

  Show description for 5bcf553
  Whojo

  committed
  5bcf553

• Add logging information to Train.jl

  Whojo

  committed
  36d869a

• fixup! Explore in Train.jl is based on environments (not states)

  Whojo

  committed
  9ecd1bd

Commits on Sep 1, 2022

• Add a dummy-run on TicTacToe_MuZero script

  Show description for 82148e7
  Whojo

  committed
  82148e7

• fixup! fixup! Explore in Train.jl is based on environments (not states)

  Whojo

  committed
  b41a43e

• Refactored dummy run of TicTacToe_MuZero script

  Whojo

  committed
  ab6670e

Commits on Sep 2, 2022

• Refacto BitwiseTicTacToe's vectorize_state

  Whojo

  committed
  ef2355b

• Improved MuZero.jl's transition_fn performance

  Whojo

  committed
  dce9f1d

Commits on Sep 7, 2022

• fixup! Add logging information to Train.jl

  Whojo

  committed
  c07bd66

• Log losses with @info

  Whojo

  committed
  067d7f1

• fixup! Log losses with @info

  Whojo

  committed
  25c1798

Commits on Sep 8, 2022

• Fixed a typo in a create_tree assert

  Whojo

  committed
  c466bb3

• Add a win-rates metrics logging to Train.jl

  Whojo

  committed
  de44286

Commits on Sep 11, 2022

• MuZero's get_env_oracle now return the real invalid actions in init_fn

  Whojo

  committed
  f4fd3c8

• Temporarily set invalid_actions_value to -2 before rescale inside completed_qvalues

  Whojo

  committed
  a23f258

• Add seed in TicTacToe_MuZero

  Whojo

  committed
  024399b

• Updated description of BatchedMcts.jl

  Whojo

  committed
  5f0d77d

• Replace use of mcts in kernel inside BatchedMcts.jl

  Whojo

  committed
  e322d3c

• Code Style

  Whojo

  committed
  1afd001

Commits on Sep 12, 2022

• Factored Policy & EnvOracle separatly from BatchedMcts.jl

  Whojo

  committed
  b95c400

• fixup! Factored Policy & EnvOracle separatly from BatchedMcts.jl

  Whojo

  committed
  4d58e66

• Temporary removing gumbel_explore from AoS

  Whojo

  committed
  b90737a

• fixup! Factored Policy & EnvOracle separatly from BatchedMcts.jl

  Whojo

  committed
  ba89bae

• Add EnvOracle to BatchedMctsAos.jl

  Whojo

  committed
  bd14a97

Commits on Sep 13, 2022

• Removed mcts from kernels in BatchedMctsAos.jl

  Whojo

  committed
  1abfcfa

• Move get_considered_visits_table to BatchedMctsUtility

  Show description for e145030
  Whojo

  committed
  e145030

• Add gumbel_explore to BatchedMctsAos

  Show description for 32e7c66
  Whojo

  committed
  32e7c66

Commits on Sep 15, 2022

• Fix GPU Compilation

  Whojo

  committed
  86b76d3

Commits on Sep 19, 2022

• fixup! Fix GPU Compilation

  Whojo

  committed
  bf82e9e

• Removed MuZero related code for PR

  Whojo

  committed
  89dc43b

Commits on Sep 24, 2022

• BatchedMcts validate_prior on CuArray

  Whojo

  committed
  90338dd

• Merge branch 'BatchedMctsAos' into 2_batched_mcts_PR

  Whojo

  committed
  65c5574

Commits on Sep 25, 2022

• Revert "BatchedMcts validate_prior on CuArray"

  Show description for 3d43a8e
  Whojo

  committed
  3d43a8e

• Handle embedding in BatchedMctsAos's internal_states

  Whojo

  committed
  9f37342

• Removed scalar indexing inside BatchedMctsAos's eval_state!

  Whojo

  committed
  13d9b56

• fixup! Removed scalar indexing inside BatchedMctsAos's eval_state!

  Whojo

  committed
  33dcc5a

• Merge branch 'BatchedMctsAos' into 2_batched_mcts_PR

  Whojo

  committed
  32747c0

Commits on Sep 26, 2022

• Adapt validate_prior to process CuArray

  Whojo

  committed
  1528fcb

• vectorize_state GPU friendly

  Whojo

  committed
  157d7a6

• Merge branch 'BatchedMctsAos' into 2_batched_mcts_PR

  Whojo

  committed
  60239a0