Skip to content

Commit

Permalink
Update default net to nn-b1e55edbea57.nnue
Browse files Browse the repository at this point in the history
Created by retraining the master big net `nn-0000000000a0.nnue` on the same
dataset with the ranger21 optimizer and more WDL skipping at training time.

More WDL skipping is meant to increase lambda accuracy and train on fewer
misevaluated positions where position scores are unlikely to correlate
with game outcomes. Inspired by:
- repeated reports in discord #events-discuss about SF misplaying due to wrong endgame
  evals, possibly due to Leela's endgame weaknesses reflected in training data
- an attempt to reduce the skewed dataset piece count distribution where there
  are much more positions with less than 16 pieces, since the target piece count
  distribution in the trainer is symmetric around 16

The faster convergence seen with ranger21 is meant to:
- prune experiment ideas more quickly since fewer epochs are needed to reach elo maxima
- research faster potential trainings by shortening each run

```yaml
experiment-name: 2560-S7-Re-514G-ranger21-more-wdl-skip
training-dataset: /data/S6-514G.binpack
early-fen-skipping: 28

start-from-engine-test-net: True
nnue-pytorch-branch: linrock/nnue-pytorch/r21-more-wdl-skip

num-epochs: 1200
lr: 4.375e-4
gamma: 0.995
start-lambda: 1.0
end-lambda: 0.7
```

Experiment yaml configs converted to easy_train.sh commands with:
https://github.com/linrock/nnue-tools/blob/4339954/yaml_easy_train.py

Implementations based off of Sopel's NNUE training & experimentation log:
https://docs.google.com/document/d/1gTlrr02qSNKiXNZ_SuO4-RjK4MXBiFlLE6jvNqqMkAY
- Experiment 336 - ranger21 https://github.com/Sopel97/nnue-pytorch/tree/experiment_336
- Experiment 351 - more WDL skipping

The version of the ranger21 optimizer used is:
https://github.com/lessw2020/Ranger21/blob/b507df6/ranger21/ranger21.py

The dataset is the exact same as in:
official-stockfish#4782

Local elo at 25k nodes per move:
nn-epoch619.nnue : 6.2 +/- 4.2

Passed STC:
https://tests.stockfishchess.org/tests/view/658a029779aa8af82b94fbe6
LLR: 2.93 (-2.94,2.94) <0.00,2.00>
Total: 46528 W: 11985 L: 11650 D: 22893
Ptnml(0-2): 154, 5489, 11688, 5734, 199

Passed LTC:
https://tests.stockfishchess.org/tests/view/658a448979aa8af82b95010f
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 265326 W: 66378 L: 65574 D: 133374
Ptnml(0-2): 153, 30175, 71254, 30877, 204

This was additionally tested with the latest DualNNUE and passed SPRTs:

Passed STC vs. official-stockfish#4919
https://tests.stockfishchess.org/tests/view/658bcd5c79aa8af82b951846
LLR: 2.93 (-2.94,2.94) <0.00,2.00>
Total: 296128 W: 76273 L: 75554 D: 144301
Ptnml(0-2): 1223, 35768, 73617, 35979, 1477

Passed LTC vs. official-stockfish#4919
https://tests.stockfishchess.org/tests/view/658c988d79aa8af82b95240f
LLR: 2.95 (-2.94,2.94) <0.50,2.50>
Total: 75618 W: 19085 L: 18680 D: 37853
Ptnml(0-2): 45, 8420, 20497, 8779, 68

closes official-stockfish#4942

Bench: 1304666
  • Loading branch information
linrock authored and Disservin committed Dec 30, 2023
1 parent bab1cc3 commit f12035c
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/evaluate.h
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ extern std::string currentEvalFileName;
// The default net name MUST follow the format nn-[SHA256 first 12 digits].nnue
// for the build process (profile-build and fishtest) to work. Do not change the
// name of the macro, as it is used in the Makefile.
#define EvalFileDefaultName "nn-0000000000a0.nnue"
#define EvalFileDefaultName "nn-b1e55edbea57.nnue"

namespace NNUE {

Expand Down

0 comments on commit f12035c

Please sign in to comment.