Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify implementation of PVS #515

Merged
merged 1 commit into from
Sep 3, 2023
Merged

Simplify implementation of PVS #515

merged 1 commit into from
Sep 3, 2023

Conversation

brunocodutra
Copy link
Owner

@brunocodutra brunocodutra commented Sep 2, 2023

SPRT

cutechess-cli -sprt elo0=-5 elo1=5 alpha=0.05 beta=0.05 -games 2 -rounds 2000 -openings file=openings-6ply-1000.pgn policy=round -concurrency 4 -ratinginterval 10 -resultformat wide -recover -engine conf=dev -engine conf=base -each proto=uci option.Threads=2 option.Hash=32 tc=3+0.025

Score of dev vs base: 980 - 930 - 2069  [0.506] 3979
...      dev playing White: 531 - 408 - 1051  [0.531] 1990
...      dev playing Black: 449 - 522 - 1018  [0.482] 1989
...      White vs Black: 1053 - 857 - 2069  [0.525] 3979
Elo difference: 4.4 +/- 7.5, LOS: 87.4 %, DrawRatio: 52.0 %
SPRT: llr 3 (101.8%), lbound -2.94, ubound 2.94 - H1 was accepted

Gauntlet

cutechess-cli -tournament gauntlet -games 2 -rounds 1500 -openings file=openings-6ply-1000.pgn policy=round -concurrency 6 -ratinginterval 10 -resultformat wide -recover -engine conf=dev -engine conf=dumb-1.11 -engine conf=Nawito-22.07 -engine conf=Fridolin-4.0 -each option.Threads=2 option.Hash=32 tc=3+0.025

Rank Name                          Elo     +/-   Games    Wins  Losses   Draws   Points   Score    Draw 
   0 dev                           -16       5    9000    2220    2635    4145   4292.5   47.7%   46.1% 
   1 Fridolin-4.0                   25       9    3000     883     671    1446   1606.0   53.5%   48.2% 
   2 Nawito-22.07                   19       9    3000     870     707    1423   1581.5   52.7%   47.4% 
   3 dumb-1.11                       5       9    3000     882     842    1276   1520.0   50.7%   42.5%

STS1-STS15_LAN_v6.epd

python sts_rating.py -f "./epd/STS1-STS15_LAN_v6.epd" -e dev -t 8 --movetime 100 --maxpoint 100

STS Rating v14.2
Engine: chessboard
Hash: 32, Threads: 8, time/pos: 0.100s

Number of positions in ./epd/STS1-STS15_LAN_v6.epd: 1188
Max score = 1188 x 100 = 118800
Test duration: 00h:02m:00s
Expected time to finish: 00h:02m:34s

  STS ID   STS1   STS2   STS3   STS4   STS5   STS6   STS7   STS8   STS9  STS10  STS11  STS12  STS13  STS14  STS15    ALL
  NumPos     85     80     86     89     85     80     82     80     71     79     70     74     75     79     73   1188
 BestCnt     56     50     54     57     62     47     41     47     50     53     42     48     46     48     43    744
   Score   7018   6347   6704   7312   7336   7381   6106   6101   6005   6690   5631   6096   6136   6488   6378  97729
Score(%)   82.6   79.3   78.0   82.2   86.3   92.3   74.5   76.3   84.6   84.7   80.4   82.4   81.8   82.1   87.4   82.3

:: STS ID and Titles ::
STS 01: Undermining
STS 02: Open Files and Diagonals
STS 03: Knight Outposts
STS 04: Square Vacancy
STS 05: Bishop vs Knight
STS 06: Re-Capturing
STS 07: Offer of Simplification
STS 08: Advancement of f/g/h Pawns
STS 09: Advancement of a/b/c Pawns
STS 10: Simplification
STS 11: Activity of the King
STS 12: Center Control
STS 13: Pawn Play in the Center
STS 14: Queens and Rooks to the 7th rank
STS 15: Avoid Pointless Exchange

:: Top 5 STS with high result ::
1. STS 06, 92.3%, "Re-Capturing"
2. STS 15, 87.4%, "Avoid Pointless Exchange"
3. STS 05, 86.3%, "Bishop vs Knight"
4. STS 10, 84.7%, "Simplification"
5. STS 09, 84.6%, "Advancement of a/b/c Pawns"

:: Top 5 STS with low result ::
1. STS 07, 74.5%, "Offer of Simplification"
2. STS 08, 76.3%, "Advancement of f/g/h Pawns"
3. STS 03, 78.0%, "Knight Outposts"
4. STS 02, 79.3%, "Open Files and Diagonals"
5. STS 11, 80.4%, "Activity of the King"

@codecov
Copy link

codecov bot commented Sep 2, 2023

Codecov Report

Patch coverage: 100.00% and project coverage change: -0.28% ⚠️

Comparison is base (dbe1484) 80.33% compared to head (7320300) 80.06%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #515      +/-   ##
==========================================
- Coverage   80.33%   80.06%   -0.28%     
==========================================
  Files          43       43              
  Lines        1246     1239       -7     
==========================================
- Hits         1001      992       -9     
- Misses        245      247       +2     
Files Changed Coverage Δ
lib/search/pvs.rs 94.00% <100.00%> (+0.36%) ⬆️

... and 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@brunocodutra brunocodutra marked this pull request as ready for review September 2, 2023 23:07
@brunocodutra brunocodutra merged commit 9a60ce6 into master Sep 3, 2023
13 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant