Potential issue leading to wrong PV with a losing move instead of drawing move

[Videodr0me]

Describe the issue

Bug also present with newest dev20230128 which includes the tb changes.

This is a drawn 8 piece position: 8/P7/5R2/6k1/6p1/7p/5P1K/5r2 b - - 0 61

If you let stockfish_23011407_x64_avx2 go infinite on this position with partial 7 piece syzygy WDL bases (and all 6p WDL + DTZ bases) it correctly shows 0.00.

If you now enter the move Rf1a1 SF often (but not always) shows that white is winning and wants to play f3?? The position in fact remains drawn and f3 is one of the few losing moves.
Even if it does not happen and it shows 0.00 with one of the many drawing moves you can go back and forth (without clearing hash) and very often SF will return to showing the bad f3 move.

All tb files have been md5 validated. Here is a list of the 7p files i use:

7p_WDL.txt

Is this a bug? Is it expected behaviour with partial 7p wdl bases? And if the latter which one's am i missing?

Expected behavior

After entering Rf1a1 and go infinite SF should also show a drawn position and come up with one of the many non losing moves.

Steps to reproduce

Just let SF calculate on the original position, enter Rf1a1, and go infinite.

Anything else?

No response

Operating system

Windows

Stockfish version

stockfish_23011407_x64_avx2

[RogerThiede]

What depths do you see an issue, and have you found if searching deeper resolves the issue?
What conditions have you set where you see it "playing a losing move"? You discussed a go infinite and so it will never actually play a move.

[Videodr0me]

What depths do you see an issue, and have you found if searching deeper resolves the issue? What conditions have you set where you see it "playing a losing move"? You discussed a go infinite and so it will never actually play a move.

Sorry, for being a little vague in the initial post. I found the time to reproduce and capture the console input and outputs. See the attached files.
Let me try to re-explain whats going on and some new findings.

I call "8/P7/5R2/6k1/6p1/7p/5P1K/5r2 b - - 0 61" the first position and "8/P7/5R2/6k1/6p1/7p/5P1K/5r2 b - - 0 61 moves f1a1" the second position. If the search on the first position reaches a high depth (approx. > 240 maybe 245) and you then search the second position (very often) SF presents a completely wrong pv leading to a a loss with mate. It is also reproducable with one thread but takes much longer. Also varying hash does not seem to matter much. I am no longer convinced it has something to do with the tablebases. It seems that the high search depth might confuse SF (maybe its hash replacement strategy and PV reconstruction).

You are correct though that SF on "go infinite" SF would never actually play a move. And here it gets interesting. SF displays the wrong losing line and never change to the correct line with "go infinite". So i reproduced without "go infinite" and used "go movetime"´instead. As you can see from the console output SF displays the wrong line just as with "go infinite". But surprisingly it will always play the correct move at the end. Also with shorter time it somehow always outputs the correct move to play (best move), while the pv remains incorrect. In the console output you can clearly see that, the moment the search of the first position reaches depth 245 (near the end of the output) the next search of the second position fails (the last search in the output).

My current understanding is now this. For actual play this might be a cosmetic bug, while for analysis it seems like a serious issue (as you almost always use the infinite mode). As the problem seems unrelated to indeterministic search (as you can reproduce on one thread) i would speculate that hash reconstruction of the PV somehow fails if previous entries are near or at high depths > 240 or so.

In the console output i used searching both positions for a bit and then alternating to reach the required depth. Maybe that somehow also confuses SF hash entries (it should not, but its a possibility). One could try to search directly to a depth of 240+ on the first search and see if that influences anything.

Stockfish console.txt

[Disservin]

I cannot reproduce any of this.

Stockfish dev-20230223-69639d764 by the Stockfish developers (see AUTHORS file)
position fen 8/P7/5R2/6k1/6p1/7p/5P1K/5r2 b - - 0 61 moves f1a1
go
info string NNUE evaluation using nn-1337b1adec5b.nnue enabled
info depth 1 seldepth 1 multipv 1 score cp -1 nodes 40 nps 20000 hashfull 0 tbhits 0 time 2 pv f6f7
info depth 2 seldepth 2 multipv 1 score cp 8 nodes 100 nps 50000 hashfull 0 tbhits 0 time 2 pv f6f8 a1a7
info depth 3 seldepth 3 multipv 1 score cp 5 nodes 237 nps 118500 hashfull 0 tbhits 0 time 2 pv f6f8 g4g3 h2h3 a1a7
info depth 4 seldepth 4 multipv 1 score cp 5 nodes 301 nps 150500 hashfull 0 tbhits 0 time 2 pv f6f8 g4g3 h2h3 a1a7
info depth 5 seldepth 5 multipv 1 score cp 6 nodes 625 nps 208333 hashfull 0 tbhits 0 time 3 pv f6f8 a1a7 h2g3 a7a2
info depth 6 seldepth 6 multipv 1 score cp 1 nodes 1400 nps 466666 hashfull 0 tbhits 0 time 3 pv f6f8 a1a7 h2g3 g5g6
info depth 7 seldepth 7 multipv 1 score cp 0 nodes 2942 nps 735500 hashfull 1 tbhits 0 time 4 pv f6f8 a1a7 f8g8 g5f5 g8f8 f5g5
info depth 8 seldepth 8 multipv 1 score cp 0 nodes 6299 nps 1049833 hashfull 2 tbhits 0 time 6 pv f6f7 a1a4 f7g7 g5f5 g7g8 a4a7
info depth 9 seldepth 10 multipv 1 score cp 0 nodes 10606 nps 1325750 hashfull 4 tbhits 0 time 8 pv f6f7 a1a3 f7h7 g5g6 h7d7
info depth 10 seldepth 10 multipv 1 score cp 0 nodes 14697 nps 1336090 hashfull 7 tbhits 0 time 11 pv f6f7 a1a3 f7b7 g5f5 b7g7 f5f6 g7h7
info depth 11 seldepth 11 multipv 1 score cp 0 nodes 21878 nps 1562714 hashfull 9 tbhits 0 time 14 pv f6f7 a1a3 f7b7 a3a2 h2g3 a2a3
info depth 12 seldepth 7 multipv 1 score cp 0 nodes 25200 nps 1575000 hashfull 10 tbhits 0 time 16 pv f6f7 a1a3 f7b7 a3a2 h2g3 a2a3
info depth 13 seldepth 14 multipv 1 score cp 0 nodes 40038 nps 1740782 hashfull 14 tbhits 0 time 23 pv f6f7 a1a3 f7b7 a3a4 b7g7 g5h5 g7h7 h5g6 h7d7 a4a2 h2g3 a2a3 g3g4
info depth 14 seldepth 15 multipv 1 score cp 0 nodes 60966 nps 1793117 hashfull 21 tbhits 0 time 34 pv f6f7 a1a3 f7b7 a3a4 b7g7 g5h6 g7f7 a4a2 h2g3 h6g6 f7d7 a2a3 f2f3 a3f3 g3g4
info depth 15 seldepth 16 multipv 1 score cp 0 nodes 74688 nps 1867200 hashfull 25 tbhits 0 time 40 pv f6f7 a1a3 f7b7 a3a4 b7d7 g5f5 f2f3 a4a2 h2g3 h3h2 a7a8q a2a8 d7d1
info depth 16 seldepth 17 multipv 1 score cp 0 nodes 88908 nps 1852250 hashfull 29 tbhits 0 time 48 pv f6f7 a1a3 f7b7 a3a4 b7d7 g5f5 f2f3 a4a2 h2g3 a2g2 g3h4 h3h2 d7d1 h2h1q d1h1 g2a2 h4g3
...
info depth 44 seldepth 48 multipv 1 score cp 0 nodes 16197337 nps 2304030 hashfull 996 tbhits 0 time 7030 pv f6f7 a1a3 f7g7 g5f5 g7d7 a3a1 h2g3 a1a2 d7f7 f5g5 f7d7 g5g6 d7b7 g6f6 b7b6 f6g5 b6b5 g5h6 b5b6
info depth 45 seldepth 42 multipv 1 score cp 0 nodes 24303720 nps 2233797 hashfull 1000 tbhits 0 time 10880 pv f6f7 a1a3 f7g7 g5f5 g7d7 a3a1 h2g3 a1a2 d7f7 f5g5 f7d7 a2a3 g3h2 g5f5 d7f7 f5g6 f7b7 a3a1 h2g3 g6g5 g3h2 g5f6 h2g3 a1a2 b7b6 f6f5 b6b5 f5g6 b5b6 g6f5

Since you also hid the "most" important part in the text file, ill repost that

info depth 241 seldepth 3 multipv 1 score cp 0 nodes 1638238 nps 32122313 hashfull 240 tbhits 153503 time 51 pv g4g3 h2h3
info depth 242 seldepth 3 multipv 1 score cp 0 nodes 1644436 nps 32243843 hashfull 241 tbhits 154237 time 51 pv g4g3 h2h3
info depth 243 seldepth 3 multipv 1 score cp 0 nodes 1660782 nps 32564352 hashfull 242 tbhits 156134 time 51 pv g4g3 h2h3
info depth 244 seldepth 3 multipv 1 score cp 0 nodes 1892952 nps 31549200 hashfull 271 tbhits 184873 time 60 pv g4g3 h2h3
info depth 245 seldepth 3 multipv 1 score cp 0 nodes 186556333 nps 18653767 hashfull 821 tbhits 16629917 time 10001 pv g4g3 h2h3
bestmove g4g3 ponder h2h3
position fen 8/P7/5R2/6k1/6p1/7p/5P1K/5r2 b - - 0 61 moves f1a1
go movetime 10000
info string NNUE evaluation using nn-1337b1adec5b.nnue enabled
info depth 1 seldepth 1 multipv 1 score mate -10 nodes 3244 nps 1622000 hashfull 0 tbhits 79 time 2 pv f2f3 a1a2 h2g1 h3h2 g1h1
info depth 2 seldepth 2 multipv 1 score cp -475 nodes 14981 nps 4993666 hashfull 0 tbhits 603 time 3 pv f2f3 g5f6
info depth 3 seldepth 3 multipv 1 score cp -19998 nodes 27737 nps 9245666 hashfull 0 tbhits 1176 time 3 pv f2f3 a1a7 h2g1 g5f6 f3g4
info depth 4 seldepth 4 multipv 1 score cp -19998 nodes 41391 nps 13797000 hashfull 0 tbhits 1760 time 3 pv f2f3 a1a7 h2g1 g5f6 f3g4
info depth 5 seldepth 5 multipv 1 score cp -19998 nodes 54686 nps 18228666 hashfull 0 tbhits 2344 time 3 pv f2f3 a1a7 f6f8 a7a2 h2g1
...
info depth 71 seldepth 21 multipv 1 score mate -10 nodes 417723836 nps 41768206 hashfull 780 tbhits 22662983 time 10001 pv f2f3 a1a2 h2g1 g4g3 a7a8q h3h2 g1h1 a2a8 h1g2 g5f6 f3f4 f6f5 g2g3 h2h1q g3f2 a8a2 f2g3 h1h2 g3f3 h2f4
info depth 179 seldepth 3 multipv 1 score cp 0 nodes 417723836 nps 41764030 hashfull 780 tbhits 22662983 time 10002 pv a7a8q a1a8
bestmove a7a8q ponder a1a8

so the previous search reached maxdepth, maybe theres some weird interaction ?

[Videodr0me]

I cannot reproduce any of this.

You did not have a previous search reaching maxdepth. This is probably what triggers the bug. Please retry exactly as i did in the log file i provided. If you alternate searches it will reach maxdepth even on slow machines reasonably quickly.

so the previous search reached maxdepth, maybe theres some weird interaction ?

Exactly - and this obviously should not happen and is some sort of bug.

[Disservin]

I tried reaching max depth by alternated searches but only ever got up to 100. Though maybe one can lower the max depth and reach the limit earlier to check if it changes anything

[Videodr0me]

I tried reaching max depth by alternated searches but only ever got up to 100. Though maybe one can lower the max depth and reach the limit earlier to check if it changes anything

I have partial 7p TB installed, this might help reaching maxdepth. Also you could try running the alternate searches for longer, and alternate more often if the machine is slower.

[vondele]

@Videodr0me can you reproduce the problem if you have only 6men in use, i.e. not partial 7men. Correct partial 7men require for each TB file available also the corresponding 7men where one of the pawns promotes. For example, you have KPPPPvKR so you will need all these files present https://syzygy-tables.info/download/KPPPPvKR.txt?source=lichess&dtz=root

[Videodr0me]

@Videodr0me can you reproduce the problem if you have only 6men in use, i.e. not partial 7men. Correct partial 7men require for each TB file available also the corresponding 7men where one of the pawns promotes. For example, you have KPPPPvKR so you will need all these files present https://syzygy-tables.info/download/KPPPPvKR.txt?source=lichess&dtz=root

With 6p tablebases only, I can only get to depth 120 or so in reasonable time. I think the missing pawn promotion wdl bases are no issue in this position, but maybe somebody with a complete set can try to reproduce to be sure.