23/10/21: Optimized for Android: SFMXnps
SFMXnps is stronger than Stockfish17dev on Android. To proof this and, in particular, to become more familiar with testing on an Android device, a large number of c-chess-cli tournaments were performed and are shown below.
All tournaments were played on a Xiaomi Poco M3 (Android 12, Snapdragon 662, 4 GB RAM) using Termux and the CETSA script, which utilizes c-chess-cli. Concurrency was set to 4 and TCs from 5+0.05s to 120+1.2s with 1 thread per engine were tested. Also one tournament with 2 threads and concurrency 2 was carried out to test the SMP performance. More details are given in the configuration files which can be, together with the played games, downloaded below.
The absolute value of the rating was set to 3100 ELO for each tournament and the rating offsets and all other values were calculated by the script with Bayeselo. Opening suite was UHO_2022_8mvs_+110_+119.epd. It should be noted that such kind of openings enlarge ELO differences but have the advantage of reduced draw rates.
TC: 5+0.05s Threads: 1
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
1 SFMXnps230801 3142 0.0 17 16 250 154.5 61.8 105 46 99 42.0 39.6 3058
2 SFnps230802 3058 83.1 16 17 250 95.5 38.2 46 105 99 18.4 39.6 3142
---------------------------------------------------------------------------------------------------------
TC: 7+0.07s Threads: 1
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
1 SFMXnps230801 3139 0.0 16 16 250 152.5 61.0 98 43 109 39.2 43.6 3061
2 SFnps230802 3061 77.1 16 16 250 97.5 39.0 43 98 109 17.2 43.6 3139
---------------------------------------------------------------------------------------------------------
TC: 10+0.1s Threads: 1
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
1 SFMXnps230801 3142 0.0 16 16 250 155.0 62.0 101 41 108 40.4 43.2 3058
2 SFnps230802 3058 84.5 16 16 250 95.0 38.0 41 101 108 16.4 43.2 3142
---------------------------------------------------------------------------------------------------------
TC: 15+0.15s Threads: 1
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
1 SFMXnps230801 3125 0.0 15 15 250 143.0 57.2 83 47 120 33.2 48.0 3075
2 SFnps230802 3075 50.4 15 15 250 107.0 42.8 47 83 120 18.8 48.0 3125
---------------------------------------------------------------------------------------------------------
TC: 20+0.2s Threads: 1
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
1 SFMXnps230801 3127 0.0 20 19 150 86.5 57.7 51 28 71 34.0 47.3 3073
2 SFnps230802 3073 53.0 19 20 150 63.5 42.3 28 51 71 18.7 47.3 3127
---------------------------------------------------------------------------------------------------------
TC: 25+0.25s Threads: 1
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
1 SFMXnps230801 3124 0.0 19 19 150 85.5 57.0 47 26 77 31.3 51.3 3076
2 SFnps230802 3076 48.2 19 19 150 64.5 43.0 26 47 77 17.3 51.3 3124
---------------------------------------------------------------------------------------------------------
TC: 25+0.25s Threads: 1
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
1 SFMXnps230801 3135 0.0 18 18 150 90.0 60.0 51 21 78 34.0 52.0 3065
2 SFnps230802 3065 69.4 18 18 150 60.0 40.0 21 51 78 14.0 52.0 3135
---------------------------------------------------------------------------------------------------------
TC: 40+0.4s Threads: 1
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
1 SFMXnps230801 3119 0.0 20 20 150 83.0 55.3 50 34 66 33.3 44.0 3081
2 SFnps230802 3081 37.3 20 20 150 67.0 44.7 34 50 66 22.7 44.0 3119
---------------------------------------------------------------------------------------------------------
TC: 60+0.6s Threads: 1
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
1 SFMXnps230801 3124 0.0 18 18 150 85.5 57.0 46 25 79 30.7 52.7 3076
2 SFnps230802 3076 48.5 18 18 150 64.5 43.0 25 46 79 16.7 52.7 3124
---------------------------------------------------------------------------------------------------------
TC: 90+0.9s Threads: 1
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
1 SFMXnps230801 3117 0.0 17 17 150 82.5 55.0 39 24 87 26.0 58.0 3083
2 SFnps230802 3083 34.4 17 17 150 67.5 45.0 24 39 87 16.0 58.0 3117
---------------------------------------------------------------------------------------------------------
TC: 120+1.2s Threads: 1
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
1 SFMXnps230801 3117 0.0 20 20 100 55.0 55.0 24 14 62 24.0 62.0 3083
2 SFnps230802 3083 34.1 20 20 100 45.0 45.0 14 24 62 14.0 62.0 3117
---------------------------------------------------------------------------------------------------------
TC: 120+1.2s Threads: 2
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
1 SFMXnps230801 3121 0.0 20 20 100 56.0 56.0 26 14 60 26.0 60.0 3079
2 SFnps230802 3079 41.1 20 20 100 44.0 44.0 14 26 60 14.0 60.0 3121
---------------------------------------------------------------------------------------------------------
TC: 180+1.0s Threads: 1
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
1 SFMXnps230801 3116 0.0 22 22 100 54.5 54.5 27 18 55 27.0 55.0 3084
2 SFnps230802 3084 31.1 22 22 100 45.5 45.5 18 27 55 18.0 55.0 3116
---------------------------------------------------------------------------------------------------------
TC: 180+1.8s Threads: 1
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
1 SFMXnps230801 3134 0.0 30 30 50 30.0 60.0 17 7 26 34.0 52.0 3066
2 SFnps230802 3066 67.4 30 30 50 20.0 40.0 7 17 26 14.0 52.0 3134
---------------------------------------------------------------------------------------------------------
Δ = delta from the next higher rated opponent
# = number of games played
Σ = total score, 1 point for win, 1/2 point for draw
Older tournaments:
Two recent tourneys added to the graph:
SF16 has a smaller net than current SF17dev and is faster than SF20230802. Could it be that Stockfish16nps is stronger than SFMX?
TC: 10+0.1s Threads: 1
Rank Name Rating Δ + - # Σ Σ% W L D W% =% OppR
1 SFMXnps230801 3137 0.0 19 19 300 173.5 57.8 107 60 133 35.7 44.3 3082
2 SFnps16 3116 20.8 19 19 300 160.5 53.5 92 71 137 30.7 45.7 3092
3 SFnps20230802 3047 69.3 19 19 300 116.0 38.7 49 117 134 16.3 44.7 3127
---------------------------------------------------------------------------------------------------------
Δ = delta from the next higher rated opponent
# = number of games played
Σ = total score, 1 point for win, 1/2 point for draw
Not very likely but not impossible either. Thus the above question will be answered by future tournaments at various TCs.
Edit 09/09/23: See https://github.com/Joachim26/StockfishNPS/releases/tag/Master_DroidSFnps-bb4c63b3
Windows tournaments included in the graph:
Reasonable results, since the speed ratio of the two engines is smaller on Windows (~1.4 for modern builds) than on Android (~1.6). Even more tournaments with (much) more games should be performed to give clearer results.