Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix calculation of best ops/sec in benchmarks #6973

Merged
merged 2 commits into from
Jun 29, 2024
Merged

Conversation

SiarheiFedartsou
Copy link
Member

@SiarheiFedartsou SiarheiFedartsou commented Jun 26, 2024

Benchmark Results

Benchmark Base PR
alias aliased u32: 1086.11
plain u32: 1081.2
aliased double: 951.047
plain double: 942.633
aliased u32: 1088.22
plain u32: 1094.7
aliased double: 961.666
plain double: 954.113
e2e_match_ch Ops: 42.77 ± 0.19 ops/s. Best: 43.00 ops/s
Total: 3062.18ms ± 13.54ms. Best: 3046.72ms
Min time: 2.23ms ± 0.02ms
Mean time: 23.38ms ± 0.10ms
Median time: 17.36ms ± 0.08ms
95th percentile: 77.55ms ± 1.55ms
99th percentile: 93.73ms ± 1.46ms
Max time: 97.74ms ± 1.67ms
Ops: 43.52 ± 0.15 ops/s. Best: 43.75 ops/s
Total: 3010.81ms ± 10.29ms. Best: 2994.11ms
Min time: 2.18ms ± 0.03ms
Mean time: 22.98ms ± 0.08ms
Median time: 16.86ms ± 0.13ms
95th percentile: 75.69ms ± 0.23ms
99th percentile: 93.31ms ± 0.95ms
Max time: 96.76ms ± 0.36ms
e2e_match_mld Ops: 62.37 ± 0.11 ops/s. Best: 62.61 ops/s
Total: 2100.32ms ± 3.72ms. Best: 2092.26ms
Min time: 1.84ms ± 0.04ms
Mean time: 16.03ms ± 0.03ms
Median time: 8.65ms ± 0.09ms
95th percentile: 53.19ms ± 0.11ms
99th percentile: 61.34ms ± 0.23ms
Max time: 71.19ms ± 0.39ms
Ops: 62.34 ± 0.07 ops/s. Best: 62.47 ops/s
Total: 2101.34ms ± 2.39ms. Best: 2096.85ms
Min time: 1.84ms ± 0.05ms
Mean time: 16.04ms ± 0.02ms
Median time: 8.55ms ± 0.07ms
95th percentile: 53.23ms ± 0.19ms
99th percentile: 61.79ms ± 0.27ms
Max time: 71.73ms ± 1.25ms
e2e_nearest_ch Ops: 784.86 ± 4.77 ops/s. Best: 791.46 ops/s
Total: 1274.31ms ± 7.79ms. Best: 1263.49ms
Min time: 1.08ms ± 0.01ms
Mean time: 1.27ms ± 0.01ms
Median time: 1.18ms ± 0.01ms
95th percentile: 1.69ms ± 0.02ms
99th percentile: 1.77ms ± 0.02ms
Max time: 4.96ms ± 3.03ms
Ops: 806.48 ± 4.84 ops/s. Best: 812.31 ops/s
Total: 1239.91ms ± 7.75ms. Best: 1231.06ms
Min time: 1.06ms ± 0.01ms
Mean time: 1.24ms ± 0.01ms
Median time: 1.15ms ± 0.01ms
95th percentile: 1.64ms ± 0.01ms
99th percentile: 1.69ms ± 0.01ms
Max time: 4.25ms ± 2.54ms
e2e_nearest_mld Ops: 781.73 ± 5.91 ops/s. Best: 794.21 ops/s
Total: 1279.38ms ± 9.80ms. Best: 1259.10ms
Min time: 1.07ms ± 0.01ms
Mean time: 1.28ms ± 0.01ms
Median time: 1.18ms ± 0.01ms
95th percentile: 1.71ms ± 0.02ms
99th percentile: 1.79ms ± 0.02ms
Max time: 5.33ms ± 3.32ms
Ops: 806.37 ± 4.01 ops/s. Best: 811.71 ops/s
Total: 1240.10ms ± 6.75ms. Best: 1231.97ms
Min time: 1.06ms ± 0.01ms
Mean time: 1.24ms ± 0.01ms
Median time: 1.15ms ± 0.01ms
95th percentile: 1.64ms ± 0.01ms
99th percentile: 1.69ms ± 0.01ms
Max time: 4.23ms ± 2.41ms
e2e_route_ch Ops: 311.72 ± 1.62 ops/s. Best: 315.09 ops/s
Total: 3208.24ms ± 17.61ms. Best: 3173.72ms
Min time: 1.33ms ± 0.02ms
Mean time: 3.21ms ± 0.02ms
Median time: 3.24ms ± 0.02ms
95th percentile: 4.28ms ± 0.03ms
99th percentile: 4.75ms ± 0.07ms
Max time: 7.42ms ± 2.15ms
Ops: 347.08 ± 1.36 ops/s. Best: 348.91 ops/s
Total: 2881.23ms ± 11.19ms. Best: 2866.05ms
Min time: 1.28ms ± 0.01ms
Mean time: 2.88ms ± 0.01ms
Median time: 2.90ms ± 0.02ms
95th percentile: 3.79ms ± 0.02ms
99th percentile: 4.19ms ± 0.04ms
Max time: 6.67ms ± 1.84ms
e2e_route_mld Ops: 247.22 ± 5.02 ops/s. Best: 252.98 ops/s
Total: 4048.20ms ± 88.16ms. Best: 3952.82ms
Min time: 1.37ms ± 0.03ms
Mean time: 4.05ms ± 0.09ms
Median time: 4.11ms ± 0.10ms
95th percentile: 5.56ms ± 0.11ms
99th percentile: 6.12ms ± 0.15ms
Max time: 8.59ms ± 1.84ms
Ops: 284.15 ± 7.15 ops/s. Best: 291.22 ops/s
Total: 3522.05ms ± 94.49ms. Best: 3433.88ms
Min time: 1.26ms ± 0.01ms
Mean time: 3.52ms ± 0.09ms
Median time: 3.56ms ± 0.10ms
95th percentile: 4.82ms ± 0.16ms
99th percentile: 5.29ms ± 0.18ms
Max time: 7.69ms ± 1.70ms
e2e_table_ch Ops: 286.55 ± 7.01 ops/s. Best: 295.50 ops/s
Total: 3489.37ms ± 89.26ms. Best: 3384.04ms
Min time: 1.85ms ± 0.07ms
Mean time: 3.49ms ± 0.09ms
Median time: 3.48ms ± 0.09ms
95th percentile: 4.79ms ± 0.10ms
99th percentile: 5.15ms ± 0.11ms
Max time: 8.47ms ± 3.26ms
Ops: 317.05 ± 1.06 ops/s. Best: 318.77 ops/s
Total: 3154.13ms ± 10.50ms. Best: 3137.01ms
Min time: 1.69ms ± 0.02ms
Mean time: 3.15ms ± 0.01ms
Median time: 3.15ms ± 0.02ms
95th percentile: 4.33ms ± 0.02ms
99th percentile: 4.65ms ± 0.02ms
Max time: 7.49ms ± 2.81ms
e2e_table_mld Ops: 110.43 ± 0.30 ops/s. Best: 110.92 ops/s
Total: 9055.48ms ± 26.82ms. Best: 9015.63ms
Min time: 3.73ms ± 0.02ms
Mean time: 9.06ms ± 0.03ms
Median time: 9.02ms ± 0.04ms
95th percentile: 13.82ms ± 0.05ms
99th percentile: 14.75ms ± 0.09ms
Max time: 17.66ms ± 2.43ms
Ops: 109.68 ± 0.88 ops/s. Best: 110.67 ops/s
Total: 9117.71ms ± 73.99ms. Best: 9036.19ms
Min time: 3.68ms ± 0.03ms
Mean time: 9.12ms ± 0.07ms
Median time: 9.05ms ± 0.07ms
95th percentile: 13.92ms ± 0.11ms
99th percentile: 14.90ms ± 0.25ms
Max time: 17.78ms ± 1.94ms
e2e_trip_ch Ops: 87.26 ± 0.78 ops/s. Best: 88.86 ops/s
Total: 11467.28ms ± 105.08ms. Best: 11253.04ms
Min time: 1.94ms ± 0.04ms
Mean time: 11.46ms ± 0.11ms
Median time: 11.00ms ± 0.12ms
95th percentile: 19.85ms ± 0.15ms
99th percentile: 21.51ms ± 0.22ms
Max time: 23.12ms ± 0.36ms
Ops: 97.20 ± 0.20 ops/s. Best: 97.48 ops/s
Total: 10289.35ms ± 21.10ms. Best: 10258.48ms
Min time: 1.58ms ± 0.10ms
Mean time: 10.29ms ± 0.02ms
Median time: 9.77ms ± 0.02ms
95th percentile: 18.21ms ± 0.05ms
99th percentile: 19.95ms ± 0.19ms
Max time: 22.48ms ± 1.12ms
e2e_trip_mld Ops: 52.56 ± 0.44 ops/s. Best: 53.66 ops/s
Total: 19026.74ms ± 157.11ms. Best: 18634.51ms
Min time: 1.92ms ± 0.29ms
Mean time: 19.03ms ± 0.16ms
Median time: 18.64ms ± 0.22ms
95th percentile: 30.68ms ± 0.19ms
99th percentile: 32.73ms ± 0.26ms
Max time: 34.74ms ± 0.66ms
Ops: 57.35 ± 0.28 ops/s. Best: 57.87 ops/s
Total: 17440.20ms ± 77.70ms. Best: 17279.90ms
Min time: 1.75ms ± 0.32ms
Mean time: 17.44ms ± 0.08ms
Median time: 16.99ms ± 0.06ms
95th percentile: 28.52ms ± 0.09ms
99th percentile: 30.37ms ± 0.16ms
Max time: 34.52ms ± 2.46ms
json-render String: 6.78794ms
Stringstream: 10.581ms
Vector: 7.00681ms
String: 6.95215ms
Stringstream: 10.7414ms
Vector: 6.96387ms
match_ch Default radius:
4.59307ms/req at 82 coordinate
0.056013ms/coordinate
Radius 10m:
16.0956ms/req at 82 coordinate
0.196288ms/coordinate
Default radius:
4.59509ms/req at 82 coordinate
0.0560377ms/coordinate
Radius 10m:
15.8894ms/req at 82 coordinate
0.193773ms/coordinate
match_mld Default radius:
3.13701ms/req at 82 coordinate
0.0382563ms/coordinate
Radius 10m:
12.0385ms/req at 82 coordinate
0.146811ms/coordinate
Default radius:
3.26805ms/req at 82 coordinate
0.0398543ms/coordinate
Radius 10m:
11.8268ms/req at 82 coordinate
0.144229ms/coordinate
osrm_contract Time: 96.03s Peak RAM: 195.78MB Time: 94.35s Peak RAM: 196.54MB
osrm_customize Time: 1.35s Peak RAM: 116.93MB Time: 1.33s Peak RAM: 116.64MB
osrm_extract Time: 12.17s Peak RAM: 414.25MB Time: 12.05s Peak RAM: 413.26MB
osrm_partition Time: 2.12s Peak RAM: 133.17MB Time: 2.02s Peak RAM: 133.48MB
packedvector random write:
std::vector 9770.43 ms
util::packed_vector 73415.5 ms
slowdown: 7.51405
random read:
std::vector 8431.23 ms
util::packed_vector 30084.1 ms
slowdown: 3.56817
random write:
std::vector 9866.67 ms
util::packed_vector 81699.9 ms
slowdown: 8.28039
random read:
std::vector 8503.38 ms
util::packed_vector 33102 ms
slowdown: 3.8928
random_match_ch 500 matches, default radius
ops: 215.85 ± 1.21 ops/s. best: 213.26ops/s.
total: 264.08 ± 1.49ms. best: 262.11ms.
avg: 4.63 ± 0.03ms
min: 0.16 ± 0.01ms
max: 23.90 ± 0.11ms
p99: 23.90 ± 0.11ms

500 matches, radius=10
ops: 65.25 ± 0.05 ops/s. best: 65.19ops/s.
total: 980.83 ± 0.71ms. best: 979.53ms.
avg: 15.33 ± 0.01ms
min: 0.16 ± 0.00ms
max: 223.36 ± 0.35ms
p99: 223.36 ± 0.35ms

500 matches, radius=20
ops: 15.72 ± 0.04 ops/s. best: 15.65ops/s.
total: 4134.45 ± 9.60ms. best: 4121.13ms.
avg: 63.61 ± 0.15ms
min: 0.35 ± 0.00ms
max: 1111.49 ± 2.65ms
p99: 1111.49 ± 2.65ms
500 matches, default radius
ops: 222.40 ± 1.06 ops/s. best: 223.78ops/s.
total: 256.31 ± 1.23ms. best: 254.72ms.
avg: 4.50 ± 0.02ms
min: 0.15 ± 0.01ms
max: 23.32 ± 0.16ms
p99: 23.32 ± 0.16ms

500 matches, radius=10
ops: 66.72 ± 0.36 ops/s. best: 67.05ops/s.
total: 959.25 ± 5.15ms. best: 954.58ms.
avg: 14.99 ± 0.08ms
min: 0.16 ± 0.00ms
max: 218.94 ± 0.85ms
p99: 218.94 ± 0.85ms

500 matches, radius=20
ops: 16.02 ± 0.03 ops/s. best: 16.07ops/s.
total: 4057.13 ± 6.89ms. best: 4044.67ms.
avg: 62.42 ± 0.11ms
min: 0.34 ± 0.00ms
max: 1089.02 ± 1.72ms
p99: 1089.02 ± 1.72ms
random_match_mld 500 matches, default radius
ops: 301.69 ± 1.50 ops/s. best: 298.09ops/s.
total: 188.94 ± 0.96ms. best: 188.00ms.
avg: 3.31 ± 0.02ms
min: 0.14 ± 0.00ms
max: 19.62 ± 0.03ms
p99: 19.62 ± 0.03ms

500 matches, radius=10
ops: 106.93 ± 0.08 ops/s. best: 106.82ops/s.
total: 598.53 ± 0.42ms. best: 597.86ms.
avg: 9.35 ± 0.01ms
min: 0.15 ± 0.00ms
max: 113.43 ± 0.09ms
p99: 113.43 ± 0.09ms

500 matches, radius=20
ops: 21.37 ± 0.07 ops/s. best: 21.20ops/s.
total: 3042.31 ± 10.40ms. best: 3032.12ms.
avg: 46.80 ± 0.16ms
min: 0.21 ± 0.00ms
max: 596.66 ± 1.60ms
p99: 596.66 ± 1.60ms
500 matches, default radius
ops: 299.80 ± 2.11 ops/s. best: 302.41ops/s.
total: 190.14 ± 1.38ms. best: 188.49ms.
avg: 3.34 ± 0.02ms
min: 0.14 ± 0.00ms
max: 19.73 ± 0.04ms
p99: 19.73 ± 0.04ms

500 matches, radius=10
ops: 105.41 ± 0.12 ops/s. best: 105.66ops/s.
total: 607.15 ± 0.69ms. best: 605.70ms.
avg: 9.49 ± 0.01ms
min: 0.15 ± 0.00ms
max: 115.88 ± 0.36ms
p99: 115.88 ± 0.36ms

500 matches, radius=20
ops: 21.04 ± 0.04 ops/s. best: 21.09ops/s.
total: 3089.47 ± 5.56ms. best: 3081.37ms.
avg: 47.53 ± 0.09ms
min: 0.21 ± 0.00ms
max: 610.30 ± 2.26ms
p99: 610.30 ± 2.26ms
random_nearest_ch 10000 nearest, number_of_results=1
ops: 23920.87 ± 55.29 ops/s. best: 23821.01ops/s.
total: 418.05 ± 1.01ms. best: 416.44ms.
avg: 0.04 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.17 ± 0.03ms
p99: 0.11 ± 0.00ms

10000 nearest, number_of_results=5
ops: 17835.70 ± 71.77 ops/s. best: 17704.04ops/s.
total: 560.68 ± 2.33ms. best: 557.31ms.
avg: 0.06 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.19 ± 0.04ms
p99: 0.12 ± 0.00ms

10000 nearest, number_of_results=10
ops: 14078.88 ± 54.80 ops/s. best: 13974.72ops/s.
total: 710.30 ± 2.82ms. best: 707.50ms.
avg: 0.07 ± 0.00ms
min: 0.03 ± 0.00ms
max: 0.19 ± 0.01ms
p99: 0.14 ± 0.00ms
10000 nearest, number_of_results=1
ops: 24614.14 ± 115.45 ops/s. best: 24737.45ops/s.
total: 406.28 ± 1.91ms. best: 404.25ms.
avg: 0.04 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.15 ± 0.02ms
p99: 0.10 ± 0.00ms

10000 nearest, number_of_results=5
ops: 18449.83 ± 32.37 ops/s. best: 18497.82ops/s.
total: 542.01 ± 0.97ms. best: 540.60ms.
avg: 0.05 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.15 ± 0.00ms
p99: 0.11 ± 0.00ms

10000 nearest, number_of_results=10
ops: 14685.76 ± 6.51 ops/s. best: 14693.08ops/s.
total: 680.93 ± 0.30ms. best: 680.59ms.
avg: 0.07 ± 0.00ms
min: 0.03 ± 0.00ms
max: 0.18 ± 0.00ms
p99: 0.13 ± 0.00ms
random_nearest_mld 10000 nearest, number_of_results=1
ops: 23859.44 ± 25.48 ops/s. best: 23817.80ops/s.
total: 419.12 ± 0.45ms. best: 418.56ms.
avg: 0.04 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.16 ± 0.03ms
p99: 0.11 ± 0.00ms

10000 nearest, number_of_results=5
ops: 17833.27 ± 44.26 ops/s. best: 17759.74ops/s.
total: 560.75 ± 1.47ms. best: 558.07ms.
avg: 0.06 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.16 ± 0.00ms
p99: 0.12 ± 0.00ms

10000 nearest, number_of_results=10
ops: 14076.92 ± 35.10 ops/s. best: 14048.94ops/s.
total: 710.39 ± 1.77ms. best: 706.32ms.
avg: 0.07 ± 0.00ms
min: 0.03 ± 0.00ms
max: 0.19 ± 0.01ms
p99: 0.14 ± 0.00ms
10000 nearest, number_of_results=1
ops: 24842.87 ± 48.75 ops/s. best: 24896.39ops/s.
total: 402.53 ± 0.79ms. best: 401.66ms.
avg: 0.04 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.15 ± 0.02ms
p99: 0.10 ± 0.00ms

10000 nearest, number_of_results=5
ops: 18550.27 ± 15.87 ops/s. best: 18573.68ops/s.
total: 539.08 ± 0.47ms. best: 538.40ms.
avg: 0.05 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.15 ± 0.01ms
p99: 0.11 ± 0.00ms

10000 nearest, number_of_results=10
ops: 14787.94 ± 11.09 ops/s. best: 14806.55ops/s.
total: 676.23 ± 0.51ms. best: 675.38ms.
avg: 0.07 ± 0.00ms
min: 0.03 ± 0.00ms
max: 0.18 ± 0.00ms
p99: 0.13 ± 0.00ms
random_route_ch 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
ops: 447.62 ± 4.96 ops/s. best: 440.99ops/s.
total: 2198.64 ± 24.38ms. best: 2169.69ms.
avg: 2.23 ± 0.02ms
min: 0.34 ± 0.00ms
max: 4.00 ± 0.09ms
p99: 3.34 ± 0.07ms

1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 519.05 ± 3.16 ops/s. best: 513.37ops/s.
total: 1926.68 ± 12.00ms. best: 1908.78ms.
avg: 1.93 ± 0.01ms
min: 0.06 ± 0.00ms
max: 4.56 ± 0.03ms
p99: 4.03 ± 0.06ms

1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 874.00 ± 11.55 ops/s. best: 853.51ops/s.
total: 1126.14 ± 15.12ms. best: 1098.90ms.
avg: 1.14 ± 0.02ms
min: 0.23 ± 0.00ms
max: 1.74 ± 0.04ms
p99: 1.63 ± 0.03ms

1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 987.18 ± 12.76 ops/s. best: 965.22ops/s.
total: 1013.20 ± 13.25ms. best: 993.91ms.
avg: 1.01 ± 0.01ms
min: 0.04 ± 0.00ms
max: 3.10 ± 0.02ms
p99: 2.15 ± 0.03ms
1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
ops: 523.98 ± 2.22 ops/s. best: 527.03ops/s.
total: 1877.98 ± 7.95ms. best: 1867.08ms.
avg: 1.91 ± 0.01ms
min: 0.33 ± 0.00ms
max: 3.41 ± 0.23ms
p99: 2.76 ± 0.03ms

1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 612.28 ± 1.28 ops/s. best: 614.07ops/s.
total: 1633.25 ± 3.49ms. best: 1628.48ms.
avg: 1.63 ± 0.00ms
min: 0.05 ± 0.00ms
max: 4.11 ± 0.26ms
p99: 3.33 ± 0.03ms

1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 1087.95 ± 3.77 ops/s. best: 1095.17ops/s.
total: 904.46 ± 3.13ms. best: 898.49ms.
avg: 0.92 ± 0.00ms
min: 0.21 ± 0.00ms
max: 1.44 ± 0.02ms
p99: 1.28 ± 0.02ms

1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 1206.32 ± 2.09 ops/s. best: 1209.35ops/s.
total: 828.97 ± 1.44ms. best: 826.89ms.
avg: 0.83 ± 0.00ms
min: 0.04 ± 0.00ms
max: 2.77 ± 0.03ms
p99: 1.77 ± 0.01ms
random_route_mld 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
ops: 222.97 ± 1.31 ops/s. best: 221.49ops/s.
total: 4413.28 ± 25.78ms. best: 4371.97ms.
avg: 4.49 ± 0.03ms
min: 0.35 ± 0.01ms
max: 9.62 ± 0.07ms
p99: 7.45 ± 0.08ms

1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 214.04 ± 2.07 ops/s. best: 209.81ops/s.
total: 4672.71 ± 45.41ms. best: 4601.81ms.
avg: 4.67 ± 0.05ms
min: 0.05 ± 0.00ms
max: 12.92 ± 2.02ms
p99: 9.51 ± 0.17ms

1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 297.46 ± 1.14 ops/s. best: 295.80ops/s.
total: 3308.04 ± 12.67ms. best: 3289.60ms.
avg: 3.36 ± 0.01ms
min: 0.30 ± 0.00ms
max: 7.89 ± 0.05ms
p99: 5.69 ± 0.03ms

1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 265.13 ± 1.12 ops/s. best: 263.30ops/s.
total: 3771.77 ± 16.21ms. best: 3749.15ms.
avg: 3.77 ± 0.02ms
min: 0.04 ± 0.00ms
max: 9.15 ± 1.14ms
p99: 7.41 ± 0.05ms
1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
ops: 247.18 ± 3.36 ops/s. best: 250.49ops/s.
total: 3981.80 ± 55.48ms. best: 3928.38ms.
avg: 4.05 ± 0.06ms
min: 0.32 ± 0.00ms
max: 8.69 ± 0.10ms
p99: 6.81 ± 0.08ms

1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 244.04 ± 1.75 ops/s. best: 246.40ops/s.
total: 4097.93 ± 29.55ms. best: 4058.52ms.
avg: 4.10 ± 0.03ms
min: 0.05 ± 0.00ms
max: 9.75 ± 0.72ms
p99: 8.12 ± 0.12ms

1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 335.61 ± 4.61 ops/s. best: 340.01ops/s.
total: 2932.68 ± 40.72ms. best: 2894.03ms.
avg: 2.98 ± 0.04ms
min: 0.29 ± 0.00ms
max: 7.09 ± 0.17ms
p99: 5.11 ± 0.10ms

1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 305.71 ± 2.23 ops/s. best: 308.97ops/s.
total: 3271.34 ± 23.95ms. best: 3236.57ms.
avg: 3.27 ± 0.02ms
min: 0.04 ± 0.00ms
max: 7.73 ± 0.79ms
p99: 6.39 ± 0.09ms
random_table_ch 250 tables, 3 coordinates
ops: 1510.62 ± 16.88 ops/s. best: 1472.49ops/s.
total: 165.52 ± 1.87ms. best: 163.53ms.
avg: 0.66 ± 0.01ms
min: 0.44 ± 0.00ms
max: 1.01 ± 0.24ms
p99: 0.86 ± 0.03ms

250 tables, 25 coordinates
ops: 176.52 ± 1.84 ops/s. best: 173.62ops/s.
total: 1416.48 ± 15.04ms. best: 1399.33ms.
avg: 5.67 ± 0.06ms
min: 5.03 ± 0.04ms
max: 6.27 ± 0.12ms
p99: 6.18 ± 0.08ms

250 tables, 50 coordinates
ops: 85.82 ± 0.19 ops/s. best: 85.55ops/s.
total: 2913.03 ± 6.48ms. best: 2902.48ms.
avg: 11.65 ± 0.03ms
min: 10.72 ± 0.05ms
max: 13.76 ± 1.48ms
p99: 13.26 ± 1.16ms
250 tables, 3 coordinates
ops: 1564.93 ± 12.93 ops/s. best: 1574.62ops/s.
total: 159.77 ± 1.34ms. best: 158.77ms.
avg: 0.64 ± 0.01ms
min: 0.42 ± 0.00ms
max: 0.96 ± 0.23ms
p99: 0.81 ± 0.03ms

250 tables, 25 coordinates
ops: 183.71 ± 0.08 ops/s. best: 183.83ops/s.
total: 1360.82 ± 0.57ms. best: 1359.97ms.
avg: 5.44 ± 0.00ms
min: 4.94 ± 0.01ms
max: 6.14 ± 0.13ms
p99: 5.88 ± 0.02ms

250 tables, 50 coordinates
ops: 90.25 ± 0.11 ops/s. best: 90.34ops/s.
total: 2770.20 ± 3.50ms. best: 2767.25ms.
avg: 11.08 ± 0.01ms
min: 10.31 ± 0.01ms
max: 12.12 ± 0.15ms
p99: 11.89 ± 0.06ms
random_table_mld 250 tables, 3 coordinates
ops: 321.72 ± 3.16 ops/s. best: 315.44ops/s.
total: 777.17 ± 7.73ms. best: 766.78ms.
avg: 3.11 ± 0.03ms
min: 2.37 ± 0.01ms
max: 4.27 ± 0.14ms
p99: 4.06 ± 0.09ms

250 tables, 25 coordinates
ops: 35.32 ± 0.21 ops/s. best: 35.12ops/s.
total: 7079.17 ± 41.01ms. best: 7007.33ms.
avg: 28.32 ± 0.16ms
min: 24.94 ± 0.21ms
max: 32.68 ± 1.10ms
p99: 31.74 ± 0.58ms

250 tables, 50 coordinates
ops: 16.47 ± 0.03 ops/s. best: 16.42ops/s.
total: 15180.47 ± 29.17ms. best: 15130.44ms.
avg: 60.72 ± 0.12ms
min: 55.14 ± 0.37ms
max: 69.05 ± 2.66ms
p99: 66.47 ± 0.86ms
250 tables, 3 coordinates
ops: 347.08 ± 0.77 ops/s. best: 347.91ops/s.
total: 720.30 ± 1.59ms. best: 718.57ms.
avg: 2.88 ± 0.01ms
min: 2.29 ± 0.01ms
max: 4.19 ± 0.33ms
p99: 3.81 ± 0.03ms

250 tables, 25 coordinates
ops: 38.36 ± 0.04 ops/s. best: 38.40ops/s.
total: 6517.15 ± 7.42ms. best: 6510.64ms.
avg: 26.07 ± 0.03ms
min: 23.37 ± 0.06ms
max: 29.75 ± 0.31ms
p99: 28.61 ± 0.10ms

250 tables, 50 coordinates
ops: 17.78 ± 0.05 ops/s. best: 17.82ops/s.
total: 14063.94 ± 38.77ms. best: 14029.38ms.
avg: 56.26 ± 0.16ms
min: 52.09 ± 0.12ms
max: 63.20 ± 4.86ms
p99: 60.20 ± 0.54ms
random_trip_ch 250 trips, 3 coordinates
ops: 435.04 ± 2.80 ops/s. best: 430.28ops/s.
total: 574.69 ± 3.81ms. best: 569.65ms.
avg: 2.30 ± 0.02ms
min: 1.18 ± 0.01ms
max: 3.39 ± 0.26ms
p99: 3.12 ± 0.02ms

250 trips, 5 coordinates
ops: 289.22 ± 1.90 ops/s. best: 286.95ops/s.
total: 864.44 ± 5.54ms. best: 853.23ms.
avg: 3.46 ± 0.02ms
min: 2.17 ± 0.03ms
max: 4.74 ± 0.36ms
p99: 4.35 ± 0.05ms
250 trips, 3 coordinates
ops: 500.25 ± 5.85 ops/s. best: 507.18ops/s.
total: 499.84 ± 5.89ms. best: 492.92ms.
avg: 2.00 ± 0.02ms
min: 1.08 ± 0.02ms
max: 2.96 ± 0.32ms
p99: 2.64 ± 0.08ms

250 trips, 5 coordinates
ops: 339.13 ± 1.09 ops/s. best: 341.17ops/s.
total: 737.19 ± 2.42ms. best: 732.77ms.
avg: 2.95 ± 0.01ms
min: 1.99 ± 0.01ms
max: 3.74 ± 0.06ms
p99: 3.61 ± 0.04ms
random_trip_mld 250 trips, 3 coordinates
ops: 152.29 ± 1.03 ops/s. best: 150.73ops/s.
total: 1641.71 ± 10.83ms. best: 1623.80ms.
avg: 6.57 ± 0.04ms
min: 4.21 ± 0.04ms
max: 9.01 ± 0.19ms
p99: 8.56 ± 0.12ms

250 trips, 5 coordinates
ops: 98.29 ± 0.61 ops/s. best: 97.50ops/s.
total: 2543.52 ± 15.29ms. best: 2519.92ms.
avg: 10.17 ± 0.06ms
min: 6.81 ± 0.12ms
max: 12.72 ± 0.38ms
p99: 12.34 ± 0.16ms
250 trips, 3 coordinates
ops: 172.17 ± 1.42 ops/s. best: 174.71ops/s.
total: 1452.16 ± 11.95ms. best: 1430.97ms.
avg: 5.81 ± 0.05ms
min: 3.82 ± 0.01ms
max: 7.93 ± 0.30ms
p99: 7.54 ± 0.08ms

250 trips, 5 coordinates
ops: 112.07 ± 1.43 ops/s. best: 114.18ops/s.
total: 2231.20 ± 28.41ms. best: 2189.54ms.
avg: 8.92 ± 0.11ms
min: 6.26 ± 0.04ms
max: 11.84 ± 1.16ms
p99: 11.28 ± 0.80ms
route_ch 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
437.546ms
0.437546ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
542.085ms
0.542085ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
156.829ms
0.156829ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
144.126ms
0.144126ms/req
1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
436.38ms
0.43638ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
525.673ms
0.525673ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
156.856ms
0.156856ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
141.093ms
0.141093ms/req
route_mld 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
582.321ms
0.582321ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
736.26ms
0.73626ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
280.23ms
0.28023ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
316.813ms
0.316813ms/req
1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
574.426ms
0.574426ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
727.031ms
0.727031ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
275.455ms
0.275455ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
305.712ms
0.305712ms/req
rtree 1 result:
202.082ms -> 0.0202082 ms/query
10 results:
237.188ms -> 0.0237188 ms/query
1 result:
201.641ms -> 0.0201641 ms/query
10 results:
237.35ms -> 0.023735 ms/query

@SiarheiFedartsou SiarheiFedartsou marked this pull request as ready for review June 26, 2024 18:01
@SiarheiFedartsou
Copy link
Member Author

I would really like to test it together with #6975, so I am taking responsibility and merging it without review 🙏🏻

@SiarheiFedartsou SiarheiFedartsou merged commit cacb1b2 into master Jun 29, 2024
22 checks passed
@SiarheiFedartsou SiarheiFedartsou deleted the sf-best-ops branch June 29, 2024 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant