Skip to content

Conversation

@SiarheiFedartsou
Copy link
Member

@SiarheiFedartsou SiarheiFedartsou commented Jun 16, 2024

Yet another microoptimisation...

Benchmark Results

Benchmark Base PR
alias aliased u32: 1095.14
plain u32: 1091.51
aliased double: 959.93
plain double: 950.851
aliased u32: 1146.41
plain u32: 1141.2
aliased double: 1178.67
plain double: 1167.56
e2e_match_ch Ops: 45.88 ± 0.12 ops/s. Best: 45.71 ops/s
Total: 2855.49ms ± 7.14ms. Best: 2841.20ms
Min time: 2.25ms ± 0.04ms
Mean time: 21.80ms ± 0.06ms
Median time: 16.13ms ± 0.10ms
95th percentile: 69.88ms ± 0.69ms
99th percentile: 84.43ms ± 0.33ms
Max time: 98.93ms ± 0.89ms
Ops: 44.53 ± 0.26 ops/s. Best: 44.10 ops/s
Total: 2941.60ms ± 18.14ms. Best: 2914.01ms
Min time: 2.21ms ± 0.03ms
Mean time: 22.46ms ± 0.14ms
Median time: 16.95ms ± 0.28ms
95th percentile: 70.79ms ± 0.50ms
99th percentile: 86.42ms ± 1.41ms
Max time: 99.67ms ± 0.36ms
e2e_match_mld Ops: 61.95 ± 0.13 ops/s. Best: 61.79 ops/s
Total: 2114.46ms ± 4.68ms. Best: 2104.36ms
Min time: 1.82ms ± 0.03ms
Mean time: 16.14ms ± 0.04ms
Median time: 8.57ms ± 0.07ms
95th percentile: 53.51ms ± 0.07ms
99th percentile: 61.94ms ± 0.31ms
Max time: 72.15ms ± 0.41ms
Ops: 61.11 ± 1.22 ops/s. Best: 58.25 ops/s
Total: 2146.06ms ± 43.88ms. Best: 2098.63ms
Min time: 1.87ms ± 0.03ms
Mean time: 16.38ms ± 0.34ms
Median time: 9.09ms ± 0.58ms
95th percentile: 54.13ms ± 1.16ms
99th percentile: 62.64ms ± 1.72ms
Max time: 71.32ms ± 0.26ms
e2e_nearest_ch Ops: 805.18 ± 4.40 ops/s. Best: 795.56 ops/s
Total: 1241.84ms ± 7.19ms. Best: 1234.70ms
Min time: 1.06ms ± 0.01ms
Mean time: 1.24ms ± 0.01ms
Median time: 1.15ms ± 0.01ms
95th percentile: 1.64ms ± 0.01ms
99th percentile: 1.69ms ± 0.01ms
Max time: 4.56ms ± 2.87ms
Ops: 678.00 ± 20.37 ops/s. Best: 641.85 ops/s
Total: 1478.15ms ± 44.71ms. Best: 1414.58ms
Min time: 1.11ms ± 0.02ms
Mean time: 1.48ms ± 0.05ms
Median time: 1.30ms ± 0.03ms
95th percentile: 2.09ms ± 0.14ms
99th percentile: 3.48ms ± 0.78ms
Max time: 8.92ms ± 1.71ms
e2e_nearest_mld Ops: 806.06 ± 3.85 ops/s. Best: 797.64 ops/s
Total: 1240.54ms ± 6.50ms. Best: 1231.79ms
Min time: 1.06ms ± 0.01ms
Mean time: 1.24ms ± 0.01ms
Median time: 1.15ms ± 0.01ms
95th percentile: 1.64ms ± 0.01ms
99th percentile: 1.70ms ± 0.01ms
Max time: 4.43ms ± 2.73ms
Ops: 790.60 ± 5.62 ops/s. Best: 777.35 ops/s
Total: 1264.77ms ± 9.60ms. Best: 1255.51ms
Min time: 1.07ms ± 0.01ms
Mean time: 1.27ms ± 0.01ms
Median time: 1.17ms ± 0.01ms
95th percentile: 1.67ms ± 0.01ms
99th percentile: 1.74ms ± 0.01ms
Max time: 4.29ms ± 2.32ms
e2e_route_ch Ops: 335.79 ± 3.02 ops/s. Best: 329.50 ops/s
Total: 2978.48ms ± 29.16ms. Best: 2940.99ms
Min time: 1.31ms ± 0.02ms
Mean time: 2.98ms ± 0.03ms
Median time: 3.00ms ± 0.03ms
95th percentile: 3.93ms ± 0.01ms
99th percentile: 4.37ms ± 0.05ms
Max time: 6.88ms ± 1.99ms
Ops: 259.22 ± 40.06 ops/s. Best: 211.50 ops/s
Total: 3998.54ms ± 583.58ms. Best: 3007.95ms
Min time: 1.42ms ± 0.12ms
Mean time: 3.98ms ± 0.58ms
Median time: 3.84ms ± 0.48ms
95th percentile: 6.35ms ± 1.37ms
99th percentile: 8.10ms ± 1.65ms
Max time: 9.99ms ± 2.25ms
e2e_route_mld Ops: 276.98 ± 1.45 ops/s. Best: 275.11 ops/s
Total: 3610.23ms ± 18.43ms. Best: 3570.98ms
Min time: 1.29ms ± 0.01ms
Mean time: 3.61ms ± 0.02ms
Median time: 3.66ms ± 0.02ms
95th percentile: 4.96ms ± 0.04ms
99th percentile: 5.39ms ± 0.09ms
Max time: 7.64ms ± 1.71ms
Ops: 279.91 ± 2.22 ops/s. Best: 276.73 ops/s
Total: 3572.49ms ± 29.03ms. Best: 3526.36ms
Min time: 1.30ms ± 0.02ms
Mean time: 3.57ms ± 0.03ms
Median time: 3.62ms ± 0.03ms
95th percentile: 4.89ms ± 0.07ms
99th percentile: 5.44ms ± 0.07ms
Max time: 7.96ms ± 1.65ms
e2e_table_ch Ops: 298.31 ± 0.93 ops/s. Best: 296.38 ops/s
Total: 3352.45ms ± 10.47ms. Best: 3336.47ms
Min time: 1.68ms ± 0.01ms
Mean time: 3.35ms ± 0.01ms
Median time: 3.36ms ± 0.03ms
95th percentile: 4.64ms ± 0.01ms
99th percentile: 5.02ms ± 0.05ms
Max time: 8.04ms ± 2.92ms
Ops: 291.07 ± 5.85 ops/s. Best: 278.50 ops/s
Total: 3439.94ms ± 72.20ms. Best: 3344.44ms
Min time: 1.76ms ± 0.03ms
Mean time: 3.44ms ± 0.07ms
Median time: 3.45ms ± 0.08ms
95th percentile: 4.79ms ± 0.12ms
99th percentile: 5.41ms ± 0.40ms
Max time: 9.30ms ± 2.22ms
e2e_table_mld Ops: 108.43 ± 0.46 ops/s. Best: 107.78 ops/s
Total: 9223.87ms ± 37.01ms. Best: 9146.23ms
Min time: 3.74ms ± 0.04ms
Mean time: 9.22ms ± 0.04ms
Median time: 9.18ms ± 0.05ms
95th percentile: 14.11ms ± 0.06ms
99th percentile: 15.09ms ± 0.08ms
Max time: 17.81ms ± 1.71ms
Ops: 103.96 ± 3.64 ops/s. Best: 95.16 ops/s
Total: 9633.65ms ± 361.92ms. Best: 9276.51ms
Min time: 3.79ms ± 0.07ms
Mean time: 9.64ms ± 0.36ms
Median time: 9.59ms ± 0.34ms
95th percentile: 14.73ms ± 0.52ms
99th percentile: 16.22ms ± 1.02ms
Max time: 20.37ms ± 1.58ms
e2e_trip_ch Ops: 94.83 ± 0.89 ops/s. Best: 92.84 ops/s
Total: 10545.80ms ± 101.30ms. Best: 10416.67ms
Min time: 1.60ms ± 0.22ms
Mean time: 10.55ms ± 0.11ms
Median time: 10.06ms ± 0.07ms
95th percentile: 18.56ms ± 0.12ms
99th percentile: 20.32ms ± 0.10ms
Max time: 21.78ms ± 0.31ms
Ops: 79.86 ± 2.13 ops/s. Best: 76.70 ops/s
Total: 12522.78ms ± 345.44ms. Best: 11917.55ms
Min time: 2.07ms ± 0.21ms
Mean time: 12.54ms ± 0.37ms
Median time: 12.01ms ± 0.35ms
95th percentile: 21.02ms ± 0.54ms
99th percentile: 23.68ms ± 0.92ms
Max time: 28.72ms ± 0.89ms
e2e_trip_mld Ops: 56.63 ± 0.30 ops/s. Best: 56.30 ops/s
Total: 17658.21ms ± 98.86ms. Best: 17450.61ms
Min time: 1.66ms ± 0.19ms
Mean time: 17.66ms ± 0.10ms
Median time: 17.24ms ± 0.17ms
95th percentile: 28.83ms ± 0.16ms
99th percentile: 31.02ms ± 0.12ms
Max time: 33.80ms ± 1.15ms
Ops: 51.72 ± 0.88 ops/s. Best: 50.67 ops/s
Total: 19339.69ms ± 332.23ms. Best: 18753.97ms
Min time: 2.08ms ± 0.17ms
Mean time: 19.34ms ± 0.33ms
Median time: 19.01ms ± 0.32ms
95th percentile: 30.96ms ± 0.44ms
99th percentile: 33.45ms ± 0.50ms
Max time: 37.86ms ± 1.41ms
json-render String: 6.95317ms
Stringstream: 10.6577ms
Vector: 7.10955ms
String: 6.65288ms
Stringstream: 10.4238ms
Vector: 6.86823ms
match_ch Default radius:
4.64066ms/req at 82 coordinate
0.0565935ms/coordinate
Radius 10m:
16.1932ms/req at 82 coordinate
0.197478ms/coordinate
Default radius:
4.58197ms/req at 82 coordinate
0.0558777ms/coordinate
Radius 10m:
16.0511ms/req at 82 coordinate
0.195745ms/coordinate
match_mld Default radius:
3.27737ms/req at 82 coordinate
0.039968ms/coordinate
Radius 10m:
11.8283ms/req at 82 coordinate
0.144248ms/coordinate
Default radius:
3.38007ms/req at 82 coordinate
0.0412204ms/coordinate
Radius 10m:
11.5173ms/req at 82 coordinate
0.140454ms/coordinate
osrm_contract Time: 96.07s Peak RAM: 197.42MB Time: 97.21s Peak RAM: 196.00MB
osrm_customize Time: 1.36s Peak RAM: 116.86MB Time: 1.39s Peak RAM: 116.78MB
osrm_extract Time: 12.02s Peak RAM: 409.83MB Time: 12.22s Peak RAM: 406.36MB
osrm_partition Time: 1.99s Peak RAM: 132.72MB Time: 2.05s Peak RAM: 135.08MB
packedvector random write:
std::vector 9760.11 ms
util::packed_vector 82326 ms
slowdown: 8.43494
random read:
std::vector 8427.84 ms
util::packed_vector 33191.2 ms
slowdown: 3.93828
random write:
std::vector 11214.2 ms
util::packed_vector 81511.3 ms
slowdown: 7.26857
random read:
std::vector 11079.3 ms
util::packed_vector 33418.5 ms
slowdown: 3.0163
random_match_ch 500 matches, default radius
ops: 213.52 ± 0.82 ops/s. best: 211.51ops/s.
total: 266.96 ± 1.03ms. best: 266.14ms.
avg: 4.68 ± 0.02ms
min: 0.15 ± 0.01ms
max: 24.30 ± 0.09ms
p99: 24.30 ± 0.09ms

500 matches, radius=10
ops: 64.15 ± 0.09 ops/s. best: 64.00ops/s.
total: 997.70 ± 1.41ms. best: 995.66ms.
avg: 15.59 ± 0.02ms
min: 0.15 ± 0.00ms
max: 225.30 ± 0.62ms
p99: 225.30 ± 0.62ms

500 matches, radius=20
ops: 14.99 ± 0.02 ops/s. best: 14.95ops/s.
total: 4337.50 ± 6.91ms. best: 4327.14ms.
avg: 66.73 ± 0.11ms
min: 0.30 ± 0.00ms
max: 1165.05 ± 5.90ms
p99: 1165.05 ± 5.90ms
500 matches, default radius
ops: 225.98 ± 0.95 ops/s. best: 224.10ops/s.
total: 252.24 ± 1.07ms. best: 250.74ms.
avg: 4.43 ± 0.02ms
min: 0.15 ± 0.01ms
max: 24.01 ± 0.06ms
p99: 24.01 ± 0.06ms

500 matches, radius=10
ops: 67.12 ± 0.16 ops/s. best: 66.87ops/s.
total: 953.47 ± 2.34ms. best: 950.36ms.
avg: 14.90 ± 0.04ms
min: 0.16 ± 0.00ms
max: 225.89 ± 0.49ms
p99: 225.89 ± 0.49ms

500 matches, radius=20
ops: 16.05 ± 0.03 ops/s. best: 15.99ops/s.
total: 4051.00 ± 8.93ms. best: 4036.04ms.
avg: 62.32 ± 0.14ms
min: 0.32 ± 0.01ms
max: 1150.28 ± 5.79ms
p99: 1150.28 ± 5.79ms
random_match_mld 500 matches, default radius
ops: 302.00 ± 1.34 ops/s. best: 298.81ops/s.
total: 188.75 ± 0.85ms. best: 187.93ms.
avg: 3.31 ± 0.01ms
min: 0.13 ± 0.00ms
max: 19.62 ± 0.05ms
p99: 19.62 ± 0.05ms

500 matches, radius=10
ops: 106.59 ± 0.14 ops/s. best: 106.35ops/s.
total: 600.41 ± 0.79ms. best: 599.29ms.
avg: 9.38 ± 0.01ms
min: 0.14 ± 0.00ms
max: 114.86 ± 0.25ms
p99: 114.86 ± 0.25ms

500 matches, radius=20
ops: 21.23 ± 0.01 ops/s. best: 21.21ops/s.
total: 3061.99 ± 1.51ms. best: 3059.28ms.
avg: 47.11 ± 0.02ms
min: 0.19 ± 0.00ms
max: 607.73 ± 1.62ms
p99: 607.73 ± 1.62ms
500 matches, default radius
ops: 303.67 ± 1.16 ops/s. best: 300.89ops/s.
total: 187.71 ± 0.72ms. best: 187.11ms.
avg: 3.29 ± 0.01ms
min: 0.13 ± 0.00ms
max: 19.56 ± 0.07ms
p99: 19.56 ± 0.07ms

500 matches, radius=10
ops: 107.23 ± 0.42 ops/s. best: 106.29ops/s.
total: 596.83 ± 2.35ms. best: 594.29ms.
avg: 9.33 ± 0.04ms
min: 0.15 ± 0.00ms
max: 113.75 ± 0.29ms
p99: 113.75 ± 0.29ms

500 matches, radius=20
ops: 21.56 ± 0.04 ops/s. best: 21.50ops/s.
total: 3015.21 ± 5.18ms. best: 3007.66ms.
avg: 46.39 ± 0.08ms
min: 0.21 ± 0.01ms
max: 594.66 ± 2.00ms
p99: 594.66 ± 2.00ms
random_nearest_ch 10000 nearest, number_of_results=1
ops: 22089.07 ± 43.16 ops/s. best: 21984.76ops/s.
total: 452.72 ± 0.89ms. best: 451.92ms.
avg: 0.05 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.19 ± 0.01ms
p99: 0.12 ± 0.00ms

10000 nearest, number_of_results=5
ops: 16900.08 ± 9.96 ops/s. best: 16878.76ops/s.
total: 591.71 ± 0.35ms. best: 591.36ms.
avg: 0.06 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.17 ± 0.01ms
p99: 0.13 ± 0.00ms

10000 nearest, number_of_results=10
ops: 13443.38 ± 40.19 ops/s. best: 13352.61ops/s.
total: 743.87 ± 2.23ms. best: 741.91ms.
avg: 0.07 ± 0.00ms
min: 0.03 ± 0.00ms
max: 0.18 ± 0.00ms
p99: 0.14 ± 0.00ms
10000 nearest, number_of_results=1
ops: 22059.58 ± 55.32 ops/s. best: 21942.26ops/s.
total: 453.32 ± 1.14ms. best: 452.31ms.
avg: 0.05 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.20 ± 0.01ms
p99: 0.12 ± 0.00ms

10000 nearest, number_of_results=5
ops: 17163.69 ± 73.64 ops/s. best: 16984.52ops/s.
total: 582.64 ± 2.54ms. best: 580.26ms.
avg: 0.06 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.18 ± 0.02ms
p99: 0.13 ± 0.00ms

10000 nearest, number_of_results=10
ops: 13804.75 ± 31.67 ops/s. best: 13736.55ops/s.
total: 724.39 ± 1.67ms. best: 722.43ms.
avg: 0.07 ± 0.00ms
min: 0.03 ± 0.00ms
max: 0.18 ± 0.01ms
p99: 0.14 ± 0.00ms
random_nearest_mld 10000 nearest, number_of_results=1
ops: 22133.62 ± 36.77 ops/s. best: 22040.32ops/s.
total: 451.80 ± 0.76ms. best: 451.17ms.
avg: 0.05 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.19 ± 0.01ms
p99: 0.12 ± 0.00ms

10000 nearest, number_of_results=5
ops: 16905.30 ± 5.73 ops/s. best: 16892.94ops/s.
total: 591.53 ± 0.20ms. best: 591.26ms.
avg: 0.06 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.18 ± 0.01ms
p99: 0.13 ± 0.00ms

10000 nearest, number_of_results=10
ops: 13447.47 ± 2.62 ops/s. best: 13444.89ops/s.
total: 743.63 ± 0.14ms. best: 743.30ms.
avg: 0.07 ± 0.00ms
min: 0.03 ± 0.00ms
max: 0.18 ± 0.00ms
p99: 0.14 ± 0.00ms
10000 nearest, number_of_results=1
ops: 22004.60 ± 32.60 ops/s. best: 21964.16ops/s.
total: 454.45 ± 0.67ms. best: 453.04ms.
avg: 0.05 ± 0.00ms
min: 0.01 ± 0.00ms
max: 0.20 ± 0.01ms
p99: 0.12 ± 0.00ms

10000 nearest, number_of_results=5
ops: 17233.14 ± 43.08 ops/s. best: 17131.08ops/s.
total: 580.28 ± 1.46ms. best: 579.06ms.
avg: 0.06 ± 0.00ms
min: 0.02 ± 0.00ms
max: 0.17 ± 0.01ms
p99: 0.13 ± 0.00ms

10000 nearest, number_of_results=10
ops: 13920.29 ± 14.55 ops/s. best: 13901.70ops/s.
total: 718.38 ± 0.75ms. best: 716.75ms.
avg: 0.07 ± 0.00ms
min: 0.03 ± 0.00ms
max: 0.18 ± 0.00ms
p99: 0.14 ± 0.00ms
random_route_ch 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
ops: 491.44 ± 1.57 ops/s. best: 487.45ops/s.
total: 2002.32 ± 6.44ms. best: 1996.89ms.
avg: 2.03 ± 0.01ms
min: 0.34 ± 0.00ms
max: 3.66 ± 0.25ms
p99: 2.91 ± 0.05ms

1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 579.37 ± 0.45 ops/s. best: 578.50ops/s.
total: 1726.02 ± 1.34ms. best: 1723.93ms.
avg: 1.73 ± 0.00ms
min: 0.06 ± 0.00ms
max: 4.46 ± 0.02ms
p99: 3.91 ± 0.04ms

1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 964.09 ± 2.63 ops/s. best: 958.14ops/s.
total: 1020.66 ± 2.80ms. best: 1018.37ms.
avg: 1.04 ± 0.00ms
min: 0.25 ± 0.00ms
max: 1.92 ± 0.34ms
p99: 1.44 ± 0.04ms

1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 1077.22 ± 20.53 ops/s. best: 1025.28ops/s.
total: 928.87 ± 18.32ms. best: 915.67ms.
avg: 0.93 ± 0.02ms
min: 0.05 ± 0.00ms
max: 3.26 ± 0.23ms
p99: 2.41 ± 0.10ms
1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
ops: 454.27 ± 14.85 ops/s. best: 432.00ops/s.
total: 2168.72 ± 68.04ms. best: 2056.84ms.
avg: 2.20 ± 0.07ms
min: 0.36 ± 0.01ms
max: 4.31 ± 0.18ms
p99: 3.32 ± 0.05ms

1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 503.94 ± 14.34 ops/s. best: 487.83ops/s.
total: 1986.49 ± 54.79ms. best: 1867.26ms.
avg: 1.99 ± 0.05ms
min: 0.06 ± 0.00ms
max: 5.21 ± 0.07ms
p99: 4.32 ± 0.17ms

1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 976.32 ± 5.65 ops/s. best: 967.48ops/s.
total: 1007.91 ± 5.83ms. best: 997.64ms.
avg: 1.02 ± 0.01ms
min: 0.26 ± 0.01ms
max: 1.62 ± 0.02ms
p99: 1.44 ± 0.01ms

1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 1029.01 ± 8.88 ops/s. best: 1009.71ops/s.
total: 971.90 ± 8.46ms. best: 963.10ms.
avg: 0.97 ± 0.01ms
min: 0.05 ± 0.00ms
max: 2.75 ± 0.22ms
p99: 2.17 ± 0.01ms
random_route_mld 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
ops: 242.30 ± 7.18 ops/s. best: 230.09ops/s.
total: 4065.69 ± 120.50ms. best: 3913.48ms.
avg: 4.13 ± 0.12ms
min: 0.34 ± 0.00ms
max: 9.10 ± 0.46ms
p99: 6.94 ± 0.33ms

1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 248.42 ± 0.24 ops/s. best: 248.00ops/s.
total: 4025.51 ± 3.83ms. best: 4020.02ms.
avg: 4.03 ± 0.00ms
min: 0.06 ± 0.00ms
max: 9.25 ± 0.15ms
p99: 8.02 ± 0.03ms

1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 336.51 ± 0.34 ops/s. best: 336.16ops/s.
total: 2924.10 ± 2.96ms. best: 2918.84ms.
avg: 2.97 ± 0.00ms
min: 0.30 ± 0.00ms
max: 7.20 ± 0.04ms
p99: 5.05 ± 0.01ms

1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 312.84 ± 0.16 ops/s. best: 312.61ops/s.
total: 3196.53 ± 1.68ms. best: 3194.44ms.
avg: 3.20 ± 0.00ms
min: 0.04 ± 0.00ms
max: 7.18 ± 0.04ms
p99: 6.20 ± 0.01ms
1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
ops: 229.51 ± 10.15 ops/s. best: 217.83ops/s.
total: 4297.41 ± 183.23ms. best: 4006.95ms.
avg: 4.37 ± 0.19ms
min: 0.35 ± 0.02ms
max: 10.67 ± 1.56ms
p99: 7.40 ± 0.39ms

1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
ops: 205.99 ± 5.63 ops/s. best: 196.67ops/s.
total: 4859.13 ± 133.88ms. best: 4664.14ms.
avg: 4.86 ± 0.13ms
min: 0.06 ± 0.00ms
max: 13.51 ± 2.40ms
p99: 10.35 ± 0.88ms

1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
ops: 292.47 ± 5.42 ops/s. best: 286.31ops/s.
total: 3365.96 ± 60.92ms. best: 3243.84ms.
avg: 3.42 ± 0.06ms
min: 0.30 ± 0.01ms
max: 8.00 ± 0.16ms
p99: 5.90 ± 0.08ms

1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
ops: 270.90 ± 4.70 ops/s. best: 262.58ops/s.
total: 3693.03 ± 63.72ms. best: 3580.40ms.
avg: 3.69 ± 0.06ms
min: 0.05 ± 0.01ms
max: 8.88 ± 0.38ms
p99: 7.46 ± 0.15ms
random_table_ch 250 tables, 3 coordinates
ops: 1387.98 ± 8.71 ops/s. best: 1367.09ops/s.
total: 180.13 ± 1.15ms. best: 179.22ms.
avg: 0.72 ± 0.00ms
min: 0.48 ± 0.00ms
max: 1.06 ± 0.27ms
p99: 0.88 ± 0.04ms

250 tables, 25 coordinates
ops: 164.36 ± 0.12 ops/s. best: 164.21ops/s.
total: 1521.01 ± 1.12ms. best: 1518.94ms.
avg: 6.08 ± 0.00ms
min: 5.46 ± 0.01ms
max: 6.71 ± 0.03ms
p99: 6.58 ± 0.01ms

250 tables, 50 coordinates
ops: 81.04 ± 0.05 ops/s. best: 80.93ops/s.
total: 3085.08 ± 1.94ms. best: 3083.15ms.
avg: 12.34 ± 0.01ms
min: 11.42 ± 0.03ms
max: 13.53 ± 0.21ms
p99: 13.05 ± 0.08ms
250 tables, 3 coordinates
ops: 1448.42 ± 10.44 ops/s. best: 1421.67ops/s.
total: 172.62 ± 1.26ms. best: 171.70ms.
avg: 0.69 ± 0.01ms
min: 0.50 ± 0.00ms
max: 1.01 ± 0.24ms
p99: 0.86 ± 0.04ms

250 tables, 25 coordinates
ops: 171.51 ± 0.86 ops/s. best: 170.22ops/s.
total: 1457.67 ± 7.30ms. best: 1447.41ms.
avg: 5.83 ± 0.03ms
min: 5.17 ± 0.02ms
max: 6.39 ± 0.10ms
p99: 6.28 ± 0.03ms

250 tables, 50 coordinates
ops: 83.86 ± 1.13 ops/s. best: 80.94ops/s.
total: 2982.14 ± 41.11ms. best: 2953.00ms.
avg: 11.93 ± 0.16ms
min: 11.02 ± 0.05ms
max: 13.46 ± 1.07ms
p99: 13.27 ± 0.90ms
random_table_mld 250 tables, 3 coordinates
ops: 345.18 ± 0.95 ops/s. best: 342.90ops/s.
total: 724.27 ± 2.01ms. best: 722.54ms.
avg: 2.90 ± 0.01ms
min: 2.31 ± 0.01ms
max: 4.00 ± 0.07ms
p99: 3.83 ± 0.04ms

250 tables, 25 coordinates
ops: 38.47 ± 0.02 ops/s. best: 38.44ops/s.
total: 6497.82 ± 3.52ms. best: 6491.53ms.
avg: 25.99 ± 0.01ms
min: 23.55 ± 0.04ms
max: 29.37 ± 0.03ms
p99: 28.58 ± 0.22ms

250 tables, 50 coordinates
ops: 17.89 ± 0.03 ops/s. best: 17.82ops/s.
total: 13977.13 ± 23.05ms. best: 13951.78ms.
avg: 55.91 ± 0.09ms
min: 52.20 ± 0.24ms
max: 60.22 ± 0.60ms
p99: 59.51 ± 0.27ms
250 tables, 3 coordinates
ops: 342.12 ± 1.86 ops/s. best: 337.61ops/s.
total: 730.77 ± 4.01ms. best: 727.24ms.
avg: 2.92 ± 0.02ms
min: 2.32 ± 0.01ms
max: 4.04 ± 0.07ms
p99: 3.88 ± 0.05ms

250 tables, 25 coordinates
ops: 38.06 ± 0.15 ops/s. best: 37.74ops/s.
total: 6568.58 ± 25.70ms. best: 6534.64ms.
avg: 26.27 ± 0.10ms
min: 23.64 ± 0.09ms
max: 32.95 ± 4.89ms
p99: 31.07 ± 3.28ms

250 tables, 50 coordinates
ops: 17.32 ± 0.33 ops/s. best: 16.56ops/s.
total: 14439.13 ± 279.36ms. best: 14148.00ms.
avg: 57.76 ± 1.12ms
min: 52.97 ± 0.14ms
max: 103.87 ± 62.89ms
p99: 94.10 ± 49.38ms
random_trip_ch 250 trips, 3 coordinates
ops: 464.91 ± 2.94 ops/s. best: 457.94ops/s.
total: 537.77 ± 3.43ms. best: 534.65ms.
avg: 2.15 ± 0.01ms
min: 1.22 ± 0.00ms
max: 2.94 ± 0.36ms
p99: 2.70 ± 0.05ms

250 trips, 5 coordinates
ops: 309.86 ± 0.18 ops/s. best: 309.48ops/s.
total: 806.81 ± 0.46ms. best: 806.24ms.
avg: 3.23 ± 0.00ms
min: 2.26 ± 0.01ms
max: 4.03 ± 0.02ms
p99: 3.88 ± 0.01ms
250 trips, 3 coordinates
ops: 467.57 ± 4.74 ops/s. best: 458.20ops/s.
total: 534.74 ± 5.62ms. best: 526.88ms.
avg: 2.14 ± 0.02ms
min: 1.21 ± 0.01ms
max: 3.23 ± 0.36ms
p99: 2.76 ± 0.08ms

250 trips, 5 coordinates
ops: 315.94 ± 2.09 ops/s. best: 313.09ops/s.
total: 791.32 ± 5.23ms. best: 781.07ms.
avg: 3.17 ± 0.02ms
min: 2.20 ± 0.01ms
max: 4.07 ± 0.26ms
p99: 3.83 ± 0.04ms
random_trip_mld 250 trips, 3 coordinates
ops: 167.68 ± 2.54 ops/s. best: 164.48ops/s.
total: 1491.35 ± 20.91ms. best: 1450.94ms.
avg: 5.97 ± 0.08ms
min: 3.90 ± 0.01ms
max: 8.09 ± 0.37ms
p99: 7.76 ± 0.25ms

250 trips, 5 coordinates
ops: 109.36 ± 2.40 ops/s. best: 106.82ops/s.
total: 2287.65 ± 49.62ms. best: 2211.33ms.
avg: 9.15 ± 0.20ms
min: 6.39 ± 0.06ms
max: 11.57 ± 0.47ms
p99: 11.16 ± 0.38ms
250 trips, 3 coordinates
ops: 168.75 ± 2.33 ops/s. best: 164.88ops/s.
total: 1481.81 ± 20.37ms. best: 1448.34ms.
avg: 5.93 ± 0.08ms
min: 3.89 ± 0.01ms
max: 8.01 ± 0.33ms
p99: 7.67 ± 0.19ms

250 trips, 5 coordinates
ops: 110.48 ± 1.47 ops/s. best: 107.97ops/s.
total: 2263.42 ± 30.43ms. best: 2225.68ms.
avg: 9.05 ± 0.12ms
min: 6.34 ± 0.05ms
max: 11.45 ± 0.39ms
p99: 11.10 ± 0.18ms
route_ch 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
466.005ms
0.466005ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
564.313ms
0.564313ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
165.038ms
0.165038ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
147.256ms
0.147256ms/req
1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
465.264ms
0.465264ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
563.936ms
0.563936ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
164.354ms
0.164354ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
147.307ms
0.147307ms/req
route_mld 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
602.218ms
0.602218ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
757.085ms
0.757085ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
300.242ms
0.300242ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
322.23ms
0.32223ms/req
1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
605.208ms
0.605208ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
767.608ms
0.767608ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
292.77ms
0.29277ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
318.29ms
0.31829ms/req
rtree 1 result:
202.736ms -> 0.0202736 ms/query
10 results:
239.886ms -> 0.0239886 ms/query
1 result:
205.476ms -> 0.0205476 ms/query
10 results:
240.866ms -> 0.0240866 ms/query

code = response.json()['code']
if code == 'NoSegment' or code == 'NoMatch':
continue
code = response.json()['code']
Copy link
Member Author

@SiarheiFedartsou SiarheiFedartsou Jun 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some requests in benchmark started returning error code which we haven't before and it is not due to this change :) I have no idea why we started having it 🤔
I have 2 theories:

  • berlin-latest.osm.pbf was updated (we don't store it in the repo, but just cache it in GH runner's cache)
  • something was updated on runner

But in general it is expected that some of requests return such errors, so I think it is okay to have it updated.

@SiarheiFedartsou SiarheiFedartsou marked this pull request as ready for review June 16, 2024 12:05
@SiarheiFedartsou SiarheiFedartsou merged commit 3d01d96 into master Jun 22, 2024
@SiarheiFedartsou SiarheiFedartsou deleted the sf-base64-perf branch June 22, 2024 06:50
eliseier pushed a commit to wanderlog/osrm-backend that referenced this pull request Mar 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants