-
Notifications
You must be signed in to change notification settings - Fork 142
2022 02 19 bench results
Andrea edited this page Feb 19, 2022
·
1 revision
Benchmarked on: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz, 32G RAM
Config:
3a8cb60 Fri, 18 Feb 2022 17:02:15 +0000 Trisfald Make the long benchmark work again
- GRPC_BENCHMARK_DURATION=120s
- GRPC_BENCHMARK_WARMUP=30s
- GRPC_SERVER_RAM=512m
- GRPC_CLIENT_CONNECTIONS=50
- GRPC_CLIENT_CONCURRENCY=1000
- GRPC_CLIENT_QPS=0
- GRPC_CLIENT_CPUS=9
- GRPC_REQUEST_SCENARIO=complex_proto
Note: I had to disable erlang and elixir benches because I couldn't get them to build successfully.
1 CPU Server
-----------------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
-----------------------------------------------------------------------------------------------------------------------------------------
| rust_tonic_st | 46683 | 20.94 ms | 24.76 ms | 26.67 ms | 29.83 ms | 100.38% | 13.15 MiB |
| rust_thruster_st | 43512 | 22.51 ms | 25.87 ms | 28.49 ms | 32.10 ms | 100.76% | 9.46 MiB |
| rust_tonic_mt | 40146 | 24.59 ms | 10.85 ms | 12.48 ms | 602.61 ms | 100.85% | 14.8 MiB |
| rust_thruster_mt | 36831 | 26.85 ms | 11.87 ms | 13.27 ms | 643.37 ms | 101.5% | 10.7 MiB |
| scala_akka | 34920 | 26.96 ms | 62.03 ms | 74.50 ms | 97.54 ms | 101.56% | 228.89 MiB |
| rust_grpcio | 33696 | 29.43 ms | 33.02 ms | 34.24 ms | 36.85 ms | 102.14% | 18.25 MiB |
| java_quarkus | 33031 | 29.57 ms | 51.97 ms | 56.77 ms | 80.91 ms | 100.1% | 113.75 MiB |
| java_hotspot_grpc_pgc | 32522 | 29.72 ms | 55.20 ms | 59.44 ms | 80.43 ms | 100.75% | 189.94 MiB |
| dotnet_grpc | 32099 | 28.39 ms | 58.78 ms | 69.62 ms | 82.62 ms | 100.9% | 109.51 MiB |
| java_hotspot_grpc_g1gc | 29896 | 32.69 ms | 58.94 ms | 65.12 ms | 84.20 ms | 100.82% | 159.77 MiB |
| cpp_grpc_mt | 28068 | 35.46 ms | 39.87 ms | 40.98 ms | 44.36 ms | 102.23% | 11.34 MiB |
| cpp_grpc_st | 27955 | 35.61 ms | 40.68 ms | 41.68 ms | 44.69 ms | 102.25% | 9.02 MiB |
| java_hotspot_grpc_she | 27819 | 35.20 ms | 68.57 ms | 83.95 ms | 100.07 ms | 101.37% | 395.68 MiB |
| java_hotspot_grpc_zgc | 27306 | 35.84 ms | 66.94 ms | 80.80 ms | 101.32 ms | 101.12% | 122.7 MiB |
| cpp_asio_grpc | 26888 | 37.04 ms | 41.72 ms | 43.26 ms | 46.38 ms | 102.61% | 9.12 MiB |
| java_openj9_grpc_gencon | 24428 | 39.82 ms | 92.16 ms | 97.64 ms | 108.29 ms | 101.37% | 119.92 MiB |
| java_hotspot_grpc_sgc | 23543 | 41.75 ms | 73.48 ms | 81.44 ms | 98.38 ms | 101.0% | 84.14 MiB |
| crystal_grpc | 22001 | 44.48 ms | 53.56 ms | 55.88 ms | 62.05 ms | 123.87% | 37.53 MiB |
| java_quarkus_native | 20184 | 49.29 ms | 54.22 ms | 55.80 ms | 58.93 ms | 100.57% | 88.22 MiB |
| kotlin_grpc | 17802 | 55.35 ms | 94.28 ms | 101.89 ms | 171.67 ms | 100.72% | 205.18 MiB |
| go_vtgrpc | 14878 | 62.91 ms | 100.51 ms | 107.10 ms | 187.02 ms | 96.54% | 30.03 MiB |
| java_micronaut_workstealing | 12423 | 80.00 ms | 102.29 ms | 110.83 ms | 189.85 ms | 101.42% | 467.16 MiB |
| go_grpc | 10143 | 94.72 ms | 193.55 ms | 206.35 ms | 303.52 ms | 95.77% | 30.3 MiB |
| java_micronaut | 9627 | 103.35 ms | 105.88 ms | 126.41 ms | 503.65 ms | 100.72% | 476.08 MiB |
| scala_zio | 5434 | 183.05 ms | 294.81 ms | 329.64 ms | 595.98 ms | 100.62% | 260.46 MiB |
| csharp_grpc | 4799 | 207.78 ms | 259.45 ms | 270.44 ms | 282.74 ms | 101.77% | 102.73 MiB |
| node_grpcjs_st | 4182 | 238.26 ms | 264.06 ms | 269.91 ms | 285.89 ms | 100.78% | 46.06 MiB |
| scala_fs2 | 2909 | 342.63 ms | 388.79 ms | 489.27 ms | 579.08 ms | 102.03% | 220.66 MiB |
| python_async_grpc | 2628 | 379.22 ms | 406.54 ms | 411.95 ms | 443.79 ms | 101.82% | 32.79 MiB |
| python_grpc | 2147 | 463.75 ms | 490.81 ms | 494.88 ms | 502.44 ms | 102.02% | 35.26 MiB |
| ruby_grpc | 1913 | 520.40 ms | 576.57 ms | 592.24 ms | 657.60 ms | 101.46% | 37.07 MiB |
| java_aot | 1694 | 586.45 ms | 997.64 ms | 1.17 s | 1.49 s | 100.46% | 188.31 MiB |
| php_grpc | 895 | 1.11 s | 1.19 s | 1.19 s | 1.20 s | 101.22% | 65.57 MiB |
-----------------------------------------------------------------------------------------------------------------------------------------
2 CPU Server
-----------------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
-----------------------------------------------------------------------------------------------------------------------------------------
| scala_akka | 70319 | 11.25 ms | 21.08 ms | 28.33 ms | 54.47 ms | 186.66% | 237.19 MiB |
| dotnet_grpc | 69564 | 11.45 ms | 18.22 ms | 22.45 ms | 31.34 ms | 195.58% | 164.42 MiB |
| cpp_grpc_mt | 69152 | 11.80 ms | 21.67 ms | 26.60 ms | 37.00 ms | 204.02% | 34.25 MiB |
| cpp_asio_grpc | 67312 | 12.54 ms | 23.65 ms | 28.16 ms | 37.49 ms | 204.18% | 35.19 MiB |
| rust_tonic_mt | 62853 | 14.48 ms | 36.81 ms | 58.07 ms | 83.36 ms | 196.24% | 17.85 MiB |
| java_hotspot_grpc_pgc | 61890 | 13.51 ms | 28.48 ms | 37.60 ms | 56.39 ms | 203.45% | 205.68 MiB |
| rust_thruster_mt | 57917 | 15.78 ms | 40.81 ms | 60.03 ms | 83.17 ms | 198.59% | 12.12 MiB |
| rust_grpcio | 56283 | 16.49 ms | 25.78 ms | 29.79 ms | 36.81 ms | 202.36% | 36.72 MiB |
| java_hotspot_grpc_g1gc | 53688 | 16.22 ms | 36.98 ms | 52.23 ms | 68.05 ms | 203.42% | 173.68 MiB |
| java_hotspot_grpc_she | 50984 | 17.30 ms | 39.47 ms | 50.53 ms | 68.28 ms | 203.56% | 396.48 MiB |
| java_hotspot_grpc_zgc | 48967 | 18.12 ms | 39.64 ms | 49.10 ms | 67.24 ms | 204.08% | 128.73 MiB |
| java_openj9_grpc_gencon | 48516 | 17.83 ms | 38.67 ms | 46.20 ms | 59.40 ms | 203.32% | 134.58 MiB |
| rust_tonic_st | 47174 | 20.72 ms | 24.87 ms | 26.51 ms | 30.05 ms | 101.4% | 12.87 MiB |
| java_hotspot_grpc_sgc | 46047 | 19.69 ms | 38.12 ms | 43.94 ms | 64.60 ms | 195.37% | 91.94 MiB |
| kotlin_grpc | 45692 | 19.42 ms | 62.24 ms | 69.51 ms | 80.33 ms | 203.46% | 226.26 MiB |
| java_quarkus | 44452 | 20.68 ms | 41.91 ms | 45.93 ms | 53.90 ms | 164.3% | 104.66 MiB |
| rust_thruster_st | 43232 | 22.67 ms | 27.52 ms | 29.19 ms | 32.64 ms | 101.33% | 9.79 MiB |
| go_vtgrpc | 42229 | 19.31 ms | 50.95 ms | 58.00 ms | 69.11 ms | 193.63% | 29.29 MiB |
| java_quarkus_native | 33920 | 29.02 ms | 34.13 ms | 36.53 ms | 40.63 ms | 194.98% | 88.46 MiB |
| cpp_grpc_st | 28857 | 34.47 ms | 38.90 ms | 40.33 ms | 43.04 ms | 102.65% | 9.0 MiB |
| go_grpc | 27987 | 31.48 ms | 76.28 ms | 83.40 ms | 96.75 ms | 203.65% | 30.2 MiB |
| java_micronaut | 26355 | 36.68 ms | 66.10 ms | 72.41 ms | 84.23 ms | 183.59% | 501.29 MiB |
| java_micronaut_workstealing | 24891 | 38.98 ms | 70.56 ms | 77.46 ms | 91.07 ms | 181.73% | 503.93 MiB |
| crystal_grpc | 21789 | 44.91 ms | 54.53 ms | 56.20 ms | 62.11 ms | 128.1% | 38.23 MiB |
| csharp_grpc | 17160 | 57.02 ms | 87.47 ms | 92.32 ms | 101.49 ms | 205.86% | 102.45 MiB |
| scala_zio | 10604 | 92.64 ms | 124.57 ms | 181.48 ms | 262.21 ms | 201.98% | 267.63 MiB |
| java_aot | 6473 | 152.45 ms | 287.09 ms | 309.77 ms | 409.35 ms | 197.6% | 214.28 MiB |
| node_grpcjs_st | 6300 | 158.03 ms | 169.86 ms | 172.94 ms | 178.73 ms | 130.84% | 47.28 MiB |
| scala_fs2 | 6188 | 161.24 ms | 188.15 ms | 200.08 ms | 293.19 ms | 206.68% | 229.39 MiB |
| python_async_grpc | 4919 | 202.85 ms | 222.77 ms | 226.03 ms | 235.51 ms | 151.14% | 36.11 MiB |
| python_grpc | 3800 | 262.44 ms | 284.87 ms | 288.63 ms | 296.21 ms | 150.47% | 34.71 MiB |
| ruby_grpc | 2535 | 392.99 ms | 422.80 ms | 431.79 ms | 470.86 ms | 132.0% | 38.17 MiB |
| php_grpc | 2178 | 457.16 ms | 491.65 ms | 492.82 ms | 494.99 ms | 206.34% | 66.67 MiB |
-----------------------------------------------------------------------------------------------------------------------------------------
3 CPU Server
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
-----------------------------------------------------------------------------------------------------------------------------------------
| scala_akka | 73637 | 10.23 ms | 17.85 ms | 22.45 ms | 33.05 ms | 197.92% | 243.02 MiB |
| cpp_grpc_mt | 72761 | 9.64 ms | 18.51 ms | 24.89 ms | 39.29 ms | 239.47% | 40.37 MiB |
| cpp_asio_grpc | 72551 | 9.72 ms | 18.33 ms | 24.19 ms | 38.24 ms | 242.92% | 41.93 MiB |
| java_hotspot_grpc_pgc | 72117 | 10.04 ms | 22.22 ms | 28.71 ms | 42.04 ms | 245.1% | 210.94 MiB |
| dotnet_grpc | 72100 | 10.59 ms | 17.31 ms | 21.75 ms | 31.98 ms | 207.01% | 172.77 MiB |
| go_vtgrpc | 71942 | 9.88 ms | 18.21 ms | 22.09 ms | 32.11 ms | 274.81% | 28.83 MiB |
| java_hotspot_grpc_g1gc | 70697 | 10.44 ms | 21.75 ms | 27.96 ms | 40.18 ms | 263.08% | 209.87 MiB |
| rust_tonic_mt | 70382 | 11.43 ms | 22.73 ms | 29.83 ms | 43.95 ms | 260.82% | 19.6 MiB |
| java_hotspot_grpc_zgc | 69698 | 10.68 ms | 22.05 ms | 29.18 ms | 43.50 ms | 271.74% | 132.86 MiB |
| java_hotspot_grpc_she | 69530 | 10.81 ms | 23.63 ms | 31.33 ms | 46.42 ms | 268.35% | 413.63 MiB |
| rust_grpcio | 69171 | 11.44 ms | 19.94 ms | 25.40 ms | 37.50 ms | 257.03% | 29.18 MiB |
| java_openj9_grpc_gencon | 68972 | 10.80 ms | 23.21 ms | 29.92 ms | 41.90 ms | 275.43% | 143.76 MiB |
| rust_thruster_mt | 66525 | 12.06 ms | 23.62 ms | 30.55 ms | 43.80 ms | 258.15% | 12.63 MiB |
| kotlin_grpc | 64433 | 11.97 ms | 30.07 ms | 44.06 ms | 66.12 ms | 291.83% | 224.17 MiB |
| java_hotspot_grpc_sgc | 58901 | 14.12 ms | 29.55 ms | 34.79 ms | 43.59 ms | 230.87% | 96.44 MiB |
| java_quarkus | 53345 | 15.70 ms | 27.55 ms | 40.28 ms | 49.46 ms | 202.86% | 113.46 MiB |
| go_grpc | 51861 | 15.15 ms | 32.76 ms | 40.88 ms | 55.76 ms | 307.96% | 30.46 MiB |
| rust_tonic_st | 47196 | 20.71 ms | 25.17 ms | 26.80 ms | 30.13 ms | 101.32% | 13.18 MiB |
| rust_thruster_st | 42880 | 22.88 ms | 26.41 ms | 29.14 ms | 32.87 ms | 100.79% | 9.56 MiB |
| java_quarkus_native | 42551 | 22.48 ms | 29.20 ms | 32.25 ms | 39.45 ms | 282.54% | 89.74 MiB |
| java_micronaut_workstealing | 41256 | 21.93 ms | 45.87 ms | 55.98 ms | 69.52 ms | 280.96% | 511.61 MiB |
| java_micronaut | 40761 | 22.43 ms | 43.08 ms | 51.66 ms | 67.15 ms | 293.93% | 510.27 MiB |
| csharp_grpc | 29183 | 32.75 ms | 54.43 ms | 59.24 ms | 67.66 ms | 307.05% | 108.81 MiB |
| cpp_grpc_st | 28500 | 34.92 ms | 39.15 ms | 40.58 ms | 42.95 ms | 102.63% | 8.82 MiB |
| crystal_grpc | 21708 | 45.09 ms | 55.05 ms | 56.60 ms | 62.99 ms | 128.46% | 38.25 MiB |
| scala_zio | 15319 | 64.13 ms | 103.81 ms | 138.97 ms | 207.28 ms | 307.8% | 301.58 MiB |
| scala_fs2 | 12386 | 80.47 ms | 97.85 ms | 104.32 ms | 165.24 ms | 317.14% | 243.53 MiB |
| java_aot | 12116 | 80.55 ms | 164.68 ms | 190.44 ms | 244.95 ms | 286.29% | 219.2 MiB |
| node_grpcjs_st | 6324 | 157.63 ms | 168.92 ms | 171.83 ms | 176.78 ms | 130.33% | 45.81 MiB |
| python_async_grpc | 4908 | 203.32 ms | 223.31 ms | 227.02 ms | 234.37 ms | 150.77% | 38.8 MiB |
| python_grpc | 3881 | 256.98 ms | 279.76 ms | 283.31 ms | 292.62 ms | 150.14% | 35.08 MiB |
| php_grpc | 3578 | 278.71 ms | 294.79 ms | 295.83 ms | 298.77 ms | 310.46% | 67.34 MiB |
| ruby_grpc | 2550 | 390.69 ms | 426.46 ms | 439.95 ms | 517.74 ms | 132.95% | 37.02 MiB |
-----------------------------------------------------------------------------------------------------------------------------------------