-
Notifications
You must be signed in to change notification settings - Fork 142
2021 04 13 bench results
LesnyRumcajs edited this page Apr 13, 2021
·
1 revision
Benchmarked on: Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz, 32G RAM
Benchmark duration per suite: 300s
Commit: a7ae82c Fri, 9 Apr 2021 12:06:54 +0200 GitHub Merge pull request #116 from LesnyRumcajs/bench-config-in-results
Config:
GRPC_BENCHMARK_DURATION=300s
GRPC_SERVER_CPUS=<1-3>
GRPC_SERVER_RAM=512m
GRPC_CLIENT_CONNECTIONS=5
GRPC_CLIENT_CONCURRENCY=50
GRPC_CLIENT_QPS=0
GRPC_CLIENT_CPUS=9
GRPC_REQUEST_PAYLOAD=100B
1 CPU Server
--------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
--------------------------------------------------------------------------------------------------------------------------------
| java_grpc_pgc | 50383 | 0.94 ms | 1.46 ms | 1.91 ms | 6.39 ms | 100.53% | 115.41 MiB |
| rust_tonic_st | 46810 | 1.01 ms | 1.21 ms | 1.32 ms | 1.86 ms | 100.2% | 4.15 MiB |
| java_grpc_g1gc | 46118 | 1.03 ms | 1.59 ms | 2.14 ms | 6.11 ms | 100.74% | 93.85 MiB |
| java_grpc_sgc | 45386 | 1.04 ms | 1.71 ms | 2.15 ms | 6.60 ms | 100.76% | 60.52 MiB |
| java_grpc_zgc | 43088 | 1.10 ms | 1.59 ms | 2.11 ms | 8.34 ms | 100.59% | 110.19 MiB |
| java_grpc_she | 41829 | 1.14 ms | 1.88 ms | 2.72 ms | 19.28 ms | 100.84% | 369.95 MiB |
| rust_thruster_st | 41417 | 1.15 ms | 1.29 ms | 1.41 ms | 2.05 ms | 101.21% | 5.82 MiB |
| rust_grpcio | 37320 | 1.29 ms | 1.54 ms | 1.64 ms | 2.09 ms | 101.85% | 9.96 MiB |
| rust_tonic_mt | 31947 | 1.51 ms | 2.28 ms | 2.51 ms | 2.97 ms | 101.33% | 3.97 MiB |
| rust_thruster_mt | 29281 | 1.65 ms | 2.53 ms | 2.79 ms | 3.29 ms | 101.17% | 5.46 MiB |
| cpp_grpc_st | 28825 | 1.68 ms | 2.09 ms | 2.21 ms | 2.56 ms | 102.04% | 3.3 MiB |
| cpp_grpc_mt | 28707 | 1.69 ms | 2.07 ms | 2.19 ms | 2.55 ms | 101.99% | 3.35 MiB |
| dotnet_grpc | 27395 | 1.76 ms | 1.88 ms | 2.35 ms | 13.40 ms | 102.0% | 77.79 MiB |
| lua_grpc_st | 18662 | 2.63 ms | 3.47 ms | 3.78 ms | 4.44 ms | 101.63% | 7.78 MiB |
| java_micronaut | 11916 | 4.12 ms | 3.74 ms | 7.68 ms | 66.51 ms | 101.72% | 148.22 MiB |
| node_grpc_st | 11894 | 4.10 ms | 4.42 ms | 4.68 ms | 8.00 ms | 100.56% | 19.09 MiB |
| go_grpc | 11701 | 4.19 ms | 2.16 ms | 4.23 ms | 76.57 ms | 101.61% | 14.62 MiB |
| swift_grpc | 10590 | 4.63 ms | 7.35 ms | 7.63 ms | 8.46 ms | 101.78% | 2.7 MiB |
| dart_grpc | 8289 | 5.97 ms | 30.39 ms | 46.18 ms | 60.97 ms | 102.02% | 25.42 MiB |
| kotlin_grpc | 7858 | 6.29 ms | 4.06 ms | 61.58 ms | 77.06 ms | 101.62% | 187.3 MiB |
| node_grpcjs_st | 6198 | 7.96 ms | 10.29 ms | 13.88 ms | 34.91 ms | 101.27% | 30.41 MiB |
| csharp_grpc | 5906 | 8.39 ms | 6.09 ms | 69.81 ms | 79.42 ms | 101.47% | 74.45 MiB |
| elixir_grpc | 4750 | 10.45 ms | 11.33 ms | 11.84 ms | 14.88 ms | 101.54% | 56.3 MiB |
| java_aot | 2672 | 18.61 ms | 85.43 ms | 98.78 ms | 125.56 ms | 101.71% | 121.51 MiB |
| python_grpc | 2350 | 21.20 ms | 47.45 ms | 48.78 ms | 50.55 ms | 101.95% | 16.28 MiB |
| ruby_grpc | 1958 | 25.43 ms | 45.74 ms | 47.03 ms | 49.06 ms | 102.0% | 18.32 MiB |
| crystal_grpc | 1925 | 25.86 ms | 44.10 ms | 44.26 ms | 47.67 ms | 16.11% | 6.12 MiB |
| php_grpc | 1903 | 26.19 ms | 87.62 ms | 88.38 ms | 89.59 ms | 101.58% | 39.75 MiB |
| scala_akka | 1419 | 35.13 ms | 93.57 ms | 97.07 ms | 188.39 ms | 100.69% | 144.48 MiB |
--------------------------------------------------------------------------------------------------------------------------------
2 CPU Server
--------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
--------------------------------------------------------------------------------------------------------------------------------
| java_grpc_pgc | 76122 | 0.61 ms | 1.11 ms | 1.48 ms | 3.84 ms | 189.73% | 116.02 MiB |
| java_grpc_g1gc | 72914 | 0.64 ms | 1.20 ms | 1.67 ms | 4.38 ms | 192.25% | 116.09 MiB |
| java_grpc_zgc | 71917 | 0.65 ms | 1.13 ms | 1.50 ms | 4.32 ms | 197.03% | 108.94 MiB |
| java_grpc_she | 69374 | 0.67 ms | 1.31 ms | 1.85 ms | 4.85 ms | 195.78% | 368.7 MiB |
| java_grpc_sgc | 63094 | 0.74 ms | 1.33 ms | 1.66 ms | 3.70 ms | 186.76% | 60.57 MiB |
| rust_grpcio | 55100 | 0.86 ms | 1.10 ms | 1.20 ms | 1.68 ms | 204.61% | 14.7 MiB |
| cpp_grpc_mt | 47077 | 1.01 ms | 1.26 ms | 1.53 ms | 16.23 ms | 206.16% | 8.42 MiB |
| rust_tonic_st | 46304 | 1.02 ms | 1.22 ms | 1.32 ms | 1.86 ms | 100.4% | 4.05 MiB |
| dotnet_grpc | 43258 | 1.10 ms | 1.45 ms | 1.75 ms | 3.92 ms | 196.55% | 91.52 MiB |
| rust_tonic_mt | 42605 | 1.12 ms | 1.68 ms | 1.86 ms | 2.27 ms | 195.87% | 4.68 MiB |
| rust_thruster_st | 41367 | 1.15 ms | 1.30 ms | 1.41 ms | 2.05 ms | 100.67% | 5.85 MiB |
| rust_thruster_mt | 37666 | 1.28 ms | 1.92 ms | 2.12 ms | 2.59 ms | 196.99% | 5.8 MiB |
| go_grpc | 29210 | 1.65 ms | 1.61 ms | 2.18 ms | 43.69 ms | 205.07% | 14.86 MiB |
| cpp_grpc_st | 29108 | 1.67 ms | 2.05 ms | 2.17 ms | 2.52 ms | 101.99% | 3.22 MiB |
| java_micronaut | 24752 | 1.96 ms | 2.24 ms | 3.15 ms | 46.72 ms | 204.68% | 144.44 MiB |
| swift_grpc | 19625 | 2.48 ms | 3.94 ms | 4.11 ms | 5.07 ms | 198.83% | 2.99 MiB |
| lua_grpc_st | 18806 | 2.61 ms | 3.49 ms | 3.81 ms | 4.43 ms | 101.58% | 7.76 MiB |
| csharp_grpc | 16176 | 3.02 ms | 2.54 ms | 4.24 ms | 54.21 ms | 204.6% | 77.19 MiB |
| kotlin_grpc | 13710 | 3.58 ms | 2.71 ms | 4.41 ms | 66.51 ms | 203.64% | 206.44 MiB |
| node_grpc_st | 12124 | 4.03 ms | 4.34 ms | 4.46 ms | 5.59 ms | 103.65% | 19.03 MiB |
| dart_grpc | 10240 | 4.83 ms | 24.82 ms | 35.06 ms | 48.28 ms | 110.19% | 25.81 MiB |
| elixir_grpc | 9037 | 5.47 ms | 7.21 ms | 7.78 ms | 9.34 ms | 203.65% | 58.13 MiB |
| node_grpcjs_st | 8048 | 6.11 ms | 7.18 ms | 8.59 ms | 11.52 ms | 116.51% | 30.37 MiB |
| java_aot | 7791 | 6.34 ms | 4.26 ms | 54.67 ms | 87.55 ms | 201.43% | 122.3 MiB |
| scala_akka | 4621 | 10.74 ms | 62.54 ms | 69.00 ms | 81.63 ms | 202.71% | 142.95 MiB |
| php_grpc | 4542 | 10.93 ms | 66.42 ms | 70.34 ms | 71.97 ms | 205.02% | 40.23 MiB |
| python_grpc | 4209 | 11.81 ms | 13.41 ms | 13.90 ms | 16.90 ms | 149.7% | 16.51 MiB |
| ruby_grpc | 2692 | 18.48 ms | 21.58 ms | 22.16 ms | 23.35 ms | 135.17% | 18.61 MiB |
| crystal_grpc | 1906 | 26.12 ms | 44.10 ms | 44.25 ms | 47.71 ms | 16.49% | 6.03 MiB |
--------------------------------------------------------------------------------------------------------------------------------
3 CPU Server (it's the best we could do with our resources, still it was not enough to properly stress all implementations)
--------------------------------------------------------------------------------------------------------------------------------
| name | req/s | avg. latency | 90 % in | 95 % in | 99 % in | avg. cpu | avg. memory |
--------------------------------------------------------------------------------------------------------------------------------
| java_grpc_pgc | 80867 | 0.57 ms | 1.10 ms | 1.48 ms | 3.46 ms | 233.89% | 147.81 MiB |
| java_grpc_zgc | 79028 | 0.58 ms | 1.11 ms | 1.50 ms | 3.42 ms | 249.63% | 116.92 MiB |
| java_grpc_g1gc | 78917 | 0.58 ms | 1.18 ms | 1.56 ms | 3.50 ms | 244.93% | 124.46 MiB |
| java_grpc_she | 77761 | 0.59 ms | 1.27 ms | 1.76 ms | 4.23 ms | 250.49% | 393.29 MiB |
| java_grpc_sgc | 75167 | 0.62 ms | 1.24 ms | 1.62 ms | 3.61 ms | 231.12% | 67.74 MiB |
| rust_grpcio | 72265 | 0.64 ms | 0.89 ms | 1.09 ms | 2.05 ms | 267.75% | 16.11 MiB |
| cpp_grpc_mt | 67708 | 0.69 ms | 1.10 ms | 1.32 ms | 2.30 ms | 288.93% | 9.31 MiB |
| dotnet_grpc | 51561 | 0.92 ms | 1.30 ms | 1.55 ms | 3.46 ms | 266.38% | 100.72 MiB |
| rust_tonic_mt | 49191 | 0.96 ms | 1.44 ms | 1.61 ms | 2.12 ms | 269.38% | 5.12 MiB |
| go_grpc | 48423 | 0.97 ms | 1.39 ms | 1.82 ms | 10.73 ms | 309.69% | 15.44 MiB |
| rust_thruster_mt | 46261 | 1.03 ms | 1.53 ms | 1.72 ms | 2.25 ms | 268.9% | 6.18 MiB |
| rust_tonic_st | 45690 | 1.04 ms | 1.20 ms | 1.30 ms | 1.85 ms | 100.58% | 3.88 MiB |
| rust_thruster_st | 40613 | 1.17 ms | 1.32 ms | 1.43 ms | 2.05 ms | 100.09% | 6.07 MiB |
| java_micronaut | 35630 | 1.35 ms | 2.03 ms | 2.80 ms | 33.31 ms | 306.98% | 158.8 MiB |
| cpp_grpc_st | 29472 | 1.65 ms | 2.05 ms | 2.17 ms | 2.53 ms | 102.06% | 3.14 MiB |
| swift_grpc | 27535 | 1.76 ms | 2.18 ms | 2.28 ms | 2.84 ms | 302.28% | 3.27 MiB |
| csharp_grpc | 27377 | 1.77 ms | 2.02 ms | 3.04 ms | 33.22 ms | 308.46% | 77.29 MiB |
| kotlin_grpc | 26463 | 1.83 ms | 2.40 ms | 3.43 ms | 53.66 ms | 306.83% | 212.95 MiB |
| lua_grpc_st | 19095 | 2.57 ms | 3.45 ms | 3.79 ms | 4.45 ms | 101.74% | 7.28 MiB |
| java_aot | 14330 | 3.42 ms | 2.78 ms | 9.12 ms | 59.06 ms | 304.15% | 122.24 MiB |
| elixir_grpc | 12797 | 3.85 ms | 5.29 ms | 6.11 ms | 7.43 ms | 306.49% | 60.25 MiB |
| node_grpc_st | 11855 | 4.12 ms | 4.42 ms | 4.54 ms | 5.79 ms | 103.14% | 19.17 MiB |
| dart_grpc | 10073 | 4.91 ms | 25.33 ms | 36.04 ms | 49.22 ms | 109.99% | 25.41 MiB |
| scala_akka | 8769 | 5.63 ms | 7.25 ms | 44.31 ms | 66.00 ms | 305.4% | 144.56 MiB |
| node_grpcjs_st | 8032 | 6.13 ms | 7.15 ms | 8.59 ms | 11.50 ms | 116.66% | 29.85 MiB |
| php_grpc | 7576 | 6.53 ms | 5.10 ms | 51.82 ms | 54.52 ms | 311.2% | 41.53 MiB |
| python_grpc | 4271 | 11.64 ms | 13.24 ms | 13.67 ms | 16.77 ms | 150.38% | 16.64 MiB |
| ruby_grpc | 2659 | 18.71 ms | 21.81 ms | 22.41 ms | 23.64 ms | 135.32% | 18.17 MiB |
| crystal_grpc | 1909 | 26.07 ms | 44.11 ms | 44.27 ms | 47.72 ms | 15.77% | 6.14 MiB |
--------------------------------------------------------------------------------------------------------------------------------