Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support HTTP/2 with TLS in OTLP HTTP receiver. #5109

Merged
merged 6 commits into from
Apr 1, 2022

Conversation

anuraaga
Copy link
Contributor

@anuraaga anuraaga commented Mar 29, 2022

Currently the HTTP receiver never negotiates HTTP/2 even when the client supports it. In a typical scenario of having many apps with batch exporters hitting a collector, HTTP/1 and HTTP/2 should perform mostly the same, but if putting a load balancer in front of the collector, HTTP/2 will be able to multiplex many apps across a single connection and can perform quite a bit better.

Benchmarks are noisy since executing actual HTTP requests. Hopefully we can see some trends

HTTP/2 seems to provide about 3x performance when using a shared client (behaves like a load balancer)
HTTP/2 performs slightly slower with a client per thread (behaves like an app), but this is probably within noise. It is something to confirm we're OK with - I think most services now adays will support HTTP/2 though even if it can be slightly slower in non-shared cases. One way to think of it is gRPC is HTTP/2-only anyways and we'd expect HTTP receiver to have a similar baseline.

After

goos: linux
goarch: amd64
pkg: go.opentelemetry.io/collector/config/confighttp
cpu: Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
BenchmarkHttpRequest
BenchmarkHttpRequest/HTTP/2.0,_shared_client_(like_load_balancer)
BenchmarkHttpRequest/HTTP/2.0,_shared_client_(like_load_balancer)-16         	   29907	     39563 ns/op
BenchmarkHttpRequest/HTTP/1.1,_shared_client_(like_load_balancer)
BenchmarkHttpRequest/HTTP/1.1,_shared_client_(like_load_balancer)-16         	    9814	    110057 ns/op
BenchmarkHttpRequest/HTTP/2.0,_client_per_thread_(like_single_app)
BenchmarkHttpRequest/HTTP/2.0,_client_per_thread_(like_single_app)-16        	   27634	     38459 ns/op
BenchmarkHttpRequest/HTTP/1.1,_client_per_thread_(like_single_app)
BenchmarkHttpRequest/HTTP/1.1,_client_per_thread_(like_single_app)-16        	   15918	     72502 ns/op
PASS

Before (HTTP/2 is actually 1.1)

goos: linux
goarch: amd64
pkg: go.opentelemetry.io/collector/config/confighttp
cpu: Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
BenchmarkHttpRequest
BenchmarkHttpRequest/HTTP/2.0,_shared_client_(like_load_balancer)
BenchmarkHttpRequest/HTTP/2.0,_shared_client_(like_load_balancer)-16         	    9368	    116053 ns/op
BenchmarkHttpRequest/HTTP/1.1,_shared_client_(like_load_balancer)
BenchmarkHttpRequest/HTTP/1.1,_shared_client_(like_load_balancer)-16         	   10587	    113067 ns/op
BenchmarkHttpRequest/HTTP/2.0,_client_per_thread_(like_single_app)
BenchmarkHttpRequest/HTTP/2.0,_client_per_thread_(like_single_app)-16        	   13497	     81537 ns/op
BenchmarkHttpRequest/HTTP/1.1,_client_per_thread_(like_single_app)
BenchmarkHttpRequest/HTTP/1.1,_client_per_thread_(like_single_app)-16        	   14283	     76041 ns/op
PASS

@anuraaga anuraaga requested review from a team and mx-psi March 29, 2022 06:12
@codecov
Copy link

codecov bot commented Mar 29, 2022

Codecov Report

Merging #5109 (34c5561) into main (699a81b) will decrease coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #5109      +/-   ##
==========================================
- Coverage   89.96%   89.94%   -0.03%     
==========================================
  Files         183      182       -1     
  Lines       11104    11081      -23     
==========================================
- Hits         9990     9967      -23     
  Misses        889      889              
  Partials      225      225              
Impacted Files Coverage Δ
config/confighttp/confighttp.go 100.00% <100.00%> (ø)
service/featuregate/flags.go 100.00% <0.00%> (ø)
component/componenthelper/component.go

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 699a81b...34c5561. Read the comment docs.

@anuraaga anuraaga force-pushed the otlp-http2 branch 2 times, most recently from 5232d60 to 7523cb1 Compare March 29, 2022 07:24
config/confighttp/confighttp.go Outdated Show resolved Hide resolved
config/confighttp/confighttp.go Outdated Show resolved Hide resolved
@tigrannajaryan
Copy link
Member

Closing/opening to trigger github actions...

@bogdandrutu bogdandrutu merged commit d2e5601 into open-telemetry:main Apr 1, 2022
Nicholaswang pushed a commit to Nicholaswang/opentelemetry-collector that referenced this pull request Jun 7, 2022
* Support HTTP/2 with TLS in OTLP HTTP receiver.

* Add benchmark

* Explicit http/1 fallback proto

* Update CHANGELOG.md

Co-authored-by: Bogdan Drutu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants