Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Teleport doesn't support proxy protocol v2 #4904

Closed
webvictim opened this issue Nov 18, 2020 · 2 comments · Fixed by #11684
Closed

Teleport doesn't support proxy protocol v2 #4904

webvictim opened this issue Nov 18, 2020 · 2 comments · Fixed by #11684
Assignees
@probakowski @zmb3
Labels
c-cx Internal Customer Reference

Comments

@webvictim
Copy link
Contributor

webvictim commented Nov 18, 2020 •
edited by programmerq
Loading

Description

What happened: Enable proxy protocol v2 (binary version) on an AWS NLB. Teleport can't parse the binary version (v2), only the human-readable version (v1), so you get a lot of multiplexer errors:

Nov 18 20:12:33 ip-172-31-2-140.ec2.internal /usr/bin/teleport[3496]: WARN [MX:PROXY:] "\nERROR REPORT:\nOriginal Error: *trace.BadParameterError failed to detect protocol by prefix: [13 10 13]\nStack Trace:\n\t/go/src/github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:348 github.com/gravitational/teleport/lib/multiplexer.detectProto\n\t/go/src/github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:263 github.com/gravitational/teleport/lib/multiplexer.detect\n\t/go/src/github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:197 github.com/gravitational/teleport/lib/multiplexer.(*Mux).detectAndForward\n\t/opt/go/src/runtime/asm_amd64.s:1374 runtime.goexit\nUser Message: failed to detect protocol by prefix: [13 10 13]\n" multiplexer/multiplexer.go:200
Nov 18 20:12:33 ip-172-31-2-140.ec2.internal /usr/bin/teleport[3496]: WARN [MX:PROXY:] "\nERROR REPORT:\nOriginal Error: *trace.BadParameterError failed to detect protocol by prefix: [13 10 13]\nStack Trace:\n\t/go/src/github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:348 github.com/gravitational/teleport/lib/multiplexer.detectProto\n\t/go/src/github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:263 github.com/gravitational/teleport/lib/multiplexer.detect\n\t/go/src/github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:197 github.com/gravitational/teleport/lib/multiplexer.(*Mux).detectAndForward\n\t/opt/go/src/runtime/asm_amd64.s:1374 runtime.goexit\nUser Message: failed to detect protocol by prefix: [13 10 13]\n" multiplexer/multiplexer.go:200
Nov 18 20:12:33 ip-172-31-2-140.ec2.internal /usr/bin/teleport[3496]: WARN [MX:PROXY:] "\nERROR REPORT:\nOriginal Error: *trace.BadParameterError failed to detect protocol by prefix: [13 10 13]\nStack Trace:\n\t/go/src/github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:348 github.com/gravitational/teleport/lib/multiplexer.detectProto\n\t/go/src/github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:263 github.com/gravitational/teleport/lib/multiplexer.detect\n\t/go/src/github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:197 github.com/gravitational/teleport/lib/multiplexer.(*Mux).detectAndForward\n\t/opt/go/src/runtime/asm_amd64.s:1374 runtime.goexit\nUser Message: failed to detect protocol by prefix: [13 10 13]\n" multiplexer/multiplexer.go:200
Nov 18 20:12:34 ip-172-31-2-140.ec2.internal /usr/bin/teleport[3496]: WARN [MX:PROXY:] "\nERROR REPORT:\nOriginal Error: *trace.BadParameterError failed to detect protocol by prefix: [13 10 13]\nStack Trace:\n\t/go/src/github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:348 github.com/gravitational/teleport/lib/multiplexer.detectProto\n\t/go/src/github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:263 github.com/gravitational/teleport/lib/multiplexer.detect\n\t/go/src/github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:197 github.com/gravitational/teleport/lib/multiplexer.(*Mux).detectAndForward\n\t/opt/go/src/runtime/asm_amd64.s:1374 runtime.goexit\nUser Message: failed to detect protocol by prefix: [13 10 13]\n" multiplexer/multiplexer.go:200
Nov 18 20:12:35 ip-172-31-2-140.ec2.internal /usr/bin/teleport[3496]: WARN [MX:PROXY:] "\nERROR REPORT:\nOriginal Error: *trace.BadParameterError failed to detect protocol by prefix: [13 10 13]\nStack Trace:\n\t/go/src/github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:348 github.com/gravitational/teleport/lib/multiplexer.detectProto\n\t/go/src/github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:263 github.com/gravitational/teleport/lib/multiplexer.detect\n\t/go/src/github.com/gravitational/teleport/lib/multiplexer/multiplexer.go:197 github.com/gravitational/teleport/lib/multiplexer.(*Mux).detectAndForward\n\t/opt/go/src/runtime/asm_amd64.s:1374 runtime.goexit\nUser Message: failed to detect protocol by prefix: [13 10 13]\n" multiplexer/multiplexer.go:200

What you expected to happen: Proxy protocol v2 headers should be parsed correctly.

How to reproduce it (as minimally and precisely as possible): Deploy Teleport in AWS using HA reference Terraform, enable proxy protocol for proxyweb group, watch errors.

Environment

  • Teleport version (use teleport version): Teleport Enterprise v4.4.5 git:v4.4.5-0-g23a2e42a8 go1.14.4

gz#4292
@russjones russjones added this to the 5.1 "San Diego" milestone Nov 24, 2020
@alex-kovoy alex-kovoy mentioned this issue Dec 11, 2020
Closed
@russjones russjones modified the milestones: 6.0 "San Diego", 6.1 Jan 26, 2021
@russjones russjones modified the milestones: 6.1, Runway Milestone Feb 3, 2021
@webvictim webvictim mentioned this issue Apr 28, 2021
Closed
@liorfranko
Copy link

+1

1 similar comment
@galgross
Copy link

galgross commented Mar 8, 2022

+1

@programmerq programmerq added the c-cx Internal Customer Reference label Mar 9, 2022
@russjones russjones removed this from the Runway Milestone milestone Mar 15, 2022
@probakowski probakowski mentioned this issue Apr 2, 2022
Merged
probakowski added a commit that referenced this issue Apr 5, 2022
@probakowski
Support proxy protocol v2 (#11684)
3ff19cf 
This change add multiplexer support for proxy protocol v2 (binary).

Closes #4904
probakowski added a commit that referenced this issue Apr 5, 2022
@probakowski
Support proxy protocol v2 (#11684)
d930139 
This change add multiplexer support for proxy protocol v2 (binary).

Closes #4904

(cherry picked from commit 3ff19cf)
probakowski added a commit that referenced this issue Apr 5, 2022
@probakowski
Support proxy protocol v2 (#11684) (#11722)
a99d3b4 
This change add multiplexer support for proxy protocol v2 (binary).

Closes #4904

(cherry picked from commit 3ff19cf)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@probakowski probakowski

@zmb3 zmb3

Labels
c-cx Internal Customer Reference
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support proxy protocol v2 gravitational/teleport
7 participants
@programmerq @webvictim @probakowski @russjones @zmb3 @liorfranko @galgross