Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add proxy-set-headers annotation for ingress #5366

Merged
merged 100 commits into from
Apr 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
100 commits
Select commit Hold shift + click to select a range
8a41ce7
Added ProxySetHeaders for annotation, added tests
AlexFenlon Mar 15, 2024
4de3299
Added ProxySetHeaders for ingress annotations
AlexFenlon Mar 19, 2024
8d27bd8
Added ProxySetHeaders for ingress annotations
AlexFenlon Mar 19, 2024
80cda82
Adds ProxySetHeaders for ingress annotations tests
AlexFenlon Mar 19, 2024
b830519
Merge branch 'main' into feat/proxy-set-header-annotation
AlexFenlon Mar 20, 2024
57655eb
change ProxySetHeaders to []Headers
AlexFenlon Mar 20, 2024
8d08b0b
Add ProxySetHeader splits, template loops and test
AlexFenlon Mar 20, 2024
3e6d8a2
Update ProxySetHeader template test
AlexFenlon Mar 20, 2024
a2cffbb
Clean up code & Add ProxySetHeader for Plus
AlexFenlon Mar 21, 2024
90bf2fa
Add value to ProxySetHeader annotation + template
AlexFenlon Mar 21, 2024
811e992
Adjust Spacing
AlexFenlon Mar 21, 2024
16f75a2
Allow custom header value or default
AlexFenlon Mar 21, 2024
841906b
Format value for ProxySetHeader
AlexFenlon Mar 21, 2024
dc0fd20
Update value for proxySetHeader + Update test
AlexFenlon Mar 21, 2024
0f20627
Add proxySetHeader template test
AlexFenlon Mar 21, 2024
63c8d86
Add proxySetHeader template test for two headers
AlexFenlon Mar 21, 2024
adc50be
Merge remote-tracking branch 'origin/main' into feat/proxy-set-header…
AlexFenlon Mar 21, 2024
033ee7e
Convert template function to go helper function
AlexFenlon Mar 21, 2024
bcb24f9
Filter headers for ProxySetHeader annotation
AlexFenlon Mar 22, 2024
eeea522
Add comment for validateProxySetHeader
AlexFenlon Mar 22, 2024
5d65bb5
Fix for spacing in template & Add two headers to test
AlexFenlon Mar 22, 2024
81a7d22
Remove spacing to clean up template
AlexFenlon Mar 22, 2024
3eb0be0
Add better header value splitting
AlexFenlon Mar 22, 2024
afb9156
Add better header value handling with override
AlexFenlon Mar 22, 2024
ebc352e
Add better header value validation with minion/master
AlexFenlon Mar 22, 2024
70280ad
Merge remote-tracking branch 'origin/main' into feat/proxy-set-header…
AlexFenlon Mar 25, 2024
e2f2ad1
Revert validation to working state
AlexFenlon Mar 25, 2024
ce626be
Remove non-functioning override
AlexFenlon Mar 25, 2024
3b4a4dd
Add template tests for NGINX and Plus for proxySetHeaderannotation
AlexFenlon Mar 25, 2024
b2ce554
Refactor tests to table tests
AlexFenlon Mar 25, 2024
9db42c4
Refactor tests to table tests with multiple line checks
AlexFenlon Mar 25, 2024
290816c
Add table test for ValidateProxySetHeader & GenerateProxySetHeader
AlexFenlon Mar 25, 2024
967d870
Merge remote-tracking branch 'origin/main' into feat/proxy-set-header…
AlexFenlon Mar 25, 2024
a8d324d
Fix template helper test
AlexFenlon Mar 25, 2024
bc5481c
Refactor ProxySetHeaders annotation and split to parsing helper
AlexFenlon Mar 25, 2024
aff39b5
Add Tests for invalid annotation & Error catch spaces in value
AlexFenlon Mar 25, 2024
1baaabc
Add Tests for Blank Header and Value & Update Validation
AlexFenlon Mar 26, 2024
f7ec268
Refactor parseProxySetHeaders
AlexFenlon Mar 26, 2024
bcae09e
Revert Change due to bug
AlexFenlon Mar 26, 2024
2881dd4
Add invalid tests & refactor generateProxySetHeader
AlexFenlon Mar 26, 2024
9f75fa2
Fix out of range with catch
AlexFenlon Mar 27, 2024
4d38167
Merge remote-tracking branch 'origin/main' into feat/proxy-set-header…
AlexFenlon Mar 27, 2024
c2e27c7
Add test for invalid header
AlexFenlon Mar 27, 2024
5bb9d03
Move ProxySetHeader to Minion ingress
AlexFenlon Mar 27, 2024
ab0dae8
Refactor test name
AlexFenlon Mar 27, 2024
606e148
Add Functionality for minion and master set by user.
AlexFenlon Mar 28, 2024
53392d5
Add master annotation to minion without annotation
AlexFenlon Mar 29, 2024
c842083
Add Minion override Master
AlexFenlon Mar 29, 2024
3548036
Revert "Add Minion override Master"
AlexFenlon Mar 29, 2024
29a9093
Add Minion override Master
AlexFenlon Mar 29, 2024
79856ab
Add tests for Master & Fix appropriate function
AlexFenlon Apr 2, 2024
c6412dd
Add tests for Minions and Master for ProxySetHeaderAnnotation
AlexFenlon Apr 2, 2024
8761fd2
Add ProxySetHeaderAnnotation to Plus
AlexFenlon Apr 2, 2024
d6e6b30
Merge remote-tracking branch 'origin/main' into feat/proxy-set-header…
AlexFenlon Apr 2, 2024
c4c991e
Add Test for ProxySetHeaderAnnotation Plus
AlexFenlon Apr 2, 2024
3042f60
Add functions to refactor generateProxySetHeaders
AlexFenlon Apr 3, 2024
51f56ad
Add functions to refactor generateProxySetHeaders
AlexFenlon Apr 3, 2024
312cd29
Merge remote-tracking branch 'origin/main' into feat/proxy-set-header…
AlexFenlon Apr 3, 2024
9bb379d
Refactor generateProxySetHeaders
AlexFenlon Apr 3, 2024
40a0ff4
Major refactor for generateProxySetHeaders
AlexFenlon Apr 3, 2024
3392718
Clean up code
AlexFenlon Apr 4, 2024
d5500a2
Refactor generateProxySetHeaders & helpers
AlexFenlon Apr 4, 2024
329d115
Fix error return
AlexFenlon Apr 4, 2024
8e74e0c
Convert functions to use string instead of strings.Builder
AlexFenlon Apr 4, 2024
e379cb3
Merge remote-tracking branch 'origin/main' into feat/proxy-set-header…
AlexFenlon Apr 4, 2024
9706887
Refactor proxySetHeader functions
AlexFenlon Apr 4, 2024
8e3bb00
Add Comments to proxy set header functions
AlexFenlon Apr 4, 2024
67bcaa5
Add Tests For generateProxySetHeaders
AlexFenlon Apr 5, 2024
b0aa430
Merge remote-tracking branch 'origin/main' into feat/proxy-set-header…
AlexFenlon Apr 5, 2024
71a3287
Add More Tests For generateProxySetHeaders
AlexFenlon Apr 5, 2024
bf93154
Add More Tests For generateProxySetHeaders
AlexFenlon Apr 5, 2024
0c473cf
Add proxy_set_header annotation to pytest
AlexFenlon Apr 9, 2024
ea14131
Merge remote-tracking branch 'origin/main' into feat/proxy-set-header…
AlexFenlon Apr 9, 2024
e06aef6
Add final pytests for proxy-set-headers annotation
AlexFenlon Apr 9, 2024
e9a9620
Merge remote-tracking branch 'origin/main' into feat/proxy-set-header…
AlexFenlon Apr 9, 2024
1ab5bdd
removed test marker
AlexFenlon Apr 9, 2024
ef46b24
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 9, 2024
382dc79
Fix indentation & comment
AlexFenlon Apr 9, 2024
38b3fd9
Merge remote-tracking branch 'origin/feat/proxy-set-header-annotation…
AlexFenlon Apr 9, 2024
8958a95
Empty commit
AlexFenlon Apr 9, 2024
8d3a281
pipeline fix
AlexFenlon Apr 10, 2024
c4d4eea
Fix annotation not working in minions only & Fix tests and added more
AlexFenlon Apr 10, 2024
d923d93
Merge remote-tracking branch 'origin/main' into feat/proxy-set-header…
AlexFenlon Apr 11, 2024
5b77f19
Add pytest for standard flow for proxysetheader annotation
AlexFenlon Apr 11, 2024
1400d18
Merge branch 'main' into feat/proxy-set-header-annotation
AlexFenlon Apr 11, 2024
0a8df87
Update comments and error message
AlexFenlon Apr 11, 2024
68473bf
Refactor functions
AlexFenlon Apr 11, 2024
9c77452
Merge branch 'main' into feat/proxy-set-header-annotation
AlexFenlon Apr 11, 2024
3a4e7a4
Merge branch 'main' into feat/proxy-set-header-annotation
AlexFenlon Apr 12, 2024
f91f145
Update annotations_test.go
AlexFenlon Apr 12, 2024
d10058b
Merge branch 'main' into feat/proxy-set-header-annotation
AlexFenlon Apr 12, 2024
6dd339c
pre-commit
AlexFenlon Apr 12, 2024
c8b5ebb
Merge remote-tracking branch 'origin/feat/proxy-set-header-annotation…
AlexFenlon Apr 12, 2024
ca44211
remove fmt import as no longer being used
AlexFenlon Apr 12, 2024
5677ca2
Merge remote-tracking branch 'origin/main' into feat/proxy-set-header…
AlexFenlon Apr 12, 2024
5b79833
Update to Regex used to be consistent with other http header annotations
AlexFenlon Apr 12, 2024
4209dff
Update to annotation format to use Header: value instead & remove/add…
AlexFenlon Apr 15, 2024
7e52eb0
Merge branch 'main' into feat/proxy-set-header-annotation
AlexFenlon Apr 15, 2024
8f05161
Merge branch 'main' into feat/proxy-set-header-annotation
AlexFenlon Apr 15, 2024
145d421
Merge branch 'main' into feat/proxy-set-header-annotation
AlexFenlon Apr 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions internal/configs/annotations.go
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,11 @@ func parseAnnotations(ingEx *IngressEx, baseCfgParams *ConfigParams, isPlus bool
cfgParams.ProxyPassHeaders = proxyPassHeaders
}

if proxySetHeaders, exists := GetMapKeyAsStringSlice(ingEx.Ingress.Annotations, "nginx.org/proxy-set-headers", ingEx.Ingress, ","); exists {
parsedHeaders := parseProxySetHeaders(proxySetHeaders)
cfgParams.ProxySetHeaders = parsedHeaders
}

if clientMaxBodySize, exists := ingEx.Ingress.Annotations["nginx.org/client-max-body-size"]; exists {
cfgParams.ClientMaxBodySize = clientMaxBodySize
}
Expand Down
2 changes: 2 additions & 0 deletions internal/configs/config_params.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package configs

import (
"github.com/nginxinc/kubernetes-ingress/internal/configs/version2"
"github.com/nginxinc/kubernetes-ingress/internal/nginx"
conf_v1 "github.com/nginxinc/kubernetes-ingress/pkg/apis/configuration/v1"
)
Expand Down Expand Up @@ -70,6 +71,7 @@ type ConfigParams struct {
ProxyHideHeaders []string
ProxyMaxTempFileSize string
ProxyPassHeaders []string
ProxySetHeaders []version2.Header
ProxyProtocol bool
ProxyReadTimeout string
ProxySendTimeout string
Expand Down
16 changes: 16 additions & 0 deletions internal/configs/configmaps.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ package configs
import (
"strings"

"github.com/nginxinc/kubernetes-ingress/internal/configs/version2"

"github.com/golang/glog"
v1 "k8s.io/api/core/v1"

Expand Down Expand Up @@ -66,6 +68,20 @@ func ParseConfigMap(cfgm *v1.ConfigMap, nginxPlus bool, hasAppProtect bool, hasA
cfgParams.ProxyPassHeaders = proxyPassHeaders
}

if proxySetHeaders, exists := GetMapKeyAsStringSlice(cfgm.Data, "proxy-set-headers", cfgm, ","); exists {
var headers []version2.Header
for _, headerAndValue := range proxySetHeaders {
parts := strings.SplitN(headerAndValue, " ", 2)
name := strings.TrimSpace(parts[0])
var value string
if len(parts) > 1 {
value = strings.TrimSpace(parts[1])
}
headers = append(headers, version2.Header{Name: name, Value: value})
}
cfgParams.ProxySetHeaders = headers
}

if clientMaxBodySize, exists := cfgm.Data["client-max-body-size"]; exists {
cfgParams.ClientMaxBodySize = clientMaxBodySize
}
Expand Down
1 change: 1 addition & 0 deletions internal/configs/ingress.go
Original file line number Diff line number Diff line change
Expand Up @@ -475,6 +475,7 @@ func createLocation(path string, upstream version1.Upstream, cfg *ConfigParams,
ProxyConnectTimeout: cfg.ProxyConnectTimeout,
ProxyReadTimeout: cfg.ProxyReadTimeout,
ProxySendTimeout: cfg.ProxySendTimeout,
ProxySetHeaders: cfg.ProxySetHeaders,
ClientMaxBodySize: cfg.ClientMaxBodySize,
Websocket: websocket,
Rewrite: rewrite,
Expand Down
16 changes: 16 additions & 0 deletions internal/configs/parsing_helpers.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ import (
"strconv"
"strings"

"github.com/nginxinc/kubernetes-ingress/internal/configs/version2"

v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
)
Expand Down Expand Up @@ -276,6 +278,20 @@ func ParseProxyBuffersSpec(s string) (string, error) {
return "", errors.New("invalid proxy buffers string")
}

// parseProxySetHeaders ensures that the string colon-separated list of headers and values
func parseProxySetHeaders(proxySetHeaders []string) []version2.Header {
var headers []version2.Header
for _, header := range proxySetHeaders {
parts := strings.SplitN(header, ":", 2)
if len(parts) == 1 {
headers = append(headers, version2.Header{Name: parts[0], Value: ""})
} else {
headers = append(headers, version2.Header{Name: parts[0], Value: parts[1]})
}
}
return headers
}

// ParsePortList ensures that the string is a comma-separated list of port numbers
func ParsePortList(s string) ([]int, error) {
var ports []int
Expand Down
6 changes: 5 additions & 1 deletion internal/configs/version1/config.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
package version1

import "github.com/nginxinc/kubernetes-ingress/internal/nginx"
import (
"github.com/nginxinc/kubernetes-ingress/internal/configs/version2"
"github.com/nginxinc/kubernetes-ingress/internal/nginx"
)

// UpstreamLabels describes the Prometheus labels for an NGINX upstream.
type UpstreamLabels struct {
Expand Down Expand Up @@ -166,6 +169,7 @@ type Location struct {
ProxyConnectTimeout string
ProxyReadTimeout string
ProxySendTimeout string
ProxySetHeaders []version2.Header
ClientMaxBodySize string
Websocket bool
Rewrite string
Expand Down
6 changes: 4 additions & 2 deletions internal/configs/version1/nginx-plus.ingress.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -260,9 +260,9 @@ server {
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
{{- else}}
{{- if $.Keepalive}}proxy_set_header Connection "";{{end}}
{{- if $.Keepalive}}
AlexFenlon marked this conversation as resolved.
Show resolved Hide resolved
proxy_set_header Connection "";{{end}}
{{- end}}

{{- if $location.LocationSnippets}}
{{range $value := $location.LocationSnippets}}
{{$value}}{{end}}
Expand All @@ -285,6 +285,8 @@ server {
proxy_read_timeout {{$location.ProxyReadTimeout}};
proxy_send_timeout {{$location.ProxySendTimeout}};
client_max_body_size {{$location.ClientMaxBodySize}};
{{- $proxySetHeaders := generateProxySetHeaders $location $.Ingress.Annotations }}
{{$proxySetHeaders}}
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
Expand Down
8 changes: 4 additions & 4 deletions internal/configs/version1/nginx.ingress.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -180,23 +180,23 @@ server {
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
{{- else}}
{{- if $.Keepalive}}proxy_set_header Connection "";{{end}}
{{- if $.Keepalive}}
proxy_set_header Connection "";{{end}}
{{- end}}

{{- if $location.LocationSnippets}}
{{range $value := $location.LocationSnippets}}
{{$value}}{{end}}
{{- end}}

{{- with $location.BasicAuth }}
auth_basic {{ printf "%q" .Realm }};
auth_basic_user_file {{ .Secret }};
{{- end }}

proxy_connect_timeout {{$location.ProxyConnectTimeout}};
proxy_read_timeout {{$location.ProxyReadTimeout}};
proxy_send_timeout {{$location.ProxySendTimeout}};
client_max_body_size {{$location.ClientMaxBodySize}};
{{- $proxySetHeaders := generateProxySetHeaders $location $.Ingress.Annotations -}}
{{$proxySetHeaders}}
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
Expand Down
137 changes: 128 additions & 9 deletions internal/configs/version1/template_helper.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
package version1

import (
"errors"
"fmt"
"regexp"
"strings"
"text/template"

Expand Down Expand Up @@ -64,14 +66,131 @@ func makePathWithRegex(path, regexType string) string {
}
}

var setHeader = regexp.MustCompile("^[-A-Za-z0-9]+$")

func validateProxySetHeader(header string) error {
header = strings.TrimSpace(header)
if !setHeader.MatchString(header) {
return errors.New("invalid header syntax - syntax must be header: value")
}
return nil
}

func defaultHeaderValues(headerParts []string, headerName string) string {
headerValue := strings.TrimSpace(headerParts[0])
headerValue = strings.ReplaceAll(headerValue, "-", "_")
headerValue = strings.ToLower(headerValue)
return fmt.Sprintf("\n\t\tproxy_set_header %s $http_%s;", headerName, headerValue)
}

func headersGreaterThanOne(headerParts []string, headerName string) string {
headerValue := strings.TrimSpace(headerParts[1])
return fmt.Sprintf("\n\t\tproxy_set_header %s %q;", headerName, headerValue)
}

func splitHeaders(header string) ([]string, string) {
header = strings.TrimSpace(header)
headerParts := strings.SplitN(header, ":", 2)
headerName := strings.TrimSpace(headerParts[0])
return headerParts, headerName
}

// minionProxySetHeaders takes a location and a bool map
// and generates proxy_set_header headers for minions based on the nginx.org/proxy-set-headers annotation in a mergeable ingress.
// It returns a string containing the generated proxy headers and a map to verify that the header exists
func minionProxySetHeaders(loc *Location, minionHeaders map[string]bool) (string, map[string]bool, error) {
proxySetHeaders, ok := loc.MinionIngress.Annotations["nginx.org/proxy-set-headers"]
if !ok {
return "", nil, nil
}
var combinedMinions string
headers := strings.Split(proxySetHeaders, ",")
for _, header := range headers {
AlexFenlon marked this conversation as resolved.
Show resolved Hide resolved
headerParts, headerName := splitHeaders(header)
err := validateProxySetHeader(headerName)
if err != nil {
return "", nil, err
}
if len(headerParts) > 1 {
output := headersGreaterThanOne(headerParts, headerName)
minionHeaders[headerName] = true
combinedMinions += output
} else {
output := defaultHeaderValues(headerParts, headerName)
combinedMinions += output
}
}
return combinedMinions, minionHeaders, nil
}

// standardIngressOrMasterProxySetHeaders takes two strings, two bools and a bool map
// and generates proxy_set_header headers based on the nginx.org/proxy-set-headers annotation in a standard ingress.
// It returns a string containing the generated proxy headers for either standardIngress/NonMergeable or Master.
func standardIngressOrMasterProxySetHeaders(result string, minionHeaders map[string]bool, proxySetHeaders string, ok bool, isMergeable bool) (string, error) {
j1m-ryan marked this conversation as resolved.
Show resolved Hide resolved
if !ok {
AlexFenlon marked this conversation as resolved.
Show resolved Hide resolved
return "", nil
}
headers := strings.Split(proxySetHeaders, ",")
for _, header := range headers {
headerParts, headerName := splitHeaders(header)
if isMergeable {
if _, ok := minionHeaders[headerName]; ok {
continue
}
}
if err := validateProxySetHeader(headerName); err != nil {
return "", err
}
if len(headerParts) > 1 {
output := headersGreaterThanOne(headerParts, headerName)
result += output
} else {
output := defaultHeaderValues(headerParts, headerName)
result += output
}
}
return result, nil
}

// generateProxySetHeaders takes a location and an ingress annotations map
// and generates proxy_set_header directives based on the nginx.org/proxy-set-headers annotation.
// It returns a string containing the generated Nginx configuration to the template.
func generateProxySetHeaders(loc *Location, ingressAnnotations map[string]string) (string, error) {
proxySetHeaders, ok := ingressAnnotations["nginx.org/proxy-set-headers"]
var ingressResult string
minionHeaders := make(map[string]bool)
isMergeable := loc.MinionIngress != nil
if !isMergeable {
ingressResult, err := standardIngressOrMasterProxySetHeaders(ingressResult, minionHeaders, proxySetHeaders, ok, isMergeable)
if err != nil {
return "", err
}
return ingressResult, nil
}
combinedHeaders, minionHeaders, err := minionProxySetHeaders(loc, minionHeaders)
if err != nil {
return "", err
}
proxySetHeaders, ok = ingressAnnotations["nginx.org/proxy-set-headers"]
if !ok {
return combinedHeaders, nil
}
combinedHeaders, err = standardIngressOrMasterProxySetHeaders(combinedHeaders, minionHeaders, proxySetHeaders, ok, isMergeable)
if err != nil {
return "", err
}
return combinedHeaders, nil
}

var helperFunctions = template.FuncMap{
"split": split,
"trim": trim,
"contains": strings.Contains,
"hasPrefix": strings.HasPrefix,
"hasSuffix": strings.HasSuffix,
"toLower": strings.ToLower,
"toUpper": strings.ToUpper,
"makeLocationPath": makeLocationPath,
"makeSecretPath": commonhelpers.MakeSecretPath,
"split": split,
"trim": trim,
"contains": strings.Contains,
"hasPrefix": strings.HasPrefix,
"hasSuffix": strings.HasSuffix,
"toLower": strings.ToLower,
"toUpper": strings.ToUpper,
"makeLocationPath": makeLocationPath,
"makeSecretPath": commonhelpers.MakeSecretPath,
"generateProxySetHeaders": generateProxySetHeaders,
}
Loading
Loading