Skip to content

Commit

Permalink
feat: auth-req caching
Browse files Browse the repository at this point in the history 
add a way to configure the `proxy_cache_*` [1] directive for external-auth.
The user-defined cache_key may contain sensitive information
(e.g. Authorization header).
We want to store *only* a hash of that key, not the key itself on disk.

[1] http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_key

Signed-off-by: Moritz Johner <[email protected]>
  • Loading branch information
@moolen
moolen committed Jul 17, 2019
1 parent e0e7b57 commit 23504db
Show file tree
Hide file tree
Showing 13 changed files with 583 additions and 52 deletions.
38 changes: 22 additions & 16 deletions docs/user-guide/nginx-configuration/annotations.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ You can add these Kubernetes annotations to specific Ingress objects to customiz
|[nginx.ingress.kubernetes.io/auth-tls-error-page](#client-certificate-authentication)|string|
|[nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream](#client-certificate-authentication)|"true" or "false"|
|[nginx.ingress.kubernetes.io/auth-url](#external-authentication)|string|
|[nginx.ingress.kubernetes.io/auth-cache-key](#external-authentication)|string|
|[nginx.ingress.kubernetes.io/auth-cache-duration](#external-authentication)|string|
|[nginx.ingress.kubernetes.io/auth-snippet](#external-authentication)|string|
|[nginx.ingress.kubernetes.io/enable-global-auth](#external-authentication)|"true" or "false"|
|[nginx.ingress.kubernetes.io/backend-protocol](#backend-protocol)|string|HTTP,HTTPS,GRPC,GRPCS,AJP|
Expand Down Expand Up @@ -113,18 +115,18 @@ In some cases, you may want to "canary" a new set of changes by sending a small

* `nginx.ingress.kubernetes.io/canary-by-header-value`: The header value to match for notifying the Ingress to route the request to the service specified in the Canary Ingress. When the request header is set to this value, it will be routed to the canary. For any other header value, the header will be ignored and the request compared against the other canary rules by precedence. This annotation has to be used together with . The annotation is an extension of the `nginx.ingress.kubernetes.io/canary-by-header` to allow customizing the header value instead of using hardcoded values. It doesn't have any effect if the `nginx.ingress.kubernetes.io/canary-by-header` annotation is not defined.

* `nginx.ingress.kubernetes.io/canary-by-cookie`: The cookie to use for notifying the Ingress to route the request to the service specified in the Canary Ingress. When the cookie value is set to `always`, it will be routed to the canary. When the cookie is set to `never`, it will never be routed to the canary. For any other value, the cookie will be ignored and the request compared against the other canary rules by precedence.
* `nginx.ingress.kubernetes.io/canary-by-cookie`: The cookie to use for notifying the Ingress to route the request to the service specified in the Canary Ingress. When the cookie value is set to `always`, it will be routed to the canary. When the cookie is set to `never`, it will never be routed to the canary. For any other value, the cookie will be ignored and the request compared against the other canary rules by precedence.

* `nginx.ingress.kubernetes.io/canary-weight`: The integer based (0 - 100) percent of random requests that should be routed to the service specified in the canary Ingress. A weight of 0 implies that no requests will be sent to the service in the Canary ingress by this canary rule. A weight of 100 means implies all requests will be sent to the alternative service specified in the Ingress.
* `nginx.ingress.kubernetes.io/canary-weight`: The integer based (0 - 100) percent of random requests that should be routed to the service specified in the canary Ingress. A weight of 0 implies that no requests will be sent to the service in the Canary ingress by this canary rule. A weight of 100 means implies all requests will be sent to the alternative service specified in the Ingress.

Canary rules are evaluated in order of precedence. Precedence is as follows:
`canary-by-header -> canary-by-cookie -> canary-weight`
Canary rules are evaluated in order of precedence. Precedence is as follows:
`canary-by-header -> canary-by-cookie -> canary-weight`

**Note** that when you mark an ingress as canary, then all the other non-canary annotations will be ignored (inherited from the corresponding main ingress) except `nginx.ingress.kubernetes.io/load-balance` and `nginx.ingress.kubernetes.io/upstream-hash-by`.

**Known Limitations**

Currently a maximum of one canary ingress can be applied per Ingress rule.
Currently a maximum of one canary ingress can be applied per Ingress rule.

### Rewrite

Expand All @@ -142,7 +144,7 @@ The annotation `nginx.ingress.kubernetes.io/affinity` enables and sets the affin
The only affinity type available for NGINX is `cookie`.

!!! attention
If more than one Ingress is defined for a host and at least one Ingress uses `nginx.ingress.kubernetes.io/affinity: cookie`, then only paths on the Ingress using `nginx.ingress.kubernetes.io/affinity` will use session cookie affinity. All paths defined on other Ingresses for the host will be load balanced through the random selection of a backend server.
If more than one Ingress is defined for a host and at least one Ingress uses `nginx.ingress.kubernetes.io/affinity: cookie`, then only paths on the Ingress using `nginx.ingress.kubernetes.io/affinity` will use session cookie affinity. All paths defined on other Ingresses for the host will be load balanced through the random selection of a backend server.

!!! example
Please check the [affinity](../../examples/affinity/cookie/README.md) example.
Expand All @@ -151,7 +153,7 @@ The only affinity type available for NGINX is `cookie`.

If you use the ``cookie`` affinity type you can also specify the name of the cookie that will be used to route the requests with the annotation `nginx.ingress.kubernetes.io/session-cookie-name`. The default is to create a cookie named 'INGRESSCOOKIE'.

The NGINX annotation `nginx.ingress.kubernetes.io/session-cookie-path` defines the path that will be set on the cookie. This is optional unless the annotation `nginx.ingress.kubernetes.io/use-regex` is set to true; Session cookie paths do not support regex.
The NGINX annotation `nginx.ingress.kubernetes.io/session-cookie-path` defines the path that will be set on the cookie. This is optional unless the annotation `nginx.ingress.kubernetes.io/use-regex` is set to true; Session cookie paths do not support regex.


### Authentication
Expand Down Expand Up @@ -294,7 +296,7 @@ CORS can be controlled with the following annotations:
Example: `nginx.ingress.kubernetes.io/cors-max-age: 600`

!!! note
For more information please see [https://enable-cors.org](https://enable-cors.org/server_nginx.html)
For more information please see [https://enable-cors.org](https://enable-cors.org/server_nginx.html)

### HTTP2 Push Preload.

Expand Down Expand Up @@ -327,11 +329,11 @@ metadata:
annotations:
nginx.ingress.kubernetes.io/server-snippet: |
set $agentflag 0;
if ($http_user_agent ~* "(Mobile)" ){
set $agentflag 1;
}
if ( $agentflag = 1 ) {
return 301 https://m.example.com;
}
Expand Down Expand Up @@ -378,6 +380,10 @@ Additionally it is possible to set:
`<Response_Header_1, ..., Response_Header_n>` to specify headers to pass to backend once authentication request completes.
* `nginx.ingress.kubernetes.io/auth-request-redirect`:
`<Request_Redirect_URL>` to specify the X-Auth-Request-Redirect header value.
* `nginx.ingress.kubernetes.io/auth-cache-key`:
`<Cache_Key>` this enables caching for auth requests. specify a lookup key for auth responses. e.g. `$remote_user$http_authorization`. Each server and location has it's own keyspace. Hence a cached response is only valid on a per-server and per-location basis.
* `nginx.ingress.kubernetes.io/auth-cache-duration`:
`<Cache_duration>` to specify a caching time for auth responses based on their response codes, e.g. `200 202 30m`. See [proxy_cache_valid](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_valid) for details. You may specify multiple, comma-separated values: `200 202 10m, 401 5m`. defaults to `200 202 401 5m`.
* `nginx.ingress.kubernetes.io/auth-snippet`:
`<Auth_Snippet>` to specify a custom snippet to use with external authentication, e.g.

Expand Down Expand Up @@ -681,7 +687,7 @@ of ingress locations. The ModSecurity module must first be enabled by enabling M
[ConfigMap](./configmap.md#enable-modsecurity). Note this will enable ModSecurity for all paths, and each path
must be disabled manually.

It can be enabled using the following annotation:
It can be enabled using the following annotation:
```yaml
nginx.ingress.kubernetes.io/enable-modsecurity: "true"
```
Expand All @@ -706,7 +712,7 @@ SecDebugLog /tmp/modsec_debug.log
```

Note: If you use both `enable-owasp-core-rules` and `modsecurity-snippet` annotations together, only the
`modsecurity-snippet` will take effect. If you wish to include the [OWASP Core Rule Set](https://www.modsecurity.org/CRS/Documentation/) or
`modsecurity-snippet` will take effect. If you wish to include the [OWASP Core Rule Set](https://www.modsecurity.org/CRS/Documentation/) or
[recommended configuration](https://github.com/SpiderLabs/ModSecurity/blob/v3/master/modsecurity.conf-recommended) simply use the include
statement:
```yaml
Expand All @@ -730,7 +736,7 @@ nginx.ingress.kubernetes.io/influxdb-server-name: "nginx-ingress"

For the `influxdb-host` parameter you have two options:

- Use an InfluxDB server configured with the [UDP protocol](https://docs.influxdata.com/influxdb/v1.5/supported_protocols/udp/) enabled.
- Use an InfluxDB server configured with the [UDP protocol](https://docs.influxdata.com/influxdb/v1.5/supported_protocols/udp/) enabled.
- Deploy Telegraf as a sidecar proxy to the Ingress controller configured to listen UDP with the [socket listener input](https://github.com/influxdata/telegraf/tree/release-1.6/plugins/inputs/socket_listener) and to write using
anyone of the [outputs plugins](https://github.com/influxdata/telegraf/tree/release-1.7/plugins/outputs) like InfluxDB, Apache Kafka,
Prometheus, etc.. (recommended)
Expand All @@ -754,7 +760,7 @@ nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
### Use Regex

!!! attention
When using this annotation with the NGINX annotation `nginx.ingress.kubernetes.io/affinity` of type `cookie`, `nginx.ingress.kubernetes.io/session-cookie-path` must be also set; Session cookie paths do not support regex.
When using this annotation with the NGINX annotation `nginx.ingress.kubernetes.io/affinity` of type `cookie`, `nginx.ingress.kubernetes.io/session-cookie-path` must be also set; Session cookie paths do not support regex.

Using the `nginx.ingress.kubernetes.io/use-regex` annotation will indicate whether or not the paths defined on an Ingress use regular expressions. The default value is `false`.

Expand All @@ -770,9 +776,9 @@ nginx.ingress.kubernetes.io/use-regex: "false"

When this annotation is set to `true`, the case insensitive regular expression [location modifier](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) will be enforced on ALL paths for a given host regardless of what Ingress they are defined on.

Additionally, if the [`rewrite-target` annotation](#rewrite) is used on any Ingress for a given host, then the case insensitive regular expression [location modifier](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) will be enforced on ALL paths for a given host regardless of what Ingress they are defined on.
Additionally, if the [`rewrite-target` annotation](#rewrite) is used on any Ingress for a given host, then the case insensitive regular expression [location modifier](https://nginx.org/en/docs/http/ngx_http_core_module.html#location) will be enforced on ALL paths for a given host regardless of what Ingress they are defined on.

Please read about [ingress path matching](../ingress-path-matching.md) before using this modifier.
Please read about [ingress path matching](../ingress-path-matching.md) before using this modifier.

### Satisfy

Expand Down
20 changes: 15 additions & 5 deletions docs/user-guide/nginx-configuration/configmap.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,8 @@ The following table shows a configuration option's name, type, and the default v
|[global-auth-response-headers](#global-auth-response-headers)|string|""|
|[global-auth-request-redirect](#global-auth-request-redirect)|string|""|
|[global-auth-snippet](#global-auth-snippet)|string|""|
|[global-auth-cache-key](#global-auth-cache-key)|string|""|
|[global-auth-cache-duration](#global-auth-cache-duration)|string|"200 202 401 5m"|
|[no-auth-locations](#no-auth-locations)|string|"/.well-known/acme-challenge"|
|[block-cidrs](#block-cidrs)|[]string|""|
|[block-user-agents](#block-user-agents)|[]string|""|
Expand Down Expand Up @@ -196,7 +198,7 @@ __Note:__ the file `/var/log/nginx/access.log` is a symlink to `/dev/stdout`

## enable-access-log-for-default-backend

Enables logging access to default backend. _**default:**_ is disabled.
Enables logging access to default backend. _**default:**_ is disabled.

## error-log-path

Expand Down Expand Up @@ -439,7 +441,7 @@ _References:_
Instructs NGINX to create an individual listening socket for each worker process (using the SO_REUSEPORT socket option), allowing a kernel to distribute incoming connections between worker processes
_**default:**_ true

## proxy-headers-hash-bucket-size
## proxy-headers-hash-bucket-size

Sets the size of the bucket for the proxy headers hash tables.

Expand Down Expand Up @@ -503,7 +505,7 @@ Enables or disables session resumption through [TLS session tickets](http://ngin
Sets the secret key used to encrypt and decrypt TLS session tickets. The value must be a valid base64 string.
To create a ticket: `openssl rand 80 | openssl enc -A -base64`

[TLS session ticket-key](http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_tickets), by default, a randomly generated key is used.
[TLS session ticket-key](http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_session_tickets), by default, a randomly generated key is used.

## ssl-session-timeout

Expand Down Expand Up @@ -622,7 +624,7 @@ _References:_

Activates the cache for connections to upstream servers. The connections parameter sets the maximum number of idle
keepalive connections to upstream servers that are preserved in the cache of each worker process. When this number is
exceeded, the least recently used connections are closed.
exceeded, the least recently used connections are closed.
_**default:**_ 32

_References:_
Expand All @@ -643,7 +645,7 @@ _References:_
Sets the maximum number of requests that can be served through one keepalive connection. After the maximum number of
requests is made, the connection is closed.
_**default:**_ 100


_References:_
[http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_requests](http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_requests)
Expand Down Expand Up @@ -922,6 +924,14 @@ Sets a custom snippet to use with external authentication. Applied to all the lo
Similar to the Ingress rule annotation `nginx.ingress.kubernetes.io/auth-request-redirect`.
_**default:**_ ""

## global-auth-cache-key

Enables caching for global auth requests. Specify a lookup key for auth responses, e.g. `$remote_user$http_authorization`.

## global-auth-cache-duration

Set a caching time for auth responses based on their response codes, e.g. `200 202 30m`. See [proxy_cache_valid](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_valid) for details. You may specify multiple, comma-separated values: `200 202 10m, 401 5m`. defaults to `200 202 401 5m`.

## no-auth-locations

A comma-separated list of locations that should not get authenticated.
Expand Down
109 changes: 94 additions & 15 deletions internal/ingress/annotations/authreq/main.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,19 @@ import (
type Config struct {
URL string `json:"url"`
// Host contains the hostname defined in the URL
Host string `json:"host"`
SigninURL string `json:"signinUrl"`
Method string `json:"method"`
ResponseHeaders []string `json:"responseHeaders,omitempty"`
RequestRedirect string `json:"requestRedirect"`
AuthSnippet string `json:"authSnippet"`
Host string `json:"host"`
SigninURL string `json:"signinUrl"`
Method string `json:"method"`
ResponseHeaders []string `json:"responseHeaders,omitempty"`
RequestRedirect string `json:"requestRedirect"`
AuthSnippet string `json:"authSnippet"`
AuthCacheKey string `json:"authCacheKey"`
AuthCacheDuration []string `json:"authCacheDuration"`
}

// DefaultCacheDuration is the fallback value if no cache duration is provided
const DefaultCacheDuration = "200 202 401 5m"

// Equal tests for equality between two Config types
func (e1 *Config) Equal(e2 *Config) bool {
if e1 == e2 {
Expand Down Expand Up @@ -77,12 +82,23 @@ func (e1 *Config) Equal(e2 *Config) bool {
return false
}

if e1.AuthCacheKey != e2.AuthCacheKey {
return false
}

match = sets.StringElementsMatch(e1.AuthCacheDuration, e2.AuthCacheDuration)
if !match {
return false
}

return true
}

var (
methods = []string{"GET", "HEAD", "POST", "PUT", "PATCH", "DELETE", "CONNECT", "OPTIONS", "TRACE"}
headerRegexp = regexp.MustCompile(`^[a-zA-Z\d\-_]+$`)
methods = []string{"GET", "HEAD", "POST", "PUT", "PATCH", "DELETE", "CONNECT", "OPTIONS", "TRACE"}
headerRegexp = regexp.MustCompile(`^[a-zA-Z\d\-_]+$`)
statusCodeRegex = regexp.MustCompile(`^[\d]{3}$`)
durationRegex = regexp.MustCompile(`^[\d]+(ms|s|m|h|d|w|M|y)$`) // see http://nginx.org/en/docs/syntax.html
)

// ValidMethod checks is the provided string a valid HTTP method
Expand All @@ -104,6 +120,31 @@ func ValidHeader(header string) bool {
return headerRegexp.Match([]byte(header))
}

// ValidCacheDuration checks if the provided string is a valid cache duration
// spec: [code ...] [time ...];
// with: code is an http status code
// time must match the time regex and may appear multiple times, e.g. `1h 30m`
func ValidCacheDuration(duration string) bool {
elements := strings.Split(duration, " ")
seenDuration := false

for _, element := range elements {
if len(element) == 0 {
continue
}
if statusCodeRegex.Match([]byte(element)) {
if seenDuration {
return false // code after duration
}
continue
}
if durationRegex.Match([]byte(element)) {
seenDuration = true
}
}
return seenDuration
}

type authReq struct {
r resolver.Resolver
}
Expand Down Expand Up @@ -143,6 +184,17 @@ func (a authReq) Parse(ing *networking.Ingress) (interface{}, error) {
klog.V(3).Infof("auth-snippet annotation is undefined and will not be set")
}

authCacheKey, err := parser.GetStringAnnotation("auth-cache-key", ing)
if err != nil {
klog.V(3).Infof("auth-cache-key annotation is undefined and will not be set")
}

durstr, _ := parser.GetStringAnnotation("auth-cache-duration", ing)
authCacheDuration, err := ParseStringToCacheDurations(durstr)
if err != nil {
return nil, err
}

responseHeaders := []string{}
hstr, _ := parser.GetStringAnnotation("auth-response-headers", ing)
if len(hstr) != 0 {
Expand All @@ -161,13 +213,15 @@ func (a authReq) Parse(ing *networking.Ingress) (interface{}, error) {
requestRedirect, _ := parser.GetStringAnnotation("auth-request-redirect", ing)

return &Config{
URL: urlString,
Host: authURL.Hostname(),
SigninURL: signIn,
Method: authMethod,
ResponseHeaders: responseHeaders,
RequestRedirect: requestRedirect,
AuthSnippet: authSnippet,
URL: urlString,
Host: authURL.Hostname(),
SigninURL: signIn,
Method: authMethod,
ResponseHeaders: responseHeaders,
RequestRedirect: requestRedirect,
AuthSnippet: authSnippet,
AuthCacheKey: authCacheKey,
AuthCacheDuration: authCacheDuration,
}, nil
}

Expand All @@ -189,3 +243,28 @@ func ParseStringToURL(input string) (*url.URL, string) {
return parsedURL, ""

}

// ParseStringToCacheDurations parses and validates the provided string
// into a list of cache durations.
// It will always return at least one duration (the default duration)
func ParseStringToCacheDurations(input string) ([]string, error) {
authCacheDuration := []string{}
if len(input) != 0 {
arr := strings.Split(input, ",")
for _, duration := range arr {
duration = strings.TrimSpace(duration)
if len(duration) > 0 {
if !ValidCacheDuration(duration) {
authCacheDuration = []string{DefaultCacheDuration}
return authCacheDuration, ing_errors.NewLocationDenied(fmt.Sprintf("invalid cache duration: %s", duration))
}
authCacheDuration = append(authCacheDuration, duration)
}
}
}

if len(authCacheDuration) == 0 {
authCacheDuration = append(authCacheDuration, DefaultCacheDuration)
}
return authCacheDuration, nil
}
Loading

0 comments on commit 23504db

Please sign in to comment.