Add websockets support for vs/vsr upstreams

docs/virtualserver-and-virtualserverroute.md

@@ -7,26 +7,27 @@ This document is the reference documentation for the resources. To see additiona
 **Feature Status**: The VirtualServer and VirtualServerRoute resources are available as a preview feature: it is suitable for experimenting and testing; however, it must be used with caution in production environments. Additionally, while the feature is in preview, we might introduce some backward-incompatible changes to the resources specification in the next releases.
 
 ## Contents
-- [VirtualServer and VirtualServerRoute Resources](#VirtualServer-and-VirtualServerRoute-Resources)
-  - [Contents](#Contents)
-  - [Prerequisites](#Prerequisites)
-  - [VirtualServer Specification](#VirtualServer-Specification)
-    - [VirtualServer.TLS](#VirtualServerTLS)
-    - [VirtualServer.Route](#VirtualServerRoute)
-  - [VirtualServerRoute Specification](#VirtualServerRoute-Specification)
-    - [VirtualServerRoute.Subroute](#VirtualServerRouteSubroute)
-  - [Common Parts of the VirtualServer and VirtualServerRoute](#Common-Parts-of-the-VirtualServer-and-VirtualServerRoute)
-    - [Upstream](#Upstream)
-    - [Upstream.TLS](#UpstreamTLS)
-    - [Upstream.Healthcheck](#UpstreamHealthcheck)
-    - [Header](#Header)
-    - [Split](#Split)
-    - [Rules](#Rules)
-    - [Condition](#Condition)
-    - [Match](#Match)
-  - [Using VirtualServer and VirtualServerRoute](#Using-VirtualServer-and-VirtualServerRoute)
-    - [Validation](#Validation)
-  - [Customization via ConfigMap](#Customization-via-ConfigMap)
+
+- [VirtualServer and VirtualServerRoute Resources](#virtualserver-and-virtualserverroute-resources)
+  - [Contents](#contents)
+  - [Prerequisites](#prerequisites)
+  - [VirtualServer Specification](#virtualserver-specification)
+    - [VirtualServer.TLS](#virtualservertls)
+    - [VirtualServer.Route](#virtualserverroute)
+  - [VirtualServerRoute Specification](#virtualserverroute-specification)
+    - [VirtualServerRoute.Subroute](#virtualserverroutesubroute)
+  - [Common Parts of the VirtualServer and VirtualServerRoute](#common-parts-of-the-virtualserver-and-virtualserverroute)
+    - [Upstream](#upstream)
+    - [Upstream.TLS](#upstreamtls)
+    - [Upstream.Healthcheck](#upstreamhealthcheck)
+    - [Header](#header)
+    - [Split](#split)
+    - [Rules](#rules)
+    - [Condition](#condition)
+    - [Match](#match)
+  - [Using VirtualServer and VirtualServerRoute](#using-virtualserver-and-virtualserverroute)
+    - [Validation](#validation)
+  - [Customization via ConfigMap](#customization-via-configmap)
 
 ## Prerequisites
 
@@ -194,6 +195,8 @@ tls:
   enable: True
 ```
 
+**Note**: The WebSocket protocol is supported without any additional configuration.
+
 | Field | Description | Type | Required |
 | ----- | ----------- | ---- | -------- |
 | `name` | The name of the upstream. Must be a valid DNS label as defined in RFC 1035. For example, `hello` and `upstream-123` are valid. The name must be unique among all upstreams of the resource. | `string` | Yes |
@@ -214,6 +217,7 @@ tls:
 | `healthCheck` | The health check configuration for the Upstream. See the [health_check](http://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html#health_check) directive. Note: this feature is supported only in NGINX Plus. | [`healthcheck`](#UpstreamHealthcheck) | No |
 
 ### Upstream.TLS
+
 | Field | Description | Type | Required |
 | ----- | ----------- | ---- | -------- |
 | `enable` | Enables HTTPS for requests to upstream servers. The default is `False`, meaning that HTTP will be used. | `boolean` | No |
@@ -262,6 +266,7 @@ healthCheck:
 | `statusMatch` | The expected response status codes of a health check.  By default, the response should have status code 2xx or 3xx. Examples: `“200”`, `“! 500”`, `"301-303 307"`. See the documentation of the [match](https://nginx.org/en/docs/http/ngx_http_upstream_hc_module.html?#match) directive. | `string` | No |
 
 ### Header
+
 The header defines an HTTP Header:
 ```yaml
 name: Host

internal/configs/version1/nginx-plus.tmpl

@@ -67,7 +67,10 @@ http {
         default upgrade;
         ''      close;
     }
-
+    map $http_upgrade $vs_connection_header {
+        default upgrade;
+        ''      $default_connection_header;
+    }
     {{if .SSLProtocols}}ssl_protocols {{.SSLProtocols}};{{end}}
     {{if .SSLCiphers}}ssl_ciphers "{{.SSLCiphers}}";{{end}}
     {{if .SSLPreferServerCiphers}}ssl_prefer_server_ciphers on;{{end}}
@@ -86,6 +89,9 @@ http {
     {{end}}
 
     server {
+        # required to support the Websocket protocol in VirtualServer/VirtualServerRoutes
+        set $default_connection_header "";
+
         listen 80 default_server{{if .ProxyProtocol}} proxy_protocol{{end}};
         listen 443 ssl default_server{{if .HTTP2}} http2{{end}}{{if .ProxyProtocol}} proxy_protocol{{end}};
 

internal/configs/version1/nginx.tmpl

@@ -66,6 +66,10 @@ http {
         default upgrade;
         ''      close;
     }
+    map $http_upgrade $vs_connection_header {
+        default upgrade;
+        ''      $default_connection_header;
+    }
     {{if .SSLProtocols}}ssl_protocols {{.SSLProtocols}};{{end}}
     {{if .SSLCiphers}}ssl_ciphers "{{.SSLCiphers}}";{{end}}
     {{if .SSLPreferServerCiphers}}ssl_prefer_server_ciphers on;{{end}}
@@ -79,6 +83,9 @@ http {
     {{end}}
 
     server {
+        # required to support the Websocket protocol in VirtualServer/VirtualServerRoutes
+        set $default_connection_header "";
+
         listen 80 default_server{{if .ProxyProtocol}} proxy_protocol{{end}};
         listen 443 ssl default_server{{if .HTTP2}} http2{{end}}{{if .ProxyProtocol}} proxy_protocol{{end}};
 

internal/configs/version2/nginx-plus.virtualserver.tmpl

@@ -97,7 +97,7 @@ server {
         proxy_read_timeout {{ $hc.ProxyReadTimeout }};
         proxy_send_timeout {{ $hc.ProxySendTimeout }};
         proxy_pass {{ $hc.ProxyPass }};
-        health_check uri={{ $hc.URI }} port={{ $hc.Port }} interval={{ $hc.Interval }} jitter={{ $hc.Jitter }} 
+        health_check uri={{ $hc.URI }} port={{ $hc.Port }} interval={{ $hc.Interval }} jitter={{ $hc.Jitter }}
             fails={{ $hc.Fails }} passes={{ $hc.Passes }}{{ if $hc.Match }} match={{ $hc.Match }}{{ end }};
     }
     {{ end }}
@@ -127,10 +127,9 @@ server {
 
         proxy_http_version 1.1;
 
-        {{ if $l.HasKeepalive }}
-        proxy_set_header Connection "";
-        {{ end }}
-
+        set $default_connection_header {{ if $l.HasKeepalive }}""{{ else }}close{{ end }};
+        proxy_set_header Upgrade $http_upgrade;
+        proxy_set_header Connection $vs_connection_header;
         proxy_set_header Host $host;
         proxy_set_header X-Real-IP $remote_addr;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
@@ -144,4 +143,4 @@ server {
         proxy_next_upstream_tries {{ $l.ProxyNextUpstreamTries }};
     }
     {{ end }}
-}
+}

internal/configs/version2/nginx.virtualserver.tmpl

@@ -107,10 +107,9 @@ server {
 
         proxy_http_version 1.1;
 
-        {{ if $l.HasKeepalive }}
-        proxy_set_header Connection "";
-        {{ end }}
-
+        set $default_connection_header {{ if $l.HasKeepalive }}""{{ else }}close{{ end }};
+        proxy_set_header Upgrade $http_upgrade;
+        proxy_set_header Connection $vs_connection_header;
         proxy_set_header Host $host;
         proxy_set_header X-Real-IP $remote_addr;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
@@ -124,4 +123,4 @@ server {
         proxy_next_upstream_tries {{ $l.ProxyNextUpstreamTries }};
     }
     {{ end }}
-}
+}

tests/suite/test_v_s_route_upstream_options.py

@@ -70,7 +70,7 @@ def test_nginx_config_upstreams_defaults(self, kube_apis, ingress_controller_pre
         assert "max_fails=1 fail_timeout=10s max_conns=0;" in config
 
         assert "keepalive" not in config
-        assert 'proxy_set_header Connection "";' not in config
+        assert 'set $default_connection_header "";' not in config
 
         assert "proxy_next_upstream error timeout;" in config
         assert "proxy_next_upstream_timeout 0s;" in config
@@ -82,7 +82,7 @@ def test_nginx_config_upstreams_defaults(self, kube_apis, ingress_controller_pre
           "keepalive": 54, "max-conns": 1024},
          ["least_conn;", "max_fails=8 ",
           "fail_timeout=13s ", "proxy_connect_timeout 55s;", "proxy_read_timeout 1s;",
-          "proxy_send_timeout 1h;", "keepalive 54;", 'proxy_set_header Connection "";', "max_conns=1024;"]),
+          "proxy_send_timeout 1h;", "keepalive 54;", 'set $default_connection_header "";', "max_conns=1024;"]),
         ({"lb-method": "ip_hash", "connect-timeout": "75", "read-timeout": "15", "send-timeout": "1h"},
          ["ip_hash;", "proxy_connect_timeout 75;", "proxy_read_timeout 15;", "proxy_send_timeout 1h;"]),
         ({"connect-timeout": "1m", "read-timeout": "1m", "send-timeout": "1s"},
@@ -140,7 +140,7 @@ def test_when_option_in_v_s_r_only(self, kube_apis,
         (f"{TEST_DATA}/virtual-server-route-upstream-options/configmap-with-keys.yaml",
          ["max_fails=3 ", "fail_timeout=33s ", "max_conns=0;",
           "proxy_connect_timeout 44s;", "proxy_read_timeout 22s;", "proxy_send_timeout 55s;",
-          "keepalive 1024;", 'proxy_set_header Connection "";'],
+          "keepalive 1024;", 'set $default_connection_header "";'],
          ["ip_hash;", "least_conn;", "random ", "hash", "least_time ",
           "max_fails=1 ", "fail_timeout=10s ", "max_conns=1000;",
           "proxy_connect_timeout 60s;", "proxy_read_timeout 60s;", "proxy_send_timeout 60s;"]),
@@ -194,7 +194,7 @@ def test_when_option_in_config_map_only(self, kube_apis,
           "keepalive": 48},
          ["least_conn;", "max_fails=12 ",
           "fail_timeout=1m ", "max_conns=0;", "proxy_connect_timeout 1m;", "proxy_read_timeout 77s;", "proxy_send_timeout 23s;",
-          "keepalive 48;", 'proxy_set_header Connection "";'],
+          "keepalive 48;", 'set $default_connection_header "";'],
          ["ip_hash;", "random ", "hash", "least_time ", "max_fails=1 ",
           "fail_timeout=10s ", "proxy_connect_timeout 44s;", "proxy_read_timeout 22s;", "proxy_send_timeout 55s;",
           "keepalive 1024;"])

tests/suite/test_virtual_server_upstream_options.py

@@ -74,7 +74,7 @@ def test_nginx_config_defaults(self, kube_apis, ingress_controller_prerequisites
         assert "max_fails=1 fail_timeout=10s max_conns=0;" in config
 
         assert "keepalive" not in config
-        assert 'proxy_set_header Connection "";' not in config
+        assert 'set $default_connection_header "";' not in config
 
         assert "proxy_next_upstream error timeout;" in config
         assert "proxy_next_upstream_timeout 0s;" in config
@@ -86,7 +86,7 @@ def test_nginx_config_defaults(self, kube_apis, ingress_controller_prerequisites
           "keepalive": 54, "max-conns": 1048},
          ["least_conn;", "max_fails=8 ",
           "fail_timeout=13s ", "proxy_connect_timeout 55s;", "proxy_read_timeout 1s;",
-          "proxy_send_timeout 1h;", "keepalive 54;", 'proxy_set_header Connection "";', "max_conns=1048;"]),
+          "proxy_send_timeout 1h;", "keepalive 54;", 'set $default_connection_header "";', "max_conns=1048;"]),
         ({"lb-method": "ip_hash", "connect-timeout": "75", "read-timeout": "15", "send-timeout": "1h"},
          ["ip_hash;", "proxy_connect_timeout 75;", "proxy_read_timeout 15;", "proxy_send_timeout 1h;"]),
         ({"connect-timeout": "1m", "read-timeout": "1m", "send-timeout": "1s"},
@@ -130,7 +130,7 @@ def test_when_option_in_v_s_only(self, kube_apis, ingress_controller_prerequisit
         (f"{TEST_DATA}/virtual-server-upstream-options/configmap-with-keys.yaml",
          ["max_fails=3 ", "fail_timeout=33s ", "max_conns=0;",
           "proxy_connect_timeout 44s;", "proxy_read_timeout 22s;", "proxy_send_timeout 55s;",
-          "keepalive 1024;", 'proxy_set_header Connection "";'],
+          "keepalive 1024;", 'set $default_connection_header "";'],
          ["ip_hash;", "least_conn;", "random ", "hash", "least_time ",
           "max_fails=1 ", "fail_timeout=10s ", "max_conns=1000;",
           "proxy_connect_timeout 60s;", "proxy_read_timeout 60s;", "proxy_send_timeout 60s;"]),
@@ -174,7 +174,7 @@ def test_when_option_in_config_map_only(self, kube_apis, ingress_controller_prer
           "keepalive": 48},
          ["least_conn;", "max_fails=12 ",
           "fail_timeout=1m ", "max_conns=0;", "proxy_connect_timeout 1m;", "proxy_read_timeout 77s;",
-          "proxy_send_timeout 23s;", "keepalive 48;", 'proxy_set_header Connection "";'],
+          "proxy_send_timeout 23s;", "keepalive 48;", 'set $default_connection_header "";'],
          ["ip_hash;", "random ", "hash", "least_time ", "max_fails=1 ",
           "fail_timeout=10s ", "proxy_connect_timeout 44s;", "proxy_read_timeout 22s;",
           "proxy_send_timeout 55s;", "keepalive 1024;"])