Enable K8 Ingress to support websockets

[manojsaxenatda]

Describe the bug

The Che UI doesn't open. Che-theia interface doesn't open for workspaces.

Che version

7.31.2

Steps to reproduce

Deploy che using helm charts from https://github.com/eclipse-che/che-server.git with the following changes to values.yaml file
cheImage: quay.io/eclipse/che-server:7.31.2
singleHostExposure: native

Expected behavior

UI should come up in a browser and a user should be able to create and launch workspaces.

Runtime

K8

Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.2", GitCommit:"f5743093fd1c663cb0cbc89748f730662345d44d", GitTreeState:"clean", BuildDate:"2020-09-16T13:41:02Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.2", GitCommit:"f5743093fd1c663cb0cbc89748f730662345d44d", GitTreeState:"clean", BuildDate:"2020-09-16T13:32:58Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}

nginx ingress controller

Image:         nginx/nginx-ingress:edge
    Image ID:      docker-pullable://nginx/nginx-ingress@sha256:2f52f6fe984f4bfb628e00286fc2f9865be2db85d77431850bb5c8e371e5eec5

Screenshots

Installation method

helm (https://github.com/eclipse-che/che-server.git)

Environment

K8 cluster with 3 linux hosts

K8

Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.2", GitCommit:"f5743093fd1c663cb0cbc89748f730662345d44d", GitTreeState:"clean", BuildDate:"2020-09-16T13:41:02Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.2", GitCommit:"f5743093fd1c663cb0cbc89748f730662345d44d", GitTreeState:"clean", BuildDate:"2020-09-16T13:32:58Z", GoVersion:"go1.15", Compiler:"gc", Platform:"linux/amd64"}

nginx ingress controller

Image:         nginx/nginx-ingress:edge
    Image ID:      docker-pullable://nginx/nginx-ingress@sha256:2f52f6fe984f4bfb628e00286fc2f9865be2db85d77431850bb5c8e371e5eec5

Eclipse Che Logs

Additional context

In k8 native strategy, eclipse che relies on nginx ingress controller to route traffic to various services based on virtual host and context path.

Che and Theia UI uses websockets.

Here is an example ingress for jwtproxy service running on port 4402 for a workspace, clearly this will not work for websockets:

---------------------------------------------------------------------------------------------------------------------------                                                                         
Name:             ingressptq9wz5w
Namespace:        che-che
Address:          10.157.173.146,10.157.175.113,10.157.175.115
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host                                                                        Path  Backends
  ----                                                                        ----  --------
  serverir1q4d2n-jwtproxy-server-4402.ec.mydomain.local  
                                                                              /   serverir1q4d2n-jwtproxy:server-4402   10.244.1.26:4402)
Annotations:                                                                  kubernetes.io/ingress.class: nginx
                                                                              nginx.ingress.kubernetes.io/proxy-connect-timeout: 3600
                                                                              nginx.ingress.kubernetes.io/proxy-read-timeout: 3600
                                                                              nginx.ingress.kubernetes.io/rewrite-target: /$1
                                                                              nginx.ingress.kubernetes.io/ssl-redirect: false
                                                                              org.eclipse.che.machine.name: theia-ideftl
                                                                              org.eclipse.che.server.theia.attributes:
                                                                                {"internal":"false","discoverable":"false","cookiesAuthEnabled":"true","secure":"true","type":"ide"}
                                                                              org.eclipse.che.server.theia.port: 3100/tcp
                                                                              org.eclipse.che.server.theia.protocol: http
---------------------------------------------------------------------------------------------------------------------------

Looking at the code, it looks like it is copying the following annotations to ingress rules CHE_INFRA_KUBERNETES_INGRESS_ANNOTATIONS__JSON and adding few more like machine.name, theia*

What would be nice is a way to add annotation
"nginx.org/websocket-services: serverir1q4d2n-jwtproxy"
via config item when k8 native ingress controller routing is in place.
The following annotation will make websockets connection possible:
nginx.org/websocket-services: serverir1q4d2n-jwtproxy

I have tested it and it works our environment.

[tolusha]

possible duplicates #19434

[sleshchenko]

I had a draft but it should be reworked in a proper generic way, where an administrator is able to use serviceName as placeholder https://github.com/eclipse-che/che-server/compare/main...sleshchenko:nginx_websocket?expand=1
I'll try to finalize it soon, but contribution is welcome.
@manojsaxenatda let me know if you want to contribute but need assistance.

[manojsaxenatda]

Hi Serhii Leshchenko,

I was looking around in the same area in code.

I was hoping to specify a config item in config map, say, CHE_INFRA_KUBERNETES_INGRESS_ANNOTATIONS_ENABLE_K8_NGINX_WEBSOCKETS_SERVICE_NAME="nginx.org/websocket-services"

Right where you pointed out in the code, if the above is set, it will add the service name value and append to annotation map.

My everyday work environment is nodejs, I will try to setup java and see if I can contribute. I will be happy to review for sure.

thank you for prompt reply

regards

[manojsaxenatda]

possible duplicates #19434

The issue you mentioned is for che-host. The issue I am describing is for workspaces. So they are different issues

[skabashnyuk]

@manojsaxenatda @sleshchenko does this eclipse-che/che-server#56 fixes the issue?

[manojsaxenatda]

Thank you so much, I have tried tag 7.35.0 and it fixes the websocket issue for nginx proxy.