diff --git a/content/en/blog/_posts/2024-XX-XX-websocket-transition.md b/content/en/blog/_posts/2024-XX-XX-websocket-transition.md new file mode 100644 index 0000000000000..c04142d6f7f06 --- /dev/null +++ b/content/en/blog/_posts/2024-XX-XX-websocket-transition.md @@ -0,0 +1,132 @@ +--- +layout: blog +title: 'Streaming Transitions from SPDY to WebSockets' +date: 2024-XX-XX +slug: websockets-transition +author: > + [Sean Sullivan](https://github.com/seans3) (Google) +--- + +With Kubernetes 1.31, PortForward over Websockets is moving to beta. +RemoteCommand over Websockets previously graduated to beta in v1.30. + +Today’s article is about streaming API’s and the protocols used to implement +these API’s. + +## Why have we modified the streaming protocol ? + +Consider the following situation: you’ve configured a proxy or gateway in front +of your Kubernetes cluster, but you’ve discovered four kubectl commands no +longer work: kubectl exec, kubectl cp, kubectl attach, and kubectl port-forward. +For these kubectl commands, the HTTP REST requests are upgraded to streaming +connections. But the streaming protocol used for these upgraded connections is +SPDY/3.1, which has been **deprecated for eight years**. Your newly configured +proxy or gateway does not support this out-of-date streaming protocol. As of the +v1.31 release, however, by default all four of these kubectl commands now +implement the more modern [Websocket streaming protocol](https://datatracker.ietf.org/doc/html/rfc6455). +The Websocket protocol is a currently supported standardized streaming protocol +that guarantees compatibility and interoperability with different components and +programming languages. If your proxy or gateway supports this Websocket protocol, +these four kubectl commands will work again. + +## Why are streaming API’s necessary ? + +Most Kubernetes developers already know a cluster exposes an HTTP/REST interface, +but not as many know that several of these endpoints are upgraded to streaming +connections (requiring a streaming protocol). For most of the cluster API’s, the +standard HTTP request/response dynamic is sufficient. But some cluster +communication requires more bi-directional, low-latency, real-time interactivity. +And a streaming protocol provides the ability to read and write arbitrary data +messages between the client and server in both directions over a previously +established connection, instead of requiring a single response to a single client +request. For example, interacting with a running container in a cluster from a +client requires significant interactivity. So the kubectl exec command is +implemented using a streaming API. + +```shell +$ kubectl exec -it nginx -- /bin/bash +root@nginx:/# whoami +root +root@nginx:/# pwd +/ +``` + +In summary, Websocket and similar streaming protocols address the limitations of +traditional HTTP by providing a more efficient, responsive, and scalable means +of handling real-time data communication. + +## How do these streaming API’s work ? + +Streaming API’s work by upgrading an HTTP connection to a streaming connection. +An HTTP upgrade request for running the date command on an nginx container +within a cluster could look like: + +```shell +$ kubectl exec -v=8 nginx -- date +GET https://127.0.0.1:43251/api/v1/namespaces/default/pods/nginx/exec?command=date… +Request Headers: + Connection: Upgrade + Upgrade: websocket + Sec-Websocket-Protocol: v5.channel.k8s.io + User-Agent: kubectl/v1.30.1 (linux/amd64) kubernetes/6911225 +``` + +If the container runtime supports the Websocket streaming protocol and at least +one of the subprotocol versions (e.g. `v5.channel.k8s.io`), the server responds +with a successful `101 Switching Protocols` status, along with the negotiated +subprotocol version: + +```shell +Response Status: 101 Switching Protocols in 3 milliseconds +Response Headers: + Upgrade: websocket + Connection: Upgrade + Sec-Websocket-Accept: j0/jHW9RpaUoGsUAv97EcKw8jFM= + Sec-Websocket-Protocol: v5.channel.k8s.io +``` + +At this point the TCP connection used for the HTTP protocol has changed to a +streaming connection. Subsequent STDIN, STDOUT, and STDERR data (as well as TTY +and process exit code data) for this shell interaction is then streamed over this +upgraded connection. + +## How do I leverage the new Websocket streaming protocol ? + +If your cluster version and kubectl are greater than or equal to v1.29, there +are two control plane feature flags and two kubectl feature flags which govern +the use of the new Websocket streaming protocol. In the case of this v1.31 +release, all streaming feature flags are on by default (beta). + +- Control Plane Feature Flags + - TranslateStreamCloseWebsocketRequests + - Alpha: v1.29 + - Beta: v1.30 + - Endpoints + - /exec + - /cp + - /attach + - PortForwardWebsockets + - Alpha: v1.30 + - Beta: v1.31 + - Endpoints + - /port-forward + +- kubectl Feature Flags (environment variables) + - KUBECTL_REMOTE_COMMAND_WEBSOCKETS + - Alpha: v1.29 + - Beta: v1.30 + - Commands + - kubectl exec + - kubectl cp + - kubectl attach + - KUBECTL_PORT_FORWARD_WEBSOCKETS + - Alpha: v1.30 + - Beta: v1.31 + - Commands + - kubectl port-forward + +## Where can I find more information on streaming API’s ? + +- [KEP - Transitioning from SPDY to Websockets](https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/4006-transition-spdy-to-websockets) +- [RFC 6455 - The Websocket Protocol](https://datatracker.ietf.org/doc/html/rfc6455) +- [Container Runtime Interface streaming explained](https://kubernetes.io/blog/2024/05/01/cri-streaming-explained/)