-
-
Notifications
You must be signed in to change notification settings - Fork 325
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How should we implement WebSocket connection? #229
Comments
IMO we should just provide a |
Yeah that makes sense. Although I guess you would actually want to provide something like |
That makes sense. It might be slightly more efficient to provide our own struct that builds the |
Actually.. Each |
Yeah building our own stream impl makes sense. Not sure I fully understand what you mean by returning a |
It doesn't map directly to That said, from what I can tell, |
|
It is claimed kubernetes/enhancements#384 that websockets have full feature parity with SPDY. SPDY is also officially deprecated, although judging from that kubernetes issue it may stick with kubernetes for a while. A breif description of the |
There's also an ongoing issue in |
I'm still very new to
If I'm understanding correctly, kubernetes-client/python-base's stream/ws_client.py keeps a map of For port forwarding, I think it pipes to the local ports based on the prefix, but I only skimmed. |
@alexjg @edio I opened a PR (#360) based on the above and implemented the output half of Pod It's currently awkward to use: // Need to specify all the fields :(
let ap = AttachParams {
container: None,
stdout: Some(tokio::io::stdout()),
stdin: Some(tokio::io::empty()),
stderr: None,
tty: false,
};
// Pod's stdout is piped to stdout
pods.attach("example", ap).await?; So I'm considering to change the params to booleans and return a struct that you can read/write as @teozkr suggested:
I haven't decided how to implement it yet. ProtocolI haven't found any official documentation on the protocol, so I'm using the official Python client implementation as a reference. As far as I can tell, it's simple and message oriented, so we don't need to write a codec. Also, a single multiplexed connection is used to send and receive messages. The message is sent in a binary frame and its first byte is used to specify its channel (destination/purpose). For For The first message for a channel is 3 bytes and it describes this association:
Subsequent messages have just the channel prefix like the |
Closing because the discussion was about how to implement WebSocket connection in general and not about portforward. WebSocket connection was first implemented (#360) using |
There are a bunch of API calls which require interactive streams to the API server. To my knowledge these are:
exec
port-forward
proxy
attach
There's not a lot of documentation on exactly how these are implemented. I've been able to find this document which - under the header "Specialized Requests" describes a websocket based streaming protocol. Apparently the api server prefixes ever frame in the websocket connection with an integer between 0 and 256 which labels the stream that frame is intended for. e.g 0 is meant for stdin, 1 for stdout, and 2 for stderr.
With respect to port-fofrwarding each frame is additionally labelled with a two byte prefix (after the stream prefix) indicating which port the frame is intended for.
This indicates that the approach we might take would be to make a
POST
request to the port forwarding endpoint (documented here) with a websocket upgrade header. Something like this:This all seemed straightforward enough so I started thinking about the API we could use, and I followed along from the logging implementation to produce something like this in
subresource.rs
:Where the returned future would be intended to run forever to drive the port forward loop. This is probably not a good API and needs work, but more importantly I haven't found a way to implement it. That's what I want to figure out here.
WebSockets, Reqwest, and SSL. A tale of woe.
If we're going to write some code which examines each frame of the websocket stream and then dispatch that frame to the relevant port on localhost then we need an asynchronous stream of websocket messages.
My first thought was that maybe reqwest supports this usecase, unfortunately it does not, though there is an issue.
There are a few websockets libraries out there. The two that people seem to use are tungstenite and it's associated tokio bindings, or websocket-rs.
Of these two Tungstenite seems more actively maintatined (indeed, the first line of the websocket repo suggests that Tungstenite might be a better option as it's more actively maintainted). Unfortunately neither library gives us an API which we can easily use.
The problem is that we have some custom SSL parameters to set up. Specifically we need to set the root certificate and client certificate. This causes us two problems:
Api::client
and then forget about itnative_tls
The first problem is not so hard to surmount, we can just add some simple structs for tracking this information. The second problem is harder. In order to overcome this problem we would need to use methods - exposed in both libraries - which run websockets over an existing stream. E.g, in Tungstenite:
In order to do that we need to do DNS resolution, then open a TLS stream to the resolved host. And we need to do this twice, once for native-tls and once for rustls. At this point it feels like we're implementing work that perhaps should be in another library.
What then?
This is mostly just a braindump of things I learned whilst trying to implement this today. I'm leaving it here in case anyone has any idea how to move it forward and as notes to myself when I come back to it later.
Maybe someone has some experience or ideas here that I haven't thought of?
The text was updated successfully, but these errors were encountered: