Support for Socks4/5 proxy in HttpClient/WebRequests

[abdullah2993]

Any chances of this happening?

[davidsh]

cc: @stephentoub @CIPop @himadrisarkar

[CIPop]

The current status:

• CoreFX (Desktop): WinHTTP does not support SOCKS at all.
• Windows Store apps: WinINet has support for SOCKS4 but not SOCKS4a or 5. I'm not sure we're exposing this through the Windows.Web APIs.
• non-Windows: I believe the CURL supports SOCKS. @stephentoub may be able to answer if this is currently exposed and working.

@abdullah2993, can you please share more details about the scenario?

[joshfree]

@ericeil can you comment on linux/mac socks4/5 proxy?

[abdullah2993]

@CIPop It would be helpful in almost all possible scenarios where you require a proxy besides socks4/5 proxies are available in abundance and most of the HTTP Stacks support them.
Another plus point would be integration with TOR

[ericeil]

It looks like CURL does support SOCKS4, 4a, and 5. It also looks like this may already be supported in the current System.Net implementation on Linux/Mac, by setting the request's Proxy property to something that returns, e.g., socks5://proxyhost.domain.com.

[CIPop]

@himadrisarkar In order to support this on Windows, the underlying components (WinINet and WinHTTP) need to be changed. Please follow-up with their respective owners internally.

[abdullah2993]

Any updates?

[davidsh]

There are no current plans to implement this functionality. Implementing this on Window will require changes to the native HTTP stacks. A linux-only implementation might be possible.

Feel free to submit a PR for this if you need it soon.

[karelz]

Windows doesn't support it (we would need managed implementation or use curl).
Curl supports it - we may support it there.

[coderb]

How about creating a pluggable proxy transport underneath the HttpWebRequest/Response?

In other words, allow the user application to register a proxy scheme and have an API for implementing it. The SOCKS protocol is extremely simple but I understand you not wanting to explicitly support it. This would allow others who need it to get by with a "good enough" internal implementation that MS is not on the hook for and have a low support burden.

From what I can tell this would likely involve altering ServicePointManager et al to allow custom ServicePoints. Seems relatively straightforward.

[lobster2012-user]

Add https/socks5 for SocketsHttpClient.
Thank you.

[mkurz]

+1

[Priya91]

+1 Please add support for socketshttpclient, and i assume clientwebsocket will automatically benefit

microsoft/live-share#844 (comment)

[karelz]

Please upvote the top post - that is searchable and sortable via GitHub ;)

[zepprebel]

how many thumbs up need? to release that

[karelz]

50+ will start to be interesting ...

[vfrz]

Interested about this feature too. Should be possible to implement it through SocketsHttpHandler.

[vova-lantsov-dev]

There is also an implementation of this feature https://github.com/MihaZupan/HttpToSocks5Proxy

[karelz]

Maybe we can ask @MihaZupan to chime in, how difficult it would be to port into our repo ...
Anyone is willing to out up a PR?

[MihaZupan]

For people that just want to use a socks5 proxy in their .Net apps right away, the library mentioned above should work fine and should work in every scenario as it implements the IWebProxy interface.

Having built-in support would be better in terms of performance overhead and resolving issues (some error information is swallowed by the library).

Having looked at the code, adding support shouldn't be too difficult now that there is a common managed implementation between platforms. It also likely wouldn't affect the public API surface at all as the type of proxy (http/socks4/socks4a/socks5) can be specified in the Uri scheme.

I think this is reasonable to implement as I've received a lot of requests for this functionality from the community, hence the aforementioned library.

[MihaZupan]

If any community member would like to take this and try implementing it, ping me so I can assign the issue to you. I can help out with PR review.

Otherwise I will look into this in the following weeks.

[caunt]

+1

[huoyaoyuan]

@MihaZupan I'm interested in this because I've read socks5 spec and done some tiny implementation.

[MihaZupan]

@huoyaoyuan That's great to hear!

The idea would be to allow this:

var handler = new HttpClientHandler
{
    Proxy = new WebProxy("socks5://127.0.0.1", 9050)
};
var httpClient = new HttpClient(handler);

Console.WriteLine(await httpClient.GetStringAsync("https://httpbin.org/ip"));

Essentially allowing socks* Uri schemes for IWebProxys and adding logic to handle them inside handlers.
Just like we have special code-paths to handle http/connect proxying, we would have another one for establishing a socks tunnel.

This will require changes in HttpConnectionPool, AuthenticationHelper and similar, but won't affect the public API.

I'll be happy to assist if you need help with the implementation/in PR review.

[huoyaoyuan]

@MihaZupan How will this interop with ConnectCallback?
For http proxy, ConnectCallback will deal with a http CONNECT request. But there isn't similar concept for socks.

[MihaZupan]

ConnectCallback will be used for the connection to the proxy server - same as with an http proxy (including connect).

In case of an HTTP CONNECT, we create the tunnel by asking the connection pool manager to send a new HttpRequestMessage with the Connect method to the proxy. This request ends up using the ConnectToTcpHostAsync helper that takes ConnectCallback into account.
The tunnel (Stream) we use for the actual request is the content stream of the Connect response.

For a Socks tunnel, we would instead use the ConnectToTcpHostAsync helper (to connect to the proxyUri) and perform a Socks tunnel handshake (for the actual host).

[huoyaoyuan]

I'm meaning that if a user specifies both Socks proxy and ConnectCallback, what will they get?

[MihaZupan]

ConnectCallback will still be used to establish the connection.
We will use the callback with the Uri of the proxy.

What we do after that (socks handshake / http connect ...) stays the same, regardless of whether the callback was used.

So the behavior for socks would be the same as for an http proxy - if specified, the callback is used for the connection to the proxy (as that is the only connection we're creating ourselves).

ConnectCallback is just used for creating the underlying transport stream. It doesn't care about whether we'll be layering socks/tls on top of it.

[huoyaoyuan]

So, the InitialRequestMessage will be just the original message? Sure

[MihaZupan]

So, the InitialRequestMessage will be just the original message?

Yes

[huoyaoyuan]

After reading implementation, I'm not sure how to carry the information from request to HttpConnectionPool. HttpConnectionKind has complex members because http proxy interacts differently with http and https. However, socks proxy is at socket level and orthogonal with http kind. A new field in HttpConnectionKey feels better to me.