Windows system proxy URL behavior differs from IE/Edge/Chrome

[filipnavara]

Description

When the system proxy is set to URL beginning with "http://" most browsers automatically strip it and work with the settings. However, .NET applications don't do that and so the same system settings appear not to work. It's easy to fix on the user's machine by deleting the prefix in the system settings but it's non-obvious problem to diagnose. Maybe it would make sense to align the behavior with the browsers and strip the prefix too.

Configuration

.NET Core 3.1.400
Windows 10 x64

[Dotnet-GitSync-Bot]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[stephentoub]

cc: @wfurt

[ghost]

Tagging subscribers to this area: @dotnet/ncl
Notify danmosemsft if you want to be subscribed.

[wfurt]

Could you be more specific @filipnavara, I don't understand the use case. Why http:// prefix would be a problem?
In case of browsers, you typically also do ftp, socks and then "just name" may make more sense.

[filipnavara]

Maybe I was not clear in describing it. What happened was that I was diagnosing an issue with customer that has a system proxy setup. The setup looked somewhat like this:

Notice that the proxy address is entered with the "http://" prefix even though it's supposed to be just plain DNS name. The browsers seem to cope with this, so if you try to open a web site in Chrome/Edge/IE it will connect the proxy (proxy.example.org in this case) just fine. However, .NET tries to use the full http://proxy.example.org as the DNS name for proxy which is invalid and hence every HTTP request fails. It's a user error to use http:// in the system settings but the specific settings work in a browser which makes it hard to find out what exactly is wrong. I am proposing that .NET HttpClient default proxy handling could be more relaxed and accept the same "invalid" value that the browsers do.

[wfurt]

ok. that would make sense. e.g. strip extra http(s):// if it exist in place where we do not expect it. I think for manual setting, we get it just once so the perf change should be trivial.

[scalablecory]

Hrm, I could swear we are already stripping that off, unless it's coming over super whack e.g. http://http://...

[stephentoub]

It certainly looks like it's trying to:

runtime/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpWindowsProxy.cs

Line 93 in b534378

if (string.Compare(proxyHelper.ProxyBypass, idx, "http://", 0, 7, StringComparison.OrdinalIgnoreCase) == 0)

[stephentoub]

Ah, that's just the bypass.

[scalablecory]

runtime/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/MultiProxy.cs

Lines 237 to 246 in b534378

	if (proxyString.StartsWith("http://"))
	{
	proxyType = INSECURE_FLAG;
	proxyString = proxyString.Slice("http://".Length);
	}
	else if (proxyString.StartsWith("https://"))
	{
	proxyType = SECURE_FLAG;
	proxyString = proxyString.Slice("https://".Length);
	}

[filipnavara]

@scalablecory Interesting. Apparently a similar code exists in the .NET Core 3.1 branch as well:

https://github.com/dotnet/corefx/blob/730e2f7af7ce3e6b141be1051a8a050b2c4d480f/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpWindowsProxy.cs#L234-L245

I wonder why it didn't work on that setup.

[scalablecory]

@filipnavara are you able to test the behavior in .NET 5?

[filipnavara]

I'll try to reproduce it on my machine with both .NET 3.1.x and .NET 5. I have very limited access to the customer's system where we noticed the issue and no way to deploy custom binaries there.

[filipnavara]

After looking at the code in .NET 3.1 I feel the problem could be that if the system proxy is specified as http://<address> it won't be used when one tries to access a https:// URI.

[scalablecory]

Triage: we should repro for 3.1 and 5.0, and decide if this needs fixing in 3.1.

[filipnavara]

So, the behavior written in the comment above is present on both .NET 5 and .NET Core 3.1. If the system proxy is prefixed with http:// then it works only for http:// requests made through HttpClient/WebClient/HttpWebRequest. HTTPS requests bypass the proxy. If the system proxy is specified with https:// prefix then it's used only for https:// prefixed URLs. This differs from how the browsers treat it where http:// prefix on the system proxy URI also affects the proxy for HTTPS requests.

[basc]

Today i stumbled around this issue. It is still present in .NET 6.
Are there any news on this?

Configuration

.NET 6.0.21
Windows 11 x64