Use HttpClient for downloads on Windows

[davidanthoff]

This completely drops the ureq dependency on Windows and instead uses https://docs.microsoft.com/en-us/uwp/api/windows.web.http.httpclient for all HTTP/download stuff.

The benefit of this is that we are getting all the default certificate and proxy handling that Windows has.

The downside is that this is Windows 10 and newer only. It might make sense to only use this in the Windows Store builds (they are Win 10 and newer by definition) and for the standalone installer use the previousl ureq download story.

[davidanthoff]

@StefanKarpinski you spent some quality time on figuring out the download story for Julia base lately, right? Does this here broadly make sense?

[davidanthoff]

Also, note to self: merge #364 first.

[davidanthoff]

@wolthom Is there a chance that you could also try this PR on your Windows machine? The idea here is that it doesn't rely on more Unixy env variables, but instead (hopefully) just uses the standard Windows system proxy configuration.

[wolthom]

@davidanthoff Sure. Just build the most recent commit to the 'win-httpclient' branch and see what happens?

As a warning though:
The corporate setup I'm in can be quite infuriating at times (this is me being diplomatic). In case things don't work there, it could still be a working solution for most people.

[davidanthoff]

Just build the most recent commit to the 'win-httpclient' branch and see what happens?

Yes, and then just try to do some juliaup add 1.0.2 or something for a version that you definitely don't have on your machine :)

The corporate setup I'm in can be quite infuriating at times

The general idea with this PR here is that it should pick up any Windows system configuration. My hope would be that if something like Microsoft Edge works on your machine, then this here should also work without any further configuration.

[wolthom]

@davidanthoff Quick update:
This approach seems to work as well 👍
I was able to install and launch a Julia version with the local build results that I definitely did not have on the system before.

Also, I got a compilation error because ureq is not a dependency for Windows machines, but the get_proxy function in operations.rs is always compiled => A #[cfg(not(windows))] should fix that issue, though.

This this part of the code is currently being touched:
What is / was the reason to treat MacOS and other Unix target differently regarding native-tls in ureq? Could this be deduplicated?

[davidanthoff]

Ah, thanks, missed that conditional compile when I merge main :) Hopefully fixed now.

What is / was the reason to treat MacOS and other Unix target differently regarding native-tls in ureq? Could this be deduplicated?

I wanted to get the native certificate story in before Juliacon, but then I found some info that made me wonder whether that is safe on Unix (essentially apparently Debian has outdated certificate lists by default) and then I ran out of time to really sort it out and thought that just sticking with the Mozialla cert list on Linux was the safe thing to do for now. The current plan is to change the Linux story here to bring it more in line with the Mac story, details at #363.

[wolthom]

Understood, thanks for the link! Not an expert on that topic, otherwise I'd love to help out.

Also, thank you for the fantastic work on juliaup (and associated tools). That in combination with Pkg and the Artifacts / BinaryBuilder ecosystem is becoming pretty amazing for simple, reliable system setups :)

Regarding the linked issue:
https://crates.io/crates/os_info => This crate may come in handy (in case you're not already using it anyway).

[davidanthoff]

Ok, I'm merging this.

I think this is essentially the best way to do downloads on Windows, in particular it seems pretty much the only way to correctly pick up system proxy configurations. Even libcurl still has that on the backlog :)

I think that probably means we will have a more robust (or at least less configuration required) download story in Juliaup than in Downloads.jl and Pkg.jl on Windows. I'm not sure how useful that is, it probably means we will end up with folks that install Julia versions but not packages without further environment variable configuration, but presumably the right move would be that at some point Downloads.jl also picks up system proxy configuration on Windows, rather than relying on env vars.