SDK install gives HTTP 407 (Proxy authentication required)

[tacker66]

Same problems e.g. with conda package manager or NetBeans updater.
The mechanism git itself uses (http.proxy) for fetching through the proxy
works.

[dscho]

Thank you for opening a new ticket (this will make it much easier to find for others with similar issues).

As to 407, it seems that this means the proxy authentication failed: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#407

So... can you tell us a bit more context, in particular about your proxy setup?

[tacker66]

I have no further information about this proxy but as mentioned above Git 2.5.0 (Windows and Cygwin) somehow manage to download files through the proxy anonymously.

[dscho]

Hmm. So maybe Pacman has problems to install with the proxy? But you specified the proxy when installing the SDK, yes?

Maybe it would be easier to help you if you could paste the output of the SDK when it fails. (If there is too personal info, please edit instead of removing)

[tacker66]

This is what I get when running git-sdk-installer-1.0.0-64.7z.exe:

:: Synchronizing package databases...
error: failed retrieving file 'git-for-windows.db' from dl.bintray.com : Connection timed out after 10001 milliseconds
error: failed to update git-for-windows (download library error)
error: failed retrieving file 'mingw32.db' from repo.msys2.org : Connection timed out after 10000 milliseconds
error: failed retrieving file 'mingw32.db' from downloads.sourceforge.net : Connection timed out after 10000 milliseconds
error: failed retrieving file 'mingw32.db' from www2.futureware.at : Connection timed out after 10000 milliseconds
error: failed to update mingw32 (download library error)
error: failed retrieving file 'mingw64.db' from repo.msys2.org : Connection timed out after 10000 milliseconds
error: failed retrieving file 'mingw64.db' from downloads.sourceforge.net : Connection timed out after 10000 milliseconds
error: failed retrieving file 'mingw64.db' from www2.futureware.at : Connection timed out after 10000 milliseconds
error: failed to update mingw64 (download library error)
error: failed retrieving file 'msys.db' from repo.msys2.org : Connection timed out after 10000 milliseconds
error: failed retrieving file 'msys.db' from downloads.sourceforge.net : Connection timed out after 10000 milliseconds
error: failed retrieving file 'msys.db' from www2.futureware.at : Connection timed out after 10000 milliseconds
error: failed to update msys (download library error)
error: failed to synchronize any databases
error: failed to init transaction (download library error)

There was a problem accessing the MSys2 repositories
If your setup requires an HTTP proxy to access the web,
please specify it here, otherwise leave it empty.

HTTP proxy: <proxyname:proxyport>

:: Synchronizing package databases...
error: failed retrieving file 'git-for-windows.db' from dl.bintray.com : Received HTTP code 407 from proxy after CONNECT
error: failed to update git-for-windows (download library error)
error: failed retrieving file 'mingw32.db' from repo.msys2.org : The requested URL returned error: 407
error: failed retrieving file 'mingw32.db' from downloads.sourceforge.net : The requested URL returned error: 407
error: failed retrieving file 'mingw32.db' from www2.futureware.at : Received HTTP code 407 from proxy after CONNECT
error: failed to update mingw32 (download library error)
error: failed retrieving file 'mingw64.db' from repo.msys2.org : The requested URL returned error: 407
error: failed retrieving file 'mingw64.db' from downloads.sourceforge.net : The requested URL returned error: 407
error: failed retrieving file 'mingw64.db' from www2.futureware.at : Received HTTP code 407 from proxy after CONNECT
error: failed to update mingw64 (download library error)
error: failed retrieving file 'msys.db' from repo.msys2.org : The requested URL returned error: 407
error: failed retrieving file 'msys.db' from downloads.sourceforge.net : The requested URL returned error: 407
error: failed retrieving file 'msys.db' from www2.futureware.at : Received HTTP code 407 from proxy after CONNECT
error: failed to update msys (download library error)
error: failed to synchronize any databases
error: failed to init transaction (download library error)

[dscho]

Could you try with an empty user name and password in the Proxy specification? I.e. if your proxy is http://proxy.internal:8080/, try specifying http://:@proxy.internal:8080/, i.e. prefix the host name with a :@.

[tacker66]

Same result ...

[dscho]

What kind of proxy and proxy authentication do you use?

[tacker66]

Seems to be an NTLM proxy.

[dscho]

@tacker66 you would not happen to know which NTLM proxy (I am trying to reproduce here...).

[tacker66]

Sorry, this is not really my area of expertise ... what do you mean by "which" proxy?
(It's a corporate proxy which seems to use NTLM authorization and is not accessible
from the outside.)

[dscho]

Right. I have troubles replicating this scenario here... :-(

[dscho]

BTW which WIndows version is your client (just so I can try my best to make as similar a setup to yours here as possible)?

[madbonez]

Hi guys! I had same issue on versions (Only they tried) 2.5.0 and 2.4.6.5. Detected request by Wire Shark:

Seems like Git trying use ntlm by default, but i am using Basic auth in my corp.
When i rollback on 2.3.5.8 everything is working. And Git attaching proper headers on request:

[dscho]

@tacker66 Actually, there is one thing I would like you to try first: could you download (manually) https://github.com/git-for-windows/build-extra/releases/download/tmp-sdk-installer-extra/curl-extra.zip and unpack it into the SDK installer directory (there should still be usr\bin\pacman.exe; make sure that the files are unpacked with full path, i.e. into usr\bin\), then edit the file usr\etc\pacman.conf and remove the # in front of the XferCommand line mentioning /usr/bin/curl, and then run (from cmd.exe, in that usr\bin\ directory, with http_config and https_config specifying your NTLM proxy):

pacman -Sy

and paste the output here?

[dscho]

@madbonez I fear you mistook this issue; it is not about Git itself, but about the SDK installer (and the reporter actually mentioned that 2.5.0 works for them...

[tacker66]

@dscho: how do I specify http_config/https_config? In the environment?

[dscho]

In the environment, as it is done in the SDK installer: https://github.com/git-for-windows/build-extra/blob/350624b44b3224605d0da0a7f8a8ceb1b1d796d4/sdk-installer/setup-git-sdk.bat#L34-L38

[tacker66]

Here the result (there is no usr/etc/pacman.conf but a etc/pacman.conf):

set http_config=http://<name>:<port>

set https_config=https://<name>:<port>

pacman -Sy

:: Synchronizing package databases...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:16 --:--:--     0

[dscho]

Is your proxy a HTTPS proxy or an HTTP proxy? You should use that protocol, not different ones for the two environment variables. The http_ and https_ prefix only state which URLs should be retrieved via the proxy, not how you want to talk to the proxy.

Also, could you specify http://:@<name>:<port>/, as I suggested earlier? The :@ -- an empty user name and an empty password -- should be interpreted by curl.exe to use NTLM authentication.

[tacker66]

Same result with

  set http_config=https://:@<name>:<port>
  set https_config=https://:@<name>:<port>

[dscho]

Bummer.

So if you start Git Bash (not in the Git SDK), does curl https://github.com/ work correctly, without setting https_config?

[tacker66]

With "Git Bash here" same result as above (and also the same when setting https_config).
(But "git fetch" from e.g. https://github.com/git/git.git works)

[dscho]

Oooh, I think we're getting somewhere. Git configures a couple of flags to be able to use NTLM authentication IIRC. So that sent me on a StackOverflow search, and it looks as if this post has the answer:

curl --proxy-ntlm --proxy %PROXY_URL%:%PORT% %SITE_URL%

Does that work in Git Bash? If so, does it work if you set https_proxy and pass only the --proxy-ntlm option (i.e. not the --proxy option)? If so, could you add the --proxy-ntlm option to the XferCommand in your etc\pacman.conf and test whether pacman -Sy works now?

[tacker66]

  curl --proxy-ntlm --proxy https://<name>:<port> https://github.com

works in Git Bash :-) But setting the proxy via environment doesn't :-(

[dscho]

Okay, let's go one step further... Can you edit the XferCommand so that it has the --proxy option, too?

[tacker66]

I didn't notice, that you silently switched from "https_config" to "https_proxy" ;-)
With "https_proxy" it works. "pacman -Sy" gives

:: Synchronizing package databases...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2424  100  2424    0     0   8728      0 --:--:-- --:--:-- --:--:--  9431
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (22) The requested URL returned error: 404 Not Found
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  196k  100  196k    0     0  2615k      0 --:--:-- --:--:-- --:--:-- 2894k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    96  100    96    0     0   5148      0 --:--:-- --:--:-- --:--:--  8000
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  197k  100  197k    0     0  2857k      0 --:--:-- --:--:-- --:--:-- 3180k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    96  100    96    0     0   4075      0 --:--:-- --:--:-- --:--:--  6857
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  124k  100  124k    0     0  2210k      0 --:--:-- --:--:-- --:--:-- 2489k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    96  100    96    0     0   5278      0 --:--:-- --:--:-- --:--:--  8727

[dscho]

Aha! So it works ;-)

Yeah, the s in https_proxy is required because both Git for Windows' and MSys2's Pacman repositories are accessed via HTTPS.

The only unclear thing before we can go and implement a fix (which I will ask you to test!) is: does --proxy-ntlm prevent non-NTLM proxies from working...

[dscho]

Ooooh! I just found the --anyauth option... Win!

[dscho]

So... @tacker66 it is testing time! Please download the (temporary) installer from https://github.com/git-for-windows/build-extra/releases/download/tmp-sdk-installer-extra/git-sdk-installer-1.0.1-64.7z.exe -- I would like you to install it and report back whether it

• still times out the first time
• asks for the proxy
• works after you type in the proxy

Thanks!

[tacker66]

After timing out the first time it asks for the proxy and then fails with

        1 Datei(en) kopiert.
:: Synchronizing package databases...
error: failed to update git-for-windows (error invoking external downloader)
error: failed to update mingw32 (error invoking external downloader)
error: failed to update mingw64 (error invoking external downloader)
error: failed to update msys (error invoking external downloader)
error: failed to synchronize any databases
error: failed to init transaction (error invoking external downloader)

There was a problem accessing the MSys2 repositories
If your setup requires an HTTP proxy to access the web,
please specify it here, otherwise leave it empty.

HTTP proxy:

[dscho]

Hmm. Could you open a cmd, cd to that directory and then call

usr\bin\pacman -Syv --debug --verbose

and paste the output here?

[tacker66]

I tried again with a clean install, and now it worked :-)

[dscho]

Yeah! Thank you for reporting back!

[dscho]

And new SDK installers have been uploaded, too: https://github.com/git-for-windows/build-extra/releases/tag/git-sdk-1.0.1