Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version 2.11.0(3) does not ask for username/password #1034

Closed
PKEuS opened this issue Jan 19, 2017 · 28 comments
Closed

Version 2.11.0(3) does not ask for username/password #1034

PKEuS opened this issue Jan 19, 2017 · 28 comments
Assignees

Comments

@PKEuS
Copy link

PKEuS commented Jan 19, 2017

Setup

  • Which version of Git for Windows are you using? Is it 32-bit or 64-bit?
$ git --version --build-options
git version 2.11.0.windows.3
built from commit: e11df2efb3072fe73153442589129d2eb8d9ea02
sizeof-long: 4
machine: x86_64
  • Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?
Windows 7 x64
  • What options did you set as part of the installation? Or did you choose the
    defaults?
$ cat /etc/install-options.txt
Path Option: Cmd
SSH Option: OpenSSH
CRLF Option: CRLFCommitAsIs
Bash Terminal Option: ConHost
Performance Tweaks FSCache: Enabled
Use Credential Manager: Disabled
Enable Symlinks: Disabled
Enable Builtin Difftool: Disabled
  • Any other interesting things about your environment that might be related
    to the issue you're seeing?
 Saw the same on other machines which were probably installed with default settings. Downgrade to 2.11.0 helps.

Details

  • Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other
 TortoiseGit
  • What commands did you run to trigger this issue?
Clone, Fetch
  • What did you expect to occur after running these commands?
 Being asked for username&password
  • What actually happened instead?
 fatal: unable to access 'https://#####/###.git/': The requested URL returned error: 500
@shiftkey
Copy link

 fatal: unable to access 'https://#####/###.git/': The requested URL returned error: 500

This status code doesn't seem to be related to an authentication error 🤔

Could you run the following commands in Git Bash and attach the output?

cd [path to existing repository]
git config -l
GIT_TRACE=1 GIT_CURL_VERBOSE=1 git fetch

@PKEuS
Copy link
Author

PKEuS commented Jan 19, 2017

The server also returns HTTP 500 if I enter wrong login data (using the old git version without the bug).

Here is the output of the commands you suggested:

E:\sfb>git config -l
core.symlinks=false
core.autocrlf=false
core.fscache=true
color.diff=auto
color.status=auto
color.branch=auto
color.interactive=true
help.format=html
http.sslcainfo=C:/Program Files/Git/mingw64/ssl/certs/ca-bundle.cr
diff.astextplain.textconv=astextplain
rebase.autosquash=true
user.name=####
user.email=####
core.autocrlf=false
core.repositoryformatversion=0
core.filemode=false
core.bare=false
core.logallrefupdates=true
core.symlinks=false
core.ignorecase=true
core.hidedotfiles=dotGitOnly
remote.origin.url=https://git.iame.uni-bonn.de/sfb.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.common_branch.remote=origin
branch.common_branch.merge=refs/heads/common_branch
branch.master.remote=origin
branch.master.merge=refs/heads/master
E:\sfb>git fetch
18:12:00.952148 git.c:371               trace: built-in: git 'fetch'
18:12:00.983398 run-command.c:350       trace: run_command: 'git-remote-https' 'origin' 'https://git.iame.uni-bonn.de/sf
b.git'
* Couldn't find host git.iame.uni-bonn.de in the _netrc file; using defaults
* timeout on name lookup is not supported
*   Trying 131.220.47.100...
* TCP_NODELAY set
* Connected to git.iame.uni-bonn.de (131.220.47.100) port 443 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: C:/Program Files/Git/mingw64/ssl/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=DE; ST=Nordrhein-Westfalen; L=Bonn; O=Universitaet Bonn; CN=git.iame.uni-bonn.de
*  start date: Oct 17 07:44:28 2012 GMT
*  expire date: Oct 16 07:44:28 2017 GMT
*  subjectAltName: host "git.iame.uni-bonn.de" matched cert's "git.iame.uni-bonn.de"
*  issuer: C=DE; ST=Nordrhein-Westfalen; L=Bonn; O=Universitaet Bonn; OU=Hochschulrechenzentrum; CN=Universitaet Bonn CA
; [email protected]
*  SSL certificate verify ok.
> GET /sfb.git/info/refs?service=git-upload-pack HTTP/1.1
Host: git.iame.uni-bonn.de
User-Agent: git/2.11.0.windows.3
Accept: */*
Accept-Encoding: gzip
Accept-Language: C, *;q=0.9
Pragma: no-cache

< HTTP/1.1 401 Unauthorized
< Date: Thu, 19 Jan 2017 17:12:44 GMT
< Server: Apache/2.4.7 (Ubuntu)
< WWW-Authenticate: Basic realm="Git"
< Content-Length: 467
< Content-Type: text/html; charset=iso-8859-1
<
* Ignoring the response-body
* Curl_http_done: called premature == 0
* Connection #0 to host git.iame.uni-bonn.de left intact
* Issue another request to this URL: 'https://git.iame.uni-bonn.de/sfb.git/info/refs?service=git-upload-pack'
* Couldn't find host git.iame.uni-bonn.de in the _netrc file; using defaults
* Found bundle for host git.iame.uni-bonn.de: 0x524b20 [can pipeline]
* Re-using existing connection! (#0) with host git.iame.uni-bonn.de
* Connected to git.iame.uni-bonn.de (131.220.47.100) port 443 (#0)
* Server auth using Basic with user ''
> GET /sfb.git/info/refs?service=git-upload-pack HTTP/1.1
Host: git.iame.uni-bonn.de
Authorization: Basic Og==
User-Agent: git/2.11.0.windows.3
Accept: */*
Accept-Encoding: gzip
Accept-Language: C, *;q=0.9
Pragma: no-cache

< HTTP/1.1 500 Internal Server Error
< Date: Thu, 19 Jan 2017 17:12:44 GMT
< Server: Apache/2.4.7 (Ubuntu)
< WWW-Authenticate: Basic realm="Git"
< Content-Length: 618
< Connection: close
< Content-Type: text/html; charset=iso-8859-1
<
* Curl_http_done: called premature == 0
* Closing connection 0
fatal: unable to access 'https://git.iame.uni-bonn.de/sfb.git/': The requested URL returned error: 500

@shiftkey
Copy link

Interesting - I don't see a credential.helper set, but it's sending Authorization: Basic Og== after being challenged, which is : in base64...

@dscho
Copy link
Member

dscho commented Jan 19, 2017

I believe that this indicates that cURL receives a header that suggests that NTLM authentication can be used (which it cannot).

Maybe it works if you set git http.emptyauth false in the config of this repository?

@dscho
Copy link
Member

dscho commented Jan 20, 2017

Maybe it works if you set git http.emptyauth false in the config of this repository?

@PKEuS ?

@PKEuS
Copy link
Author

PKEuS commented Jan 20, 2017

This works indeed, but why does it get broken when a working installation is updated? I just updated it on another machine, and there it now fails, too (unless I manually edit the config).

@dscho
Copy link
Member

dscho commented Jan 23, 2017

This works indeed, but why does it get broken when a working installation is updated?

It was intended as a bug fix for a problem introduced by upstream Git v2.11.0. We discussed back and forth whether it is possible that the NTLM authentication is attempted by mistake, and concluded that it would be a server misconfiguration if that happened. Your report suggests that there might have been a mistake in our assessment?

In any case, the problems that were solved by switching the default to allowing "empty" authentication appear to outweigh the downsides, as major platforms rely on that "empty" authentication to switch to NTLM authentication automatically (Visual Studio Team Services, and AFAIK GitHub for Enterprise, for example).

I am far from an expert in the details of HTTPS authentication, but maybe there is a header that is sent back by the server from which you try to fetch that fools cURL into attempting NTLM authentication instead of falling back to asking for credentials interactively?

@dscho dscho self-assigned this Jan 23, 2017
@PKEuS
Copy link
Author

PKEuS commented Jan 23, 2017

In any case, the problems that were solved by switching the default to allowing "empty" authentication appear to outweigh the downsides as major platforms rely on that "empty" authentication to switch to NTLM authentication automatically (Visual Studio Team Services, and AFAIK GitHub for Enterprise, for example).

Is the usage from command line or via TortoiseGit not an important use-case? Both are affected from this. Or is it a TortoiseGit bug which does not set the config properly?

@dscho
Copy link
Member

dscho commented Jan 23, 2017

Is the usage from command line or via TortoiseGit not an important use-case?

You misunderstand. Command-line usage of Git works plenty fine with the bug fix.

What causes the problem here is the server you try to access. It apparently reports via the HTTP response header that it is okay for cURL to go ahead and attempt NTLM authentication, but that fails, so it was lying.

@DaveRinker
Copy link

What causes the problem here is the server you try to access. It apparently reports via the HTTP response header that it is okay for cURL to go ahead and attempt NTLM authentication, but that fails, so it was lying.

We're seeing a similar problem. The server is responding with "WWW-Authenticate: Basic" so I don't see that this would be a signal that it's OK to attempt NTLM.

Here's our trace (somewhat redacted) which is similar to the above although we're reporting 403 on the empty username attempt.

< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Basic realm=""
< Content-Type: text/xml
< Content-Length: 95
< Date: Sun, 22 Jan 2017 19:12:12 GMT
<

  • Ignoring the response-body
  • Curl_http_done: called premature == 0
  • Connection #0 to host git-server left intact
  • Issue another request to this URL: 'https://git-server/repos/myrepo/info/refs?service=git-upload-pack'
  • Couldn't find host git-server in the _netrc file; using defaults
  • Found bundle for host git-server: 0x149ad50 [can pipeline]
  • Re-using existing connection! (#0) with host git-server
  • Connected to git-server (x.x.x.x) port 443 (#0)
  • Server auth using Basic with user ''

GET /repos/myrepo/info/refs?service=git-upload-pack HTTP/1.1
Host: git-server
Authorization: Basic Og==
User-Agent: git/2.11.0.windows.3
Accept: /
Accept-Encoding: gzip
Accept-Language: en-US, *;q=0.9
Pragma: no-cache

< HTTP/1.1 403 Forbidden
< Content-Type: text/xml
< Content-Length: 87
< Date: Sun, 22 Jan 2017 19:12:11 GMT
<

  • Curl_http_done: called premature == 0
  • Connection #0 to host git-server left intact
    fatal: unable to access 'https://git-server/repos/myrepo/': The requested URL returned error: 403

@DaveRinker
Copy link

Based on these two RFCs, the server response in this case is correctly telling the client that Basic authentication is supported, and NOT NTLM

https://tools.ietf.org/html/rfc2617#page-5
https://tools.ietf.org/html/rfc4559#page-2

The Git client should use these headers to make the decision to send the empty username and password rather than just making it the default without any further checks as apparently was done here? 713b253

@PhilipOakley
Copy link

The 'right way' of doing such negatiation occurs on the git list a number of times with different fall backs depending on who is reporting a misconfigured server in the wild ;-)

a Quick search of the list come up with Brian Carson's https://public-inbox.org/git/[email protected]/ Brian has teneded to be the person who has done the most work (AFAICT).

Maybe have a look back at some of the discussion to see what solution was found for these tricky 'how to pass an empty username' issues.

@DaveRinker
Copy link

The server isn't misconfigured, it's correctly sending "WWW-Authenticate: Basic" which means the server does not support NTML so why would the client try to send an empty username and password by default when it would fail. The client, in this case is wrong. "Wrong" as in it's not following the RFC rules the rest of the world is following.

The Git client should respect the HTTP header that's explicitly telling the client what authentication method it supports.

@dscho
Copy link
Member

dscho commented Jan 24, 2017

@DaveRinker telling me that I should break the bug again that was fixed by the change is unlikely to have the intended effect.

I would appreciate it if you could work with us to get this fixed both for you as well as for users of Visual Studio Team Services. As I stated before, I am far from an expert when it comes to NTLM authentication (and how cURL can figure out whether it is available or not). So it is even more important that you help with the resolution of the technical issue.

@DaveRinker
Copy link

@dscho fair enough. Do you have a git trace from VSTS by chance?

@DaveRinker
Copy link

I've got one... looks like VSTS sends "WWW-Authenticate: TFS-Federated". I think we can use the server response to decide if we need to send blank (e.g. if it's WWW-Authenticate: Basic don't send blank).

Response from VSTS:

HTTP/1.1 401 Unauthorized
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/10.0
X-TFS-FedAuthRealm: https://tfsprodscussu3.app.visualstudio.com/
X-TFS-FedAuthIssuer: https://redacted.visualstudio.com/
X-VSS-ResourceTenant: 00000000-0000-0000-0000-000000000000
X-TFS-SoapException: %3c%3fxml+version%3d%221.0%22+encoding%3d%22utf-8%22%3f%3e%3csoap%3aEnvelope+xmlns%3asoap%3d%22http%3a%2f%2fwww.w3.org%2f2003%2f05%2fsoap-envelope%22%3e%3csoap%3aBody%3e%3csoap%3aFault%3e%3csoap%3aCode%3e%3csoap%3aValue%3esoap%3aReceiver%3c%2fsoap%3aValue%3e%3csoap%3aSubcode%3e%3csoap%3aValue%3eUnauthorizedRequestException%3c%2fsoap%3aValue%3e%3c%2fsoap%3aSubcode%3e%3c%2fsoap%3aCode%3e%3csoap%3aReason%3e%3csoap%3aText+xml%3alang%3d%22en%22%3eTF400813%3a+Resource+not+available+for+anonymous+access.+Client+authentication+required.%3c%2fsoap%3aText%3e%3c%2fsoap%3aReason%3e%3c%2fsoap%3aFault%3e%3c%2fsoap%3aBody%3e%3c%2fsoap%3aEnvelope%3e
X-TFS-ServiceError: TF400813%3a+Resource+not+available+for+anonymous+access.+Client+authentication+required.
WWW-Authenticate: TFS-Federated
X-TFS-FedAuthRedirect: https://app.vssps.visualstudio.com/_signin?realm=redacted.visualstudio.com&reply_to=https%3A%2F%2Fredacted.visualstudio.com%2F_git%2FTestGit%2Finfo%2Frefs%3Fservice%3Dgit-upload-pack&redirect=1&context=eyJodCI6MiwiaGlkIjoiYmJjMzVhYTgtMDIyYS00ZDU5LWE2YjctZGRlOWFmZWYzZWZkIiwicXMiOnt9LCJyciI6IiIsInZoIjoiIiwiY3YiOiIiLCJjcyI6IiJ90#ctx=eyJTaWduSW5Db29raWVEb21haW5zIjpbImh0dHBzOi8vbG9naW4ubWljcm9zb2Z0b25saW5lLmNvbSIsImh0dHBzOi8vbG9naW4ubWljcm9zb2Z0b25saW5lLmNvbSJdfQ2
WWW-Authenticate: Basic realm="https://tfsprodscussu3.app.visualstudio.com/"
X-VSS-S2STargetService: 00000002-0000-8888-8000-000000000000/visualstudio.com
WWW-Authenticate: Bearer
X-Powered-By: ASP.NET
P3P: CP="CAO DSP COR ADMa DEV CONo TELo CUR PSA PSD TAI IVDo OUR SAMi BUS DEM NAV STA UNI COM INT PHY ONL FIN PUR LOC CNT"
X-Content-Type-Options: nosniff
Date: Tue, 24 Jan 2017 17:41:54 GMT
Content-Length: 9822

@whoisj
Copy link

whoisj commented Jan 24, 2017

@DaveRinker that's true for .microsoft.com, but for local Windows hosted instances (like the ones enterprises deploy internally) they rely on Active Directory authentication via NTLM. In those cases the server will send WWW-Authenticate: Negotiate, and the client can then enter into NTLM by supplying a username:password pair of :.

If an empty username and password pair cannot be sent, there's no way for libcurl to enter into NTLM via negotiate. This cripples thousands of users who rely on Windows hosted services.

@DaveRinker
Copy link

@whoisj If the server sends only WWW-Authenticate: Basic, it seems like it would be safe to assume that we shouldn't send : at least as the default behavior

@whoisj
Copy link

whoisj commented Jan 24, 2017

@DaveRinker I believe Git tries with no basic authentication initially, and if it gets a 401/403 error it'll then prompt you for credentials. Though, to be honest I'm not completely certain.

@DaveRinker
Copy link

Yes, that's right. So the flow is:

Git Client -> request with no auth
Server -> 401 WWW-Authenticate: foo
Git Client -> request with auth

What changed recently was the second Git Client "request with auth" is now sending ":" by default and we'd like to make that only the default if "foo" is not basic. If the server sends 401 and only supports basic, a blank user ID and password is not going to work. This should continue to be a happy path for Negotiate and TFS but have the advantage of not breaking traditional servers that only use basic auth.

As it stands now, people with traditional servers would have to override the server behavior of rejecting the well-formed but improper authentication (e.g. I have sent a blank user ID and password) instead of sending the 403, re-send the 401 challenge a second time. This causes the client to finally prompt/send a user ID and password. Changing this low-level auth logic may be tricky depending on the server.

Once I have time, I'll see if I can find where we should make this change.

@whoisj
Copy link

whoisj commented Jan 25, 2017

I wonder if this additional side-effects from the http.emptyAuth=true change.

@dscho
Copy link
Member

dscho commented Jan 25, 2017

I wonder if this additional side-effects from the http.emptyAuth=true change.

I am quite certain about it. The question is whether we can ask cURL to set the credentials only when the first (401) reply contained a non-Basic header.

@DaveRinker
Copy link

@dscho right

@dscho
Copy link
Member

dscho commented Feb 21, 2017

@DaveRinker any progress on that front?

@DaveRinker
Copy link

Changing the server to re-send a 401 challenge when receiving an empty auth (":") does work even if it's technically a bit weird. I haven't looked into the modification to be able to look at the response header from the server on the cURL side

@dscho
Copy link
Member

dscho commented Feb 22, 2017

Changing the server to re-send a 401 challenge when receiving an empty auth (":") does work even if it's technically a bit weird.

Thanks for the feedback. I had a very quick look and could not easily find out how to intercept when cURL chooses Negotiate vs Basic. But then, I am far from an expert how to use cURL. If you have the time, the best way forward is probably to connect with the cURL developers, describe the problem, and hope that they have a really easy suggestion how we can fix Git.

@whoisj
Copy link

whoisj commented Feb 22, 2017

Not a cURL expert either, but from looking at enough traces it appears that cURL evaluates potential authentication schemes in the order they are presented as options by the challenging host.

@dscho
Copy link
Member

dscho commented Feb 23, 2017

from looking at enough traces it appears that cURL evaluates potential authentication schemes in the order they are presented as options by the challenging host.

Yes, I also came to the conclusion from my cursory look that there is not actually any provision to intercept or hook into that part. Hence my suggestion to involve the cURL developers (who are quite active and helpful).

dscho added a commit to dscho/git that referenced this issue Feb 25, 2017
Turning on http.emptyAuth globally turned out to cause a regression
in case of some servers that support Basic authentication but do not
respond with *yet* another 401 upon receiving the empty user/password:

	git-for-windows#1034

Jeff King came up with a superior approach. Let's revert our change in
preparation for including those patches.

This reverts commit de4b3c5.

Signed-off-by: Johannes Schindelin <[email protected]>
@dscho dscho closed this as completed in 44ae0bc Feb 25, 2017
dscho added a commit to git-for-windows/build-extra that referenced this issue Feb 25, 2017
Git for Windows [no longer attempts to send empty credentials
to HTTP(S) servers that handle only Basic and/or Digest
authentication](git-for-windows/git#1034).

Signed-off-by: Johannes Schindelin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants