Cannot "git pull" if any remote branch contains invalid char - regression from 2.21 to 2.24

[will-bartlett]

• [ x ] I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

$ git --version --build-options

git version 2.24.1.windows.2
cpu: x86_64
built from commit: 992f0773022527b1b0cb1e0c13aec97dd5248053
sizeof-long: 4
sizeof-size_t: 8

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver

Microsoft Windows [Version 10.0.18363.535]

• What options did you set as part of the installation? Or did you choose the
  defaults?

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

Editor Option: VisualStudioCode
Custom Editor Path:
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: WinSSL
CRLF Option: CRLFAlways
Bash Terminal Option: ConHost
Performance Tweaks FSCache: Enabled
Use Credential Manager: Enabled
Enable Symlinks: Enabled
Enable Builtin Interactive Add: Disabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

The problem appears related to a branch name containing a dot/period character. You should be able to reproduce the problem with a remote branch named branch./name.

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

Cmd.

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

git checkout master
git pull

• What did you expect to occur after running these commands?

I expected git to update its local state (git fetch behavior), log an error for the branch with the invalid name, and then update the state of master to match origin/master.

• What actually happened instead?

Git updated its local state, logged an error, and then exited, leaving master in its previous state (X commits behind origin/master).

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

N/A.

Additional thoughts: it occurs to me that this behavior may be by design / a "bugfix" - exiting with a failed error code where an error was previously silently ignored. However, it doesn't seem right that a badly named remote without a corresponding local branch would block "git pull" in master.

Workaround: it is possible to workaround this bug by using "git merge <tip of remote>" instead of "git pull"

[dscho]

This does not appear to be a new bug, actually. It was just hidden by yet another bug, and now it is made obvious.

In their unpacked format, refs (including branch names) are represented as files, and file names with a trailing dot are illegal on Windows. If you try to write them, though, no error will be thrown, but the file name without a trailing dot will be used automatically.

Which means: if you have a branch name branch./name and a branch name branch/name, they will be backed by the exact same file, which leads to a lot of confusion.

In short: you are hitting a bug that was ignored for years, and that is no longer ignored as of v2.24.1(2).

[Oeffner]

I think I have encountered the same error in git version 2.25.0.windows.1
When I try to clone the repo https://git.uis.cam.ac.uk/cimr-phaser/phaser.git Git fails with the error message of invalid path. There is a file in the repo named doc/phaserwiki/articles/a/b/o/Phaserwiki:About_932f.html. Obviously the ":" character is illegal on Windows and in previous version of git the file would be renamed to doc\phaserwiki\articles\a\b\o\Phaserwiki~About_932f.html and Git would carry on cloning the full repo. But in version 2.25 git fails. My workaround for now is to use an older version of Git.

[PhilipOakley]

Has this been fixed in the new V2.26 release? Worth testing. I'm not sure if the fixes mentioned are in the upstream yet. Just a thought. https://github.com/git-for-windows/git/releases/

[dscho]

There is a file in the repo named doc/phaserwiki/articles/a/b/o/Phaserwiki:About_932f.html. Obviously the ":" character is illegal on Windows and in previous version of git the file would be renamed to doc\phaserwiki\articles\a\b\o\Phaserwiki~About_932f.html and Git would carry on cloning the full repo.

I highly doubt that the file would have been renamed in the indicated manner. Instead, the file would be marked as deleted right off the bat, i.e. right after cloning, and an "untracked" file doc\phaserwiki\articles\a\b\o\Phaserwiki would appear.

[Oeffner]

There is a file in the repo named doc/phaserwiki/articles/a/b/o/Phaserwiki:About_932f.html. Obviously the ":" character is illegal on Windows and in previous version of git the file would be renamed to doc\phaserwiki\articles\a\b\o\Phaserwiki~About_932f.html and Git would carry on cloning the full repo.

I highly doubt that the file would have been renamed in the indicated manner. Instead, the file would be marked as deleted right off the bat, i.e. right after cloning, and an "untracked" file doc\phaserwiki\articles\a\b\o\Phaserwiki would appear.

I'm using Git 2.18 and that's what I'm experiencing. But that's less important. What matters is the failure of version 2.25 to clone the repository.

[dscho]

I'm using Git 2.18 and that's what I'm experiencing. But that's less important. What matters is the failure of version 2.25 to clone the repository.

I disagree. As long as we do not understand how on earth your Git (for Windows?) v2.18 can munge the path of an otherwise illegal file name, we won't understand how to address this properly.

You could try setting core.protectNTFS to false, of course, but that is pretty dangerous.

[Oeffner]

All I can say is that this is what I have experienced with this repository on older versions of Git for Windows on Windows 10. It's never borthered me since the particular file was not important to me. But now where it prevents cloning the repo on Windows it's a different matter.
If you want to prove me wrong the try the following command on Windows 10 with Git 2.18 and Git 2.25
git clone https://git.uis.cam.ac.uk/cimr-phaser/phaser.git

[PhilipOakley]

I see (for the record):

phili@Philip-Win10 MINGW64 ~
$ git clone https://git.uis.cam.ac.uk/cimr-phaser/phaser.git
Cloning into 'phaser'...
remote: Enumerating objects: 70879, done.
remote: Total 70879 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (70879/70879), 65.67 MiB | 190.00 KiB/s, done.
Resolving deltas: 100% (53258/53258), done.
error: invalid path 'doc/phaserwiki/articles/a/b/o/Phaserwiki:About.html'
fatal: unable to checkout working tree
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

phili@Philip-Win10 MINGW64 ~
$ cd phaser

phili@Philip-Win10 MINGW64 ~/phaser (master)
$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        deleted:    README.md
        deleted:    SConscript
        deleted:    ccp4i/etc/modules.def
...... [snip lots]

phili@Philip-Win10 MINGW64 ~/phaser (master)
$ git restore --source=HEAD :/
error: invalid path 'doc/phaserwiki/articles/a/b/o/Phaserwiki:About.html'
error: invalid path 'doc/phaserwiki/articles/c/o/p/Phaserwiki:Copyrights.html'
error: invalid path 'doc/phaserwiki/articles/d/o/c/Template:Documentation.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser-2.2:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser-2.3:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser-2.4:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser-2.5.1:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser-2.5.2:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser-2.5.3:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser-2.5.4:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser-2.5.5:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser-2.5.6:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser-2.5.7:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser-2.5:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser-2.6.0:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser-2.7.0:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser-2.7.12:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser-2.7.14:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/a/n/Phaser:_Manual.html'
error: invalid path 'doc/phaserwiki/articles/m/o/d/Phaser-2.2:_Modes.html'
error: invalid path 'doc/phaserwiki/articles/r/e/d/Help:Redirects.html'

phili@Philip-Win10 MINGW64 ~/phaser (master)
$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        deleted:    README.md
        deleted:    SConscript
        deleted:    ccp4i/etc/modules.def

... [snipped]
        deleted:    write_version.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        README.md
        SConscript
        ccp4i/
        ccp4i2/
        codebase/
        conda_envs/
        defaults
        doc/
        libtbx_config
        licences/
        ncs/
        phaser/
        process_default_settings.py
        test/
        tutorial/
        write_version.py

Is that what you see?

Do check the final restore command. The option you need is likely not the suggested one.

[dscho]

Yes, I expected pretty much precisely what @PhilipOakley sees.

[PhilipOakley]

While the clone ended with advice git restore --source=HEAD :/, the status command then suggests git restore --staged <file>... to unstage the deletions.

I'm not up on usage of the restore command but if it really need all the deleted files to be listed for unstaging, it's perhaps not a helpful as needed, but hopefully a closer reading of the man page would 'do the right thing'.

It could be that in this (initial clone) case we have commands/advice acting at cross purposes i.e. unstage a deletion vs git add untracked files (from the status report)?