CMD does not consistently respect PathExt

[MatthijsBlom]

Windows build number: Microsoft Windows [Version 10.0.17134.165]

Steps to reproduce

• .BAT and .PY are both on my PATHEXT.
• These two directories are on my Path:
  %AppData%\local\bin
%LocalAppData%\Packages\CanonicalGr...\LocalState\rootfs\home\matthijs
• The files hello.py, hullo.bat, hallo.py, and hollo.bat all contain a simple greeting and are distributed over the above two directories.

CMD can find the files in the Windows directory just fine, but fails to find the files in the WSL directory if the extensions are omitted:

Microsoft Windows [Version 10.0.17134.165]
(c) 2018 Microsoft Corporation. All rights reserved.

> hello & hullo & hallo & hollo
Hello!
Hullo!
'hallo' is not recognized as an internal or external command,
operable program or batch file.
'hollo' is not recognized as an internal or external command,
operable program or batch file.

> hello.py & hullo.bat & hallo.py & hollo.bat
Hello!
Hullo!
Hallo!
Hollo!

Remarkably, CMD's where has no trouble finding any of them:

> where hello hullo hallo hollo
C:\...\local\bin\hello.py
C:\...\local\bin\hullo.bat
C:\...\rootfs\home\matthijs\hallo.py
C:\...\rootfs\home\matthijs\hollo.bat

Hence the consistently in the title.

---

I did some comparing with PowerShell and the Run dialog.

PowerShell shows no difficulty:

Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

> hello ; hullo ; hallo ; hollo
Hello!
Hullo!
Hallo!
Hollo!
> hello.py ; hullo.bat ; hallo.py ; hollo.bat
Hello!
Hullo!
Hallo!
Hollo!

The Run dialog seems to simply not mind PATHEXT: it fails only on hello and hallo – the Python file names without extensions. Note that these failures are independent of file locations.

Some notes & Conclusion

I first came across this issue at leongrdic/wsl-alias#3. Apparently the behavior of CMD has recently changed?

The above tests indicate that, at the very least, CMD's behavior is inconsistent with that of PowerShell and the Run dialog. It may be inconsistent with its past behavior as well. I also find it surprising, and a bit confusing.

I report this as

• an issue because CMD behaves in an unexpected and inconsistent(!) way;
• with WSL because it has to do with how the WSL file system is handled.

Many thanks for giving us the opportunity to even have this issue! 😄

[therealkenc]

with WSL because it has to do with how the WSL file system is handled.

Can't speak to unexpected and/or inconsistent CMD behavior versus PowerShell or the Run Dialog. But on the WSL front, this is dupe #2003. GNU Bash (the shell that launches your program) doesn't have clue-one about PATHEXT.

[MatthijsBlom]

Thanks for your rapid response!

I already found #2003, but do not think this issue is a duplicate. It appears to me that #2003 is about calling files from bash (or some other WSL shell). But as far as I can tell, this issue has nothing to do with bash, as I did not even use it. Instead, this issue is about calling files located in certain directories, using a standard Windows tool.

As far as I know, it may well be possible that somewhere, somehow, bash (or some other WSL shell) is implicitly called whenever CMD crawls WSL directories. This, however, seems to me unlikely, because

• my Python files did not contain shebangs, but still managed to run;
• the Python files, when called from CMD, are interpreted by Python 3.7, while my WSL only has Python 3.6 installed;
• I would expect the same implicit call (leading to the same ultimate behavior) when using PowerShell.

[therealkenc]

I already found #2003, but do not think this issue is a duplicate

You're right I see what you are doing now.

%AppData%\local\bin
%LocalAppData%\Packages\CanonicalGr...\LocalState\rootfs\home\matthijs

Don't do that. For win32's purposes that directory does not exist.

somehow, bash (or some other WSL shell) is implicitly called whenever CMD crawls WSL directories

CMD and PS do not implicitly call anything in WSL (/usr/bin/python, /bin/bash, or otherwise). Separately, neither CMD nor PS interpret shebangs. If you are using wsl-alias, those WSL programs may be called explicitly after being bounced off a batch file that is also in your %PATH%. Also n.b. Ben's advice over here is enlightened. If the (now becoming incredibly unfortunately misnamed) MSFT bash.exe and a win32 GNU bash.exe are in your path (and the latter is interpreting shebangs), and/or you have the .py extension registered with the Windows shell to a win32 python.exe, and/or you are bouncing /usr/bin/python off a .bat file via wsl-alias, and/or you have .py in your Windows PATHEXT, you are going to need a large whiteboard and a lot of arrows to explain various observed behaviors.

But as far as I can tell, this issue has nothing to do with bash

I'll re-categorize 'discussion' for good sport. But we're not in WSL actionable territory. You are going to find MSFT bash.exe is behaving by-design. Whether powershell.exe and/or cmd.exe are behaving by-design, I can't (?) speak to that.

[leongrdic]

Don't do that. For win32's purposes that directory does not exist.

what wsl-alias does is creates a .bat file inside the wsl directory. that bat file is never modified by any tools outside the linux enviroment and all it does is calls wsl.exe file to execute the wanted linux command from windows. i am aware that files inside the wsl directory shouldn't be modified by windows and can confirm that this never happens.

regarding the problem @MatthijsBlom discovered, the biggest confusion is caused by CMD acting differently when calling a batch file without an extension while navigated inside a wsl directory and while navigated in a regular directory (this has nothing to do with PATH)

in the following screenshot, you can see that powershell doesn't have the same behavior as cmd.

[leongrdic]

Update: i accidentally came across the root of this problem
The problem happens when working with a folder that was created through wsl, and it can be anywhere in the windows filesystem!

Take a look at this

C:\Users\Leon\folder-created-in-wsl>dir
 Directory of C:\Users\Leon\folder-created-in-wsl
2018-09-17  02:18 PM                23 file.bat

C:\Users\Leon\folder-created-in-wsl>file
'file' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\Leon\folder-created-in-wsl>file.bat
success

this means there is some kind of a difference between the directories created through wsl (using the regular mkdir command) and through windows

[therealkenc]

I wasn't able to reproduce that here on 18237. Might be cursed directory related #3356, given the default /mnt/c mount options changed between 17134 and now. Or it might not.

$ mkdir /mnt/c/Users/there/source/batfoo
$ echo "echo hello" > /mnt/c/Users/there/source/batfoo/foo.bat

...and

C:\Users\there\source\batfoo>foo

C:\Users\there\source\batfoo>echo hello
hello

[leongrdic]

You're indeed right - after I turned off case sensitivity for that directory, I could run the same file by typing its name without extension in cmd, as expected.

C:\Users\Leon\folder-created-in-wsl>file
'file' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\Leon\folder-created-in-wsl>fsutil.exe file setCaseSensitiveInfo . disable
Case sensitive attribute on directory C:\Users\Leon\folder-created-in-wsl is disabled.

C:\Users\Leon\folder-created-in-wsl>file
success

Though it's still unclear why powershell behaves differently (detects the right file without its extension even in directories with case sensitivity enabled)?

[therealkenc]

Though it's still unclear why powershell behaves differently

Possibly somewhere in the code cMd is being case sloppy but PowerShelL is not. Perhaps case-normalizing somewhere on the way with toupper().

Experiment: Try doing everything all old-tymie and make the path all uppercase like it is the MSDOS days. Make the path C:\TEST\BATCH\FOO.BAT. You'll need to create C:\TEST in Windows as Admin and make it world-writable before mkdir /mnt/c/TEST/BATCH in WSL. Dunno whether it will work (I don't have a '134 on hand) but it would be fun(ish) to look.

[jonlm]

I'm so glad I found this conversation. I've been tearing my hair out trying to figure out why I can't launch exe files without the .exe extension from cmd shells in directories that were created in WSL. This only started happening in 1803, previously this was not an issue. Sure enough, calling "fsutil.exe file setCaseSensitiveInfo . disable" fixes the problem! So the question is: is this a regression or an intentional change that improves something else? If so, what?

[jonlm]

UPDATE: I figured it out. First off, I read this post explaining the new case sensitivity features:

https://blogs.msdn.microsoft.com/commandline/2018/02/28/per-directory-case-sensitivity-and-wsl/

A comment on that post got me thinking about how PATHEXT works. The extensions in PATHEXT are all uppercase (.COM, .EXE, .BAT; etc..). The reason PATHEXT works with lowercase ".exe" files is because of case insensitivity. Doing this fixed my problem without requiring me to disable case sensitivity on the directory:

set PATHEXT=.exe;.bat;%PATHEXT%

Whew. That is pretty subtle. One either needs to add both lower and uppecase versions of all desired extensions to PATHEXT, or disable default case sensitivity in WSL per that blog post.

[jonlm]

With WSL becoming increasingly popular, perhaps Microsoft should add lowercase versions of all the default extensions to the default PATHEXT. (And remove .JS while they're at it ... but that's another ball of wax.)

[therealkenc]

With WSL becoming increasingly popular, perhaps Microsoft should add lowercase versions of all the default extensions to the default PATHEXT.

Reasonable enough suggestion.

That said, MSFT jettisoned the case sensitivity stuff as the default midway though this iteration. After the Fall Release rolls out we'll probably hear less and less about it. Majority of folks will come to think of it as "broken in Spring 2018 release fixed in Fall 2018 release" and that will be that. [This is all quite sad because anyone revisiting #101 will be told the solution is to turn on case sensitivity. Which is broken. So they turn it off. Which is a loop construct with no exit conditional.]

[therealkenc]

Putting to bed as dupe #3356 #2937 (message).