VSWhere.exe Hard-Coded Path Fails on 32-Bit Windows #3333

jcbrill · 2019-03-26T00:19:20Z

There is a bug in the hard-code path specification for the vswhere.exe utility on 32-bit platforms.

Configuration:

Scons: 3.0.4
Python: 2.7.14 on win32
OS: Windows 7 (32-bit)

The vswhere.exe path is hard-coded in vc.py (SCons\Tool\MsCommon\vc.py: line 291 in Scons 3.0.4 and line 289 in Master) as follows:

vswhere_path = os.path.join(
    'C:\\',
    'Program Files (x86)',
    'Microsoft Visual Studio',
    'Installer',
    'vswhere.exe'
)

This causes VS 2017 (14.1) queries to fail on 32-bit platforms with the following cryptic error message:

            scons: warning: VC version 14.1 not installed.  C/C++ compilers are most likely not set correctly.

The appropriate hard-coded path for 32-bit platforms would be:

vswhere_path = os.path.join(
    'C:\\',
    'Program Files',
    'Microsoft Visual Studio',
    'Installer',
    'vswhere.exe'
)

Program files Folder:

'Program Files' [32-bit windows]
'Program Files (x86)' [64-bit windows]

SCons Users Reference: https://pairlist4.pair.net/pipermail/scons-users/2019-March/007645.html

The text was updated successfully, but these errors were encountered:

…win_install Fix Issue #3333 - Find vswhere under 32 bit windows installs

bdbaddog · 2019-03-26T21:22:06Z

Fixed in #3334

bdbaddog · 2019-03-26T21:41:46Z

If you get a chance try this out via:
virtualenv venv
. venv/bin/activate
pip install --index-url https://test.pypi.org/simple/ scons==3.0.5a5
scons --version

jcbrill · 2019-03-26T23:31:18Z

Good news: this successfully resolved the path problem on 32-bit windows.

Bad news: a production build using an extremely customized build script fails with 3.0.5a5 that runs successfully with 3.0.4.

Sheepish admission: I've been using VS 2017 for a long time under Python 2.7 and 32-bit Windows. I must have manually fixed the path in an earlier release and forgot to report the bug. For that, I apologize.

Long-winded description: The custom build script compiles DLLs using mingw-w64 and various MS compilers targeting: release and debug builds, 32-bit and 64-bit architectures, and MSVC runtime compatibility with msvcrt, msvcr80, msvcr90, msvcr100, msvcr120, and ucrt. For each compiled DLL, import libraries are generated for both the GNU and MSVC compilers using gendef, dlltool, and lib.

Failure in 3.0.5a5:

gendef - lib-random/__build/64vc140d\stk-random-64-crt140-d.dll > lib-random/__build/64vc140d\stk-random-d.def
 * [lib-random/__build/64vc140d\stk-random-64-crt140-d.dll] Found PE+ image
dlltool --as-flags=--64 -m i386:x86-64 --dllname stk-random-64-crt140-d.dll --def lib-random/__build/64vc140d\stk-random-d.def --output-lib lib-random/__build/64vc140d\libstk-random-d.a
dlltool: Can't create .lib file: lib-random/__build/64vc140d\libstk-random-d.a: Invalid bfd target
scons: *** [lib-random\__build\64vc140d\stk-random-64-crt140-d.dll] Error 1

Success in 3.0.4:

gendef - lib-random/__build/64vc140d\stk-random-64-crt140-d.dll > lib-random/__build/64vc140d\stk-random-d.def
 * [lib-random/__build/64vc140d\stk-random-64-crt140-d.dll] Found PE+ image
dlltool --as-flags=--64 -m i386:x86-64 --dllname stk-random-64-crt140-d.dll --def lib-random/__build/64vc140d\stk-random-d.def --output-lib lib-random/__build/64vc140d\libstk-random-d.a
Mkdir("lib-random/_usr/lib/64-140")
ClearReadOnly("lib-random/_usr/lib/64-140\libstk-random-d.a")
Copy("lib-random/_usr/lib/64-140\libstk-random-d.a", "lib-random/__build/64vc140d\libstk-random-d.a")

The first difference in the combined stdout+stderr for both builds is the dlltool error. This is an error that I have not seen before.

bdbaddog · 2019-03-27T00:08:06Z

I just pushed out 3.0.5 release. Please bring this issue the scons-users mailing list. Can you make a simple reproducer? Do you think the issue is due to some difference in PATH ?

…

On Tue, Mar 26, 2019 at 4:31 PM jcbrill ***@***.***> wrote: Good news: this successfully resolved the path problem on 32-bit windows. Bad news: a production build using an extremely customized build script fails with 3.0.5a5 that runs successfully with 3.0.4. *Sheepish admission: I've been using VS 2017 for a long time under Python 2.7 and 32-bit Windows. I must have manually fixed the path in an earlier release and forgot to report the bug. For that, I apologize.* Long-winded description: The custom build script compiles DLLs using mingw-w64 and various MS compilers targeting: release and debug builds, 32-bit and 64-bit architectures, and MSVC runtime compatibility with msvcrt, msvcr80, msvcr90, msvcr100, msvcr120, and ucrt. For each compiled DLL, import libraries are generated for both the GNU and MSVC compilers using gendef, dlltool, and lib. Failure in 3.0.5a5: gendef - lib-random/__build/64vc140d\stk-random-64-crt140-d.dll > lib-random/__build/64vc140d\stk-random-d.def * [lib-random/__build/64vc140d\stk-random-64-crt140-d.dll] Found PE+ image dlltool --as-flags=--64 -m i386:x86-64 --dllname stk-random-64-crt140-d.dll --def lib-random/__build/64vc140d\stk-random-d.def --output-lib lib-random/__build/64vc140d\libstk-random-d.a dlltool: Can't create .lib file: lib-random/__build/64vc140d\libstk-random-d.a: Invalid bfd target scons: *** [lib-random\__build\64vc140d\stk-random-64-crt140-d.dll] Error 1 Success in 3.0.4: gendef - lib-random/__build/64vc140d\stk-random-64-crt140-d.dll > lib-random/__build/64vc140d\stk-random-d.def * [lib-random/__build/64vc140d\stk-random-64-crt140-d.dll] Found PE+ image dlltool --as-flags=--64 -m i386:x86-64 --dllname stk-random-64-crt140-d.dll --def lib-random/__build/64vc140d\stk-random-d.def --output-lib lib-random/__build/64vc140d\libstk-random-d.a Mkdir("lib-random/_usr/lib/64-140") ClearReadOnly("lib-random/_usr/lib/64-140\libstk-random-d.a") Copy("lib-random/_usr/lib/64-140\libstk-random-d.a", "lib-random/__build/64vc140d\libstk-random-d.a") The first difference in the combined stdout+stderr for both builds is the dlltool error. This is an error that I have not seen before. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#3333 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAFBNHKvytRxk_aLDp6uTvBC9muSXy76ks5vaq3GgaJpZM4cKI3T> .

jcbrill · 2019-03-27T00:56:57Z

I have installed 3.0.5 and get the same behavior. This time the path was the bare minimum for python, scons, and windows.

Time permitting, I will see if I can isolate the problem tomorrow.

I believe the first failure is when running dlltool on the msvc cross-compiled 32-bit to 64-bit binary. I am not ruling out a path problem. For the mingw-w64 tools to generate the import library I am manually adding the mingw-w64 path (i.e., env.AppendENVPath('PATH', cls.MINGW_BIN)) to the environment.

It may be easier to isolate the problem in the source rather than conjure a simple reproducer.

bdbaddog · 2019-03-27T01:23:04Z

o.k. one of the notes for the release is: - Change the default for AppendENVPath to delete_existing=0, so path order will not be changed, unless explicitly set (Issue #3276) So AppendENVPath's default behavior is changed slightly. If you want the same behavior as before add delete_existing=1 to all your AppendEnvPath, or at least your mingw related ones?

…

-Bill

On Tue, Mar 26, 2019 at 5:56 PM jcbrill ***@***.***> wrote: I have installed 3.0.5 and get the same behavior. This time the path was the bare minimum for python, scons, and windows. Time permitting, I will see if I can isolate the problem tomorrow. I believe the first failure is when running dlltool on the msvc cross-compiled 32-bit to 64-bit binary. I am not ruling out a path problem. For the mingw-w64 tools to generate the import library I am manually adding the mingw-w64 path (i.e., env.AppendENVPath('PATH', cls.MINGW_BIN)) to the environment. It may be easier to isolate the problem in the source rather than conjure a simple reproducer. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#3333 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAFBNGvmomc-NBEf4kVflNbRL34tc3oUks5vasHZgaJpZM4cKI3T> .

jcbrill · 2019-03-27T02:27:18Z

Adding delete_existing=1 to the AppendENVPath did not change the erroneous behavior.

However, changing AppendENVPath to PrependENVPath for mingw-w64 resulted in successful builds.

I need to try to narrow down the problem case(s). It appears to be for an MSVC 64-bit build when the Mingw-w64 tools (gendef and dlltool) are added to the end of the path. It seems like a conflicting change to the environment perhaps. I am surprised moving the Mingw-w64 tools to the front of the path works. There is an unintended consequence that I do not fully understand yet.

jcbrill · 2019-03-27T03:16:19Z

More information.

When comparing the environment dumps between 3.0.4 and 3.0.5, the following folders are at the end of the 'ENV': { 'PATH': '...'} in 3.0.5 but not in 3.0.4:

C:\ProgramData\chocolatey\bin
C:\msys64
C:\msys
C:\cygwin64\bin
C:\cygwin\bin
C:/Program Files\Java\jdk1.8.0_172\bin

Only the C:\cygwin\bin and java folders actually exist locally.

AppendENVPath for the Mingw-w64 tools after the cygwin tools is the root cause of the problem. The cygwin version was never intended to be used. This explains why adding delete_files=1 to AppendENVPath did nothing while PrependENVPath was successful (i.e., the mingw folder is before the cygwin folder).

I don't know where or when these folders are added yet. None of them were on the system path when SCons was launched.

bdbaddog · 2019-03-27T15:46:15Z

Hmm. I thought the changes that were looking new locations only added paths they found tools to PATH. Can you make a simple SConstruct and dump PATH for 3.0.4 and 3.0.5?

…

On Tue, Mar 26, 2019 at 8:16 PM jcbrill ***@***.***> wrote: *More information.* When comparing the environment dumps between 3.0.4 and 3.0.5, the following folders are at the end of the 'ENV': { 'PATH': '...'} in 3.0.5 *but not in 3.0.4*: - C:\ProgramData\chocolatey\bin - C:\msys64 - C:\msys - C:\cygwin64\bin - *C:\cygwin\bin* - *C:/Program Files\Java\jdk1.8.0_172\bin* Only the C:\cygwin\bin and java folders actually exist locally. AppendENVPath for the Mingw-w64 tools after the cygwin tools is the root cause of the problem. The cygwin version was never intended to be used. This explains why adding delete_files=1 to AppendENVPath did nothing while PrependENVPath was successful (i.e., the mingw folder is before the cygwin folder). I don't know where or when these folders are added yet. *None of them were on the system path when SCons was launched.* — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#3333 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAFBNHEZt3p8SpReWS8-a_g4OwHzJ2cVks5vauKEgaJpZM4cKI3T> .

jcbrill · 2019-03-27T15:57:43Z

I opened a new issue with more information: #3336

This contains annotated selections from the environment dump.

There is another difference that I am trying to document at the moment:

There are also 4 additional builders in the MSVC environment for 3.0.5 that did not exist in 3.0.4:

Ignore the null addressed. I replaced the actual adresses with 0x00000000 for diff purposes.

    'BUILDERS': {
=>      'CFile': <SCons.Builder.CompositeBuilder object at 0x00000000>, 
        'Zip': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'MSVSProject': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'RES': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'TypeLibrary': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'StaticLibrary': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'PCH': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'Program': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'ProgramAllAtOnce': <SCons.Builder.BuilderBase object at 0x00000000>, 
=>      'Substfile': <SCons.Builder.BuilderBase object at 0x00000000>, 
=>      'CXXFile': <SCons.Builder.CompositeBuilder object at 0x00000000>, 
        'Object': <SCons.Builder.CompositeBuilder object at 0x00000000>, 
        'JavaClassDir': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'LoadableModule': <SCons.Builder.BuilderBase object at 0x00000000>, 
=>      'Textfile': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'MSVSSolution': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'CopyAs': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'SharedLibrary': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'CopyTo': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'StaticObject': <SCons.Builder.CompositeBuilder object at 0x00000000>, 
        'SharedObject': <SCons.Builder.CompositeBuilder object at 0x00000000>, 
        'JarFile': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'JavaClassFile': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'Library': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'RMIC': <SCons.Builder.BuilderBase object at 0x00000000>, 
        'JavaFile': <SCons.Builder.CompositeBuilder object at 0x00000000>
    },

jcbrill · 2019-03-27T16:11:58Z

The attached files are the env dumps for 3.0.4 and 3.0.5 with the addresses changed to zero.

Environments sequence:

Names: target bits [32|64]
       compiler [mgw|vc]
       msvcr [60|80|90|100|110|120|140]
       mode [d|r]

Environment sequence:
    32mgw60d
    32mgw60r
    32vc80d
    32vc80r
    32vc90d
    32vc90r
    32vc100d
    32vc100r
    32vc110d
    32vc110r
    32vc120d
    32vc120r
    32vc140d
    32vc140r
    64mgw60d
    64mgw60r
    64vc90d
    64vc90r
    64vc100d
    64vc100r
    64vc110d
    64vc110r
    64vc120d
    64vc120r
    64vc140d
    64vc140r

note: 64-bit msvc 2005 was not run

scons_jcb_304_zero.txt
scons_jcb_305_zero.txt

mwichmann · 2019-03-27T16:41:22Z

fwiw, substfile/textfile were intentionally added to defaults, should not affect anything you're doing.

bdbaddog · 2019-03-28T02:25:27Z

The attached files are the env dumps for 3.0.4 and 3.0.5 with the addresses changed to zero.

Environments sequence:

Names: target bits [32|64]
       compiler [mgw|vc]
       msvcr [60|80|90|100|110|120|140]
       mode [d|r]

Environment sequence:
    32mgw60d
    32mgw60r
    32vc80d
    32vc80r
    32vc90d
    32vc90r
    32vc100d
    32vc100r
    32vc110d
    32vc110r
    32vc120d
    32vc120r
    32vc140d
    32vc140r
    64mgw60d
    64mgw60r
    64vc90d
    64vc90r
    64vc100d
    64vc100r
    64vc110d
    64vc110r
    64vc120d
    64vc120r
    64vc140d
    64vc140r

note: 64-bit msvc 2005 was not run

scons_jcb_304_zero.txt
scons_jcb_305_zero.txt

Please put this info in the new bug. This bug is closed.

bdbaddog added a commit that referenced this issue Mar 26, 2019

Fix Issue #3333 - Find vswhere under 32 bit windows installs

49e73ba

bdbaddog mentioned this issue Mar 26, 2019

Fix Issue #3333 - Find vswhere under 32 bit windows installs #3334

Merged

3 tasks

bdbaddog added a commit that referenced this issue Mar 26, 2019

Merge pull request #3334 from bdbaddog/fix_gh_3333_vswhere_for_32bit_…

0113c8f

…win_install Fix Issue #3333 - Find vswhere under 32 bit windows installs

bdbaddog closed this as completed Mar 26, 2019

ghost mentioned this issue Jul 28, 2019

Not working build from source on 32-bit Windows nvaccess/nvda#9990

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

VSWhere.exe Hard-Coded Path Fails on 32-Bit Windows #3333

VSWhere.exe Hard-Coded Path Fails on 32-Bit Windows #3333

jcbrill commented Mar 26, 2019

bdbaddog commented Mar 26, 2019

bdbaddog commented Mar 26, 2019

jcbrill commented Mar 26, 2019

bdbaddog commented Mar 27, 2019 via email

jcbrill commented Mar 27, 2019

bdbaddog commented Mar 27, 2019 via email

jcbrill commented Mar 27, 2019

jcbrill commented Mar 27, 2019

bdbaddog commented Mar 27, 2019 via email

jcbrill commented Mar 27, 2019

jcbrill commented Mar 27, 2019

mwichmann commented Mar 27, 2019

bdbaddog commented Mar 28, 2019

VSWhere.exe Hard-Coded Path Fails on 32-Bit Windows #3333

VSWhere.exe Hard-Coded Path Fails on 32-Bit Windows #3333

Comments

jcbrill commented Mar 26, 2019

bdbaddog commented Mar 26, 2019

bdbaddog commented Mar 26, 2019

jcbrill commented Mar 26, 2019

bdbaddog commented Mar 27, 2019 via email

jcbrill commented Mar 27, 2019

bdbaddog commented Mar 27, 2019 via email

jcbrill commented Mar 27, 2019

jcbrill commented Mar 27, 2019

bdbaddog commented Mar 27, 2019 via email

jcbrill commented Mar 27, 2019

jcbrill commented Mar 27, 2019

mwichmann commented Mar 27, 2019

bdbaddog commented Mar 28, 2019