Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Windows support #78

Closed
wants to merge 11 commits into from
Closed

[WIP] Windows support #78

wants to merge 11 commits into from

Conversation

isuruf
Copy link
Member

@isuruf isuruf commented Nov 30, 2017

@conda-forge-linter
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@isuruf
Copy link
Member Author

isuruf commented Nov 30, 2017

@xoviat, do you know the reason for the failure?

@isuruf
Copy link
Member Author

isuruf commented Dec 1, 2017

There are 2 problems

  • Default appveyor image has VS 2015 Update 3 and cl.exe crashes with an ICE
  • VS 2015 Update 2 can't import numpy. I tried compiling and testing numpy on that image, but it can't import numpy.

@xoviat, do you know if we can compile scipy with clang-cl.exe?

@ghost
Copy link

ghost commented Dec 1, 2017

Theoretically possible but the time commitment is a open ended. The question is why can I compile scipy but you can't.

@ghost
Copy link

ghost commented Dec 1, 2017

@isuruf
Copy link
Member Author

isuruf commented Dec 1, 2017

This is the error,

c:\bld\scipy_1512059958270\work\scipy-1.0.0\scipy\stats\_stats.c(15362) : fatal error C1001: An internal error has occurred in the compiler.
(compiler file 'f:\dd\vctools\compiler\utc\src\p2\main.c', line 255)
 To work around this problem, try simplifying or changing the program near the locations listed above.
Please choose the Technical Support command on the Visual C++ 
 Help menu, or open the Technical Support help file for more information
INTERNAL COMPILER ERROR in 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\amd64\link.exe'
    Please choose the Technical Support command on the Visual C++
    Help menu, or open the Technical Support help file for more information

Internal compiler errors are hard to reproduce.

@ghost
Copy link

ghost commented Dec 1, 2017

Idea: compile numpy with vs2017 and numpy PR #9965.

@ghost
Copy link

ghost commented Dec 1, 2017

Then compile scipy with vs2017.

@isuruf
Copy link
Member Author

isuruf commented Dec 1, 2017

Can numpy.distutils use VS2017?

@ghost
Copy link

ghost commented Dec 1, 2017

Yes

@isuruf
Copy link
Member Author

isuruf commented Dec 1, 2017

How do I use VS2017?

@ghost
Copy link

ghost commented Dec 1, 2017

Put os: Visual Studio 2017 in appveyor.yml

@isuruf
Copy link
Member Author

isuruf commented Dec 1, 2017

Yes, but that doesn't mean numpy.distutils will pick up VS2017

@ghost
Copy link

ghost commented Dec 1, 2017

On the latest Python 3.6 it will. On older versions it still will if vs 2015 is not found.

@isuruf
Copy link
Member Author

isuruf commented Dec 1, 2017

Fixed in the last commit. See the commit message. Your logs helped a lot. Thanks.

x86_amd64 is a 32-bit compiler producing amd64 code, while amd64 is
a 64-bit compiler producing amd64 code. Latter gives a ICE
while former does not
@isuruf
Copy link
Member Author

isuruf commented Dec 1, 2017

26/12056 failed for flang+openblas which is more than flang+mkl. I think OpenBLAS is not well tested in windows with MSVC. I'll look into running OpenBLAS test-suite.

@isuruf
Copy link
Member Author

isuruf commented Feb 3, 2018

@conda-forge-admin, please rerender

@jschueller
Copy link
Contributor

Hi @isuruf what's up on this ? Are there any important defects revealed by the testsuite here of could we ignore some ?

@jakirkham
Copy link
Member

Looking at the failures, they are things like computing decompositions, determinants, inversions, solving for matrices, etc. These are things that I would typically associate with LAPACK. Though could be wrong. Did not check the code of the failing tests.

Assuming LAPACK is the culprit, seeing failures on Windows here isn't too surprising given the LAPACK part of the OpenBLAS code base is basically unused on Windows. Perhaps we should look into alternative LAPACK implementations? OpenBLAS 0.20.0 includes ReLAPACK support, which could be a nice option. Raised issue ( conda-forge/openblas-feedstock#43 ) to explore it. Another option would be to use our own build of Netlib's LAPACK. Other suggestions?

@jakirkham jakirkham mentioned this pull request May 3, 2018
@tadeu
Copy link
Member

tadeu commented May 3, 2018

Thanks for following up on this @jakirkham (and everyone working before on this), getting SciPy to work on Windows is really not an easy task :)

I'll add some things I just dig hoping that maybe you find something useful there:

SciPy has an official build "script" used for testing on AppVeyor here: https://github.com/scipy/scipy/blob/master/appveyor.yml

They use OpenBLAS 0.2.20 and mingw-gfortran, and it seems they use the LAPACK that is provided with OpenBLAS [side note: weird that a BLAS lib provides a LAPACK implementation and not the other way around]. There is this documentation about building SciPy on Windows that also seems to confirm it.

Their builds are currently passing here: https://ci.appveyor.com/project/scipy/scipy

@tadeu
Copy link
Member

tadeu commented May 3, 2018

The SciPy "official" build outputs this after openblas_lapack_info::

     FOUND:
       library_dirs = ['C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\pip-req-build-_s1qezof\\build\\openblas']
       libraries = ['openblas']
       language = f77
       define_macros = [('HAVE_CBLAS', None)]

while on conda-forge, it outputs this:

  FOUND:
    libraries = ['openblas', 'openblas']
    library_dirs = ['C:\\bld\\scipy_1524225237806\\_b_env\\Library\\lib']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

I have no idea why it finds it twice in conda-forge, and why the language is f77 in one an c in the other (perhaps it's because setup couldn't find an msvc compiler in the scipy official build).

I've inspected the OpenBLAS pre-compiled package they are using, it's from SciPy wheels "channel", it's version 0.2.20, and seems to be using the standard LAPACK, not ReLAPACK, but I'm still not 100% sure on this.

@tadeu
Copy link
Member

tadeu commented May 3, 2018

Hmm, from here I've found the source of the OpenBLAS binaries they are using: https://github.com/matthew-brett/build-openblas

Here are the flags they use:

make BINARY=$BUILD_BITS DYNAMIC_ARCH=1 USE_THREAD=1 USE_OPENMP=0 \
     NUM_THREADS=24 NO_WARMUP=1 NO_AFFINITY=1 CONSISTENT_FPCSR=1 \
     BUILD_LAPACK_DEPRECATED=1 \
     COMMON_OPT="$cflags" \
     FCOMMON_OPT="$fflags" \
MAX_STACK_ALLOC=2048

perhaps it's the case to compare this against the conda-forge build of OpenBLAS on Windows?

@tadeu
Copy link
Member

tadeu commented May 3, 2018

Hmm, this failure indicate that it might actually be a problem with LAPACK configuration for complex numbers?

_________________________ TestInv.test_random_complex _________________________
self = <scipy.linalg.tests.test_basic.TestInv object at 0x000000B45F8A9400>
    def test_random_complex(self):
        n = 20
        for i in range(4):
            a = random([n, n])+2j*random([n, n])
            for i in range(n):
                a[i, i] = 20*(.1+a[i, i])
            a_inv = inv(a)
            assert_array_almost_equal(dot(a, a_inv),
>                                     identity(n))
E           AssertionError: 
E           Arrays are not almost equal to 6 decimals
E           
E           (mismatch 80.0%)
E            x: array([[ 9.787941e-01-1.260238e-02j, -1.318547e-02-1.933605e-03j,
E                   -1.024285e-01-5.050906e-04j, -1.695463e-02+2.560524e-04j,
E                   -1.235516e-02-1.183319e-02j, -1.123503e-02-2.985081e-03j,...
E            y: array([[1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
E                   0., 0., 0., 0.],
E                  [0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,...
a          = array([[5.83038901e+00+2.85997553e+01j, 6.22108771e-01+1.44818297e+00j,
        4.37727739e-01+3.73528788e-02j, 7.8535...3467485e-01j, 9.02555026e-01+7.41271734e-01j,
        2.25995526e-01+1.51916906e+00j, 8.08307477e+00+1.22473564e+01j]])
a_inv      = array([[ 6.72549153e-03-3.38476215e-02j, -3.15675610e-04+1.29006546e-03j,
         1.78181454e-03+6.08326700e-04j,  4....0989e-04j, -4.39107047e-04+1.62316713e-03j,
        -2.65091529e-03+1.90687418e-03j,  3.81396360e-02-5.72156374e-02j]])
i          = 19
n          = 20
self       = <scipy.linalg.tests.test_basic.TestInv object at 0x000000B45F8A9400>

there many possibilities of complex numbers configuration in LAPACK from OpenBLAS (in lapacke_config.h): "structure", "C99" and "CPP". Perhaps something related to this is misconfigured? Just guessing...

@jakirkham
Copy link
Member

I have no idea why it finds it twice in conda-forge, and why the language is f77 in one an c in the other (perhaps it's because setup couldn't find an msvc compiler in the scipy official build).

It might be picking up LAPACKE, which are the C bindings to LAPACK. Just guessing though.

@FSund
Copy link

FSund commented Nov 14, 2018

I'm really keen to get this working, but I have no idea how I can help.

How do I get started? I have never worked on a feedstock before.
How do I compile the feedstock on my own machine?
Are the only files that need to be changed conda-forge.yml and the files in the recipe folder?

Sorry if this is not the correct place for these questions.

@tadeu
Copy link
Member

tadeu commented Nov 14, 2018

@FSund try to start from here: https://conda-forge.org/docs/buildwin.html#local-testing

You should be able to build it locally by cloning this recipe and using conda-build: https://conda.io/docs/commands/build/conda-build.html

There's a chat here where you might find help: https://gitter.im/conda-forge/conda-forge.github.io

@FSund
Copy link

FSund commented Nov 16, 2018

@tadeu I did what you said, but I'm getting the following warnings/errors during building/testing

(build36) C:\Users\IEUser\code\scipy-feedstock>conda build recipe

[...]

============================= test session starts =============================
platform win32 -- Python 3.6.6, pytest-4.0.0, py-1.7.0, pluggy-0.8.0 -- C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542357493199\_test_env\python.exe
cachedir: .pytest_cache
rootdir: C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542357493199\test_tmp, inifile:
collecting ... collected 9287 items / 1 errors / 577 deselected

=================================== ERRORS ====================================
_________________ ERROR collecting sparse/tests/test_base.py __________________
..\_test_env\lib\site-packages\pluggy\hooks.py:284: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
..\_test_env\lib\site-packages\pluggy\manager.py:67: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
..\_test_env\lib\site-packages\pluggy\manager.py:61: in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
..\_test_env\lib\site-packages\_pytest\python.py:224: in pytest_pycollect_makeitem
    res = list(collector._genfunctions(name, obj))
..\_test_env\lib\site-packages\_pytest\python.py:432: in _genfunctions
    originalname=name,
..\_test_env\lib\site-packages\_pytest\python.py:1382: in __init__
    self.keywords[mark.name] = mark
..\_test_env\lib\site-packages\_pytest\mark\structures.py:25: in warned
    warnings.warn(warning, stacklevel=2)
E   _pytest.warning_types.RemovedInPytest4Warning: MarkInfo objects are deprecated as they contain merged marks which are hard to deal with correctly.
E   Please use node.get_closest_marker(name) or node.iter_markers(name).
E   Docs: https://docs.pytest.org/en/latest/mark.html#updating-code
============================== warnings summary ===============================

I'm using a VM from here, have installed Microsoft Build Tools for Visual Studio 2017, Miniconda, and all packages in requirements: build: in meta.yaml in an empty conda env

conda install conda-build conda-verify m2-patch python=3.6
conda install toolchain flang setuptools blas=1.1 openblas=0.2.20 numpy=1.14

I had to adjust the path to vcvarsall.bat in bld.bat to my install, but otherwise everything is stock.

Any idea what I'm doing wrong?

@nicoddemus
Copy link
Member

It seems scipy is the deprecated MarkInfo objects. I'm afraid this needs to be addressed upstream. Meanwhile I suggest to pin pytest to <4.

@FSund
Copy link

FSund commented Nov 20, 2018

Thanks. Perhaps pytest <4 should be added to the recipe? I added it to requirements: run: and test: requires: on my copy. That did the trick for me, but not sure where it should really go.

I'm running into a new issue with one of the tests. Specifically this one, which makes python crash

linalg/tests/test_decomp_update.py::TestQRinsert_D::test_economic_1_col_bad_update 

More detailed logs:

===== testing package: scipy-1.0.1-py36_blas_openblas_200 =====
running run_test.py
============================= test session starts =============================
platform win32 -- Python 3.6.6, pytest-3.10.1, py-1.7.0, pluggy-0.8.0 -- C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542705286025\_test_env\python.exe

[...]

linalg/tests/test_decomp_update.py::TestQRinsert_D::test_economic_1_row PASSED [ 14%]
linalg/tests/test_decomp_update.py::TestQRinsert_D::test_economic_p_row PASSED [ 14%]
linalg/tests/test_decomp_update.py::TestQRinsert_D::test_economic_1_col PASSED [ 14%]
linalg/tests/test_decomp_update.py::TestQRinsert_D::test_economic_1_col_bad_update 
(C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542705286025\_test_env) C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542705286025\test_tmp>IF 255 NEQ 0 exit 1 
Tests failed for scipy-1.0.1-py36_blas_openblas_200.tar.bz2 - moving package to C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\broken
WARNING:conda_build.build:Tests failed for scipy-1.0.1-py36_blas_openblas_200.tar.bz2 - moving package to C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\broken
TESTS FAILED: scipy-1.0.1-py36_blas_openblas_200.tar.bz2

Seems like this could be the same issue: scipy/scipy#5338
I tried adding this to patches, but get the same result. I'm not sure if I added the patch correctly though.

Here is my fix_error.patch

diff --git a/scipy/linalg/_decomp_update.pyx.in b/scipy/linalg/_decomp_update.pyx.in
index 03875ff7d92..dbaa9061cdc 100644
--- a/scipy/linalg/_decomp_update.pyx.in
+++ b/scipy/linalg/_decomp_update.pyx.in
@@ -1890,6 +1890,8 @@ cdef qr_insert_col(Q, R, u, int k, rcond, bint overwrite_qru, bint check_finite)
     cdef bint economic
     cdef float frc = libc.float.FLT_EPSILON
     cdef double drc = libc.float.DBL_EPSILON
+    cdef float_complex cfrc = frc
+    cdef double_complex cdrc = drc
 
     # 1 eco q alloc, r alloc, u any
     # p eco q alloc, r alloc, u any unless eco->fat then F
@@ -1963,7 +1965,7 @@ cdef qr_insert_col(Q, R, u, int k, rcond, bint overwrite_qru, bint check_finite)
         rptr = extract(rnew, rs)
         uptr = extract(u1, us)
 {{py:
-RCONDS = ['&frc', '&drc', '<float_complex*>&frc', '<double_complex*>&drc']
+RCONDS = ['&frc', '&drc', '<float_complex*>&cfrc', '<double_complex*>&cdrc']
 }}
         with nogil:
             {{for COND, TYPECODE, CNAME, RC in zip(CONDS, TCODES, CNAMES, RCONDS)}}

which I added in meta.yaml

source:
  url: https://pypi.io/packages/source/s/scipy/scipy-{{ version }}.tar.gz
  sha256: 8739c67842ed9a1c34c62d6cca6301d0ade40d50ef14ba292bd331f0d6c940ba
  patches:
    - delete_test_segfault.patch
    - fix_error.patch

It gets applied right before the resource usage statistics is listed, so I think I did it correctly, but the patch itself might not be correctly formatted.

Applying patch: 'C:\\Users\\IEUser\\code\\scipy-feedstock\\recipe\\fix_error.patch'
source tree in: C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542711051699\work

Resource usage statistics from building scipy:
   Process count: 5
   CPU time: Sys=0:00:58.6, User=0:11:41.8
   Memory: 262.4M
   Disk usage: 291.6M
   Time elapsed: 0:20:44.5

@nicoddemus
Copy link
Member

Thanks. Perhaps pytest <4 should be added to the recipe? I added it to requirements: run: and test: requires: on my copy.

pytest should appear only on test/requires I think:

test:
  requires:
    - pytest <4
    - mpmath

Also ideally we should add a comment to a related upstream issue, so the pin can be taken out in the future when it scipy is made compatible with pytest 4.

@tadeu
Copy link
Member

tadeu commented Nov 20, 2018

Nice find @FSund! It seems that patch was already released in version 1.1.0: scipy/scipy@ce36940

This PR is over an older version of SciPy (1.0.1), perhaps rebase into master to see what happens?

@FSund
Copy link

FSund commented Nov 21, 2018

I tried rebasing into master, but there were a lot of conflicts I an not sure how to resolve.

@jakirkham
Copy link
Member

Might be easier to merge with master so as to resolve conflicts once. For everything outside of the recipe directory would prefer the master changes. Would resolve the recipe changes as you think is best.

@FSund
Copy link

FSund commented Nov 21, 2018

I tried merging (locally) now.

It seems like something with the build setup must have changed, because I'm getting the following error during building which I didn't get before

C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542800623024\work>for /F "skip=2 tokens=2,*" %A in ('reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\14.0" /v InstallDir') do SET "VSINSTALLDIR=%B" 
ERROR: The system was unable to find the specified registry key or value.

It's looking for the registry key HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\14.0\InstallDir, which was used to determine the Visual Studio install directory on older versions of VS. But it seems like on VS 2017 this registry key is not set anymore [1]

I will try setting the registry key manually and look for any further issues during build.

@FSund
Copy link

FSund commented Nov 21, 2018

I got past the InstallDir issue, but here's where progress stopped today

TL;DR:
distutils.errors.DistutilsExecError: Command "cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt" -IC:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_h_env\Library\include /TcC:\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\source.c /FoC:\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\source.obj" failed with exit status 127

Full traceback

Traceback (most recent call last):
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\distutils\_msvccompiler.py", line 423, in compile
    self.spawn(args)
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\distutils\_msvccompiler.py", line 542, in spawn
    return super().spawn(cmd)
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\ccompiler.py", line 89, in <lambda>
    m = lambda self, *args, **kw: func(self, *args, **kw)
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\ccompiler.py", line 152, in CCompiler_spawn
    raise DistutilsExecError('Command "%s" failed with exit status %d%s' % (cmd, s, msg))
distutils.errors.DistutilsExecError: Command "cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt" -IC:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_h_env\Library\include /TcC:\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\source.c /FoC:\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\source.obj" failed with exit status 127

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 474, in <module>
    setup_package()
  File "setup.py", line 470, in setup_package
    setup(**metadata)
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\core.py", line 135, in setup
    config = configuration()
  File "setup.py", line 388, in configuration
    config.add_subpackage('scipy')
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\misc_util.py", line 1024, in add_subpackage
    caller_level = 2)
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\misc_util.py", line 993, in get_subpackage
    caller_level = caller_level + 1)
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\misc_util.py", line 930, in _get_configuration_from_setup_py
    config = setup_module.configuration(*args)
  File "scipy\setup.py", line 9, in configuration
    config.add_subpackage('cluster')
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\misc_util.py", line 1024, in add_subpackage
    caller_level = 2)
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\misc_util.py", line 993, in get_subpackage
    caller_level = caller_level + 1)
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\misc_util.py", line 930, in _get_configuration_from_setup_py
    config = setup_module.configuration(*args)
  File "scipy\cluster\setup.py", line 16, in configuration
    blas_opt = get_info('lapack_opt')
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\system_info.py", line 432, in get_info
    return cl().get_info(notfound_action)
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\system_info.py", line 624, in get_info
    self.calc_info()
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\system_info.py", line 1534, in calc_info
    openblas_info = get_info('openblas_lapack')
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\system_info.py", line 432, in get_info
    return cl().get_info(notfound_action)
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\system_info.py", line 624, in get_info
    self.calc_info()
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\system_info.py", line 1831, in calc_info
    if not (assume_lapack or self.check_embedded_lapack(info)):
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\site-packages\numpy\distutils\system_info.py", line 1897, in check_embedded_lapack
    obj = c.compile([src], output_dir=tmpdir)
  File "C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_build_env\lib\distutils\_msvccompiler.py", line 425, in compile
    raise CompileError(msg)
distutils.errors.CompileError: Command "cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt" -IC:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_h_env\Library\include /TcC:\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\source.c /FoC:\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\source.obj" failed with exit status 127
source tree in: C:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\work
Traceback (most recent call last):
  File "C:\Users\IEUser\Miniconda3\envs\build36\Scripts\conda-build-script.py", line 10, in <module>
    sys.exit(main())
  File "C:\Users\IEUser\Miniconda3\envs\build36\lib\site-packages\conda_build\cli\main_build.py", line 439, in main
    execute(sys.argv[1:])
  File "C:\Users\IEUser\Miniconda3\envs\build36\lib\site-packages\conda_build\cli\main_build.py", line 430, in execute
    verify=args.verify, variants=args.variants)
  File "C:\Users\IEUser\Miniconda3\envs\build36\lib\site-packages\conda_build\api.py", line 201, in build
    notest=notest, need_source_download=need_source_download, variants=variants)
  File "C:\Users\IEUser\Miniconda3\envs\build36\lib\site-packages\conda_build\build.py", line 2275, in build_tree
    notest=notest,
  File "C:\Users\IEUser\Miniconda3\envs\build36\lib\site-packages\conda_build\build.py", line 1449, in build
    windows.build(m, build_file, stats=build_stats)
  File "C:\Users\IEUser\Miniconda3\envs\build36\lib\site-packages\conda_build\windows.py", line 316, in build
    check_call_env(cmd, cwd=src_dir, stats=stats)
  File "C:\Users\IEUser\Miniconda3\envs\build36\lib\site-packages\conda_build\utils.py", line 319, in check_call_env
    return _func_defaulting_env_to_os_environ('call', *popenargs, **kwargs)
  File "C:\Users\IEUser\Miniconda3\envs\build36\lib\site-packages\conda_build\utils.py", line 299, in _func_defaulting_env_to_os_environ
    raise subprocess.CalledProcessError(proc.returncode, _args)
subprocess.CalledProcessError: Command '['cmd.exe', '/c', 'bld.bat']' returned non-zero exit status 1.

@FSund
Copy link

FSund commented Nov 22, 2018

I did some thinking, tried reinstalling VS Build Tools again, but nothing helped. Then I noticed the a DeprecationWarning just above the wall of text I posted above. Is the build failing on that warning, and not on the cl.exe step?

Here's the output from just before the lines I posted in my previous comment

creating C:\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\Users
creating C:\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\Users\IEUser
creating C:\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\Users\IEUser\AppData
creating C:\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\Users\IEUser\AppData\Local
creating C:\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\Users\IEUser\AppData\Local\Temp
creating C:\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8
cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt" -I"C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt" -IC:\Users\IEUser\Miniconda3\envs\build36\conda-bld\scipy_1542805286838\_h_env\Library\include /TcC:\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\source.c /FoC:\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\Users\IEUser\AppData\Local\Temp\tmpegu9e4n8\source.obj
setup.py:114: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp

Would the DeprecationWarning trigger a traceback and exiting the build?

@hmaarrfk
Copy link
Contributor

I just managed to build numpy 1.11.3 with the windows compilers on python 3.6+

conda-forge/numpy-feedstock#128

Maybe that recipe can be merge and we can try this again?
Should simplify the pinings.

@ocefpaf
Copy link
Member

ocefpaf commented Nov 9, 2019

@isuruf closing this as stale. I guess that if we are to try this again it is better to start fresh.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants