no warning if numpy not present when installing pip

[ashander]

When installing from github via pip, no warning shows up in a conda env that has no numpy. I think there should be one, based on current code in setup.py.

env PYTHONNOUSERSITE=1 conda create -n msprime-numpy pip
source activate msprime-numpy
python -c 'import numpy'  # raises an error as expected
pip install git+https://github.com/jeromekelleher/msprime.git@master#egg=msprime  # no warning

example output

 $ env PYTHONNOUSERSITE=1 conda create -n msprime-numpy pip
Fetching package metadata .........
Solving package specifications: ..........

Package plan for installation in environment /home/jaime/miniconda3/envs/msprime-numpy:

The following NEW packages will be INSTALLED:

    ca-certificates: 2017.1.23-0      conda-forge
    certifi:         2017.1.23-py36_0 conda-forge
    ncurses:         5.9-10           conda-forge
    openssl:         1.0.2k-0         conda-forge
    pip:             9.0.1-py36_0     conda-forge
    python:          3.6.1-0          conda-forge
    readline:        6.2-0            conda-forge
    setuptools:      33.1.1-py36_0    conda-forge
    sqlite:          3.13.0-1         conda-forge
    tk:              8.5.19-1         conda-forge
    wheel:           0.29.0-py36_0    conda-forge
    xz:              5.2.2-0          conda-forge
    zlib:            1.2.11-0         conda-forge

Proceed ([y]/n)? y

Linking packages ...
[      COMPLETE      ]|#######<snip>###| 100%
#
# To activate this environment, use:
# > source activate msprime-numpy
#
# To deactivate this environment, use:
# > source deactivate msprime-numpy
#
 $ source activate msprime-numpy
 $ python -c 'import numpy'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
 $ pip install git+https://github.com/jeromekelleher/msprime.git@master#egg=msprime
Collecting msprime from git+https://github.com/jeromekelleher/msprime.git@master#egg=msprime
  Cloning https://github.com/jeromekelleher/msprime.git (to master) to /tmp/pip-build-xap8rm9h/msprime
Collecting svgwrite (from msprime)
  Using cached svgwrite-1.1.10-py2.py3-none-any.whl
Collecting pyparsing>=2.0.1 (from svgwrite->msprime)
  Using cached pyparsing-2.2.0-py2.py3-none-any.whl
Installing collected packages: pyparsing, svgwrite, msprime
  Running setup.py install for msprime ... done
Successfully installed msprime-0.4.1.dev374+g4951950 pyparsing-2.2.0 svgwrite-1.1.10

[jeromekelleher]

Thanks for this @ashander, and sorry for the hassle it caused upstream. I don't understand how this can be happening --- could the pip inside in your environment be invoking a different Python somehow? Would you mind putting a bit of debug into setup.py to see if you can figure it out?

The rationale for making numpy optional is that it's a hard requirement for a lot of installations, and compiling from scratch through pip isn't pleasant. For users of the basic library it's not needed, as we only use it when importing and exporting tree sequences though the tables API.

[ashander]

It totally makes sense to have numpy optional, and once I saw the warning I was surprised it hadn't been triggered here. I'll report back after trying some debug in setup.py. My knowledge of setup.py / setuptools is limited and I could stand to improve it.

…

On Sat, Apr 15, 2017 at 12:41 AM, Jerome Kelleher ***@***.***> wrote: Thanks for this @ashander <https://github.com/ashander>, and sorry for the hassle it caused upstream. I don't understand how this can be happening --- could the pip inside in your environment be invoking a different Python somehow? Would you mind putting a bit of debug into setup.py to see if you can figure it out? The rationale for making numpy optional is that it's a hard requirement for a lot of installations, and compiling from scratch through pip isn't pleasant. For users of the basic library it's not needed, as we only use it when importing and exporting tree sequences though the tables API. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/jeromekelleher/msprime/issues/189#issuecomment-294278178>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAfLOMA1emh4o52v_XeM828AkrHzdFexks5rwHS0gaJpZM4M-Ci1> .

[ashander]

So I've poked around a bit. In the same env as above:

• running python setup.py install or python setup.py develop does show the warning (in the first line to standard out: Warning: numpy not available. Some features will not work.) but of course this method prints lots of output to stdout
• on an experimental branch I tried printing to stderr instead and this also installs quietly with pip but if I manually run python setup.py install I can see the warning (in stderr this time)

My working hypothesis is that pip doesn't care about or propagate setup.py's to stdout (or stderr!).
This is borne out by the tip of that experimental branch which raise s a warning (instead of just printing) and so fails to install.

So, the question is how to address. Some options:

• make pip somehow tell user about the missing numpy?
• replace with a warning on import if numpy is missing instead

[jeromekelleher]

Ah, that makes sense, thanks @ashander. I had a feeling that just putting a print in setup.py would be a bad idea...

We should probably be using the warnings.warn function instead of a print in setup.py. Would you mind trying this out? Also we should probably have a warning on import too, but (I guess) silence this when we're running the CLI programs.

[ashander]

Heh, I forgot to mention, but should have. I tried warnings.warn before resorting to raise. pip didn't care about it either :\

I'll poke around a bit and see if there's a known way to get pip install to print something.

[ashander]

OK. So it seems we can't warn pip users on install.

It used to be possible to print warnings, but no longer is. From the closed issue on this it appears pypa have no plans to fix. Some packages that used to print warnings no longer do (e.g., conda references the issue but doesn't have a work around).

[jeromekelleher]

Thanks for looking into this @ashander. OK, I think the only sensible thing to do here is to have a warnings.warn() on import of msprime if numpy is missing. This can be turned off for the CLIs. We can keep a warning in setup.py and acknowledge that this won't work in pip.

[ashander]

Great. I'll contribute a patch

[jeromekelleher]

I think we can close this issue. There's not a lot we can do about pip swallowing up our warnings.