Unbreak the build when pyenv is present

[dimpase]

The build fails when pyenv is present, as reported in report and also in #32749/#33103.

We fix it by rejecting the pyenv shims for sage-bootstrap-python.

Tickets:

• sage --gdb not working #29461 sage --gdb not working
• SAGE_ROOT/tox.ini: Add variants that provision a system python using pyenv #32531: SAGE_ROOT/tox.ini: Add variants that provision a system python using pyenv

CC: @embray @mkoeppe @slel @tobiasdiez @jhpalmieri

Component: build: configure

Author: Matthias Koeppe

Branch/Commit: 376932f

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/29285

[embray]

comment:1

Who's doing something with pyenv?

I don't think we need to provide work-arounds for every conceivable user environment misconfiguration.

[dimpase]

comment:2

Replying to @embray:

Who's doing something with pyenv?

https://dblp.org/pers/k/Kruppa:Alexander.html

I don't think we need to provide work-arounds for every conceivable user environment misconfiguration.

[Shlokatadistance]

comment:3

Pyenv creates major issues with packages sometimes, I can imagine what Sagemath might be going about. My easiest fix would be symlinking the two paths together which allows communication, between your bin and other paths, it did work for me when I was trying simple builds, of course now I use Sagemath as an application separately.

Pyenv is popular with MacOs users, so this will definitely need a fix

[Shlokatadistance]

comment:4

In fact, the breaking of the path isn't something restricted to Sage, it existed with pip too, where pip was not able to define the paths correctly for the packages to go in

[Volker-Weissmann]

comment:6

Replying to @embray:

Who's doing something with pyenv?

I do. Because I tried installing a standalone cygdb on Arch Linux for 3 days and only succeeded when I used pyenv, because I wanted two different python versions on my machine: A normal one and one compiled with --with-pydebug. Achieving this without pyenv is very hard (on Arch Linux).

I don't think we need to provide work-arounds for every conceivable user environment misconfiguration.

If it is too hard or not possible to make sagemath work with pyenv that is ok. But there should be a good error message, because no one looks at the message:

ImportError: /home/volker/.pyenv/versions/3.8.1-debug/lib/python3.8/lib-dynload/math.cpython-38d-x86_64-linux-gnu.so: undefined symbol: PyFloat_Type
/tmp/tmppfahgne0:19: Error in sourced command file:
Error while executing Python code.
/home/volker/Sync/git/sage/src/bin/sage-gdb-commands:1: Error in sourced command file:

And thinks: "Ok, it looks like I need to uninstall pyenv." An error message similar to this:
"It looks like you installed pyenv. Sagemath is incompatible with pyenv. Please uninstall pyenv."
can save someone hours of work. If mkoeppe hadn't added me to the Cc, I would be going through the source code of sagemath right now to find the error. That would have costed me multiple days.

In General, I think it is really important that misconfigurations result in nice error messages.

Note: My issue can be found here: #29461

[dimpase]

comment:9

I'm checking this with the up to date pyenv, and it seems that the issue with the PATH is no longer. Currently building latest beta with python 3.9-dev from pyenv on Debian stable, will report the results.

[dimpase]

comment:10

I was able to successfully install Sage using python 3.9 from pyenv (installed following
https://github.com/pyenv/pyenv#basic-github-checkout) on Debian 10.

Perhaps one should check this on Homebrew, but it seems it's OK.

[dimpase]

comment:11

Can be fixed as worksforme, that is.

[slel]

comment:12

Let us close this if nobody objects.

[slel]

Reviewer: Samuel Lelièvre

[mkoeppe]

comment:14

New issues with pyenv reported in #32749.

[mkoeppe]

comment:19

Test on macOS with pyenv from homebrew:

$ brew install pyenv
$ pyenv install 3.7.8
$ eval "$(pyenv init --path)"
$ echo $PATH
/Users/mkoeppe/.pyenv/shims:/Users/mkoeppe/miniconda3/condabin:/Users/mkoeppe/bin:/usr/local/bin:/Users/mkoeppe/google-cloud-sdk/bin:/Users/mkoeppe/perl5/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin
$ pyenv global 3.7.8
$ ./configure --enable-download-from-upstream-url --with-python=python3

(--with-python=python3 is needed here because otherwise we reject it as misconfigured - see #32531 comment:3)

pip's self-installation gets tricked into installing in the wrong place:

[pip-21.3.1] Using pip 21.3.1 from /Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/var/lib/sage/venv-python3.7/var/tmp/sage/build/pip-21.3.1/src/src/pip (python 3.7)
[pip-21.3.1] Looking in links: /Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/var/lib/sage/venv-python3.7/var/lib/sage/wheels
[pip-21.3.1] Processing /Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/var/lib/sage/venv-python3.7/var/tmp/sage/build/pip-21.3.1/inst/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/var/lib/sage/venv-python3.7/var/lib/sage/wheels/pip-21.3.1-py3-none-any.whl
[pip-21.3.1] Installing collected packages: pip
[pip-21.3.1]   changing mode of /Users/mkoeppe/.pyenv/versions/3.7.8/bin/pip to 755
[pip-21.3.1]   changing mode of /Users/mkoeppe/.pyenv/versions/3.7.8/bin/pip3 to 755
[pip-21.3.1]   changing mode of /Users/mkoeppe/.pyenv/versions/3.7.8/bin/pip3.7 to 755
[pip-21.3.1] Successfully installed pip-21.3.1
[pip-21.3.1] 

[mkoeppe]

comment:20

sage-pip-install -> sage-flock -> sage-bootstrap-python -> $HOME/.pyenv/shims -> $HOME/.pyenv/versions/3.7.8/bin/python3; the python3 shim manipulates PATH.

(sage-buildsh) mkoeppe@egret:pip-21.3.1$ python3 -c 'import os; print(os.environ["PATH"])'
/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/src/bin:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/libexec/ccache:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/build/bin:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/var/lib/sage/venv-python3.7/bin:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/bin:/Users/mkoeppe/.pyenv/shims:/Users/mkoeppe/miniconda3/condabin:/Users/mkoeppe/bin:/usr/local/bin:/Users/mkoeppe/google-cloud-sdk/bin:/Users/mkoeppe/perl5/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin
(sage-buildsh) mkoeppe@egret:pip-21.3.1$ sage-bootstrap-python -c 'import os; print(os.environ["PATH"])'
/Users/mkoeppe/.pyenv/versions/3.7.8/bin:/usr/local/Cellar/pyenv/2.2.3/libexec:/usr/local/Cellar/pyenv/2.2.3/plugins/python-build/bin:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/src/bin:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/libexec/ccache:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/build/bin:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/var/lib/sage/venv-python3.7/bin:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/bin:/Users/mkoeppe/.pyenv/shims:/Users/mkoeppe/miniconda3/condabin:/Users/mkoeppe/bin:/usr/local/bin:/Users/mkoeppe/google-cloud-sdk/bin:/Users/mkoeppe/perl5/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin

[mkoeppe]

Branch: u/mkoeppe/unbreak_the_build_when_pyenv_is_present

[mkoeppe]

Commit: 376932f

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

New commits:

376932f

build/bin/sage-bootstrap-python: Reject pyenv shims

[mkoeppe]

Changed reviewer from Samuel Lelièvre to none

[mkoeppe]

Reviewer: https://github.com/mkoeppe/sage/runs/4682227249?check_suite_focus=true

[mkoeppe]

Changed reviewer from https://github.com/mkoeppe/sage/runs/4682227249?check_suite_focus=true to https://github.com/mkoeppe/sage/runs/4682289891?check_suite_focus=true

[mkoeppe]

comment:25

Let's get this into 9.5 please

[jhpalmieri]

Changed reviewer from https://github.com/mkoeppe/sage/runs/4682289891?check_suite_focus=true to John Palmieri

[jhpalmieri]

comment:28

Works for me on OS X with homebrew's pyenv: without this branch, sage-bootstrap runs the "wrong" python3, while with this branch, it runs the right python3.

[mkoeppe]

comment:29

Thank you!

[vbraun]

Changed branch from u/mkoeppe/unbreak_the_build_when_pyenv_is_present to 376932f