Tox can't find python3.x executable in virtualenv #202

gangefors · 2017-03-03T12:08:41Z

After creating a virtualenv with:

pyenv virtualenv 3.5.3 myenv

I'm not able to run tox with this ini file:

[tox]
envlist = py35

[testenv]
commands =
    pytest {posargs: --pep8 --pylint}

deps = .[test]

The following error appears:

ERROR: InvocationError: Failed to get version_info for python3.5: b"pyenv: python3.5: command not found\n\nThe `python3.5' command exists in these Python versions:\n  3.5.3\n\n"

After checking my virtualenv it's clear that it doesn't contain a python3.5 symlink.

(myenv) ➜  ~ ll /home/stefanga/.pyenv/versions/myenv/bin/      
total 32K
-rw-r--r-- 1 stefanga axusers 2.2K Mar  3 13:06 activate
-rw-r--r-- 1 stefanga axusers 1.3K Mar  3 13:06 activate.csh
-rw-r--r-- 1 stefanga axusers 2.4K Mar  3 13:06 activate.fish
-rwxr-xr-x 1 stefanga axusers  276 Mar  3 13:06 easy_install
-rwxr-xr-x 1 stefanga axusers  276 Mar  3 13:06 easy_install-3.5
-rwxr-xr-x 1 stefanga axusers  248 Mar  3 13:06 pip
-rwxr-xr-x 1 stefanga axusers  248 Mar  3 13:06 pip3
-rwxr-xr-x 1 stefanga axusers  248 Mar  3 13:06 pip3.5
lrwxrwxrwx 1 stefanga axusers   47 Mar  3 13:06 python -> /home/stefanga/.pyenv/versions/3.5.3/bin/python
lrwxrwxrwx 1 stefanga axusers    6 Mar  3 13:06 python3 -> python

After creating the python3.5 -> python symlink manually tox executed without any errors.

Shouldn't all executables (python, pythonx and python x.y) be available in the virtualenv?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

The text was updated successfully, but these errors were encountered:

blueyed · 2017-03-03T19:50:57Z

See pyenv/pyenv#92 (comment).

gangefors · 2017-03-06T09:34:50Z

Thanks! But should I really have to use another plugin to get the expected behaviour?

To me it just feels like there should be a pythonX.Y symlink in the virtualenv's bin folder after creating a new virtualenv. Since the executables are present in the system's python and also pyenv installed python versions I'd expect to find similar symlinks in the virtualenvs as well.

But if you disagree, feel free to close this issue.

eddie-dunn · 2017-03-07T09:08:31Z

I'm with @gangefors on this. Is there a reason that said symlinks can't be created automatically by pyenv?

yyuu · 2017-03-07T12:53:39Z

If you think so, just write something by yourself.

gangefors · 2017-03-07T13:58:49Z

To anyone that finds this thread and experience the same issue and don't want to use yet another plugin to solve it.

I've found a workaround to this issue.

By specifying exactly which python binary you want to use you can get the expected behaviour.

pyenv virtualenv -p python3.5 3.5.3 myenv

will create a virtualenv with the following contents in ./bin

ll ~/.pyenv/versions/myenv/bin 
total 12M
-rw-r--r-- 1 stefanga axusers 2.1K Mar  7 14:51 activate
-rw-r--r-- 1 stefanga axusers 1.1K Mar  7 14:51 activate.csh
-rw-r--r-- 1 stefanga axusers 2.2K Mar  7 14:51 activate.fish
-rw-r--r-- 1 stefanga axusers 1.2K Mar  7 14:51 activate_this.py
-rwxr-xr-x 1 stefanga axusers  277 Mar  7 14:51 easy_install
-rwxr-xr-x 1 stefanga axusers  277 Mar  7 14:51 easy_install-3.5
-rwxr-xr-x 1 stefanga axusers  249 Mar  7 14:51 pip
-rwxr-xr-x 1 stefanga axusers  249 Mar  7 14:51 pip3
-rwxr-xr-x 1 stefanga axusers  249 Mar  7 14:51 pip3.5
lrwxrwxrwx 1 stefanga axusers    9 Mar  7 14:51 python -> python3.5
lrwxrwxrwx 1 stefanga axusers    9 Mar  7 14:51 python3 -> python3.5
-rwxr-xr-x 1 stefanga axusers  12M Mar  7 14:51 python3.5
-rwxr-xr-x 1 stefanga axusers 2.4K Mar  7 14:51 python-config
-rwxr-xr-x 1 stefanga axusers  256 Mar  7 14:51 wheel

The plugin (probably the python venv module) now creates a virtualenv with python3.5 binary as the primary binary and creates symlinks to this. The result is that I've got my pythonX.Y binary in the virtualenv and tox is happy again.

anentropic · 2017-10-26T10:36:57Z

I have same problem:

$ pyenv virtualenv 3.6.2 myenv
$ pyenv activate myenv
$ tox -e py36-dj111
py36-dj111 create: /App/.tox/py36-dj111
ERROR: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.6: b"pyenv: python3.6: command not found\\n\\nThe `python3.6\' command exists in these Python versions:\\n  3.6.2\\n\\n"',)
_______________________________________________________ summary _______________________________________________________
ERROR:   py36-dj111: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.6: b"pyenv: python3.6: command not found\\n\\nThe `python3.6\' command exists in these Python versions:\\n  3.6.2\\n\\n"',)

I could not get tox-pyenv plugin to solve this issue (it seems to be intended to solve a slightly different problem?)

However @gangefors suggestion above does work

If I delete my virtualenv and recreate it with pyenv virtualenv -p python3.6 myenv then tox will work

So it seems to be down to pyenv-virtualenv not creating the python3.6 symlink. I note that pip3.6 and easy_install-3.6 links are created.

eddie-dunn · 2017-10-27T09:59:37Z

@yyuu I would like to solve this, preferrably in the same place as where the python and python3 symlinks are created. I couldn't find where this is, though.

Could you point me in the right direction?

gangefors · 2017-10-30T18:30:22Z

Possibly related to #206

madhurabhogate · 2017-11-29T17:22:13Z

@gangefors : The solution you suggested (pyenv virtualenv -p python3.5 3.5.3 myenv) works.
But in my tox.ini I want to verify for multiple python versions say py35, py36. By creating the the virtualenv in above manner; tox runs successfully for py35 version but fails for py36.

ERROR: py36: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.6: b"pyenv: python3.6: command not found\n\nThe `python3.6' command exists in these Python versions:\n 3.6.0\n 3.6.1\n\n"',)

How do we overcome this issues ? Can you suggest please ?

gangefors · 2017-11-29T17:55:55Z

@madhurabhogate I would suggest that you create different venvs for different python versions. Then you can use them at the same time with pyenv <shell|local> <py35venv> <py36venv>. The python binaries from both venvs should be available in your path for tox to find.

# Do the following in your shell
$ pyenv virtualenv -p python3.5 3.5.4 py35
$ pyenv virtualenv -p python3.6 3.6.3 py36
$ pyenv shell py36 py35

# Then you should have two active environments you can use
$ pyenv versions
  system
  3.5.4
  3.5.4/envs/py35
  3.6.3
  3.6.3/envs/py36
* py35 (set by PYENV_VERSION environment variable)
* py36 (set by PYENV_VERSION environment variable)

# Now you should be able to run tox for both py35 and py36 envlists

madhurabhogate · 2017-11-29T18:06:55Z

@gangefors : Thank you. An example would be really helpful :)

hhstore · 2018-04-07T14:53:02Z

you can try:

pyenv shell 3.5.3

ref: pyenv/pyenv#856

bsolomon1124 · 2018-10-25T14:27:40Z

Note that there is a small typo in @gangefors 's example (at least for pyenv 1.2.7) with a flag:

$ brew install pyenv-virtualenv
$ pyenv virtualenv -p python3.5 3.5.6 py35
$ pyenv virtualenv -p python3.6 3.6.6 py36
$ pyenv virtualenv -p python3.7 3.7.0 py37
$ pyenv shell py35 py36 py37

If using with tox, order of cmds seems to be a tiny bit finnicky and very particular:

$ pyenv shell py35 py36 py37
$ python3.7 -m venv venv
$ source ./venv/bin/activate
$ tox

GhostofGoes · 2019-01-26T00:54:43Z

The issue for me was simply a case of not having the Pythons on the pyenv path. Fixed by adding them to the path:

pyenv local 3.7.2 3.6.8 3.5.6 pypy3.5-6.0.0 pypy2.7-6.0.0

mark0978 · 2019-02-13T18:28:19Z

I did pyenv local 3.4.9 3.5.6 3.6.8 but that did NOT fix the shims. For that I had to use rm (to remove the old shim) and ln -s to create the new shim

cicorias · 2019-02-23T18:13:30Z

i was able to fix this in a docker build here
https://github.com/Microsoft/agogosml/blob/7a41e4d136534c9b8caf93dc1a87cb52a62aa845/agogosml_cli/Dockerfile

The "gist" of it all is below:

RUN curl https://pyenv.run | bash
#see https://www.loganasherjones.com/2018/03/setting-up-your-python-environment/

RUN pyenv install 3.5.6
RUN pyenv install 3.6.8
RUN pyenv install 3.7.2
RUN pip install pytest tox
RUN pip install tox-globinterpreter

WORKDIR /usr/src/agogosml_cli

RUN echo "3.5.6" >> .python-version
RUN echo "3.6.8" >> .python-version
RUN echo "3.7.2" >> .python-version

yyuu closed this as completed Mar 7, 2017

kroitor mentioned this issue Sep 7, 2017

use python3.5 for flake8 ccxt/ccxt#180

Merged

This was referenced Sep 27, 2017

Travis CI currently broken on some Python 3.5 environments pytest-dev/pytest#2779

Closed

pyenv for python3.5 misconfigured? travis-ci/travis-ci#8363

Closed

gangefors mentioned this issue Oct 30, 2017

python -m venv does not create python3.X symlink (but using python3.6 does) #206

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tox can't find python3.x executable in virtualenv #202

Tox can't find python3.x executable in virtualenv #202

gangefors commented Mar 3, 2017 •

edited

Loading

blueyed commented Mar 3, 2017

gangefors commented Mar 6, 2017 •

edited

Loading

eddie-dunn commented Mar 7, 2017

yyuu commented Mar 7, 2017

gangefors commented Mar 7, 2017 •

edited

Loading

anentropic commented Oct 26, 2017

eddie-dunn commented Oct 27, 2017

gangefors commented Oct 30, 2017

madhurabhogate commented Nov 29, 2017

gangefors commented Nov 29, 2017 •

edited

Loading

madhurabhogate commented Nov 29, 2017

hhstore commented Apr 7, 2018

bsolomon1124 commented Oct 25, 2018 •

edited

Loading

GhostofGoes commented Jan 26, 2019

mark0978 commented Feb 13, 2019

cicorias commented Feb 23, 2019

Tox can't find python3.x executable in virtualenv #202

Tox can't find python3.x executable in virtualenv #202

Comments

gangefors commented Mar 3, 2017 • edited Loading

blueyed commented Mar 3, 2017

gangefors commented Mar 6, 2017 • edited Loading

eddie-dunn commented Mar 7, 2017

yyuu commented Mar 7, 2017

gangefors commented Mar 7, 2017 • edited Loading

anentropic commented Oct 26, 2017

eddie-dunn commented Oct 27, 2017

gangefors commented Oct 30, 2017

madhurabhogate commented Nov 29, 2017

gangefors commented Nov 29, 2017 • edited Loading

madhurabhogate commented Nov 29, 2017

hhstore commented Apr 7, 2018

bsolomon1124 commented Oct 25, 2018 • edited Loading

GhostofGoes commented Jan 26, 2019

mark0978 commented Feb 13, 2019

cicorias commented Feb 23, 2019

gangefors commented Mar 3, 2017 •

edited

Loading

gangefors commented Mar 6, 2017 •

edited

Loading

gangefors commented Mar 7, 2017 •

edited

Loading

gangefors commented Nov 29, 2017 •

edited

Loading

bsolomon1124 commented Oct 25, 2018 •

edited

Loading