Use of virtualenv-burrito results in a tainted PYTHONPATH element #707

stuhood · 2019-04-15T22:48:27Z

To reproduce:

launch a centos6 docker image:

#!/bin/bash
docker run \
  -v "$(pwd):/workdir" \
  -w '/workdir' \
  --rm \
  -it \
  pantsbuild/centos6:latest \
  /bin/bash

install virtualenv-burrito:

yum install -y unzip
curl -sL https://raw.githubusercontent.com/brainsik/virtualenv-burrito/master/virtualenv-burrito.sh | $SHELL

start a new environment:

mkvirtualenv example

pip install something in that environment:

pip install snakeviz

download a pantsbuild pants pex:

curl -L -O 'https://github.com/pantsbuild/pants/releases/download/release_1.15.0rc3/pants.1.15.0rc3.py27.pex'
chmod +x pants.1.15.0rc3.py27.pex

run the pex, and inspect verbose pex output:

PEX_VERBOSE=9 ./pants.1.15.0rc3.py27.pex

The result is:

pex: Please build pex with the subprocess32 module for more reliable requirement installation and interpreter execution.
pex: Selecting runtime interpreter based on pexrc: 0.1ms
pex: Found site-library: /root/.virtualenvs/example/lib/python2.7/site-packages
pex: Found site-library: /root/.virtualenvs/example/lib64/python2.7/site-packages
pex: Not a tainted path element: /workdir/virtualenv-test/pants.1.15.0rc3.py27.pex/.bootstrap
pex: Not a tainted path element: /workdir/virtualenv-test/pants.1.15.0rc3.py27.pex
pex: Not a tainted path element: /root/.venvburrito/lib/python2.7/site-packages
pex: Not a tainted path element: /opt/rh/devtoolset-7/root/usr/lib64/python2.6/site-packages
pex: Not a tainted path element: /opt/rh/devtoolset-7/root/usr/lib/python2.6/site-packages
pex: Not a tainted path element: /root/.venvburrito/lib/python2.7/site-packages/setuptools-40.8.0-py2.7.egg
pex: Not a tainted path element: /root/.venvburrito/lib/python2.7/site-packages/pip-19.0.2-py2.7.egg
pex: Not a tainted path element: /root/.virtualenvs/example/lib64/python27.zip
pex: Not a tainted path element: /root/.virtualenvs/example/lib64/python2.7
pex: Not a tainted path element: /root/.virtualenvs/example/lib64/python2.7/plat-linux2
pex: Not a tainted path element: /root/.virtualenvs/example/lib64/python2.7/lib-tk
pex: Not a tainted path element: /root/.virtualenvs/example/lib64/python2.7/lib-old
pex: Not a tainted path element: /root/.virtualenvs/example/lib64/python2.7/lib-dynload
pex: Not a tainted path element: /opt/rh/python27/root/usr/lib64/python2.7
pex: Not a tainted path element: /opt/rh/python27/root/usr/lib/python2.7
pex: Tainted path element: /root/.virtualenvs/example/lib/python2.7/site-packages
pex: Not a tainted path element: /opt/rh/devtoolset-7/root/usr/lib64/python2.6/site-packages
pex: Not a tainted path element: /opt/rh/devtoolset-7/root/usr/lib/python2.6/site-packages
pex: Scrubbing from user site: /root/.local/lib/python2.7/site-packages
pex: Scrubbing from site-packages: /root/.virtualenvs/example/lib/python2.7/site-packages
pex: Scrubbing from path_importer_cache: /root/.virtualenvs/example/lib/python2.7/site-packages
pex: Scrubbing from path_importer_cache: /root/.local/lib/python2.7/site-packages

...which shows entries for both ~/.venvburrito and ~/.virtualenvs.

The text was updated successfully, but these errors were encountered:

jsirois · 2019-04-21T01:20:45Z

The issue s triggered by exporting of PYTHONPATH here:
https://github.com/brainsik/virtualenv-burrito/blob/c6029a77b1f8ee9275171103138f8ecee7f0e9d3/virtualenv-burrito.py#L91-L103

Pex currently honors PYTHONPATH, only scrubbing site-packages, site-extras and user-site-packages as gleaned from the executing interpreter's sysconfig and site modules. Plain old virtualenv works fine - not messing with the PYTHONPATH.

Pex could learn about .venvburrito/lib/pythonX.Y/site-packages but that is not appealing. Thinking on other approaches here that do not disturb the very old contract of honoring PYTHONPATH. The right answer would seem to be to not honor PYTHONPATH unless PEX_INHERIT_PATH but the tendrils here are probably long and tangled. More thought is needed and considered input is appreciated as well.

jsirois · 2019-09-09T00:48:50Z

I hit an instance of this issue trying to fix up how we do pants integration tests to support the v2 python test rule better. This was enough to get me to look more closely and think about this more, I think I found a fix that does the right thing.

jsirois · 2019-09-09T02:23:45Z

OK, fix is now showing the following with a simpler version of your example that uses pex directly cutting out the Pants middleman:

$ docker run -v $PWD:/workdir --rm -it pantsbuild/centos6:latest /bin/bash
[root@b795e6ecc8fd /]# yum install -y unzip
[root@b795e6ecc8fd /]# curl -fsSL https://raw.githubusercontent.com/brainsik/virtualenv-burrito/master/virtualenv-burrito.sh | $SHELL
[root@b795e6ecc8fd /]# curl -fsSL https://github.com/pantsbuild/pex/releases/download/v1.6.10/pex -O
[root@b795e6ecc8fd /]# chmod +x pex
root@b795e6ecc8fd /]# ./pex requests -o requests.pex1.6.10
[root@b795e6ecc8fd /]# PEX_INTERPRETER=1 PEX_VERBOSE=9 ./requests.pex1.6.10 -c 'import snakeviz'
[root@b795e6ecc8fd /]# source /root/.venvburrito/startup.sh
[root@b795e6ecc8fd /]# mkvirtualenv example
(example) [root@b795e6ecc8fd /]# pip install snakeviz
(example) [root@b795e6ecc8fd /]# diff -u <(PEX_INTERPRETER=1 PEX_VERBOSE=9 ./requests.pex1.6.10 -c 'import setuptools' 2>&1) <(PEX_INTERPRETER=1 PEX_VERBOSE=9 ./requests.pexHEAD -c 'import setuptools' 2>&1)
--- /dev/fd/63	2019-09-09 02:17:17.801452820 +0000
+++ /dev/fd/62	2019-09-09 02:17:17.804786108 +0000
...
 pex: PYTHONPATH contains:
-pex:     /requests.pex1.6.10
-pex:     /root/.venvburrito/lib/python2.7/site-packages
-pex:   * /opt/rh/devtoolset-7/root/usr/lib64/python2.6/site-packages
-pex:   * /opt/rh/devtoolset-7/root/usr/lib/python2.6/site-packages
-pex:     /root/.venvburrito/lib/python2.7/site-packages/setuptools-40.8.0-py2.7.egg
-pex:     /root/.venvburrito/lib/python2.7/site-packages/pip-19.0.2-py2.7.egg
+pex:     /requests.pexHEAD
 pex:   * /root/.virtualenvs/example/lib/python27.zip
 pex:     /root/.virtualenvs/example/lib/python2.7
 pex:   * /root/.virtualenvs/example/lib/python2.7/plat-linux2
@@ -193,5 +201,15 @@
 pex:     /root/.pex/install/chardet-3.0.4-py2.py3-none-any.whl.c4332e34e38b781695ce775973fe40663558a897/chardet-3.0.4-py2.py3-none-any.whl
 pex:     /root/.pex/install/certifi-2019.6.16-py2.py3-none-any.whl.e7502e9b88c49c666599cb7a4cc64504fe1f9022/certifi-2019.6.16-py2.py3-none-any.whl
 pex:     /root/.pex/install/idna-2.8-py2.py3-none-any.whl.645e82ab4157606d15a8766060dd690deebcc4f2/idna-2.8-py2.py3-none-any.whl
-pex:   * /requests.pex1.6.10/.bootstrap
+pex:   * /requests.pexHEAD/.bootstrap
 pex:   * - paths that do not exist or will be imported via zipimport
+Traceback (most recent call last):
+  File ".bootstrap/pex/pex.py", line 374, in execute
+  File ".bootstrap/pex/pex.py", line 306, in _wrap_coverage
+  File ".bootstrap/pex/pex.py", line 337, in _wrap_profiling
+  File ".bootstrap/pex/pex.py", line 404, in _execute
+  File ".bootstrap/pex/pex.py", line 461, in execute_interpreter
+  File ".bootstrap/pex/pex.py", line 512, in execute_content
+  File "<exec_function>", line 4, in exec_function
+  File "-c <cmd>", line 1, in <module>
+ImportError: No module named setuptools

In particular:

diff -u <(PEX_INTERPRETER=1 PEX_VERBOSE=9 ./requests.pex1.6.10 -c 'import setuptools' 2>&1) <(PEX_INTERPRETER=1 PEX_VERBOSE=9 ./requests.pexHEAD -c 'import setuptools' 2>&1)
--- /dev/fd/63	2019-09-09 02:17:17.801452820 +0000
+++ /dev/fd/62	2019-09-09 02:17:17.804786108 +0000
...
 pex: Found site-library: /root/.virtualenvs/example/lib/python2.7/site-packages
-pex: Not a tainted path element: /requests.pex1.6.10/.bootstrap
-pex: Not a tainted path element: /requests.pex1.6.10
-pex: Not a tainted path element: /root/.venvburrito/lib/python2.7/site-packages
-pex: Not a tainted path element: /opt/rh/devtoolset-7/root/usr/lib64/python2.6/site-packages
-pex: Not a tainted path element: /opt/rh/devtoolset-7/root/usr/lib/python2.6/site-packages
-pex: Not a tainted path element: /root/.venvburrito/lib/python2.7/site-packages/setuptools-40.8.0-py2.7.egg
-pex: Not a tainted path element: /root/.venvburrito/lib/python2.7/site-packages/pip-19.0.2-py2.7.egg
+pex: Not a tainted path element: /requests.pexHEAD/.bootstrap
+pex: Not a tainted path element: /requests.pexHEAD
 pex: Not a tainted path element: /root/.virtualenvs/example/lib/python27.zip
 pex: Not a tainted path element: /root/.virtualenvs/example/lib/python2.7
 pex: Not a tainted path element: /root/.virtualenvs/example/lib/python2.7/plat-linux2
@@ -35,60 +46,62 @@
 pex: Not a tainted path element: /pyenv-docker-build/versions/2.7.13/lib/python2.7/plat-linux2
 pex: Not a tainted path element: /pyenv-docker-build/versions/2.7.13/lib/python2.7/lib-tk
 pex: Tainted path element: /root/.virtualenvs/example/lib/python2.7/site-packages
-pex: Not a tainted path element: /opt/rh/devtoolset-7/root/usr/lib64/python2.6/site-packages
-pex: Not a tainted path element: /opt/rh/devtoolset-7/root/usr/lib/python2.6/site-packages
 pex: Scrubbing from user site: /root/.local/lib/python2.7/site-packages
 pex: Scrubbing from site-packages: /root/.virtualenvs/example/lib/python2.7/site-packages
+pex: Extracted user PYTHONPATH of /root/.venvburrito/lib/python2.7/site-packages:/opt/rh/devtoolset-7/root/usr/lib64/python2.6/site-packages:/opt/rh/devtoolset-7/root/usr/lib/python2.6/site-packages from original PYTHONPATH of /root/.venvburrito/lib/python2.7/site-packages:/opt/rh/devtoolset-7/root/usr/lib64/python2.6/site-packages:/opt/rh/devtoolset-7/root/usr/lib/python2.6/site-packages
+pex: Scrubbing user PYTHONPATH element: /root/.venvburrito/lib/python2.7/site-packages
+pex: Scrubbing user PYTHONPATH element: /opt/rh/devtoolset-7/root/usr/lib64/python2.6/site-packages
+pex: Scrubbing user PYTHONPATH element: /opt/rh/devtoolset-7/root/usr/lib/python2.6/site-packages
 pex: Scrubbing from path_importer_cache: /root/.virtualenvs/example/lib/python2.7/site-packages
 pex: Scrubbing from path_importer_cache: /root/.local/lib/python2.7/site-packages
...
 pex: PYTHONPATH contains:
-pex:     /requests.pex1.6.10
-pex:     /root/.venvburrito/lib/python2.7/site-packages
-pex:   * /opt/rh/devtoolset-7/root/usr/lib64/python2.6/site-packages
-pex:   * /opt/rh/devtoolset-7/root/usr/lib/python2.6/site-packages
-pex:     /root/.venvburrito/lib/python2.7/site-packages/setuptools-40.8.0-py2.7.egg
-pex:     /root/.venvburrito/lib/python2.7/site-packages/pip-19.0.2-py2.7.egg
+pex:     /requests.pexHEAD
 pex:   * /root/.virtualenvs/example/lib/python27.zip
 pex:     /root/.virtualenvs/example/lib/python2.7
 pex:   * /root/.virtualenvs/example/lib/python2.7/plat-linux2
@@ -193,5 +201,15 @@
 pex:     /root/.pex/install/chardet-3.0.4-py2.py3-none-any.whl.c4332e34e38b781695ce775973fe40663558a897/chardet-3.0.4-py2.py3-none-any.whl
 pex:     /root/.pex/install/certifi-2019.6.16-py2.py3-none-any.whl.e7502e9b88c49c666599cb7a4cc64504fe1f9022/certifi-2019.6.16-py2.py3-none-any.whl
 pex:     /root/.pex/install/idna-2.8-py2.py3-none-any.whl.645e82ab4157606d15a8766060dd690deebcc4f2/idna-2.8-py2.py3-none-any.whl
-pex:   * /requests.pex1.6.10/.bootstrap
+pex:   * /requests.pexHEAD/.bootstrap
 pex:   * - paths that do not exist or will be imported via zipimport
+Traceback (most recent call last):
+  File ".bootstrap/pex/pex.py", line 374, in execute
+  File ".bootstrap/pex/pex.py", line 306, in _wrap_coverage
+  File ".bootstrap/pex/pex.py", line 337, in _wrap_profiling
+  File ".bootstrap/pex/pex.py", line 404, in _execute
+  File ".bootstrap/pex/pex.py", line 461, in execute_interpreter
+  File ".bootstrap/pex/pex.py", line 512, in execute_content
+  File "<exec_function>", line 4, in exec_function
+  File "-c <cmd>", line 1, in <module>
+ImportError: No module named setuptools

Previously Pex would inherit `PYTHONPATH` by default despite the default `--inherit-path` setting being false. This was buggy behavior running against the primary Pex goal of providing an isolated execution environment. Now, to use PYTHONPATH to ammend the `sys.path` of a running is accomplished by either of: ``` PEX_INHERIT_PATH=prefer PYTHONPATH=... ./my.pex PEX_INHERIT_PATH=fallback PYTHONPATH=... ./my.pex ``` Update corresponding settings documentation and fix pex to consider PYTHONPATH when adjusting sys.path. Fixes pex-tool#707

This enables uniform execution of Pants in a subprocess whether running in pantsbuild/pants from our custom venv, from a pants PEX or from a pantsbuild/setup venv. In order to support this mode, apply the `PYTHONPATH` scrubbing hack in `PytestRun` to `PythonRun` and `PythonRepl` as well pending a fix for: pex-tool/pex#707

stuhood · 2019-09-10T06:25:24Z

Awesome! Thanks a ton for diving in here.

Previously `PEX` would inherit `PYTHONPATH` by default despite the default `--inherit-path` setting being false. This was buggy behavior running against a primary Pex goal of providing an isolated execution environment. Now, in order to use `PYTHONPATH` to ammend the `sys.path` of a running `PEX` a `PEX_INHERIT_PATH` (`pex --inherit-path=...`) of `prefer` or `fallback` must be specified. Update corresponding settings documentation and fix `PEX` to consider `PYTHONPATH` when adjusting sys.path. Fixes #707

This enables uniform execution of Pants in a subprocess whether running in pantsbuild/pants from our custom venv, from a pants PEX or from a pantsbuild/setup venv. In order to support this mode, apply the `PYTHONPATH` scrubbing hack in `PytestRun` to `PythonRun` and `PythonRepl` as well pending a fix for: pex-tool/pex#707

stuhood added the bug label Apr 15, 2019

jsirois self-assigned this Apr 21, 2019

jsirois added the in progress label Apr 21, 2019

jsirois removed the in progress label Apr 21, 2019

jsirois added the in progress label Sep 9, 2019

jsirois mentioned this issue Sep 9, 2019

Error attempting to import local files as packages #340

Closed

jsirois mentioned this issue Sep 9, 2019

Include PYTHONPATH in --inherit-path logic. #765

Merged

jsirois mentioned this issue Sep 9, 2019

Invoke Pants via module in ITs. pantsbuild/pants#8265

Merged

jsirois closed this as completed in #765 Sep 10, 2019

jsirois removed the in progress label Sep 10, 2019

jsirois mentioned this issue Sep 13, 2019

Upgrade to Pex 1.6.11. pantsbuild/pants#8287

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use of virtualenv-burrito results in a tainted PYTHONPATH element #707

Use of virtualenv-burrito results in a tainted PYTHONPATH element #707

stuhood commented Apr 15, 2019 •

edited

Loading

jsirois commented Apr 21, 2019

jsirois commented Sep 9, 2019

jsirois commented Sep 9, 2019 •

edited

Loading

stuhood commented Sep 10, 2019

Use of virtualenv-burrito results in a tainted PYTHONPATH element #707

Use of virtualenv-burrito results in a tainted PYTHONPATH element #707

Comments

stuhood commented Apr 15, 2019 • edited Loading

jsirois commented Apr 21, 2019

jsirois commented Sep 9, 2019

jsirois commented Sep 9, 2019 • edited Loading

stuhood commented Sep 10, 2019

stuhood commented Apr 15, 2019 •

edited

Loading

jsirois commented Sep 9, 2019 •

edited

Loading