Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tox parallelized subprocesses should remove TOX_PARALLEL_ENV from passenv #1275

Closed
rpkilby opened this issue Apr 23, 2019 · 1 comment
Closed
Labels
bug:normal affects many people or has quite an impact help:wanted Issues that have been acknowledged, a solution determined and a PR might likely be accepted.

Comments

@rpkilby
Copy link
Member

rpkilby commented Apr 23, 2019

I don't know if this is worth tackling, as this is an edge case that should only realistically affect plugin authors. In short, the TOX_PARALLEL_ENV is always added to the passenv, which is then propagated to the downstream tox/python subprocesses. Normally, this shouldn't matter, since most tox users won't care about the env var. However, this affects test cases that in turn invoke tox itself (i.e., integration tests for plugins like tox-factor and tox-travis). I think this issue should also mean that you can't run tox's test suite in parallel.

As an example, take a potential integration test. It would setup a fake project with a tox config, then run some tox test command. The process tree would look something like:

tox 
 └ python test command
    └ tox test command

However, if we run this test suite in parallel, the internal tox invocation will already have TOX_PARALLEL_ENV set, which is propagated to the various subprocesses, affecting the internal tox test. e.g., it looks something like this:

tox --parallel auto
 └ TOX_PARALLEL_ENV=py37 tox 
    └ TOX_PARALLEL_ENV=py37 python test command
       └ TOX_PARALLEL_ENV=py37 tox test command

Instead of running normally, the nested tox test command thinks it's running as a parallelized subprocess. I think this can be fixed by having the parallelized tox subprocess remove the TOX_PARALLEL_ENV from either its environment variables, or from its subprocess's environment variables.


My current workaround is to modify the subprocess environ in my own code. e.g., my plugin tests invoke tox like so:

    def _tox_call(self, arguments):
        env = os.environ.copy()
        env.pop(TOX_PARALLEL_ENV, None)

        proc = subprocess.Popen(
            arguments, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)
        stdout, stderr = proc.communicate()

        return proc.returncode, stdout.decode('utf-8'), stderr.decode('utf-8')
@rpkilby rpkilby added the bug:normal affects many people or has quite an impact label Apr 23, 2019
@rpkilby rpkilby changed the title Tox parallelized subprocesses should remove PARALLEL_ENV_VAR_KEY from passenv tox parallelized subprocesses should remove PARALLEL_ENV_VAR_KEY from passenv Apr 23, 2019
@rpkilby rpkilby changed the title tox parallelized subprocesses should remove PARALLEL_ENV_VAR_KEY from passenv tox parallelized subprocesses should remove TOX_PARALLEL_ENV from passenv Apr 23, 2019
@gaborbernat gaborbernat added the help:wanted Issues that have been acknowledged, a solution determined and a PR might likely be accepted. label Apr 24, 2019
@gaborbernat
Copy link
Member

TOX_PARALLEL_ENV has been removed with tox 4 so it's no longer a problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug:normal affects many people or has quite an impact help:wanted Issues that have been acknowledged, a solution determined and a PR might likely be accepted.
Projects
None yet
Development

No branches or pull requests

2 participants