Fixed PYTHONPATH for ddp test model #4528
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gianscarpe
requested review from
ananyahjha93,
awaelchli,
Borda,
justusschock,
nateraw,
SeanNaren,
tchaton,
teddykoker and
williamFalcon
as code owners
Codecov Report
@@ Coverage Diff @@
## master #4528 +/- ##
======================================
Coverage 93% 93%
======================================
Files 129 129
Lines 9359 9359
======================================
Hits 8677 8677
Misses 682 682 |
Borda
reviewed
Nov 5, 2020
tests/utilities/dist.py
Outdated
| @@ -29,11 +29,10 @@ def call_training_script(module_file, cli_args, method, tmpdir, timeout=60): | |||
|
|
|||
| # need to set the PYTHONPATH in case pytorch_lightning was not installed into the environment | |||
| env = os.environ.copy() | |||
| env['PYTHONPATH'] = f'{pytorch_lightning.__file__}:' + env.get('PYTHONPATH', '') | |||
| env['PYTHONPATH'] = env.get('PYTHONPATH', '') + f'{pytorch_lightning.__file__}:' | |||
There was a problem hiding this comment.
I am not sure why the self-file shall not be the first one... ?
There was a problem hiding this comment.
In my setup, I installed pytorch-lightning and its dependencies. When test_model is launched as subprocess, it uses tests module from the virtualenv instead of the local 'tests' as supposed (and PYTHONPATH contains local pytorch-lightning first)
|
@tchaton @awaelchli mind check on this DDP? |
|
@gianscarpe mind resolve conflicts? |
…torch-lightning into bugfix/3827_test_ddp_error
|
Yes, verified, this should be fine :) |
awaelchli
approved these changes
Dec 3, 2020
SeanNaren
approved these changes
Dec 5, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
I found an issue related to PYTHONPATH env when launching test_ddp.py. In particular, the pytorch-lightning local directory should be placed before the installation directory to access
testsdirectory locally. This solved the issue and tests now passFixes #3827
Before submitting
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Yes! I want to continue improving pytorch-lightning and cooperate!🙃