New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor data fetcher selection in loops #18494

Merged

awaelchli merged 4 commits into master from refactor/select-data-fetcher

Sep 12, 2023

Contributor

awaelchli commented Sep 6, 2023 •

edited by github-actions bot

Loading

What does this PR do?

Addresses #18441 (comment)
Follow up to #18441
This was suggested by @carmocca

📚 Documentation preview 📚: https://pytorch-lightning--18494.org.readthedocs.build/en/18494/

cc @Borda @justusschock @awaelchli @carmocca


          refactor data fetcher selection

e4e3132

github-actions bot added the pl label

awaelchli added the bug label

awaelchli added this to the 2.0.x milestone

awaelchli removed the bug label

awaelchli modified the milestones: 2.0.x, 2.1

awaelchli added refactor loops labels


          Merge branch 'master' into refactor/select-data-fetcher

3bc58bb

awaelchli marked this pull request as ready for review

September 7, 2023 15:54

awaelchli requested review from williamFalcon, carmocca and justusschock as code owners

September 7, 2023 15:54

Contributor

github-actions bot commented Sep 7, 2023 •

edited

Loading

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow

Check ID	Status
pl-cpu (macOS-11, lightning, 3.8, 1.11)	success	✅
pl-cpu (macOS-11, lightning, 3.9, 1.12)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 1.13)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
pl-cpu (macOS-11, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.9, 1.12)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 1.11)	success	✅
pl-cpu (windows-2022, lightning, 3.9, 1.12)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 1.13)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 1.11, oldest)	success	✅
pl-cpu (macOS-11, pytorch, 3.8, 1.13)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.13)	success	✅
pl-cpu (windows-2022, pytorch, 3.8, 1.13)	success	✅
pl-cpu (macOS-12, pytorch, 3.11, 2.0)	success	✅
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.0)	success	✅
pl-cpu (windows-2022, pytorch, 3.11, 2.0)	success	✅

These checks are required after the changes to src/lightning/pytorch/loops/evaluation_loop.py, src/lightning/pytorch/loops/fit_loop.py, src/lightning/pytorch/loops/prediction_loop.py, src/lightning/pytorch/loops/utilities.py.

🟢 pytorch_lightning: Azure GPU

Check ID	Status
[pytorch-lightning (GPUs) (testing Lightning	latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=173560&view=logs&jobId=47e66f3c-897a-5428-da11-bf5c7745762e)	success
[pytorch-lightning (GPUs) (testing PyTorch	latest)](https://dev.azure.com/Lightning-AI/72ab7ed8-b00f-4b6e-b131-3388f7ffafa7/_build/results?buildId=173560&view=logs&jobId=3f274fac-2e11-54ca-487e-194c91f3ae9f)	success

These checks are required after the changes to src/lightning/pytorch/loops/evaluation_loop.py, src/lightning/pytorch/loops/fit_loop.py, src/lightning/pytorch/loops/prediction_loop.py, src/lightning/pytorch/loops/utilities.py.

🟢 pytorch_lightning: Benchmarks

Check ID	Status
lightning.Benchmarks	success	✅

These checks are required after the changes to src/lightning/pytorch/loops/evaluation_loop.py, src/lightning/pytorch/loops/fit_loop.py, src/lightning/pytorch/loops/prediction_loop.py, src/lightning/pytorch/loops/utilities.py.

🟢 pytorch_lightning: Docs

Check ID	Status
docs-make (pytorch, doctest)	success	✅
docs-make (pytorch, html)	success	✅

These checks are required after the changes to src/lightning/pytorch/loops/evaluation_loop.py, src/lightning/pytorch/loops/fit_loop.py, src/lightning/pytorch/loops/prediction_loop.py, src/lightning/pytorch/loops/utilities.py.

🟢 mypy

Check ID	Status
mypy	success	✅

These checks are required after the changes to src/lightning/pytorch/loops/evaluation_loop.py, src/lightning/pytorch/loops/fit_loop.py, src/lightning/pytorch/loops/prediction_loop.py, src/lightning/pytorch/loops/utilities.py.

🟢 install

Check ID	Status
install-pkg (ubuntu-22.04, app, 3.8)	success	✅
install-pkg (ubuntu-22.04, app, 3.11)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.8)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.11)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.8)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.11)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.8)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.11)	success	✅
install-pkg (ubuntu-22.04, notset, 3.8)	success	✅
install-pkg (ubuntu-22.04, notset, 3.11)	success	✅
install-pkg (macOS-12, app, 3.8)	success	✅
install-pkg (macOS-12, app, 3.11)	success	✅
install-pkg (macOS-12, fabric, 3.8)	success	✅
install-pkg (macOS-12, fabric, 3.11)	success	✅
install-pkg (macOS-12, pytorch, 3.8)	success	✅
install-pkg (macOS-12, pytorch, 3.11)	success	✅
install-pkg (macOS-12, lightning, 3.8)	success	✅
install-pkg (macOS-12, lightning, 3.11)	success	✅
install-pkg (macOS-12, notset, 3.8)	success	✅
install-pkg (macOS-12, notset, 3.11)	success	✅
install-pkg (windows-2022, app, 3.8)	success	✅
install-pkg (windows-2022, app, 3.11)	success	✅
install-pkg (windows-2022, fabric, 3.8)	success	✅
install-pkg (windows-2022, fabric, 3.11)	success	✅
install-pkg (windows-2022, pytorch, 3.8)	success	✅
install-pkg (windows-2022, pytorch, 3.11)	success	✅
install-pkg (windows-2022, lightning, 3.8)	success	✅
install-pkg (windows-2022, lightning, 3.11)	success	✅
install-pkg (windows-2022, notset, 3.8)	success	✅
install-pkg (windows-2022, notset, 3.11)	success	✅

These checks are required after the changes to src/lightning/pytorch/loops/evaluation_loop.py, src/lightning/pytorch/loops/fit_loop.py, src/lightning/pytorch/loops/prediction_loop.py, src/lightning/pytorch/loops/utilities.py.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

carmocca approved these changes

View reviewed changes

Contributor

carmocca left a comment

Thank you!

You'll need to assert that the stage is not None when accessed trough trainer.state.stage (see mypy failure)

fix

64e8ac3

codecov bot commented Sep 7, 2023 •

edited

Loading

Codecov Report

Merging #18494 (c1418a4) into master (cc18bd7) will decrease coverage by 35%.
Report is 15 commits behind head on master.
The diff coverage is 100%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #18494      +/-   ##
==========================================
- Coverage      85%      50%     -35%     
==========================================
  Files         427      419       -8     
  Lines       33365    33218     -147     
==========================================
- Hits        28370    16684   -11686     
- Misses       4995    16534   +11539


          mypy

c1418a4

awaelchli requested a review from Borda

September 7, 2023 16:31

Borda approved these changes

View reviewed changes

mergify bot added the ready label

awaelchli merged commit 1bee50b into master

89 checks passed

awaelchli deleted the refactor/select-data-fetcher branch

September 12, 2023 11:25

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

carmocca carmocca approved these changes

Borda Borda approved these changes

williamFalcon Awaiting requested review from williamFalcon

justusschock Awaiting requested review from justusschock justusschock is a code owner

Labels

loops pl ready refactor