diff --git a/.github/BECOMING_A_CORE_CONTRIBUTOR.md b/.github/BECOMING_A_CORE_CONTRIBUTOR.md index a5e7d3830bdd9..2fe58ff726c25 100644 --- a/.github/BECOMING_A_CORE_CONTRIBUTOR.md +++ b/.github/BECOMING_A_CORE_CONTRIBUTOR.md @@ -6,22 +6,26 @@ We're currently recruiting for a team of 5 core maintainers. As a core maintainer you will have a strong say in the direction of the project. Big changes will require a majority of maintainers to agree. ## Code of conduct + First and foremost, you'll be evaluated against [these core values](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/.github/CONTRIBUTING.md). Any code we commit or feature we add needs to align with those core values. ## The bar for joining the team + Lightning is being used to solve really hard problems at the top AI labs in the world. As such, the bar for adding team members is extremely high. Candidates must have solid engineering skills, have a good eye for user experience, and must be a power user of Lightning and PyTorch. With that said, the Lightning team will be diverse and a reflection of an inclusive AI community. You don't have to be an engineer to contribute! Scientists with great usability intuition and PyTorch ninja skills are welcomed! ## Responsibilities: + The responsibilities mainly revolve around 3 things. ### Github issues + - Here we want to help users have an amazing experience. These range from questions from new people getting into DL to questions from researchers about doing something esoteric with Lightning -Often, these issues require some sort of bug fix, document clarification or new functionality to be scoped out. + Often, these issues require some sort of bug fix, document clarification or new functionality to be scoped out. - To become a core member you must resolve at least 10 Github issues which align with the API design goals for Lightning. By the end of these 10 issues I should feel comfortable in the way you answer user questions -Pleasant/helpful tone. + Pleasant/helpful tone. - Can abstract from that issue or bug into functionality that might solve other related issues or makes the platform more flexible. @@ -37,22 +41,25 @@ Pleasant/helpful tone. - Ask yourself, could a non-engineer understand whatโ€™s happening here? - Make sure new tests are written - Is this NECESSARY for Lightning? There are some PRs which are just purely about adding engineering complexity which have no place in Lightning. -Guidance + Guidance - Some other PRs are for people who are wanting to get involved and add something unnecessary. We do want their help though! So donโ€™t approve the PR, but direct them to a Github issue that they might be interested in helping with instead! - To be considered for core contributor, please review 10 PRs and help the authors land it on master. Once you've finished the review, ping me -for a sanity check. At the end of 10 PRs if your PR reviews are inline with expectations described above, then you can merge PRs on your own going forward, -otherwise we'll do a few more until we're both comfortable :) + for a sanity check. At the end of 10 PRs if your PR reviews are inline with expectations described above, then you can merge PRs on your own going forward, + otherwise we'll do a few more until we're both comfortable :) ### Project directions + There are some big decisions which the project must make. For these I expect core contributors to have something meaningful to add if itโ€™s their area of expertise. ### Diversity + Lightning should reflect the broader community it serves. As such we should have scientists/researchers from different fields contributing! The first 5 core contributors will fit this profile. Thus if you overlap strongly with experiences and expertise as someone else on the team, you might have to wait until the next set of contributors are added. ### Summary: Requirements to apply + The goal is to be inline with expectations for solving issues by the last one so you can do them on your own. If not, I might ask you to solve a few more specific ones. - Solve 10+ Github issues. diff --git a/.github/CODE_OF_CONDUCT.md b/.github/CODE_OF_CONDUCT.md index c3e3b0ecf9997..94333b0479e06 100644 --- a/.github/CODE_OF_CONDUCT.md +++ b/.github/CODE_OF_CONDUCT.md @@ -14,22 +14,22 @@ appearance, race, religion, or sexual identity and orientation. Examples of behavior that contributes to creating a positive environment include: -* Using welcoming and inclusive language -* Being respectful of differing viewpoints and experiences -* Gracefully accepting constructive criticism -* Focusing on what is best for the community -* Showing empathy towards other community members +- Using welcoming and inclusive language +- Being respectful of differing viewpoints and experiences +- Gracefully accepting constructive criticism +- Focusing on what is best for the community +- Showing empathy towards other community members Examples of unacceptable behavior by participants include: -* The use of sexualized language or imagery and unwelcome sexual attention or - advances -* Trolling, insulting/derogatory comments, and personal or political attacks -* Public or private harassment -* Publishing others' private information, such as a physical or electronic - address, without explicit permission -* Other conduct which could reasonably be considered inappropriate in a - professional setting +- The use of sexualized language or imagery and unwelcome sexual attention or + advances +- Trolling, insulting/derogatory comments, and personal or political attacks +- Public or private harassment +- Publishing others' private information, such as a physical or electronic + address, without explicit permission +- Other conduct which could reasonably be considered inappropriate in a + professional setting ## Our Responsibilities @@ -70,7 +70,7 @@ members of the project's leadership. This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html -[homepage]: https://www.contributor-covenant.org - For answers to common questions about this code of conduct, see https://www.contributor-covenant.org/faq + +[homepage]: https://www.contributor-covenant.org diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index ee9706172e2ac..05461baf81e0b 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -56,7 +56,7 @@ As a researcher, you can't have any part of your code going wrong. So, make thor Have a favorite feature from other libraries like fast.ai or transformers? Those should just work with lightning as well. Grab your favorite model or learning rate scheduler from your favorite library and run it in Lightning. ---- +______________________________________________________________________ ## Contribution Types @@ -73,26 +73,28 @@ A lot of good work has already been done in project mechanics (requirements.txt, - Add details on how to reproduce the issue - a minimal test case is always best, colab is also great. Note, that the sample code shall be minimal and if needed with publicly available data. -2. Try to fix it or recommend a solution. We highly recommend to use test-driven approach: +1. Try to fix it or recommend a solution. We highly recommend to use test-driven approach: - Convert your minimal code example to a unit/integration test with assert on expected results. - Start by debugging the issue... You can run just this particular test in your IDE and draft a fix. - Verify that your test case fails on the master branch and only passes with the fix applied. -3. Submit a PR! +1. Submit a PR! -_**Note**, even if you do not find the solution, sending a PR with a test covering the issue is a valid contribution, and we can help you or finish it with you :]_ +_**Note**, even if you do not find the solution, sending a PR with a test covering the issue is a valid contribution, and we can help you or finish it with you :\]_ ### New Features: 1. Submit a GitHub issue - describe what is the motivation of such feature (adding the use case, or an example is helpful). -2. Determine the feature scope with us. -3. Submit a PR! We recommend test driven approach to adding new features as well: + +1. Determine the feature scope with us. + +1. Submit a PR! We recommend test driven approach to adding new features as well: - Write a test for the functionality you want to add. - Write the functional code until the test passes. -4. Add/update the relevant tests! +1. Add/update the relevant tests! - [This PR](https://github.com/PyTorchLightning/pytorch-lightning/pull/2671) is a good example for adding a new metric, and [this one for a new logger](https://github.com/PyTorchLightning/pytorch-lightning/pull/2721). @@ -102,12 +104,14 @@ Want to keep Lightning healthy? Love seeing those green tests? So do we! How to Most of the tests in PyTorch Lightning train a trial MNIST model under various trainer conditions (ddp, ddp2+amp, etc...). The tests expect the model to perform to a reasonable degree of testing accuracy to pass. Want to add a new test case and not sure how? [Talk to us!](https://join.slack.com/t/pytorch-lightning/shared_invite/zt-pw5v393p-qRaDgEk24~EjiZNBpSQFgQ) ---- +______________________________________________________________________ ## Guidelines ### Developments scripts + To build the documentation locally, simply execute the following commands from project root (only for Unix): + - `make clean` cleans repo from temp/generated files - `make docs` builds documentation under _docs/build/html_ - `make test` runs all project's tests with coverage @@ -121,7 +125,7 @@ In case you adding new dependencies, make sure that they are compatible with the ### Coding Style 1. Use f-strings for output formation (except logging when we stay with lazy `logging.info("Hello %s!", name)`. -2. You can use `pre-commit` to make sure your code style is correct. +1. You can use `pre-commit` to make sure your code style is correct. ### Documentation @@ -222,18 +226,18 @@ We welcome any useful contribution! For your convenience here's a recommended wo - Create a branch and prepare your changes. - Tip: do not work with your master directly, it may become complicated when you need to rebase. - Tip: give your PR a good name! It will be useful later when you may work on multiple tasks/PRs. -2. Test your code! +1. Test your code! - It is always good practice to start coding by creating a test case, verifying it breaks with current behaviour, and passes with your new changes. - Make sure your new tests cover all different edge cases. - Make sure all exceptions are handled. -3. Create a "Draft PR" which is clearly marked, to let us know you don't need feedback yet. -4. When you feel ready for integrating your work, mark your PR "Ready for review". +1. Create a "Draft PR" which is clearly marked, to let us know you don't need feedback yet. +1. When you feel ready for integrating your work, mark your PR "Ready for review". - Your code should be readable and follow the project's design principles. - Make sure all tests are passing. - Make sure you add a GitHub issue to your PR. -5. Use tags in PR name for following cases: - - **[blocked by #]** if your work is dependent on other PRs. - - **[wip]** when you start to re-edit your work, mark it so no one will accidentally merge it in meantime. +1. Use tags in PR name for following cases: + - **\[blocked by #\]** if your work is dependent on other PRs. + - **\[wip\]** when you start to re-edit your work, mark it so no one will accidentally merge it in meantime. ### Question & Answer @@ -274,20 +278,21 @@ git rebase upstream/master git push -f ``` -#### How to add new tests?** +#### How to add new tests? We are using [pytest](https://docs.pytest.org/en/stable/) in Pytorch Lightning. Here are tutorials: -* (recommended) [Visual Testing with pytest](https://www.youtube.com/playlist?list=PLCTHcU1KoD99Rim2tzg-IhYY2iu9FFvNo) from JetBrains on YouTube -* [Effective Python Testing With Pytest](https://realpython.com/pytest-python-testing/) article on realpython.com + +- (recommended) [Visual Testing with pytest](https://www.youtube.com/playlist?list=PLCTHcU1KoD99Rim2tzg-IhYY2iu9FFvNo) from JetBrains on YouTube +- [Effective Python Testing With Pytest](https://realpython.com/pytest-python-testing/) article on realpython.com Here is the process to create a new test -* 0. Optional: Follow tutorials ! -* 1. Find a file in tests/ which match what you want to test. If none, create one. -* 2. Use this template to get started ! -* 3. Use `BoringModel and derivates to test out your code`. +- 0. Optional: Follow tutorials ! +- 1. Find a file in tests/ which match what you want to test. If none, create one. +- 2. Use this template to get started ! +- 3. Use `BoringModel and derivates to test out your code`. ```python # TEST SHOULD BE IN YOUR FILE: tests/..../...py @@ -320,47 +325,49 @@ def test_explain_what_is_being_tested(tmpdir): # assert the behaviour is correct. assert ... ``` + run our/your test with + ```bash python -m pytest tests/..../...py::test_explain_what_is_being_tested --verbose --capture=no ``` - #### How to fix PR with mixed base and target branches? Sometimes you start your PR as a bug-fix but it turns out to be more of a feature (or the other way around). Do not panic, the solution is very straightforward and quite simple. All you need to do are these two steps in arbitrary order: - - Ask someone from Core to change the base/target branch to the correct one - - Rebase or cherry-pick your commits onto the correct base branch... + +- Ask someone from Core to change the base/target branch to the correct one +- Rebase or cherry-pick your commits onto the correct base branch... Let's show how to deal with the git... the sample case is moving a PR from `master` to `release/1.2-dev` assuming my branch name is `my-branch` and the last true master commit is `ccc111` and your first commit is `mmm222`. - * **Cherry-picking** way - ```bash - git checkout my-branch - # create a local backup of your branch - git checkout -b my-branch-backup - # reset your branch to the correct base - git reset release/1.2-dev --hard - # ACTION: this step is much easier to do with IDE - # so open one and cherry-pick your last commits from `my-branch-backup` - # resolve all eventual conflict as the new base may contain different code - # when all done, push back to the open PR - git push -f - ``` - * **Rebasing way**, see more about [rebase onto usage](https://womanonrails.com/git-rebase-onto) - ```bash - git checkout my-branch - # rebase your commits on the correct branch - git rebase --onto release/1.2-dev ccc111 - # if there is no collision you shall see just success - # eventually you would need to resolve collision and in such case follow the instruction in terminal - # when all done, push back to the open PR - git push -f - ``` +- **Cherry-picking** way + ```bash + git checkout my-branch + # create a local backup of your branch + git checkout -b my-branch-backup + # reset your branch to the correct base + git reset release/1.2-dev --hard + # ACTION: this step is much easier to do with IDE + # so open one and cherry-pick your last commits from `my-branch-backup` + # resolve all eventual conflict as the new base may contain different code + # when all done, push back to the open PR + git push -f + ``` +- **Rebasing way**, see more about [rebase onto usage](https://womanonrails.com/git-rebase-onto) + ```bash + git checkout my-branch + # rebase your commits on the correct branch + git rebase --onto release/1.2-dev ccc111 + # if there is no collision you shall see just success + # eventually you would need to resolve collision and in such case follow the instruction in terminal + # when all done, push back to the open PR + git push -f + ``` ### Bonus Workflow Tip diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 92f292d7bbb56..3a94ef6758910 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -4,8 +4,8 @@ about: Create a report to help us improve title: '' labels: bug / fix, help wanted assignees: '' - --- + ## ๐Ÿ› Bug @@ -46,15 +46,15 @@ python collect_env_details.py You can also fill out the list below manually. --> - - PyTorch Lightning Version (e.g., 1.3.0): - - PyTorch Version (e.g., 1.8) - - Python version: - - OS (e.g., Linux): - - CUDA/cuDNN version: - - GPU models and configuration: - - How you installed PyTorch (`conda`, `pip`, source): - - If compiling from source, the output of `torch.__config__.show()`: - - Any other relevant information: +- PyTorch Lightning Version (e.g., 1.3.0): +- PyTorch Version (e.g., 1.8) +- Python version: +- OS (e.g., Linux): +- CUDA/cuDNN version: +- GPU models and configuration: +- How you installed PyTorch (`conda`, `pip`, source): +- If compiling from source, the output of `torch.__config__.show()`: +- Any other relevant information: ### Additional context diff --git a/.github/ISSUE_TEMPLATE/documentation.md b/.github/ISSUE_TEMPLATE/documentation.md index 363a0a54bd130..a6a84c1351e77 100644 --- a/.github/ISSUE_TEMPLATE/documentation.md +++ b/.github/ISSUE_TEMPLATE/documentation.md @@ -4,7 +4,6 @@ about: Typos and doc fixes title: '' labels: typo, documentation assignees: '' - --- ## ๐Ÿ“š Documentation @@ -12,8 +11,8 @@ assignees: '' For typos and doc fixes, please go ahead and: 1. Create an issue. -2. Fix the typo. -3. Submit a PR. +1. Fix the typo. +1. Submit a PR. For very simple fixes, you can submit a PR without a linked issue. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index c73cda501eaba..fd91ee6f64471 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -4,7 +4,6 @@ about: Suggest an idea for this project title: '' labels: enhancement, help wanted assignees: '' - --- ## ๐Ÿš€ Feature @@ -27,7 +26,7 @@ assignees: '' ---- +______________________________________________________________________ #### If you enjoy Lightning, check out our other projects! โšก diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 5603dd70ce7b5..2d6c957870d49 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -9,7 +9,8 @@ If we didn't discuss your PR in Github issues there's a high chance it will not The following links the related issue to the PR (https://docs.github.com/en/free-pro-team@latest/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) --> -Fixes # + +Fixes #\ ### Does your PR introduce any breaking changes? If yes, please list them. @@ -25,6 +26,7 @@ Fixes # - [ ] Did you verify new and **existing tests pass** locally with your changes? - [ ] Did you list all the **breaking changes** introduced by this pull request? - [ ] Did you **update the [CHANGELOG](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/CHANGELOG.md)**? (not for typos, docs, test updates, or internal minor changes/refactorings) + ## PR review @@ -32,10 +34,10 @@ Fixes # Anyone in the community is welcome to review the PR. Before you start reviewing make sure you have read [Review guidelines](https://github.com/PyTorchLightning/pytorch-lightning/wiki/Review-guidelines). In short, see the following bullet-list: - - [ ] Is this pull request ready for review? (if not, please submit in draft mode) - - [ ] Check that all items from **Before submitting** are resolved - - [ ] Make sure the title is self-explanatory and the description concisely explains the PR - - [ ] Add labels and milestones (and optionally projects) to the PR so it can be classified +- [ ] Is this pull request ready for review? (if not, please submit in draft mode) +- [ ] Check that all items from **Before submitting** are resolved +- [ ] Make sure the title is self-explanatory and the description concisely explains the PR +- [ ] Add labels and milestones (and optionally projects) to the PR so it can be classified ## Did you have fun? diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 682fac3570b57..a497aa7b00871 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -46,14 +46,14 @@ repos: - id: detect-private-key - repo: https://github.com/asottile/pyupgrade - rev: v2.23.0 + rev: v2.23.2 hooks: - id: pyupgrade args: [--py36-plus] name: Upgrade code - repo: https://github.com/PyCQA/isort - rev: 5.9.2 + rev: 5.9.3 hooks: - id: isort name: Format imports @@ -64,6 +64,23 @@ repos: - id: black name: Format code + - repo: https://github.com/asottile/blacken-docs + rev: v1.10.0 + hooks: + - id: blacken-docs + args: [ --line-length=120 ] + additional_dependencies: [ black==21.7b0 ] + + - repo: https://github.com/executablebooks/mdformat + rev: 0.7.8 + hooks: + - id: mdformat + additional_dependencies: + - mdformat-gfm + #- mdformat-black + - mdformat_frontmatter + exclude: CHANGELOG.md + - repo: https://github.com/PyCQA/flake8 rev: 3.9.2 hooks: diff --git a/README.md b/README.md index 14ca61cea9f42..2da4b524be209 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,10 @@ - **The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.** ---- +______________________________________________________________________

Website โ€ข @@ -20,6 +19,7 @@ Scale your models, not the boilerplate.**

+ [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/pytorch-lightning)](https://pypi.org/project/pytorch-lightning/) [![PyPI Status](https://badge.fury.io/py/pytorch-lightning.svg)](https://badge.fury.io/py/pytorch-lightning) [![PyPI Status](https://pepy.tech/badge/pytorch-lightning)](https://pepy.tech/project/pytorch-lightning) @@ -34,19 +34,22 @@ Scale your models, not the boilerplate.** + -###### *Codecov is > 90%+ but build delays may show less +###### \*Codecov is > 90%+ but build delays may show less ---- +______________________________________________________________________ ## PyTorch Lightning is just organized PyTorch + Lightning disentangles PyTorch code to decouple the science from the engineering. ![PT to PL](docs/source/_static/images/general/pl_quick_start_full_compressed.gif) ---- +______________________________________________________________________ ## Lightning Design Philosophy + Lightning structures PyTorch code with these principles:
@@ -64,86 +67,93 @@ Once you do this, you can train on multiple-GPUs, TPUs, CPUs and even in 16-bit Get started with our [2 step guide](https://pytorch-lightning.readthedocs.io/en/latest/starter/new-project.html) ---- +______________________________________________________________________ ## Continuous Integration + Lightning is rigorously tested across multiple GPUs, TPUs CPUs and against major Python and PyTorch versions.
Current build statuses -
+
- | System / PyTorch ver. | 1.6 (min. req.) | 1.7 | 1.8 (LTS) | 1.9 (latest) | 1.10 (nightly) | - | :---: | :---: | :---: | :---: | :---: | :---: | - | Conda py3.7 [linux] | [![PyTorch & Conda](https://github.com/PyTorchLightning/pytorch-lightning/workflows/PyTorch%20&%20Conda/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22PyTorch+%26+Conda%22+branch%3Amaster) | [![PyTorch & Conda](https://github.com/PyTorchLightning/pytorch-lightning/workflows/PyTorch%20&%20Conda/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22PyTorch+%26+Conda%22+branch%3Amaster) | [![PyTorch & Conda](https://github.com/PyTorchLightning/pytorch-lightning/workflows/PyTorch%20&%20Conda/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22PyTorch+%26+Conda%22+branch%3Amaster) | [![PyTorch & Conda](https://github.com/PyTorchLightning/pytorch-lightning/workflows/PyTorch%20&%20Conda/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22PyTorch+%26+Conda%22+branch%3Amaster) | [![PyTorch & Conda](https://github.com/PyTorchLightning/pytorch-lightning/workflows/PyTorch%20&%20Conda/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22PyTorch+%26+Conda%22+branch%3Amaster) | [![PyTorch & Conda](https://github.com/PyTorchLightning/pytorch-lightning/workflows/PyTorch%20&%20Conda/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22PyTorch+%26+Conda%22+branch%3Amaster) | - | Linux py3.7 [GPUs**] | - | - | [![Build Status](https://dev.azure.com/PytorchLightning/pytorch-lightning/_apis/build/status/PL.pytorch-lightning%20(GPUs)?branchName=master)](https://dev.azure.com/PytorchLightning/pytorch-lightning/_build/latest?definitionId=6&branchName=master) | - | - | - | Linux py3.{6,7} [TPUs***] | [![TPU tests](https://github.com/PyTorchLightning/pytorch-lightning/workflows/TPU%20tests/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22TPU+tests%22+branch%3Amaster) | - | [![TPU tests](https://github.com/PyTorchLightning/pytorch-lightning/workflows/TPU%20tests/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22TPU+tests%22+branch%3Amaster) | - | - | - | Linux py3.{6,7,8,9} | [![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) | - | - | [![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) | - | - | OSX py3.{6,7,8,9} |[![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) | - | - | [![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) | - | - | Windows py3.{6,7,8,9} | [![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) | - | - | [![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) | - | +| System / PyTorch ver. | 1.6 (min. req.) | 1.7 | 1.8 (LTS) | 1.9 (latest) | 1.10 (nightly) | +| :----------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | +| Conda py3.7 \[linux\] | [![PyTorch & Conda](https://github.com/PyTorchLightning/pytorch-lightning/workflows/PyTorch%20&%20Conda/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22PyTorch+%26+Conda%22+branch%3Amaster) | [![PyTorch & Conda](https://github.com/PyTorchLightning/pytorch-lightning/workflows/PyTorch%20&%20Conda/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22PyTorch+%26+Conda%22+branch%3Amaster) | [![PyTorch & Conda](https://github.com/PyTorchLightning/pytorch-lightning/workflows/PyTorch%20&%20Conda/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22PyTorch+%26+Conda%22+branch%3Amaster) | [![PyTorch & Conda](https://github.com/PyTorchLightning/pytorch-lightning/workflows/PyTorch%20&%20Conda/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22PyTorch+%26+Conda%22+branch%3Amaster) | [![PyTorch & Conda](https://github.com/PyTorchLightning/pytorch-lightning/workflows/PyTorch%20&%20Conda/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22PyTorch+%26+Conda%22+branch%3Amaster) | +| Linux py3.7 \[GPUs\*\*\] | - | - | [![Build Status]()](https://dev.azure.com/PytorchLightning/pytorch-lightning/_build/latest?definitionId=6&branchName=master) | - | - | +| Linux py3.{6,7} \[TPUs\*\*\*\] | [![TPU tests](https://github.com/PyTorchLightning/pytorch-lightning/workflows/TPU%20tests/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22TPU+tests%22+branch%3Amaster) | - | [![TPU tests](https://github.com/PyTorchLightning/pytorch-lightning/workflows/TPU%20tests/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22TPU+tests%22+branch%3Amaster) | - | - | +| Linux py3.{6,7,8,9} | [![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) | - | - | [![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) | - | +| OSX py3.{6,7,8,9} | [![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) | - | - | [![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) | - | +| Windows py3.{6,7,8,9} | [![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) | - | - | [![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=master&event=push)](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) | - | - - _\** tests run on two NVIDIA P100_ - - _\*** tests run on Google GKE TPUv2/3_ - - _TPU py3.7 means we support Colab and Kaggle env._ +- _\*\* tests run on two NVIDIA P100_ +- _\*\*\* tests run on Google GKE TPUv2/3_ +- _TPU py3.7 means we support Colab and Kaggle env._ -
+
---- +______________________________________________________________________ ## How To Use ### Step 0: Install Simple installation from PyPI + ```bash pip install pytorch-lightning ``` +
Other installation options - #### Install with optional dependencies +#### Install with optional dependencies - ```bash - pip install pytorch-lightning['extra'] - ``` +```bash +pip install pytorch-lightning['extra'] +``` - #### Conda +#### Conda - ```bash - conda install pytorch-lightning -c conda-forge - ``` +```bash +conda install pytorch-lightning -c conda-forge +``` - #### Install stable 1.4.x +#### Install stable 1.4.x - the actual status of 1.4 [stable] is following: +the actual status of 1.4 \[stable\] is following: - ![CI base testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20base%20testing/badge.svg?branch=release%2F1.4.x&event=push) - ![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=release%2F1.4.x&event=push) - ![PyTorch & Conda](https://github.com/PyTorchLightning/pytorch-lightning/workflows/PyTorch%20&%20Conda/badge.svg?branch=release%2F1.4.x&event=push) - ![TPU tests](https://github.com/PyTorchLightning/pytorch-lightning/workflows/TPU%20tests/badge.svg?branch=release%2F1.4.x&event=push) - ![Docs check](https://github.com/PyTorchLightning/pytorch-lightning/workflows/Docs%20check/badge.svg?branch=release%2F1.4.x&event=push) +![CI base testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20base%20testing/badge.svg?branch=release%2F1.4.x&event=push) +![CI complete testing](https://github.com/PyTorchLightning/pytorch-lightning/workflows/CI%20complete%20testing/badge.svg?branch=release%2F1.4.x&event=push) +![PyTorch & Conda](https://github.com/PyTorchLightning/pytorch-lightning/workflows/PyTorch%20&%20Conda/badge.svg?branch=release%2F1.4.x&event=push) +![TPU tests](https://github.com/PyTorchLightning/pytorch-lightning/workflows/TPU%20tests/badge.svg?branch=release%2F1.4.x&event=push) +![Docs check](https://github.com/PyTorchLightning/pytorch-lightning/workflows/Docs%20check/badge.svg?branch=release%2F1.4.x&event=push) - Install future release from the source - ```bash - pip install git+https://github.com/PytorchLightning/pytorch-lightning.git@release/1.4.x --upgrade - ``` +Install future release from the source + +```bash +pip install git+https://github.com/PytorchLightning/pytorch-lightning.git@release/1.4.x --upgrade +``` - #### Install bleeding-edge - future 1.5 +#### Install bleeding-edge - future 1.5 - Install nightly from the source (no guarantees) - ```bash - pip install https://github.com/PyTorchLightning/pytorch-lightning/archive/master.zip - ``` +Install nightly from the source (no guarantees) + +```bash +pip install https://github.com/PyTorchLightning/pytorch-lightning/archive/master.zip +``` + +or from testing PyPI + +```bash +pip install -iU https://test.pypi.org/simple/ pytorch-lightning +``` - or from testing PyPI - ```bash - pip install -iU https://test.pypi.org/simple/ pytorch-lightning - ```
@@ -161,6 +171,7 @@ import pytorch_lightning as pl ``` ### Step 2: Define a LightningModule (nn.Module subclass) + A LightningModule defines a full *system* (ie: a GAN, autoencoder, BERT or a simple Image Classifier). ```python @@ -205,6 +216,7 @@ trainer.fit(autoencoder, DataLoader(train), DataLoader(val)) ``` ## Advanced features + Lightning has over [40+ advanced features](https://pytorch-lightning.readthedocs.io/en/latest/common/trainer.html#trainer-flags) designed for professional AI research at scale. Here are some examples: @@ -216,87 +228,89 @@ Here are some examples:
Highlighted feature code snippets - ```python - # 8 GPUs - # no code changes needed - trainer = Trainer(max_epochs=1, gpus=8) +```python +# 8 GPUs +# no code changes needed +trainer = Trainer(max_epochs=1, gpus=8) - # 256 GPUs - trainer = Trainer(max_epochs=1, gpus=8, num_nodes=32) - ``` +# 256 GPUs +trainer = Trainer(max_epochs=1, gpus=8, num_nodes=32) +``` - Train on TPUs without code changes +Train on TPUs without code changes + +```python +# no code changes needed +trainer = Trainer(tpu_cores=8) +``` - ```python - # no code changes needed - trainer = Trainer(tpu_cores=8) - ``` +16-bit precision - 16-bit precision +```python +# no code changes needed +trainer = Trainer(precision=16) +``` - ```python - # no code changes needed - trainer = Trainer(precision=16) - ``` +Experiment managers - Experiment managers +```python +from pytorch_lightning import loggers - ```python - from pytorch_lightning import loggers +# tensorboard +trainer = Trainer(logger=TensorBoardLogger("logs/")) - # tensorboard - trainer = Trainer(logger=TensorBoardLogger('logs/')) +# weights and biases +trainer = Trainer(logger=loggers.WandbLogger()) - # weights and biases - trainer = Trainer(logger=loggers.WandbLogger()) +# comet +trainer = Trainer(logger=loggers.CometLogger()) - # comet - trainer = Trainer(logger=loggers.CometLogger()) +# mlflow +trainer = Trainer(logger=loggers.MLFlowLogger()) - # mlflow - trainer = Trainer(logger=loggers.MLFlowLogger()) +# neptune +trainer = Trainer(logger=loggers.NeptuneLogger()) - # neptune - trainer = Trainer(logger=loggers.NeptuneLogger()) +# ... and dozens more +``` - # ... and dozens more - ``` +EarlyStopping - EarlyStopping +```python +es = EarlyStopping(monitor="val_loss") +trainer = Trainer(callbacks=[es]) +``` - ```python - es = EarlyStopping(monitor='val_loss') - trainer = Trainer(callbacks=[es]) - ``` +Checkpointing - Checkpointing +```python +checkpointing = ModelCheckpoint(monitor="val_loss") +trainer = Trainer(callbacks=[checkpointing]) +``` - ```python - checkpointing = ModelCheckpoint(monitor='val_loss') - trainer = Trainer(callbacks=[checkpointing]) - ``` +Export to torchscript (JIT) (production use) - Export to torchscript (JIT) (production use) +```python +# torchscript +autoencoder = LitAutoEncoder() +torch.jit.save(autoencoder.to_torchscript(), "model.pt") +``` - ```python - # torchscript - autoencoder = LitAutoEncoder() - torch.jit.save(autoencoder.to_torchscript(), "model.pt") - ``` +Export to ONNX (production use) - Export to ONNX (production use) +```python +# onnx +with tempfile.NamedTemporaryFile(suffix=".onnx", delete=False) as tmpfile: + autoencoder = LitAutoEncoder() + input_sample = torch.randn((1, 64)) + autoencoder.to_onnx(tmpfile.name, input_sample, export_params=True) + os.path.isfile(tmpfile.name) +``` - ```python - # onnx - with tempfile.NamedTemporaryFile(suffix='.onnx', delete=False) as tmpfile: - autoencoder = LitAutoEncoder() - input_sample = torch.randn((1, 64)) - autoencoder.to_onnx(tmpfile.name, input_sample, export_params=True) - os.path.isfile(tmpfile.name) - ```
### Pro-level control of training loops (advanced users) + For complex/professional level work, you have optional full control of the training loop and optimizers. ```python @@ -320,54 +334,61 @@ class LitAutoEncoder(pl.LightningModule): opt_b.step() opt_b.zero_grad() ``` ---- + +______________________________________________________________________ ## Advantages over unstructured PyTorch -* Models become hardware agnostic -* Code is clear to read because engineering code is abstracted away -* Easier to reproduce -* Make fewer mistakes because lightning handles the tricky engineering -* Keeps all the flexibility (LightningModules are still PyTorch modules), but removes a ton of boilerplate -* Lightning has dozens of integrations with popular machine learning tools. -* [Tested rigorously with every new PR](https://github.com/PyTorchLightning/pytorch-lightning/tree/master/tests). We test every combination of PyTorch and Python supported versions, every OS, multi GPUs and even TPUs. -* Minimal running speed overhead (about 300 ms per epoch compared with pure PyTorch). +- Models become hardware agnostic +- Code is clear to read because engineering code is abstracted away +- Easier to reproduce +- Make fewer mistakes because lightning handles the tricky engineering +- Keeps all the flexibility (LightningModules are still PyTorch modules), but removes a ton of boilerplate +- Lightning has dozens of integrations with popular machine learning tools. +- [Tested rigorously with every new PR](https://github.com/PyTorchLightning/pytorch-lightning/tree/master/tests). We test every combination of PyTorch and Python supported versions, every OS, multi GPUs and even TPUs. +- Minimal running speed overhead (about 300 ms per epoch compared with pure PyTorch). ---- +______________________________________________________________________ ## Examples ###### Hello world + - [MNIST hello world](https://pytorch-lightning.readthedocs.io/en/latest/notebooks/lightning_examples/mnist-hello-world.html) ###### Contrastive Learning + - [BYOL](https://lightning-bolts.readthedocs.io/en/latest/self_supervised_models.html#byol) - [CPC v2](https://lightning-bolts.readthedocs.io/en/latest/self_supervised_models.html#cpc-v2) - [Moco v2](https://lightning-bolts.readthedocs.io/en/latest/self_supervised_models.html#moco-v2) - [SIMCLR](https://lightning-bolts.readthedocs.io/en/latest/self_supervised_models.html#simclr) ###### NLP + - [GPT-2](https://lightning-bolts.readthedocs.io/en/latest/convolutional.html#gpt-2) - [BERT](https://pytorch-lightning.readthedocs.io/en/latest/notebooks/lightning_examples/text-transformers.html) - ###### Reinforcement Learning + - [DQN](https://lightning-bolts.readthedocs.io/en/latest/reinforce_learn.html#dqn-models) - [Dueling-DQN](https://lightning-bolts.readthedocs.io/en/latest/reinforce_learn.html#dueling-dqn) - [Reinforce](https://lightning-bolts.readthedocs.io/en/latest/reinforce_learn.html#reinforce) ###### Vision + - [GAN](https://pytorch-lightning.readthedocs.io/en/latest/notebooks/lightning_examples/basic-gan.html) ###### Classic ML + - [Logistic Regression](https://lightning-bolts.readthedocs.io/en/latest/classic_ml.html#logistic-regression) - [Linear Regression](https://lightning-bolts.readthedocs.io/en/latest/classic_ml.html#linear-regression) ---- +______________________________________________________________________ ## Community The lightning community is maintained by + - [10+ core contributors](https://pytorch-lightning.readthedocs.io/en/latest/governance.html) who are all a mix of professional engineers, Research Scientists, and Ph.D. students from top AI labs. - 480+ active community contributors. @@ -376,16 +397,21 @@ Want to help us build Lightning and reduce boilerplate for thousands of research Lightning is also part of the [PyTorch ecosystem](https://pytorch.org/ecosystem/) which requires projects to have solid testing, documentation and support. ### Asking for help + If you have any questions please: + 1. [Read the docs](https://pytorch-lightning.rtfd.io/en/latest). -2. [Search through existing Discussions](https://github.com/PyTorchLightning/pytorch-lightning/discussions), or [add a new question](https://github.com/PyTorchLightning/pytorch-lightning/discussions/new) -3. [Join our slack](https://join.slack.com/t/pytorch-lightning/shared_invite/zt-pw5v393p-qRaDgEk24~EjiZNBpSQFgQ). +1. [Search through existing Discussions](https://github.com/PyTorchLightning/pytorch-lightning/discussions), or [add a new question](https://github.com/PyTorchLightning/pytorch-lightning/discussions/new) +1. [Join our slack](https://join.slack.com/t/pytorch-lightning/shared_invite/zt-pw5v393p-qRaDgEk24~EjiZNBpSQFgQ). + ### Funding + [We're venture funded](https://techcrunch.com/2020/10/08/grid-ai-raises-18-6m-series-a-to-help-ai-researchers-and-engineers-bring-their-models-to-production/) to make sure we can provide around the clock support, hire a full-time staff, attend conferences, and move faster through implementing features you request. ---- +______________________________________________________________________ ## Grid AI + Grid AI is our platform for training models at scale on the cloud! **Sign up for our FREE community Tier [here](https://www.grid.ai/pricing/)** @@ -405,7 +431,7 @@ grid train --grid_gpus 4 my_model.py --learning_rate 'uniform(1e-6, 1e-1, 20)' - The above command will launch (20 * 4) experiments each running on 4 GPUs (320 GPUs!) - by making ZERO changes to your code. ---- +______________________________________________________________________ ## Licence @@ -413,6 +439,7 @@ Please observe the Apache 2.0 license that is listed in this repository. In addition, the Lightning framework is Patent Pending. ## BibTeX + If you want to cite the framework feel free to use this (but only if you loved it ๐Ÿ˜Š) or [zenodo](https://zenodo.org/record/3828935#.YC45Lc9Khqs): ```bibtex diff --git a/dockers/README.md b/dockers/README.md index c747b3af79500..6ab45b49a5154 100644 --- a/dockers/README.md +++ b/dockers/README.md @@ -20,7 +20,9 @@ docker image build \ --build-arg PYTORCH_VERSION=1.8 \ . ``` + or nightly version from Conda + ```bash git clone docker image build \ @@ -48,6 +50,7 @@ docker image rm pytorch-lightning:latest ## Run docker image with GPUs To run docker image with access to you GPUs you need to install + ```bash # Add the package repositories distribution=$(. /etc/os-release;echo $ID$VERSION_ID) @@ -69,17 +72,17 @@ docker run --rm -it --gpus all pytorchlightning/pytorch_lightning:base-cuda-py3. Inspiration comes from https://u.group/thinking/how-to-put-jupyter-notebooks-in-a-dockerfile 1. Build the docker image: - ```bash - docker image build \ - -t pytorch-lightning:v1.3.1 \ - -f dockers/nvidia/Dockerfile \ - --build-arg LIGHTNING_VERSION=1.3.1 \ - . - ``` -2. start the server and map ports: - ```bash - docker run --rm -it --runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=all -p 8888:8888 pytorch-lightning:v1.3.1 - ``` -3. Connect in local browser: - - copy the generated path e.g. `http://hostname:8888/?token=0719fa7e1729778b0cec363541a608d5003e26d4910983c6` - - replace the `hostname` by `localhost` + ```bash + docker image build \ + -t pytorch-lightning:v1.3.1 \ + -f dockers/nvidia/Dockerfile \ + --build-arg LIGHTNING_VERSION=1.3.1 \ + . + ``` +1. start the server and map ports: + ```bash + docker run --rm -it --runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=all -p 8888:8888 pytorch-lightning:v1.3.1 + ``` +1. Connect in local browser: + - copy the generated path e.g. `http://hostname:8888/?token=0719fa7e1729778b0cec363541a608d5003e26d4910983c6` + - replace the `hostname` by `localhost` diff --git a/legacy/README.md b/legacy/README.md index 3ce6d15f65568..5ffcecf971f3a 100644 --- a/legacy/README.md +++ b/legacy/README.md @@ -5,12 +5,14 @@ The aim of this section is set some baselines and workflows/guidelines for maint At this moment we focus on ability running old checkpoints, so the flow here is to create a checkpoint with every release and store it in our public AWS storage and so each CI testing will pull this archive and test loading and resuming training with this model. If you want to pull all saved version-checkpoints for local testing/development, call + ```bash wget https://pl-public-data.s3.amazonaws.com/legacy/checkpoints.zip unzip -o checkpoints.zip ``` To back populate collection with past version you can use following bash: + ```bash bash generate_checkpoints.sh 1.0.2 1.0.3 1.0.4 zip -r checkpoints.zip checkpoints/ diff --git a/pl_examples/README.md b/pl_examples/README.md index e09c79e081c25..c6d729c9aef33 100644 --- a/pl_examples/README.md +++ b/pl_examples/README.md @@ -1,19 +1,22 @@ # Examples + Our most robust examples showing all sorts of implementations can be found in our sister library [lightning-bolts](https://lightning-bolts.readthedocs.io/en/latest/convolutional.html#gpt-2). ---- +______________________________________________________________________ ## Basic examples + In this folder we add 3 simple examples: -* [MNIST Classifier](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pl_examples/basic_examples/simple_image_classifier.py) (defines the model inside the `LightningModule`). -* [Image Classifier](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pl_examples/basic_examples/backbone_image_classifier.py) (trains arbitrary datasets with arbitrary backbones). -* [Autoencoder](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pl_examples/basic_examples/autoencoder.py) (shows how the `LightningModule` can be used as a system) +- [MNIST Classifier](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pl_examples/basic_examples/simple_image_classifier.py) (defines the model inside the `LightningModule`). +- [Image Classifier](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pl_examples/basic_examples/backbone_image_classifier.py) (trains arbitrary datasets with arbitrary backbones). +- [Autoencoder](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pl_examples/basic_examples/autoencoder.py) (shows how the `LightningModule` can be used as a system) ---- +______________________________________________________________________ ## Domain examples + This folder contains older examples. You should instead use the examples in [lightning-bolts](https://lightning-bolts.readthedocs.io/en/latest/convolutional.html#gpt-2) for advanced use cases. diff --git a/pl_examples/basic_examples/README.md b/pl_examples/basic_examples/README.md index bd8dfd7426eb8..fd7824140d470 100644 --- a/pl_examples/basic_examples/README.md +++ b/pl_examples/basic_examples/README.md @@ -1,8 +1,11 @@ ## Basic Examples + Use these examples to test how lightning works. #### MNIST + Trains MNIST where the model is defined inside the `LightningModule`. + ```bash # cpu python simple_image_classifier.py @@ -14,17 +17,23 @@ python simple_image_classifier.py --trainer.gpus 2 python simple_image_classifier.py --trainer.gpus 2 --trainer.accelerator 'dp' ``` ---- +______________________________________________________________________ + #### MNIST with DALI + The MNIST example above using [NVIDIA DALI](https://developer.nvidia.com/DALI). Requires NVIDIA DALI to be installed based on your CUDA version, see [here](https://docs.nvidia.com/deeplearning/dali/user-guide/docs/installation.html). + ```bash python dali_image_classifier.py ``` ---- +______________________________________________________________________ + #### Image classifier + Generic image classifier with an arbitrary backbone (ie: a simple system) + ```bash # cpu python backbone_image_classifier.py @@ -36,9 +45,12 @@ python backbone_image_classifier.py --trainer.gpus 2 python backbone_image_classifier.py --trainer.gpus 2 --trainer.accelerator 'dp' ``` ---- +______________________________________________________________________ + #### Autoencoder + Showing the power of a system... arbitrarily complex training loops + ```bash # cpu python autoencoder.py @@ -49,12 +61,14 @@ python autoencoder.py --trainer.gpus 2 # dataparallel python autoencoder.py --trainer.gpus 2 --trainer.accelerator 'dp' ``` ---- + +______________________________________________________________________ + # Multi-node example This demo launches a job using 2 GPUs on 2 different nodes (4 GPUs total). To run this demo do the following: 1. Log into the jumphost node of your SLURM-managed cluster. -2. Create a conda environment with Lightning and a GPU PyTorch version. -3. Choose a script to submit +1. Create a conda environment with Lightning and a GPU PyTorch version. +1. Choose a script to submit diff --git a/tests/README.md b/tests/README.md index 01d0d835f9fe7..fc63d353ebeca 100644 --- a/tests/README.md +++ b/tests/README.md @@ -1,9 +1,11 @@ # PyTorch-Lightning Tests + Most PL tests train a full MNIST model under various trainer conditions (ddp, ddp2+amp, etc...). This provides testing for most combinations of important settings. The tests expect the model to perform to a reasonable degree of testing accuracy to pass. ## Running tests + ```bash git clone https://github.com/PyTorchLightning/pytorch-lightning cd pytorch-lightning @@ -23,6 +25,7 @@ such as Horovod, FairScale, NVIDIA/apex, NVIDIA/DALI, etc. You can rely on our CI to make sure all these tests pass. ## Running Coverage + Make sure to run coverage on a GPU machine with at least 2 GPUs and NVIDIA apex installed. ```bash @@ -41,11 +44,14 @@ coverage xml ## Building test image You can build it on your own, note it takes lots of time, be prepared. + ```bash git clone docker image build -t pytorch_lightning:devel-torch1.9 -f dockers/cuda-extras/Dockerfile --build-arg TORCH_VERSION=1.9 . ``` + To build other versions, select different Dockerfile. + ```bash docker image list docker run --rm -it pytorch_lightning:devel-torch1.9 bash