Lightning Lite Examples (#9987)

Co-authored-by: Kaushik B <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Justus Schock <[email protected]> Co-authored-by: SeanNaren <[email protected]> Co-authored-by: Kaushik B <[email protected]> Co-authored-by: thomas chaton <[email protected]> Co-authored-by: Rohit Gupta <[email protected]> Co-authored-by: four4fish <[email protected]> Co-authored-by: Nicki Skafte Detlefsen <[email protected]> Co-authored-by: Carlos Mocholi <[email protected]> Co-authored-by: Pietro Lesci <[email protected]>
Lightning-AI · Nov 2, 2021 · 3cd65b5 · 3cd65b5
1 parent e4ee6df
commit 3cd65b5
Show file tree

Hide file tree

Showing 24 changed files with 1,016 additions and 168 deletions.
diff --git a/.gitignore b/.gitignore
@@ -137,6 +137,9 @@ ENV/
 Datasets/
 mnist/
 legacy/checkpoints/
+*.gz
+*ubyte
+
 
 # pl tests
 ml-runs/

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -119,6 +119,8 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
     * Updated precision attributes in `DeepSpeedPlugin` ([#10164](https://github.com/PyTorchLightning/pytorch-lightning/pull/10164))
     * Added the ability to return a result from rank 0 in `DDPSpawnPlugin.spawn` ([#10162](https://github.com/PyTorchLightning/pytorch-lightning/pull/10162))
     * Added `pytorch_lightning.lite` package ([#10175](https://github.com/PyTorchLightning/pytorch-lightning/pull/10175))
+    * Added `LightningLite` documentation ([#10043](https://github.com/PyTorchLightning/pytorch-lightning/pull/10043))
+    * Added `LightningLite` examples ([#9987](https://github.com/PyTorchLightning/pytorch-lightning/pull/9987))
     * Make the `_LiteDataLoader` an iterator and add supports for custom dataloader ([#10279](https://github.com/PyTorchLightning/pytorch-lightning/pull/10279))
 - Added `use_omegaconf` argument to `save_hparams_to_yaml` plugin ([#9170](https://github.com/PyTorchLightning/pytorch-lightning/pull/9170))
 - Added `ckpt_path` argument for `Trainer.fit()` ([#10061](https://github.com/PyTorchLightning/pytorch-lightning/pull/10061))

diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -16,6 +16,7 @@
 import os
 import shutil
 import sys
+import warnings
 from importlib.util import module_from_spec, spec_from_file_location
 
 import pt_lightning_sphinx_theme
@@ -26,10 +27,13 @@
 sys.path.insert(0, os.path.abspath(PATH_ROOT))
 sys.path.append(os.path.join(PATH_RAW_NB, ".actions"))
 
+_SHOULD_COPY_NOTEBOOKS = True
+
 try:
     from helpers import HelperCLI
 except Exception:
-    raise ModuleNotFoundError("To build the code, please run: `git submodule update --init --recursive`")
+    _SHOULD_COPY_NOTEBOOKS = False
+    warnings.warn("To build the code, please run: `git submodule update --init --recursive`", stacklevel=2)
 
 FOLDER_GENERATED = "generated"
 SPHINX_MOCK_REQUIREMENTS = int(os.environ.get("SPHINX_MOCK_REQUIREMENTS", True))
@@ -41,8 +45,8 @@
 spec.loader.exec_module(about)
 
 # -- Project documents -------------------------------------------------------
-
-HelperCLI.copy_notebooks(PATH_RAW_NB, PATH_HERE, "notebooks")
+if _SHOULD_COPY_NOTEBOOKS:
+    HelperCLI.copy_notebooks(PATH_RAW_NB, PATH_HERE, "notebooks")
 
 
 def _transform_changelog(path_in: str, path_out: str) -> None:

diff --git a/pl_examples/README.md b/pl_examples/README.md
@@ -5,20 +5,31 @@ can be found in our sister library [Lightning Bolts](https://pytorch-lightning.r
 
 ______________________________________________________________________
 
-## Basic examples
+## MNIST Examples
 
-In this folder we add several starter examples:
+5 MNIST examples showing how to gradually convert from pure PyTorch to PyTorch Lightning.
 
-- [MNIST Classifier](./basic_examples/simple_image_classifier.py): Shows how to define the model inside the `LightningModule`.
-- [Image Classifier](./basic_examples/backbone_image_classifier.py): Trains arbitrary datasets with arbitrary backbones.
-- [Autoencoder](./basic_examples/autoencoder.py): Shows how the `LightningModule` can be used as a system.
-- [Profiler](./basic_examples/profiler_example.py): Shows the basic usage of the PyTorch profilers and how to inspect traces in Google Chrome.
-- [Image Classifier with DALI](./basic_examples/dali_image_classifier.py): Shows how to use [NVIDIA DALI](https://developer.nvidia.com/DALI) with Lightning.
-- [Mnist Datamodule](.basic_examples/mnist_datamodule.py): Shows how to define a simple `LightningDataModule` using the MNIST dataset.
+The transition through [LightningLite](https://pytorch-lightning.readthedocs.io/en/latest/starter/lightning_lite.html) from pure PyTorch is optional but it might be helpful to learn about it.
+
+- [MNIST with vanilla PyTorch](./basic_examples/mnist_examples/image_classifier_1_pytorch.py)
+- [MNIST with LightningLite](./basic_examples/mnist_examples/image_classifier_2_lite.py)
+- [MNIST LightningLite to LightningModule](./basic_examples/mnist_examples/image_classifier_3_lite_to_lightning_module.py)
+- [MNIST with LightningModule](./basic_examples/mnist_examples/image_classifier_4_lightning_module.py)
+- [MNIST with LightningModule + LightningDataModule](./basic_examples/mnist_examples/image_classifier_5_lightning_datamodule.py)
+
+______________________________________________________________________
+
+## Basic Examples
+
+In this folder, we have 2 simple examples:
+
+- [Image Classifier](./basic_examples/backbone_image_classifier.py) (trains arbitrary datasets with arbitrary backbones).
+- [Image Classifier + DALI](./basic_examples/mnist_examples/image_classifier_4_dali.py) (defines the model inside the `LightningModule`).
+- [Autoencoder](./basic_examples/autoencoder.py) (shows how the `LightningModule` can be used as a system)
 
 ______________________________________________________________________
 
-## Domain examples
+## Domain Examples
 
 This folder contains older examples. You should instead use the examples
 in [Lightning Bolts](https://pytorch-lightning.readthedocs.io/en/latest/ecosystem/bolts.html)

diff --git a/pl_examples/basic_examples/README.md b/pl_examples/basic_examples/README.md
@@ -2,62 +2,70 @@
 
 Use these examples to test how Lightning works.
 
-#### MNIST
+## MNIST Examples
 
-Trains MNIST where the model is defined inside the `LightningModule`.
+Here are 5 MNIST examples showing you how to gradually convert from pure PyTorch to PyTorch Lightning.
 
-```bash
-# cpu
-python simple_image_classifier.py
+The transition through [LightningLite](https://pytorch-lightning.readthedocs.io/en/stable/starter/lightning_lite.rst) from pure PyTorch is optional but it might be helpful to learn about it.
+
+#### 1. Image Classifier with Vanilla PyTorch
 
-# gpus (any number)
-python simple_image_classifier.py --trainer.gpus 2
+Trains a simple CNN over MNIST using vanilla PyTorch.
 
-# Distributed Data Parallel
-python simple_image_classifier.py --trainer.gpus 2 --trainer.accelerator ddp
+```bash
+# CPU
+python image_classifier_1_pytorch.py
 ```
 
 ______________________________________________________________________
 
-#### MNIST with DALI
+#### 2. Image Classifier with LightningLite
 
-The MNIST example above using [NVIDIA DALI](https://developer.nvidia.com/DALI).
-Requires NVIDIA DALI to be installed based on your CUDA version, see [here](https://docs.nvidia.com/deeplearning/dali/user-guide/docs/installation.html).
+This script shows you how to scale the previous script to enable GPU and multi-GPU training using [LightningLite](https://pytorch-lightning.readthedocs.io/en/stable/starter/lightning_lite.html).
 
 ```bash
-python dali_image_classifier.py
+# CPU / multiple GPUs if available
+python image_classifier_2_lite.py
 ```
 
 ______________________________________________________________________
 
-#### Image classifier
+#### 3. Image Classifier - Conversion from Lite to Lightning
 
-Generic image classifier with an arbitrary backbone (ie: a simple system)
+This script shows you how to prepare your conversion from [LightningLite](https://pytorch-lightning.readthedocs.io/en/stable/starter/lightning_lite.html) to `LightningModule`.
 
 ```bash
-# cpu
-python backbone_image_classifier.py
+# CPU / multiple GPUs if available
+python image_classifier_3_lite_to_lightning_module.py
+```
 
-# gpus (any number)
-python backbone_image_classifier.py --trainer.gpus 2
+______________________________________________________________________
+
+#### 4. Image Classifier with LightningModule
+
+This script shows you the result of the conversion to the `LightningModule` and finally all the benefits you get from the Lightning ecosystem.
+
+```bash
+# CPU
+python image_classifier_4_lightning_module.py
 
-# Distributed Data Parallel
-python backbone_image_classifier.py --trainer.gpus 2 --trainer.accelerator ddp
+# GPUs (any number)
+python image_classifier_4_lightning_module.py --trainer.gpus 2
 ```
 
 ______________________________________________________________________
 
-#### Autoencoder
+#### 5. Image Classifier with LightningModule and LightningDataModule
 
-Showing the power of a system... arbitrarily complex training loops
+This script shows you how to extract the data related components into a `LightningDataModule`.
 
 ```bash
-# cpu
-python autoencoder.py
+# CPU
+python image_classifier_5_lightning_datamodule.py
 
-# gpus (any number)
-python autoencoder.py --trainer.gpus 2
+# GPUs (any number)
+python image_classifier_5_lightning_datamodule.py --trainer.gpus 2
 
-# Distributed Data Parallel
-python autoencoder.py --trainer.gpus 2 --trainer.accelerator ddp
+# Distributed Data Parallel (DDP)
+python image_classifier_5_lightning_datamodule.py --trainer.gpus 2 --trainer.strategy 'ddp'
 ```
diff --git a/pl_examples/basic_examples/mnist_examples/README.md b/pl_examples/basic_examples/mnist_examples/README.md
@@ -0,0 +1,67 @@
+## MNIST Examples
+
+Here are 5 MNIST examples showing you how to gradually convert from pure PyTorch to PyTorch Lightning.
+
+The transition through [LightningLite](https://pytorch-lightning.readthedocs.io/en/latest/stable/lightning_lite.rst) from pure PyTorch is optional but it might be helpful to learn about it.
+
+#### 1. Image Classifier with Vanilla PyTorch
+
+Trains a simple CNN over MNIST using vanilla PyTorch.
+
+```bash
+# CPU
+python image_classifier_1_pytorch.py
+```
+
+______________________________________________________________________
+
+#### 2. Image Classifier with LightningLite
+
+This script shows you how to scale the previous script to enable GPU and multi-GPU training using [LightningLite](https://pytorch-lightning.readthedocs.io/en/stable/starter/lightning_lite.html).
+
+```bash
+# CPU / multiple GPUs if available
+python image_classifier_2_lite.py
+```
+
+______________________________________________________________________
+
+#### 3. Image Classifier - Conversion from Lite to Lightning
+
+This script shows you how to prepare your conversion from [LightningLite](https://pytorch-lightning.readthedocs.io/en/stable/starter/lightning_lite.html) to `LightningModule`.
+
+```bash
+# CPU / multiple GPUs if available
+python image_classifier_3_lite_to_lightning_module.py
+```
+
+______________________________________________________________________
+
+#### 4. Image Classifier with LightningModule
+
+This script shows you the result of the conversion to the `LightningModule` and finally all the benefits you get from Lightning.
+
+```bash
+# CPU
+python image_classifier_4_lightning_module.py
+
+# GPUs (any number)
+python image_classifier_4_lightning_module.py --trainer.gpus 2
+```
+
+______________________________________________________________________
+
+#### 5. Image Classifier with LightningModule and LightningDataModule
+
+This script shows you how to extract the data related components into a `LightningDataModule`.
+
+```bash
+# CPU
+python image_classifier_5_lightning_datamodule.py
+
+# GPUs (any number)
+python image_classifier_5_lightning_datamodule.py --trainer.gpus 2
+
+# Distributed Data parallel
+python image_classifier_5_lightning_datamodule.py --trainer.gpus 2 --trainer.strategy 'ddp'
+```
diff --git a/pl_examples/basic_examples/mnist_examples/__init__.py b/pl_examples/basic_examples/mnist_examples/__init__.py