From d779bd5334b2399af832406cc0f79df0f5b24b1a Mon Sep 17 00:00:00 2001
From: Shaden Smith <Shaden.Smith@microsoft.com>
Date: Thu, 12 Nov 2020 14:29:52 -0800
Subject: [PATCH] Installation documentation updates. (#525)
* Adds torch install requirement to documentation.
* build ops documentation
---
README.md | 5 ++-
docs/_tutorials/advanced-install.md | 70 +++++++++++++++++------------
2 files changed, 46 insertions(+), 29 deletions(-)
diff --git a/README.md b/README.md
index 42c91f288cab..d94938da1ce5 100755
--- a/README.md
+++ b/README.md
@@ -72,11 +72,14 @@ just-in-time (JIT) using [torch's JIT C++ extension loader that relies on
ninja](https://pytorch.org/docs/stable/cpp_extension.html) to build and
dynamically link them at runtime.
+**Note:** [PyTorch](https://pytorch.org/) must be installed _before_ installing
+DeepSpeed.
+
```bash
pip install deepspeed
```
-After installation you can validate your install and see which extensions/ops
+After installation, you can validate your install and see which extensions/ops
your machine is compatible with via the DeepSpeed environment report.
```bash
diff --git a/docs/_tutorials/advanced-install.md b/docs/_tutorials/advanced-install.md
index 4d8061e4c987..ccb38e334ab3 100644
--- a/docs/_tutorials/advanced-install.md
+++ b/docs/_tutorials/advanced-install.md
@@ -11,11 +11,15 @@ just-in-time (JIT) using [torch's JIT C++ extension loader that relies on
ninja](https://pytorch.org/docs/stable/cpp_extension.html) to build and
dynamically link them at runtime.
+**Note:** [PyTorch](https://pytorch.org/) must be installed _before_ installing
+DeepSpeed.
+{: .notice--info}
+
```bash
pip install deepspeed
```
-After installation you can validate your install and see which ops your machine
+After installation, you can validate your install and see which ops your machine
is compatible with via the DeepSpeed environment report with `ds_report` or
`python -m deepspeed.env_report`. We've found this report useful when debugging
DeepSpeed install or compatibility issues.
@@ -24,23 +28,6 @@ DeepSpeed install or compatibility issues.
ds_report
```
-## Install DeepSpeed from source
-
-After cloning the DeepSpeed repo from github you can install DeepSpeed in
-JIT mode via pip (see below). This install should complete
-quickly since it is not compiling any C++/CUDA source files.
-
-```bash
-pip install .
-```
-
-For installs spanning multiple nodes we find it useful to install DeepSpeed
-using the
-[install.sh](https://github.com/microsoft/DeepSpeed/blob/master/install.sh)
-script in the repo. This will build a python wheel locally and copy it to all
-the nodes listed in your hostfile (either given via --hostfile, or defaults to
-/job/hostfile).
-
## Pre-install DeepSpeed Ops
Sometimes we have found it useful to pre-install either some or all DeepSpeed
@@ -53,23 +40,50 @@ want to attempt to install all of our ops by setting the `DS_BUILD_OPS`
environment variable to 1, for example:
```bash
-DS_BUILD_OPS=1 pip install .
+DS_BUILD_OPS=1 pip install deepspeed
+```
+
+DeepSpeed will only install any ops that are compatible with your machine.
+For more details on which ops are compatible with your system please try our
+`ds_report` tool described above.
+
+If you want to install only a specific op (e.g., FusedLamb), you can toggle
+with `DS_BUILD` environment variables at installation time. For example, to
+install DeepSpeed with only the FusedLamb op use:
+
+```bash
+DS_BUILD_FUSED_LAMB=1 pip install deepspeed
```
-We will only install any ops that are compatible with your machine, for more
-details on which ops are compatible with your system please try our `ds_report`
-tool described above.
+Available `DS_BUILD` options include:
+* `DS_BUILD_OPS` toggles all ops
+* `DS_BUILD_CPU_ADAM` builds the CPUAdam op
+* `DS_BUILD_FUSED_ADAM` builds the FusedAdam op (from [apex](https://github.com/NVIDIA/apex))
+* `DS_BUILD_FUSED_LAMB` builds the FusedLamb op
+* `DS_BUILD_SPARSE_ATTN` builds the sparse attention op
+* `DS_BUILD_TRANSFORMER` builds the transformer op
+* `DS_BUILD_STOCHASTIC_TRANSFORMER` builds the stochastic transformer op
+* `DS_BUILD_UTILS` builds various optimized utilities
+
-If you want to install only a specific op (e.g., FusedLamb) you can view the op
-specific build environment variable (set as `BUILD_VAR`) in the corresponding
-op builder class in the
-[op\_builder](https://github.com/microsoft/DeepSpeed/tree/master/op_builder)
-directory. For example to install only the Fused Lamb op you would install via:
+## Install DeepSpeed from source
+
+After cloning the DeepSpeed repo from GitHub, you can install DeepSpeed in
+JIT mode via pip (see below). This install should complete
+quickly since it is not compiling any C++/CUDA source files.
```bash
-DS_BUILD_FUSED_LAMB=1 pip install .
+pip install .
```
+For installs spanning multiple nodes we find it useful to install DeepSpeed
+using the
+[install.sh](https://github.com/microsoft/DeepSpeed/blob/master/install.sh)
+script in the repo. This will build a python wheel locally and copy it to all
+the nodes listed in your hostfile (either given via --hostfile, or defaults to
+/job/hostfile).
+
+
## Feature specific dependencies
Some DeepSpeed features require specific dependencies outside of the general