Migrate useful notebooks to the deployment docs repo

[jacobtomlinson]

In rapidsai/deployment#108 I copied a couple of the notebooks from this repo over into the deployment docs so that I had something to work with when updating the Sphinx theme.

As part of #197 we need to migrate the rest of the notebooks in this repo over, but those can be done in separate PRs. Not all notebooks here are in a good state, so we should just focus on migrating the ones which are up to date and confirmed to be working/useful.

My preference would be for each notebook to be placed in its own directory in the source/examples section of the deployment repo with no nesting. The notebook should probably be renamed to notebook.ipynb too so that the URL looks nice.

Any supporting files should be also placed in the directory alongside the notebook. Again ideally without any nested directories unless there is a technical reason why we need to do so.

source/examples
├── index.md
├── rapids-sagemaker-higgs
│   ├── Dockerfile
│   ├── notebook.ipynb
│   └── rapids-higgs.py
└── xgboost-gpu-hpo-job-parallel-k8s
    └── notebook.ipynb

Each notebook also needs to be added to the toctree of the source/examples/index.md file.

Once the notebook has been copied and merged it needs to be removed from this repo and replaced with a link so folks can find them again.

Tasks

Preview Give feedback

1. Migrate dask/kubernetes/Dask_cuML_Exploration.ipynb to deployment docs #206
   1 of 9
   migrate: no notebook-migration +2
   
2. Migrate dask/kubernetes/Dask_cuML_Exploration_Full.ipynb to deployment docs #207
   8 of 9
   migrate: yes notebook-migration +2
   
3. Migrate dask/notebooks/HPO_demo.ipynb to deployment docs #208
   6 of 9
   migrate: yes notebook-migration +2
   
4. Migrate azure/kubernetes/MNMG_XGBoost.ipynb to deployment docs #209
   0 of 9
   notebook-migration +1
   
5. Migrate azure/kubernetes/Dask_cuML_Exploration_Full.ipynb to deployment docs #210
   0 of 9
   notebook-migration +1
6. Migrate azure/notebooks/Azure-MNMG-XGBoost.ipynb to deployment docs #211
   9 of 9
   notebook-migration +1
   
7. Migrate azure/notebooks/Azure-MNMG-RF.ipynb to deployment docs #212
   0 of 9
   notebook-migration +1
8. Migrate azure/notebooks/remote-explanation/azure-gpu-shap.ipynb to deployment docs #213
   0 of 9
   notebook-migration +1
9. Migrate azure/notebooks/Train-RAPIDS.ipynb to deployment docs #214
   0 of 9
   notebook-migration +1
   
10. Migrate azure/notebooks/HPO-RAPIDS.ipynb to deployment docs #215
    migrate: yes notebook-migration +2
    
11. Migrate azure/notebooks/HPO-SKLearn.ipynb to deployment docs #216
    0 of 9
    notebook-migration +1
    
12. Migrate azure/notebooks/Train-SKLearn.ipynb to deployment docs #217
    0 of 9
    notebook-migration +1
    
13. Migrate databricks/notebooks/rapids_airline_hyperopt.ipynb to deployment docs #218
    1 of 9
    migrate: yes notebook-migration +2
14. Migrate databricks/notebooks/rapids_intro.ipynb to deployment docs #219
    1 of 9
    migrate: no notebook-migration +2
    
15. Migrate triton/kubernetes/GCP/FIL/interact_with_triton.ipynb to deployment docs #220
    0 of 9
    notebook-migration +1
16. Migrate triton/kubernetes/AWS/FIL/triton_inference.ipynb to deployment docs #221
    0 of 9
    notebook-migration +1
17. Migrate mlflow/local_environment/notebooks/databricks_mlflow_rapids.ipynb to deployment docs #222
    0 of 9
    notebook-migration +1
18. Migrate gcp/notebooks/container_build.ipynb to deployment docs #223
    1 of 9
    migrate: no notebook-migration +2
    
19. Migrate gcp/notebooks/custom_hpo.ipynb to deployment docs #224
    0 of 9
    notebook-migration +1
20. Migrate optuna/notebooks/jupytercon_demo/RAPIDS_xfeat_Optuna-CPU.ipynb to deployment docs #225
    0 of 9
    migrate: no notebook-migration +2
21. Migrate optuna/notebooks/jupytercon_demo/RAPIDS_xfeat_Optuna-GPU.ipynb to deployment docs #226
    0 of 9
    migrate: no notebook-migration +2
22. Migrate optuna/notebooks/optuna_rapids.ipynb to deployment docs #227
    1 of 9
    migrate: yes notebook-migration +2
    
23. Migrate optuna/notebooks/azure-optuna/run_optuna.ipynb to deployment docs #228
    0 of 9
    migrate: yes notebook-migration +2
24. Migrate aws/rapids_sagemaker_hpo.ipynb to deployment docs #229
    1 of 9
    migrate: no notebook-migration +2
    
25. Migrate aws/autogluon/autogluon_airline.ipynb to deployment docs #230
    0 of 9
    notebook-migration +1
26. Migrate aws/rapids_sagemaker_higgs/sagemaker_rapids_higgs.ipynb to deployment docs #231
    9 of 9
    migrate: yes notebook-migration +2
    
27. Migrate aws/gpu_tree_shap/gpu_tree_shap.ipynb to deployment docs #232
    0 of 9
    notebook-migration +1
28. Migrate aws/rapids_ec2_mnmg.ipynb to deployment docs #233
    9 of 9
    migrate: yes notebook-migration +2
    
29. Migrate aws/environment_setup/docker_ebs_volume.ipynb to deployment docs #234
    0 of 9
    notebook-migration +1
30. Migrate aws/rapids_studio_hpo.ipynb to deployment docs #235
    0 of 9
    notebook-migration +1
31. Migrate aws/rapids_intro.ipynb to deployment docs #236
    2 of 9
    migrate: yes notebook-migration +2
    
32. Migrate aws/rapids_sagemaker_hpo_extended.ipynb to deployment docs #237
    9 of 9
    migrate: yes notebook-migration +2
    
33. Migrate k8s-dask/notebooks/xgboost-gpu-hpo-job-parallel-k8s.ipynb to deployment docs #238
    9 of 9
    migrate: yes notebook-migration +2
    
34. Migrate ray/notebooks/Ray_RAPIDS_HPO.ipynb to deployment docs #239
    0 of 9
    notebook-migration +1

[jacobtomlinson]

Notebook migration guide

We are migrating the notebooks from this repo to the deployment docs. To get started pick any unassigned issue from the tasklist above and assign yourself. Unassigned notebooks with migrate: yes already set are the highest priority.

Decide if notebook should be migrated

For each notebook we need to assess it and decide whether we actually want to migrate it. If the notebook is very out of date or irrelevant to the state of RAPIDS today we shouldn't waste time with it and just let it be archived. Some notebooks are fully up to date (as noted by a link a green tick on the repo README) so definitely should be migrated and have already been marked as such. Others are in the middle.

Use your best judgement and add a migrate: yes or migrate: no label to the issue, if you decide no write a small comment in the issue on why it won't be migrated and then close the issue.

Test if notebook works

Run through the notebook and check that it works and the instructions make sense.

If the notebook needs some attention fix things up and get it into a good state.

Ensure notebook has good title and metadata tags in the first cell

The first cell of each notebook should be markdown and contain a descriptive title and a few sentences about what the notebook does.

This cell should also have cell tags to allow it to be organised by the notebook gallery Sphinx extension.

For example if the notebook is an HPO workflow running on Azure ML and uses XGBoost and cuDF it should have workflows/hpo, cloud/azure/ml, library/xgboost and library/cudf tags.

Remove any deployment instructions and link instead

Many of the notebooks have deployment instructions baked into them. We should remove or at least slim these down and link out to the deployment documentation pages.

For example if a notebook contains instructions on starting a Kubernetes cluster on Google Cloud that should be replaced with a callout to the GKE documentation page.

Copy notebook to deployment docs examples

In the Deployment docs repo create a new directory for the notebook in source/examples and copy the notebook into that directory. The notebook should also be renamed to notebook.ipynb to make the URL neater.

Copy any supporting files to the folder

If the notebook has supporting files such as Python scripts, Dockerfiles or any other config files also copy those into the directory. The notebook gallery Sphinx extension will list these files automatically in the sidebar and link to them on GitHub once they have been merged.

Add notebook to examples toctree

Also update the toctree in source/examples/index.md with the relative path to the notebook.

Make PR to deployment docs repo

Open a new Pull Request for each notebook so we can review them individually. Mark the PR as closing the issue in this repo associated with that notebook.