Skip to content

Commit

Permalink
change: adding reinvent notebook examples (#941)
Browse files Browse the repository at this point in the history
* Sagemaker debugger example notebook (#38)

* add files for sagemaker debugger notebook

* updated notebook

* added subfolder

* update files

* Notebook to enable monitoring for existing Endpoints (#43)

* MNIST training with TensorFlow and CloudWatch monitoring of Rule jobs (#49)

* MNIST training with TensorFlow and CloudWatch monitoring of Rule jobs

* Model monitor end to end notebook (#53)

This notebook demonstrates the end to end flow of setting up an Endpoint with data capture, creating a baseline and setting up a monitoring schedule.

* Fixing minor issues (#51)

* SMDebugger notebook for plotting tensors in realtime (#39)

* SMDebugger notebook for plotting tensors in realtime

This shows how to run training job asynchronously and plot output of first convolutional layer activations and weights.

* Update mxnet-realtime-analysis.ipynb

* Adding TensorFlow MNIST analysis example with a custom gradient rule (#41)

* Adding TensorFlow MNIST analysis example with a custom gradient rule

* Addressing comments

* Apply suggestions from code review

* Correcting errors in the TensorFlow MNIST Custom Rule example (#47)

* Correcting errors in the TensorFlow MNIST Custom Rule example

* Correcting typos

* Tornasole -> Debugger

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Edit pass for grammar in code review

Simple editorial pass for readability.

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* SMDebugger notebook editorial review

Edit pass for readability.

* Update to use some new methods in smdebug and pysdk

* First version or README for SM Debugger (#56)

* First version or README for SM Debugger

* Update README.md

* Model monitoring visualization example (#52)

Notebook for visualizing the results generated by a monitoring execution.

* Debugger readme brief edit (#60)

Terminology udpate

* Adding example of MNIST TensorFlow analysis with Rules and reacting o… (#55)

* Adding example of MNIST TensorFlow analysis with Rules and reacting off Rule statuses with CloudWatch Events

* Correcting Typos and more instructions for Lambda setup

* One more Typo fix

* fixed example (#61)

* fixed example

* Add installing section

* Modified realtime analysis notebook for MEAD/Loosleaf (#58)

* Modified realtime analysis notebook for MEAD/Loosleaf

As a result of this notebook will only be runnable after GA as setup section is missing.

* Update mxnet version

* Update mxnet version

* Update install smdebug section

* Notebook demonstrating how to enable spot training with sagemaker debugger (#44)

* Notebook demonstrating the usage of spot training with SageMaker Debugger

* Fixed the notebook to add the checkpoint s3 uri

* Moved the files to right location.

* Updated the notebook for GA

* Added the description regarding how SageMaker works with Spot Training.

* Making some content changes

* Restructure notebooks, and update the rules notebooks (#57)

* Move tensorflow debugging to own folder

* Rename folder

* Rename files
add readme

* Move and update custom rule notebook

* Update links to notebooks in README

* Test and update for GA, the rule notebooks for TF

* Brought cloudwatch notebook to using_rules folder, and updated script name and import line, Also added to readme

* Update main readme

* Ishan's comments addressed

* fix train_volume and mxnet version

- train_volume_size should be 400GB
- mxnet version should be 1.6.0

* fixed kernel name

* fixed kernel to conda_mxnet_p36

* Add SageMaker Debugger XGBoost Rules notebook (#54)

* Add SageMaker Debugger XGBoost Rules notebook

* Rename notebook

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* XGBoost Rules notebook edit review

Simple edit pass for grammar and style.

* Address comments by rahul003 and john-andrilla

* Address comments by john-andrilla

* Add debughookconfig and rules description

* Final test and add one more feature importance plot

* Add description for SHAP

* Rename 1p rules to builtin rules

* Updated to imports

* Add SageMaker Debugger XGBoost realtime analysis notebook (#50)

* Add SageMaker Debugger XGBoost realtime analysis notebook

* Remove pre-GA setup

* Change all Debugger to Amazon SageMaker Debugger

* clean up output of cells

* Add an example notebook for data preprocessing using SageMaker Processing and Spark (#59)

Add an example notebook for data preprocessing using SageMaker Processing and Spark

* Add processing sklearn example (#63)

Add processing sklearn example

* tf-mnist-custom-rule.ipynb editorial review (#65)

Simple edit pass for grammar and style.

* XGBoost realtime analysis text edit (#64)

* Start new review

* Move notebook to a temp file for text review

* Produce text diff

* Copy to another file for edit

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* XGBoost Rules notebook edit review

Simple edit pass for grammar and readability.

* Change temp file with edits back to original

* Changes to catch up with SDK (#66)

* Add keras example, and rename folders to make it clear that they are tensorflow (#67)

* Rename to tensorflow_using_rules

* Add keras example and split notebooks into three different folders

* Rename notebook

* Incorporate editorial changes for custom rule notebook

* Another rename

* Editorial changes

* Address review comments by adding description at top of scripts, and making tf_keras tensorflow_keras

* Update tf_keras_resnet_byoc.py

* Doc update and bug fixes in TF MNIST stop training job example (#70)

* Doc update and bug fixes in TF MNIST stop training job example

* Cleanup

* Updated the text in notebook from Sagemaker-dbugger to Amazon SageMak… (#68)

* Updated the text in notebook from Sagemaker-dbugger to Amazon SageMaker Debugger

* Added try-except block around invoke_rule to catch NoMoreData exception

* Removed the empty cell

* Modified SageMaker Debugger to Amazon SageMaker Debugger

* TF MNIST buildin rule doc fix and bug fix (#69)

* TF MNIST buildin rule doc fix and bug fix

* remove output selectively

* Remove output selectively

* Fix typos (#72)

* Name Change (#73)

* name change

* fix

* fix

* Update tf-mnist-builtin-rule.ipynb

Remove the in front of Amazon SageMaker

* Update tf-keras-custom-rule.ipynb

* Update links in readme (#74)

* add sagemaker experiment management sample notebook (#40)

* add sagemaker experiment management sample notebook

* Downloading sdks and service2.json from a different s3 uri

* Address comments and update text.

* remove pre-GA cmds

* remove cleanup and rename the directory

* delete old dir

* remove cell output

* Fix intro text

* Removed unnencessary imports and used the right pip command to install smdebug (#77)

* Clear outputs except plots in xgboost debugger notebooks (#75)

* Add regression plots

* Add confusion matrix gif

* Add plots back in

* Use python -m instead of pip and use latest pysdk api

* Remove extra exclamation point

* Fix typos

* SMDebugger Example with BYOC (#45)

* Example with BYOC

* Addressed review comments

* Updates to text of notebook

* Added a rule and hook from config

* Change folder and notebook name

* Remove old notebook and script

* Remove extra files

* Update README

* Dont use import all

* Updated notebook

* Add Deep Graph Library Amazon examples for SageMaker (#48)

* Add gcn_tox21 example

* Add DGL examples for sagemaker

Including GCMC, KGE and GCN

* Add KGE hypertune example

* add more details

* Add sample notebook for AP (#62)

* Create sagemaker_automl_direct_marketing.ipynb

* Add files via upload

* Notebook for AP editorial review

Simple edit pass for grammar and style.

* Apply suggestions from code review

Applied changes from editor review.

Co-Authored-By: John Andrilla <[email protected]>

* Add files via upload

Updated to show the results of batch transform from Tanya

* Delete sagemaker_automl_direct_marketing.ipynb

Remove incorrectly named file.

* Add files via upload

Removed output

* rebase staging repo with public repo (#78)

* Adding a link to DeepRacer ArXiv paper (#915)

* adding a video that shows deepracer working in various tracks

* adding deepracer paper link to readme

* fixing line breaks

* fixing line breaks

* fixing line breaks

* fixing back slash

* Add example notebooks for AWS Step Functions Data Science SDK (#917)

* Support restoring checkpoint in ray; Add packages in dockerfile (#919)

* Support restoring checkpoint in ray; Add packages in dockerfile

* Minor formatting fix

* Allow user to specify checkpoint in two ways; move notice to launch()

* Remove extra space

* Add dict and list in _autotype()

* Recursively looking for checkpoint path

* Fixed broken links (#897)

* Adding a new sample notebook demonstrating using data from AWS Data Exchange for training a machine learning model" (#930)

* Renamed sample notebook file (#932)

* Adding a new sample notebook demonstrating using data from AWS Data Exchange for training a machine learning model"

* Rename file

* Sagemaker public notebook consistant with the AWS Deepracer console code (#927)

* Sagemaker public notebook consistant with the AWS Deepracer console code

1. Fix Exception being Thrown By 4XX Errors
2. Separate User and System Errors for S3 Calls
3. Fix Out of Memory Issue in Training Worker
4. Reduced the time to upload sim traces to S3 to 1minute befor job exits
5. Remove sending training metrics to cloudwatch as reward graphs uses training metrics from the s3 bucket
6. Upload SIM_TRACE data to s3 bucket of the launched job for training and evaluation
7. Modified mameory backend system errors as inofrmation logs as SIMAPP is not halted by these errors
8. Don't allow the car to move backwards
7. Call os._exit when simapp encounters system or user error leading to exit

* Changing the S3 bucket for the robomaker simapp

* Add sample notebooks for multi-model endpoints functionality (#935)

* Add sample notebooks for multi-model endpoints functionality.

* Add batch RL example notebook (#926)

* adding mxnet embedding serving notebook (#882)

* adding ImagNet embedding notebook from gluon vision zoo

* modified the resnet embedding following PR review

* removed the training API part, updated to mxnet 1.4, enriched the markdown doc.

* Embedding demo fix (#938)

* using a more robust artifact finding mechanism

using an environment variable to explicitely call model file by name instead of using os.listdir()

* added the working demo to the fix branch for the PR

* Add three ORL examples (#934)

* Initial commit for ORL examples

* Address comments on bin packing nb; Add news vendor

* Finish three notebooks; Add README

* Delete duplicated noteboks; Clean up cells

* Fix typo & address comments

* Add README for RL directory; Typo fix in network compression README file (#940)

* Add README for RL directory; Typo fix in network compression README file

* Modify README for batch example

* Sagemaker debugger example notebook (#38)

* add files for sagemaker debugger notebook

* updated notebook

* added subfolder

* update files

* Notebook to enable monitoring for existing Endpoints (#43)

* MNIST training with TensorFlow and CloudWatch monitoring of Rule jobs (#49)

* MNIST training with TensorFlow and CloudWatch monitoring of Rule jobs

* Model monitor end to end notebook (#53)

This notebook demonstrates the end to end flow of setting up an Endpoint with data capture, creating a baseline and setting up a monitoring schedule.

* Fixing minor issues (#51)

* SMDebugger notebook for plotting tensors in realtime (#39)

* SMDebugger notebook for plotting tensors in realtime

This shows how to run training job asynchronously and plot output of first convolutional layer activations and weights.

* Update mxnet-realtime-analysis.ipynb

* Adding TensorFlow MNIST analysis example with a custom gradient rule (#41)

* Adding TensorFlow MNIST analysis example with a custom gradient rule

* Addressing comments

* Apply suggestions from code review

* Correcting errors in the TensorFlow MNIST Custom Rule example (#47)

* Correcting errors in the TensorFlow MNIST Custom Rule example

* Correcting typos

* Tornasole -> Debugger

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Edit pass for grammar in code review

Simple editorial pass for readability.

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* SMDebugger notebook editorial review

Edit pass for readability.

* Update to use some new methods in smdebug and pysdk

* First version or README for SM Debugger (#56)

* First version or README for SM Debugger

* Update README.md

* Model monitoring visualization example (#52)

Notebook for visualizing the results generated by a monitoring execution.

* Debugger readme brief edit (#60)

Terminology udpate

* Adding example of MNIST TensorFlow analysis with Rules and reacting o… (#55)

* Adding example of MNIST TensorFlow analysis with Rules and reacting off Rule statuses with CloudWatch Events

* Correcting Typos and more instructions for Lambda setup

* One more Typo fix

* fixed example (#61)

* fixed example

* Add installing section

* Modified realtime analysis notebook for MEAD/Loosleaf (#58)

* Modified realtime analysis notebook for MEAD/Loosleaf

As a result of this notebook will only be runnable after GA as setup section is missing.

* Update mxnet version

* Update mxnet version

* Update install smdebug section

* Notebook demonstrating how to enable spot training with sagemaker debugger (#44)

* Notebook demonstrating the usage of spot training with SageMaker Debugger

* Fixed the notebook to add the checkpoint s3 uri

* Moved the files to right location.

* Updated the notebook for GA

* Added the description regarding how SageMaker works with Spot Training.

* Making some content changes

* Restructure notebooks, and update the rules notebooks (#57)

* Move tensorflow debugging to own folder

* Rename folder

* Rename files
add readme

* Move and update custom rule notebook

* Update links to notebooks in README

* Test and update for GA, the rule notebooks for TF

* Brought cloudwatch notebook to using_rules folder, and updated script name and import line, Also added to readme

* Update main readme

* Ishan's comments addressed

* fix train_volume and mxnet version

- train_volume_size should be 400GB
- mxnet version should be 1.6.0

* fixed kernel name

* fixed kernel to conda_mxnet_p36

* Add SageMaker Debugger XGBoost Rules notebook (#54)

* Add SageMaker Debugger XGBoost Rules notebook

* Rename notebook

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* XGBoost Rules notebook edit review

Simple edit pass for grammar and style.

* Address comments by rahul003 and john-andrilla

* Address comments by john-andrilla

* Add debughookconfig and rules description

* Final test and add one more feature importance plot

* Add description for SHAP

* Rename 1p rules to builtin rules

* Updated to imports

* Add SageMaker Debugger XGBoost realtime analysis notebook (#50)

* Add SageMaker Debugger XGBoost realtime analysis notebook

* Remove pre-GA setup

* Change all Debugger to Amazon SageMaker Debugger

* clean up output of cells

* Add an example notebook for data preprocessing using SageMaker Processing and Spark (#59)

Add an example notebook for data preprocessing using SageMaker Processing and Spark

* Add processing sklearn example (#63)

Add processing sklearn example

* tf-mnist-custom-rule.ipynb editorial review (#65)

Simple edit pass for grammar and style.

* XGBoost realtime analysis text edit (#64)

* Start new review

* Move notebook to a temp file for text review

* Produce text diff

* Copy to another file for edit

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* XGBoost Rules notebook edit review

Simple edit pass for grammar and readability.

* Change temp file with edits back to original

* Changes to catch up with SDK (#66)

* Add keras example, and rename folders to make it clear that they are tensorflow (#67)

* Rename to tensorflow_using_rules

* Add keras example and split notebooks into three different folders

* Rename notebook

* Incorporate editorial changes for custom rule notebook

* Another rename

* Editorial changes

* Address review comments by adding description at top of scripts, and making tf_keras tensorflow_keras

* Update tf_keras_resnet_byoc.py

* Doc update and bug fixes in TF MNIST stop training job example (#70)

* Doc update and bug fixes in TF MNIST stop training job example

* Cleanup

* Updated the text in notebook from Sagemaker-dbugger to Amazon SageMak… (#68)

* Updated the text in notebook from Sagemaker-dbugger to Amazon SageMaker Debugger

* Added try-except block around invoke_rule to catch NoMoreData exception

* Removed the empty cell

* Modified SageMaker Debugger to Amazon SageMaker Debugger

* TF MNIST buildin rule doc fix and bug fix (#69)

* TF MNIST buildin rule doc fix and bug fix

* remove output selectively

* Remove output selectively

* Fix typos (#72)

* Name Change (#73)

* name change

* fix

* fix

* Update tf-mnist-builtin-rule.ipynb

Remove the in front of Amazon SageMaker

* Update tf-keras-custom-rule.ipynb

* Update links in readme (#74)

* add sagemaker experiment management sample notebook (#40)

* add sagemaker experiment management sample notebook

* Downloading sdks and service2.json from a different s3 uri

* Address comments and update text.

* remove pre-GA cmds

* remove cleanup and rename the directory

* delete old dir

* remove cell output

* Fix intro text

* Removed unnencessary imports and used the right pip command to install smdebug (#77)

* Clear outputs except plots in xgboost debugger notebooks (#75)

* Add regression plots

* Add confusion matrix gif

* Add plots back in

* Use python -m instead of pip and use latest pysdk api

* Remove extra exclamation point

* Fix typos

* Add readme for dgl_kge example (#79)

Add Readme for dgl_kge example

* Add sagemaker experiments LL sample notebook. (#80)

* rebase staging with public repo (#81)

* Adding a link to DeepRacer ArXiv paper (#915)

* adding a video that shows deepracer working in various tracks

* adding deepracer paper link to readme

* fixing line breaks

* fixing line breaks

* fixing line breaks

* fixing back slash

* Add example notebooks for AWS Step Functions Data Science SDK (#917)

* Support restoring checkpoint in ray; Add packages in dockerfile (#919)

* Support restoring checkpoint in ray; Add packages in dockerfile

* Minor formatting fix

* Allow user to specify checkpoint in two ways; move notice to launch()

* Remove extra space

* Add dict and list in _autotype()

* Recursively looking for checkpoint path

* Fixed broken links (#897)

* Adding a new sample notebook demonstrating using data from AWS Data Exchange for training a machine learning model" (#930)

* Renamed sample notebook file (#932)

* Adding a new sample notebook demonstrating using data from AWS Data Exchange for training a machine learning model"

* Rename file

* Sagemaker public notebook consistant with the AWS Deepracer console code (#927)

* Sagemaker public notebook consistant with the AWS Deepracer console code

1. Fix Exception being Thrown By 4XX Errors
2. Separate User and System Errors for S3 Calls
3. Fix Out of Memory Issue in Training Worker
4. Reduced the time to upload sim traces to S3 to 1minute befor job exits
5. Remove sending training metrics to cloudwatch as reward graphs uses training metrics from the s3 bucket
6. Upload SIM_TRACE data to s3 bucket of the launched job for training and evaluation
7. Modified mameory backend system errors as inofrmation logs as SIMAPP is not halted by these errors
8. Don't allow the car to move backwards
7. Call os._exit when simapp encounters system or user error leading to exit

* Changing the S3 bucket for the robomaker simapp

* Add sample notebooks for multi-model endpoints functionality (#935)

* Add sample notebooks for multi-model endpoints functionality.

* Add batch RL example notebook (#926)

* adding mxnet embedding serving notebook (#882)

* adding ImagNet embedding notebook from gluon vision zoo

* modified the resnet embedding following PR review

* removed the training API part, updated to mxnet 1.4, enriched the markdown doc.

* Embedding demo fix (#938)

* using a more robust artifact finding mechanism

using an environment variable to explicitely call model file by name instead of using os.listdir()

* added the working demo to the fix branch for the PR

* Add three ORL examples (#934)

* Initial commit for ORL examples

* Address comments on bin packing nb; Add news vendor

* Finish three notebooks; Add README

* Delete duplicated noteboks; Clean up cells

* Fix typo & address comments

* Add README for RL directory; Typo fix in network compression README file (#940)

* Add README for RL directory; Typo fix in network compression README file

* Modify README for batch example

* Sagemaker debugger example notebook (#38)

* add files for sagemaker debugger notebook

* updated notebook

* added subfolder

* update files

* Notebook to enable monitoring for existing Endpoints (#43)

* MNIST training with TensorFlow and CloudWatch monitoring of Rule jobs (#49)

* MNIST training with TensorFlow and CloudWatch monitoring of Rule jobs

* Model monitor end to end notebook (#53)

This notebook demonstrates the end to end flow of setting up an Endpoint with data capture, creating a baseline and setting up a monitoring schedule.

* Fixing minor issues (#51)

* SMDebugger notebook for plotting tensors in realtime (#39)

* SMDebugger notebook for plotting tensors in realtime

This shows how to run training job asynchronously and plot output of first convolutional layer activations and weights.

* Update mxnet-realtime-analysis.ipynb

* Adding TensorFlow MNIST analysis example with a custom gradient rule (#41)

* Adding TensorFlow MNIST analysis example with a custom gradient rule

* Addressing comments

* Apply suggestions from code review

* Correcting errors in the TensorFlow MNIST Custom Rule example (#47)

* Correcting errors in the TensorFlow MNIST Custom Rule example

* Correcting typos

* Tornasole -> Debugger

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Edit pass for grammar in code review

Simple editorial pass for readability.

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* SMDebugger notebook editorial review

Edit pass for readability.

* Update to use some new methods in smdebug and pysdk

* First version or README for SM Debugger (#56)

* First version or README for SM Debugger

* Update README.md

* Model monitoring visualization example (#52)

Notebook for visualizing the results generated by a monitoring execution.

* Debugger readme brief edit (#60)

Terminology udpate

* Adding example of MNIST TensorFlow analysis with Rules and reacting o… (#55)

* Adding example of MNIST TensorFlow analysis with Rules and reacting off Rule statuses with CloudWatch Events

* Correcting Typos and more instructions for Lambda setup

* One more Typo fix

* fixed example (#61)

* fixed example

* Add installing section

* Modified realtime analysis notebook for MEAD/Loosleaf (#58)

* Modified realtime analysis notebook for MEAD/Loosleaf

As a result of this notebook will only be runnable after GA as setup section is missing.

* Update mxnet version

* Update mxnet version

* Update install smdebug section

* Notebook demonstrating how to enable spot training with sagemaker debugger (#44)

* Notebook demonstrating the usage of spot training with SageMaker Debugger

* Fixed the notebook to add the checkpoint s3 uri

* Moved the files to right location.

* Updated the notebook for GA

* Added the description regarding how SageMaker works with Spot Training.

* Making some content changes

* Restructure notebooks, and update the rules notebooks (#57)

* Move tensorflow debugging to own folder

* Rename folder

* Rename files
add readme

* Move and update custom rule notebook

* Update links to notebooks in README

* Test and update for GA, the rule notebooks for TF

* Brought cloudwatch notebook to using_rules folder, and updated script name and import line, Also added to readme

* Update main readme

* Ishan's comments addressed

* fix train_volume and mxnet version

- train_volume_size should be 400GB
- mxnet version should be 1.6.0

* fixed kernel name

* fixed kernel to conda_mxnet_p36

* Add SageMaker Debugger XGBoost Rules notebook (#54)

* Add SageMaker Debugger XGBoost Rules notebook

* Rename notebook

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_1p_rules/xgboost-regression-debugger-rules.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* XGBoost Rules notebook edit review

Simple edit pass for grammar and style.

* Address comments by rahul003 and john-andrilla

* Address comments by john-andrilla

* Add debughookconfig and rules description

* Final test and add one more feature importance plot

* Add description for SHAP

* Rename 1p rules to builtin rules

* Updated to imports

* Add SageMaker Debugger XGBoost realtime analysis notebook (#50)

* Add SageMaker Debugger XGBoost realtime analysis notebook

* Remove pre-GA setup

* Change all Debugger to Amazon SageMaker Debugger

* clean up output of cells

* Add an example notebook for data preprocessing using SageMaker Processing and Spark (#59)

Add an example notebook for data preprocessing using SageMaker Processing and Spark

* Add processing sklearn example (#63)

Add processing sklearn example

* tf-mnist-custom-rule.ipynb editorial review (#65)

Simple edit pass for grammar and style.

* XGBoost realtime analysis text edit (#64)

* Start new review

* Move notebook to a temp file for text review

* Produce text diff

* Copy to another file for edit

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* Update sagemaker-debugger/xgboost_realtime_analysis/temp-xgboost-realtime-analysis.ipynb

Co-Authored-By: John Andrilla <[email protected]>

* XGBoost Rules notebook edit review

Simple edit pass for grammar and readability.

* Change temp file with edits back to original

* Changes to catch up with SDK (#66)

* Add keras example, and rename folders to make it clear that they are tensorflow (#67)

* Rename to tensorflow_using_rules

* Add keras example and split notebooks into three different folders

* Rename notebook

* Incorporate editorial changes for custom rule notebook

* Another rename

* Editorial changes

* Address review comments by adding description at top of scripts, and making tf_keras tensorflow_keras

* Update tf_keras_resnet_byoc.py

* Doc update and bug fixes in TF MNIST stop training job example (#70)

* Doc update and bug fixes in TF MNIST stop training job example

* Cleanup

* Updated the text in notebook from Sagemaker-dbugger to Amazon SageMak… (#68)

* Updated the text in notebook from Sagemaker-dbugger to Amazon SageMaker Debugger

* Added try-except block around invoke_rule to catch NoMoreData exception

* Removed the empty cell

* Modified SageMaker Debugger to Amazon SageMaker Debugger

* TF MNIST buildin rule doc fix and bug fix (#69)

* TF MNIST buildin rule doc fix and bug fix

* remove output selectively

* Remove output selectively

* Fix typos (#72)

* Name Change (#73)

* name change

* fix

* fix

* Update tf-mnist-builtin-rule.ipynb

Remove the in front of Amazon SageMaker

* Update tf-keras-custom-rule.ipynb

* Update links in readme (#74)

* add sagemaker experiment management sample notebook (#40)

* add sagemaker experiment management sample notebook

* Downloading sdks and service2.json from a different s3 uri

* Address comments and update text.

* remove pre-GA cmds

* remove cleanup and rename the directory

* delete old dir

* remove cell output

* Fix intro text

* Removed unnencessary imports and used the right pip command to install smdebug (#77)

* Clear outputs except plots in xgboost debugger notebooks (#75)

* Add regression plots

* Add confusion matrix gif

* Add plots back in

* Use python -m instead of pip and use latest pysdk api

* Remove extra exclamation point

* Fix typos

* SMDebugger Example with BYOC (#45)

* Example with BYOC

* Addressed review comments

* Updates to text of notebook

* Added a rule and hook from config

* Change folder and notebook name

* Remove old notebook and script

* Remove extra files

* Update README

* Dont use import all

* Updated notebook

* Add Deep Graph Library Amazon examples for SageMaker (#48)

* Add gcn_tox21 example

* Add DGL examples for sagemaker

Including GCMC, KGE and GCN

* Add KGE hypertune example

* add more details

* Add sample notebook for AP (#62)

* Create sagemaker_automl_direct_marketing.ipynb

* Add files via upload

* Notebook for AP editorial review

Simple edit pass for grammar and style.

* Apply suggestions from code review

Applied changes from editor review.

Co-Authored-By: John Andrilla <[email protected]>

* Add files via upload

Updated to show the results of batch transform from Tanya

* Delete sagemaker_automl_direct_marketing.ipynb

Remove incorrectly named file.

* Add files via upload

Removed output

* rebase staging repo with public repo (#78)

* Adding a link to DeepRacer ArXiv paper (#915)

* adding a video that shows deepracer working in various tracks

* adding deepracer paper link to readme

* fixing line breaks

* fixing line breaks

* fixing line breaks

* fixing back slash

* Add example notebooks for AWS Step Functions Data Science SDK (#917)

* Support restoring checkpoint in ray; Add packages in dockerfile (#919)

* Support restoring checkpoint in ray; Add packages in dockerfile

* Minor formatting fix

* Allow user to specify checkpoint in two ways; move notice to launch()

* Remove extra space

* Add dict and list in _autotype()

* Recursively looking for checkpoint path

* Fixed broken links (#897)

* Adding a new sample notebook demonstrating using data from AWS Data Exchange for training a machine learning model" (#930)

* Renamed sample notebook file (#932)

* Adding a new sample notebook demonstrating using data from AWS Data Exchange for training a machine learning model"

* Rename file

* Sagemaker public notebook consistant with the AWS Deepracer console code (#927)

* Sagemaker public notebook consistant with the AWS Deepracer console code

1. Fix Exception being Thrown By 4XX Errors
2. Separate User and System Errors for S3 Calls
3. Fix Out of Memory Issue in Training Worker
4. Reduced the time to upload sim traces to S3 to 1minute befor job exits
5. Remove sending training metrics to cloudwatch as reward graphs uses training metrics from the s3 bucket
6. Upload SIM_TRACE data to s3 bucket of the launched job for training and evaluation
7. Modified mameory backend system errors as inofrmation logs as SIMAPP is not halted by these errors
8. Don't allow the car to move backwards
7. Call os._exit when simapp encounters system or user error leading to exit

* Changing the S3 bucket for the robomaker simapp

* Add sample notebooks for multi-model endpoints functionality (#935)

* Add sample notebooks for multi-model endpoints functionality.

* Add batch RL example notebook (#926)

* adding mxnet embedding serving notebook (#882)

* adding ImagNet embedding notebook from gluon vision zoo

* modified the resnet embedding following PR review

* removed the training API part, updated to mxnet 1.4, enriched the markdown doc.

* Embedding demo fix (#938)

* using a more robust artifact finding mechanism

using an environment variable to explicitely call model file by name instead of using os.listdir()

* added the working demo to the fix branch for the PR

* Add three ORL examples (#934)

* Initial commit for ORL examples

* Address comments on bin packing nb; Add news vendor

* Finish three notebooks; Add README

* Delete duplicated noteboks; Clean up cells

* Fix typo & address comments

* Add README for RL directory; Typo fix in network compression README file (#940)

* Add README for RL directory; Typo fix in network compression README file

* Modify README for batch example

* Sagemaker debugger example notebook (#38)

* add files for sagemaker debugger notebook

* updated notebook

* added subfolder

* update files

* Notebook to enable monitoring for existing Endpoints (#43)

* MNIST training with TensorFlow and CloudWatch monitoring of Rule jobs (#49)

* MNIST training with TensorFlow and CloudWatch monitoring of Rule jobs

* Model monitor end to end notebook (#53)

This notebook demonstrates the end to end flow of setting up an Endpoint with data capture, creating a baseline and setting up a monitoring schedule.

* Fixing minor issues (#51)

* SMDebugger notebook for plotting tensors in realtime (#39)

* SMDebugger notebook for plotting tensors in realtime

This shows how to run training job asynchronously and plot output of first convolutional layer activations and weights.

* Update mxnet-realtime-analysis.ipynb

* Adding TensorFlow MNIST analysis example with a custom gradient rule (#41)

* Adding TensorFlow MNIST analysis example with a custom gradient rule

* Addressing comments

* Apply suggestions from code review

* Correcting errors in the TensorFlow MNIST Custom Rule example (#47)

* Correcting errors in the TensorFlow MNIST Custom Rule example

* Correcting typos

* Tornasole -> Debugger

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Edit pass for grammar in code review

Simple editorial pass for readability.

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* Editorial change

Co-Authored-By: John Andrilla <[email protected]>

* SMDebugger notebook editorial review

Edit pass for readability.

* Update to use some new methods in smdebug and pysdk

* First version or README for SM Debugger (#56)

* First version or README for SM Debugger

* Update README.md

* Model monitoring visualization example (#52)

Notebook for visualizing the results g…
  • Loading branch information
chuyang-deng authored and yuanzhua committed Dec 4, 2019
1 parent 70042b8 commit 714652a
Show file tree
Hide file tree
Showing 98 changed files with 23,653 additions and 0 deletions.
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,19 @@ These examples provide more thorough mathematical treatment on a select group of
- [Latent Dirichlet Allocation (LDA)](scientific_details_of_algorithms/lda_topic_modeling) dives into Amazon SageMaker's spectral decomposition approach to LDA.
- [Linear Learner features](scientific_details_of_algorithms/linear_learner_class_weights_loss_functions) shows how to use the class weights and loss functions features of the SageMaker Linear Learner algorithm to improve performance on a credit card fraud prediction task

### Amazon SageMaker Debugger
These examples provide and introduction to SageMaker Debugger which allows debugging and monitoring capabilities for training of machine learning and deep learning algorithms. Note that although these notebooks focus on a specific framework, the same approach works with all the frameworks that Amazon SageMaker Debugger supports. The notebooks below are listed in the order in which we recommend you review them.

- [Using a built-in rule with TensorFlow](sagemaker-debugger/tensorflow_builtin_rule/)
- [Using a custom rule with TensorFlow Keras](sagemaker-debugger/tensorflow_keras_custom_rule/)
- [Interactive tensor analysis in notebook with MXNet](sagemaker-debugger/mnist_tensor_analysis/)
- [Real-time analysis in notebook with MXNet](sagemaker-debugger/mxnet_realtime_analysis/)
- [Using a built in rule with XGBoost](sagemaker-debugger/xgboost_builtin_rules/)
- [Real-time analysis in notebook with XGBoost](sagemaker-debugger/xgboost_realtime_analysis/)
- [Using SageMaker Debugger with Managed Spot Training and MXNet](sagemaker-debugger/mxnet_spot_training/)
- [Reacting to CloudWatch Events from Rules to take an action based on status with TensorFlow](sagemaker-debugger/tensorflow_action_on_rule/)
- [Using SageMaker Debugger with a custom PyTorch container](sagemaker-debugger/pytorch_custom_container/)

### Advanced Amazon SageMaker Functionality

These examples that showcase unique functionality available in Amazon SageMaker. They cover a broad range of topics and will utilize a variety of methods, but aim to provide the user with sufficient insight or inspiration to develop within Amazon SageMaker.
Expand Down Expand Up @@ -123,6 +136,13 @@ These examples provide you an introduction to how to use Neo to optimizes deep l
- [Distributed TensorFlow](sagemaker_neo_compilation_jobs/tensorflow_distributed_mnist) Adapts form [tensorflow mnist](sagemaker-python-sdk/tensorflow_distributed_mnist) including Neo API and comparsion between the baseline
- [Predicting Customer Churn](sagemaker_neo_compilation_jobs/xgboost_customer_churn) Adapts form [xgboost customer churn](introduction_to_applying_machine_learning/xgboost_customer_churn) including Neo API and comparsion between the baseline

### Amazon SageMaker Procesing

These examples show you how to use SageMaker Processing jobs to run data processing workloads.

- [Scikit-Learn Data Processing and Model Evaluation](sagemaker_processing/scikit_learn_data_processing_and_model_evaluation) shows how to use SageMaker Processing and the Scikit-Learn container to run data preprocessing and model evaluation workloads.
- [Feature transformation with Amazon SageMaker Processing and SparkML](sagemaker_processing/feature_transformation_with_sagemaker_processing) shows how to use SageMaker Processing to run data processing workloads using SparkML prior to training.

### Amazon SageMaker Pre-Built Framework Containers and the Python SDK

#### Pre-Built Deep Learning Framework Containers
Expand Down
551 changes: 551 additions & 0 deletions autopilot/sagemaker_autopilot_direct_marketing.ipynb

Large diffs are not rendered by default.

Loading

0 comments on commit 714652a

Please sign in to comment.