Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up RTD Website #1 #3312

Merged
merged 29 commits into from
Apr 11, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
a73da85
rst edits
atqy Apr 8, 2022
f266799
edit introduction_to_amazon_algorithms/blazingtext_hosting_pretrained…
Apr 8, 2022
9817be5
Merge branch 'main' of https://github.com/atqy/amazon-sagemaker-examp…
Apr 8, 2022
6dcbbf2
edit introduction_to_amazon_algorithms/xgboost_mnist/xgboost_mnist.ipynb
Apr 8, 2022
4d096b2
edit aws_sagemaker_studio/sagemaker_studio_image_build/xgboost_bring_…
Apr 8, 2022
5525f98
fix autopilot/custom-feature-selection/Feature_selection_autopilot.ipynb
Apr 8, 2022
c664b2a
fix advanced_functionality/search/ml_experiment_management_using_sear…
Apr 8, 2022
143d367
edit ground_truth_labeling_jobs/from_unlabeled_data_to_deployed_machi…
Apr 8, 2022
f59ddc6
edit ground_truth_labeling_jobs/3d_point_cloud_input_data_processing/…
Apr 8, 2022
0a2d10a
edit ground_truth_labeling_jobs/ground_truth_object_detection_tutoria…
Apr 8, 2022
9a45d5a
edit autopilot/custom-feature-selection/Feature_selection_autopilot.i…
Apr 8, 2022
180542b
edit ground_truth_labeling_jobs/pretrained_model/pretrained_model_lab…
Apr 8, 2022
b2133a0
edit ground_truth_labeling_jobs/bring_your_own_model_for_sagemaker_la…
Apr 8, 2022
583510f
edit ground_truth_labeling_jobs/3d_point_cloud_input_data_processing/…
Apr 8, 2022
6b57dcb
remove broken link for deleted notebook training/algorithms.rst
Apr 8, 2022
fa8f4c1
edit aws_marketplace/creating_marketplace_products/algorithms/Bring_Y…
Apr 9, 2022
5b8a710
edit aws_marketplace/curating_aws_marketplace_listing_and_sample_note…
Apr 9, 2022
3cce924
edit aws_marketplace/curating_aws_marketplace_listing_and_sample_note…
Apr 9, 2022
2cf6acf
edit aws_marketplace/using_data/image_classification_with_shutterstoc…
Apr 9, 2022
448fbcc
edit aws_marketplace/using_model_packages/auto_insurance/automating_a…
Apr 9, 2022
3559a8b
edit aws_marketplace/using_model_packages/improving_industrial_workpl…
Apr 9, 2022
6eef950
edit aws_marketplace/using_model_packages/generic_sample_notebook/A_g…
Apr 9, 2022
4b00c4e
aws_marketplace/using_model_packages/creative-writing-using-gpt-2-te…
Apr 9, 2022
d56e1ac
edit aws_marketplace/using_model_packages/amazon_augmented_ai_with_aw…
Apr 9, 2022
61d67c7
edit aws_marketplace/using-open-source-model-packages/pytorch-ic-mode…
Apr 9, 2022
7e821ea
edit advanced_functionality/pytorch_extending_our_containers/pytorch_…
Apr 9, 2022
153fac0
advanced_functionality/scikit_bring_your_own/scikit_bring_your_own.ipynb
Apr 9, 2022
3f4d7ad
edit advanced_functionality/tensorflow_bring_your_own/tensorflow_brin…
Apr 9, 2022
a6559cf
edit sagemaker-python-sdk/tensorflow_serving_using_elastic_inference_…
Apr 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ These examples provide a gentle introduction to machine learning concepts as the

- [Targeted Direct Marketing](introduction_to_applying_machine_learning/xgboost_direct_marketing) predicts potential customers that are most likely to convert based on customer and aggregate level metrics, using Amazon SageMaker's implementation of [XGBoost](https://github.com/dmlc/xgboost).
- [Predicting Customer Churn](introduction_to_applying_machine_learning/xgboost_customer_churn) uses customer interaction and service usage data to find those most likely to churn, and then walks through the cost/benefit trade-offs of providing retention incentives. This uses Amazon SageMaker's implementation of [XGBoost](https://github.com/dmlc/xgboost) to create a highly predictive model.
- [Time-series Forecasting](introduction_to_applying_machine_learning/linear_time_series_forecast) generates a forecast for topline product demand using Amazon SageMaker's Linear Learner algorithm.
- [Cancer Prediction](introduction_to_applying_machine_learning/breast_cancer_prediction) predicts Breast Cancer based on features derived from images, using SageMaker's Linear Learner.
- [Ensembling](introduction_to_applying_machine_learning/ensemble_modeling) predicts income using two Amazon SageMaker models to show the advantages in ensembling.
- [Video Game Sales](introduction_to_applying_machine_learning/video_game_sales) develops a binary prediction model for the success of video games based on review scores.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Part 1: Packaging and Uploading your Algorithm for use with Amazon SageMaker\n",
"## Part 1: Packaging and Uploading your Algorithm for use with Amazon SageMaker\n",
"\n",
"### An overview of Docker\n",
"\n",
Expand Down Expand Up @@ -517,7 +517,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Part 2: Training and Hosting your Algorithm in Amazon SageMaker\n",
"## Part 2: Training and Hosting your Algorithm in Amazon SageMaker\n",
"Once you have your container packaged, you can use it to train and serve models. Let's do that with the algorithm we made above.\n",
"\n",
"## Set up the environment\n",
Expand Down Expand Up @@ -684,7 +684,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Reference\n",
"## Reference\n",
"- [How Amazon SageMaker interacts with your Docker container for training](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-training-algo.html)\n",
"- [How Amazon SageMaker interacts with your Docker container for inference](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-inference-code.html)\n",
"- [CIFAR-10 Dataset](https://www.cs.toronto.edu/~kriz/cifar.html)\n",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Part 1: Packaging and Uploading your Algorithm for use with Amazon SageMaker\n",
"## Part 1: Packaging and Uploading your Algorithm for use with Amazon SageMaker\n",
"\n",
"### An overview of Docker\n",
"\n",
Expand Down Expand Up @@ -303,7 +303,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Part 2: Using your Algorithm in Amazon SageMaker\n",
"## Part 2: Using your Algorithm in Amazon SageMaker\n",
"\n",
"Once you have your container packaged, you can use it to train models and use the model for hosting or batch transforms. Let's do that with the algorithm we made above.\n",
"\n",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
"tags": []
},
"source": [
"# Introduction\n",
"## Introduction\n",
"\n",
"Welcome to our example introducing Amazon SageMaker Search! Amazon SageMaker Search lets you quickly find and evaluate the most relevant model training runs from potentially hundreds and thousands of your Amazon SageMaker model training jobs.\n",
"Developing a machine learning model requires continuous experimentation, trying new learning algorithms and tuning hyper parameters, all the while observing the impact of such changes on model performance and accuracy. This iterative exercise often leads to explosion of hundreds of model training experiments and model versions, slowing down the convergence and discovery of “winning” model. In addition, the information explosion makes it very hard down the line to trace back the lineage of a model version i.e. the unique combination of datasets, algorithms and parameters that brewed that model in the first place. \n",
Expand Down Expand Up @@ -292,7 +292,7 @@
"tags": []
},
"source": [
"# Training the linear model\n",
"## Training the linear model\n",
"Once we have the data preprocessed and available in the correct format for training, the next step is to actually train the model using the data. First, let's specify our algorithm container. More details on algorithm containers can be found in [AWS documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-algo-docker-registry-paths.html)."
]
},
Expand Down Expand Up @@ -457,7 +457,7 @@
"tags": []
},
"source": [
"# Use Amazon SageMaker Search to organize and evaluate experiments\n",
"## Use Amazon SageMaker Search to organize and evaluate experiments\n",
"Usually you will experiment with tuning multiple hyperparameters or even try new learning algorithms and training datasets resulting in potentially hundreds of model training runs and model versions. However, for the sake of simplicity, we are only tuning mini_batch_size in this example, trying only three different values resulting in as many model versions. Now we will use [Search](https://docs.aws.amazon.com/sagemaker/latest/dg/search.html) to **group together** the three model training runs and **evaluate** the best performing model by ranking and comparing them on a metric of our choice. \n",
"\n",
"**For grouping** the relevant model training runs together, we will search the model training jobs by the unique label or tag that we have been using as a tracking label to track our experiments. \n",
Expand Down Expand Up @@ -556,7 +556,7 @@
"tags": []
},
"source": [
"# Set up hosting for the model\n",
"## Set up hosting for the model\n",
"Now that we've found our best performing model (in this example the one with mini_batch_size=100), we can deploy it behind an Amazon SageMaker real-time hosted endpoint. This will allow out to make predictions (or inference) from the model dyanamically."
]
},
Expand Down Expand Up @@ -601,7 +601,7 @@
"tags": []
},
"source": [
"# Tracing the lineage of a model starting from an endpoint\n",
"## Tracing the lineage of a model starting from an endpoint\n",
"Now we will present an example of how you can use the Amazon SageMaker Search to trace the antecedents of a model deployed at an endpoint i.e. unique combination of algorithms, datasets, and parameters that brewed the model in first place."
]
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Part 1: Packaging and Uploading your Algorithm for use with Amazon SageMaker\n",
"## Part 1: Packaging and Uploading your Algorithm for use with Amazon SageMaker\n",
"\n",
"### An overview of Docker\n",
"\n",
Expand Down Expand Up @@ -449,7 +449,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Part 2: Training and Hosting your Algorithm in Amazon SageMaker\n",
"## Part 2: Training and Hosting your Algorithm in Amazon SageMaker\n",
"Once you have your container packaged, you can use it to train and serve models. Let's do that with the algorithm we made above.\n",
"\n",
"## Set up the environment\n",
Expand Down Expand Up @@ -608,7 +608,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Reference\n",
"## Reference\n",
"- [How Amazon SageMaker interacts with your Docker container for training](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-training-algo.html)\n",
"- [How Amazon SageMaker interacts with your Docker container for inference](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-inference-code.html)\n",
"- [CIFAR-10 Dataset](https://www.cs.toronto.edu/~kriz/cifar.html)\n",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"In some cases, customer wants to have the flexibility to bring custom data processing code to SageMaker Autopilot. For example, customer might have datasets with large number of independent variables. Customer would like to have a custom feature selection step to remove irrelevant variables first. The resulted smaller dataset is then used to launch SageMaker Autopilot job. Customer would also like to include both the custom processing code and models from SageMaker Autopilot for easy deployment—either on a real-time endpoint or for batch processing. We will demonstrate how to achieve this in this notebook. \n",
"\n",
"\n",
"### Table of contents\n",
"## Table of contents\n",
"* [Setup](#setup)\n",
" * [Generate dataset](#data_gene)\n",
" * [Upload data to S3](#upload)\n",
Expand All @@ -29,7 +29,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Setup <a class=\"anchor\" id=\"setup\"></a>"
"## Setup <a class=\"anchor\" id=\"setup\"></a>"
]
},
{
Expand Down Expand Up @@ -139,7 +139,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Feature Selection <a class=\"anchor\" id=\"feature_selection\"></a>\n",
"## Feature Selection <a class=\"anchor\" id=\"feature_selection\"></a>\n",
"\n",
"We use Scikit-learn on Sagemaker `SKLearn` Estimator with a feature selection script as an entry point. The script is very similar to a training script you might run outside of SageMaker, but you can access useful properties about the training environment through various environment variables, such as:\n",
"\n",
Expand Down Expand Up @@ -439,7 +439,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Autopilot <a class=\"anchor\" id=\"autopilot\"></a>"
"## Autopilot <a class=\"anchor\" id=\"autopilot\"></a>"
]
},
{
Expand Down Expand Up @@ -652,7 +652,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Serial Inference Pipeline that combines feature selection and autopilot <a class=\"anchor\" id=\"serial_inference\"></a>\n"
"## Serial Inference Pipeline that combines feature selection and autopilot <a class=\"anchor\" id=\"serial_inference\"></a>\n"
]
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Part 1: Packaging and Uploading your Algorithm for use with Amazon SageMaker\n",
"## Part 1: Packaging and Uploading your Algorithm for use with Amazon SageMaker\n",
"\n",
"### An overview of Docker\n",
"\n",
Expand Down Expand Up @@ -286,7 +286,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Testing your algorithm on your local machine or on an Amazon SageMaker notebook instance\n",
"### Testing your algorithm on your local machine or on an Amazon SageMaker notebook instance\n",
"\n",
"While you're first packaging an algorithm use with Amazon SageMaker, you probably want to test it yourself to make sure it's working right. In the directory `container/local_test`, there is a framework for doing this. It includes three shell scripts for running and using the container and a directory structure that mimics the one outlined above.\n",
"\n",
Expand All @@ -303,11 +303,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Part 2: Training, Batch Inference and Hosting your Algorithm in Amazon SageMaker\n",
"## Part 2: Training, Batch Inference and Hosting your Algorithm in Amazon SageMaker\n",
"\n",
"Once you have your container packaged, you can use it to train and serve models. Let's do that with the algorithm we made above.\n",
"\n",
"## Set up the environment\n",
"### Set up the environment\n",
"\n",
"Here we specify a bucket to use and the role that will be used for working with Amazon SageMaker."
]
Expand All @@ -333,7 +333,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create the session\n",
"### Create the session\n",
"\n",
"The session remembers our connection parameters to Amazon SageMaker. We'll use it to perform all of our SageMaker operations."
]
Expand All @@ -353,7 +353,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Upload the data for training\n",
"### Upload the data for training\n",
"\n",
"When training large models with huge amounts of data, you'll typically use big data tools, like Amazon Athena, AWS Glue, or Amazon EMR, to create your data in S3. For the purposes of this example, we're using some the classic [Iris dataset](https://en.wikipedia.org/wiki/Iris_flower_data_set), which we have included. \n",
"\n",
Expand All @@ -376,7 +376,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create an estimator and fit the model\n",
"### Create an estimator and fit the model\n",
"\n",
"In order to use Amazon SageMaker to fit our algorithm, we'll create an `Estimator` that defines how to use the container to train. This includes the configuration we need to invoke SageMaker training:\n",
"\n",
Expand Down Expand Up @@ -422,7 +422,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Batch Transform Job\n",
"### Batch Transform Job\n",
"\n",
"Now let's use the model built to run a batch inference job and verify it works.\n"
]
Expand All @@ -431,7 +431,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Batch Transform Input Preparation\n",
"#### Batch Transform Input Preparation\n",
"\n",
"The snippet below is removing the \"label\" column (column indexed at 0) and retaining the rest to be batch transform's input. \n",
"\n",
Expand Down Expand Up @@ -463,7 +463,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Run Batch Transform\n",
"#### Run Batch Transform\n",
"\n",
"Now that our batch transform input is setup, we run the transformation job next"
]
Expand All @@ -485,7 +485,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Inspect the Batch Transform Output in S3"
"##### Inspect the Batch Transform Output in S3"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does 5 headers deep look okay on GitHub and RTD?

Copy link
Member Author

@atqy atqy Apr 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks fine to me at https://atqy-amazon-sagemaker-examples.readthedocs.io/en/atqy-correct-rtd-rst/aws_marketplace/index.html (it is the first notebook listed). I generally don't agree with that many level of headers but am just following what the author initially wrote. Let me know if you think it is not fine though.

]
},
{
Expand All @@ -511,7 +511,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deploy the model\n",
"### Deploy the model\n",
"\n",
"Deploying the model to Amazon SageMaker hosting just requires a `deploy` call on the fitted model. This call takes an instance count, instance type, and optionally serializer and deserializer functions. These are used when the resulting predictor is created on the endpoint."
]
Expand All @@ -532,7 +532,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Choose some data and use it for a prediction\n",
"#### Choose some data and use it for a prediction\n",
"\n",
"In order to do some predictions, we'll extract some of the data we used for training and do predictions against it. This is, of course, bad statistical practice, but a good way to see how the mechanism works."
]
Expand Down Expand Up @@ -576,7 +576,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Cleanup Endpoint\n",
"#### Cleanup Endpoint\n",
"\n",
"When you're done with the endpoint, you'll want to clean it up."
]
Expand All @@ -594,7 +594,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Part 3 - Package your resources as an Amazon SageMaker Algorithm\n",
"## Part 3 - Package your resources as an Amazon SageMaker Algorithm\n",
"(If you looking to sell a pretrained model (ModelPackage), please skip to Part 4.)\n",
"\n",
"Now that you have verified that the algorithm code works for training, live inference and batch inference in the above sections, you can start packaging it up as an Amazon SageMaker Algorithm."
Expand All @@ -615,7 +615,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Algorithm Definition\n",
"### Algorithm Definition\n",
"\n",
"SageMaker Algorithm is comprised of 2 parts:\n",
"\n",
Expand Down Expand Up @@ -741,7 +741,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Putting it all together\n",
"### Putting it all together\n",
"\n",
"Now we put all the pieces together in the next cell and create an Amazon SageMaker Algorithm"
]
Expand Down Expand Up @@ -777,7 +777,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Describe the algorithm\n",
"#### Describe the algorithm\n",
"\n",
"The next cell describes the Algorithm and waits until it reaches a terminal state (Completed or Failed)"
]
Expand Down Expand Up @@ -805,11 +805,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Part 4 - Package your resources as an Amazon SageMaker ModelPackage\n",
"## Part 4 - Package your resources as an Amazon SageMaker ModelPackage\n",
"\n",
"In this section, we will see how you can package your artifacts (ECR image and the trained artifact from your previous training job) into a ModelPackage. Once you complete this, you can list your product as a pretrained model in the AWS Marketplace.\n",
"\n",
"## Model Package Definition\n",
"### Model Package Definition\n",
"A Model Package is a reusable model artifacts abstraction that packages all ingredients necessary for inference. It consists of an inference specification that defines the inference image to use along with an optional model weights location.\n"
]
},
Expand Down Expand Up @@ -892,7 +892,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Putting it all together\n",
"### Putting it all together\n",
"\n",
"Now we put all the pieces together in the next cell and create an Amazon SageMaker Model Package."
]
Expand Down Expand Up @@ -951,7 +951,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Debugging Creation Issues\n",
"### Debugging Creation Issues\n",
"\n",
"Entity creation typically never fails in the synchronous path. However, the validation process can fail for many reasons. If the above Algorithm creation fails, you can investigate the cause for the failure by looking at the \"AlgorithmStatusDetails\" field in the Algorithm object or \"ModelPackageStatusDetails\" field in the ModelPackage object. You can also look for the Training Jobs / Transform Jobs created in your account as part of our validation and inspect their logs for more hints on what went wrong. \n",
"\n",
Expand All @@ -963,7 +963,7 @@
"metadata": {},
"source": [
"\n",
"## List on AWS Marketplace\n",
"### List on AWS Marketplace\n",
"\n",
"Next, please go back to the Amazon SageMaker console, click on \"Algorithms\" (or \"Model Packages\") and you'll find the entity you created above. If it was successfully created and validated, you should be able to select the entity and \"Publish new ML Marketplace listing\" from SageMaker console.\n",
"<img src=\"images/publish-to-marketplace-action.png\"/>"
Expand Down
Loading