Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed Clean up step to use variable instead of a String. Formatted example to be in compliance with guidelines #3289

Merged
merged 5 commits into from
May 26, 2022
Merged

Fixed Clean up step to use variable instead of a String. Formatted example to be in compliance with guidelines #3289

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
54bcda9
Fixed Clean up step to use variable instead of a String. Formatted e…
Apr 5, 2022
5be5428
Update markdown for header
marckarp Apr 5, 2022
9e41372
Ran black-nb -l 100
marckarp Apr 5, 2022
16a8acd
Fix typo
marckarp Apr 5, 2022
afaa77a
Merge branch 'main' into my-fix-branch
jkroll-aws May 26, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 20 additions & 39 deletions async-inference/Async-Inference-Walkthrough.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Amazon SageMaker Asynchronous Inference\n",
"# Amazon SageMaker Asynchronous Inference\n",
"_**A new near real-time Inference option for generating machine learning model predictions**_"
]
},
Expand Down Expand Up @@ -34,7 +34,7 @@
"Asynchronous inference is a new inference option for near real-time inference needs. Requests can take up to 15 minutes to process and have payload sizes of up to 1 GB. Asynchronous inference is suitable for workloads that do not have sub-second latency requirements and have relaxed latency requirements. For example, you might need to process an inference on a large image of several MBs within 5 minutes. In addition, asynchronous inference endpoints let you control costs by scaling down endpoints instance count to zero when they are idle, so you only pay when your endpoints are processing requests. \n",
"\n",
"### Notebook scope <a id='scope'></a> \n",
"This notebook provides an introduction to the SageMaker Asynchronous inference capability. This notebook will cover the steps required to create an asynchonous inference endpoint and test it with some sample requests. \n",
"This notebook provides an introduction to the SageMaker Asynchronous inference capability. This notebook will cover the steps required to create an asynchronous inference endpoint and test it with some sample requests. \n",
"\n",
"### Overview and sample end to end flow <a id='overview'></a>\n",
"Asynchronous inference endpoints have many similarities (and some key differences) compared to real-time endpoints. The process to create asynchronous endpoints is similar to real-time endpoints. You need to create: a model, an endpoint configuration, and then an endpoint. However, there are specific configuration parameters specific to asynchronous inference endpoints which we will explore below. \n",
Expand Down Expand Up @@ -66,13 +66,7 @@
"\n",
"> The original Titanic dataset, describing the survival status of individual passengers on the Titanic. The titanic data does not contain information from the crew, but it does contain actual ages of half of the passengers. The principal source for data about Titanic passengers is the Encyclopedia Titanica. The datasets used here were begun by a variety of researchers. One of the original sources is Eaton & Haas (1994) Titanic: Triumph and Tragedy, Patrick Stephens Ltd, which includes a passenger list created by many researchers and edited by Michael A. Findlay.\n",
">\n",
"> Thomas Cason of UVa has greatly updated and improved the Titanic data frame using the Encyclopedia Titanica and created the dataset here. Some duplicate passengers have been dropped, many errors corrected, many missing ages filled in, and new variables created.\n",
">\n",
"> For more information about how this dataset was constructed:\n",
"http://biostat.mc.vanderbilt.edu/wiki/pub/Main/DataSets/titanic3info.txt\n",
">\n",
"> [1] Author: Frank E. Harrell Jr., Thomas Cason\n",
"Source: [Vanderbilt Biostatistics](http://biostat.mc.vanderbilt.edu/wiki/pub/Main/DataSets/titanic.html)"
"> Thomas Cason of UVa has greatly updated and improved the Titanic data frame using the Encyclopedia Titanica and created the dataset here. Some duplicate passengers have been dropped, many errors corrected, many missing ages filled in, and new variables created.\n"
]
},
{
Expand Down Expand Up @@ -116,6 +110,7 @@
"import sagemaker\n",
"import boto3\n",
"from time import gmtime, strftime\n",
"from datetime import datetime\n",
"\n",
"boto_session = boto3.session.Session()\n",
"sm_session = sagemaker.session.Session()\n",
Expand All @@ -129,28 +124,11 @@
"metadata": {},
"source": [
"Specify your IAM role. Go the AWS IAM console (https://console.aws.amazon.com/iam/home) and add the following policies to your IAM Role:\n",
"* SageMakerFullAccessPolicy\n",
"* Amazon S3 access: Apply this to get and put objects in your Amazon S3 bucket. Replace `bucket_name` with the name of your Amazon S3 bucket: \n",
"\n",
"```json\n",
"{\n",
" \"Version\": \"2012-10-17\",\n",
" \"Statement\": [\n",
" {\n",
" \"Action\": [\n",
" \"s3:GetObject\",\n",
" \"s3:PutObject\",\n",
" \"s3:AbortMultipartUpload\",\n",
" \"s3:ListBucket\"\n",
" ],\n",
" \"Effect\": \"Allow\",\n",
" \"Resource\": \"arn:aws:s3:::bucket_name/*\"\n",
" }\n",
" ]\n",
"}\n",
"```\n",
" * SageMakerFullAccessPolicy\n",
"\n",
"\n",
"* (Optional) Amazon SNS access: Add `sns:Publish` on the topics you define. Apply this if you plan to use Amazon SNS to receive notifications.\n",
" * (Optional) Amazon SNS access: Add `sns:Publish` on the topics you define. Apply this if you plan to use Amazon SNS to receive notifications.\n",
"\n",
"```json\n",
"{\n",
Expand All @@ -161,7 +139,7 @@
" \"sns:Publish\"\n",
" ],\n",
" \"Effect\": \"Allow\",\n",
" \"Resource\": \"arn:aws:sns:us-east-2:123456789012:MyTopic\"\n",
" \"Resource\": \"arn:aws:sns:<aws-region>:<account-id>:<topic-name>\"\n",
" }\n",
" ]\n",
"}\n",
Expand Down Expand Up @@ -209,7 +187,7 @@
"outputs": [],
"source": [
"bucket_prefix = \"async-inference-demo\"\n",
"resource_name = \"AsyncInferenceDemo\""
"resource_name = \"AsyncInferenceDemo-{}-{}\""
]
},
{
Expand Down Expand Up @@ -273,7 +251,7 @@
"metadata": {},
"outputs": [],
"source": [
"model_name = resource_name.format(\"Model\")\n",
"model_name = resource_name.format(\"Model\", datetime.now().strftime(\"%Y-%m-%d-%H-%M-%S\"))\n",
"create_model_response = sm_client.create_model(\n",
" ModelName=model_name,\n",
" ExecutionRoleArn=sm_role,\n",
Expand Down Expand Up @@ -306,7 +284,9 @@
"metadata": {},
"outputs": [],
"source": [
"endpoint_config_name = resource_name.format(\"EndpointConfig\")\n",
"endpoint_config_name = resource_name.format(\n",
" \"EndpointConfig\", datetime.now().strftime(\"%Y-%m-%d-%H-%M-%S\")\n",
")\n",
"create_endpoint_config_response = sm_client.create_endpoint_config(\n",
" EndpointConfigName=endpoint_config_name,\n",
" ProductionVariants=[\n",
Expand All @@ -322,8 +302,8 @@
" \"S3OutputPath\": f\"s3://{s3_bucket}/{bucket_prefix}/output\",\n",
" # Optionally specify Amazon SNS topics\n",
" # \"NotificationConfig\": {\n",
" # \"SuccessTopic\": \"arn:aws:sns:us-east-2:123456789012:MyTopic\",\n",
" # \"ErrorTopic\": \"arn:aws:sns:us-east-2:123456789012:MyTopic\",\n",
" # \"SuccessTopic\": \"arn:aws:sns:<aws-region>:<account-id>:<topic-name>\",\n",
" # \"ErrorTopic\": \"arn:aws:sns:<aws-region>:<account-id>:<topic-name>\",\n",
" # }\n",
" },\n",
" \"ClientConfig\": {\"MaxConcurrentInvocationsPerInstance\": 4},\n",
Expand Down Expand Up @@ -352,7 +332,8 @@
"metadata": {},
"outputs": [],
"source": [
"endpoint_name = resource_name.format(\"Endpoint\")\n",
"endpoint_name = resource_name.format(\"Endpoint\", datetime.now().strftime(\"%Y-%m-%d-%H-%M-%S\"))\n",
"\n",
"create_endpoint_response = sm_client.create_endpoint(\n",
" EndpointName=endpoint_name, EndpointConfigName=endpoint_config_name\n",
")\n",
Expand Down Expand Up @@ -623,14 +604,14 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"response = client.deregister_scalable_target(\n",
" ServiceNamespace='sagemaker',\n",
" ServiceNamespace=\"sagemaker\",\n",
" ResourceId=resource_id,\n",
" ScalableDimension='sagemaker:variant:DesiredInstanceCount'\n",
" ScalableDimension=\"sagemaker:variant:DesiredInstanceCount\",\n",
")"
]
},
Expand Down