reverting to v22.07

multi-model-endpoints/mme-on-gpu/cv/resnet50_mme_with_gpu.ipynb

@@ -119,7 +119,7 @@
     "runtime_sm_client = boto3.client(\"sagemaker-runtime\")\n",
     "sagemaker_session = sagemaker.Session(boto_session=boto3.Session())\n",
     "bucket = sagemaker_session.default_bucket()\n",
-    "prefix = \"resnet-mme-gpu-v1\"\n",
+    "prefix = \"resnet50-mme-gpu\"\n",
     "\n",
     "# endpoint variables\n",
     "sm_model_name = f\"{prefix}-mdl-{ts}\"\n",
@@ -159,7 +159,7 @@
     "\n",
     "base = \"amazonaws.com.cn\" if region.startswith(\"cn-\") else \"amazonaws.com\"\n",
     "mme_triton_image_uri = (\n",
-    "    \"{account_id}.dkr.ecr.{region}.{base}/sagemaker-tritonserver:22.09-py3\".format(\n",
+    "    \"{account_id}.dkr.ecr.{region}.{base}/sagemaker-tritonserver:22.07-py3\".format(\n",
     "        account_id=account_id_map[region], region=region, base=base\n",
     "    )\n",
     ")"
@@ -187,7 +187,7 @@
    "source": [
     "#### Prepare PyTorch Model \n",
     "\n",
-    "`generate_model_pytorch.sh` file in the `workspace` directory contains scripts to generate a PyTorch model. First, we load a pre-trained ResNet50 model using torchvision models package. We save the model as model.pt file in TorchScript optimized and serialized format. TorchScript needs an example inputs to do a model forward pass, so we pass one instance of a RGB image with 3 color channels of dimension 224X224."
+    "`generate_model_pytorch.sh` file in the `workspace` directory contains scripts to generate a PyTorch model. First, we load a pre-trained ResNet50 model using torchvision models package. We save the model as model.pt file in TorchScript optimized and serialized format. TorchScript needs an example inputs to do a model forward pass, so we pass one instance of a RGB image with 3 color channels of dimension 224X224. The script for exporting this model can be found [here](./workspace/generate_model_pytorch.sh)"
    ]
   },
   {
@@ -200,7 +200,7 @@
    "outputs": [],
    "source": [
     "!docker run --gpus=all --rm -it \\\n",
-    "            -v `pwd`/workspace:/workspace nvcr.io/nvidia/pytorch:22.09-py3 \\\n",
+    "            -v `pwd`/workspace:/workspace nvcr.io/nvidia/pytorch:22.07-py3 \\\n",
     "            /bin/bash generate_model_pytorch.sh"
    ]
   },
@@ -211,13 +211,14 @@
    "source": [
     "#### PyTorch Model Respository\n",
     "\n",
-    "The model repository contains model to serve, in our case it will be the model.pt and configuration file with input/output specifications and metadata.\n",
+    "The model repository contains model to serve, in our case it will be the `model.pt` and configuration file with input/output specifications and metadata.\n",
     "\n",
     "```\n",
     "resnet\n",
     "├── 1\n",
     "│   └── model.pt\n",
     "└── config.pbtxt\n",
+    "\n",
     "```"
    ]
   },
@@ -228,7 +229,7 @@
    "source": [
     "#### PyTorch Model configuration\n",
     "\n",
-    "Model configuration file config.pbtxt must specify name of the model(resnet), the platform and backend properties (pytorch_libtorch), max_batch_size(128) and the input and output tensors along with the data type(TYPE_FP32) information. Additionally, you can specify instance_group and dynamic_batching properties to achieve high performance inference."
+    "Model configuration file `config.pbtxt` must specify name of the model(`resnet`), the platform and backend properties (`pytorch_libtorch`), max_batch_size(128) and the input and output tensors along with the data type(TYPE_FP32) information. Additionally, you can specify `instance_group` and `dynamic_batching` properties to achieve high performance inference."
    ]
   },
   {
@@ -273,7 +274,7 @@
    "source": [
     "#### Prepare TensorRT Model\n",
     "\n",
-    "1. We export the pre-trained ResNet model into an ONNX file, which runs the model once to trace its execution and then export the traced model to the specified file. It is one of the better options in terms model conversion and deployment when converting using ONNX.\n",
+    "1. We export the pre-trained ResNet50 model into an ONNX file, which runs the model once to trace its execution and then export the traced model to the specified file. It is one of the better options in terms model conversion and deployment when converting using ONNX.\n",
     "\n",
     "2. We use `trtexec` to automatically convert ONNX model to TensorRT plan. As ONNX is framework agnostic it works with models in TF, PyTorch and more. You will export the weights of your model from the framework and load them into your TensorRT network.\n",
     "\n",
@@ -289,7 +290,7 @@
    "outputs": [],
    "source": [
     "!docker run --gpus=all --rm -it \\\n",
-    "            -v `pwd`/workspace:/workspace nvcr.io/nvidia/pytorch:22.09-py3 \\\n",
+    "            -v `pwd`/workspace:/workspace nvcr.io/nvidia/pytorch:22.07-py3 \\\n",
     "            /bin/bash generate_model_trt.sh"
    ]
   },
@@ -317,7 +318,7 @@
    "source": [
     "#### TensorRT Model configuration\n",
     "\n",
-    " For the TensorRT model, we specify tensorrt_plan as platform, input tensor specification of the image of dimension 224X224 which has 3 color channels. Output tensor with 1000 dimensions of type TYPE_FP32 corresponding the different object categories."
+    " For the TensorRT model, we specify `tensorrt_plan` as platform, input tensor specification of the image of dimension 224X224 which has 3 color channels. Output tensor with 1000 dimensions of type TYPE_FP32 corresponding the different object categories."
    ]
   },
   {
@@ -862,6 +863,14 @@
     "sm_client.delete_endpoint_config(EndpointConfigName=endpoint_config_name)\n",
     "sm_client.delete_endpoint(EndpointName=endpoint_name)"
    ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cf87a993",
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {

multi-model-endpoints/mme-on-gpu/cv/workspace/generate_model_pytorch.sh

@@ -6,7 +6,4 @@ python pt_exporter.py
 
 # Optional Scripts
 # use this script to convert Pytorch model to ONNX format
-# python onnx_exporter.py
-
-#use this command to generate a model plan that will be used to host SageMaker Endpoint
-#trtexec --onnx=model.onnx --saveEngine=model.plan --explicitBatch --minShapes=input:1x3x224x224 --optShapes=input:128x3x224x224 --maxShapes=input:128x3x224x224 --fp16 --verbose | tee conversion.txt
+# python onnx_exporter.py