llama2-finetune-sagemaker-jumpstart.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "af2ee7fb-e888-4e38-a349-c7c40dfd2963",
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "tags": []
   },
   "source": [
    "# Fine-tune LLaMA 2 models on SageMaker JumpStart\n",
    "From https://github.com/generative-ai-on-aws/generative-ai-on-aws"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc9539dd-ad5e-4fd9-bc65-b75dc0ea4db0",
   "metadata": {},
   "source": [
    "# Setup "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b674854d-4a29-4737-afe0-d8b9b530e71b",
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip install datasets[s3]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13274b9b-87bd-4090-a6aa-294570c31e0e",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Deploy Pre-trained Model\n",
    "\n",
    "---\n",
    "\n",
    "First we will deploy the Llama-2 model as a SageMaker endpoint and do some inference.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "97e1d98f",
   "metadata": {
    "jumpStartAlterations": [
     "modelIdVersion"
    ],
    "tags": []
   },
   "outputs": [],
   "source": [
    "model_id, model_version = \"meta-textgeneration-llama-2-7b\", \"2.*\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1722b230-b7bc-487f-b4ee-98ca42848423",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker.jumpstart.model import JumpStartModel\n",
    "\n",
    "pretrained_model = JumpStartModel(model_id=model_id, model_version=model_version)\n",
    "pretrained_predictor = pretrained_model.deploy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b795a085-048f-42b2-945f-0cd339c1cf91",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def print_response(payload, response):\n",
    "    print(payload[\"inputs\"])\n",
    "    print(f\"> {response[0]['generation']}\")\n",
    "    print(\"\\n==================================\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5dd833f8-1ddc-4805-80b2-19e7db629880",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "payload = {\n",
    "    \"inputs\": \"I believe the meaning of life is\",\n",
    "    \"parameters\": {\n",
    "        \"max_new_tokens\": 64,\n",
    "        \"top_p\": 0.9,\n",
    "        \"temperature\": 0.6,\n",
    "        \"return_full_text\": False,\n",
    "    },\n",
    "}\n",
    "try:\n",
    "    response = pretrained_predictor.predict(payload, custom_attributes=\"accept_eula=true\")\n",
    "    print_response(payload, response)\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e19e16f-d459-40c6-9d6b-0272938b3878",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Dataset preparation for fine-tuning\n",
    "\n",
    "---\n",
    "\n",
    "You can fine-tune on the dataset with domain adaptation format or instruction tuning format. We will use a subset of [Dolly dataset](https://huggingface.co/datasets/databricks/databricks-dolly-15k) in an instruction tuning format. Dolly dataset contains roughly 15,000 instruction following records for various categories such as question answering, summarization, information extraction etc. It is available under Apache 2.0 license. \n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6dd20a0d-15a5-49b0-a330-a75755d046ed",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from datasets import load_dataset\n",
    "\n",
    "# load smaller dataset for faster training\n",
    "dolly_dataset = load_dataset(\"databricks/databricks-dolly-15k\", split=\"train[:10%]\")\n",
    "\n",
    "# Filter for question answering examples\n",
    "qa_dataset = dolly_dataset.filter(lambda example: example[\"category\"] == \"closed_qa\")\n",
    "qa_dataset = qa_dataset.remove_columns(\"category\")\n",
    "\n",
    "# We split the dataset into two where test data is used to evaluate at the end.\n",
    "train_and_test_dataset = qa_dataset.train_test_split(test_size=0.1)\n",
    "train_dataset = train_and_test_dataset[\"train\"]\n",
    "\n",
    "# Dumping the training data to a local file to be used for training.\n",
    "train_dataset.to_json(\"training.jsonl\")\n",
    "\n",
    "train_dataset[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b2e5489-33dc-4623-92da-f6fc97bd25ab",
   "metadata": {},
   "source": [
    "---\n",
    "Next, we create a prompt template for using the data in an instruction / input format for the training job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "90451114-7cf5-445c-88e3-02ccaa5d3a4b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "template = {\n",
    "    \"prompt\": \"Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\\n\\n### Instruction:\\n{instruction}\\n\\n### Input:\\n{context}\\n\\n\",\n",
    "    \"completion\": \"{response}\",\n",
    "}\n",
    "with open(\"template.json\", \"w\") as f:\n",
    "    json.dump(template, f)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a22171b1-1cec-4cec-9ce4-db62761633d9",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Upload dataset to S3\n",
    "---\n",
    "\n",
    "We will upload the prepared dataset to S3 which will be used for fine-tuning.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92ac02d1-3a59-4779-8a43-42262302bc96",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.s3 import S3Uploader\n",
    "import sagemaker\n",
    "import random\n",
    "\n",
    "bucket = sagemaker.Session().default_bucket()\n",
    "local_data_file = \"training.jsonl\"\n",
    "train_data_location = f\"s3://{bucket}/dolly_dataset\"\n",
    "S3Uploader.upload(local_data_file, train_data_location)\n",
    "S3Uploader.upload(\"template.json\", train_data_location)\n",
    "print(f\"Training data: {train_data_location}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86e61340-bc81-477d-aaf1-f37e8c554863",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Train the model\n",
    "\n",
    "Next, we fine-tune the LLaMA v2 7B model. Finetuning scripts are based on scripts provided by [this repo](https://github.com/facebookresearch/llama-recipes/tree/main).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9a71087e-9c9e-42d7-999e-5f3fac07bc4a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker.jumpstart.estimator import JumpStartEstimator\n",
    "\n",
    "estimator = JumpStartEstimator(\n",
    "    model_id=model_id,\n",
    "    model_version=model_version,\n",
    "    instance_type=\"ml.g5.12xlarge\",\n",
    "    instance_count=1,\n",
    "    environment={\"accept_eula\": \"true\"}\n",
    ")\n",
    "\n",
    "# By default, instruction tuning is set to false\n",
    "estimator.set_hyperparameters(instruction_tuned=\"True\", \n",
    "                              epoch=\"1\", \n",
    "                              max_input_length=\"1024\")\n",
    "estimator.fit({\"training\": train_data_location})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e9decbf-08c6-4cb4-8644-4a96afb5bebf",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Deploy the fine-tuned model\n",
    "---\n",
    "Next, we deploy fine-tuned model. We will compare the performance of fine-tuned and pre-trained model.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "016e591b-63f8-4e0f-941c-4b4e0b9dc6fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "finetuned_predictor = estimator.deploy()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb57904a-9631-45fe-bc3f-ae2fbb992960",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Evaluate the pre-trained and fine-tuned model\n",
    "---\n",
    "Next, we use the test data to evaluate the performance of the fine-tuned model and compare it with the pre-trained model. \n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "87085bf6-dc7e-46f3-8563-d2e4aafd0820",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from IPython.display import display, HTML\n",
    "\n",
    "test_dataset = train_and_test_dataset[\"test\"]\n",
    "\n",
    "inputs, ground_truth_responses, responses_before_finetuning, responses_after_finetuning = (\n",
    "    [],\n",
    "    [],\n",
    "    [],\n",
    "    [],\n",
    ")\n",
    "\n",
    "\n",
    "def predict_and_print(datapoint):\n",
    "    # For instruction fine-tuning, we insert a special key between input and output\n",
    "    input_output_demarkation_key = \"\\n\\n### Response:\\n\"\n",
    "\n",
    "    instruction = f\"### Instruction {datapoint['instruction']}\"\n",
    "    context = f\"### Context\\n{datapoint['context']}\" \n",
    "    prompt = \"\\n\\n\".join([i for i in [instruction, context] if i is not None])\n",
    "    \n",
    "    payload = {\n",
    "        #\"inputs\": template[\"prompt\"].format(\n",
    "        \"inputs\": prompt.format(\n",
    "            instruction=datapoint[\"instruction\"], context=datapoint[\"context\"]\n",
    "        )\n",
    "        + input_output_demarkation_key,\n",
    "        \"parameters\": {\"max_new_tokens\": 100},\n",
    "    }\n",
    "    inputs.append(payload[\"inputs\"])\n",
    "    ground_truth_responses.append(datapoint[\"response\"])\n",
    "    # Please change the following line to \"accept_eula=True\"\n",
    "    pretrained_response = pretrained_predictor.predict(\n",
    "        payload, custom_attributes=\"accept_eula=true\"\n",
    "    )\n",
    "    responses_before_finetuning.append(pretrained_response[0][\"generation\"])\n",
    "    # Please change the following line to \"accept_eula=True\"\n",
    "    finetuned_response = finetuned_predictor.predict(payload, custom_attributes=\"accept_eula=true\")\n",
    "    responses_after_finetuning.append(finetuned_response[0][\"generation\"])\n",
    "\n",
    "\n",
    "try:\n",
    "    for i, datapoint in enumerate(test_dataset.select(range(5))):\n",
    "        predict_and_print(datapoint)\n",
    "\n",
    "    df = pd.DataFrame(\n",
    "        {\n",
    "            \"Inputs\": inputs,\n",
    "            \"Ground Truth\": ground_truth_responses,\n",
    "            \"Response from non-finetuned model\": responses_before_finetuning,\n",
    "            \"Response from fine-tuned model\": responses_after_finetuning,\n",
    "        }\n",
    "    )\n",
    "    display(HTML(df.to_html()))\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6b0a0f5-ef34-40db-8ab7-c24a5d14b525",
   "metadata": {},
   "source": [
    "### Clean up resources"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d73ab2da-d00f-46db-90eb-81812898653b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Delete resources\n",
    "pretrained_predictor.delete_model()\n",
    "pretrained_predictor.delete_endpoint()\n",
    "finetuned_predictor.delete_model()\n",
    "finetuned_predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "759ce98f-a35a-4c64-9fae-50894b5e9f37",
   "metadata": {
    "tags": []
   },
   "source": [
    "# What to do next\n",
    "- Understand the various parameters in finetuning"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa35a3af-c45d-4f27-bbc6-af2c3e8bd49f",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Appendix\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d1c8c86-bfe2-4828-a7aa-dbd7a5ee075f",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Supported Inference Parameters\n",
    "\n",
    "---\n",
    "This model supports the following inference payload parameters:\n",
    "\n",
    "* **max_new_tokens:** Model generates text until the output length (excluding the input context length) reaches max_new_tokens. If specified, it must be a positive integer.\n",
    "* **temperature:** Controls the randomness in the output. Higher temperature results in output sequence with low-probability words and lower temperature results in output sequence with high-probability words. If `temperature` -> 0, it results in greedy decoding. If specified, it must be a positive float.\n",
    "* **top_p:** In each step of text generation, sample from the smallest possible set of words with cumulative probability `top_p`. If specified, it must be a float between 0 and 1.\n",
    "* **return_full_text:** If True, input text will be part of the output generated text. If specified, it must be boolean. The default value for it is False.\n",
    "\n",
    "You may specify any subset of the parameters mentioned above while invoking an endpoint. \n",
    "\n",
    "\n",
    "### Notes\n",
    "- If `max_new_tokens` is not defined, the model may generate up to the maximum total tokens allowed, which is 4K for these models. This may result in endpoint query timeout errors, so it is recommended to set `max_new_tokens` when possible. For 7B, 13B, and 70B models, we recommend to set `max_new_tokens` no greater than 1500, 1000, and 500 respectively, while keeping the total number of tokens less than 4K.\n",
    "- In order to support a 4k context length, this model has restricted query payloads to only utilize a batch size of 1. Payloads with larger batch sizes will receive an endpoint error prior to inference.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19f7e4f8-970f-4a1d-b6ee-86bc77b8b9a9",
   "metadata": {},
   "source": [
    "### Supported Hyper-parameters for fine-tuning\n",
    "---\n",
    "- epoch: The number of passes that the fine-tuning algorithm takes through the training dataset. Must be an integer greater than 1. Default: 5\n",
    "- learning_rate: The rate at which the model weights are updated after working through each batch of training examples. Must be a positive float greater than 0. Default: 1e-4.\n",
    "- instruction_tuned: Whether to instruction-train the model or not. Must be 'True' or 'False'. Default: 'False'\n",
    "- per_device_train_batch_size: The batch size per GPU core/CPU for training. Must be a positive integer. Default: 4.\n",
    "- per_device_eval_batch_size: The batch size per GPU core/CPU for evaluation. Must be a positive integer. Default: 1\n",
    "- max_train_samples: For debugging purposes or quicker training, truncate the number of training examples to this value. Value -1 means using all of training samples. Must be a positive integer or -1. Default: -1. \n",
    "- max_val_samples: For debugging purposes or quicker training, truncate the number of validation examples to this value. Value -1 means using all of validation samples. Must be a positive integer or -1. Default: -1. \n",
    "- max_input_length: Maximum total input sequence length after tokenization. Sequences longer than this will be truncated. If -1, max_input_length is set to the minimum of 1024 and the maximum model length defined by the tokenizer. If set to a positive value, max_input_length is set to the minimum of the provided value and the model_max_length defined by the tokenizer. Must be a positive integer or -1. Default: -1. \n",
    "- validation_split_ratio: If validation channel is none, ratio of train-validation split from the train data. Must be between 0 and 1. Default: 0.2. \n",
    "- train_data_split_seed: If validation data is not present, this fixes the random splitting of the input training data to training and validation data used by the algorithm. Must be an integer. Default: 0.\n",
    "- preprocessing_num_workers: The number of processes to use for the preprocessing. If None, main process is used for preprocessing. Default: \"None\"\n",
    "- lora_r: Lora R. Must be a positive integer. Default: 8.\n",
    "- lora_alpha: Lora Alpha. Must be a positive integer. Default: 32\n",
    "- lora_dropout: Lora Dropout. must be a positive float between 0 and 1. Default: 0.05. \n",
    "- int8_quantization: If True, model is loaded with 8 bit precision for training. Default for 7B/13B: False. Default for 70B: True.\n",
    "- enable_fsdp: If True, training uses Fully Sharded Data Parallelism. Default for 7B/13B: True. Default for 70B: False.\n",
    "\n",
    "Note 1: int8_quantization is not supported with FSDP. Also, int8_quantization = 'False' and enable_fsdp = 'False' is not supported due to CUDA memory issues for any of the g5 family instances. Thus, we recommend setting exactly one of int8_quantization or enable_fsdp to be 'True'\n",
    "Note 2: Due to the size of the model, 70B model can not be fine-tuned with enable_fsdp = 'True' for any of the supported instance types.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72b6d023-3487-4571-8b52-f332790c1ad7",
   "metadata": {},
   "source": [
    "### Supported Instance types\n",
    "\n",
    "---\n",
    "We have tested our scripts on the following instances types:\n",
    "\n",
    "- 7B: ml.g5.12xlarge, nl.g5.24xlarge, ml.g5.48xlarge, ml.p3dn.24xlarge\n",
    "- 13B: ml.g5.24xlarge, ml.g5.48xlarge, ml.p3dn.24xlarge\n",
    "- 70B: ml.g5.48xlarge\n",
    "\n",
    "Other instance types may also work to fine-tune. Note: When using p3 instances, training will be done with 32 bit precision as bfloat16 is not supported on these instances. Thus, training job would consume double the amount of CUDA memory when training on p3 instances compared to g5 instances.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "770d4350-cf4d-40a0-be1c-eba44efd33ab",
   "metadata": {},
   "source": [
    "### Few notes about the fine-tuning method\n",
    "\n",
    "---\n",
    "- Fine-tuning scripts are based on [this repo](https://github.com/facebookresearch/llama-recipes/tree/main). \n",
    "- Instruction tuning dataset is first converted into domain adaptation dataset format before fine-tuning. \n",
    "- Fine-tuning scripts utilize Fully Sharded Data Parallel (FSDP) as well as Low Rank Adaptation (LoRA) method fine-tuning the models\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4fbd6dcf",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 57,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.trn1.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 58,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1.32xlarge",
    "vcpuNum": 128
   },
   {
    "_defaultOrder": 59,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1n.32xlarge",
    "vcpuNum": 128
   }
  ],
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}