penguins/penguins-endpoint.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5558c008-6e62-49de-9139-a074dcf3c18f",
   "metadata": {},
   "source": [
    "# Penguins Endpoint\n",
    "\n",
    "This notebook contains routines and examples to interact with a SageMaker Endpoint.\n",
    "\n",
    "This notebook is part of the [Machine Learning School](https://www.ml.school) program."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "09e73d7a-a972-4d75-ae0b-57e69e65580a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "from pathlib import Path\n",
    "\n",
    "CODE_FOLDER = Path(\"code\")\n",
    "sys.path.append(f\"./{CODE_FOLDER}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "7a118412-1ce3-4db1-821a-4e7c73772361",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "import json\n",
    "import numpy as np\n",
    "import random\n",
    "import sagemaker\n",
    "import pandas as pd\n",
    "\n",
    "from constants import *\n",
    "from time import sleep\n",
    "from datetime import datetime\n",
    "from threading import Thread, Event\n",
    "from pathlib import Path\n",
    "from sagemaker import ModelPackage\n",
    "from sagemaker.predictor import Predictor\n",
    "from sagemaker.serializers import JSONSerializer\n",
    "from sagemaker.deserializers import JSONDeserializer\n",
    "from sagemaker.s3 import S3Uploader, S3Downloader\n",
    "\n",
    "\n",
    "ENDPOINT = \"penguins-endpoint\"\n",
    "MODEL_PACKAGE_GROUP = \"penguins\"\n",
    "GROUND_TRUTH_LOCATION = f\"{S3_LOCATION}/monitoring/groundtruth\" \n",
    "\n",
    "\n",
    "def get_predictor(endpoint_name, json_predictor=False):\n",
    "    \"\"\"\n",
    "    This function waits for the endpoint to be ready to serve traffic and then returns \n",
    "    a Predictor using the appropriate serializer and a deserializer. \n",
    "    \"\"\"\n",
    "    \n",
    "    waiter = sagemaker_client.get_waiter(\"endpoint_in_service\")\n",
    "    waiter.wait(\n",
    "        EndpointName=endpoint_name,\n",
    "        WaiterConfig={\n",
    "            \"Delay\": 10,\n",
    "            \"MaxAttempts\": 30\n",
    "        }\n",
    "    )\n",
    "    \n",
    "    return (\n",
    "        Predictor(endpoint_name=endpoint_name, serializer=JSONSerializer(), deserializer=JSONDeserializer()) \n",
    "        if json_predictor \n",
    "        else Predictor(endpoint_name=endpoint_name)\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75cf322a-4b8b-4d28-8b79-7689edf3c373",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "## Deploy Latest Model From Registry\n",
    "\n",
    "We can deploy a model from the Model Registry using SageMaker's SDK or the [boto3 SageMaker Client API](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b3fd68c-9d3e-4def-9fe9-bf5cc7b7e089",
   "metadata": {},
   "source": [
    "Let's create a function that we can use to test whether the model is working properly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "91e613a1-de76-4286-94ff-dab42413d956",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def test_endpoint(predictor):\n",
    "    \"\"\"\n",
    "    This is a simple function that sends a request to the endpoint and prints out\n",
    "    the results.\n",
    "    \n",
    "    Notice the payload we need to provide the model is in CSV format. The model\n",
    "    expects data that's already transformed. We can't provide the original data\n",
    "    from our dataset because the model will not work with it.\n",
    "    \"\"\"\n",
    "\n",
    "    payload = \"\"\"\n",
    "    0.6569590202313976, -1.0813829646495108, 1.2097102831892812, 0.9226343641317372, 1.0, 0.0, 0.0\n",
    "    -0.7751048801481084, 0.8822689351285553,  -1.2168066120762704, 0.9226343641317372, 0.0, 1.0, 0.0\n",
    "    -0.837387834894918, 0.3386660813829646, -0.26237731892812, -1.92351941317372, 0.0, 0.0, 1.0\n",
    "    \"\"\"\n",
    "\n",
    "    response = predictor.predict(payload, initial_args={\"ContentType\": \"text/csv\"})\n",
    "    response = json.loads(response.decode(\"utf-8\"))\n",
    "\n",
    "    print(json.dumps(response, indent=2))\n",
    "    print(f\"\\nSpecies: {np.argmax(response['predictions'], axis=1)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0eb4e2f4-2871-4e57-b192-3dc356ceec6a",
   "metadata": {},
   "source": [
    "To deploy a model from the Model Registry, we need to find its model package ARN. Let's query the list of approved models and get the latest one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "0a17312f-4ac8-44fe-90a5-fd69c0422cd7",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ModelPackageGroupName': 'penguins',\n",
       " 'ModelPackageVersion': 38,\n",
       " 'ModelPackageArn': 'arn:aws:sagemaker:us-east-1:325223348818:model-package/penguins/38',\n",
       " 'CreationTime': datetime.datetime(2023, 7, 29, 12, 52, 50, 699000, tzinfo=tzlocal()),\n",
       " 'ModelPackageStatus': 'Completed',\n",
       " 'ModelApprovalStatus': 'Approved'}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = sagemaker_client.list_model_packages(\n",
    "    ModelPackageGroupName=MODEL_PACKAGE_GROUP,\n",
    "    ModelApprovalStatus=\"Approved\",\n",
    "    SortBy=\"CreationTime\",\n",
    "    MaxResults=1,\n",
    ")\n",
    "\n",
    "package = response[\"ModelPackageSummaryList\"][0]\n",
    "package"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09b3867b-e5b8-429b-a5b2-01a543d538b3",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Deploying the Model using SageMaker's SDK"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6353871b-644c-42de-a147-0e37a6826356",
   "metadata": {},
   "source": [
    "Using the ARN of the model package from the Model Registry, we can deploy the model by creating a [ModelPackage](https://sagemaker.readthedocs.io/en/stable/api/inference/model.html#sagemaker.model.ModelPackage) instance and calling its `deploy()` function. The model information lives in the Model Registry, so we don't need to specify anything else."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "8f5ad5cd-62cc-4992-8191-22b1e4308316",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---!"
     ]
    }
   ],
   "source": [
    "model_package_arn = package[\"ModelPackageArn\"]\n",
    "\n",
    "model_package = ModelPackage(\n",
    "    model_package_arn=model_package_arn, \n",
    "    sagemaker_session=sagemaker_session,\n",
    "    role=role, \n",
    ")\n",
    "\n",
    "model_package.deploy(\n",
    "    endpoint_name=ENDPOINT,\n",
    "    initial_instance_count=1, \n",
    "    instance_type=\"ml.m5.large\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6d82447-f13e-4286-9cd9-28d33942f102",
   "metadata": {},
   "source": [
    "Let's test the endpoint to make sure it works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "efa586b5-6e17-4dd0-88c6-b5391f962f02",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"predictions\": [\n",
      "    [\n",
      "      0.0330601558,\n",
      "      0.0195660368,\n",
      "      0.947373867\n",
      "    ],\n",
      "    [\n",
      "      0.874199688,\n",
      "      0.109029919,\n",
      "      0.0167703032\n",
      "    ],\n",
      "    [\n",
      "      0.93585372,\n",
      "      0.044174917,\n",
      "      0.0199713483\n",
      "    ]\n",
      "  ]\n",
      "}\n",
      "\n",
      "Species: [2 0 0]\n"
     ]
    }
   ],
   "source": [
    "predictor = get_predictor(ENDPOINT)\n",
    "test_endpoint(predictor)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02478c1b-6b64-47fb-a411-9428daf9c32f",
   "metadata": {},
   "source": [
    "We can now delete the endpoint using the predictor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "1bd4d79b-0b33-4e7e-af99-9bfc9aaa7371",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d8fcf64-ab20-4d4f-919d-01e72b57adf1",
   "metadata": {},
   "source": [
    "### Deploying the Model using Boto3\n",
    "\n",
    "We can also deploy the model using the [boto3 SageMaker Client API](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html). We can do this in three steps:\n",
    "\n",
    "\n",
    "1. Create a model. See the [create_model](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker/client/create_model.html) function.\n",
    "2. Create the configuration of the endpoint. This configuration will specify the model you created in the first step. See the [create_endpoint_config](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker/client/create_endpoint_config.html) function.\n",
    "3. Create the endpoint using the endpoint configuration that you created before. See the [create_endpoint](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker/client/create_endpoint.html) function.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "3673636d-6a0f-4c4f-bb2a-03250d547f33",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'EndpointArn': 'arn:aws:sagemaker:us-east-1:325223348818:endpoint/penguins-endpoint',\n",
       " 'ResponseMetadata': {'RequestId': 'ee1a1e05-60d3-4119-9664-0bdce6d67c6d',\n",
       "  'HTTPStatusCode': 200,\n",
       "  'HTTPHeaders': {'x-amzn-requestid': 'ee1a1e05-60d3-4119-9664-0bdce6d67c6d',\n",
       "   'content-type': 'application/x-amz-json-1.1',\n",
       "   'content-length': '85',\n",
       "   'date': 'Tue, 25 Jul 2023 13:31:56 GMT'},\n",
       "  'RetryAttempts': 0}}"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_package_arn = package[\"ModelPackageArn\"]\n",
    "model_name = \"penguins-model\"\n",
    "endpoint_config_name = \"penguins-endpoint-config\"\n",
    "\n",
    "sagemaker_client.create_model(\n",
    "    ModelName=model_name, \n",
    "    ExecutionRoleArn=role, \n",
    "    Containers=[{\n",
    "        \"ModelPackageName\": model_package_arn\n",
    "    }] \n",
    ")\n",
    "\n",
    "sagemaker_client.create_endpoint_config(\n",
    "    EndpointConfigName=endpoint_config_name,\n",
    "    ProductionVariants=[\n",
    "        {\n",
    "            \"ModelName\": model_name,\n",
    "            \"InstanceType\": \"ml.m5.large\",\n",
    "            \"InitialVariantWeight\": 1,\n",
    "            \"InitialInstanceCount\": 1,\n",
    "            \"VariantName\": \"AllTraffic\",\n",
    "        }\n",
    "    ]\n",
    ")\n",
    "\n",
    "sagemaker_client.create_endpoint(\n",
    "    EndpointName=ENDPOINT, \n",
    "    EndpointConfigName=endpoint_config_name,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "546fe1b3-c47a-4961-ae78-6cd6365e4e4c",
   "metadata": {},
   "source": [
    "Let's test the endpoint to make sure it works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "96cd585a-dc62-4f69-b823-838bcb6d85a3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"predictions\": [\n",
      "    [\n",
      "      0.0330601558,\n",
      "      0.0195660368,\n",
      "      0.947373867\n",
      "    ],\n",
      "    [\n",
      "      0.874199688,\n",
      "      0.109029919,\n",
      "      0.0167703032\n",
      "    ],\n",
      "    [\n",
      "      0.93585372,\n",
      "      0.044174917,\n",
      "      0.0199713483\n",
      "    ]\n",
      "  ]\n",
      "}\n",
      "\n",
      "Species: [2 0 0]\n"
     ]
    }
   ],
   "source": [
    "test_endpoint(get_predictor(ENDPOINT))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f18e2e35-71c2-4efb-bba8-57ed4848c680",
   "metadata": {},
   "source": [
    "We can now delete the endpoint, the endpoint configuration, and the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "3a41af68-0c30-46e8-b9b7-3e680b8d0669",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ResponseMetadata': {'RequestId': 'e84f4a89-401c-4a34-b49a-7c14bb306d5a',\n",
       "  'HTTPStatusCode': 200,\n",
       "  'HTTPHeaders': {'x-amzn-requestid': 'e84f4a89-401c-4a34-b49a-7c14bb306d5a',\n",
       "   'content-type': 'application/x-amz-json-1.1',\n",
       "   'content-length': '0',\n",
       "   'date': 'Tue, 25 Jul 2023 13:34:11 GMT'},\n",
       "  'RetryAttempts': 0}}"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sagemaker_client.delete_endpoint(EndpointName=ENDPOINT)\n",
    "sagemaker_client.delete_endpoint_config(EndpointConfigName=endpoint_config_name)\n",
    "sagemaker_client.delete_model(ModelName=model_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00b7ba7b-433c-463c-8e5e-8b975a5be463",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Testing Custom Endpoint\n",
    "\n",
    "In this section we can test the endpoint that uses a custom inference script. We will use the function to create a predictor with a JSON encoder and decoder. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "dd1b8372-b9d7-43cc-b2b1-c45a1983252a",
   "metadata": {},
   "outputs": [
    {
     "ename": "WaiterError",
     "evalue": "Waiter EndpointInService failed: Waiter encountered a terminal failure state: Matched expected service error code: ValidationException",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mWaiterError\u001b[0m                               Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-20-7980b49c1af0>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mpredictor\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mget_predictor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mENDPOINT\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mjson_predictor\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m<ipython-input-6-2159ca705690>\u001b[0m in \u001b[0;36mget_predictor\u001b[0;34m(endpoint_name, json_predictor)\u001b[0m\n\u001b[1;32m     30\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     31\u001b[0m     \u001b[0mwaiter\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msagemaker_client\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_waiter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"endpoint_in_service\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 32\u001b[0;31m     waiter.wait(\n\u001b[0m\u001b[1;32m     33\u001b[0m         \u001b[0mEndpointName\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mendpoint_name\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     34\u001b[0m         WaiterConfig={\n",
      "\u001b[0;32m/usr/local/lib/python3.8/site-packages/botocore/waiter.py\u001b[0m in \u001b[0;36mwait\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m     53\u001b[0m     \u001b[0;31m# method.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     54\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mwait\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 55\u001b[0;31m         \u001b[0mWaiter\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwait\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     56\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     57\u001b[0m     wait.__doc__ = WaiterDocstring(\n",
      "\u001b[0;32m/usr/local/lib/python3.8/site-packages/botocore/waiter.py\u001b[0m in \u001b[0;36mwait\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m    373\u001b[0m                     \u001b[0macceptor\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexplanation\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    374\u001b[0m                 )\n\u001b[0;32m--> 375\u001b[0;31m                 raise WaiterError(\n\u001b[0m\u001b[1;32m    376\u001b[0m                     \u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    377\u001b[0m                     \u001b[0mreason\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mreason\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mWaiterError\u001b[0m: Waiter EndpointInService failed: Waiter encountered a terminal failure state: Matched expected service error code: ValidationException"
     ]
    }
   ],
   "source": [
    "predictor = get_predictor(ENDPOINT, json_predictor=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb2a35f3-9763-4ed6-b718-636907db2147",
   "metadata": {},
   "source": [
    "Running one example through the endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "d5e83ed2-8f05-4e09-8f7f-acacdb051be0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'species': 'Chinstrap', 'prediction': 1, 'confidence': 0.697722673}"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictor.predict({\n",
    "    \"island\": \"Dream\",\n",
    "    \"culmen_length_mm\": 46.4,\n",
    "    \"culmen_depth_mm\": 18.6,\n",
    "    \"flipper_length_mm\": 190.0,\n",
    "    \"body_mass_g\": 3450.0,\n",
    "})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdd50240-1c77-4b39-aa49-a05dba5b24bd",
   "metadata": {},
   "source": [
    "Running another example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "640694fd-072c-4dd6-a0a6-558e6d2bc3f6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'species': 'Gentoo', 'prediction': 2, 'confidence': 0.99367708}"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictor.predict({\n",
    "    \"island\": \"Biscoe\",\n",
    "    \"culmen_length_mm\": 48.6,\n",
    "    \"culmen_depth_mm\": 16.0,\n",
    "    \"flipper_length_mm\": 230.0,\n",
    "    \"body_mass_g\": 5800.0,\n",
    "})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28096b63-f528-44ba-9cdf-664853723ac5",
   "metadata": {},
   "source": [
    "Let's now delete the endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "d6c375f8-4805-4556-8b8c-cf1267a04f70",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3362d05f-3dac-4c8e-9243-20ebb640792e",
   "metadata": {},
   "source": [
    "## Generating Traffic\n",
    "\n",
    "To test the monitoring functionality, we need to generate some traffic to the endpoint We will repeatedly send every sample from the dataset to the endpoint to simulate real prediction requests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "28818214-a9ed-4044-8941-4dfeae6005a4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def generate_traffic(predictor):\n",
    "    \n",
    "    def _predict(data, predictor, stop_traffic_thread):\n",
    "        for index, row in data.iterrows():\n",
    "            predictor.predict(row.to_dict(), inference_id=str(index))\n",
    "            \n",
    "            sleep(1)\n",
    "\n",
    "            if stop_traffic_thread.is_set():\n",
    "                break\n",
    "\n",
    "    def _generate_prediction_data(data, predictor, stop_traffic_thread):\n",
    "        while True:\n",
    "            print(f\"Generating {data.shape[0]} predictions...\")\n",
    "            _predict(data, predictor, stop_traffic_thread)\n",
    "            \n",
    "            if stop_traffic_thread.is_set():\n",
    "                break\n",
    "\n",
    "                \n",
    "    stop_traffic_thread = Event()\n",
    "    \n",
    "    data = pd.read_csv(DATA_FILEPATH).dropna()\n",
    "    data.drop([\"sex\", \"species\"], axis=1, inplace=True)\n",
    "    \n",
    "    traffic_thread = Thread(\n",
    "        target=_generate_prediction_data,\n",
    "        args=(data, predictor, stop_traffic_thread,)\n",
    "    )\n",
    "    \n",
    "    traffic_thread.start()\n",
    "    \n",
    "    return stop_traffic_thread, traffic_thread\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37d8703e-f3dd-4373-84db-2bc564189696",
   "metadata": {},
   "source": [
    "Let's wait for the endpoint to be ready and create a predictor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "da5b6bc4-1a85-4f96-ba7d-3ec1c373aa40",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'species': 'Chinstrap', 'prediction': 1, 'confidence': 0.842793047}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictor = _get_predictor(ENDPOINT)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55cd313e-5271-431b-8e96-97e5c7aabd01",
   "metadata": {},
   "source": [
    "We can now start generating traffic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "bfeac7b4-aebc-489a-b5a0-fc330d4b9174",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generating 334 predictions...\n"
     ]
    }
   ],
   "source": [
    "stop_traffic_thread, traffic_thread = generate_traffic(predictor)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cce58505-8089-4eeb-bc22-fc2bb6f65867",
   "metadata": {},
   "source": [
    "### Introducing a Violation\n",
    "\n",
    "Let's make a prediction for a penguin and include extra fields in the request. This should be flagged by the monitoring job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "27b616f9-c9db-4c50-8908-c9feb57d5015",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "predictor.predict({\n",
    "    \"island\": \"Dream\",\n",
    "    \"culmen_length_mm\": 46.4,\n",
    "    \"culmen_depth_mm\": 18.6,\n",
    "    \"flipper_length_mm\": 190.0,\n",
    "    \"body_mass_g\": 5608.0,\n",
    "    \n",
    "    # These two columns are not in the baseline data,\n",
    "    # so they will be reported by the monitoring job\n",
    "    # as a violation.\n",
    "    \"name\": \"Johnny\",\n",
    "    \"height\": 28.0\n",
    "})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1008a068-e3fd-4fa0-9acb-c5ba09b20342",
   "metadata": {},
   "source": [
    "### Deleting Resources\n",
    "\n",
    "First, let's stop the thread genering traffic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "66e7e493-22a2-4f00-afd8-7547d060ea37",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "stop_traffic_thread.set()\n",
    "traffic_thread.join()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ed60e39-9558-4dcb-a4cf-9e4e4a8294ce",
   "metadata": {},
   "source": [
    "Let's now delete the endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "283ce0a7-ca8f-46c0-bcdf-2347b0d2f1fc",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4388c339-6c5c-4952-aef6-2e618e43682a",
   "metadata": {},
   "source": [
    "## Generating Ground Truth Data\n",
    "\n",
    "To monitor our model, we need to generate ground truth data for the samples captured by the endpoint. We can simulate this by generating a random ground truth for every sample. Check [Ingest Ground Truth Labels and Merge Them With Predictions](https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-model-quality-merge.html) for more information about this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "d828181f-312d-44a1-8f9f-e4d15a46f9db",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def generate_ground_truth_data(predictor, ground_truth_location):\n",
    "    \n",
    "    def _generate_ground_truth_record(inference_id):\n",
    "        random.seed(inference_id)\n",
    "\n",
    "        return {\n",
    "            \"groundTruthData\": {\n",
    "                \"data\": random.choice([\"Adelie\", \"Chinstrap\", \"Gentoo\"]),\n",
    "                \"encoding\": \"CSV\",\n",
    "            },\n",
    "            \"eventMetadata\": {\n",
    "                \"eventId\": str(inference_id),\n",
    "            },\n",
    "            \"eventVersion\": \"0\",\n",
    "        }\n",
    "\n",
    "\n",
    "    def _upload_ground_truth(records, upload_time):\n",
    "        records = [json.dumps(r) for r in records]\n",
    "        data = \"\\n\".join(records)\n",
    "        uri = f\"{ground_truth_location}/{upload_time:%Y/%m/%d/%H/%M%S}.jsonl\"\n",
    "\n",
    "        print(f\"Uploading ground truth data to {uri}...\")\n",
    "\n",
    "        S3Uploader.upload_string_as_file_body(data, uri)    \n",
    "\n",
    "                \n",
    "    def _generate_ground_truth_data(max_records, stop_ground_truth_thread):\n",
    "        while True:\n",
    "            records = [_generate_ground_truth_record(i) for i in range(max_records)]\n",
    "            _upload_ground_truth(records, datetime.utcnow())\n",
    "\n",
    "            if stop_ground_truth_thread.is_set():\n",
    "                break\n",
    "\n",
    "            sleep(30)\n",
    "\n",
    "                \n",
    "    stop_ground_truth_thread = Event()\n",
    "    data = pd.read_csv(DATA_FILEPATH).dropna()\n",
    "    \n",
    "    groundtruth_thread = Thread(\n",
    "        target=_generate_ground_truth_data,\n",
    "        args=(len(data), stop_ground_truth_thread,)\n",
    "    )\n",
    "    \n",
    "    groundtruth_thread.start()\n",
    "    \n",
    "    return stop_ground_truth_thread, groundtruth_thread"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba5a72f8-5365-4fdb-9603-660d246ff4d8",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor = _get_predictor(ENDPOINT)\n",
    "\n",
    "stop_ground_truth_thread, groundtruth_thread = generate_ground_truth_data(\n",
    "    predictor, \n",
    "    GROUND_TRUTH_LOCATION\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "6a972500-5650-4225-bfd4-68a9408ef6e3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Uploading ground truth data to s3://mlschool/penguins/monitoring/groundtruth/2023/07/13/13/0713.jsonl...\n"
     ]
    }
   ],
   "source": [
    "stop_ground_truth_thread.set()\n",
    "groundtruth_thread.join()"
   ]
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   }
  ],
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (TensorFlow 2.6 Python 3.8 CPU Optimized)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/tensorflow-2.6-cpu-py38-ubuntu20.04-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  },
  "lcc_arn": "arn:aws:sagemaker:us-east-1:325223348818:studio-lifecycle-config/packages",
  "toc-autonumbering": false,
  "toc-showcode": false,
  "toc-showmarkdowntxt": false
 },
 "nbformat": 4,
 "nbformat_minor": 5
}