UseCases/CancerPrediction_TDApiClient/CancerPrediction_tdapiClient_VertexAI_CustomModel.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<header>\n",
    "   <p  style='font-size:36px;font-family:Arial;color:#F0F0F0; background-color: #00233c; padding-left: 20pt; padding-top: 20pt;padding-bottom: 10pt; padding-right: 20pt;'>\n",
    "       Cancer Prediction using Google Vertex AI with TDApiClient using Custom Model\n",
    "  <br>\n",
    "       <img id=\"teradata-logo\" src=\"https://storage.googleapis.com/clearscape_analytics_demo_data/DEMO_Logo/teradata.svg\" alt=\"Teradata\" style=\"width: 125px; height: auto; margin-top: 20pt;\">\n",
    "    </p>\n",
    "</header>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Introduction</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Worldwide, breast cancer is the most common type of cancer in women and the second highest in terms of mortality rates. Diagnosis of breast cancer is performed when an abnormal lump is found (from self-examination or x-ray) or a tiny speck of calcium is seen (on an x-ray). After a suspicious lump is found, the doctor will conduct a diagnosis to determine whether it is cancerous and, if so, whether it has spread to other parts of the body. Vantage Clearscape Analytics provides us various machine learning techniques to develop predictive models for cancer diagnosis. Features are computed from a digitized image of a fine needle aspirate (FNA) of a breast mass, using descriptions that define the characteristics of the cell nuclei.</p> \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Vertex AI empowers machine learning developers, data scientists, and data engineers to take their projects from ideation to deployment, quickly and cost-effectively. With the Teradata Vantage API_Request feature directly from Vantage, we can connect to these Vertex AI endpoints through a function to do real-time scoring on data. Google provides the Vertex AI Python SDK, which includes TrainingJob classes for training models on the Vertex AI platform. For the workflow, training runs on the Vertex AI platform. The model is then deployed on a Vertex AI online endpoint and on Teradata Vantage.</p>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Business Values</b></p>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Comprehensive health predictions and a reduced number of false positive and false negative results.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Reduced cost to patients and hospitals caused by cancer.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Identify patterns and symptoms leading to breast cancer to ensure early intervention.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Advanced research and development stemming from the results of the data and models produced.</li></p>\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Why Vantage?</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Machine Learning and AI have a proven track record of improving patient outcomes and well-being across the entire healthcare industry. Traditional approaches to data preparation, model development, and deployment rely on manual, error-prone processes that prevent enterprises from realizing the true value of these tools and techniques.</p>\n",
    " \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>However, Vantage provides these same proven data preparation and machine learning capabilities, integrated as native ClearScape Analytic functions.  This allows organizations to drastically reduce data preparation, model development, and testing time, while allowing for much more frequent and iterative testing and tuning to ensure maximum life-critical accuracy.</p>\n",
    " \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Furthermore, the exact same development pipeline can be deployed seamlessly to production, eliminating the traditional development-to-deployment gap in the ML and AI industry.</p>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>1. Initial setup</b></p>\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Install packages</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In the section, we will install the TDApiClient packages along with the necessary packages needed for the TDApiClient package.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "# # '%%capture' suppresses the display of installation steps of the following packages\n",
    "!pip install --upgrade numpy pyopenssl\n",
    "!pip install tdapiclient\n",
    "!pip install google-cloud-aiplatform"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Note: </b><i>The above statements may need to be uncommented if you run the notebooks on a platform other than ClearScape Analytics Experience that does not have the libraries installed. If you uncomment those installs, be sure to restart the kernel after executing those lines to bring the installed libraries into memory. The simplest way to restart the Kernel is by typing zero zero: <b> 0 0</b></i></p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Setting up Google Cloud Vertex AI credentials</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-warning\">\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Note: </b><i>This notebook cannot be executed if you do not have a Vertex AI service account with the necessary permissions. Information regarding the Vertex AI account and the necessary permissions is given below</i></p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Required Vertex AI Credentials:</b>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>GOOGLE_APPLICATION_CREDENTIALS:</b> The json file which has credentials for the service account.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>GCP_REGION:</b> Region for the service account and the storage bucket.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>GCP_PROJECT_ID:</b> Project ID</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>GCP_TD_AUTH_OBJ:</b> Authorization object created in the database for Google Cloud</li>\n",
    "</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>How to Get These Inputs:</b>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>GOOGLE_APPLICATION_CREDENTIALS, GCP_REGION, GCP_PROJECT_ID and GCP_TD_AUTH_OBJ: These credentials are related to your Google account and subscription. If you already have an Google account and an active subscription for the Vertex AI service account, you can find these credentials in the Google cloud portal. Here's how:\n",
    "    \n",
    "<ul style=\"font-size: 16px; font-family: Arial;;color:#00233C\">\n",
    "            <li><a href=\"https://developers.google.com/workspace/guides/create-credentials#:~:text=Click%20Keys%20%3E%20Add%20key%20%3E%20Create,json%20in%20your%20working%20directory.\">Download json with GCP Application credentials</a></li>\n",
    "            <li><a href=\"https://cloud.google.com/compute/docs/regions-zones\">Find your GCP Region</a></li>\n",
    "            <li><a href=\"https://cloud.google.com/resource-manager/docs/creating-managing-projects\">Find your Project ID</a></li>\n",
    "    <li><a href=\"https://docs.teradata.com/r/Enterprise_IntelliFlex_VMware/SQL-Data-Definition-Language-Syntax-and-Examples/Authorization-Statements-for-External-Routines/CREATE-AUTHORIZATION-and-REPLACE-AUTHORIZATION/CREATE-AUTHORIZATION-and-REPLACE-AUTHORIZATION-Syntax\">GCP Authorization object in Teradata database</a></li>\n",
    "        </ul>\n",
    "</p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Below is the sample SQL for creation of the GCP Authorization object in Teradata database</p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><code>CREATE AUTHORIZATION Auth_S3_USR_IVO\n",
    "USER 'service-account-name@project-id.iam.gserviceaccount.com'\n",
    "PASSWORD '-----BEGIN PRIVATE KEY-----\\n\n",
    "MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCyfg3398iOxjt...almSAvk9SqPoyZ\n",
    "R7JJFs=\\n -----END PRIVATE KEY-----\\n';\n",
    "</code>\n",
    "</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We will also have to create a Google cloud bucket where we will be uploading our code and artifacts when using the Vertex AI models. If you already have a bucket created you can use the same else to create a new bucket check this <a href=\"https://cloud.google.com/storage/docs/creating-buckets?_gl=1*lboq4y*_ga*MTQyNzA2MDA1OC4xNzEwMzEzMTcw*_ga_WH2QY8WWF5*MTcxMTA4MDY3MC4xOS4xLjE3MTEwODIzMzguMC4wLjA.&_ga=2.269165169.-1427060058.1710313170\">link</a>. We also need to check the permissions needed for these buckets to be used by the Vertex AI API calls ad mentioned <a href=\"https://cloud.google.com/vertex-ai/docs/general/access-control\"> here.</a></p> \n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Since we are going to use the Vertex AI Pre Built Containers and access them using the APIs, we will have to Enable the API as mentioned <a href= \"https://cloud.google.com/vertex-ai/docs/start/cloud-environment\">here</a></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We can also create a new service account using the<a href= \" https://cloud.google.com/vertex-ai/docs/general/custom-service-account\"> Vertex AI custom service account </a></p>    \n",
    "   \n",
    "</p>\n",
    "\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In case you do not have a Google cloud account you can create one by following the steps mentioned <a href=\"https://cloud.google.com/free/?utm_source=google&utm_medium=cpc&utm_campaign=japac-IN-all-en-dr-BKWS-all-cloud-trial-EXA-dr-1605216&utm_content=text-ad-none-none-DEV_c-CRE_634320416384-ADGP_Hybrid+%7C+BKWS+-+EXA+%7C+Txt+-GCP-General-google+cloud+account-how-KWID_43700074200360382-kwd-310570337956&userloc_9062101-network_g&utm_term=KW_how+to+create+google+cloud+account&gad_source=1&gclid=Cj0KCQjw2PSvBhDjARIsAKc2cgOOL7cfu4LRkX9-nIT7sVizw8ubKIl2aYUXkAdbxxFnXpHo6lYk7CAaAiTNEALw_wcB&gclsrc=aw.ds&hl=en\">here</a>.</p> "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>2. Connect to Vantage</b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In the section, we import the required libraries and set environment variables and environment paths (if required).</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from teradataml import *\n",
    "from tdapiclient import create_tdapi_context, TDApiClient, remove_tdapi_context\n",
    "from sklearn import metrics\n",
    "from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay\n",
    "import matplotlib.pyplot as plt\n",
    "configure.byom_install_location = \"mldb\"\n",
    "configure.val_install_location = \"val\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We will be prompted to provide the password. We will enter the password, press the Enter key, and then use the down arrow to go to the next cell. Begin running steps with Shift + Enter keys.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%run -i ../startup.ipynb\n",
    "eng = create_context(host = 'host.docker.internal', username='demo_user', password = password)\n",
    "print(eng)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "execute_sql('''SET query_band='DEMO=CancerPrediction_TDApiClient_VertexAI_CustomModel.ipynb;' UPDATE FOR SESSION; ''')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>3. Getting Data for This Demo</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We have provided data for this demo on cloud storage. We have the option of either running the demo using foreign tables to access the data without using any storage on our environment or downloading the data to local storage, which may yield somewhat faster execution. However, we need to consider available storage. There are two statements in the following cell, and one is commented out. We may switch which mode we choose by changing the comment string.</p>   \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# %run -i ../run_procedure.py \"call get_data('DEMO_CancerPrediction_cloud');\"\n",
    " # Takes about 50 seconds\n",
    "%run -i ../run_procedure.py \"call get_data('DEMO_CancerPrediction_local');\"\n",
    " # Takes about 2 minute 30 secs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Optional step – We should execute the below step only if we want to see the status of databases/tables created and space used.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%run -i ../run_procedure.py \"call space_report();\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>4. Analyze the raw data set</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Let us start by creating a teradataml dataframe. A \"Virtual DataFrame\" that points directly to the dataset in Vantage.</p>\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Creating a teradataml dataframe using the table.\n",
    "df = DataFrame(in_schema(\"DEMO_CancerPrediction\",\"Patient_Data\"))\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>5. Data Preparation</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Label encoding a categorical data column is done to re-express existing values of a column (variable) into a new coding scheme or to correct data quality problems and focus an analysis of a particular value. It allows\n",
    "    for mapping individual values, NULL values, or any number of remaining values (ELSE option) to a new value, a NULL value or the same value. Label encoding supports charter, numeric, and date type columns.</p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Output of this function is passed to \"label_encode\" argument of \"Transform\" function from Vantage Analytic Library.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Encoding the target column using label encoder.\n",
    "from teradataml import LabelEncoder \n",
    "rc = LabelEncoder(values=(\"M\", 1), columns=[\"diagnosis\"], default=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_columns_names= Retain(columns=[\"radius_mean\",\"texture_mean\",\"perimeter_mean\",\"area_mean\",\"smoothness_mean\",\n",
    "                                       \"compactness_mean\",\"concavity_mean\",\"concave_points_mean\",\"symmetry_mean\",\n",
    "                                       \"fractal_dimension_mean\",\"radius_se\",\"texture_se\",\"perimeter_se\",\"area_se\",\n",
    "                                       \"smoothness_se\",\"compactness_se\",\"concavity_se\",\"concave_points_se\",\n",
    "                                       \"symmetry_se\",\"fractal_dimension_se\",\"radius_worst\",\"texture_worst\",\n",
    "                                       \"perimeter_worst\",\"area_worst\",\"smoothness_worst\",\"compactness_worst\",\n",
    "                                       \"concavity_worst\",\"concave_points_worst\",\"symmetry_worst\",\n",
    "                                       \"fractal_dimension_worst\" ])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'> The Variable Transformation analysis reads a teradataml DataFrame and produces an output containing transformed columns. This is useful when preparing data for input to an analytic algorithm. For example, a K-Means Clustering algorithm typically produces better results when the input columns are first converted to their Z-Score values to put all input variables on an equal footing, regardless of their magnitude.</p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Function supports following transformations:</p>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>Binning</code> - Binning replaces a continuous numeric column with a categorical one to produce ordinal values (for example, numeric categorical values where order is meaningful).</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>Derive</code> - The Derive transformation requires the free-form transformation be specified as a formula.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>One Hot Encoding</code> - One Hot Encoding is useful when a categorical data element must be re-expressed as one or more numeric data elements, creating a binary numeric field for each categorical data value.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>Missing Value</code> Treatment or Null Replacement.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>Label Encoding</code> - Allows to re-express existing values of a categorical data column (variable) into a new coding scheme or to correct data quality problems and focus an analysis on a value.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>Min-Max Scaling</code> - Limits the upper and lower boundaries of the data in a continuous numeric column using a linear rescaling function based on maximum and minimum data values.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>Retain</code> - Allows copying of one or more columns into the final analytic data set.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>Sigmoid</code> - Provides rescaling of continuous numeric data using a type of sigmoid or s-shaped function.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>ZScore</code> - Provides rescaling of continuous numeric data using Z-Scores.</li></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Here, we will be using the Lable Encode option for the diagnosis column</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = valib.Transform(data=df, label_encode=rc,index_columns=\"id\",unique_index=True,retain=feature_columns_names)\n",
    "df=data.result\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We use the sample function on teradataml dataframe. This function allows to sample few rows from dataframe directly or based on conditions. It creates a new column 'sampleid' which has a unique id for each sample, it helps to uniquely identify each sample.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create 2 samples of input data - sample 1 will have 80% of total rows and sample 2  will have 20% of total rows.\n",
    "cancer_sample = df.sample(frac=[0.80, 0.20], seed=42, id_column=\"id\")\n",
    "df_train = cancer_sample[cancer_sample.sampleid == \"1\"].drop(\"sampleid\", axis = 1)\n",
    "df_test = cancer_sample[cancer_sample.sampleid == \"2\"].drop(\"sampleid\", axis = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>6. Setup TDApiClient and Teradata contexts for Google Vertex AI</b></p>\n",
    "\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The Google Vertex AI teradataml extension library (TDApiClient) uses Vantage DataFrame to train Google Vertex AI models.</p>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Enable users to prepare the training data by leveraging Teradata’s Python based in-DB analytics functions.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Leveraging teradataml and Vertex AI Python SDK capabilities which allow users to create model over Vertex AI directly from Vantage.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Once created models are deployed as endpoint in Google Cloud or over Vantage, business users can get access of Vertex AI analytic services for real-time scoring directly from Vantage.</li></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Create the following environment variables before invoking this API. Required environment variables:</p>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>GOOGLE_APPLICATION_CREDENTIALS:</b> Specifies the path to the JSON file containing the Google Cloud service account credentials.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>GCP_REGION:</b> Specifies the location of the Google Cloud project.</li>\n",
    "<li style = 'font-size:14px;font-family:Arial;color:#00233C'><i><b>**Note :- The region for the VertexAI Service account and the region in which the bucket is created should be same</b></i></li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>GCP_PROJECT_ID:</b> Specifies the Project ID of the Google Cloud project.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>GCP_TD_AUTH_OBJ:</b> Specifies the name of the Google Cloud authorization object in Vantage.</li></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>**Please enter your valid GCP credentials.</b></p>\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "credentials_json = input(\"Name of the json file which has credentials:\")\n",
    "region = input(\"GCP Region: \")\n",
    "project_id = input(\"Enter the GCP project ID: \")\n",
    "td_auth_object = input(\"Enter the Authorization object name along with database name: \")\n",
    "\n",
    "# Example of the required values --- these are not valid values \n",
    "# credentials_json = \"path/to/service/account/file.json\"\n",
    "# region = \"us-east1\"\n",
    "# project_id = \"demo-environment\"\n",
    "# td_auth_object = \"demodb.td_auth\" ---demodb is the teradata database where the authorization object td_auth is created"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-warning\">\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Note: </b><i>If the credentials entered are not valid or do not have the necessary permissions, we may get an error in the estimator step.</i></p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We need to assign values based on the GCP Credentials.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "os.environ[\"GOOGLE_APPLICATION_CREDENTIALS\"] = credentials_json\n",
    "os.environ[\"GCP_REGION\"] = region\n",
    "os.environ[\"GCP_PROJECT_ID\"] = project_id\n",
    "os.environ[\"GCP_TD_AUTH_OBJ\"] = td_auth_object"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We will use the create_tdapi_context method to create a TDAPI context to be used to run TDApiClient functions. Creating TDAPI context object is the first step of using TDApiClient library.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Required Arguments:</p>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>type:</b> Specifies the cloud type of the TDAPI context.\n",
    "Permitted values are: \"aws\", \"azure\", or \"gcp\".</li></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>If the cloud type is Google Cloud, the only accepted value is \"gcp\".</p>\n",
    "\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>gcp_bucket_name:</b> Specifies the name of the Google Cloud storage bucket within project.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>gcp_bucket_path:</b> Specifies a path within the given bucket name. This acts as parent folder for all files that TDApiClient creates.</li></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gcp_bucket = input(\"Name of the storage bucket in project with same region :\")\n",
    "gcp_path = input(\"Path within the given bucket: \")\n",
    "# Examples of parameters\n",
    "# gcp_bucket = \"clearscape-*********\"\n",
    "# gcp_path = \"custom-******\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Assign variables \n",
    "\n",
    "gcp_context = create_tdapi_context(\n",
    "    \"gcp\",\n",
    "    gcp_bucket_name=gcp_bucket,\n",
    "    gcp_bucket_path=gcp_path\n",
    "    )\n",
    "tdapi_client = TDApiClient(gcp_context)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>7. Training job and Deploy model</b></p>\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>7.1 Create and run training job</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Vertex AI provides Docker container images that you run as prebuilt containers for custom training. These containers, which are organized by machine learning (ML) framework and framework version, include common dependencies that you might want to use in training code. Often, using a prebuilt container is simpler than creating your own custom container for training.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We can find the list of all precreated images <a href = 'https://cloud.google.com/vertex-ai/docs/training/pre-built-containers'>here</a>. For our demo we will be using the scikit-learn training and prediction images for our training and prediuction</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "TRAINING_IMAGE = \"us-docker.pkg.dev/vertex-ai/training/scikit-learn-cpu.0-23:latest\"\n",
    "PREDICTION_IMAGE = \"us-docker.pkg.dev/vertex-ai/prediction/sklearn-cpu.1-0:latest\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We will use the TDApiClient class to create training job objects of Vertex AI Python package. This feature is implemented using getattr method of this class. Exact input for this method is determined by the class name of TrainingJob, such as CustomTrainingJob or AutoMLTabularTrainingJob.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>There are different training job classes of Vertex AI package that can be used. For example:\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>CustomTrainingJob:</code> Useful for taking a training implementation as a python script</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>CustomPythonPackageTrainingJob:</code> Useful for taking a training implementation as a python package</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>CustomContainerTrainingJob:</code> Useful for training on a custom Docker container</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>AutoMLTabularTrainingJob:</code> Useful for training an AutoML model with tabular dataset</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>AutoMLForecastingTrainingJob:</code> Useful for training an AutoML model with time series dataset</li>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Here we are using the CustomTrainingJob</p>    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "job = tdapi_client.CustomTrainingJob(\n",
    "    display_name=\"tdapiclient-custom-demo\",\n",
    "    script_path=\"train.py\",\n",
    "    container_uri=TRAINING_IMAGE,\n",
    "    requirements=[\"gcsfs\", \"nyoka\"],\n",
    "    model_serving_container_image_uri=PREDICTION_IMAGE\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>7.2 Train a model</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We will use the TrainingJob.fit method to train a teradataml DataFrame with optional keyword arguments, and return a Google Vertex AI Model object. Below are the different arguments that can be used.</p>\n",
    "\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>LOCATION:</code> The region where the container or Python package will be run.</li>\n",
    "\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>JOB_NAME:</code> Required. A display name for the CustomJob.</li>\n",
    "\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>MACHINE_TYPE:</code> The type of the machine. Refer to available machine types for training.</li>\n",
    "\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>REPLICA_COUNT:</code> The number of worker replicas to use. In most cases, set this to 1 for your first worker pool.</li>\n",
    "\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>EXECUTOR_IMAGE_URI:</code> The URI of the container image that runs the provided code. Refer to the available prebuilt containers for training. This image acts as the base image for the new Docker image that you are building with this command.</li>\n",
    "\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>WORKING_DIRECTORY:</code> A directory in your local file system containing the entry point script that runs your training code (see the following list item). We can use the parent directory of the script, or a higher-level directory. We can also use a higher-level directory if it contains a requirements.txt or setup.py file. To learn more, see Install dependencies.</li>\n",
    "\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>SCRIPT_PATH:</code> The path, relative to WORKING_DIRECTORY on your local file system, to the script that is the entry point for your training code. This can be a Python script (ending in .py) or a Bash script. For example, if you want to run /hello-world/trainer/task.py and WORKING_DIRECTORY is /hello-world, then use trainer/task.py for this value.</li></p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = job.fit(\n",
    "    df_train,\n",
    "    replica_count=1,\n",
    "    model_display_name=\"tdapiclient-custom-demo\"\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>7.3 Deploy model to online endpoint</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We will use the TrainingJob.deploy method to deploy a trained model in Google Vertex AI environment or in Vantage system. A TDPredictor object is returned which can be used to run prediction.</p>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'>For Vertex AI environment, it requires argument format the same as the Model.deploy function in Vertex AI Python SDK.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'>For Vantage system, it requires arguments that are required by BYOM and Predictor functions in teradataml.</li></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Required Arguments:</b>\n",
    "    <li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>model:</b> Specifies a Vertex AI model object.\n",
    "platform: Specifies the platform to deploy the given model:</li>\n",
    "    <ul style = 'font-size:14px;font-family:Arial;color:#00233C'>\n",
    "    <li>'vantage'</li>\n",
    "<li>'vx-endpoint'</li></ul></p>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>model_type:</b> Required if platform is 'vantage'. Specifies the type of the model:</li>\n",
    "<ul style = 'font-size:14px;font-family:Arial;color:#00233C'>\n",
    "<li>'pmml'</li>\n",
    "<li>'onnx'</li>\n",
    "<li>'h2o'</li></ul></p>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>model_filename:</b> Required if platform is 'vantage'. Specifies the Google Cloud Storage file name of the model artifact.</li></p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor = job.deploy(\n",
    "    model,\n",
    "    \"vx-endpoint\",\n",
    "    vertex_kwargs={\"machine_type\": \"n1-standard-4\"}\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>8. Predict using the deployed model</b></p>\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>8.1 Prepare test data</b></p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_test = df_test.drop([\"diagnosis\"], axis=1)\n",
    "df_test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>8.2 Predict using the TDApiClient predictor object </b></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We will use the TDPredictor.predict method to perform prediction using teradataml DataFrame and VertexAI endpoint represented by this predictor object.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Required Arguments:</p>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>input:</code> Specifies the teradataml DataFrame used as input for scoring.</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>mode:</code> Specifies the mode for scoring.\n",
    "<ol style = 'font-size:16px;font-family:Arial;color:#00233C'>Permitted values include:\n",
    "<li style = 'font-size:14px;font-family:Arial;color:#00233C'><code>'UDF':</code> Score in database using a Teradata UDF. This is the default value. For this mode, the return is a teradataml DataFrame. This mode provides faster scoring with the data from Teradata.</li>\n",
    "\n",
    "<li style = 'font-size:14px;font-family:Arial;color:#00233C'><code>'CLIENT':</code> Score at client side using a library. For this mode, the return is an array or JSON. When using mode, data is pulled from Teradata and serialized for scoring at client.</li></ol></p>\n",
    "\n",
    "\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Optional Argument:\n",
    "<ol style = 'font-size:16px;font-family:Arial;color:#00233C'>options: Specifies the predict method with the following key-value arguments:\n",
    "<li style = 'font-size:14px;font-family:Arial;color:#00233C'><code>udf_name:</code> Specifies the name of the UDF used to invoke predict with UDF mode. Default value is 'tapidb.API_Request'.</li>\n",
    "<li style = 'font-size:14px;font-family:Arial;color:#00233C'><code>content_type:</code> Specifies content type required for VertexAI endpoint present in the predictor. Default value is 'csv'.</li>\n",
    "<li style = 'font-size:14px;font-family:Arial;color:#00233C'><code>key_start_index:</code> Specifies the index in DataFrame columns to be the key for scoring starts. Default value is 0.</li></ol>\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.predict(df_test, mode=\"udf\", content_type=\"json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>8.3 Deploy created model to Vantage </b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We can use the BYOMPredictor class to hold the name and type of the model teradataml DataFrame obtained using deploy() method. This allows user to export models that are built on VertexAI to Vantage through BYOM, for in-database analysis.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Required Arguments:</p>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>model_df:</code> Specifies the teradataml DataFrame containing the model data to be used for scoring.\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><code>model_type:</code> Specifies the type of the model:</li>\n",
    "<ul style = 'font-size:14px;font-family:Arial;color:#00233C'>    \n",
    "<li>'pmml'</li>\n",
    "    <li>'onnx'</li>\n",
    "    <li>'h2o'</li>\n",
    "    </ul>\n",
    "    </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "byom_predictor = job.deploy(\n",
    "    model,\n",
    "    \"vantage\",\n",
    "    model_type=\"pmml\",\n",
    "    model_filename=\"model.pmml\"\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "byom_predictor.model_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>8.3 Predict in database using the BYOM Predictor</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We can use the BYOMPredictor.predict method to score data in Vantage with a model that has been created outside Vantage and exported to Vantage using the deploy method.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>This method supports prediction using models in the following formats:\n",
    "<li style = 'font-size:14px;font-family:Arial;color:#00233C'>PMML</li>\n",
    "<li style = 'font-size:14px;font-family:Arial;color:#00233C'>ONNX</li>\n",
    "    <li style = 'font-size:14px;font-family:Arial;color:#00233C'>MOJO (H2O)</li></p>\n",
    "    \n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Required Arguments:</p>\n",
    "<li style = 'font-size:14px;font-family:Arial;color:#00233C'><code>input:</code> Specifies the teradataml DataFrame containing the input test data.</li>\n",
    "<li style = 'font-size:14px;font-family:Arial;color:#00233C'><code>input_cols:</code> Specifies the name(s) of input teradataml DataFrame column(s) to copy to the output.</li></p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_predict = byom_predictor.predict(df_test, df_test.columns)\n",
    "df_predict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_join = df.select([\"id\",\"diagnosis\"])\n",
    "df_final = df_join.merge(right=df_predict, how='inner', on=\"id\", lsuffix = 't1', rsuffix='t2')\n",
    "df_final"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>9. Evaluate the model</b></p>\n",
    "\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>9.1 Classification Evaluator</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The ClassificationEvaluator() function evaluate and emits various metrics of classification model based on its predictions on the data. Apart from accuracy, the secondary output data returns micro, macro, and weighted-averaged metrics of precision, recall, and F1-score values.</p>\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pred_df = df_final.assign(drop_columns=True,\n",
    "                          id = df_final.id_t1,\n",
    "                          diagnosis = df_final.diagnosis.cast(type_=INTEGER),\n",
    "                          prediction = df_final.prediction.cast(type_=INTEGER))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ClassificationEvaluator_obj = ClassificationEvaluator(data=df_final,\n",
    "                                                          observation_column='diagnosis',\n",
    "                                                          prediction_column='prediction',\n",
    "                                                          labels=['0', '1'])\n",
    "classeval_df = ClassificationEvaluator_obj.output_data\n",
    "classeval_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>9.2 Show AUC-ROC Curve</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The <a href = 'https://docs.teradata.com/search/all?query=TD_ROC&content-lang=en-US'>ROC</a> curve shows the performance of a binary classification model as its discrimination threshold varies. For a range of thresholds, the curve plots the true positive rate against false-positive rate.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>This function accepts a set of prediction-actual pairs as input and calculates the following values for a range of discrimination thresholds.</p>\n",
    "    <ul style = 'font-size:16px;font-family:Arial;color:#00233C'>\n",
    "        <li style = 'font-size:16px;font-family:Arial;color:#00233C'>True-positive rate (TPR)</li>\n",
    "        <li style = 'font-size:16px;font-family:Arial;color:#00233C'>False-positive rate (FPR)</li>\n",
    "        <li style = 'font-size:16px;font-family:Arial;color:#00233C'>The area under the ROC curve (AUC)</li>\n",
    "        <li style = 'font-size:16px;font-family:Arial;color:#00233C'>Gini coefficient</li>\n",
    "        <li style = 'font-size:16px;font-family:Arial;color:#00233C'>Other details are mentioned in the documentation</li>\n",
    "    </ul>\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from teradataml import ROC \n",
    "roc_df = ROC(data = pred_df, \n",
    "                    probability_column = \"prediction\",\n",
    "                    observation_column = \"diagnosis\",\n",
    "                    positive_class=\"1\"\n",
    "                    )\n",
    "roc_df.output_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "auc = roc_df.result.get_values()[0][0]\n",
    "auc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Plot ROC Curves</b></p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_cm=pred_df.to_pandas(all_rows=True)\n",
    "fpr, tpr, thresholds = metrics.roc_curve(df_cm['diagnosis'], df_cm['prediction'])\n",
    "# auc = metrics.auc(fpr, tpr)\n",
    "plt.plot(fpr,tpr,label=\"AUC=\"+str(auc))\n",
    "plt.legend(loc=4)\n",
    "plt.xlabel(\"False Positive Rate\")\n",
    "plt.ylabel(\"True Positive Rate\")\n",
    "plt.title(\"Receiver Operating Characteristic (ROC) Curve\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The closer the ROC curve is to the upper left corner of the graph, the higher the accuracy of the test because in the upper left corner, the sensitivity = 1 and the false positive rate = 0 (specificity = 1). The ideal ROC curve thus has an AUC = 1.0. As seen in the above graph the AUC for both the models is close to 1 so the accuracy of both models is very good. </p>\n",
    "\n",
    "<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>9.3 Show Confusion Matrix</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Confusion Matrix is a performance measurement for machine learning classification problem where output can be two or more classes. It is a table with 4 different combinations of predicted and actual values.</p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Confusion matrices represent counts from predicted and actual values. The output “TN” stands for True Negative which shows the number of negative examples classified accurately. Similarly, “TP” stands for True Positive which indicates the number of positive examples classified accurately. The term “FP” shows False Positive value, i.e., the number of actual negative examples classified as positive; and “FN” means a False Negative value which is the number of actual positive examples classified as negative.</p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "confusion_matrix_df = pred_df.to_pandas(all_rows=True)\n",
    "cm = confusion_matrix(confusion_matrix_df['diagnosis'], confusion_matrix_df['prediction'])\n",
    "cmd = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=['DoesNotHaveCancer', 'HasCancer'],)\n",
    "cmd.plot(cmap='Blues', colorbar=False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Conclusion</b></p>\n",
    "\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Thus we have seen that with the Teradata Vantage API_Request feature, we can connect to VertexAI endpoints through a function to do real-time scoring on data. A VertexAI endpoint was used to orchestrate model training and deploy the solution’s ML model. Also the model can be stored in Vantage and then used to score in Vantage. Vantage and ClearScape Anlaytics has helped drastically reduce data preparation, model development, and testing time, while allowing for much more frequent and iterative testing and tuning to ensure maximum life-critical accuracy.</p>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>10. Cleanup</b></p>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Databases and Tables</b></p>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We will use the following code to clean up tables and databases created for this demonstration.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%run -i ../run_procedure.py \"call remove_data('DEMO_CancerPrediction');\" \n",
    "#Takes 40 seconds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-danger\">\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>**Note: </b><i>Please make sure to delete the VertexAI model and endpoints after use using the code in below cell. If these are not deleted the cost will keep increasing till the time it is not deleted.</i></p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.cloudObj.delete(force=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "remove_tdapi_context(\n",
    "    gcp_context,\n",
    "    delete_byom_models=True,\n",
    "    table_name=\"tdapiclient_byom_models\"\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "remove_context()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
    "<b style = 'font-size:20px;font-family:Arial;color:#00233C'>Required Materials</b>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Let’s look at the elements we have available for reference for this notebook:</p>\n",
    "<b style = 'font-size:18px;font-family:Arial;color:#00233C'>Dataset:</b>\n",
    "<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The dataset for this analysis has been taken from \n",
    "<a href = 'https://archive.ics.uci.edu/dataset/17/breast+cancer+wisconsin+diagnostic'>Breast Cancer Wisconsin (Diagnostic) - UCI Machine Learning Repository.</a>\n",
    "\n",
    "<b style = 'font-size:18px;font-family:Arial;color:#00233C'>Filters:</b> \n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Industry:</b> Healthcare</li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Functionality:</b> Machine Learning</li> \n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Use Case:</b> Prediction Analysis</li></p>\n",
    "<b style = 'font-size:18px;font-family:Arial;color:#00233C'>Related Resources:</b>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><a href = 'https://usc-word-edit.officeapps.live.com/we/%E2%80%A2%09https:/www.teradata.com/Blogs/Predicting-Heart-Failure-with-Teradata'>Saving Lives, Saving Costs: Predicting Heart Failure with Teradata</a> </li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><a href = 'https://www.teradata.com/Blogs/Hyper-scale-time-series-forecasting-done-right'>Hyper-scale time series forecasting done right</a></li>\n",
    "<li style = 'font-size:16px;font-family:Arial;color:#00233C'><a href = 'https://www.teradata.com/Blogs/Forecasting-COVID-19-Using-Teradata-Vantage'>Forecasting COVID-19 Using Teradata Vantage</a></li>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "<footer style=\"padding-bottom:35px; background:#f9f9f9; border-bottom:3px solid #00233C\">\n",
    "    <div style=\"float:left;margin-top:14px\">ClearScape Analytics™</div>\n",
    "    <div style=\"float:right;\">\n",
    "        <div style=\"float:left; margin-top:14px\">\n",
    "            © 2023, 2024 Teradata. All rights reserved.\n",
    "        </div>\n",
    "    </div>\n",
    "</footer>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}