iml_07.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "30b61190-61c1-4dd2-a9ec-acbadc8a52ef",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Introduction to Machine Learning - Exercise 7\n",
    "Goal of the excercise is to code selected part of the Decision tree algorithm which is focused on the optimum split part using gini index. \n",
    "\n",
    "After that the scikit-learn implementation of the Decision tree basic usage will be demonstrated.\n",
    "\n",
    "![meme02](https://github.com/rasvob/EFREI-Introduction-to-Machine-Learning/blob/main/images/fml_07_meme_02.jpg?raw=true)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a70c5553-ad01-4d43-93fb-50c085f09a64",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "import math"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d685ada-68b5-474a-84d5-5bfacd4a9eca",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Load the *Iris* dataset\n",
    "https://archive.ics.uci.edu/dataset/53/iris\n",
    "* One of the earliest datasets used in the literature on classification methods and widely used in statistics and machine learning.\n",
    "* The data set contains 3 classes of 50 instances each, where each class refers to a type of iris plant.\n",
    "* One class is linearly separable from the other 2 - the latter are not linearly separable from each other.uted?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5164551-7881-46fc-84b9-85781099fd8a",
   "metadata": {},
   "source": [
    "|Variable Name | Role | Type | Units |\n",
    "|:-------|:-------|:-------|:-------|\n",
    "|sepal length | Feature | Continuous | cm |\n",
    "|sepal width | Feature | Continuous | cm |\n",
    "|petal length | Feature | Continuous | cm |\n",
    "|petal width | Feature | Continuous | cm |\n",
    "|class | Target | Categorical | Iris Setosa, Iris Versicolour, or Iris Virginica |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0b9043cd-4e41-40b0-93e1-712d94d122b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('https://github.com/rasvob/EFREI-Introduction-to-Machine-Learning/raw/refs/heads/main/datasets/iris.csv')\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "876ad5c0-86b3-4f28-8d5c-b13f8814c5d3",
   "metadata": {},
   "source": [
    "# 💡 Let's take a look at the data first\n",
    "* 🔎 Do you have an idea how would you divide the data into groupd based on the assigned classed by vertical or horizontal lines?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a226485c-3300-436b-a05b-2761dd04ede3",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.scatterplot(data = df, x='sepal_length', y='sepal_width', hue='species')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "120d6aec-119b-49c3-bfed-fe2c290359ad",
   "metadata": {},
   "source": [
    "# And now? 😊"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aaf18814-5547-48f8-92c1-3f7ead6a9e67",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.scatterplot(data = df, x='petal_length', y='petal_width', hue='species')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54cd1e35-c508-4ff4-a42c-0679d5bd67ed",
   "metadata": {},
   "source": [
    "# Also it's always reasonable to take a look at the number of classes in the dataset\n",
    "* 💡You need to know beforehand about missing values or imbalance of labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "be1dfcb0-ac1c-4271-986a-fc5c017c760d",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.species.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85aa015b-a8e7-4a9a-857c-c11a2007adcc",
   "metadata": {},
   "source": [
    "# 🔎 What is the purpose of the classification algorithms?\n",
    "* How the algorithms work in general?\n",
    "* How does **Decision tree** work in a very simple terms?\n",
    "    * What is a split?\n",
    "        * 🔎 Is the data separated only by vertical/horizontal lines or are there any inclined straight lines or curves used?\n",
    "    * How can we decide on **which feature** and on **which condition** we should split?\n",
    "    * What is in the leaf of the **Decision tree**?\n",
    "        * **When** will the decision tree **stop the growth**?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78504964-e946-44f5-b321-b121d3f019e7",
   "metadata": {},
   "source": [
    "# 📊 Gini Index\n",
    "* What does it tell us?\n",
    "    * What is the **minimum** and **maximum** value for 2 classes?\n",
    "    * And for three?\n",
    "    * What result should we expect for the whole data set?\n",
    "\n",
    "## 🚀 Compute the gini index for the whole data set\n",
    "* 💡 Take a look at [these slides](https://homel.vsb.cz/~pla06/files/fml/fml_06.pdf) for the formulas and examples\n",
    "\n",
    "![meme03](https://github.com/rasvob/EFREI-Introduction-to-Machine-Learning/blob/main/images/fml_07_meme_03.jpg?raw=true)\n",
    "\n",
    "### Formulas\n",
    "\n",
    "* Gini index:\n",
    "  \n",
    "$$\n",
    "G(S) = 1 - \\sum_{j=1}^{k} p_{j}^{2}\n",
    "$$\n",
    "\n",
    "* where the $p_{j}$  is a fraction of points that belongs to the class j from a set S$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68c0aa60-daa9-4acd-81e6-0851d241eb51",
   "metadata": {},
   "source": [
    "* Gini impurity:\n",
    "$$\n",
    "G(S \\Rightarrow S_{1},...,S_{r}) = \\sum_{i=1}^{r} \\frac{S_i}{S} G(S_i)\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a5822b5c-c463-4875-816d-2ab2f3a2a53b",
   "metadata": {},
   "outputs": [],
   "source": [
    "def gini_index(data: pd.DataFrame):\n",
    "    probs = 0\n",
    "    labels = data.species.unique()\n",
    "    for l in labels:\n",
    "        p = data.loc[data.species == l].shape[0] / data.shape[0]\n",
    "        probs += p ** 2\n",
    "    return 1 - probs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee00bd16-1d23-4608-96a3-ebf22f677dbe",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "gini_index(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "77c091d4-a420-432a-86f0-8ee70272b5b5",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 🚀 Take only the *sepal_width* attribute\n",
    "* Try to split the by attribute by value *3.35 (<= 3.35)*\n",
    "* Compute the gini index for the left and the right side of the split\n",
    "    * Left: 0.629\n",
    "    * Right: 0.285\n",
    "* Compute the total gini impurity for this split\n",
    "    * Total: 0.546\n",
    "\n",
    "    \n",
    "**How to solve the example?**\n",
    "$$\n",
    "Gini(left) = 1 - \\left[ \\left(\\frac{45}{114}\\right)^2 + \\left(\\frac{49}{114}\\right)^2 + \\left(\\frac{20}{114}\\right)^2 \\right] = 0.629\n",
    "$$\n",
    "\n",
    "$$\n",
    "Gini(right) = 1 - \\left[ \\left(\\frac{5}{36}\\right)^2 + \\left(\\frac{1}{36}\\right)^2 + \\left(\\frac{30}{36}\\right)^2 \\right] = 0.285\n",
    "$$\n",
    "\n",
    "$$\n",
    "GiniImpurity = \\frac{|df_{left}|}{|df|} * Gini(left) + \\frac{|df_{right}|}{|df|} * Gini(right) =  \\frac{114}{150} * 0.629 + \\frac{36}{150} * 0.285 = 0.546\n",
    "$$\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ae5b4ee-3fc0-4f2b-a2f7-6f6e131341b8",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_left = df[df.sepal_width <= 3.35]\n",
    "df_left.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee18e789-7ccc-4140-a58e-46ed5b6bc6ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_left.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c75a13f2-b217-463b-b034-9112ac5042fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_left.species.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9328bebc-0933-4829-ab78-cd515cf4cb77",
   "metadata": {},
   "outputs": [],
   "source": [
    "gini_index(df_left)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "388c331b-22e0-4cb4-bec9-571982385e71",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_right = df[df.sepal_width > 3.35]\n",
    "df_right.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8adc4083-667c-4b39-85eb-8f8b47cdb9c6",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_right.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96a667ea-456b-45d1-a704-e18f5813744c",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_right.species.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "83942d58-c899-425f-93be-6b9f54786f7e",
   "metadata": {},
   "outputs": [],
   "source": [
    "gini_index(df_right)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d4a7c2c4-802c-40b2-9f9a-d0c779b1d1d1",
   "metadata": {},
   "outputs": [],
   "source": [
    "gini_imp = (df_left.shape[0] / df.shape[0]) * gini_index(df_left) + (df_right.shape[0] / df.shape[0]) * gini_index(df_right)\n",
    "gini_imp"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3b6c0d8-5303-4c5d-94d4-6d0f883abd7a",
   "metadata": {},
   "source": [
    "# 🎯 Repeat the process for the all 4 attributes and find the best split\n",
    "* You will try all the possible boundaries along all the features, i.e. all the axes petal width and sepal width.\n",
    "    * 💡 How to set the boundary? \n",
    "        * Select only the unique values of the chosen attribute (.unique() function)\n",
    "        * Sort the values (np.sort())\n",
    "        * Iterate over the array of the values, take every two consecutive numbers and compute their mean\n",
    "        * The mean is the current split boundary\n",
    "* For each split the algorithm will compute the **Gini impurity** of the two groups created.\n",
    "* 💡 Finally you will choose the decision boundary (best split) that gives the lowest Gini impurity for the two groups from all the 4 attbibutes\n",
    "\n",
    "### Results:\n",
    "* index = 0, Sepal Length < 5.45, Gini = 0.44\n",
    "* index = 1, Sepal Width < 3.35, Gini = 0.54\n",
    "* index = 2, Petal Length < 2.45, Gini = 0.33\n",
    "* index = 3, Petal Width < 0.80, Gini = 0.33\n",
    "\n",
    "* **Selected split:**\n",
    "    * **index = 2, Petal Length < 2.45, Gini = 0.33**\n",
    "    * ... or index = 3, Petal Width < 0.80, Gini = 0.33\n",
    "\n",
    "# 🔎 If we have only one split - how will the classification rule-set look like?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f534f148-11aa-4121-acab-29363bac7fba",
   "metadata": {},
   "outputs": [],
   "source": [
    "def print_splits(df: pd.DataFrame, column: str) -> None:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a20af5bd-014d-4782-9f6c-518b55141efc",
   "metadata": {},
   "source": [
    "## sepal_length"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4054ef8-9b42-448e-92b3-c8be3dbc694c",
   "metadata": {},
   "source": [
    "* Gini: 0.662, Boundary: 4.35\n",
    "* Gini: 0.648, Boundary: 4.45\n",
    "* Gini: 0.644, Boundary: 4.55\n",
    "* Gini: 0.624, Boundary: 4.65\n",
    "* Gini: 0.614, Boundary: 4.75\n",
    "* Gini: 0.587, Boundary: 4.85\n",
    "* Gini: 0.581, Boundary: 4.95\n",
    "* Gini: 0.547, Boundary: 5.05\n",
    "* Gini: 0.498, Boundary: 5.15\n",
    "* Gini: 0.482, Boundary: 5.25\n",
    "* Gini: 0.474, Boundary: 5.35\n",
    "* Gini: 0.439, Boundary: 5.45\n",
    "* Gini: 0.449, Boundary: 5.55\n",
    "* Gini: 0.475, Boundary: 5.65\n",
    "* Gini: 0.475, Boundary: 5.75\n",
    "* Gini: 0.492, Boundary: 5.85\n",
    "* Gini: 0.500, Boundary: 5.95\n",
    "* Gini: 0.512, Boundary: 6.05\n",
    "* Gini: 0.520, Boundary: 6.15\n",
    "* Gini: 0.528, Boundary: 6.25\n",
    "* Gini: 0.558, Boundary: 6.35\n",
    "* Gini: 0.580, Boundary: 6.45\n",
    "* Gini: 0.598, Boundary: 6.55\n",
    "* Gini: 0.591, Boundary: 6.65\n",
    "* Gini: 0.603, Boundary: 6.75\n",
    "* Gini: 0.608, Boundary: 6.85\n",
    "* Gini: 0.617, Boundary: 6.95\n",
    "* Gini: 0.609, Boundary: 7.05\n",
    "* Gini: 0.614, Boundary: 7.15\n",
    "* Gini: 0.629, Boundary: 7.25\n",
    "* Gini: 0.634, Boundary: 7.35\n",
    "* Gini: 0.639, Boundary: 7.50\n",
    "* Gini: 0.644, Boundary: 7.65\n",
    "* Gini: 0.662, Boundary: 7.80\n",
    "* Best -> Gini: 0.439, Boundary: 5.45"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "44a3e1d0-6ff6-4632-9d8d-6a321777b2c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "print_splits(df, 'sepal_length')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3881d935-4582-44ed-9346-b9bc5a64e762",
   "metadata": {},
   "source": [
    "## sepal_width"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e768d0b1-4aff-4caf-b284-2d69c77f8512",
   "metadata": {},
   "outputs": [],
   "source": [
    "print_splits(df, 'sepal_width')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "945b4908-8d83-4bd5-9834-71ff6a7a5c38",
   "metadata": {},
   "source": [
    "## petal_length"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "172fa8b7-7750-4026-b722-615bc908e7fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "print_splits(df, 'petal_length')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de4948cf-9b66-4b94-b9af-db330b8c061a",
   "metadata": {},
   "source": [
    "## petal_width"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "26d2b561-0a51-4217-b6f1-7686c6d67671",
   "metadata": {},
   "outputs": [],
   "source": [
    "print_splits(df, 'petal_width')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2042a76e-da61-4822-a321-179b1c5979be",
   "metadata": {},
   "source": [
    "# 🎯 Split the data using the selected split criterion \n",
    "* 💡 **Use the one using petal_length 😊**\n",
    "\n",
    "1) Repeat the whole process one more time for the part of the data with more than 1 class\n",
    "2) Find the best split again\n",
    "3) Draw the decision tree with the two defined levels you have created this way -\n",
    "    * You can use any graphic editor you like, even the MS Paint 😃"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aef5d44f-f45a-4bbd-aa40-dc226e7eba12",
   "metadata": {},
   "outputs": [],
   "source": [
    "# df_petal_left = ...\n",
    "# df_petal_right = ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b743f4a3-88d1-4141-8ee3-96d4c6357cb6",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_petal_left.species.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "215b98ef-75e8-4b38-be1c-cab256f222a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_petal_right.species.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96cb987c-9039-4479-902d-6287971d2af6",
   "metadata": {},
   "outputs": [],
   "source": [
    "for c in df_petal_right.columns[:-1]:\n",
    "    print(f'{c}')\n",
    "    print_splits(df_petal_right, c)\n",
    "    print(''.join(['-' for _ in range(50)]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fef2e5d-124c-4bca-afdc-fa44df8b46a3",
   "metadata": {},
   "source": [
    "# 📊 Introduction to the scikit-learn (Sklearn) library\n",
    "* Open source library for predictive data analysis. Contains algorithms for classification, clustering and others.\n",
    "\n",
    "![meme01](https://github.com/rasvob/EFREI-Introduction-to-Machine-Learning/blob/main/images/fml_07_meme_01.jpg?raw=true)\n",
    "\n",
    "- Decision tree https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html\n",
    "- Train test split https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html\n",
    "- Accuracy https://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "be61fad6-a531-47be-9f33-4a1ba0da6950",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier, export_text, plot_tree\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f2ee01e-a17e-41bc-8706-cb7acabb6787",
   "metadata": {},
   "source": [
    "# Let's start with splitting the data into the input and output part\n",
    "* 💡 Usually named as a *X* and *y* variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2257c875-3070-4153-8ec8-76bdb40b0d67",
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = df.loc[:, df.columns != 'species'], df.loc[:, 'species']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ea2ff8e-1857-4490-a373-03009419bd5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "411cfe95-46ef-4f0c-9b05-0814a4d4f3a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "y.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19178e66-00d6-4009-992d-e6ac171865bc",
   "metadata": {},
   "source": [
    "# What is the size of our data?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "22b6a5d8-b00c-4f5f-b845-f7eacb417e5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "X.shape, y.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50d3479c-173a-486e-a359-01335268b4f5",
   "metadata": {},
   "source": [
    "## 🔎 Doing evaluation process for every classification or regressions task is mandatory\n",
    "* Do you know how to evaluate models?\n",
    "* Do you know **train test split** method for evaluation?\n",
    "* Do you know any **different** evaluation process?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53036baf-2105-45be-84fd-53cc7aeecb6b",
   "metadata": {},
   "source": [
    "## Lets continue with train test split process. Split dataset to train:test ratio 60:40. Check how many records/rows do you have in splitted variables.\n",
    "* 💡 Number of rows in the *X* and *y* in the Train/Test part of the data has to be equal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e6d81bb3-e0e8-412f-939c-55c460976077",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=42)\n",
    "X_train.shape, X_test.shape, y_train.shape, y_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f0188db-498f-40e1-a755-7ad744f16375",
   "metadata": {},
   "source": [
    "## Take a look on data in variable y_test \n",
    "- Can you detect how many elements of each class is represented in test split?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6537945a-4b29-4649-958a-398a74e82420",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.Series(y_test).value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4dd1830-fd70-4d83-a7b0-25f1f12b1f59",
   "metadata": {},
   "source": [
    "# 🚀 Create the Decision tree classifier instance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a0cde27-cd89-424d-87a5-8720546d2f90",
   "metadata": {},
   "outputs": [],
   "source": [
    "clf = DecisionTreeClassifier()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2f1b95c-308b-4a25-9d99-a3a47b796720",
   "metadata": {},
   "source": [
    "## 💡 Use *fit()* method for training of classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e2fbb2e6-12b0-460e-9457-c12066425f87",
   "metadata": {},
   "outputs": [],
   "source": [
    "clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8b70c69-f460-4fc9-8c07-7122bbd4c8a9",
   "metadata": {},
   "source": [
    "## Lets use trained model for the prediction part\n",
    "* 💡 Get predictions via the method *predict()*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33bcfaf7-5c8e-4c7b-9d9c-aca6edfbd92a",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = clf.predict(X_test)\n",
    "y_pred"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3791870-d3b6-4fd7-91c5-67a9424c7888",
   "metadata": {},
   "source": [
    "# 🔎 Can we evaluate our model? \n",
    "* How good is it?\n",
    "* What does accuracy mean?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "27d1b774-53ba-4ae6-88a6-8c42c64a9220",
   "metadata": {},
   "outputs": [],
   "source": [
    "accuracy_score(y_test, y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9485e5c7-e2a7-40bb-9105-32121aadd6b2",
   "metadata": {},
   "source": [
    "## How does the trained Decision tree look like?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a9eb03-cf24-40b1-bebf-9144028877a1",
   "metadata": {},
   "source": [
    "### We can use either text representation as follows..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "09263452-a660-4933-bd36-01eca1e54d81",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(export_text(clf, show_weights=True, feature_names=list(X.columns.values)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "282b0d26-9255-4113-bcc2-014ae75550b9",
   "metadata": {},
   "source": [
    "### ...or more convinient graphical diagram 😊"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "94a0eeb2-757d-47d3-aae1-6d2c8afd8bce",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(25,20))\n",
    "_ = plot_tree(clf, feature_names=list(X.columns.values), filled=True, class_names=clf.classes_)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a65dc8f-cb95-40d7-ba81-c779df06e77b",
   "metadata": {
    "tags": []
   },
   "source": [
    "# ✅ Task (2p)\n",
    "* Finish the implementation task from the exercise"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}