SIT744_prac_5.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "SIT744_prac_5.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Lzb2RpiAhc3L",
        "colab_type": "text"
      },
      "source": [
        "# SIT744 Practical 5: tf.data input pipeline\n",
        "\n",
        "*Dr Wei Luo*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YDxKpHrsIaN5",
        "colab_type": "text"
      },
      "source": [
        "<div class=\"alert alert-info\">\n",
        "We suggest that you run this notebook using Google Colab.\n",
        "</div>\n",
        "\n",
        "\n",
        "## Pre-practical readings\n",
        "\n",
        "- [tf.data: Build TensorFlow input pipelines](https://www.tensorflow.org/guide/data)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "44hpSshgh2al",
        "colab_type": "text"
      },
      "source": [
        "## Task 1 Experiments with neural networks\n",
        "\n",
        "Last week, we saw how neural networks can be used for classification and regression. In this task, you will complete some experiments suggested in the textbook.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rtdzMbOnVXP-",
        "colab_type": "text"
      },
      "source": [
        "### Task 1.1 Experiments with binary classification\n",
        "\n",
        "The code for binary classification is copied below for you.\n",
        "\n",
        "Complete the following experiments:\n",
        "\n",
        "- You used two representation layers before the final classification layer. Try using one or three representation layers, and see how doing so affects validation and test accuracy.\n",
        "- Try using layers with more units or fewer units: 32 units, 64 units, and so on.\n",
        "- Try using the mse loss function instead of binary_crossentropy.\n",
        "- Try using the tanh activation (an activation that was popular in the early days of neural networks) instead of relu."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gQ0y0guOVhUw",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "import numpy as np\n",
        "from keras.datasets import imdb\n",
        "\n",
        "\n",
        "\n",
        "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)\n",
        "\n",
        "\n",
        "\n",
        "def vectorize_sequences(sequences, dimension=10000):\n",
        "    # Create an all-zero matrix of shape (len(sequences), dimension)\n",
        "    results = np.zeros((len(sequences), dimension))\n",
        "    for i, sequence in enumerate(sequences):\n",
        "        results[i, sequence] = 1.  # set specific indices of results[i] to 1s\n",
        "    return results\n",
        "\n",
        "# Our vectorized training data\n",
        "x_train = vectorize_sequences(train_data)\n",
        "# Our vectorized test data\n",
        "x_test = vectorize_sequences(test_data)\n",
        "\n",
        "\n",
        "# Our vectorized labels\n",
        "y_train = np.asarray(train_labels).astype('float32')\n",
        "y_test = np.asarray(test_labels).astype('float32')\n",
        "\n",
        "\n",
        "x_val = x_train[:10000]\n",
        "partial_x_train = x_train[10000:]\n",
        "\n",
        "y_val = y_train[:10000]\n",
        "partial_y_train = y_train[10000:]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Dhx9ZVzlV9CE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from tensorflow.keras import models\n",
        "from tensorflow.keras import layers\n",
        "from tensorflow.keras import optimizers\n",
        "\n",
        "model = models.Sequential()\n",
        "model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))\n",
        "model.add(layers.Dense(16, activation='relu'))\n",
        "model.add(layers.Dense(1, activation='sigmoid'))\n",
        "\n",
        "\n",
        "\n",
        "model.compile(optimizer=optimizers.RMSprop(lr=0.001),\n",
        "              loss='binary_crossentropy',\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "history = model.fit(partial_x_train,\n",
        "                    partial_y_train,\n",
        "                    epochs=20,\n",
        "                    batch_size=512,\n",
        "                    validation_data=(x_val, y_val))\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9ObvEJffXxiC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "acc = history.history['accuracy']\n",
        "val_acc = history.history['val_accuracy']\n",
        "loss = history.history['loss']\n",
        "val_loss = history.history['val_loss']\n",
        "\n",
        "epochs = range(1, len(acc) + 1)\n",
        "\n",
        "# \"bo\" is for \"blue dot\"\n",
        "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
        "# b is for \"solid blue line\"\n",
        "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
        "plt.title('Training and validation loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "\n",
        "plt.show()\n",
        "\n",
        "results = model.evaluate(x_test, y_test)\n",
        "results"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UWxj7LoGYK69",
        "colab_type": "text"
      },
      "source": [
        "### Task 2.2 Experiments with multiclass classification\n",
        "\n",
        "The code for multiclass classification is copied below for you.\n",
        "\n",
        "Complete the following experiments:\n",
        "- Try using larger or smaller layers: 32 units, 128 units, and so on.\n",
        "- You used two intermediate layers before the final softmax classification layer. Now try using a single intermediate layer, or three intermediate layers.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UMCgBB1FYfQg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from keras.datasets import reuters\n",
        "from keras.utils.np_utils import to_categorical\n",
        "\n",
        "(train_data, train_labels), (test_data, test_labels) = reuters.load_data(num_words=10000)\n",
        "\n",
        "def vectorize_sequences(sequences, dimension=10000):\n",
        "    results = np.zeros((len(sequences), dimension))\n",
        "    for i, sequence in enumerate(sequences):\n",
        "        results[i, sequence] = 1.\n",
        "    return results\n",
        "\n",
        "# Our vectorized training data\n",
        "x_train = vectorize_sequences(train_data)\n",
        "# Our vectorized test data\n",
        "x_test = vectorize_sequences(test_data)\n",
        "\n",
        "\n",
        "one_hot_train_labels = to_categorical(train_labels)\n",
        "one_hot_test_labels = to_categorical(test_labels)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k5JSN_yEY_gb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model = models.Sequential()\n",
        "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))\n",
        "model.add(layers.Dense(64, activation='relu'))\n",
        "model.add(layers.Dense(46, activation='softmax'))\n",
        "\n",
        "model.compile(optimizer='rmsprop',\n",
        "              loss='categorical_crossentropy',\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "x_val = x_train[:1000]\n",
        "partial_x_train = x_train[1000:]\n",
        "\n",
        "y_val = one_hot_train_labels[:1000]\n",
        "partial_y_train = one_hot_train_labels[1000:]\n",
        "\n",
        "\n",
        "history = model.fit(partial_x_train,\n",
        "                    partial_y_train,\n",
        "                    epochs=20,\n",
        "                    batch_size=512,\n",
        "                    validation_data=(x_val, y_val))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IZULRpXNZKWf",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "loss = history.history['loss']\n",
        "val_loss = history.history['val_loss']\n",
        "\n",
        "epochs = range(1, len(loss) + 1)\n",
        "\n",
        "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
        "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
        "plt.title('Training and validation loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "\n",
        "plt.show()\n",
        "\n",
        "\n",
        "model = models.Sequential()\n",
        "model.add(layers.Dense(64, activation='relu', input_shape=(10000,)))\n",
        "model.add(layers.Dense(64, activation='relu'))\n",
        "model.add(layers.Dense(46, activation='softmax'))\n",
        "\n",
        "model.compile(optimizer='rmsprop',\n",
        "              loss='categorical_crossentropy',\n",
        "              metrics=['accuracy'])\n",
        "model.fit(partial_x_train,\n",
        "          partial_y_train,\n",
        "          epochs=8,\n",
        "          batch_size=512,\n",
        "          validation_data=(x_val, y_val))\n",
        "results = model.evaluate(x_test, one_hot_test_labels)\n",
        "results"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BIlfHMv6Pg5S",
        "colab_type": "text"
      },
      "source": [
        "## Task 2 TensorFlow input pipelines\n",
        "\n",
        "If you are familiar with the Scikit-learn [Pipeline](https://scikit-learn.org/stable/modules/compose.html#pipeline), you would appreciate the convenience it brings to data preprocessing. In TensorFlow, the `tf.data` API provides similar functionality. \n",
        "\n",
        "1. When you dealing with a huge dataset or a data stream that cannot fit into memory, the `tf.data` API provides a way to feed training data batches into TensorFlow.\n",
        "2. As we see in the lecture, often data from different raw formats (for example text) need to be preprocessed. The `tf.data` allows this and other transformations to be performed on-the-fly. \n",
        "\n",
        "The key abstraction in the `tf.data` API is the `tf.data.Dataset` interface, which consists of a sequence of **elements** and an iterator for the sequence. (If you are not familiar with Iterator, see [here](https://wiki.python.org/moin/Iterator).) Often an element is a pair of batched data *(training features, labels)*. \n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gw01UtvMiVhD",
        "colab_type": "text"
      },
      "source": [
        "### Task 2.1 TensorFlow Datasets: a collection of ready-to-use datasets\n",
        "\n",
        "If you are looking for some common datasets to test your model, you can find example Datasets in `tensorflow_datasets`. These are similar to the NumPy datasets provided by `tf.keras.datasets`. \n",
        "\n",
        "**Warning**: You should not confuse \"TensorFlow Datasets\" provided by the `tensorflow_datasets` package (aka `tfds`, see below) with the `tf.data.Dataset` API (aka `Dataset`). The former is a collection of ready-to-go datasets packaged using the latter. (Yes I know. Unfortunately, TensorFlow has a lot of confusingly named modules. But you should feel lucky that you do not have to learn TensorFlow v1 anymore.)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-QSqLVpEixHR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import tensorflow as tf\n",
        "import tensorflow_datasets as tfds\n",
        "\n",
        "# Construct a tf.data.Dataset\n",
        "ds = tfds.load('mnist', split='train', shuffle_files=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VnRQe26pC_Se",
        "colab_type": "text"
      },
      "source": [
        "Every Dataset may have a different data structure for its elements. You can see how each element is organised via `Dataset.element_spec`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "clR9CaszDHcT",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(ds.element_spec)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q-O689dijeIM",
        "colab_type": "text"
      },
      "source": [
        "Dataset is designed to be used for large datasets. Therefore it is not meant to be loaded into the memory all at once. Instead, a Dataset has an iterable interface and you can use a for-loop to progressively access the elements in a Dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3eWsND_YjxzD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for example in ds.take(1):\n",
        "  image, label = example[\"image\"], example[\"label\"]\n",
        "\n",
        "\n",
        "## Show the image and label\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "\n",
        "def show(image, label):\n",
        "  plt.figure()\n",
        "  plt.imshow(np.squeeze(image), cmap='gray')\n",
        "  plt.title(label.numpy())\n",
        "  plt.axis('off')\n",
        "\n",
        "show(image, label)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0aFVABPCltsu",
        "colab_type": "text"
      },
      "source": [
        "In the above example, the function `take(n)` limits the number of elements returned by the iterator to `n`.\n",
        "\n",
        "**exercise** Find out what other datasets are available from `tensorflow_datasets`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8AmCAa9rxSoH",
        "colab_type": "text"
      },
      "source": [
        "### Task 2.2 Define a data source\n",
        "\n",
        "When you want to use your own data, a Dataset can be sourced from either the memory or physical files. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3VqtgKFP8JlY",
        "colab_type": "text"
      },
      "source": [
        "#### Dataset from the memory\n",
        "\n",
        "To define a data source in the memory, you use the function `tf.data.Dataset.from_tensor_slices()`. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3YGhhUrE1VRR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "dataset = tf.data.Dataset.from_tensor_slices([8, 3, 0, 8, 2, 1])\n",
        "\n",
        "for item in dataset:\n",
        "    print(item)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x7nhtI3-51Rg",
        "colab_type": "text"
      },
      "source": [
        "In most cases, you do not want to use the closely-named function `tf.data.Dataset.from_tensors()`. \n",
        "\n",
        "**exercise** Repeat the above experiment replacing `tf.data.Dataset.from_tensor_slices()` with  `tf.data.Dataset.from_tensors()`. What will you get?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "skB6ckc464De",
        "colab_type": "text"
      },
      "source": [
        "#### Dataset from files\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t2bshfjz7MsN",
        "colab_type": "text"
      },
      "source": [
        "##### TextLineDataset\n",
        "\n",
        "If you have a text file, you can create a Dataset so that each line is an element."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mlV4qJkbyKzw",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "## You can find the file in your Google Colab host machine\n",
        "file_paths = [\"sample_data/california_housing_train.csv\"]\n",
        "\n",
        "housing_dataset = tf.data.TextLineDataset(file_paths)\n",
        "\n",
        "print(housing_dataset.element_spec)\n",
        "\n",
        "for line in housing_dataset.take(5):\n",
        "    print(line.numpy())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l0AXgR6Y5lg0",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "\n",
        "For a CSV file, each line still needs to be parsed. You can follow the code snippet below for that."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JDigDwXr00zL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for line in housing_dataset.skip(1).take(5):\n",
        "  column_default_values = [[]] * 9\n",
        "  columns = tf.io.decode_csv(line, record_defaults = column_default_values)\n",
        "  print(tf.stack(columns).numpy())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X3lBLCg8xxet",
        "colab_type": "text"
      },
      "source": [
        "##### make_csv_dataset\n",
        "\n",
        "Alternatively you can use the function `tf.data.experimental.make_csv_dataset` to read CSV files."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sXjSY6hi6ovD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "housing_batches = tf.data.experimental.make_csv_dataset(\n",
        "    file_paths, batch_size=4,\n",
        "    label_name=\"median_house_value\")\n",
        "\n",
        "print(housing_batches.element_spec)\n",
        "\n",
        "housing_batches.element_spec[0]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j4yGjrjQ68Ae",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for features, labels in housing_batches.take(1):\n",
        "  print(f\"median_price: {labels}\")\n",
        "  for key, value in features.items():\n",
        "      print(f\"{key}: {value}\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "22OhQe387CMs",
        "colab_type": "text"
      },
      "source": [
        "Below is another example."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Tvh0mP4ZAqTP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "TRAIN_DATA_URL = \"https://storage.googleapis.com/tf-datasets/titanic/train.csv\"\n",
        "train_file_path = tf.keras.utils.get_file(\"train.csv\", TRAIN_DATA_URL)\n",
        "\n",
        "titanic_batches = tf.data.experimental.make_csv_dataset(\n",
        "    train_file_path, batch_size=4,\n",
        "    label_name=\"survived\")\n",
        "\n",
        "print(titanic_batches.element_spec)\n",
        "\n",
        "print()\n",
        "print(\"Features:\")\n",
        "titanic_batches.element_spec[0]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_qF1D6HmFDZb",
        "colab_type": "text"
      },
      "source": [
        "You can use `take(n)` to get the first n elements from a Dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8zG4_NHF7Kyb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for features, labels in titanic_batches.take(1):\n",
        "  print(f\"survived: {labels}\")\n",
        "  for key, value in features.items():\n",
        "      print(f\"{key}: {value}\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9gr2LdbJvAtg",
        "colab_type": "text"
      },
      "source": [
        "Later on, the features should be processed into numerical tensors. In particular, categorical features should be encoded by either one-hot vectors or more sophisticated embeddings, which we will learn later."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dzXe35bYFsoy",
        "colab_type": "text"
      },
      "source": [
        "##### TFRecord\n",
        "\n",
        "Besides CSV files, TensorFlow actually recommends the binary TFRecord files for your data. You have encountered this file format in Practical 2, when we learned TensorBoard. The event files generated by `tf.summary` are just TFRecord files. A TFRecord file contains a sequence of `tf.train.Example`. You can use `tf.data.TFRecordDataset` to construct a Dataset from one or more TFRecord files. As it is more complex, we will not go into the details in this practical.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BX8mZZhsGqu-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "DATA_URL = \"https://storage.googleapis.com/download.tensorflow.org/data/fsns-20160927/testdata/fsns-00000-of-00001\"\n",
        "fsns_test_file = tf.keras.utils.get_file(\"fsns.tfrec\", DATA_URL )\n",
        "dataset = tf.data.TFRecordDataset(filenames = [fsns_test_file])\n",
        "\n",
        "print(dataset.element_spec)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ce0trJzEGnl9",
        "colab_type": "text"
      },
      "source": [
        "Here `Dataset.element_spec` is not of much use as each element is stored as a byte string. To access the features, follow the example below. As you can see, the data itself is deeply embedded in a nested data structure. This is for reversing space for future extensions of the API."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PlzGcOLIF6JS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for element in dataset.take(1):\n",
        "  example = tf.train.Example.FromString(element.numpy())\n",
        "\n",
        "print(dict(example.features.feature).keys())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z20Cki33Hi8q",
        "colab_type": "text"
      },
      "source": [
        "### Task 2.3 Batching\n",
        "\n",
        "Batching can be achieved through the `batch()` function."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RZIn3mw_JUFT",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(\"Individual element\")\n",
        "for element in housing_dataset.skip(1).take(8): #Skipping the CSV heading\n",
        "  print(element)\n",
        "\n",
        "print()\n",
        "print(\"Three batches\")\n",
        "for batches in housing_dataset.skip(1).take(8).batch(3):\n",
        "  print(batches)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5pXFGa8pLcvi",
        "colab_type": "text"
      },
      "source": [
        "**question** What is the size of the last batch?\n",
        "\n",
        "**exercise** Read the tf.Dataset documentation to find out how to do repeat and shuffle training data. Then apply them to the housing dataset."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DfG8EASdZ7I-",
        "colab_type": "text"
      },
      "source": [
        "### Task 2.4 Dataset transformation \n",
        "\n",
        "Data preprocessing can be achieved by chaining transformations over the original Dataset. The main function for element-wise transforms is `Dataset.map()`, which is similar to the `map()` function built in other programming languages (such as [Python](https://docs.python.org/3/library/functions.html#map) or [R](https://purrr.tidyverse.org/reference/map.html)).\n",
        "\n",
        "Below is the MNIST example we saw earlier."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0Bw6tzPfuiFF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for example in ds.take(2):\n",
        "  image, label = example[\"image\"], example[\"label\"]\n",
        "  show(image, label)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Umb6u5eVcCRR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def resize_image(example):\n",
        "  image, label = example[\"image\"], example[\"label\"]\n",
        "  image = tf.image.resize(image, [14, 14])\n",
        "  return image, label \n",
        "                     \n",
        "\n",
        "resized_ds = ds.map(resize_image)\n",
        "\n",
        "\n",
        "for image, label in resized_ds.take(2):\n",
        "  show(image, label)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DOhOH0GfbB5X",
        "colab_type": "text"
      },
      "source": [
        "**exercise** Use the map function to scale the image pixels to have values between 0 and 1."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sgzCS-03Z4su",
        "colab_type": "text"
      },
      "source": [
        "## Additional resources\n",
        "\n",
        "- [A blog article on TensorFlow Data Pipeline](https://heartbeat.fritz.ai/building-a-data-pipeline-with-tensorflow-3047656b5095)"
      ]
    }
  ]
}