material/deep_learning/template_text_sequence_data_prediction.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "template_text_sequence_data_prediction.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mZAFnuGfwqy-"
      },
      "source": [
        "# 텍스트 순차열 데이터 예측 Template"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hb0EOx6OyNIw"
      },
      "source": [
        "# 데이터 준비"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VBfCpbwh0h9R"
      },
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B5AtaEi6yPhX"
      },
      "source": [
        "## 기반 문자열 생성"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "V6wKtOwFwqXM"
      },
      "source": [
        "BASE_STRING = \"abcde12345\"\n",
        "text_data = BASE_STRING*10\n",
        "CATEGORY_NUM = len(BASE_STRING)"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NRl4H3Zrgjso",
        "outputId": "6cf6b031-12b0-4165-a13b-7aa65eb9bdf2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "source": [
        "print(text_data)"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "abcde12345abcde12345abcde12345abcde12345abcde12345abcde12345abcde12345abcde12345abcde12345abcde12345\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6RAkHiJbxNfE"
      },
      "source": [
        "SEQUENCE_LENGTH = 5\n",
        "OFFSET = 1"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f-z-YNfNDBn5"
      },
      "source": [
        "## 입출력 데이터 생성\n",
        "\n",
        "'abcde12345abcde'의 문자열로 다음의 입출력 데이터를 생성한다.\n",
        "```\n",
        "'abcde' -> '1'\n",
        "'bcde1' -> '2'\n",
        "'cde12' -> '3'\n",
        "'de123' -> '4'\n",
        "...\n",
        "'5abcd' -> 'e'\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yYxpmUQtwqTY"
      },
      "source": [
        "raw_x = []\n",
        "raw_y = []\n",
        "\n",
        "for i in range(len(text_data)):\n",
        "  if i+SEQUENCE_LENGTH+OFFSET >= len(text_data): break\n",
        "  raw_x.append(text_data[i:i+SEQUENCE_LENGTH])\n",
        "  raw_y.append(text_data[i+SEQUENCE_LENGTH+OFFSET-1])\n",
        "#            0         1         2      \n",
        "#            012345678901234567890123456789\n",
        "# textdata = abcde12345abcde12345abcde12345\n",
        "#               i = 3\n",
        "#               <-x->   = [3:8] = 'de123'\n",
        "#                    y  = [8]   = '4'\n"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "15RBiNi9wqQU",
        "outputId": "0be7cf17-d23f-4171-e302-f6d8c2e32fd2"
      },
      "source": [
        "print(len(raw_x))\n",
        "# len(raw_x) = 94 = len(text_data) - SEQUENCE_LENGTH - OFFSET = 100 - 5 - 1"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "94\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S7eFX6jIyrnx"
      },
      "source": [
        "## 데이터를 2차원 벡터화 하기\n",
        "\n",
        "RNN은 2차원 데이터를 입력으로 받는다. 1차원 문자열의 각 문자를 one-hot 인코딩하여 2차원 데이터로 만드렁 준다.<br>\n",
        "입력 데이터와 출력 데이터를 다음의 형태로 준비한다.<br>\n",
        "입력 데이터\n",
        "```\n",
        "문자열 'abc12'을 인코딩한\n",
        "[[0. 0. 0. 0. 0. 1. 0. 0. 0. 0.]\n",
        " [0. 0. 0. 0. 0. 0. 1. 0. 0. 0.]\n",
        " [0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]\n",
        " [1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
        " [0. 1. 0. 0. 0. 0. 0. 0. 0. 0.]]\n",
        "```\n",
        "\n",
        "출력 데이터\n",
        "```\n",
        "문자열 '3'을 인코딩한\n",
        " [0. 0. 1. 0. 0. 0. 0. 0. 0. 0.]]\n",
        "```\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4jO8skDb7p3f"
      },
      "source": [
        "인코딩 디코딩을 위한 유틸 클래스 CharacterTable"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-atmjECTwqNi"
      },
      "source": [
        "class CharacterTable:\n",
        "    \"\"\" 특정 문자열 집합을 가지고:\n",
        "    + one-hot 인코딩\n",
        "    + one-hot된 혹은 숫자로 인코딩된 값을 원 무자열로 디코딩\n",
        "    + softmax 같은 확율 벡터를 원 문자열로 디코딩\n",
        "    \"\"\"\n",
        "\n",
        "    def __init__(self, chars):\n",
        "        \"\"\"문자 테이블을 초기화\n",
        "        # Arguments\n",
        "            chars: 입력에 사용되는 문자들\n",
        "        \"\"\"\n",
        "        self.chars = sorted(set(chars))\n",
        "        self.char_indices = dict((c, i) for i, c in enumerate(self.chars))\n",
        "        # self.char_indices = { ' ': 0, \n",
        "        #                       '+': 1,\n",
        "        #                       '0': 2, \n",
        "        #                       '1': 3, \n",
        "        #                       ...\n",
        "        #                       '9': 11}\n",
        "        self.indices_char = dict((i, c) for i, c in enumerate(self.chars))\n",
        "        # self.indices_char = { 0: ' ', \n",
        "        #                       1: '+', \n",
        "        #                       2: '0', \n",
        "        #                       3: '1', \n",
        "        #                       ...\n",
        "        #                       11: '9'}\n",
        "\n",
        "    def encode(self, C):\n",
        "        \"\"\"전달된 문자열 C를 One-hot 인코딩 한다.\n",
        "        # Arguments\n",
        "            C: 인코딩할 문자열. 예 ' 123+45'\n",
        "        \"\"\"\n",
        "        x = np.zeros((len(C), len(self.chars)))\n",
        "        for i, c in enumerate(C): # 각 자리 문자마다\n",
        "            x[i, self.char_indices[c]] = 1 # 각 줄의 index만 1로 만든다. one-hot encoding한다.\n",
        "\n",
        "        # 반환되는 x는 num_rows의 행이고, 각 행은 각 문자에 대한 one-hot 인코딩된 값이다.\n",
        "        return x\n",
        "\n",
        "    def decode(self, x, calc_argmax=True):\n",
        "        \"\"\"전달된 벡터값 혹은 2D 배열을 해당 문자열로 디코딩한다.\n",
        "        # Arguments\n",
        "            x: 벡터 혹은 one-hot 인코딩된 2D 배열 혹은 softmax된 확률 2D 배열 \n",
        "               혹은 calc_argmax=False일 때는 문자 인덱스 벡터\n",
        "            calc_argmax: 최대 값의 index 값을 찾을 지 여부. default는 True\n",
        "        \"\"\"\n",
        "\n",
        "        # one-hot 인코딩 혹은 softmax 값이면 최대 index를 구한다.\n",
        "        if calc_argmax:\n",
        "            # x = [ [0 0 0 0 1 0 0 0 0 0 0 0]\n",
        "            #       [0 0 0 0 0 1 0 0 0 0 0 0]\n",
        "            #       [0 0 0 0 0 0 1 0 0 0 0 0]]\n",
        "            x = x.argmax(axis=-1)\n",
        "            # x = [ 4 5 6 ]\n",
        "        return \"\".join(self.indices_char[x] for x in x)\n"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MW8bFIwAwqKa"
      },
      "source": [
        "ctable = CharacterTable(BASE_STRING)"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I_fJZA7b7Y-e"
      },
      "source": [
        "다음은 인코딩, 디코딩 예이다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vwm40AA-r4tt",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4ba88e44-22a8-4ded-bdc9-42210598809c"
      },
      "source": [
        "# 원 문자열\n",
        "org_str = 'abc12'\n",
        "print(f\"org_str='{org_str}'\")\n",
        "\n",
        "# 인코딩된 값\n",
        "encoded = ctable.encode(org_str)\n",
        "# [[0. 0. 0. 0. 0. 1. 0. 0. 0. 0.]    <==== 'a'\n",
        "#  [0. 0. 0. 0. 0. 0. 1. 0. 0. 0.]    <==== 'b'\n",
        "#  [0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]    <==== 'c'\n",
        "#  [1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]    <==== '1'\n",
        "#  [0. 1. 0. 0. 0. 0. 0. 0. 0. 0.]]   <==== '2'\n",
        "print(f\"encoded=\\n{encoded}\")\n",
        "print(f\"encoded.shape={encoded.shape}\")\n",
        "\n",
        "# 다시 디코딩해서 원복한 값\n",
        "decoded = ctable.decode(encoded)\n",
        "print(f\"decoded='{decoded}'\")"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "org_str='abc12'\n",
            "encoded=\n",
            "[[0. 0. 0. 0. 0. 1. 0. 0. 0. 0.]\n",
            " [0. 0. 0. 0. 0. 0. 1. 0. 0. 0.]\n",
            " [0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]\n",
            " [1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
            " [0. 1. 0. 0. 0. 0. 0. 0. 0. 0.]]\n",
            "encoded.shape=(5, 10)\n",
            "decoded='abc12'\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4mg54AkZ3boa",
        "outputId": "8ee80baf-31e5-404f-e3f2-4b165a0ce789"
      },
      "source": [
        "print(\"Vectorization...\")\n",
        "\n",
        "x = np.zeros((len(raw_x), len(raw_x[0]), CATEGORY_NUM), dtype=np.bool)\n",
        "y = np.zeros((len(raw_y), len(raw_y[0]), CATEGORY_NUM), dtype=np.bool)\n",
        "for i, sentence in enumerate(raw_x):\n",
        "    x[i] = ctable.encode(sentence)\n",
        "for i, sentence in enumerate(raw_y):\n",
        "    y[i] = ctable.encode(sentence)\n",
        "\n",
        "print(x.shape)\n",
        "print(y.shape)\n",
        "\n",
        "shuffled_index = np.arange(len(x))\n",
        "np.random.shuffle(shuffled_index)\n",
        "x = x[shuffled_index]\n",
        "y = y[shuffled_index]\n",
        "\n",
        "\n",
        "split_at = int(len(x)*0.9)\n",
        "(train_x, test_x) = x[:split_at], x[split_at:]\n",
        "(train_y, test_y) = y[:split_at], y[split_at:]\n",
        "\n",
        "print(\"Training Data:\")\n",
        "print(train_x.shape)\n",
        "print(train_y.shape)\n",
        "\n",
        "print(\"Test Data:\")\n",
        "print(test_x.shape)\n",
        "print(test_y.shape)\n"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Vectorization...\n",
            "(94, 5, 10)\n",
            "(94, 1, 10)\n",
            "Training Data:\n",
            "(84, 5, 10)\n",
            "(84, 1, 10)\n",
            "Test Data:\n",
            "(10, 5, 10)\n",
            "(10, 1, 10)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lMq2Om3w8qNy"
      },
      "source": [
        "준비된 데이터는 다음과 같습니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "41UcKyn-q5Ez",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3122d751-57ca-43fe-b749-08560d7bfedd"
      },
      "source": [
        "print(\"train_x[0]\\n\", train_x[0].astype(np.int16))\n",
        "print()\n",
        "print(\"train_y[0]\\n\", train_y[0].astype(np.int16))"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "train_x[0]\n",
            " [[0 0 0 1 0 0 0 0 0 0]\n",
            " [0 0 0 0 1 0 0 0 0 0]\n",
            " [0 0 0 0 0 1 0 0 0 0]\n",
            " [0 0 0 0 0 0 1 0 0 0]\n",
            " [0 0 0 0 0 0 0 1 0 0]]\n",
            "\n",
            "train_y[0]\n",
            " [[0 0 0 0 0 0 0 0 1 0]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wa-gM3Nq84XK"
      },
      "source": [
        "준비된 데이터는 인코딩된 값들입니다. 디코딩에서 본 원 문자열은 다음과 같습니다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zw_RFfJvrGFM",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "975fb5fd-1428-460f-b02b-adef2e4eefe7"
      },
      "source": [
        "for i in range(10):\n",
        "  print(\"'\"+ctable.decode(train_x[i])+\"' -> '\"+ctable.decode(train_y[i])+\"'\")"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "'45abc' -> 'd'\n",
            "'bcde1' -> '2'\n",
            "'45abc' -> 'd'\n",
            "'de123' -> '4'\n",
            "'cde12' -> '3'\n",
            "'e1234' -> '5'\n",
            "'bcde1' -> '2'\n",
            "'5abcd' -> 'e'\n",
            "'5abcd' -> 'e'\n",
            "'12345' -> 'a'\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0-mHtIAk_qU9"
      },
      "source": [
        "# Template"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k3CG02lDAI73"
      },
      "source": [
        "## LSTM"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 806
        },
        "id": "GEffW4RR3bgJ",
        "outputId": "7d7baae4-e8ab-428d-a047-a858862310d3"
      },
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras.layers import Dense, RepeatVector, Flatten\n",
        "from tensorflow.keras.layers import Bidirectional, LSTM, GRU\n",
        "\n",
        "input_shape = train_x.shape[1:]\n",
        "# input_shape = (5,10)\n",
        "\n",
        "\n",
        "model = keras.Sequential()\n",
        "model.add(LSTM(128, input_shape=input_shape))\n",
        "model.add(RepeatVector(1))\n",
        "model.add(Dense(CATEGORY_NUM, activation=\"softmax\"))\n",
        "model.compile(loss=\"categorical_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n",
        "model.summary()\n",
        "\n",
        "history = model.fit(train_x, train_y, epochs=100, verbose=1, validation_split=0.1, batch_size=128)\n",
        "\n",
        "\n",
        "plt.plot(history.history['loss'])\n",
        "plt.plot(history.history['val_loss'])\n",
        "plt.show()\n",
        "\n",
        "\n",
        "\n",
        "loss, acc = model.evaluate(test_x, test_y)\n",
        "print(\"loss=\", loss)\n",
        "print(\"acc=\", acc)\n",
        "\n",
        "\n",
        "\n",
        "y_ = model.predict(test_x)\n",
        "predicted = np.argmax(y_, axis=-1)\n",
        "test_y_index = np.argmax(test_y, axis=-1)\n",
        "\n",
        "plt.plot(test_y_index, \"o\")\n",
        "plt.plot(predicted, \".\")\n",
        "plt.show()"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "lstm (LSTM)                  (None, 128)               71168     \n",
            "_________________________________________________________________\n",
            "repeat_vector (RepeatVector) (None, 1, 128)            0         \n",
            "_________________________________________________________________\n",
            "dense (Dense)                (None, 1, 10)             1290      \n",
            "=================================================================\n",
            "Total params: 72,458\n",
            "Trainable params: 72,458\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3xUVfrH8c+TXkgjJCEkgVBDJ0AEFBBERbABKth2bShrX137z67rru66rm3Xih0bihRFWVGkiJTQey8JLY30npzfH3fQiCAB7s0kM8/79ZpXpuU853r16+TMueeIMQallFJNn4+7O6CUUsoeGuhKKeUhNNCVUspDaKArpZSH0EBXSikP4eeuwi1atDDJycnuKq+UUk3SsmXLcowxMUd6zW2BnpycTHp6urvKK6VUkyQiu472mg65KKWUh9BAV0opD6GBrpRSHkIDXSmlPIQGulJKeQgNdKWU8hAa6Eop5SHcNg/9hGVthHVTICTadWsOIS1+eewf5O4eKqWUWzS5QK/ctxb/uf9AOMo67v6hEHoo7F1BH9ril+APjXE9dj0fGA4iDXsQSinlgCYX6N/7DuKW8vfoH+/DWa39GNjK0CGsCt+yPCjJgdI8KM2x7pdkQ/Ym63FV6ZEb9A2wQj4k2vrZLNYK+tBY1/0YaBZn3UKag49vwx6wUkrVU5ML9G6twvnLOV35YVMWTy3Np6bWEBEcxOCOPRmaEsuQ3jHEhAX+9hcrS38J+tLcXwK/bviX5EDOZijOgpqK37YhvlbAh7Wsc2sF4fHWz4hEiEiAwDDn/0EopdRhxF1b0KWlpZmTXculoLSK+Vuz+WFTNnM3Z5NdZIVwj4QIzkiJYWjnWHolRuLrc5xDKsZARZEV8sVZUHzAdf8AFO2DogNQtN+6X5rz298PjIDI1hCZZP2MSoaottbP5m3B7wj/w1FKqXoQkWXGmLQjvtaUA72u2lrD+n2F/LApix82ZbN890FqDUSF+DOkUwxndI5lSKcYIkMCbKsJQHWFFe6Fe6FwDxRkum4ZkJ8B+bugsviX94uP9Uk+ugO0SIGYFIjpDLFdIDjS3r4ppTyOVwT64fJLK5m3JYcfNmbxw+Zs8koq8RHo0zqKYV1iObNzHJ3imiFOfyFqjDXEk7cDDu6A3G2QuxVyt0DOll+P7Ue0hpbdIb4XtOpt3ZrFOts/pVST4pWBXldNrWF1Zj5zNmbx3cYs1u0tBCAxKpgzO8dyZpc4+rdrTqBfA3/hWVtrfZLP3gQH1lq3/WutcfxDs3giWkPr/pDUH9qcBjFdwEcvH1DKW3l9oB9uf0E5czZl8d2GAyzYmkN5VS2hAb4MSYlheNeWnJESS0SIv1v6BkBFMexfDXuWQeZS2L0Yivdbr4VEQ/JgaH8GdDjb+hJWKeU1NNB/R1llDQu35TB7wwG+XZ9FTnEFfj7CgHbRnNMtjuHdWhIX7uaLlYyB/N2w60fYMQ+2z4WivdZrsd2g0znQ9UKIT9U59Up5OA30eqqtNazMzOfb9QeYtW4/27NLAOjTOpJze8QzontLEqNC3NxLrIDP3ghbvoUt/4NdC8HUWDNquo6GXpdBXDd391Ip5QAN9BO0NauIb9bu5+u1+38ed++VFMkFPeM5r2c88RHBbu6hS2kebPwKNkyHbd9DbTW07AmpV1jhHhzl7h4qpWyigW6DnTklzFy7j5lr9rF2jxXu/ZKbM6p3K87rEW//dMgTVZIDaz+HlR/CvpXgFww9x8IpN0B8T3f3Til1kjTQbbY9u5gvV+9j+qq9bM0qxt9XGJoSy8V9EhjWOY4Av0YyC2XfKlg6EdZMtqZHtjsDBt0JbU/XsXalmigNdIcYY1i3t5BpK/cwbeVesooqiArx58JerRh3ShLdWkW4u4uWsnxY9g789B8oyYKEvjDsIWg/zN09U0odJw30BlBdU8uCrTl8tiyT/60/QGV1LT0TI7j0lCRGpSbQLLARLJtTVQ6rPoT5z1nz39ueDmc+Bol93d0zpVQ9aaA3sPzSSqau2MPHSzPYuL+I0ABfxvRJ4A8D2tC5Zbi7u2ctV5D+Nsz7p7UWTa8r4OzH9apUpZoADXQ3McawMiOfDxbtZsbqvVRW13Jqu2iuG9SWYZ1jj3/RMLtVFFmf1he+BP4hMOxBOOV6XSJYqUZMA70ROFhSySfpGby3cCd7C8ppEx3C+EFtGds3ieAANwdozhb4+l5rymNSfxj9CkS3d2+flFJHpIHeiFTX1DJr3QHeXLCdFbvzaR4awDWnJXP1qcnuXW7AGFj9KXx9D1RXwtlPuD6tN5IZO0opQAO9UTLGsGRHHq/N2873G7MIC/TjmoHJjB/U1r1z2gv3wvTbYOts6DQSxryiFyYp1YhooDdyG/YV8tL3W5i5Zj+hAb6MH9SW609vR3iQmz6xGwOLX4P/PQThrWDce9Aq1T19UUr9igZ6E7FpfxEvfreFr9bsIzLEn5uHtueqU5MJ8nfTGHvGUph8jbVb04UvWssIKKXcSgO9iVm7p4B/ztrE3M3ZJEQGc9/IzlzQM975zTiOpCQXPrvGWuVxyH0w9AG9ylQpN/q9QNdvvBqh7gkRvHtdPz68oT8Rwf7c/tEKLn5lIasy8hu+M6HRcOXnkPoHmPsMTJlgzWNXSjU6xwx0EUkSkTkisl5E1onIn4/wHhGRF0Vkq4isFpE+znTXu5zWvgUzbhvEPy7uScbBMkb/90ce/GIN+aWVDdsRvwAY9TIMexjWfAofjoPKkobtg1LqmOrzCb0auMsY0xUYANwiIl0Pe89IoKPrNgF4xdZeejFfH2HcKUl8f9cQrj2tLR8t2c2wf83l82WZNOhwmQicfrc1R33HPPjgYigvbLj6SqljOmagG2P2GWOWu+4XARuAw/c9GwW8ZyyLgEgRibe9t14sLMifRy7oyozbBpEcHcJdk1dx7TtL2Ztf1rAdSb0CLp5obY333ihrLXalVKNwXGPoIpIM9AYWH/ZSApBR53Emvw19RGSCiKSLSHp2dvbx9VQB0K1VBJNvPI1HL+jK4u15DP/3PD5asrthP613vwgu/cDa1PqDi/STulKNRL0DXUSaAZ8DdxhjTui/YGPM68aYNGNMWkxMzIk0obCGYa4d2JZZd5xOj4QIHpiyhgnvLyO3uAG/rEwZCePeh/1r4KPLoLK04WorpY6oXoEuIv5YYT7JGDPlCG/ZAyTVeZzoek45qHV0CJOu789D53Vh7qZsRrwwn7mbG/Avn5QRMOY1a0/TT6+ylgxQSrlNfWa5CDAR2GCMee4ob5sOXOWa7TIAKDDG7LOxn+oofHyE6we3Y9qtA2keEsDVby3h2VmbqKltoCGYHpfABS/A1m9h6o1QW9swdZVSv1GfXRcGAn8E1ojIStdz/we0BjDGvArMBM4FtgKlwLX2d1X9ni7x4Uy7dSCPTlvHy3O2snz3QV64rDcxYYHOF+97NZTlwezHICoZznzE+ZpKqd/QK0U90OT0DB6etpbwIH9evyqN1KRI54saAzP+DMvfhQtfhj5/dL6mUl5IrxT1MmPTkvji5oEE+Pkw7rWfmLqiAb7OEIHz/mXtU/rlHbBtjvM1lVK/ooHuobrEhzP91kH0Torkjk9W8vTXG6l1elzd1x/GvgstUmDy1ZC33dl6Sqlf0UD3YM1DA3h/fH+u6N+aV+du4/aPV1BRXeNs0aBwuGwSIPDxlVBR7Gw9pdTPNNA9XICfD0+N7s79Izvz5ep9XP3WEgrKqpwt2rwtjH0bsjfCtJut8XWllOM00L2AiHDjkPa8cFkqy3YdZNyrP5FVWO5s0fbD4KzHYP00+PF5Z2sppQANdK8yKjWBd67tR8bBUsa+9hOZBx2+uvO026HbRfDdE7DzR2drKaU00L3NwA4t+OD6/uSVVDLu1Z/YkePgMrgi1k5HzdvB5+OhWNfvUcpJGuheqE/rKD66YQDl1bWMe+0ntmY5+MVlYJg186XsIHwxQa8kVcpBGuheqntCBJ9MGIAxcMUbi9jp5Cf1lt1h5DOw7XuY/y/n6ijl5TTQvVjHuDAmXd+fqpparnhjERl5Do6p97kaeoyFH/4Ouw9ffVkpZQcNdC+X0jKMD67vT0llDZe/sYh9BQ5tmCEC5z0HEYkw5XooL3CmjlJeTANd0a1VBO+P70d+aRVXTVzi3J6lQeFw8ZtQsAe+utuZGkp5MQ10BUDPxEhev6ovu3JLGf9uOmWVDl1RmtQPhtxnbTa96hNnaijlpTTQ1c9Oa9+C5y9LZfnug9zy4XKqahyakTL4Lmh9Knx1F+TvdqaGUl5IA139yrk94nlyVHe+35jFo9PXObNXqa8fjHkVMDD1Zp3KqJRNNNDVb/xhQBtuHtqeDxfv5s35O5wpEpUMI56GnfNh0X+dqaGUl9FAV0d09/AUzu3Rkr99vYFZ6/Y7U6T3HyDlXGtpgKwNztRQyotooKsj8vERnhuXSs/ESO74eCVr9zgwzVAELnjRupp0ygTdZFqpk6SBro4qyN+XN69KIyrEnwnvpZNTXGF/kWYx1ibT+1fD/Gftb18pL6KBrn5XTFggr/0xjdySSm6Z5NDMly7nQ89LYd6zsHeF/e0r5SU00NUx9UiM4OmLe7B4Rx5PfeXQWPfIZ6BZLHxxE1Q5vFa7Uh5KA13Vy5jeiYwf1JZ3Fu7k82WZ9hcIjoILX4LsDfDD3+xvXykvoIGu6u2BkZ0Z0K45D05dw6b9RfYX6Hg29LkKFr4EGUvsb18pD6eBrurNz9eHFy/vTbNAf26atIziimr7iwx/CsITYOpNUOXQQmFKeSgNdHVcYsOCeOny3uzMKeGBKWvsv5I0KNwaesndCt//1d62lfJwGujquJ3aPpq7hqcwY9VeJi12YC2W9mdA32vhp//o2ulKHQcNdHVCbhrSntM7xfDkl+udGU8f/iREJFlDL5UOb2atlIfQQFcnxMdH+NfYXoQF+XH7Rysor7J5ud3AMBj1MuRtg++ftLdtpTyUBro6YTFhgTw7thebDhTxt5kOzE9vNwROuQEWvQK7FtrfvlIeRgNdnZShKbGMH9SW937axbfrD9hf4KzHIKqNtcxupYMbWSvlATTQ1Um7d0QK3VqFc//nq+1f7yWwGYz6LxzcAbMft7dtpTyMBro6aYF+vjx/aSpFFdXc/7kDUxmTB0L/m2DJa7Bjnr1tK+VBNNCVLTrGhXHvOSnM3nCAyekOLA1w5iPQvD1MuwUqHJhVo5QH0EBXtrluYFtObRfN4zPWkZFn81TDgBAY/QoUZML/HrK3baU8xDEDXUTeEpEsEVl7lNeHikiBiKx03R6xv5uqKfDxEZ4d1wsfEe6evIraWpuHXlr3h1NvhWXvwNbZ9ratlAeozyf0d4ARx3jPfGNMquv2xMl3SzVVCZHBPHxBVxbvyOP9RbvsL3DGgxDTGabdBmX59revVBN2zEA3xswD8hqgL8pDjO2byNCUGJ7+eiO7c20eevEPsoZeig/A1/fZ27ZSTZxdY+inisgqEflaRLod7U0iMkFE0kUkPTs726bSqrEREf5+UQ/8fIR7PnNg6CWhD5x+D6z+GNZPs7dtpZowOwJ9OdDGGNMLeAmYerQ3GmNeN8akGWPSYmJibCitGqv4iGAePt/BoZfT74b4VJhxBxQ5cEGTUk3QSQe6MabQGFPsuj8T8BeRFifdM9XkjU1L5PROMTzzzUYyD9o89OLrDxe9bl09OuPPYPfcd6WaoJMOdBFpKSLiut/P1Wbuybarmj4R4W9jugPw4Bdr7b/gKCYFznoUNn8NK963t22lmqD6TFv8CPgJSBGRTBEZLyI3isiNrrdcAqwVkVXAi8Blxvb/clVTlRgVwj3npDB3czZTV+6xv0D/myB5MHzzAOTtsL99pZoQcVf2pqWlmfT0dLfUVg2rptZwyasL2ZlTwuy/DCG6WaC9BfIz4JWBENsZrv0afHztbV+pRkRElhlj0o70ml4pqhzn6yM8c3FPiiuqeXzGevsLRCbBec9CxmJY8G/721eqidBAVw2iU1wYNw3twPRVe5m72YEpqz3GQreL4Ie/w94V9revVBOgga4azM1D29OuRSgPTV1DWaXNOxyJwHn/gtBY+PwG3bZOeSUNdNVggvx9eWpMDzLyynjx+y32FwhpDmNegdwt8L8H7W9fqUZOA101qFPbR3NJ30TemLedjfsL7S/QbiicdhukvwWbvra/faUaMQ101eAePLcL4cH+/N+UNfYvCwAw7GFo2cNaO12vIlVeRANdNbio0AAePLcLy3fn80l6hv0F/ALhojetq0in3gS1tfbXUKoR0kBXbnFRnwT6t23O019vtH8fUrDmpJ/zFGz7Dha/an/7SjVCGujKLUSEp8Z0p7Symr/P3OhMkbTxkHIezH4U9q1ypoZSjYgGunKbDrFh3DC4HZ8vz2TxdgeW/xGBC1+CkGj4/HprCEYpD6aBrtzqtmEdSYwK5qGpa6msdmCsOzQaxrwGOVus9V6U8mAa6MqtggN8eWJUN7ZkFfPWjw4trtVuCAy6A5a/C2unOFNDqUZAA1253bDOcZzdNY4XZm9hT36ZM0XOeBAST7HWTj+405kaSrmZBrpqFB69oCsGw5NOLN4F1oYYF79p3f9sPNRUOVNHKTfSQFeNQmJUCLcN68g36/YzZ1OWM0WikuGCF2BPOnz/V2dqKOVGGuiq0bhhcDvaxYTy6LR1lFfZvHjXId0vgj5Xw4/Pw9bZztRQyk000FWjEeDnw5OjurM7r5RX525zrtCIpyG2K0z5ExTtd66OUg1MA101KgM7tOD8nvH894dt7Mp1aN54QAhc8rY1L33KDVDr0F8DSjUwDXTV6Dx0Xlf8fYTHpq+zf2PpQ2I7W7sc7ZgH8551poZSDUwDXTU6LSOCuOOsTszZlM236x1cLTH1Suh5Kcx9GnYucK6OUg1EA101StcMTKZTXDMen7He/t2NDjm0y1HzdtZUxmIHtsZTqgFpoKtGyd/X+oJ0T34ZL89xYHejQwLDYOw7UHYQvpigS+2qJk0DXTVa/dtFc1HvBF6ft52tWcXOFWrZA0Y+Ddu+hwXPOVdHKYdpoKtG7YFzuxDk78uj09c69wUpQN9rodtFMOcp2LXQuTpKOUgDXTVqMWGB3HtOCj9uzWXG6n3OFRKxriKNSrbG00tynKullEM00FWjd0X/NvRIiOCvX66nqNzBNViCwq3x9NIc+OJPOp6umhwNdNXo+foIT47uTnZxBc99u9nZYvG9YMTfrWUBfnze2VpK2UwDXTUJqUmRXNm/Ne8u3MnaPQXOFksbD11HWwt47V7kbC2lbKSBrpqMe87pTPPQAB6cupaaWge/IBWBC1+EyCT47DoozXOullI20kBXTUZEsD8PndeVVRn5fLRkt7PFgiKs8fSSbJh6Ezg5w0Ypm2igqyZlVGorTmsfzTPfbCS7qMLZYq16w/C/wuZv4KeXna2llA000FWTImJ9QVpRVcuTXzq0u1Fd/SZA5/Nh9mOwZ5nz9ZQ6CRroqslpH9OMm4a2Z/qqvczd7PD6KyIw6mUIi7fG08sd/kJWqZOgga6apJvPaE+7mFAemrrGucW7DgmOgosnQn4GzLhDx9NVo3XMQBeRt0QkS0TWHuV1EZEXRWSriKwWkT72d1OpXwv08+VvY3qQkVfGC985uHjXIa37w7AHYd0UWPG+8/WUOgH1+YT+DjDid14fCXR03SYAr5x8t5Q6tgHtohmXlsgb87ezYV+h8wUH3gnthsLMeyF7k/P1lDpOxwx0Y8w84Pcm4o4C3jOWRUCkiMTb1UGlfs8DI7sQGezP/Z+vdnZuOoCPD4x5HQJCrfVeqsqdrafUcbJjDD0ByKjzONP13G+IyAQRSReR9Oxs3UxAnbyo0AAevbAbqzILePvHHc4XDIuD0f+FA2vgu8edr6fUcWjQL0WNMa8bY9KMMWkxMTENWVp5sAt6xnNm51ie/d8mdueWOl+w0znQ70+w6L+w5Vvn6ylVT3YE+h4gqc7jRNdzSjUIEeGvY7rj5+PD/32xxtl10w85+wmI7QZf3AjFWc7XU6oe7Aj06cBVrtkuA4ACY4yDC1cr9VvxEcHcN7IzC7bm8Gl6xrF/4WT5B8ElE6GyGKbdqlMZVaNQn2mLHwE/ASkikiki40XkRhG50fWWmcB2YCvwBnCzY71V6ndc2a81/do2569fbmBfQZnzBWO7WJ/Ut8yC9InO11PqGKRB/jw9grS0NJOenu6W2spz7cotYcTz8+nfrjlvX3MKIuJswdpamHSJtW3dn+ZBTCdn6ymvJyLLjDFpR3pNrxRVHqVNdCj3jUjhh03ZfLYs0/mCPj7WrBf/YJhyPVRXOl9TqaPQQFce56pTk+mX3JwnvlzP/oIGmCse1tJaP33fKpj3T+frKXUUGujK4/j4CP+4pCdVNbXc9/nqhpn10uUCSL0S5v8LMnUoUbmHBrrySMktQvm/c7swd3M2kxY7vBnGISP+DuGtrA2mKxtgPrxSh9FAVx7rjwPaMLhjC576agM7c0qcLxgUYY2n526F2Y86X0+pw2igK48lIvzzkl74+wp/+XQl1TW1zhdtezoMuBmWvA7b5jhfT6k6NNCVR2sZEcSTo7uzfHc+r87d1jBFz3wEWnSyLjjSDTFUA9JAVx7vwl6tuKBXK56fvYWVGfnOF/QPhtGvQNFemPV/ztdTykUDXXk8EeGvo7sTFx7Enz9eQXFFtfNFE9Ng4B2w4gPYPMv5ekqhga68RESwP/++NJWMvFIem76uYYoOvR9iu8L026H097YUUMoeGujKa/Rr25xbz+jAZ8symbFqr/MF/QKtoZfSHPjmfufrKa+nga68yu1ndqRP60gemLKGXbkNMJWxVSoMvgtWfwIbv3K+nvJqGujKq/j5+vDi5b3x9RFu+XA5FdU1zhcdfDfE9YAZd+jQi3KUBrryOolRIfzzkp6s3VPI32dudL6gX4B1wVFZHnx9r/P1lNfSQFdeaXi3llw3sC3vLNzJN2sbYD+W+J5w+r2wZjJsmOF8PeWVNNCV17p/ZGd6JUZwz+TV7GiIpQEG/wVa9oQv74SSXOfrKa+jga68VoCfD/+5sg9+vsKN7y+jtNLh+em+/tasl7J8mHm3s7WUV9JAV14tMSqEFy/vzeasIh6Y0gAbTLfsDkPvg3VTYN0XztZSXkcDXXm9wR1juHt4CtNW7uWdhTudLzjwTohPha/uguJs5+spr6GBrhRw05D2nNUljqe+2sDCbTnOFvP1gzGvQkURfHUnuGlfX+V5NNCVwtrl6N+X9iK5RSi3TFpORp7DG1TEdoEzHrRmvKz+1NlaymtooCvlEhbkz5tXpVFr4Ib30ilxehGv026DpAEw8x4o2ONsLeUVNNCVqiO5RSgvX9GbzQeKuPOTldTWOjgc4uNrXXBUWwXTb9OhF3XSNNCVOszgjjE8fH5X/rf+AM984/CVpNHtYfiTsO07SJ/obC3l8TTQlTqCa05L5upT2/DavO1MWrzL2WJp46H9mTDrIcjZ4mwt5dE00JU6AhHh4fO7ckZKDI9MW8fczQ5OLxSBUf+xdjqacgPUVDlXS3k0DXSljsLP14eXruhDp7gwbpm0nLV7HNwfNDweLngB9q6AH552ro7yaBroSv2OZoF+vH3NKYQH+XHN20udnc7Y9ULo/QdY8Bzs+sm5OspjaaArdQwtI4J4b3w/qmpqueqtJeQWVzhXbMTTENnGGnopO+hcHeWRNNCVqocOsWG8dU0a+wrKuPadpc5tNB0YBpdMhKJ9OpVRHTcNdKXqqW+b5rx8eR/W7S1k/DtLKa9yaLejhL5w1mPWVaQ6lVEdBw10pY7DWV3jeG5cL5bszOOmD5ZRWV3rTKEBt0CHs+Gb/4P9a52poTyOBrpSx2lUagJPje7BnE3Z3PnJSqprHAh1Hx9r7fTgKJh8NZQX2l9DeRwNdKVOwBX9W/PguV34as0+7pq8yplQbxYDl7wFeTtg2i06nq6OqV6BLiIjRGSTiGwVkfuP8Po1IpItIitdt+vt76pSjcsNp7fj3hHWOup3TV5FjRPrviQPhLMfhw3T4aeX7W9feRS/Y71BRHyB/wBnA5nAUhGZboxZf9hbPzHG3OpAH5VqtG4e2gFj4J+zNiHAs2N74edr8x++p94KGUvg20ehVR8r5JU6gvr8m9cP2GqM2W6MqQQ+BkY52y2lmo5bzujAPeekMHXlXv788Ur7vyg9tDRA87bWeHp+hr3tK49Rn0BPAOr+G5Tpeu5wF4vIahH5TESSjtSQiEwQkXQRSc/O1q23lOe45YwOP4+p3/jBMvunNAaFw2UfQnUFfHwFVJbY277yCHb9bTgDSDbG9AS+Bd490puMMa8bY9KMMWkxMTE2lVaqcbjh9HY8NaY7czZlcd07S+3fICMmBS6eCPvXwNSb9UtS9Rv1CfQ9QN1P3Imu535mjMk1xhy6HvpNoK893VOqabmyfxv+NbYXi3fkcfkbi8ixe5mATsOtL0nXT4W5/7C3bdXk1SfQlwIdRaStiAQAlwHT675BROLrPLwQ2GBfF5VqWi7qk8jrf+zL5gNFXPLKQnbn2ryg12m3Q6/L4Ye/wapP7G1bNWnHDHRjTDVwKzALK6g/NcasE5EnRORC19tuF5F1IrIKuB24xqkOK9UUnNkljknXDyC/rIqLXlnImkwbl94VsZbaTR5szU/fMc++tlWTJsZN43BpaWkmPT3dLbWVaihbs4q4+q2l5JVU8vxlqZzTraV9jZcdhLdGQOE+GD8LYrvY17ZqtERkmTEm7Uiv6ZWiSjmoQ2wYU28ZSErLMG78YBlvzNuObR+igqPgysngHwQfXKzTGZUGulJOiwkL5OMJAzi3ezxPzdzAvZ+tpqLapmmNka3hys+gohjeHw3FWfa0q5okDXSlGkCQvy8vXd6b24d1YPKyTC59bREHCsvtaTy+J1z5KRTsgfcvgrJ8e9pVTY4GulINxMdH+MvwFF79Qx82Hyji/JcWkL4zz57GWw+Ayz6A7I3w4TioKLKnXdWkaKAr1cBGdI/ni5sHEhrgy6WvL7JvXL3DWdZuR5npMGmshroX0kBXyg1SWoYx/bZBnN0ljqdmbmDC+8soKBA4/VcAAA2BSURBVKs6+Ya7jrJCPWOJhroX0kBXyk3Cg/x55Q99ePj8rszZmMW5L8xnqR1DMN3G/BLqH1yim2N4EQ10pdxIRBg/qC2f3XQavj7Cpa/9xL+/3XzyG2YcCvU96fDu+VCSY0+HVaOmga5UI5CaFMlXtw9idGoCL3y3hbGv/cT27OKTa7TbGLjsI8jeBG+PhIJMezqrGi0NdKUaibAgf567NJUXLktle3YJ5744n3d+3EHtyeyE1Gk4/PELKNpvXVWatdG+DqtGRwNdqUZmVGoC/7vzdE5tF81jM9Zz+RuL2JFzEuuftzkNrp5hraU+cThsn2tfZ1WjooGuVCMUFx7EW9ecwj8u7sn6fYWc8/w8/vvDVqpOdGy9VSpcPxvC461lAlZ+ZG+HVaOgga5UIyUijDslie/+MoRhKbH845tNXPDSApbtOsGZMFFt4LpZ0OZUmHojfPsI1Nq8s5JyKw10pRq52PAgXv1jX179Q18Kyqq4+JWfuGfyKnJPZPOM4Ei48nNIGw8/vgCTLoFSm65WVW6nga5UEzGie0tm/2UIfxrSji9W7OGMZ3/grQU7jn8Yxi8Azn8OLngRdi6A14fC3pWO9Fk1LA10pZqQ0EA/HhjZha//PJheSZE88eV6znl+Ht9vPHD8ywf0vRqumQk1VTDxbFj0qu5T2sRpoCvVBHWMC+O96/ox8eo0jIHr3knn8jcWsSrjOFdaTDoFbvoR2p8J39wHH10OJbnOdFo5TncsUqqJq6yu5aMlu3nxuy3kllRyXo947jy7Ix1iw+rfiDGw+DX49mEIDLeGZLqOcq7T6oT93o5FGuhKeYii8iremLediQt2UFZVw6jUBG4/syNtW4TWv5ED62DqTbBvFXS/GEb+A0JbONdpddw00JXyInkllbw2bxvvLtxJZXUtF/Zqxc1ndKBTXD0/sddUwYLnYe4zEBACZz4Cfa8FH19nO67qRQNdKS+UVVTOm/N38MGiXZRW1jC8axx/GtKOvm2a17OBjTDzbtg5H+JTrU/rrfs722l1TBroSnmxgyWVvL1wJ+8u3ElBWRW9W0dyw+B2DO8ah5/vMeZFGANrP4dZD0Lxfkg5z/rEHtu5YTqvfkMDXSlFaWU1k9MzmbhgB7vzSomPCOKKfq25rF9rYsICf/+XK0tg0X9hwQtQVQI9xsHgv0BMSsN0Xv1MA10p9bOaWsN3Gw7w/qJdzN+Sg7+vMLxrS8adksSgDi3w9ZGj/3JJLix4DtLfgqoy6HweDLwDEtNAfuf3lG000JVSR7Qtu5hJi3bzxYpMDpZWkRAZzJjeCYzu3er3pz2W5MDiV2Hx61BRAPG9oN8Ea2aMf3DDHYAX0kBXSv2uiuoaZq/P4tP0DOZvyabWQPeEcM7v2Ypzu8fTOjrkKL9YBKs+hqVvQvZGaw57t9HQ6wpoPUA/tTtAA10pVW9ZReV8uWof01buYVVmAWCF+zldW3Jmlzi6xIchhwe1Mda6MCsnwfrp1jh7RGvocgF0vRAS+4GPXphuBw10pdQJycgr5Zu1+/lqzT5WupYVSIgMZkhKDKd3bMGp7VsQEez/61+qKIaNX8LaKbB9DtRUQmgMdDjLurUfBiH1nDqpfkMDXSl10rKKypmzMYvZG7JYuDWHksoafAR6JEYyoG1zBrSLpm9yFOFBdQK+vBA2z4LN38C276DsICAQ1x2SB1m3pH7QLNZtx9XUaKArpWxVVVPLyox85m3OZtH2XFZm5FNVYxCBjrHN6NM6itSkSHokRtApLgx/Xx9rM409y61P7TvnQ8YSqC63GoxsA4mnWDsrxadCfE8IinDvQTZSGuhKKUeVVdawfPdB0nceZPnug6zYfZDC8moAAvx86NIyjC7x4XRuGUZKy3A6xTUjOgjYuwIyl1rhvmcZFO75pdGIJIjrBrFdrfnuLTpCdEcICnfPQTYSGuhKqQZVW2vYlVfKmj0FrMnMZ93eQjbsK+RgadXP74kK8adDbDPatgilbYtmtG0RQnJwKa0rthCSsxay1sOB9ZC7BWqrf2k8NBaat7NuUW0gsrX1CT8iEcJbga//EXrkOTTQlVJuZ4whu6iCjfuL2JpVzJasYrZlFbMjt4Tsol9vpxcV4k9CVDCtIoJJCvejU0AOyewhviqDqIpMQop343twBxTtA+pmmECzOGsz7LBWENbSujWLtZ4PjbFWjwxpAQGhTXJapQa6UqpRKyqvYlduKbvzfrntzS9jb34Zew6WUVL5282sQwJ8aRkqdA4uoENAHkk+ubQklxYmh8iqHJpVZRNUloV/5VE2/fALgpBoCG4OIVEQ7LoFRVp7rwZFWLfACAgMs4Z6AppZ9wOaga+fw/9Ujuz3Ar1ePRKREcALgC/wpjHm6cNeDwTeA/oCucClxpidJ9NppZT3CAvyp3tCBN0TjvxFaFF5FQcKy9lfUEFWUTlZRRVkFVaQV1JBbkkk3xbHc7CkkrzSSiqrf73HagBVRFNIjOQTLYW08ism3r+IGCkmuqKEyIpCIvJzaWZ20ay2iJCaQnxN9RH7UVetbyC1/s0wAaHgHwIBoYjr5hMQggQEW8/7BVk//YOs+35B1pe+CX1t+WdX1zEDXUR8gf8AZwOZwFIRmW6MWV/nbeOBg8aYDiJyGfAMcKntvVVKeaWwIH/CgvyPuQuTMYbSyhryy6ooKK0iv6ySwrJqCsurKCyroqi8muKKajLKq9lQWU1JxaFbDWVVNZRWVlNaVY2pKiOktoRwKSGcUppJGWGUEer62YwyQqrLCa0oJ1TKCaGCYCoIkSKCqSCISkLF+hkkFQRR9at+rmxzLanXuiHQgX7AVmPMdgAR+RgYBdQN9FHAY677nwEvi4gYd43nKKW8kogQGuhHaKAfCZEnt6ZMVU0tpZU1VFTVUF5VS1lVDRXVNVRU11JeVUNldS0V1bVUVNeQX13LgepaKqtrqaxx/ayupbrWUFldS1V1Naa6Aqkux1SXMzAlkVSbjrmu+gR6ApBR53EmcPgq9z+/xxhTLSIFQDSQY0cnlVKqofn7+hAR7AOHXwnbiDXo4goiMkFE0kUkPTs7uyFLK6WUx6tPoO8Bkuo8TnQ9d8T3iIgfEIH15eivGGNeN8akGWPSYmJiTqzHSimljqg+gb4U6CgibUUkALgMmH7Ye6YDV7vuXwJ8r+PnSinVsI45hu4aE78VmIU1bfEtY8w6EXkCSDfGTAcmAu+LyFYgDyv0lVJKNaB6zUM3xswEZh723CN17pcDY+3tmlJKqeOhK84rpZSH0EBXSikPoYGulFIewm2Lc4lINrDrBH+9Bd550ZI3Hrc3HjN453F74zHD8R93G2PMEed9uy3QT4aIpB9ttTFP5o3H7Y3HDN553N54zGDvceuQi1JKeQgNdKWU8hBNNdBfd3cH3MQbj9sbjxm887i98ZjBxuNukmPoSimlfqupfkJXSil1GA10pZTyEE0u0EVkhIhsEpGtInK/u/vjBBFJEpE5IrJeRNaJyJ9dzzcXkW9FZIvrZ5S7++oEEfEVkRUi8qXrcVsRWew655+4Vv30GCISKSKfichGEdkgIqd6w7kWkTtd/36vFZGPRCTIE8+1iLwlIlkisrbOc0c8v2J50XX8q0Wkz/HUalKBXmd/05FAV+ByEenq3l45ohq4yxjTFRgA3OI6zvuB74wxHYHvXI890Z+BDXUePwP82xjTATiItYetJ3kB+MYY0xnohXXsHn2uRSQBuB1IM8Z0x1rJ9dB+xJ52rt8BRhz23NHO70igo+s2AXjleAo1qUCnzv6mxphK4ND+ph7FGLPPGLPcdb8I6z/wBKxjfdf1tneB0e7poXNEJBE4D3jT9ViAYVh71YKHHbeIRACnYy1BjTGm0hiTjxeca6zVXoNdm+KEAPvwwHNtjJmHtax4XUc7v6OA94xlERApIvH1rdXUAv1I+5smuKkvDUJEkoHewGIgzhizz/XSfiDOTd1y0vPAvUCt63E0kG+MqXY99rRz3hbIBt52DTO9KSKhePi5NsbsAZ4FdmMFeQGwDM8+13Ud7fyeVMY1tUD3KiLSDPgcuMMYU1j3NdeOUB4151REzgeyjDHL3N2XBuQH9AFeMcb0Bko4bHjFQ891FNan0bZAKyCU3w5LeAU7z29TC/T67G/qEUTEHyvMJxljpriePnDozy/Xzyx39c8hA4ELRWQn1nDaMKzx5UjXn+Xgeec8E8g0xix2Pf4MK+A9/VyfBewwxmQbY6qAKVjn35PPdV1HO78nlXFNLdDrs79pk+caN54IbDDGPFfnpbp7t14NTGvovjnJGPOAMSbRGJOMdW6/N8ZcCczB2qsWPOy4jTH7gQwRSXE9dSawHg8/11hDLQNEJMT17/uh4/bYc32Yo53f6cBVrtkuA4CCOkMzx2aMaVI34FxgM7ANeNDd/XHoGAdh/Qm2Gljpup2LNZ78HbAFmA00d3dfHfxnMBT40nW/HbAE2ApMBgLd3T+bjzUVSHed76lAlDeca+BxYCOwFngfCPTEcw18hPU9QRXWX2Tjj3Z+AcGaybcNWIM1C6jetfTSf6WU8hBNbchFKaXUUWigK6WUh9BAV0opD6GBrpRSHkIDXSmlPIQGulJKeQgNdKWU8hD/DzDiSWRnqv5QAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 16ms/step - loss: 0.0205 - accuracy: 1.0000\n",
            "loss= 0.02045532502233982\n",
            "acc= 1.0\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAARPUlEQVR4nO3dcWikeX3H8fc3O0mdO4uRXpBmT7on9NKRA10JknpFWk8breLuH/3j7CrUUrYUq6fIiskfPSj0tiUiCi0Ly6ltcattz7gVsUZBRYTmMHs5u3rTgD31vOT0xj+ynpJ2MubbP5L1drfZy2ydyfPbzPsFy2Zmnsx8eHaeD8/+5vfMLzITSVK5hqoOIEl6bha1JBXOopakwlnUklQ4i1qSClfrx5PedttteeTIkX48tSQdSBcuXPhRZo7t9lhfivrIkSMsLS3146kl6UCKiO9d7zGHPiSpcBa1JBXOopakwlnUklQ4i1qSCtfVrI+IeA/wx0ACF4G3Z+Z/9zOYdNn55VXmFlZYW99gfLTOqekJjh89XHUsad/seUYdEYeBdwGTmXkXcAi4t9/BJNgu6Zn5i7zo0jf400P/yosufYOZ+YucX16tOpq0b7qdR10D6hGxCdwCrPUvkvSsuYUVGp0m50YeYJgOm9Q40Z5lbmHEs2oNjD3PqDNzFfgA8ATwFHApM79w7XYRcTIiliJiqdVq9T6pBtLa+gZTQ02G6VCLLYbpMDXUZG19o+po0r7pZujjhcAx4A5gHLg1It567XaZeTYzJzNzcmxs16sgpRs2PlpncavBJjU6OcQmNRa3GoyP1quOJu2bboY+Xgt8JzNbABExD7wK+Hg/g0kAp6YnmJlvc6I9y9RQk8WtBs1ag9PTE1VHk/ZNN0X9BDAVEbcAG8A9gF/koX1xeRx6bmGEM+t3Mj5a57SzPjRg9izqzHw4Ih4CHgE6wDJwtt/BpMuOHz1sMWugdTXrIzPvB+7vcxZJ0i68MlGSCmdRS1LhLGpJKpxFLUmFs6glqXAWtSQVzqKWpMJZ1JJUOItakgpnUUtS4SxqSSqcRS1JhbOoJalwFrUkFc6ilqTCWdSSVLg9Fw6IiAngn6646yXAn2fmh/qWSpJuIueXV5lbWGFtfYPx0TqnerxcXDdLca0ALweIiEPAKvDpniWQpJvY+eVVZuYv0ug0OXaoyeKlBjPzbYCelXVXS3Fd4R7gvzLzez15dUm6yc0trNDoNDk38gDDdNikxon2LHMLIz0r6hsdo74X+MRuD0TEyYhYioilVqv1iyeTpJvA2voGU0NNhulQiy2G6TA11GRtfaNnr9F1UUfECPBm4F92ezwzz2bmZGZOjo2N9SqfJBVtfLTO4laDTWp0cohNaixuNRgfrffsNW5k6OMNwCOZ+cOevbok3eROTU8wM9/mRHuWqaEmi1sNmrUGp6cnevYaN1LUb+E6wx6SNKguj0PPLYxwZv1OxkfrnN7vWR8AEXEr8DrgT3r2ypJ0QBw/erinxXytroo6M38K/ErfUkiSrssrEyWpcBa1JBXOopakwlnUklQ4i1qSCmdRS1LhLGpJKpxFLUmFs6glqXAWtSQVzqKWpMJZ1JJUOItakgpnUUtS4SxqSSqcRS1Jhet2hZdR4EHgLiCBP8rMf+9nMKk055dXmVtYYW19g/HROqd6vNySdD3drpn4YeDzmfn7O6uR39LHTFJxzi+vMjN/kUanybFDTRYvNZiZbwNY1uq7PYs6Il4AvBr4Q4DMbAPt/saSyjK3sEKj0+TcyAMM02GTGifas8wtjFjU6rtuxqjvAFrAxyJiOSIe3Fns9ioRcTIiliJiqdVq9TyoVKW19Q2mhpoM06EWWwzTYWqoydr6RtXRNAC6Keoa8ArgTGYeBX4KvP/ajTLzbGZOZubk2NhYj2NK1RofrbO41WCTGp0cYpMai1sNxkfrVUfTAOhmjPpJ4MnMfHjn9kPsUtTSQXZqeoKZ+TYn2rNMDTVZ3GrQrDU4PT1RdTQNgD2LOjN/EBHfj4iJzFwB7gEe6380qRyXx6HnFkY4s34n46N1TjvrQ/uk21kf7wTO7cz4eBx4e/8iSWU6fvSwxaxKdFXUmfkoMNnnLJKkXXhloiQVzqKWpMJZ1JJUOItakgpnUUtS4SxqSSqcRS1JhbOoJalwFrUkFc6ilqTCWdSSVDiLWpIKZ1FLUuEsakkqnEUtSYXr6vuoI+K7wDPAz4BOZvrd1H10fnmVuYUV1tY3GB+tc6qilURKyaEy+f7YP92u8ALwO5n5o74lEbD95p+Zv0ij0+TYoSaLlxrMzLcB9vUgKCWHyuT7Y3/dSFFrH8wtrNDoNDk38gDDdNikxon2LHMLI/t6AJSSQ2Xy/bG/uh2jTuALEXEhIk7utkFEnIyIpYhYarVavUs4YNbWN5gaajJMh1psMUyHqaEma+sbA5lDZfL9sb+6LerfysxXAG8A3hERr752g8w8m5mTmTk5NjbW05CDZHy0zuJWg01qdHKITWosbjUYH60PZA6VyffH/up2cdvVnb+fjohPA68EvtrPYIPq1PQEM/NtTrRnmRpqsrjVoFlrcHp6YiBzqEy+P/bXnkUdEbcCQ5n5zM7Pvwv8Rd+TDajL43tzCyOcWb+T8dE6pyv4NL2UHCqT74/9FZn53BtEvAT49M7NGvCPmfmXz/U7k5OTubS01JuEkjQAIuLC9aY+73lGnZmPAy/reSpJUle8MlGSCmdRS1LhLGpJKpxFLUmFs6glqXAWtSQVzqKWpMJZ1JJUOItakgpnUUtS4SxqSSqcRS1JhbOoJalwFrUkFc6ilqTCWdSSVLiuizoiDkXEckR8tp+BJElXu5Ez6vuAZr+CSJJ211VRR8TtwBuBB/sbR5J0rW7PqD8EvA/Yut4GEXEyIpYiYqnVavUknCSpi6KOiDcBT2fmhefaLjPPZuZkZk6OjY31LKAkDbpuzqjvBt4cEd8FPgm8JiI+3tdUkqSf27OoM3MmM2/PzCPAvcCXMvOtfU8mSQKcRy1JxavdyMaZ+RXgK31JIknalWfUklQ4i1qSCmdRS1LhLGpJKpxFLUmFs6glqXAWtSQVzqKWpMJZ1JJUOItakgpnUUtS4SxqSSqcRS1JhbOoJalwFrUkFc6ilqTC7blwQEQ8D/gq8Es72z+Umff3O5ik3Z1fXmVuYYW19Q3GR+ucmp7g+NHDVcdSH3Wzwsv/AK/JzJ9ExDDwtYj4t8xc7HM2Sdc4v7zKzPxFGp0mxw41WbzUYGa+DWBZH2B7FnVmJvCTnZvDO3+yn6Ek7W5uYYVGp8m5kQcYpsMmNU60Z5lbGLGoD7Cuxqgj4lBEPAo8DXwxMx/eZZuTEbEUEUutVqvXOSUBa+sbTA01GaZDLbYYpsPUUJO19Y2qo6mPuirqzPxZZr4cuB14ZUTctcs2ZzNzMjMnx8bGep1TEjA+Wmdxq8EmNTo5xCY1FrcajI/Wq46mPrrRVcjXI+LLwOuBb/YnkqTrOTU9wcx8mxPtWaaGmixuNWjWGpyenqg6mvqom1kfY8DmTknXgdcBf933ZJL+j8vj0HMLI5xZv5Px0TqnnfVx4HVzRv2rwN9HxCG2h0r+OTM/299Ykq7n+NHDFvOA6WbWx38AR/chiyRpF16ZKEmFs6glqXAWtSQVzqKWpMJZ1JJUOItakgpnUUtS4SxqSSqcRS1JhbOoJalwFrUkFc6ilqTCWdSSVDiLWpIKZ1FLUuH2LOqIeHFEfDkiHouIb0XEffsRTJK0rZsVXjrAezPzkYj4ZeBCRHwxMx/rczZJEl2cUWfmU5n5yM7PzwBNwHWAJGmf3NAYdUQcYXtZrod3eexkRCxFxFKr1epNOklS90UdEc8HPgW8OzN/fO3jmXk2Myczc3JsbKyXGSVpoHVV1BExzHZJn8vM+f5GkiRdqZtZHwF8BGhm5gf7H0mSdKVuzqjvBt4GvCYiHt3583t9ziVJ2rHn9LzM/BoQ+5BFkrQLr0yUpMJZ1JJUOItakgpnUUtS4SxqSSqcRS1JhbOoJalwFrUkFc6ilqTCWdSSVDiLWpIKZ1FLUuEsakkqnEUtSYWzqCWpcBa1JBVuz4UDIuKjwJuApzPzrv5Hqtb55VXmFlZYW99gfLTOqekJjh89XHUsSbsYlON1z6IG/g74G+Af+huleueXV5mZv0ij0+TYoSaLlxrMzLcBDuQ/vnQzG6TjtZuluL4aEUf6H6V6cwsrNDpNzo08wDAdNqlxoj3L3MLIgfuHl252g3S89myMOiJORsRSRCy1Wq1ePe2+WlvfYGqoyTAdarHFMB2mhpqsrW9UHU3SNQbpeO1ZUWfm2cyczMzJsbGxXj3tvhofrbO41WCTGp0cYpMai1sNxkfrVUeTdI1BOl67GaMeGKemJ5iZb3OiPcvUUJPFrQbNWoPT0xNVR5N0jUE6Xi3qK1we15pbGOHM+p2Mj9Y5fUA/RZZudoN0vEZmPvcGEZ8Afhu4DfghcH9mfuS5fmdycjKXlpZ6lVGSDryIuJCZk7s91s2sj7f0PpIkqVtemShJhbOoJalwFrUkFc6ilqTCWdSSVDiLWpIKZ1FLUuEsakkqnEUtSYWzqCWpcBa1JBXOopakwlnUklQ4i1qSCmdRS1LhLGpJKlxXS3FFxOuBDwOHgAcz8696HeT88ipzCyusrW8wPlrn1AFdUkeSbtSeRR0Rh4C/BV4HPAl8PSI+k5mP9SrE+eVVZuYv0ug0OXaoyeKlBjPzbQDLWtLA6+aM+pXAtzPzcYCI+CRwDOhZUc8trNDoNDk38gDDdNikxon2LHMLIxa1pIHXzRj1YeD7V9x+cue+q0TEyYhYioilVqt1QyHW1jeYGmoyTIdabDFMh6mhJmvrGzf0PJJ0EPXsw8TMPJuZk5k5OTY2dkO/Oz5aZ3GrwSY1OjnEJjUWtxqMj9Z7FU+SblrdDH2sAi++4vbtO/f1zKnpCWbm25xozzI11GRxq0Gz1uD09EQvX0aSbkrdFPXXgV+PiDvYLuh7gT/oZYjL49BzCyOcWb+T8dE6p531IUlAF0WdmZ2I+DNgge3peR/NzG/1Osjxo4ctZknaRVfzqDPzc8Dn+pxFkrQLr0yUpMJZ1JJUOItakgpnUUtS4SIze/+kES3ge//PX78N+FEP49zM3BdXc39czf3xrIOwL34tM3e9WrAvRf2LiIilzJysOkcJ3BdXc39czf3xrIO+Lxz6kKTCWdSSVLgSi/ps1QEK4r64mvvjau6PZx3ofVHcGLUk6WolnlFLkq5gUUtS4Yop6oh4fUSsRMS3I+L9VeepUkS8OCK+HBGPRcS3IuK+qjNVLSIORcRyRHy26ixVi4jRiHgoIv4zIpoR8ZtVZ6pSRLxn5zj5ZkR8IiKeV3WmXiuiqK9YQPcNwEuBt0TES6tNVakO8N7MfCkwBbxjwPcHwH1As+oQhfgw8PnM/A3gZQzwfomIw8C7gMnMvIvtr2K+t9pUvVdEUXPFArqZ2QYuL6A7kDLzqcx8ZOfnZ9g+EAf2y7oj4nbgjcCDVWepWkS8AHg18BGAzGxn5nq1qSpXA+oRUQNuAdYqztNzpRR1VwvoDqKIOAIcBR6uNkmlPgS8D9iqOkgB7gBawMd2hoIejIhbqw5VlcxcBT4APAE8BVzKzC9Um6r3Silq7SIing98Cnh3Zv646jxViIg3AU9n5oWqsxSiBrwCOJOZR4GfAgP7mU5EvJDt/33fAYwDt0bEW6tN1XulFHXfF9C92UTEMNslfS4z56vOU6G7gTdHxHfZHhJ7TUR8vNpIlXoSeDIzL/8P6yG2i3tQvRb4Tma2MnMTmAdeVXGmniulqH++gG5EjLD9YcBnKs5UmYgItscgm5n5warzVCkzZzLz9sw8wvb74kuZeeDOmLqVmT8Avh8REzt33QM8VmGkqj0BTEXELTvHzT0cwA9Xu1ozsd/2awHdm8jdwNuAixHx6M59sztrV0rvBM7tnNQ8Dry94jyVycyHI+Ih4BG2Z0stcwAvJ/cSckkqXClDH5Kk67CoJalwFrUkFc6ilqTCWdSSVDiLWpIKZ1FLUuH+FzLzmJjeDOPXAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M1sV7tBtAWEz"
      },
      "source": [
        "## GRU"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 819
        },
        "id": "usltj8hI3bdL",
        "outputId": "c3e360ef-66ec-4c1a-8993-d44600315d6c"
      },
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras.layers import Dense, RepeatVector, Flatten\n",
        "from tensorflow.keras.layers import Bidirectional, LSTM, GRU\n",
        "\n",
        "input_shape = train_x.shape[1:]\n",
        "# input_shape = (5,10)\n",
        "\n",
        "model = keras.Sequential()\n",
        "model.add(GRU(128, input_shape=input_shape))\n",
        "model.add(RepeatVector(1))\n",
        "model.add(Dense(CATEGORY_NUM, activation=\"softmax\"))\n",
        "model.compile(loss=\"categorical_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n",
        "model.summary()\n",
        "\n",
        "history = model.fit(train_x, train_y, epochs=100, verbose=1, validation_split=0.1, batch_size=128)\n",
        "\n",
        "\n",
        "plt.plot(history.history['loss'])\n",
        "plt.plot(history.history['val_loss'])\n",
        "plt.show()\n",
        "\n",
        "\n",
        "\n",
        "loss, acc = model.evaluate(test_x, test_y)\n",
        "print(\"loss=\", loss)\n",
        "print(\"acc=\", acc)\n",
        "\n",
        "\n",
        "\n",
        "y_ = model.predict(test_x)\n",
        "predicted = np.argmax(y_, axis=-1)\n",
        "test_y_index = np.argmax(test_y, axis=-1)\n",
        "\n",
        "plt.plot(test_y_index, \"o\")\n",
        "plt.plot(predicted, \".\")\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "gru (GRU)                    (None, 128)               53760     \n",
            "_________________________________________________________________\n",
            "repeat_vector_1 (RepeatVecto (None, 1, 128)            0         \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 1, 10)             1290      \n",
            "=================================================================\n",
            "Total params: 55,050\n",
            "Trainable params: 55,050\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhU5f3+8fdnliwQCEsChM2AIogWFVNFkSpoVXBBXGmtWqvi0ha1trW2dW+xbpVFRBBRQIuK4I6iKK6AEkSQfd+3AIFAyDLJPL8/MvrND0ECTHIyM/fruubKzJyTnPtw4GZy5szzmHMOERGJfT6vA4iISHSo0EVE4oQKXUQkTqjQRUTihApdRCROBLzacEZGhsvOzvZq8yIiMWnWrFlbnXOZ+1rmWaFnZ2eTm5vr1eZFRGKSma3e3zKdchERiRMqdBGROKFCFxGJEyp0EZE4oUIXEYkTKnQRkTihQhcRiRMxV+hbdhXz73cXsLmg2OsoIiK1SswV+uwFi8mYMYBLH5nA3RO/Y822PV5HEhGpFTz7pOihOrfOUlzgXa7nfSZ+243rci/gZ506c8uZR9G+WT2v44mIeMa8mrEoJyfHHfJH/7evgGlP4Wa/BOUlTHE5DC29kMxjunLrmUdyYuuG0Q0rIlJLmNks51zOPpfFZKF/b3cefPUM4Zkj8RXvIJdjGFp6ASXZZ/H7Hu047cjGmFl0AouI1ALxW+jfK9kF34whPO0pfLs2sIzWPF3ai1VZPbmpRwd+eUxTfD4Vu4jEvvgv9O+VlcK8CYS/HIgvbxGbLYMRpefxdaMLua77cVx4fHOC/ph7H1hE5AeJU+jfcw6WfoD7YiC2Zhq7LI3nQ2fzQdrFXHlmZy4/qSUpQX/1bFtEpBolXqFXtnYm7suBsOhdSgnyStkZTEjpQ69uXbiqyxGkJcfchT4iksASu9C/t3Up7stBuDkv48LlvFPehRf9fTit65lc1zWbBnWSai6LiMghUqFXVrABZjxN+cxR+EOFTC0/nlHWh45dzuP6bm1oUi+l5jOJiFSRCn1fivJh5kjKpg0jULyN3PDRDHd9aJFzEf3OOJLmDVK9yyYish8q9J9Sugdmj6Xsi0EEdq1nQfgIhoUvpt6Jl3BL96Np1aiO1wlFRH6gQq+K8hDMfYXQp/8luGM5y11zhpVfTOD4y7ilRweOaFzX64QiIir0gxIuhwVvEvrkUYJbF7LaNeXp8t7QqS+/P+sYWjfWK3YR8Y4K/VCEw7DkPUJTHyG4eQ5rXROGll+MHf8rbj2rg07FiIgnVOiHI/IhpdBHAwhu/jZS7H0Idq4o9qx0vXkqIjVHhR4NkWIvnfIvkrbMZZVrxpDwZTQ8uS+3dD+axmnJXicUkQSgQo8m52DxpIpi37qAxeFWDKEvbbtexo1nHEm9lKDXCUUkjqnQq0M4DAtep/TDh0jauZJvwkfxdOAaTutxEVd1aU1yQGPFiEj0/VSha+jBQ+XzwXGXktR/Jlw4iOPSdjMyfC+tJv+OGx4by1tzNuDVf5YikphU6IfLH4STfkvS7bNxPe6le8oSXii5g13j/8C1T01i5qrtXicUkQShUy7Rtmc74U/+AzOfo8gFGRLqzcYO1/Hn8zvpUkcROWw6h+6FrUspn3wP/qXvsdo1ZUD4Gtp1vYxbexxFnSQN2Ssih+awzqGbWSszm2pmC8xsvpndto91zMwGm9kyM5trZp2jETymZbTDf9XLcPXrNG9cn+H+x8j5sh+/fWwck77bqPPrIhJ1VTmHXgbc6ZzrCHQBfm9mHfdapyfQLnLrBwyLaspYdmQPgr+fDuc+zOkpK3gpdDtLX/k7vxv5OSvydnudTkTiyAEL3Tm30Tn3TeT+LmAh0GKv1XoDY1yFGUADM8uKetpY5Q/CqbcS6J+L/7iLuS0wkQfX38CDg4by5IdLKA6Ve51QROLAQV3lYmbZwInAV3stagGsrfR4HT8ufcysn5nlmlluXl7ewSWNB/Wa4bvsObjmTbIapvFCYACtP/sTfQe+y/Tl27xOJyIxrsqFbmZpwATgdudcwaFszDk3wjmX45zLyczMPJQfER/ankng1mnQ7c/0CUzn+cI/8OJzA/n769+xqzjkdToRiVFVKnQzC1JR5i855ybuY5X1QKtKj1tGnpP9CabAWffgu/kz0rOOZGjSYLp+cydX/Pdtpi7e4nU6EYlBVbnKxYDngIXOuf/uZ7W3gGsiV7t0AXY65zZGMWf8anosvhs+hLPupWfwG/4Xup1xo5/m7olz2V1S5nU6EYkhVXmF3hW4GuhhZt9Gbr3M7GYzuzmyziRgBbAMeBa4tXrixil/ALrdie+mz0hvcgQjkp7khNn30OfJyXy1QufWRaRq9MGi2qasFD4ZgPtiIOutKX8ouZVuZ55H/7PaEfRrpAaRRKfBuWJJIAnOvh/77bs0rxdgQtID7Pl0EJcPm8aabXu8TicitZgKvbbK7orvli/wd+jJPcGXuH3rPfx68CTen6e3JkRk31TotVlqQ7jyRTjvEc7wfccE/90MeWkCD7w9n9KysNfpRKSWUaHXdmbQ5Wbs+sk0SQvyZsoD7J7xAlcMn86mncVepxORWkSFHitanITd9BmB7C48FhzBlVuepPfgTzTeuoj8QIUeS+pmwG9eh9P68yv7kKfdv7h5xBTGzljtdTIRqQVU6LHGH4BzHoKLh9HZFjOpzn288OZk7n9rPmXlOq8ukshU6LHqhF9j175Dk+QQ79R5kAXT3+OGMbkaC0YkganQY1nrU7AbPiK1YRbjUv5DveVvc/kzerNUJFGp0GNdwyPgd5Pxt8phSGAwPfLHc8nTX7Jsyy6vk4lIDVOhx4M6jeDqN6Bjb/5qY7ih9EUufXoas1brChiRRKJCjxfBFLjseeh8Lb9zE3kg+Dy/GTmdT5ck4EQiIglK08/HE58fLhwEKelcPG0w9VOLuXm0Y+Cvczj32GZepxORaqZX6PHGDH75IPS4hx6lUxmeNoI/vjSTN2ZrvhGReKdX6PHIDH7xZ/D5+cWU+3m+Plz76g04HH1ObOl1OhGpJir0eHb6HYDRdcp9jEmvKHW/z8dFxzf3OpmIVAMVerw7/XYATptyHyMaJHHjKz78ZpzfKcvjYCISbSr0RHD67RAqovun/2FQeh36v2ykJuXQo0NTr5OJSBSp0BPFmX+DkgLOn/E0u9JTueVFHy/ecAo/z27kdTIRiRIVeqIwg3MHQMku+s4ey+Y69fndCz5evelUjsmq73U6EYkCXbaYSMwqrlNv34v+pc/SKzCLq5/7mrXbNVepSDxQoScanx8ufQ5rcRIPu4EcU7aQ616Yyc49GqVRJNap0BNRUh349Sv40lswKvkJ3Lbl3PziLM1TKhLjVOiJqm4GXPUaQR+83nAI81as5W8T5+Kc8zqZiBwiFXoia3wkXDGW+nvW8HbWKN74Zi1Pf7Lc61QicohU6ImuTTfo9TjZ+dMYmfUmj3+wmI8XbfY6lYgcAhW6QM51cPJN9Mgfzx8azeS2cd+yPG+316lE5CCp0KXCuQMguxt/Kh5GR/8abhyTS4HmJxWJKSp0qeAPwGWjsDoNGV1nMPnbtvDX8XqTVCSWqNDl/6Q1gctHk7JnA29kjWHy/A2M+nKV16lEpIpU6PL/a30KnDuAI7Z9zuPNP+HhSQv5Zk2+16lEpApU6PJjJ/eDjr25ZMcLnFVvDX946Ru2F5Z6nUpEDkCFLj9mBhcOxuo1Z0jwKUp253PXBJ1PF6ntVOiyb6kN4LLnSCrcwPgWr/Dhgk387+s1XqcSkZ9wwEI3s1FmtsXM5u1n+ZlmttPMvo3c7o1+TPFEq5Ohxz9ou+UD/pE1i4feWcCyLbo+XaS2qsor9BeA8w6wzufOuRMitwcPP5bUGl3vgOxuXL97OEcGtnLby7MpKSv3OpWI7MMBC9059xmwvQaySG3k88HFT+MzH2Mbj2bBhh0MnLLU61Qisg/ROod+qpnNMbP3zOzY/a1kZv3MLNfMcvPy8qK0aal2DVpDz0dotHUmQ7KnM/zT5czWpYwitU40Cv0b4Ajn3PHAEOCN/a3onBvhnMtxzuVkZmZGYdNSY074NbQ/n/PznuW0tM38efwcikM69SJSmxx2oTvnCpxzuyP3JwFBM8s47GRSu0Smr7Pkegyr/zyr8gp48sMlXqcSkUoOu9DNrJmZWeT+yZGfue1wf67UQmmZ0PNR6m2by5A2Mxjx+QpmrdbbKyK1RVUuWxwHTAfam9k6M7vezG42s5sjq1wGzDOzOcBgoK/TJ1Di13GXQvte9Mx7jp/X28FdE77TVS8itYR51b05OTkuNzfXk23LYSrYAENPIT+9A53X9Of2sztw29ntvE4lkhDMbJZzLmdfy/RJUTl49ZvDOf+i4ZavefiI2QyduoxlW3Z5nUok4anQ5dB0vgayu3HFjpG0CO7i7onfEQ7rTJuIl1TocmjM4Pz/4gvt4fkWbzJzVT4vz1zrdSqRhKZCl0OXeTScfjvZ69/h+hZreOT9RWzbXeJ1KpGEpUKXw9PtTmiYzV3lzxIqKeLR9xd7nUgkYanQ5fAEU6HXEyTtWM6wNl/wSu5azXAk4hEVuhy+dmdDx978YvNYOtUr4J435lGuN0hFapwKXaLjnH9hwDNN3mD+hgL+99VqrxOJJBwVukRHg9Zw+h00X/8+N7Rcy+MfLCFf85CK1CgVukRP1/7QoDV/KX+OPcXFDJyiwbtEapIKXaInmArnDiA5fwlPts3lxa/WsGSzPkEqUlNU6BJdHS6Att3ptW00WUlFPPTOAjRWm0jNUKFLdJnBuf/GV1LAM62m8PnSrXy8aIvXqUQSggpdoq/psdD5Go5d/ypnNM7n3+8uJFQe9jqVSNxToUv16P4PLJDK4+kTWbG1kHFfr/E6kUjcU6FL9UhrAt3+ROaGj7ihxVoGTllKQXHI61QicU2FLtWny63QoDV3Mob8wmKe+WS514lE4poKXapPMAXOuo/UbfN5qO1CnvtiJet3FHmdSiRuqdCleh17CWSdQN9dLxCklCcmazRGkeqiQpfq5fPBOQ8R2LWewW1n8vq361mwocDrVCJxSYUu1a/NL6DdOXTfPIaWycU8OnmR14lE4pIKXWrG2Q9gpbt4qtVUPlmcx7TlW71OJBJ3VOhSM5p2hON/TacNr3BC/V088v5iDQkgEmUqdKk53e/GMJ5s+h5z1u7g/XmbvE4kEldU6FJz0lvCyTeSve4tzs7YzqOTF2tIAJEoUqFLzep2J5aUxoD6r7NyayGvzVrndSKRuKFCl5pVpxF07U+TDR/RN2sjg6YspThU7nUqkbigQpea1+VWqNuEvwdfYVNBEWOna/5RkWhQoUvNS6oLZ/yV+lu+5taWq3j6k2Xs0sBdIodNhS7e6HwtNDiCP7px7NhTwrOfr/Q6kUjMU6GLNwJJ0P3vpG6bxz+ylzDy8xVs3V3idSqRmKZCF+/87HLIPIZril8kFCplmIbXFTksKnTxjs8PZ91D0o4VDGjzHWNnrGaDhtcVOWQqdPFW+17Q8uf0KXiRJFfKkI+Xep1IJGYdsNDNbJSZbTGzeftZbmY22MyWmdlcM+sc/ZgSt8zgrHsJ7N7A49m5vJq7jhV5u71OJRKTqvIK/QXgvJ9Y3hNoF7n1A4YdfixJKG1+AW3P5JztL9HAX8KTU/QqXeRQHLDQnXOfAdt/YpXewBhXYQbQwMyyohVQEsRZ9+Ir2sagI77k7TkbmL9hp9eJRGJONM6htwDWVnq8LvKcSNW1OAk6XEDXLeNolVLEEx8s8TqRSMyp0TdFzayfmeWaWW5eXl5NblpiQY9/YqWFDG71CR8v2kLuqp/6xVBE9haNQl8PtKr0uGXkuR9xzo1wzuU453IyMzOjsGmJK02OgeP7csLG8RxTdzePTtYkGCIHIxqF/hZwTeRqly7ATufcxij8XElEZ/4NC5czKGsyX6/czmdLNVWdSFVV5bLFccB0oL2ZrTOz683sZjO7ObLKJGAFsAx4Fri12tJK/GuYDTnX0W79G3RJz+exyYsIh/UqXaQqAgdawTn3qwMsd8Dvo5ZI5Bd/wWa/yGON36Hbiqt5b94mzu+kC6dEDkSfFJXaJ60JdLmFVhveo2dGHo9/oKnqRKpChS6102n9IaUBD6VNYOXWQl7NXXvg7xFJcCp0qZ1SG8Dpd5Cx6TOuzlrHoClLKSrVVHUiP0WFLrXXKTdBvebcFRjHll3FPD9Nk2CI/BQVutRewVTofjdpebP5c6ulDPtkOTv2lHqdSqTWUqFL7Xb8ryGjPTeGxlJUUsLQqcu8TiRSa6nQpXbzB+Ds+0jesZyHs+cyetpq1m7f43UqkVpJhS61X/te0OoULikYQ6qV8MQHi71OJFIrqdCl9jODsx/AX7iZIW2m88a3G5i3XsPriuxNhS6x4YhTocMFdNs8liNTCxkwaaEG7hLZiwpdYsfZD2BlJTzV4gOmLd/GJ4s1BLNIZSp0iR0ZR0HO7+iwfiJnNNrOv95doCEBRCpRoUtsOeMuLKkuj6VPYHleIeO+XuN1IpFaQ4UusaVuBnT7E002TuX6Fmt58sMl7NwT8jqVSK2gQpfYc8otkN6aPzOagqIShny81OtEIrWCCl1iTzAFznmQ1G0LeLTNHEZPX8XKrYVepxLxnApdYlPHi6H1afTJH0XjQAkPvbPA60QinlOhS2wyg/Mexle0nRGtP+LjRVv4aOFmr1OJeEqFLrGr+Qlw4lX8bP3LnNF4Bw++s4DikMZMl8SlQpfY1uNeLJDCk/VfZvW2Qp77QmOmS+JSoUtsq9cUuv+dRhs/467s5Qz5eCnrdxR5nUrEEyp0iX0n94Mmx3Jj4XBSKOHBt+d7nUjEEyp0iX3+AJz/OIFd6xnZ5jMmz9/Mhwv0BqkkHhW6xIcjToNOfTlp/Vh6ZBRw35vzKCwp8zqVSI1SoUv8+OWDWCCVgfXGsGFnEQOnLPE6kUiNUqFL/KjXFH55P/U3TuOxo+Yx6stVmghDEooKXeJL599C61O5dOsztE3dw10T5mqIXUkYKnSJLz4fXDgIX+luRjefyPwNBQz/dLnXqURqhApd4k9me+h2J83Xvstf265m8EfLWLJ5l9epRKqdCl3iU7c/QUZ7bioYTLPkEv4yfg5lOvUicU6FLvEpkAx9huEv3MKLLd9gzrqdjPh8hdepRKqVCl3iV4uT4PQ7aL3mdf7aZiVPfrhEV71IXFOhS3w746/Q5FhuLhhEdp0Sbnt5NkWlGpFR4pMKXeJbIBn6PIOvaDv/y3qV5Xm7efi9hV6nEqkWVSp0MzvPzBab2TIz+9s+lv/WzPLM7NvI7YboRxU5RFmdoPvfyVwzicEd5jNm+mqmLtridSqRqDtgoZuZHxgK9AQ6Ar8ys477WPUV59wJkdvIKOcUOTxdb4fsbly4fiBnZ+7kzvFz2LhTw+xKfKnKK/STgWXOuRXOuVLgZaB39cYSiTKfHy4ZgQVSGJo8lHComD/+b7Y+RSpxpSqF3gJYW+nxushze7vUzOaa2Wtm1ioq6USiqX5z6P0UyVvnMbHdB+SuzufxyYu9TiUSNdF6U/RtINs51wn4EBi9r5XMrJ+Z5ZpZbl5eXpQ2LXIQOpwPJ99E2+Vj+E+H5Qz/bIXGTpe4UZVCXw9UfsXdMvLcD5xz25xzJZGHI4GT9vWDnHMjnHM5zrmczMzMQ8krcvjO+Re0/DlXbniEns0K+NMr37Jsy26vU4kctqoU+kygnZm1MbMkoC/wVuUVzCyr0sOLAF0XJrVXIAkuH40FUhjse4IGgRL6jcllZ1HI62Qih+WAhe6cKwP+AEymoqhfdc7NN7MHzeyiyGr9zWy+mc0B+gO/ra7AIlGR3gIuG0UwfzlvthrH2vxC+o+bTXnYeZ1M5JCZc978Bc7JyXG5ubmebFvkB18Ohg/v4bujbuHCed244fQ2/POCfV2VK1I7mNks51zOvpYFajqMSK1y2h8hbzE/+3YYj3doxp+/gNaN63DNqdleJxM5aProvyQ2M7jgSWh9GpeuHcCNbbZz31vzmTx/k9fJRA6aCl0kkARXvojVa8bfd97Pec0qzqfPWp3vdTKRg6JCFwGo2xh+MwHD8VT4ITrW28P1o2eyeJNmOpLYoUIX+V5GO7hqPP4923il7hM08hfxm+e+YuXWQq+TiVSJCl2kshYnwZVjSdq+hHczhhIsL+KqZ2ewLn+P18lEDkiFLrK3o86CS0aQumkmHzYdSnnJbq4a+RXrd2h0RqndVOgi+3LcJdBnBHU3fcWUrGHsKdzFFc9MZ802vVKX2kuFLrI/nS6HPsOpt+krpmYNw5Xs4orh01mep3FfpHZSoYv8lE5XQJ/hpG36io8ynqBu+Q6uHD5dk01LraRCFzmQTldA35dIzV/Me/X+TWv/dq4cPp1Pl2gIaKldVOgiVdG+J1z9OklFWxkfvI/u6Zu4/oWZjM9de+DvFakhKnSRqjriNLhuEn6fnyHFd/PHZgv4y2tz+c97izRKo9QKKnSRg9HsOOg3FWt6HLdtf4iR2R8z/NOlXD96psZTF8+p0EUOVloTuPZt6NSXszeN5ItWI5i/bCUXD/2ShRsLvE4nCUyFLnIoginQ5xno9Tgtts/giwb307ZoPr2HfsmLM1bj1TwDkthU6CKHygxOvhGu/4DkYJCR4Xt4pNG73P/Gt/z+f9+wY0+p1wklwajQRQ5X8xPhps+wn11Gn4KxTM8YwMoFuZzz5Gd8tHCz1+kkgajQRaIhtQFcMgKuGEtmeCvvJv+T/r7XuHX0NO58dY5erUuNUKGLRFPHi+DWGfg6XshvSsYxo8E95M15jx5PfMr43LWEdXmjVCMVuki0pWXCZaPg6tdpWCfImODDPB34L0MnTObKERo2QKqPCl2kuhzZA26ZDt3/ySnhOXyUche9Nz3F1U+9x59e/ZYNGo5XokyFLlKdgilwxl+w/rPxn3gVV9l7zEi9g7bzBnHR4+8yYNJCtu4u8TqlxAnz6nrZnJwcl5ub68m2RTyzZSF88jAseJMiXxrPlp7DOOvJRaf9jBu7tSUjLdnrhFLLmdks51zOPpep0EU8sHEufPoILHqHUkvmpdCZjOV8Tj2pMzd2a0t2Rl2vE0otpUIXqa22LIJpg3FzX8GFw3wc7szzZedQp30Prj41m9OPysDnM69TSi2iQhep7Xauh9xRhHOfx1e0jVU0Z1zoDGbWP4dzuxxPnxNb0KR+itcppRZQoYvEilAxzJ9IeNZofGtnUI6Pj8tP4J3waRS3PYfzc9pxVocm1E0OeJ1UPKJCF4lFeUtg9ljK5rxKoHATRSQzpfxEptrJuCPPpvsJ7Tjj6EzSU4NeJ5UapEIXiWXhMKyZTvi78ZTPf4tg8TZCBJhR3oEv3PFszzqdo449mdOPzuSYZvV1zj3OqdBF4kW4HNblEl74NiUL3yd1x1IAtrgGfB3uwLzAsZS36kLzdp3JaZPJMVn1CPj1cZN4okIXiVc718HyjylaMhW36kvqFFeM7ljokvnOtWU+R7G7QQeCLTuR1fZntG/emCOb1CU54Pc4uBwqFbpIInAOdqyGNTPYvfJrQqtnUm/HQgKuYmq8EhdgpctihWvOttQ2lDVsQ1LmUaS3aE9WVnNaN04jIy0JM52yqc1U6CKJqjwEW5dSvnEuO1fNpnTTIpLzl5Fesh4f//dvv9Als95lsMkyKUhqSkmdZrh6zQmkZ5HaMIu0jBakN25GZnpdGtdN0mkcDx12oZvZecAgwA+MdM79Z6/lycAY4CRgG3Clc27VT/1MFbqIh0LFsGMNoa3L2LFuMcV5KwnnryFp9zrqluRRP7zjR98SdsZO6rLd1aPAV589gXRKAumUJaUTTk6H1HR8Ken4U9MJ1k0nqW4DkuvUJzUtnZS66dSpm0adlCTqBP164/Yw/FShH/BiVjPzA0OBXwLrgJlm9pZzbkGl1a4H8p1zR5lZX+AR4MrDjy4i1SKYAplHE8w8msxjev14eVkJFGygdOdGCrauZ8+2DZTu3EJ5YR5WuJX0knwyQ9tILV1BneLdpFJcpc0WuSTySaaYZEosmVJLpsySKPMnU2bJlPuTCfuSCfuTCPuTcb4kCCTh/MngD4I/GQsEMX8S5g/ii9z/4as/gAWC+PxB/P4g5g/iDwTw+QP4AkF8/gD+QADzBfD7K573+wP4An78/orlPr8fvz+A3+fHF/DjMx9+ny8m/hOqyqcTTgaWOedWAJjZy0BvoHKh9wbuj9x/DXjKzMxpplyR2BRIhkZtSGrUhow2VVi/rBSKdxIq2smegnyKduVTXLiD0j27KC/aRXlxAeGSQlyoEEr3YGVFWFkRvrJiguXFJIdLCYQLCZSWEHAhgq6EoAsRpIwgIYKUV/su/5RyZ5Tiw+EjjFGOD4cR/v6r2Q/L3A/LIvfNcHs9v/HIKzn1N/dGPWdVCr0FsLbS43XAKftbxzlXZmY7gcbA1miEFJFaLpAEaZkE0zJJz4T0aP/8cBjCIcKhYkKhUspCJYRKSigPlRIKFVNeFqq4hUKUl5cSLg8RLisjXFaKC5cTLi/DlYdw5WW4cJhweQjCZbhw+Q83vv/qKu7jwj98da4cC5cDLvJcOTgq1nHlmAtXLKv0vLmK9c25ivVwP6yXlN4k2n9CQNUKPWrMrB/QD6B169Y1uWkRiWU+H/iS8QWSSU4FDTK8b1V5q3o90KrS45aR5/a5jpkFqPgPetveP8g5N8I5l+Ocy8nMzDy0xCIisk9VKfSZQDsza2NmSUBf4K291nkLuDZy/zLgY50/FxGpWQc85RI5J/4HYDIVly2Ocs7NN7MHgVzn3FvAc8BYM1sGbKei9EVEpAZV6Ry6c24SMGmv5+6tdL8YuDy60URE5GDo414iInFChS4iEidU6CIicUKFLiISJzwbbdHM8oDVh/jtGSTmp1ATcb8TcZ8hMfc7EfcZDn6/j3DO7fODPJ4V+uEws9z9jTYWzxJxvxNxnyEx9zsR9xmiu9865SIiEidU6CIicSJWC32E1wE8koj7nYj7DIm534m4zxDF/Y7Jc+giIvJjsfoKXURE9qJCFxGJEzFX6GZ2npktNrNlZvY3r/NUB54rlCEAAANeSURBVDNrZWZTzWyBmc03s9sizzcysw/NbGnka0Ovs1YHM/Ob2WwzeyfyuI2ZfRU55q9EhnGOG2bWwMxeM7NFZrbQzE5NhGNtZndE/n7PM7NxZpYSj8fazEaZ2RYzm1fpuX0eX6swOLL/c82s88FsK6YKvdKE1T2BjsCvzKyjt6mqRRlwp3OuI9AF+H1kP/8GfOScawd8FHkcj24DFlZ6/AjwpHPuKCCfiknJ48kg4H3nXAfgeCr2Pa6PtZm1APoDOc6546gYmvv7Cebj7Vi/AJy313P7O749gXaRWz9g2MFsKKYKnUoTVjvnSoHvJ6yOK865jc65byL3d1HxD7wFFfs6OrLaaOBibxJWHzNrCZwPjIw8NqAHFZOPQ5ztt5mlA7+gYk4BnHOlzrkdJMCxpmL47tTILGd1gI3E4bF2zn1GxTwRle3v+PYGxrgKM4AGZpZV1W3FWqHva8LqFh5lqRFmlg2cCHwFNHXObYws2gQ09ShWdRoI/BUIRx43BnY458oij+PtmLcB8oDnI6eZRppZXeL8WDvn1gOPA2uoKPKdwCzi+1hXtr/je1gdF2uFnlDMLA2YANzunCuovCwyxV9cXXNqZhcAW5xzs7zOUoMCQGdgmHPuRKCQvU6vxOmxbkjFq9E2QHOgLj8+LZEQonl8Y63QqzJhdVwwsyAVZf6Sc25i5OnN3//6Ffm6xat81aQrcJGZraLidFoPKs4vN4j8Wg7xd8zXAeucc19FHr9GRcHH+7E+G1jpnMtzzoWAiVQc/3g+1pXt7/geVsfFWqFXZcLqmBc5b/wcsNA5999KiypPxn0t8GZNZ6tOzrm7nXMtnXPZVBzbj51zVwFTqZh8HOJsv51zm4C1ZtY+8tRZwALi/FhTcaqli5nVifx9/36/4/ZY72V/x/ct4JrI1S5dgJ2VTs0cmHMupm5AL2AJsBz4h9d5qmkfT6fiV7C5wLeRWy8qzid/BCwFpgCNvM5ajX8GZwLvRO63Bb4GlgHjgWSv80V5X08AciPH+w2gYSIca+ABYBEwDxgLJMfjsQbGUfE+QYiK38iu39/xBYyKK/mWA99RcRVQlbelj/6LiMSJWDvlIiIi+6FCFxGJEyp0EZE4oUIXEYkTKnQRkTihQhcRiRMqdBGROPH/ADOq1B4EwOXXAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 15ms/step - loss: 0.0080 - accuracy: 1.0000\n",
            "loss= 0.008012698031961918\n",
            "acc= 1.0\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAMs0lEQVR4nO3dX2id93nA8e9jy6ZyulaFiICcMPti9RQCw0UMr4Eylg53a6hzsYtsSaEZwzC2Nu26lMo3vWpTcCnNxQgYd4URs124wisj1B1svRjMpnKc4tVnhpKuif+Unl7YLcVDFnp6IbmW1aPoVX2O3qd+v58rWzqRHl6iL8c/nfM+kZlIkura1vYAkqS3Z6glqThDLUnFGWpJKs5QS1JxY6P4og8++GDu2bNnFF9aku5L586d+0lmTg763EhCvWfPHubn50fxpSXpvhQRP1zvcx59SFJxhlqSijPUklScoZak4gy1JBU3kld9SPejU+evcPT0Ja5ev8nUxDgvHNzHU/t3tz2WOsBQSw2cOn+F2bkLTC/2OLS9x5kb08zOLQAYa42coZYaOHr6EtOLPU7s/AI7WOQWYzyzcISjp3caao2cZ9RSA1ev3+TAth47WGQsltjBIge29bh6/Wbbo6kDDLXUwNTEOGeWprnFGIu5jVuMcWZpmqmJ8bZHUwd49CE18MLBfczOLfDMwhEObOtxZmma3tg0Lx7c1/Zo6gBDLTVw+xz66OmdvHz9vUxNjPOir/rQFjHUUkNP7d9tmNUKz6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqzlBLUnGGWpKKM9SSVFyj25xGxKeAvwISuAA8l5n/P8rBpNvc/q2u2/AZdUTsBj4BzGTmY8B24OlRDybBne3fD934Ln+9/V956MZ3mZ27wKnzV9oeTdoyTRcHjAHjEXEL2AVcHd1I0h1u/5YaPKPOzCvAl4A3gWvAjcz81trHRcThiJiPiPl+vz/8SdVJbv+Wmh19vAc4BOwFpoAHIuLZtY/LzGOZOZOZM5OTk8OfVJ3k9m+p2dHHB4EfZGYfICLmgPcDr4xyMAnc/i1Bs1C/CRyIiF3ATeAJYH6kU0kr3P4tNQh1Zp6NiJPAa8AicB44NurBpNvc/q2ua/Sqj8z8HPC5Ec8iSRrAdyZKUnGGWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1JxRlqSSrOUEtScYZakooz1JJUnKGWpOIMtSQV13RnojrI7d9SDYZaA93e/j292OPQ9h5nbkwzO7cAYKylLWaoNZDbv6U6PKPWQG7/luow1BrI7d9SHR59aCC3f0t1GGoN5PZvqQ5DrXW5/VuqwTNqSSrOUEtScYZakooz1JJUnKGWpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBXX6DanETEBHAceAxL4y8z871EO1mVu/5a0WtP7Ub8EfDMz/ywidgK7RjhTp7n9W9JaG4Y6It4NfAD4GEBmLgALox2ru9z+LWmtJmfUe4E+8LWIOB8RxyPigbUPiojDETEfEfP9fn/og3aF278lrdUk1GPA+4CXM3M/8HPgs2sflJnHMnMmM2cmJyeHPGZ3uP1b0lpNzqgvA5cz8+zK308yINQaDrd/S1prw1Bn5o8i4q2I2JeZl4AngIujH62b3P4taa2mr/r4OHBi5RUfbwDPjW4kuf1b0mqNQp2ZrwMzI55FkjSA70yUpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqrultTjvDDeCSqjHUq7gBXFJFhnoVN4BLqsgz6lXcAC6pIkO9ihvAJVXk0ccqbgCXVJGhXsUN4JIqMtRruAFcUjWeUUtScYZakooz1JJUnKGWpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpuMa3OY2I7cA8cCUznxz2IG7/lqTBNnM/6ueBHvCuYQ/h9m9JWl+jUEfEw8CHgc8DfzfsIdz+LUnra3pG/RXgM8DSeg+IiMMRMR8R8/1+f1NDuP1bkta3Yagj4kngx5l57u0el5nHMnMmM2cmJyc3NYTbvyVpfU2OPh4HPhIRfwq8A3hXRLySmc8Oawi3f0vS+jYMdWbOArMAEfGHwN8PM9Lg9m9JejtltpC7/VuSBttUqDPz28C3RzKJJGkg35koScUZakkqzlBLUnGGWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1JxRlqSSrOUEtScYZakooz1JJUXJnbnErSb6pT569w9PQlrl6/ydTEOC8M+X76hlqS7sGp81eYnbvA9GKPQ9t7nLkxzezcAsDQYm2oJekeHD19ienFHid2foEdLHKLMZ5ZOMLR0zuHFmrPqCXpHly9fpMD23rsYJGxWGIHixzY1uPq9ZtD+x6GWpLuwdTEOGeWprnFGIu5jVuMcWZpmqmJ8aF9D48+JOkevHBwH7NzCzyzcIQD23qcWZqmNzbNiwf3De17GGpJuge3z6GPnt7Jy9ffy9TEOC/6qg9JquWp/buHGua1PKOWpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqzlBLUnEb3uY0Ih4B/gl4CEjgWGa+NOrBJGkjo97+XUWT+1EvAp/OzNci4reAcxHx75l5ccSzSdK6tmL7dxUbhjozrwHXVv78s4joAbsBQy2pNVux/buKTZ1RR8QeYD9wdsDnDkfEfETM9/v94UwnSevYiu3fVTQOdUS8E/g68MnM/Onaz2fmscycycyZycnJYc4oSb9iK7Z/V9FoZ2JE7GA50icyc260I0nSxrZi+3cVTV71EcBXgV5mfnn0I0nSxrZi+3cVTZ5RPw58FLgQEa+vfOxIZr46urEkaWOj3v5dRZNXffwXEFswiyRpAN+ZKEnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqzlBLUnGGWpKKM9SSVFyjxQGStFZXNoBXYKglbVqXNoBXYKglbVqXNoBX4Bm1pE3r0gbwCgy1pE3r0gbwCjz6kLRpXdoAXoGhlrRpXdoAXoGhlvRr6coG8Ao8o5ak4gy1JBVnqCWpOEMtScUZakkqzlBLUnGGWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1JxRlqSSrOUEtScY1ucxoRHwJeArYDxzPziyOdStK63P7dPRuGOiK2A/8A/DFwGfhORHwjMy+OejhJd3P7dzc1eUb9+8D3M/MNgIj4F+AQYKilLeb2725qcka9G3hr1d8vr3zsLhFxOCLmI2K+3+8Paz5Jq7j9u5uG9svEzDyWmTOZOTM5OTmsLytpFbd/d1OTo48rwCOr/v7wysckbTG3f3dTk1B/B/idiNjLcqCfBv5ipFNJGsjt3920YagzczEi/hY4zfLL8/4xM7838skkDeT27+5p9DrqzHwVeHXEs0iSBvCdiZJUnKGWpOIMtSQVZ6glqbjIzOF/0Yg+8MNf8z9/EPjJEMf5Tea1uJvX425ejzvuh2vx25k58N2CIwn1vYiI+cycaXuOCrwWd/N63M3rccf9fi08+pCk4gy1JBVXMdTH2h6gEK/F3bwed/N63HFfX4tyZ9SSpLtVfEYtSVrFUEtScWVCHREfiohLEfH9iPhs2/O0KSIeiYj/jIiLEfG9iHi+7ZnaFhHbI+J8RPxb27O0LSImIuJkRPxvRPQi4g/anqlNEfGplZ+T/4mIf46Id7Q907CVCPWqBbp/AjwK/HlEPNruVK1aBD6dmY8CB4C/6fj1AHge6LU9RBEvAd/MzN8Ffo8OX5eI2A18ApjJzMdYvhXz0+1ONXwlQs2qBbqZuQDcXqDbSZl5LTNfW/nzz1j+QezsDYgj4mHgw8DxtmdpW0S8G/gA8FWAzFzIzOvtTtW6MWA8IsaAXcDVlucZuiqhbrRAt4siYg+wHzjb7iSt+grwGWCp7UEK2Av0ga+tHAUdj4gH2h6qLZl5BfgS8CZwDbiRmd9qd6rhqxJqDRAR7wS+DnwyM3/a9jxtiIgngR9n5rm2ZyliDHgf8HJm7gd+DnT2dzoR8R6W//W9F5gCHoiIZ9udaviqhNoFumtExA6WI30iM+fanqdFjwMfiYj/Y/lI7I8i4pV2R2rVZeByZt7+F9ZJlsPdVR8EfpCZ/cy8BcwB7295pqGrEupfLtCNiJ0s/zLgGy3P1JqICJbPIHuZ+eW252lTZs5m5sOZuYfl/y/+IzPvu2dMTWXmj4C3IuL22vEngIstjtS2N4EDEbFr5efmCe7DX6422pk4ai7Q/RWPAx8FLkTE6ysfO7Kyu1L6OHBi5UnNG8BzLc/Tmsw8GxEngddYfrXUee7Dt5P7FnJJKq7K0YckaR2GWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1Jxf0CnP++rG/VJ1oAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jCBiKCg9AeHT"
      },
      "source": [
        "## Bidirectional + LSTM"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "azGH_FDbwqHT",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 819
        },
        "outputId": "0e471b5e-f73b-41ab-d09b-44970f7ebfa8"
      },
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras.layers import Dense, RepeatVector, Flatten\n",
        "from tensorflow.keras.layers import Bidirectional, LSTM, GRU\n",
        "\n",
        "input_shape = train_x.shape[1:]\n",
        "# input_shape = (5,10)\n",
        "\n",
        "model = keras.Sequential()\n",
        "model.add(Bidirectional(LSTM(128), input_shape=input_shape))\n",
        "model.add(RepeatVector(1))\n",
        "model.add(Dense(CATEGORY_NUM, activation=\"softmax\"))\n",
        "model.compile(loss=\"categorical_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n",
        "model.summary()\n",
        "\n",
        "history = model.fit(train_x, train_y, epochs=100, verbose=1, validation_split=0.1, batch_size=128)\n",
        "\n",
        "\n",
        "plt.plot(history.history['loss'])\n",
        "plt.plot(history.history['val_loss'])\n",
        "plt.show()\n",
        "\n",
        "\n",
        "\n",
        "loss, acc = model.evaluate(test_x, test_y)\n",
        "print(\"loss=\", loss)\n",
        "print(\"acc=\", acc)\n",
        "\n",
        "\n",
        "\n",
        "y_ = model.predict(test_x)\n",
        "predicted = np.argmax(y_, axis=-1)\n",
        "test_y_index = np.argmax(test_y, axis=-1)\n",
        "\n",
        "plt.plot(test_y_index, \"o\")\n",
        "plt.plot(predicted, \".\")\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_2\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "bidirectional (Bidirectional (None, 256)               142336    \n",
            "_________________________________________________________________\n",
            "repeat_vector_2 (RepeatVecto (None, 1, 256)            0         \n",
            "_________________________________________________________________\n",
            "dense_2 (Dense)              (None, 1, 10)             2570      \n",
            "=================================================================\n",
            "Total params: 144,906\n",
            "Trainable params: 144,906\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3hUZd7G8e9vJpNGCKGEGroUsYEELAgiiIt9FXXVta4uq8va1ndZe9tVVlSsiIKCZV27rohYEEERaaGX0BUISwkdkkCSmef9Y0Y3siABkpzMzP25rrkyM+eQcx+O3kxOznkec84hIiLRz+d1ABERqRgqdBGRGKFCFxGJESp0EZEYoUIXEYkRCV5tuF69eq5FixZebV5EJCrNnDlzk3Muc1/LPCv0Fi1akJOT49XmRUSikpmt2t8ynXIREYkRKnQRkRihQhcRiREqdBGRGKFCFxGJESp0EZEYoUIXEYkRUVfoG3fs5pGxuazbXuR1FBGRaiXqCn368vVMnfwVPQZP4P/encvSDTu9jiQiUi14dqfooTrHP5VzAnexKu04npjXmzNndqJn+0b84dTWdGlRGzPzOqKIiCei7hM6bfvCGQ/TPGErz/iHMLvWQI5e9SrXvziOC4d9x2cL1hMKaRYmEYk/5tUUdNnZ2e6wxnIJBWHJWJg6DFZNptSfzMfWk2cLTod6bejfvRUXHN+EpAR/xYUWEfGYmc10zmXvc1nUFnpZ6+bBtBdw89/FgsXMCGTzdEEflqZ25nfdW3H5Cc1ITw5UzLZERDwU+4X+o10bIWcUbsZLWMFG1iQ057miM/gy4VQuOakN13ZrQf2ayRW7TRGRKhQ/hf6j0j2w4H2Y8jxsmM9OfwYj9/TiTX5Fr85H8YcerWhet0blbFtEpBLFX6H/yDn4YRJMGQpLP6PEEvkweArDS8/kqGO7cGPP1rRvmF65GUREKlD8FnpZ+Uth6lDcnLew4G6+cp15vvhsMtr14E+929CxaUbVZREROUQq9LIKNsH0EYSmD8dXtIU5tOW54nMpbtWHP/VuR9eWdao+k4hIOanQ96W4EOa8QWjyM/i2r2YFWTxXfC4bmp/DTacfyUmt63qXTURkP1TovyRYCgs/JDRpCL78ReTRgGdLzmNN0/O45YyjOKGVil1Eqg8VenmEQrD0M0JfD8a3bjbrqcczJeeT1+ICbjnjKDo316kYEfGeCv1gOAcrxhOaMAjf2hzWUY+nS37NliMu4ra+R3FkI10VIyLeUaEfikixB796BP9/ZvIDjXiipB8JR1/In391JE3rpHqdUETikAr9cDgHSz8n+OVD+PMXkuuaMzh4Oa1PPI8Bpx1B7RqJXicUkTjyS4UefaMtVjUzaNcX/43fQr+XaVPLMSphEKdO70//x0bx8rffU1wa8jqliIgKvdx8PjjmIhJuzoFfPcLJKat5m7+S8tlt/GbIaL5ctAGvftoREQEV+sFLSIKTBuC/dS524o1cGpjE64U3MuWNB7l+1BR+2FTgdUIRiVM6h3648pcS+uwOfCvGs9Q15f7gdXTufhZ/6nUEyQGNxS4iFUvn0CtTZlt8V34Al/6L1umONxMeoPmk/+OiIWOYtCzf63QiEkdU6BWl/dn4b5oO3W6lX+A7Xt99E2+NeoZb35zFloJir9OJSBw4YKGbWVMzm2Bmi8xsoZndso91zMyeMbPlZjbPzI6vnLjVXGIN6PMgvj98Ta0GLRma+Axn5/6Fy4b8m0/nr/M6nYjEuPJ8Qi8FbnfOdQBOBAaYWYe91jkTaBN59AeGVWjKaNPwaHy/Hw99/kbvwHzeCd3O6DeHMeBf+rQuIpXngIXunFvnnJsVeb4TyAWa7LXa+cBrLmwqkGFmjSo8bTTxJ0C3m/HdMImaDVszLPFp+iy+l35Pfso3S3VuXUQq3kGdQzezFkAnYNpei5oAa8q8zuN/Sx8z629mOWaWk58fJ6WW2Q7f9eOg552c75/Cv4J/Yciof/HA6IXsLgl6nU5EYki5C93M0oD3gVudczsOZWPOueHOuWznXHZmZuahfIvo5A9Azzuw331Gw/RE3k96kMRpz3Lh0G9Zmb/L63QiEiPKVehmFiBc5m845z7YxyprgaZlXmdF3pOymnbFbvgW/5Fnc1fgTf667UEuf/ZzPpqjvyoROXzlucrFgJeBXOfckP2sNhq4KnK1y4nAduecLuvYl5QMuOQ1OHMwPXxz+TBwD0Pf/ph7/71AY8KIyGEpzyf0bsCVQC8zmxN5nGVmN5jZDZF1xgIrgeXACOCPlRM3RpjBCX/Arh5Dw5QgY1LuZ9P0d7h8xFQ27tztdToRiVK69d9rO9fD21dC3nSeDP2Gt5Iu5sWrutCxaYbXyUSkGtKt/9VZzYZw9cdwzCXc5nubB0PPcuWLXzNWNyKJyEFSoVcHgWS4cDicdg99g1/zVspg7nhjEsMmrtCQvCJSbir06sIMTv0L9HuZDsElfJo+iFGfTeGuDxcQDKnUReTAVOjVzTEXYb99l8ZsZFz635k2YyoD3pilm5BE5IBU6NVR69Owa8ZQK1DKmJqDWLloBteMms6O3SVeJxORakyFXl017gTXjCU1KZGPaw6icNUsfjtiGtsKNbiXiOybCr06y2wL144lKaUm76f+g8QNc7hsxDSN2Cgi+6RCr+7qtIJrxxKoUZu3Uh4lMX8Blw2fSv7OPV4nE5FqRoUeDTKawTVjCKSk816NwQS2LOXyEVP1SV1EfkaFHi0ymsHVowkEAnyQ9g9sywquGjlNvygVkZ+o0KNJ3dZw1WgSLcRH6Y+zZf1qrh01g8LiUq+TiUg1oEKPNvXbw2/fI6VkG5/Ve5Zlq9fyh9dnaqRGEVGhR6Umx8Mlr5G+cxnjmrzE1GXr+ev78zRMgEicU6FHqzanw3nP0WDTVMY0e5MPZ+cx+PMlXqcSEQ8leB1ADkPHy2DHWtp99TeGt8ii/0SjUa1krjqphdfJRMQDKvRo1/122LSUM+aNYGCzxjwwGprXrcGpbeNozlYRAXTKJfqZwbnPQFYXbtz6GGfW3cif/jWL5Rs1+bRIvFGhx4JAMvzmDSylDk8zmPq+nVz/6gyN+yISZ1TosaJmA7j0nyQUbebDBiNZv62QAf+aRWlQlzOKxAsVeixp3AnOfpz0dZP54MiJTF6+mSHjlnqdSkSqiAo91hx/FXS6kg7Lh/Ng+9U8P3EFXyxc73UqEakCKvRYdNbj0KgjV60fRK+Ge7j93bn8sKnA61QiUslU6LEokAwXv4KFQgxLeZ4AIW7UNHYiMU+FHqvqtIRznyJp3Qw+OGoSuet2MGhsrtepRKQSqdBj2TEXQccraLHweR46diuvTlnFuEUbvE4lIpVEhR7rznwU6rbmynV/54SGxsD35rJ++26vU4lIJVChx7qkNOj3ElaQz0uZ77C7JMRtb88hFNLIjCKxRoUeDxp3gh4DqbnsQ17uspYpKzczcvL3XqcSkQqmQo8X3f8MjTtxUu7fubBNgMGfL2H5xp1epxKRCqRCjxf+AFzwIlZcwKDACGoEfNz+zlwNDSASQ1To8SSzHZx+P0krv2Bk51XMzdvOsIkrvE4lIhVEhR5vTrgBmmTTaeEgLjsqlafHLyN33Q6vU4lIBVChxxufH85/Dvbs5IHE18lIDXDH+/MI6qoXkainQo9H9Y+E7reTlPs+Q7vkMzdvO6N01YtI1DtgoZvZSDPbaGYL9rO8p5ltN7M5kcd9FR9TKlz3P0Nme7oufJiz26bx+BdLWL250OtUInIYyvMJ/RWg7wHWmeSc6xh5PHT4saTSJSTBec9iO9byaL2xJPh83PXhfJzTqReRaHXAQnfOfQNsqYIsUtWadoXOV5M2ewSDuvn4dvkmPpy91utUInKIKuoc+klmNtfMPjWzoyroe0pV6H0/pGRwzprH6ZiVziNjc9leVOJ1KhE5BBVR6LOA5s6544BngX/vb0Uz629mOWaWk5+fXwGblsOWWgf6PIStmcpzHRazpaCYIV8s8TqViByCwy5059wO59yuyPOxQMDM6u1n3eHOuWznXHZmZubhbloqynGXQ9MTyZrxCP2za/P61FXMz9vudSoROUiHXehm1tDMLPK8a+R7bj7c7ytVyOeDs5+A3du4LfAedWokcc9HCzQio0iUKc9li28CU4B2ZpZnZteZ2Q1mdkNklYuABWY2F3gGuNTpUono0/BoyL6OpNmjGNzdz9w123g7Z43XqUTkIJhX3Zudne1ycnI82bbsR+EWePZ4XIOjuaToTlZuKuSr/+tJrZSA18lEJMLMZjrnsve1THeKyn+l1oHT7sZ+mMQTx6xmS2Exz4xf5nUqESknFbr8XOdrof5RNJvxCFd0rs+r3/2gcdNFooQKXX7OnxCeh3T7au7MGE9Kop+HxuTqDlKRKKBCl//Vsju0P4fU6c9yxym1+WZpPl8t3uh1KhE5ABW67NvpD0Lpbi4t/Cet6tXgkbG5lGh2I5FqTYUu+1bvCOhyPf7Zr/P3k32syC/gremrvU4lIr9AhS77d+pfIakmJ614ihNa1uHJL5exY7fGeRGprlTosn+pdaDHQGzFeP5xXD5bCop5foLmIBWprlTo8su6/h4ymtNy9mD6dWzEyMnfs2aLJsIQqY5U6PLLEpKg932wYT73NF+IAU9oNEaRakmFLgd21IXQ6DhqTx3M709uzL/n/IcFazUao0h1o0KXA/P5wpcxbl/NgLSvqZUSYPDn+pQuUt2o0KV8Wp8GrU4jZcoQ/nxKA75Zms/k5Zu8TiUiZajQpfz6PAhFW/lt8EOaZKTwj08Xa8x0kWpEhS7l1+g4OPoiEma8yJ3dazN/7XbGzF/ndSoRiVChy8E57S4IFnPWtjdo37AmT3yxREMCiFQTKnQ5OHVbQ6cr8c18hXu7pbBqcyHvaGYjkWpBhS4H79SB4PNz8prhdG5em2fGL2N3SdDrVCJxT4UuBy+9MXTtj817l/u7Ojbs2MOr3/3gdSqRuKdCl0Nzym2QlM6xS5/j1LaZPD9xBduLNHCXiJdU6HJoUuvAyTfBkrHc16mQ7UUljPhmpdepROKaCl0O3Yk3QGpdWs9/knOODQ/ctWnXHq9TicQtFbocuqSa0P12WDmRu47MZ09pSMPrinhIhS6HJ/s6qNmYxjMfp1+nxvxz6irWbivyOpVIXFKhy+EJJIcvY8ybzsBWqwB4dvwyj0OJxCcVuhy+TldA7ZbUm/4Yv+2axbsz81iZv8vrVCJxR4Uuh88fgJ53wob53NYkl0S/jye/1Kd0kaqmQpeKccxFkNme9CmD+d3JWXw89z/krtvhdSqRuKJCl4rh88Npd8PmZQyoM4uayQk88cVSr1OJxBUVulScI8+FRh1JnfIYN57SlC9zNzBr9VavU4nEDRW6VBwz6HUvbFvNdamTqFsjURNKi1QhFbpUrCN6Q7OTSPpuCH/q3pjJyzfznaaqE6kSKnSpWGbQ+z7YtZ4r/ONomJ7MY18swTlNVSdS2VToUvGanwytexP47in+3KMRs1dvY3zuRq9TicQ8FbpUjt73QtFW+hV/RIu6qTz+xRJNKC1SyQ5Y6GY20sw2mtmC/Sw3M3vGzJab2TwzO77iY0rUadwJjjwP/9ShDDy1PovX7+Tjef/xOpVITCvPJ/RXgL6/sPxMoE3k0R8YdvixJCacdjcU7+LMrW/SvmFNhoxbqgmlRSrRAQvdOfcNsOUXVjkfeM2FTQUyzKxRRQWUKFa/PRx3KTZjBPd0r8WqzYW8m5PndSqRmFUR59CbAGWnfc+LvPc/zKy/meWYWU5+fn4FbFqqvZ53QihIt7UjOb5ZBk+PX6oJpUUqSZX+UtQ5N9w5l+2cy87MzKzKTYtXajeH7Gux2a9z38lJbNixh1c0obRIpaiIQl8LNC3zOivynkhYj79AQjIdlw3ltHaZPD9hOdsLNaG0SEWriEIfDVwVudrlRGC7c25dBXxfiRVp9eHEG2HhB9zbuYSde0p54RtNVSdS0cpz2eKbwBSgnZnlmdl1ZnaDmd0QWWUssBJYDowA/lhpaSV6nXwTJGfQau4TnH9cY0ZN/p4NO3Z7nUokppTnKpfLnHONnHMB51yWc+5l59wLzrkXIsudc26Ac661c+4Y51xO5ceWqJOSEZ5QesV47myfTzDkeEqTYIhUKN0pKlWna39Iz6LB9Ef4bZcs3slZw/KNmqpOpKKo0KXqBJKh193wn9n8OWsxKQE/gz9b7HUqkZihQpeqdexvoP5RpE9+hD92b8oXizYw44dfum9NRMpLhS5Vy+eH0x+Ard9zfeokGqQn8cjYXA2vK1IBVOhS9dr0geankPjtYAb2bMzs1dv4fOF6r1OJRD0VulQ9Mzjjb1C4iQsK3qFtgzQe/WwJxaUauEvkcKjQxRtNjodjLsE3bRgP9KjF95sK+OfUVV6nEolqKnTxTu97wTlOWjWU7m3q8fT4ZWwrLPY6lUjUUqGLdzKawUl/xOa9w9+6FLNzdwlPj9fNRiKHSoUu3jrlz5BajxYzB/Gb7Ka8PmUVK/J1s5HIoVChi7eS0+G0O2HVZO5osZTkgJ9BY3O9TiUSlVTo4r3jr4H6Hag16UFu6pHFl7kb+WapJkAROVgqdPGePwH6DoJtq7nOP5bmdVN58OOFmn9U5CCp0KV6aNUT2p9DwuQn+XvvuqzIL+BVzWwkclBU6FJ9nPE3CJVwyqqh9GyXydNfLiN/5x6vU4lEDRW6VB91WsFJA7C5b/FwdiG7S4M89rlGYxQpLxW6VC/d/w9qNqbJ5Hu5rlsz3snJY9bqrV6nEokKKnSpXpLS4FcPw/p53JoxmYbpydzz4QJK9QtSkQNSoUv1c9QF0LIHyd88zMNnNGTRuh28NkXjvIgciApdqh8zOOtxKC6gV97znNo2kyHjlmpSaZEDUKFL9ZTZLvwL0jn/5NEuBRQHQzw0ZpHXqUSqNRW6VF89BkKtpjScdBc39WjBJ/PWMXHJRq9TiVRbKnSpvpLSwqdeNi7ixqRPaJ1Zg7s/XEDBnlKvk4lUSyp0qd7a9YUjzyNh0mM81acWa7cV8cQXS71OJVItqdCl+jvzUfAFOGbuQ1x5QjNGffc9c9Zs8zqVSLWjQpfqL70x9L4PVnzFXU3n0aBmMne8P0+Dd4nsRYUu0aHLddD0BFLG382jfRuyeP1Ohk5Y7nUqkWpFhS7RweeH856D4kJOXf4ov+7YmOe+Ws7C/2z3OplItaFCl+iR2RZ63gGLPuLhdiuoXSOR29+ZS3GpTr2IgApdos3JN0Oj46jx5R08flYWi9fv5LmvNLG0CKjQJdr4E+D8oVC0jVOXP0q/47MYOnEF8/J01YuICl2iT8NjwqdeFn7A31ovpn7NJG59aw6FxbrhSOKbCl2iU7dbIasLqeMG8uw5Dfh+cwEPf5LrdSoRT6nQJTr5E+CCFyFYTPac+/j9KS15Y9pqvly0wetkIp4pV6GbWV8zW2Jmy83sjn0sv8bM8s1sTuRxfcVHFdlL3dbQ5yFYMZ6/1PmGDo3S+ev789i4U8PsSnw6YKGbmR8YCpwJdAAuM7MO+1j1bedcx8jjpQrOKbJvXa6HI/oQ+PI+hvVJZNeeUm5/Zy6hkPM6mUiVK88n9K7AcufcSudcMfAWcH7lxhIpJzP49TBIrkXzCTfzt7NbMWnZJp6fqLtIJf6Up9CbAGvKvM6LvLe3fmY2z8zeM7Om+/pGZtbfzHLMLCc/P/8Q4orsQ1omXPgi5C/m4k0vcN5xjRkybinTVm72OplIlaqoX4p+DLRwzh0LjANe3ddKzrnhzrls51x2ZmZmBW1aBGjdC06+GZs5kkc7fE/zujW4+a3ZbN61x+tkIlWmPIW+Fij7iTsr8t5PnHObnXM//p/zEtC5YuKJHIRe90KTzqSMvYXhZ2ewtbCEW96aQ1Dn0yVOlKfQZwBtzKylmSUClwKjy65gZo3KvDwP0AXBUvUSEuHiV8Dnp83EAQw65wi+Xb6Jxz5f4nUykSpxwEJ3zpUCfwI+J1zU7zjnFprZQ2Z2XmS1m81soZnNBW4GrqmswCK/KKMZXPgSbFhIv/VPcfkJzXjh6xV8On+d18lEKp05582Po9nZ2S4nJ8eTbUsc+Oph+GYwJWc9ySU57ViyficfDehGmwY1vU4mcljMbKZzLntfy3SnqMSmnndA614EPhvISz2DpCYmcP1rOWwtKPY6mUilUaFLbPL54aKRkNGUup/8jpEXNGTd9t384Z8zNX66xCwVusSulNpw2VtQsptjJ93IkF+3Yfr3W7j7w/l4dapRpDKp0CW2ZbaDi16G9fM5Z8UD3HJaS96dmccLX6/0OplIhVOhS+xr+ys44++Q+zG3hl7j3OMa8+hni/lgVp7XyUQqVILXAUSqxEkDYPsabNownuyTxeZd2Qx8bx5105I4ta3uWpbYoE/oEh/M4FePQPtzSBh3Dy93WUubBjW58Z8zmbtG09dJbFChS/zw+aHfS9C0Kymj/8Cbp+2ibloi14yazuL1O7xOJ3LYVOgSXwIpcPnbUL89GaOv5d2+kJTg54qXprF84y6v04kcFhW6xJ+U2nDFh1Ari4ZjruK985MB4/IRU/lhU4HX6UQOmQpd4lNaJlz1EaTWJuvjy/jgvERKQ47LRkxlZb4+qUt0UqFL/KrVBK75BFJq02zM5Xxwjo/i0hCXvDhV59QlKqnQJb5lNINrxkLNBrQYeyWjzwnh98Glw6cyL09Xv0h0UaGL/PhJPaMpTcZcyZjeW0hLSuCy4VOZtExTJUr0UKGLANRsCNd+Co2OI/PT/ow9YRFN66Ry7agZvD9Td5RKdFChi/wotQ5cPRran036xLv5qO1nnNiyFre/O5dnxy/TgF5S7anQRcoKpMAlr0GX60maPpTXUp7k8mNr8cS4pdz05myKioNeJxTZLxW6yN58fjj7CTh7CL6VE3h486080iOFT+avo9+w78jbWuh1QpF9UqGL7E+X6+Cq0VjRFi6fezUf98pnzdZCzntuMhOWbPQ6ncj/UKGL/JIW3aD/RMhsx9GTb2Fyh4/ISoNrR81g0NhcSoKa/UiqDxW6yIFkNAtfAdPtVtIXvcG/E+/hL8cW8eI3K7n4hSl8r+ECpJpQoYuUhz8AfR6EK97HV7SNAcv680XHb1mdv40zn/6Gkd9+Tyikq2DEWyp0kYNxxOkwYCocczFtFz/PtHp/58omG3hozCIuHT6V5Rt3ep1Q4pgKXeRgpdSGC1+ES98kULydu9ffwoQ277Bx/Rr6PjWJQZ/mUrCn1OuUEodU6CKHqv1ZMGA6dLuVlms/4auk23kmawKvfp1LnyFf88GsPII6DSNVyLy6+y07O9vl5OR4sm2RCpe/FL64B5Z9TnFqA4bbxTy9uSutG9ZmYN92nNauPmbmdUqJAWY20zmXvc9lKnSRCrTqOxh3P+RNpyilESOCZzFsRzfaZjXgxp6tOaNDQ3w+FbscOhW6SFVyDpZ/CZOGwOrv2B2ozXv0YtjOHiRntuCabi25oFMT0pISvE4qUUiFLuKV1VNh8jO4pZ+Cg+mBbF4u6Mb0hM6ce3xLLsluytFN0nU6RspNhS7itW1rYOYruFmvYgX5FPpq8lHpCXxc2pX8Op05t1NzzjqmEUfUT/M6qVRzKnSR6iJYAisnwry3cYs/wUoK2WVpjCs9jgnBjqzN6Erno9vRs20mxzevTXLA73ViqWZU6CLVUXEBrJgAS8YSXPwp/t1bAMh1zZgaPJJ51o5g42xaH3EknZrXpmOzDNKTAx6HFq+p0EWqu1AQ1s2FlRMpXTEBWzMDf7AIgE0unYWhFixwLdiS1hZ/ww7UbdaBtk3qckRmGk0yUnTlTBxRoYtEm2ApbFwIq6dRnDeL4rw5pGxbit+FJ9gIOmONq88q14A8a0hBahauVhaJdZpRs34zamdm0ahOTRrVSiYjNaBfusaQwy50M+sLPA34gZecc//Ya3kS8BrQGdgM/MY598MvfU8VushBKt0Dm5bBpiXsXruQgnVLYOv3pO5aTUrw52PIhJyxmXTyXQZbSKcgoTZ7kuoQTMrApdTGl1qHhNQMEtMySEqrQ0paLVLSMqhRsxZpKYnUTAqQHPDpH4Jq6JcK/YAXwpqZHxgK9AHygBlmNto5t6jMatcBW51zR5jZpcCjwG8OP7qI/CQhCRoeDQ2PJvnofiSXXVa0DXasJbR1DbvyV1GwOY+SbetI27WBWkWbSS5eRurubaQUFcG2X95MoUtiF0nkk8wektjjS6bUkijxJRH0JxHyJRLyJxHyJ+L8yTh/IvgTcf5ELCER8yVCQgCfP4D9+J4/gC8hAfMH8PsTfnrt8wUwf0LkvQR8/gT8fj/mS8CfEPnq8+Pz+8F8+P0J+Pw+zOfH7/NjkWU+nw+fzx9++H2Y+fCZ4fP5MCNu/mEqz50NXYHlzrmVAGb2FnA+ULbQzwceiDx/D3jOzMxpVl2RqpGSASkZ+BocRXp7SN/feqXFULQVV7SF4l3bKNixmaKdWygu3EFJ4Q6CRTsIFRfAngIoKcRXWkSgtIjkYBH+0C4SQptJKN1DgishwZWQ6IoJUEIi1XMwspAzQkAIHyEMMBzgMEIYDt9Pr8FwFl43XFyGizyIrOMif569n9vP16PMOuH3/rsewLpWl3DiFfdX+P6Wp9CbAGvKvM4DTtjfOs65UjPbDtQFNpVdycz6A/0BmjVrdoiRReSQJSRCzQZYzQYk1Yekivq+zoUvyQwWEywtoaR4N8XFewiW7CFYWkxpcTHBYAnBkvDyUDBIKFRCqLQEFwoSCpaGn7sgLhjEhUpxoRChUBBCQVwohHPh57gQLhQCF4x8DYXfx+HKPMc5cMFwm7rIei5c3+HlkdcuMutU5Ku50H//fNmvPy3jp/fsp+UAIeynj7D//TM//z7hmk9Ib1BRf/M/U6X3HjvnhgPDIXwOvSq3LSKVyCz8j0VCIv4k8Nfg56eEpEqUZ/jctUDTMq+zIu/tcx0zSwBqEf7lqIiIVJHyFPoMoI2ZtTSzROBSYPRe64wGro48vwj4SufPRUSq1gFPuUTOif8J+JzwZYsjnXMLzewhIMc5Nxp4GXjdzJYDWwiXvoiIVKFynUN3zo0FxmJecKwAAAQQSURBVO713n1lnu8GLq7YaCIicjA0BZ2ISIxQoYuIxAgVuohIjFChi4jECM9GWzSzfGDVIf7xeux1F2qciMf9jsd9hvjc73jcZzj4/W7unMvc1wLPCv1wmFnO/kYbi2XxuN/xuM8Qn/sdj/sMFbvfOuUiIhIjVOgiIjEiWgt9uNcBPBKP+x2P+wzxud/xuM9QgfsdlefQRUTkf0XrJ3QREdmLCl1EJEZEXaGbWV8zW2Jmy83sDq/zVAYza2pmE8xskZktNLNbIu/XMbNxZrYs8rW211krg5n5zWy2mY2JvG5pZtMix/ztyDDOMcPMMszsPTNbbGa5ZnZSPBxrM7st8t/3AjN708ySY/FYm9lIM9toZgvKvLfP42thz0T2f56ZHX8w24qqQi8zYfWZQAfgMjPr4G2qSlEK3O6c6wCcCAyI7OcdwHjnXBtgfOR1LLoFyC3z+lHgSefcEcBWwpOSx5Kngc+cc+2B4wjve0wfazNrAtwMZDvnjiY8NPePE8zH2rF+Bei713v7O75nAm0ij/7AsIPZUFQVOmUmrHbOFQM/TlgdU5xz65xzsyLPdxL+H7wJ4X19NbLaq8CvvUlYecwsCzgbeCny2oBehCcfhxjbbzOrBfQgPKcAzrli59w24uBYEx6+OyUyy1kqsI4YPNbOuW8IzxNR1v6O7/nAay5sKpBhZo3Ku61oK/R9TVjdxKMsVcLMWgCdgGlAA+fcusii9UDlzDTrraeAgUBkNl7qAtuccz9OKx9rx7wlkA+MipxmesnMahDjx9o5txZ4HFhNuMi3AzOJ7WNd1v6O72F1XLQVelwxszTgfeBW59yOsssiU/zF1DWnZnYOsNE5N9PrLFUoATgeGOac6wQUsNfplRg91rUJfxptCTQGavC/pyXiQkUe32gr9PJMWB0TzCxAuMzfcM59EHl7w48/fkW+bvQqXyXpBpxnZj8QPp3Wi/D55YzIj+UQe8c8D8hzzk2LvH6PcMHH+rE+HfjeOZfvnCsBPiB8/GP5WJe1v+N7WB0XbYVengmro17kvPHLQK5zbkiZRWUn474a+Kiqs1Um59ydzrks51wLwsf2K+fcb4EJhCcfhxjbb+fcemCNmbWLvNUbWESMH2vCp1pONLPUyH/vP+53zB7rvezv+I4Gropc7XIisL3MqZkDc85F1QM4C1gKrADu9jpPJe3jKYR/BJsHzIk8ziJ8Pnk8sAz4EqjjddZK/DvoCYyJPG8FTAeWA+8CSV7nq+B97QjkRI73v4Ha8XCsgQeBxcAC4HUgKRaPNfAm4d8TlBD+iey6/R1fwAhfybcCmE/4KqByb0u3/ouIxIhoO+UiIiL7oUIXEYkRKnQRkRihQhcRiREqdBGRGKFCFxGJESp0EZEY8f9H0bZVY1QeuQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 15ms/step - loss: 0.0037 - accuracy: 1.0000\n",
            "loss= 0.0037450515665113926\n",
            "acc= 1.0\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAMs0lEQVR4nO3dX2id93nA8e9jy6ZyulaFiICcMPti9RQCw0UMr4Eylg53a6hzsYtsSaEZwzC2Nu26lMo3vWpTcCnNxQgYd4URs124wisj1B1svRjMpnKc4tVnhpKuif+Unl7YLcVDFnp6IbmW1aPoVX2O3qd+v58rWzqRHl6iL8c/nfM+kZlIkura1vYAkqS3Z6glqThDLUnFGWpJKs5QS1JxY6P4og8++GDu2bNnFF9aku5L586d+0lmTg763EhCvWfPHubn50fxpSXpvhQRP1zvcx59SFJxhlqSijPUklScoZak4gy1JBU3kld9SPejU+evcPT0Ja5ev8nUxDgvHNzHU/t3tz2WOsBQSw2cOn+F2bkLTC/2OLS9x5kb08zOLQAYa42coZYaOHr6EtOLPU7s/AI7WOQWYzyzcISjp3caao2cZ9RSA1ev3+TAth47WGQsltjBIge29bh6/Wbbo6kDDLXUwNTEOGeWprnFGIu5jVuMcWZpmqmJ8bZHUwd49CE18MLBfczOLfDMwhEObOtxZmma3tg0Lx7c1/Zo6gBDLTVw+xz66OmdvHz9vUxNjPOir/rQFjHUUkNP7d9tmNUKz6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqzlBLUnGGWpKKM9SSVFyj25xGxKeAvwISuAA8l5n/P8rBpNvc/q2u2/AZdUTsBj4BzGTmY8B24OlRDybBne3fD934Ln+9/V956MZ3mZ27wKnzV9oeTdoyTRcHjAHjEXEL2AVcHd1I0h1u/5YaPKPOzCvAl4A3gWvAjcz81trHRcThiJiPiPl+vz/8SdVJbv+Wmh19vAc4BOwFpoAHIuLZtY/LzGOZOZOZM5OTk8OfVJ3k9m+p2dHHB4EfZGYfICLmgPcDr4xyMAnc/i1Bs1C/CRyIiF3ATeAJYH6kU0kr3P4tNQh1Zp6NiJPAa8AicB44NurBpNvc/q2ua/Sqj8z8HPC5Ec8iSRrAdyZKUnGGWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1JxRlqSSrOUEtScYZakooz1JJUnKGWpOIMtSQV13RnojrI7d9SDYZaA93e/j292OPQ9h5nbkwzO7cAYKylLWaoNZDbv6U6PKPWQG7/luow1BrI7d9SHR59aCC3f0t1GGoN5PZvqQ5DrXW5/VuqwTNqSSrOUEtScYZakooz1JJUnKGWpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBXX6DanETEBHAceAxL4y8z871EO1mVu/5a0WtP7Ub8EfDMz/ywidgK7RjhTp7n9W9JaG4Y6It4NfAD4GEBmLgALox2ru9z+LWmtJmfUe4E+8LWIOB8RxyPigbUPiojDETEfEfP9fn/og3aF278lrdUk1GPA+4CXM3M/8HPgs2sflJnHMnMmM2cmJyeHPGZ3uP1b0lpNzqgvA5cz8+zK308yINQaDrd/S1prw1Bn5o8i4q2I2JeZl4AngIujH62b3P4taa2mr/r4OHBi5RUfbwDPjW4kuf1b0mqNQp2ZrwMzI55FkjSA70yUpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqrultTjvDDeCSqjHUq7gBXFJFhnoVN4BLqsgz6lXcAC6pIkO9ihvAJVXk0ccqbgCXVJGhXsUN4JIqMtRruAFcUjWeUUtScYZakooz1JJUnKGWpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpuMa3OY2I7cA8cCUznxz2IG7/lqTBNnM/6ueBHvCuYQ/h9m9JWl+jUEfEw8CHgc8DfzfsIdz+LUnra3pG/RXgM8DSeg+IiMMRMR8R8/1+f1NDuP1bkta3Yagj4kngx5l57u0el5nHMnMmM2cmJyc3NYTbvyVpfU2OPh4HPhIRfwq8A3hXRLySmc8Oawi3f0vS+jYMdWbOArMAEfGHwN8PM9Lg9m9JejtltpC7/VuSBttUqDPz28C3RzKJJGkg35koScUZakkqzlBLUnGGWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1JxRlqSSrOUEtScYZakooz1JJUXJnbnErSb6pT569w9PQlrl6/ydTEOC8M+X76hlqS7sGp81eYnbvA9GKPQ9t7nLkxzezcAsDQYm2oJekeHD19ienFHid2foEdLHKLMZ5ZOMLR0zuHFmrPqCXpHly9fpMD23rsYJGxWGIHixzY1uPq9ZtD+x6GWpLuwdTEOGeWprnFGIu5jVuMcWZpmqmJ8aF9D48+JOkevHBwH7NzCzyzcIQD23qcWZqmNzbNiwf3De17GGpJuge3z6GPnt7Jy9ffy9TEOC/6qg9JquWp/buHGua1PKOWpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqzlBLUnEb3uY0Ih4B/gl4CEjgWGa+NOrBJGkjo97+XUWT+1EvAp/OzNci4reAcxHx75l5ccSzSdK6tmL7dxUbhjozrwHXVv78s4joAbsBQy2pNVux/buKTZ1RR8QeYD9wdsDnDkfEfETM9/v94UwnSevYiu3fVTQOdUS8E/g68MnM/Onaz2fmscycycyZycnJYc4oSb9iK7Z/V9FoZ2JE7GA50icyc260I0nSxrZi+3cVTV71EcBXgV5mfnn0I0nSxrZi+3cVTZ5RPw58FLgQEa+vfOxIZr46urEkaWOj3v5dRZNXffwXEFswiyRpAN+ZKEnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqzlBLUnGGWpKKM9SSVFyjxQGStFZXNoBXYKglbVqXNoBXYKglbVqXNoBX4Bm1pE3r0gbwCgy1pE3r0gbwCjz6kLRpXdoAXoGhlrRpXdoAXoGhlvRr6coG8Ao8o5ak4gy1JBVnqCWpOEMtScUZakkqzlBLUnGGWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1JxRlqSSrOUEtScY1ucxoRHwJeArYDxzPziyOdStK63P7dPRuGOiK2A/8A/DFwGfhORHwjMy+OejhJd3P7dzc1eUb9+8D3M/MNgIj4F+AQYKilLeb2725qcka9G3hr1d8vr3zsLhFxOCLmI2K+3+8Paz5Jq7j9u5uG9svEzDyWmTOZOTM5OTmsLytpFbd/d1OTo48rwCOr/v7wysckbTG3f3dTk1B/B/idiNjLcqCfBv5ipFNJGsjt3920YagzczEi/hY4zfLL8/4xM7838skkDeT27+5p9DrqzHwVeHXEs0iSBvCdiZJUnKGWpOIMtSQVZ6glqbjIzOF/0Yg+8MNf8z9/EPjJEMf5Tea1uJvX425ejzvuh2vx25k58N2CIwn1vYiI+cycaXuOCrwWd/N63M3rccf9fi08+pCk4gy1JBVXMdTH2h6gEK/F3bwed/N63HFfX4tyZ9SSpLtVfEYtSVrFUEtScWVCHREfiohLEfH9iPhs2/O0KSIeiYj/jIiLEfG9iHi+7ZnaFhHbI+J8RPxb27O0LSImIuJkRPxvRPQi4g/anqlNEfGplZ+T/4mIf46Id7Q907CVCPWqBbp/AjwK/HlEPNruVK1aBD6dmY8CB4C/6fj1AHge6LU9RBEvAd/MzN8Ffo8OX5eI2A18ApjJzMdYvhXz0+1ONXwlQs2qBbqZuQDcXqDbSZl5LTNfW/nzz1j+QezsDYgj4mHgw8DxtmdpW0S8G/gA8FWAzFzIzOvtTtW6MWA8IsaAXcDVlucZuiqhbrRAt4siYg+wHzjb7iSt+grwGWCp7UEK2Av0ga+tHAUdj4gH2h6qLZl5BfgS8CZwDbiRmd9qd6rhqxJqDRAR7wS+DnwyM3/a9jxtiIgngR9n5rm2ZyliDHgf8HJm7gd+DnT2dzoR8R6W//W9F5gCHoiIZ9udaviqhNoFumtExA6WI30iM+fanqdFjwMfiYj/Y/lI7I8i4pV2R2rVZeByZt7+F9ZJlsPdVR8EfpCZ/cy8BcwB7295pqGrEupfLtCNiJ0s/zLgGy3P1JqICJbPIHuZ+eW252lTZs5m5sOZuYfl/y/+IzPvu2dMTWXmj4C3IuL22vEngIstjtS2N4EDEbFr5efmCe7DX6422pk4ai7Q/RWPAx8FLkTE6ysfO7Kyu1L6OHBi5UnNG8BzLc/Tmsw8GxEngddYfrXUee7Dt5P7FnJJKq7K0YckaR2GWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1Jxf0CnP++rG/VJ1oAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TfY_p_SJA5Rf"
      },
      "source": [
        "## Bidirectional + GRU"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OcjycCe8wqEH",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 819
        },
        "outputId": "dc8ed707-de34-488b-dd1d-2fff1edc23fa"
      },
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras.layers import Dense, RepeatVector, Flatten\n",
        "from tensorflow.keras.layers import Bidirectional, LSTM, GRU\n",
        "\n",
        "input_shape = train_x.shape[1:]\n",
        "# input_shape = (5,10)\n",
        "\n",
        "model = keras.Sequential()\n",
        "model.add(Bidirectional(GRU(128), input_shape=input_shape))\n",
        "model.add(RepeatVector(1))\n",
        "model.add(Dense(CATEGORY_NUM, activation=\"softmax\"))\n",
        "model.compile(loss=\"categorical_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n",
        "model.summary()\n",
        "\n",
        "history = model.fit(train_x, train_y, epochs=100, verbose=1, validation_split=0.1, batch_size=128)\n",
        "\n",
        "\n",
        "plt.plot(history.history['loss'])\n",
        "plt.plot(history.history['val_loss'])\n",
        "plt.show()\n",
        "\n",
        "\n",
        "\n",
        "loss, acc = model.evaluate(test_x, test_y)\n",
        "print(\"loss=\", loss)\n",
        "print(\"acc=\", acc)\n",
        "\n",
        "\n",
        "\n",
        "y_ = model.predict(test_x)\n",
        "predicted = np.argmax(y_, axis=-1)\n",
        "test_y_index = np.argmax(test_y, axis=-1)\n",
        "\n",
        "plt.plot(test_y_index, \"o\")\n",
        "plt.plot(predicted, \".\")\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_3\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "bidirectional_1 (Bidirection (None, 256)               107520    \n",
            "_________________________________________________________________\n",
            "repeat_vector_3 (RepeatVecto (None, 1, 256)            0         \n",
            "_________________________________________________________________\n",
            "dense_3 (Dense)              (None, 1, 10)             2570      \n",
            "=================================================================\n",
            "Total params: 110,090\n",
            "Trainable params: 110,090\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3hUZd7G8e9vJiGFkoTei4AUBQQixO4q7ioKuooFXUXFuvKqq66rrg3LrgUFRVFpSlkRRVcREBVEKYIaOqEGkCZICBIgJKQ97x8ZNIsgQSY5mZn7c11zJXPOSeY+HrwzeXLOc8w5h4iIhD6f1wFERCQ4VOgiImFChS4iEiZU6CIiYUKFLiISJqK8euGaNWu6pk2bevXyIiIhaf78+Tucc7UOtc6zQm/atCmpqalevbyISEgysw2HW6chFxGRMKFCFxEJEyp0EZEwoUIXEQkTKnQRkTChQhcRCRMqdBGRMBFyhb4zO4/+H6eRvb/A6ygiIhVKyBX67PQdvPX191z86hxW/7jH6zgiIhVGyBV6z/b1eP+y6uzal8fFr8zh/fmbvY4kIlIhhFyhs/gdOk2+kK+S59GhQRXufW8x909YTE5eodfJREQ8FXqF3ro7tLucyvMGMC7qcR46JZZ3UzdzyatzSN++1+t0IiKeCb1Cj02AS9+Ay0ZgGau5Ja0PU8/eTMaeXHq+Mpv/LtQQjIhEptAr9APa9YLbZ0PddrSedz9zmo+ha10ffxu/mH9MWKIhGBGJOKFb6ACJjeH6SXDuY8StncLInLt4ttNPvDt/Exe/Ops1OgtGRCJIaBc6gM8PZ9wDN03DKlXmyuV3MLPDdHbvyabnK3N4L3WT1wlFRMpF6Bf6AfU7wq0z4eSbaLRyBLOqP0GPuj/x9wlLuGf8Il2IJCJhL3wKHaBSPFz4Alz9LtE5O3h2512MbvMdHy3aRI9XZrP8h91eJxQRKTPhVegHHP8nuH0u1uJczlw/kPlNhhCf8yOXDJnD2HkbcM55nVBEJOjCs9ABqtSCq96GHi+RmLmIiVH3c2fdZTz84TL6jVvI7tx8rxOKiARV+BY6gBl0vh5um42v+nH02/EUU5uMY/aydVz08myWbN7ldUIRkaAJ70I/oEZz6PsZnPl3Wm+fzLfVH6d1/goue+1rRs5eryEYEQkLkVHoAP5oOOdhuOETYqKMNwoe5oVaU3h60lJuHTOfrH0aghGR0BY5hX5A4xS4bTbW7nJ67hrD13WeZ/XKZVw4eBaLNmkIRkRCV+QVOvzPfDB1cr9neuWH6VYwk8tf1xCMiISuyCz0A9r1gttm4697Ao/nD2RU0khemDSf28cuICtHQzAiEloiu9ABkprA9ZPhrAc4JXs6Xyf1Z+vKefQYPJtlW7K8TiciUmoqdAB/FPzhQazPJBKiCvhvzGNcuv8jLn1tDm9/s1FDMCISElToJTU9DW6fg6/ledxd+CbjqrzMs/+dy73v6o5IIlLxqdAPFl+9+ArT85+hU14qsxIeYcPiGfx5yBy+35HtdToRkcNSoR+KGaTcjvX9lGpxsbwX8xTdsibQY/AsPkvb5nU6EZFDUqH/lgad4daZ+Fqdz31uFG/EvMS9Y2bxwmerKCzSuLqIVCxHLHQza2RmM8xsuZmlmdldh9jGzOxlM0s3syVm1qls4nogLhGuHAt/fJpTCr5lRrXH+GzGF/Qd9Z2uLhWRCqU079ALgHudc22BFOAOM2t70DYXAC0Dj1uA14Ka0mtmcGo/7PrJ1KhUyOT4x0la+xE9XpnNqm26zZ2IVAxHLHTn3Fbn3ILA53uAFUCDgza7GBjtis0DEs2sXtDTeq1xCnbrTKIadGJg1CvcljucXkNmMnWZxtVFxHtHNYZuZk2BjsA3B61qAJS8eedmfl36mNktZpZqZqkZGRlHl7SiqFoH+kyErrdzddEkxsY8ywNjv2TQtNUUaVxdRDxU6kI3syrA+8DdzrnfdS8359xQ51yycy65Vq1av+dbVAz+aLjgGbjkNdoXrWR61cf5ZPp0+o1boPPVRcQzpSp0M4umuMz/45z74BCbbAEalXjeMLAsvJ10NXbDFKrHOCbF96dg+SSueGMu27JyvU4mIhGoNGe5GDACWOGce/Ewm00Erguc7ZICZDnntgYxZ8XVMBm75Uui67ThjeiBnJXxNj0Hz2LpZs0DIyLlK6oU25wGXAssNbNFgWUPAY0BnHOvA1OA7kA6sA+4IfhRK7Bq9eD6ydhHf+W+tP/Qym3jmjf282Lvk+nWto7X6UQkQhyx0J1zswE7wjYOuCNYoUJSpXi4bCTUaEmPmc/RKDaT68b04+4Lk7nx9GZepxORCKArRYPJ54Nz/gmXvEaHwmVMqfo0QyfN4omPl+sMGBEpcyr0snDS1dg1E2hgmXxWtT9zvv6K/3tnIbn5OgNGRMqOCr2sNP8DduNUqsZGM7Hy0/y4dAbXjfxWd0ISkTKjQi9LdU7A+n5GTEJdxsc9S9Kmz7nyjbls363TGkUk+FToZS2xMdz4Kf56J/J69CBO2vkJvV6fy4ZMza0uIsGlQi8PlWvAdROxZmfwjG8I5+dM4rLX5rJi6++64FZE5JBU6OUlpgr0Hg+tuvOQG84N7r9cNXQeizft8jqZiIQJFXp5io6FK0bDib24o3Asd/kncM3weXy7fqfXyUQkDJTmSlEJJn80XDoUomK5cdFYomMc140sYvh1XTi9ZU2v04lICFOhe8Hnh56Dwefj2gWjiYp39B0Fw/uczBktQ3gWShHxlArdKz4fXPQSmI/e89+CysZNo4xh1yVz5vEqdRE5eip0L/l8cOFAcI7eC0aRVyWGm0bDiD7JeqcuIkdNhe41nw8uGgQF++mzZAz7q1Ti5tHw1g1dSDmuhtfpRCSEqNArAp8PLn4VCnK5ZfkI9lWuRN+3jDE3daVT4ySv04lIiNBpixWFPwouGw7Hn89dua9zedx39Bn5Lcu26EYZIlI6KvSKxB8Nl7+FNT6Fx/Jfolv0MvqM/JZ1GXu9TiYiIUCFXtFEx8HV72C1WvOCe54T3WquHfGt7lMqIkekQq+IYhPg2g/wVa3LiOjnSczZwLUjvmHXvjyvk4lIBaZCr6iq1Ia/vE+U38f7VV9kT+ZWbnzrO90kQ0QOS4VekdVoDle/S2xuBlNrv8LKTdu4+51FFOp2diJyCCr0iq5hMvQaQeKuNKY0GM2naT/w1OTlXqcSkQpIhR4KWl8If/o3TXd8ydgmn/DmnO8ZMXu916lEpILRhUWhouutsGMVp6WOpH+jBjw+GZrWiOfcNnW8TiYiFYTeoYcKM7jgOWh2FtdlDuSKWhu5c9xCVm7TXY9EpJgKPZT4o+GKUVhSU/6d/zzHVfqJvm+lsmPvfq+TiUgFoEIPNXFJ0HscvqJ8xie8wp69e7h1zHzyCoq8TiYiHlOhh6KaLeHSocRnLmNyswnM37CTJyaleZ1KRDymQg9VrS6Asx+i0aaJDDs+lbHzNjL+u41epxIRD6nQQ9mZf4dWF9Jt82BubPwjj3yYxqJNu7xOJSIeUaGHMp8PLhmCJTTk4ZznaVUtl9vGzCdTfyQViUgq9FAXlwhXjMa3L5NxNUaya18ud49fRJGmBxCJOCr0cFCvA1zwLFU2z+TdNrOZtWYHr85I9zqViJQzFXq46Hw9tL+Sdumvc2+rHQyctpqv03d4nUpEypEKPVyYwYUvYElNuWPnM5xU03HnOwvJ2KPxdJFIccRCN7ORZrbdzJYdZv3ZZpZlZosCj0eDH1NKJaYq9BqJLzuD0TVHsyc3n/veW6zxdJEIUZp36G8B5x9hm1nOuZMCjyeOPZb8bvU7QrfHqLL+U0a3X8ZXqzMYOUczM4pEgiMWunNuJrCzHLJIsKTcAc3PpcvqF7iuRS7PTl3Jsi1ZXqcSkTIWrDH0U8xssZl9YmYnHG4jM7vFzFLNLDUjIyNILy2/cuD89Oh4Hs1/idrxfv5v3EL25RV4nUxEylAwCn0B0MQ51wEYDHx4uA2dc0Odc8nOueRatWoF4aXlsKrWhR6DiPpxMe+0mcX6Hdn8e8pKr1OJSBk65kJ3zu12zu0NfD4FiDazmsecTI5d24uh/VU0WjqER07ax5h5G/hqtX4zEglXx1zoZlbXzCzweZfA98w81u8rQdL9Oahajxu2/5sTakVz/4TFZO3L9zqViJSB0py2OA6YC7Qys81m1tfMbjOz2wKb9AKWmdli4GXgKueczpOrKGIT4JIh+Hau5a0mn5G5N49HPjrkGagiEuKOeE9R51zvI6x/BXglaIkk+I47C5JvpFbqcP518inc/80PdG9Xl/NPrOd1MhEJIl0pGinOewISGnL55n/TsV4MD3+Yxk/ZeV6nEpEgUqFHipiq0PNlLHMNwxpPY9e+PJ6ctNzrVCISRCr0SNL8HOh0HTWXvMHjJ+fzwcItfLHyR69TiUiQqNAjzXlPQuXaXP3jANrUjuehD5axO1dnvYiEAxV6pIlLhO7P4du2hBGtv2P7nlyen7rK61QiEgQq9EjUpie06k79hQO5s3MMY7/ZwIKNP3mdSkSOkQo9EplB9+fBfPTbN4S6VWN46IOl5BcWeZ1MRI6BCj1SJTSEcx4hat10Xu+0kZXb9jB8lqbZFQllKvRI1uVmqNueDmnP0bNNVV6avppNO/d5nUpEficVeiTz+eHCF2HPVv5VfTI+M/p/nOZ1KhH5nVToka7RydCpD1UWDKN/ijFtxXamLde56SKhSIUu0O1xiE3gsm0DaVmrMv0npZGbX+h1KhE5Sip0gfjq0O1xfBvn8mr7dDbtzOG1L9d6nUpEjpIKXYp1vBbqd+T4JQPo1S6R175ay4bMbK9TichRUKFLMZ8PLnge9mylf9JUonzGU5NXeJ1KRI6CCl1+0ehk6NCbyvNf56GUGD5f/iOz1uiWdSKhQoUu/6vb4+CvRO+f3qBx9Xie+Hg5BbqCVCQkqNDlf1WtC2f+Hf+aT3ixcyZrtu9l7LwNXqcSkVJQocuvpdwOSc3ovHIAZzRP5MXPV7NTdzcSqfBU6PJrUTFw3hNYxgqeb76EvfsLeGnaaq9TicgRqNDl0Nr0gCanUTd1ANd3rsHYbzaSvn2v16lE5Deo0OXQzOBPT8O+HdwXP4m4aD/PfKLTGEUqMhW6HF79jtChN/Hz3+AfKbFMW7Gdr9fu8DqViByGCl1+27mPgvm5eu8oGiTG8dSkFRQWOa9TicghqNDlt1WrD6fcgT/tff7VNZ/lW3fzwYLNXqcSkUNQocuRnXYXxNfkzO9fokODarz4+WrNxihSAanQ5chiq8HZD2Ab5vBM+61szcrlzTnfe51KRA6iQpfS6Xw91GhBm6UDOK9VDYZ8mc5PuthIpEJRoUvp+KOL53nZsYqnmi4ie38Br8xI9zqViJSgQpfSa30RNOxCnfkDubpjLUbP/V43lRapQFToUnpmcF5/2LuNfyR9id9nDPhsldepRCRAhS5Hp8mpcPwFVE19hb92rc5Hi34g7Ycsr1OJCCp0+T3OfRT27+ZW30ckxEXz3FS9SxepCI5Y6GY20sy2m9myw6w3M3vZzNLNbImZdQp+TKlQ6rSFk64mZv5w7j8lnq9WZ2hKAJEKoDTv0N8Czv+N9RcALQOPW4DXjj2WVHhnPwjAlXvHUi8hlmenrsI5TQkg4qUjFrpzbiaw8zc2uRgY7YrNAxLNrF6wAkoFldgIutxM1NLxPNbVz+JNu5i6bJvXqUQiWjDG0BsAm0o83xxYJuHu9HsgujJ//HEYLWpX4fnPVun+oyIeKtc/iprZLWaWamapGRm6m3zIq1wDTrsT36pJPJm8n3UZ2XywYIvXqUQiVjAKfQvQqMTzhoFlv+KcG+qcS3bOJdeqVSsILy2eS/krxNckZd3LdGiYwMBpmrhLxCvBKPSJwHWBs11SgCzn3NYgfF8JBTFV4Kz7se9n8a/2GWzNymXsvA1epxKJSKU5bXEcMBdoZWabzayvmd1mZrcFNpkCrAPSgWHAX8ssrVRMna+HxMacsGIQZ7Sowasz0tmTm+91KpGIE3WkDZxzvY+w3gF3BC2RhJ6oGDj7IfjwNp46dy1npScybNZ67jnveK+TiUQUXSkqwdH+CqjVhiaLBnLRibUYMWsdmXv3e51KJKKo0CU4fH449xHIXMMjDReTk1/IqzPWep1KJKKo0CV4WnWHhidTZ8FArupYi7HzNrBlV47XqUQihgpdgseseOKu3Vv4R43ZYDDo89VepxKJGCp0Ca5mZ0Lzc0hIHcxNyTV4f8Fm0rfv8TqVSERQoUvwnfso5OykX+wnxEX7GfCp3qWLlAcVugRf/Y5wwp+Jn/8Gd6UkMDVtG4s27fI6lUjYU6FL2fjDw1CQy/WF71OjciWe/WSlptcVKWMqdCkbNVtAp2uptPAtHkiJZe66TGat0U0wRMqSCl3Kzln/AJ+fS7NG0TApjuc+XUlRkd6li5QVFbqUnWr1oett+JdNoH+XQpZt2c3kpZq3TaSsqNClbJ1+N8QmcM7m12lVpyoDPltFvm6CIVImVOhStuKS4Mz7sLXTeKbTTjZk7mPctxu9TiUSllToUvZOvhmqNeSkVYNIaZrIS9PWaHpdkTKgQpeyFx0L5/wT+2Ehz7RZT2Z2HsNmrvM6lUjYUaFL+Wh/JdQ+gaaLBtDzxJoMm7We7btzvU4lElZU6FI+fH44rz/8tJ7H6s0jv7CIQdPXeJ1KJKyo0KX8tOgGx51NjdSB9O2cxPjvNmniLpEgUqFL+TGD856EnF3cHfsx8dF+/jVlpdepRMKGCl3KV7320KE3cQuG8cAp8Xyxcjtz0jUlgEgwqNCl/J3zMJiPq/a8ScOkOJ6avIJCTQkgcsxU6FL+EhrAKf3wp73Pv7vksmLrbt6fv9nrVCIhT4Uu3jj9b1ClDqenv0CnRtUY8NkqsvcXeJ1KJKSp0MUbMVXg3MewLam82Dad7Xv28+qMdK9TiYQ0Fbp4p0NvqN+Rpgue46qTqjN81no2ZGZ7nUokZKnQxTs+H5z/DOz5gYcTpxHlN56avMLrVCIhS4Uu3mqcAidcSpXUV3ng1Mp8vvxHZq3J8DqVSEhSoYv3/vgkmHHNrtdpUiOe/h8v15zpIr+DCl28l9AQzrwP/8pJDOy8k/Ttexk5e73XqURCjgpdKoZT+kH15nRK+xfnt05i0LQ1/LArx+tUIiFFhS4VQ1QMXPAcZKbzTP1ZOBxPfLzc61QiIUWFLhVHy27Q+iISvxvEQ6dWYWraNmas2u51KpGQoUKXiuX8ZwDjLzsH07xmPI99lEZufqHXqURCggpdKpbERvCHh/Ct+ZRXO21h4859DJqmG2GIlEapCt3MzjezVWaWbmYPHGL99WaWYWaLAo+bgh9VIkbX26Bue1oveJI+HZMYNmsdy7ZkeZ1KpMI7YqGbmR94FbgAaAv0NrO2h9h0vHPupMBjeJBzSiTxR0GPlyB7Ow/Fvkf1ypW4f8ISnZsucgSleYfeBUh3zq1zzuUB7wAXl20siXgNOkGXW4hZ+CaDT9vP8q27GTZrndepRCq00hR6A2BTieebA8sOdpmZLTGzCWbW6FDfyMxuMbNUM0vNyNDl3XIE5zwMCY1IWfIIPdoUn5uevn2v16lEKqxg/VH0Y6Cpc6498Dkw6lAbOeeGOueSnXPJtWrVCtJLS9iKqQoXD4ada3m2+kTiK/m5591FGnoROYzSFPoWoOQ77oaBZT9zzmU65/YHng4HOgcnnkS8486G5BuJn/86Q87IZ8nmLIbMWOt1KpEKqTSF/h3Q0syamVkl4CpgYskNzKxeiac9Ac2BKsFz3hOQ0JBTlz1Kr/Y1GPzFGpZs3uV1KpEK54iF7pwrAPoBn1Jc1O8659LM7Akz6xnY7E4zSzOzxcCdwPVlFVgiUExV6DkYMtN5usp71KhSiXveXawLjkQOYs55c7f15ORkl5qa6slrS4ia+iDMG8LSs4bT49N4/pLSmKcuaed1KpFyZWbznXPJh1qnK0UldJz7GNQ+gXapD3L3KYmMnbeRKUu3ep1KpMJQoUvoiI6Fy4ZD7m7u3DOIkxom8I8JS9i0c5/XyUQqBBW6hJY6beG8J/Clf8abbVLBoN+4heQV6FRGERW6hJ6ut0KrC0ma8yTD/lDI4k27eHqy5k4XUaFL6DGDS4ZAtQakzL+Pu1ISGTV3A++lbjry14qEMRW6hKa4RLhiNGTv4K7dAzi9eSL//HAZizfp/HSJXCp0CV31T4Luz+Fb9wXDGn5K7aox3DpmPtv35HqdTMQTKnQJbZ36QKc+xH3zEuNTNrIrJ4+bR6WyL6/A62Qi5U6FLqHNDLoPgCan02Dm/Yz+o4+lW7K4c9wiCou8uWhOxCsqdAl9UZXgyjFQrR5d5vXjuW7VmbbiR/p/nIZXV0KLeEGFLuEhvjr0Hg8FufRacSd3nlKd0XM38NpXmplRIocKXcJH7dbQexz8tIG//fhPerVL5Lmpq3hzznqvk4mUCxW6hJemp0OvkdjWhTxXOIDubavT/+PlvP3NRq+TiZQ5FbqEnzYXQY+X8a37gsHRr3Du8Yn888OlvKsLjyTMRXkdQKRMdLoW8rLxT/0HQ1sWcFPzftw/YQnZ+wu44bRmXqcTKRMqdAlfKbeBz49/yn0Mb1HInW3uof/Hy8nKyeeuc1tiZl4nFAkqFbqEty43gy8K/6S7Gdwkh6STHmTQtDXszM7j0YvaEuXXqKOED/1rlvCXfAP8eSi+Td/wZOa93Nc1ntFzN3DDW9+RlZPvdTqRoFGhS2TocCX85X1s91b6rb2Vod18zF2byZ+HzGH9jmyv04kEhQpdIsdxZ0HfT8FfiT/O7cOnZ33PT9l59Bw8m0lLfvA6ncgxU6FLZKndBm6eAY1TaD73QWa1/oA2taLp9/ZCHvrvUnLzC71OKPK7qdAl8lSpBdf+F878O1WWj2O8PchjnfN4+5uN9Bg8m4Ubf/I6ocjvokKXyOTzwzkPwzXvY7lZ3LC8L192nkNOTg6XvfY1T09eTk6e3q1LaFGhS2Rr2Q3+OhfaX0HTtFf5KuExHmqTwbBZ6/njoK/4ZOlWzdgoIUOFLhKXBH9+HXq/g79gHzetu4v5x4+mmT+T2/+zgCuHzmPZliyvU4ockQpd5IBWF8Ad38IfHqbGD18xKvt2Pm35IXt+/J6LBs/mltGppP2gYpeKy7z6dTI5OdmlpqZ68toiR5S1GWYOgIVjcWYsrdWDh7adwbLc2nRrU4ebzmhG12bVNX2AlDszm++cSz7kOhW6yG/YtRFmvQCL3obCPNYnncbzu87mk5w2tK6XyA2nNuXC9vWoHKNZNKR8qNBFjtXe7ZA6Er4bDtkZZMfW5SN3JkN3d2V7dEO6t6tHr84N6dK0Oj6f3rVL2VGhiwRLwX5YOQkWjcOtnY65In6IbcH7OZ2YmJdMVuXj+NOJ9bjgxLqc3Kw60Zr8S4JMhS5SFnZvhbQPYPlE3KZvMByZUbWZlteOLwrasSy6HSe2aMrZrWpzavMaNK4erzF3OWYqdJGytmcbrJoC6dNx677E8vYCsNYa8XV+KxYVtWBL5bbUaXYiHZtUp0OjRNrWr0ZMlN/j4BJqVOgi5akwHzanwoY5uA1f4zbOw5dfPKPjXuJYXtSYlUWNWWONyU1oSXz9NjRq2Ijj61ajRe0q1EuI1Tt5OSwVuoiXigphx2rYsgC3ZQH5PyzFtqcRXbD35012ucp87+qwydVmq68uuZUb4KvWgJiajalWuwk1a9amXmI89RJiSYyPVuFHsN8q9FKda2Vm5wMvAX5guHPumYPWxwCjgc5AJnClc+77YwktEjZ8/uJZHmu3wTpeQyUA54pPidyxBjLXELN1BU23r6V51gbi932Hf18h7AO2FX+L/S6aDBL43iXyE9XIjk4ir1ISRbFJEJeEv3Ii0fFJVKqcRFzVROKqJBJfLYkqlatQJTaaKrFRGt6JAEcsdDPzA68C5wGbge/MbKJzbnmJzfoCPznnWpjZVcCzwJVlEVgkLJhBUpPiR8tuxAFxB9YVFsDebZC1haKszWTv2ETOzi3Yrq3Uzs6gfm4msXkbqZyzi6icAviNySELnZFNLLuIYR+x7LcY8i2WfF8MBb4YCn0xFPpjKPLF4KJicL5KuKhK4K8E/hgsKhrzV4KoSvj80Zg/Cp+/EuaPxhcVXfzRH3XQIxqf34/5/fj90ZjPH1jux8yP3x+Fz+cLrPdjPh9mPvz+qOJtzYf5ffh9fsxn+HzF2/h8fgwr/lpDv6UcQmneoXcB0p1z6wDM7B3gYqBkoV8MPB74fALwipmZ06xGIkfPHwUJDSGhIT66UhWoeqjtnIP8fZDzE27fTvZn7yI7ayc5e3aSv283+Tm7KczZg8vLxuVlY/n78BXkEFOQQ3xhLv6iXUQV7ic6fz9RLp9ol0c0+USTTxRF5bzTR6fIGUVAET4c4H7+aIEHgFEUeA6Gs1/WH/DL9vbz84M//mr7n3+QHHgdSmzzv9sf+utg63FXkPKXx47xv8KvlabQGwCbSjzfDHQ93DbOuQIzywJqADtKbmRmtwC3ADRu3Ph3RhYRoPhdfqXKUKkyltCQWCA2WN+7qBAK83CFeRTk55G/P5eC/Hzy8/dTWJBHYX4BBYV5FOXnUVhYQFFBAUWFeRQVFlJUVIArLKCosACKCikqKsQVFuBcEa6oMLCsCA48dw7nipfjiopntywqBBy4op+XWeBzHDhXXOc4h7nAdoFKx5X46AI/mAIfDffL7Jnulx8BP39N8YrA9+SX7xlYjjtom//xy48S5/637g983YFlUdXqBOEg/Vq5Xq/snBsKDIXiP4qW52uLyFHw+cEXh0XHER0L0Yf8FUEqmtJcxrYFaFTiecPAskNuY2ZRQALFfxwVEZFyUppC/w5oaWbNzKwScBUw8aBtJgJ9Ap/3Ar7Q+LmISPk64pBLYEy8H/ApxactjnTOpZnZE0Cqc24iMAIYY2bpwE6KS19ERMpRqcbQnXNTgCkHLXu0xOe5wOXBjSYiIkdDU8GJiIQJFbqISJhQoYuIhAkVuohImPBstkUzywA2/M4vr8lBV6FGiEjc70jcZ4jM/Y7EfYaj3+8mzrlah1rhWaEfC87A8gQAAAO/SURBVDNLPdz0keEsEvc7EvcZInO/I3GfIbj7rSEXEZEwoUIXEQkToVroQ70O4JFI3O9I3GeIzP2OxH2GIO53SI6hi4jIr4XqO3QRETmICl1EJEyEXKGb2flmtsrM0s3sAa/zlAUza2RmM8xsuZmlmdldgeXVzexzM1sT+JjkddayYGZ+M1toZpMCz5uZ2TeBYz4+MI1z2DCzRDObYGYrzWyFmZ0SCcfazP4W+Pe9zMzGmVlsOB5rMxtpZtvNbFmJZYc8vlbs5cD+LzGzTkfzWiFV6CVuWH0B0BbobWZtvU1VJgqAe51zbYEU4I7Afj4ATHfOtQSmB56Ho7uAFSWePwsMdM61oPiWyH09SVV2XgKmOudaAx0o3vewPtZm1gC4E0h2zp1I8dTcB24wH27H+i3g/IOWHe74XgC0DDxuAV47mhcKqUKnxA2rnXN5wIEbVocV59xW59yCwOd7KP4fvAHF+zoqsNko4BJvEpYdM2sIXAgMDzw34ByKbz4OYbbfZpYAnEnxPQVwzuU553YRAcea4um74wJ3OYsHthKGx9o5N5Pi+0SUdLjjezEw2hWbBySaWb3SvlaoFfqhbljdwKMs5cLMmgIdgW+AOs65rYFV24CyudOstwYB98PPt52vAexyzhUEnofbMW8GZABvBoaZhptZZcL8WDvntgADgI0UF3kWMJ/wPtYlHe74HlPHhVqhRxQzqwK8D9ztnNtdcl3gFn9hdc6pmV0EbHfOzfc6SzmKAjoBrznnOgLZHDS8EqbHOonid6PNgPpAZX49LBERgnl8Q63QS3PD6rBgZtEUl/l/nHMfBBb/eODXr8DH7V7lKyOnAT3N7HuKh9POoXh8OTHwazmE3zHfDGx2zn0TeD6B4oIP92PdDVjvnMtwzuUDH1B8/MP5WJd0uON7TB0XaoVemhtWh7zAuPEIYIVz7sUSq0rejLsP8FF5ZytLzrkHnXMNnXNNKT62XzjnrgFmUHzzcQiz/XbObQM2mVmrwKJzgeWE+bGmeKglxcziA//eD+x32B7rgxzu+E4Erguc7ZICZJUYmjky51xIPYDuwGpgLfBPr/OU0T6eTvGvYEuARYFHd4rHk6cDa4BpQHWvs5bhf4OzgUmBz48DvgXSgfeAGK/zBXlfTwJSA8f7QyApEo410B9YCSwDxgAx4XisgXEU/50gn+LfyPoe7vgCRvGZfGuBpRSfBVTq19Kl/yIiYSLUhlxEROQwVOgiImFChS4iEiZU6CIiYUKFLiISJlToIiJhQoUuIhIm/h8PbdY7P7ei1QAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 16ms/step - loss: 0.0026 - accuracy: 1.0000\n",
            "loss= 0.002641375409439206\n",
            "acc= 1.0\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAMs0lEQVR4nO3dX2id93nA8e9jy6ZyulaFiICcMPti9RQCw0UMr4Eylg53a6hzsYtsSaEZwzC2Nu26lMo3vWpTcCnNxQgYd4URs124wisj1B1svRjMpnKc4tVnhpKuif+Unl7YLcVDFnp6IbmW1aPoVX2O3qd+v58rWzqRHl6iL8c/nfM+kZlIkura1vYAkqS3Z6glqThDLUnFGWpJKs5QS1JxY6P4og8++GDu2bNnFF9aku5L586d+0lmTg763EhCvWfPHubn50fxpSXpvhQRP1zvcx59SFJxhlqSijPUklScoZak4gy1JBU3kld9SPejU+evcPT0Ja5ev8nUxDgvHNzHU/t3tz2WOsBQSw2cOn+F2bkLTC/2OLS9x5kb08zOLQAYa42coZYaOHr6EtOLPU7s/AI7WOQWYzyzcISjp3caao2cZ9RSA1ev3+TAth47WGQsltjBIge29bh6/Wbbo6kDDLXUwNTEOGeWprnFGIu5jVuMcWZpmqmJ8bZHUwd49CE18MLBfczOLfDMwhEObOtxZmma3tg0Lx7c1/Zo6gBDLTVw+xz66OmdvHz9vUxNjPOir/rQFjHUUkNP7d9tmNUKz6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqzlBLUnGGWpKKM9SSVFyj25xGxKeAvwISuAA8l5n/P8rBpNvc/q2u2/AZdUTsBj4BzGTmY8B24OlRDybBne3fD934Ln+9/V956MZ3mZ27wKnzV9oeTdoyTRcHjAHjEXEL2AVcHd1I0h1u/5YaPKPOzCvAl4A3gWvAjcz81trHRcThiJiPiPl+vz/8SdVJbv+Wmh19vAc4BOwFpoAHIuLZtY/LzGOZOZOZM5OTk8OfVJ3k9m+p2dHHB4EfZGYfICLmgPcDr4xyMAnc/i1Bs1C/CRyIiF3ATeAJYH6kU0kr3P4tNQh1Zp6NiJPAa8AicB44NurBpNvc/q2ua/Sqj8z8HPC5Ec8iSRrAdyZKUnGGWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1JxRlqSSrOUEtScYZakooz1JJUnKGWpOIMtSQV13RnojrI7d9SDYZaA93e/j292OPQ9h5nbkwzO7cAYKylLWaoNZDbv6U6PKPWQG7/luow1BrI7d9SHR59aCC3f0t1GGoN5PZvqQ5DrXW5/VuqwTNqSSrOUEtScYZakooz1JJUnKGWpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBXX6DanETEBHAceAxL4y8z871EO1mVu/5a0WtP7Ub8EfDMz/ywidgK7RjhTp7n9W9JaG4Y6It4NfAD4GEBmLgALox2ru9z+LWmtJmfUe4E+8LWIOB8RxyPigbUPiojDETEfEfP9fn/og3aF278lrdUk1GPA+4CXM3M/8HPgs2sflJnHMnMmM2cmJyeHPGZ3uP1b0lpNzqgvA5cz8+zK308yINQaDrd/S1prw1Bn5o8i4q2I2JeZl4AngIujH62b3P4taa2mr/r4OHBi5RUfbwDPjW4kuf1b0mqNQp2ZrwMzI55FkjSA70yUpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqrultTjvDDeCSqjHUq7gBXFJFhnoVN4BLqsgz6lXcAC6pIkO9ihvAJVXk0ccqbgCXVJGhXsUN4JIqMtRruAFcUjWeUUtScYZakooz1JJUnKGWpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpuMa3OY2I7cA8cCUznxz2IG7/lqTBNnM/6ueBHvCuYQ/h9m9JWl+jUEfEw8CHgc8DfzfsIdz+LUnra3pG/RXgM8DSeg+IiMMRMR8R8/1+f1NDuP1bkta3Yagj4kngx5l57u0el5nHMnMmM2cmJyc3NYTbvyVpfU2OPh4HPhIRfwq8A3hXRLySmc8Oawi3f0vS+jYMdWbOArMAEfGHwN8PM9Lg9m9JejtltpC7/VuSBttUqDPz28C3RzKJJGkg35koScUZakkqzlBLUnGGWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1JxRlqSSrOUEtScYZakooz1JJUXJnbnErSb6pT569w9PQlrl6/ydTEOC8M+X76hlqS7sGp81eYnbvA9GKPQ9t7nLkxzezcAsDQYm2oJekeHD19ienFHid2foEdLHKLMZ5ZOMLR0zuHFmrPqCXpHly9fpMD23rsYJGxWGIHixzY1uPq9ZtD+x6GWpLuwdTEOGeWprnFGIu5jVuMcWZpmqmJ8aF9D48+JOkevHBwH7NzCzyzcIQD23qcWZqmNzbNiwf3De17GGpJuge3z6GPnt7Jy9ffy9TEOC/6qg9JquWp/buHGua1PKOWpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqzlBLUnEb3uY0Ih4B/gl4CEjgWGa+NOrBJGkjo97+XUWT+1EvAp/OzNci4reAcxHx75l5ccSzSdK6tmL7dxUbhjozrwHXVv78s4joAbsBQy2pNVux/buKTZ1RR8QeYD9wdsDnDkfEfETM9/v94UwnSevYiu3fVTQOdUS8E/g68MnM/Onaz2fmscycycyZycnJYc4oSb9iK7Z/V9FoZ2JE7GA50icyc260I0nSxrZi+3cVTV71EcBXgV5mfnn0I0nSxrZi+3cVTZ5RPw58FLgQEa+vfOxIZr46urEkaWOj3v5dRZNXffwXEFswiyRpAN+ZKEnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqzlBLUnGGWpKKM9SSVFyjxQGStFZXNoBXYKglbVqXNoBXYKglbVqXNoBX4Bm1pE3r0gbwCgy1pE3r0gbwCjz6kLRpXdoAXoGhlrRpXdoAXoGhlvRr6coG8Ao8o5ak4gy1JBVnqCWpOEMtScUZakkqzlBLUnGGWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1JxRlqSSrOUEtScY1ucxoRHwJeArYDxzPziyOdStK63P7dPRuGOiK2A/8A/DFwGfhORHwjMy+OejhJd3P7dzc1eUb9+8D3M/MNgIj4F+AQYKilLeb2725qcka9G3hr1d8vr3zsLhFxOCLmI2K+3+8Paz5Jq7j9u5uG9svEzDyWmTOZOTM5OTmsLytpFbd/d1OTo48rwCOr/v7wysckbTG3f3dTk1B/B/idiNjLcqCfBv5ipFNJGsjt3920YagzczEi/hY4zfLL8/4xM7838skkDeT27+5p9DrqzHwVeHXEs0iSBvCdiZJUnKGWpOIMtSQVZ6glqbjIzOF/0Yg+8MNf8z9/EPjJEMf5Tea1uJvX425ejzvuh2vx25k58N2CIwn1vYiI+cycaXuOCrwWd/N63M3rccf9fi08+pCk4gy1JBVXMdTH2h6gEK/F3bwed/N63HFfX4tyZ9SSpLtVfEYtSVrFUEtScWVCHREfiohLEfH9iPhs2/O0KSIeiYj/jIiLEfG9iHi+7ZnaFhHbI+J8RPxb27O0LSImIuJkRPxvRPQi4g/anqlNEfGplZ+T/4mIf46Id7Q907CVCPWqBbp/AjwK/HlEPNruVK1aBD6dmY8CB4C/6fj1AHge6LU9RBEvAd/MzN8Ffo8OX5eI2A18ApjJzMdYvhXz0+1ONXwlQs2qBbqZuQDcXqDbSZl5LTNfW/nzz1j+QezsDYgj4mHgw8DxtmdpW0S8G/gA8FWAzFzIzOvtTtW6MWA8IsaAXcDVlucZuiqhbrRAt4siYg+wHzjb7iSt+grwGWCp7UEK2Av0ga+tHAUdj4gH2h6qLZl5BfgS8CZwDbiRmd9qd6rhqxJqDRAR7wS+DnwyM3/a9jxtiIgngR9n5rm2ZyliDHgf8HJm7gd+DnT2dzoR8R6W//W9F5gCHoiIZ9udaviqhNoFumtExA6WI30iM+fanqdFjwMfiYj/Y/lI7I8i4pV2R2rVZeByZt7+F9ZJlsPdVR8EfpCZ/cy8BcwB7295pqGrEupfLtCNiJ0s/zLgGy3P1JqICJbPIHuZ+eW252lTZs5m5sOZuYfl/y/+IzPvu2dMTWXmj4C3IuL22vEngIstjtS2N4EDEbFr5efmCe7DX6422pk4ai7Q/RWPAx8FLkTE6ysfO7Kyu1L6OHBi5UnNG8BzLc/Tmsw8GxEngddYfrXUee7Dt5P7FnJJKq7K0YckaR2GWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1Jxf0CnP++rG/VJ1oAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P3aO3WEwA-lR"
      },
      "source": [
        "## stacked"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xIYi9GUDwqA6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 893
        },
        "outputId": "86877aab-bfbd-4995-853e-4d320e74eaed"
      },
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras.layers import Dense, RepeatVector, Flatten\n",
        "from tensorflow.keras.layers import Bidirectional, LSTM, GRU\n",
        "\n",
        "input_shape = train_x.shape[1:]\n",
        "# input_shape = (5,10)\n",
        "\n",
        "model = keras.Sequential()\n",
        "model.add(LSTM(128, input_shape=input_shape, return_sequences=True))\n",
        "model.add(LSTM(128))\n",
        "model.add(RepeatVector(1))\n",
        "model.add(Dense(CATEGORY_NUM, activation=\"softmax\"))\n",
        "model.compile(loss=\"categorical_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n",
        "model.summary()\n",
        "\n",
        "history = model.fit(train_x, train_y, epochs=100, verbose=1, validation_split=0.1, batch_size=128)\n",
        "\n",
        "\n",
        "plt.plot(history.history['loss'])\n",
        "plt.plot(history.history['val_loss'])\n",
        "plt.show()\n",
        "\n",
        "\n",
        "\n",
        "loss, acc = model.evaluate(test_x, test_y)\n",
        "print(\"loss=\", loss)\n",
        "print(\"acc=\", acc)\n",
        "\n",
        "\n",
        "\n",
        "y_ = model.predict(test_x)\n",
        "predicted = np.argmax(y_, axis=-1)\n",
        "test_y_index = np.argmax(test_y, axis=-1)\n",
        "\n",
        "plt.plot(test_y_index, \"o\")\n",
        "plt.plot(predicted, \".\")\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_4\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "lstm_2 (LSTM)                (None, 5, 128)            71168     \n",
            "_________________________________________________________________\n",
            "lstm_3 (LSTM)                (None, 128)               131584    \n",
            "_________________________________________________________________\n",
            "repeat_vector_4 (RepeatVecto (None, 1, 128)            0         \n",
            "_________________________________________________________________\n",
            "dense_4 (Dense)              (None, 1, 10)             1290      \n",
            "=================================================================\n",
            "Total params: 204,042\n",
            "Trainable params: 204,042\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd5xU9b3/8dfnzMw2ysLKImWRRSkKEZGsCthALKiJehNbolGTKPEnRmxJbpJfzFUfN4k/jTGW6EWsoKIUySIYlBJpUhZEmiIdFilLr9tmvr8/ZuQuSFnY2T07M+/n4zGPPW3nvA/HvDN75hRzziEiIonP8zuAiIjEhwpdRCRJqNBFRJKECl1EJEmo0EVEkkTQrxU3a9bM5efn+7V6EZGENHfu3C3OudzDzfOt0PPz8ykqKvJr9SIiCcnM1hxpng65iIgkCRW6iEiSUKGLiCQJFbqISJJQoYuIJAkVuohIklChi4gkiYQr9DVb9/LU+KV8umIrZZVhv+OIiNQbvl1YdKKK50/gqul/YM7Ujgz3zqCy9Xl0Pv0MLmjfjM4tG+N55ndEERFfJFyhn39aMyrX5dOxeDq3V34MG2Dt+lxmTziDEcEuRNpeQJfOZ3JBx+a0bpLpd1wRkTpjfj2xqKCgwNXo0v9wJWxaCGtmULpiGrZ2BunlOwAods2YFTmDZQ0KyOh4Ced07cw5+TmkBRPuCJOIyEHMbK5zruCw8xK20A/lHJR8iVs1lT1L/01w3XQyK6IF/2WkDTPsbHa16UO7sy+hd+fWZGeG4rduEZE6khqFfqhIBDYtpPyriexe/BHZJXMIukp2uSw+cd1Y27wvLQu+T9+zTlW5i0jCSM1CP1TZbiIrJrPtszFkrvqIBpU72O/SmOy6s6b11bTvdR0Xn5GnwzIiUq+p0A8VrsStncGW2cPJWlZIg8odbHcNGe9dyN4zb+Py3r1pk5PlTzYRkaNQoR9NuILKZRPZMuNNTlr7ESEqmBPpRFHzH3LWFbfTs/3JmOlUSBGpH1To1bV3K7tmvk5kzqs0KS1mXSSXsQ1/wCl9f0G/s0/VOe4i4jsV+vGKRChf8gG7JjxJsx0LKHGNeS/zZtpfeS+Xdz1Fn9hFxDdHK3R9A3g4nkfad66h2cApRG4fh+WezoDSQXQZdQl/++vjFK3a6ndCEZFvUaEfjRleu/NpNuAjwj8eSVaTk3lwz1+JvHYlT7w+go07S/1OKCJygAq9OswIdLyUnIHTKLv6Wb6TtomHVt3F+Kd/xrAZX+LXYSsRkapU6MfD80g/53ayHvyM/Wfewu02loJ//Qd/eHEoxdv3+Z1ORFKcCv1EZOXQ6PrncT8ZTavMSv64aSAjn3mAcQvW+51MRFKYCr0G7LQ+ZA2cRUXH7zHQhpE2/Bb+MnoW5ZURv6OJSApSoddUVg5ZP36DyiueoE9gATfMu52H/vEeJbvL/E4mIilGhR4PZgR73k3gjjHkZZbx563389hzL7F88x6/k4lIClGhx1P++aTfM5VgThv+Wv4Yz//jGWav2uZ3KhFJESr0eMvOI+Ou8bgWZ/FX/sqoV55g0peb/E4lIilAhV4bsnJI/9kYIvkX8ZfgS0wa+iSTv9zsdyoRSXIq9NqS1oDQrcOpOPVSHgsOpnDoc0xeqlIXkdqjQq9NwTRCNw8hkteDJ4MvMGzoIGas2OJ3KhFJUscsdDNrY2aTzWyJmS02s4GHWcbM7FkzW25mC8yse+3ETUBpWQRvfQ9anMlzgWd4achbLNu02+9UIpKEqvMJvRJ4yDnXGegBDDCzzocscyXQIfbqD7wY15SJLqMxwdvex5q04Rme4jevjmXzbt3YS0Ti65iF7pzb4JybFxveDXwBtD5ksWuBN13UTKCJmbWMe9pElpVD6JZ3yU6DP5X+N/e8NoW9ZZV+pxKRJHJcx9DNLB84G5h1yKzWwLoq48V8u/Qxs/5mVmRmRSUlJceXNBnkdiRw42t08oq5q+T/8buRn+tOjSISN9UudDNrCIwE7nfO7TqRlTnnBjnnCpxzBbm5uSfyFomv/aXY5f/NFYE5tF78EkNnrfU7kYgkiWoVupmFiJb5W865UYdZZD3Qpsp4XmyaHE6P/4Pr8gMeCo2g8IN/sqB4h9+JRCQJVOcsFwNeAb5wzj19hMUKgdtiZ7v0AHY65zbEMWdyMcO+9zfIzuPvwef41dCp7NxX4XcqEUlw1fmEfj7wE+ASM5sfe11lZneb2d2xZcYBK4HlwMvAPbUTN4lkNiFw/Su0tK38cu8L/GH0Qr8TiUiCCx5rAefcNOCoj7l30W/2BsQrVMpocy7W53d8b9LjTFr0Lh8ubMmVZ+rkIBE5MbpS1G8XPECkzXk8mj6Up9+fxtY9uo+6iJwYFbrfvADeNc/TwMp5oHIwj/xzsd+JRCRBqdDrg9yOeL1/zVXeTCoWFzJ2gb5PFpHjp0KvL86/H9e8C39Of4Onx8xmj64iFZHjpEKvLwIh7LoXyGEHt+8fwnMTl/mdSEQSjAq9Pml1NnbOndwanMjU6Z+wfLPuyigi1adCr296/xYysvmv0Js8MnqR7vUiItWmQq9vsnLwLvk957KYRqv/xdiF+oJURKpHhV4fffenuOadeTTjbZ75cCHllRG/E4lIAlCh10eBINbvL7SIbObyXSN5Z7buyCgix6ZCr69OvRjX6UoGpH3A6xPm6TRGETkmFXo9Zpc8Qpbbx03loxg8daXfcUSknlOh12cnd8a63sjPQuMZPaWILbrPi4gchQq9vuv9W0IW4S43kucnLfc7jYjUYyr0+i6nHfbdO7g5MJmps2ezeVep34lEpJ5SoSeCi36FBUPcayMYNEXH0kXk8FToiaBRC7xz7+LawAymzZqle6aLyGGp0BNFz19CII2f8T6Dp63yO42I1EMq9ETR6GS8gjv4YWAaE2bMZvvecr8TiUg9o0JPJOcPxLwAd0RG89p0fUoXkYOp0BNJ41Z43W/lxuAUPpwxj33lunpURP6XCj3RnH8/QXP8uPJ9hhcV+51GROoRFXqiadoWO+smfhz8N8Onzicc0f3SRSRKhZ6Iet1HOmX03VXI+MUb/U4jIvWECj0RNT8d17EfPw19xGufLNFTjUQEUKEnLDv/fpqwm9M3FFK0ZrvfcUSkHlChJ6q2PQnnncvdobG8/O9lfqcRkXpAhZ7AAhfcT2tKSF9WyNqt+/yOIyI+U6Enso5XUpnTgV8EPuDNGbrQSCTVqdATmecRPP9evuOtZsXc8ezVY+pEUpoKPdF1vYmK9Bx+HB7DqM/W+51GRHykQk90oUyC593JZYF5TJg6XacwiqQwFXoSsHPvIuylcdnOEUxfvtXvOCLiExV6MmjYHLreyPXBqQyfMt/vNCLik2MWupm9amabzWzREeb3NrOdZjY/9nok/jHlWAK97iWDctqsepd123QKo0gqqs4n9NeBfsdYZqpzrlvs9VjNY8lxa34GpW37cHtgPO98qguNRFLRMQvdOTcF2FYHWaSGMi68l1zbye6i9yitCPsdR0TqWLyOofc0s8/N7EMz63Kkhcysv5kVmVlRSUlJnFYtB5zWl33ZHbg5PIaxn3/tdxoRqWPxKPR5QFvn3FnAc8DoIy3onBvknCtwzhXk5ubGYdVyEDMyL7yXLt4aPpv6gd9pRKSO1bjQnXO7nHN7YsPjgJCZNatxMjkhdtZNlIaacvG24Sws3ul3HBGpQzUudDNrYWYWGz439p46GdovoUzsnJ/R15vH2H9P8zuNiNSh6py2+A7wKdDJzIrN7OdmdreZ3R1b5HpgkZl9DjwL3Ox0uaKv0nv2J+IFaP3VG+zYV+53HBGpI8FjLeCc+9Ex5j8PPB+3RFJzjVqwp8N1/GDpGEbOXMJtl3TzO5GI1AFdKZqkmvQZSAMrY++nrxLRg6RFUoIKPVm17EpJs/O4tnwM07/a4HcaEakDKvQk1qTv/bSybXwxaajfUUSkDqjQk1ioUz+2ZZxCj43vsH677u8ikuxU6MnM87Ce99DVW8mUCWP8TiMitUyFnuSa9ryNPV4jTl7yCmWVur+LSDJToSe7tAZsO+NWekdmM2XWbL/TiEgtUqGngLzLBxK2ABXTdLmASDJToacAL7slq1pdTZ99H7Fk+Wq/44hILVGhp4hWV/6KTCtnzUfP+h1FRGqJCj1FNGxzJksb9eCcTSPYtnOX33FEpBao0FNIVu8HaGY7WTB2kN9RRKQWqNBTSJvuV7Aq1J78Za8SDusURpFko0JPJWbsOvsX5Lv1zJ/4rt9pRCTOVOgp5juX3cEGyyVrjk5hFEk2KvQUEwilsbrDHZxRsZgV8yb6HUdE4kiFnoI6X30vO1xD9k1+2u8oIhJHKvQUlJ3dhPktb+DM3dPYunqh33FEJE5U6Cmq3dUPUupCfD3uSb+jiEicqNBTVNs2pzCj8ZV02jyW0m3FfscRkThQoaewxpc8QMCFWfXBU35HEZE4UKGnsO92O5upaRfRduUwInu3+x1HRGpIhZ7CzIzI+QPJYj8r//V3v+OISA2p0FPchRf2YYbXndzFr0LFfr/jiEgNqNBTXCjgsa3bPWRHdlI86WW/44hIDajQhd6XX8d8OpIx5wUIV/odR0ROkApdaJgRYnnHu2hWuZEts97xO46InCAVugBw4dW3sszlUTnlaYhE/I4jIidAhS4AnJydxWen3EGL0pXs+HyM33FE5ASo0OWA875/F8WuGXsmPgnO+R1HRI6TCl0OaNu8CZ+efAt5exayZ+knfscRkeOkQpeDnPn9AZS4xmwd/xe/o4jIcVKhy0FOb3MynzS9nrbbP2X/2rl+xxGR46BCl2857eqB7HKZbPjgz35HEZHjcMxCN7NXzWyzmS06wnwzs2fNbLmZLTCz7vGPKXXp7A75TG50DfmbJ1C2canfcUSkmqrzCf11oN9R5l8JdIi9+gMv1jyW+K1Vv4cod0HWFOpTukiiOGahO+emANuOssi1wJsuaibQxMxaxiug+KOgS0f+3aAf7b4eQ9m2tX7HEZFqiMcx9NbAuirjxbFp32Jm/c2syMyKSkpK4rBqqS1mxkmXPYy5CCv/+YTfcUSkGur0S1Hn3CDnXIFzriA3N7cuVy0noKDbWUzL7E3+muGU79L/AYvUd/Eo9PVAmyrjebFpkuDMjKxLHiaTMr4q1MOkReq7eBR6IXBb7GyXHsBO59yGOLyv1APnnNOLmWk9OWX5UMr37vQ7jogcRXVOW3wH+BToZGbFZvZzM7vbzO6OLTIOWAksB14G7qm1tFLnzIxQ74dozF4WFf7N7zgichTBYy3gnPvRMeY7YEDcEkm9073npcyf3I22S1+jrPRXpGc08DuSiByGrhSVYzIzvAsf4iR2ML/wBb/jiMgRqNClWs684Ht8FepEmyUvU1pW5nccETkMFbpUi3ke4V4P0orNzC4c5HccETkMFbpU2+kX38DaYD55i19if1mF33FE5BAqdKk28wKU93yAUylmauFrfscRkUOo0OW4tO/zEzYGW9Nm8T/Yvb/c7zgiUoUKXY6PF6Ci50DOYBWTPnjb7zQiUoUKXY5bm4t/yrZAc9os+gc79uqMF5H6QoUuxy+YRnnPX9LdljJu7Ei/04hIjApdTkiLi+9iV6Ap+YtfYPOuUr/jiAgqdDlRoUwqe95HL1vE6EJ9ShepD1TocsJyLvoFe4JNOOOrl1i9Za/fcURSngpdTlxaA+h5Hxd6CxhV+L7faURSngpdaqThBb9gXzCbbqteZvHXul+6iJ9U6FIz6Q3xev2SSwLzGf7PQr/TiKQ0FbrUWEavX1AabMyFX7/C9OVb/I4jkrJU6FJzGY0JXDCQvoHPGFX4PpGI8zuRSEpSoUtchHreTVlaDtdtf50xC772O45ISlKhS3ykNyR08YNcGFjEx+NGUlYZ9juRSMpRoUvceOfeSVlmc35S+hZDZqz2O45IylGhS/yEMknv82vO875k9qRRbN+r2+uK1CUVusRX99uoaNiaeyNv8+yEpX6nEUkpKnSJr2A6oUv/QFdvJVvnvMfyzbv9TiSSMlToEn9db6Qytwu/Dg7jiQ8+9zuNSMpQoUv8eQGCVzxGHpvJWzGMT74q8TuRSEpQoUvtOK0vkfyLuT80mqfHzKEiHPE7kUjSU6FL7TDDu/wxstnNFdvf4fXpq/1OJJL0VOhSe1p1gzNv5K7gh4yYMJVNerKRSK1SoUvtuuxRAqEQDzOEP437wu80IklNhS61q3ErvIse5jJvDtsW/IuZK7f6nUgkaanQpfb1GECkaTseTx/Co6PnU16pL0hFaoMKXWpfKAOv35/Jd+vptXUU//PJCr8TiSQlFbrUjY79oP2lPJw+ivcmzWJFyR6/E4kkHRW61A0zuOpJMrwIfwy9wW9HLdSDMETirFqFbmb9zGypmS03s/88zPw7zKzEzObHXnfGP6okvJxTsYt/w6XMpsma8bxXtM7vRCJJ5ZiFbmYB4AXgSqAz8CMz63yYRd91znWLvQbHOacki16/xDXvzJ8zhvDMuLls2Lnf70QiSaM6n9DPBZY751Y658qBYcC1tRtLklYghF3zHDmRrQyIDOPXIxbgnA69iMRDdQq9NVD1b+Pi2LRD/dDMFpjZCDNrc7g3MrP+ZlZkZkUlJbphU8rKK8DO7c+t3nhKl0/j7dlr/U4kkhTi9aXoGCDfOdcV+Bh443ALOecGOecKnHMFubm5cVq1JKS+j0DTtjyf9TJ/G/sZa7fu8zuRSMKrTqGvB6p+4s6LTTvAObfVOVcWGx0MfDc+8SRppTfErnuR5uGNPGRv8/CIzwnrrBeRGqlOoc8BOphZOzNLA24GCqsuYGYtq4xeA+imHXJsbXthPf4PP7LxBNdM4SVdcCRSI8csdOdcJXAvMJ5oUb/nnFtsZo+Z2TWxxe4zs8Vm9jlwH3BHbQWWJNP3EdxJ7Xk+6xVe/vgz5q3d7ncikYRlfp1hUFBQ4IqKinxZt9QzxUW4Vy5noteTR9MeZOzAi2icEfI7lUi9ZGZznXMFh5unK0XFf3kFWJ/fcWl4Gj12f8QfRi/SqYwiJ0CFLvXDBQ9A2wv477Q3mP/5PN6do6tIRY6XCl3qBy8AP/gfQmnpvNbwJR4v/JxF63f6nUokoajQpf7IzsOufZ5TK77ij2lvc89b89i5v8LvVCIJQ4Uu9csZ34ceA7gxMo5uuybx8PDPdTxdpJpU6FL/XPYotDmPp9IHs/KLebwwebnfiUQSggpd6p9ACG54nVB6FkMbvcCLHy9g4heb/E4lUu+p0KV+atwK++FgWlSs5eWGg3hg2DyWb9ZTjkSORoUu9ddpfbAr/kSvipnc5w2n/5AifUkqchQqdKnfzrsbzv4Jd7qRdN0+gXvemkt5ZcTvVCL1kgpd6jczuPppOKUnf00bxL4VM/n9+wt15ovIYajQpf4LpsGNQwhkt+LtBk8zb94snp+kM19EDqVCl8TQMBd+MoqM9DSGN3iKoR/P1EOmRQ6hQpfEkXMqdssImnp7GdHoKf40cgbjFm7wO5VIvaFCl8TSqht201DywusZ0eBJ/u+wafx76Wa/U4nUCyp0STyn9cFuGsppkdUMy3iCh4dMYcbyLX6nEvGdCl0SU6d+2E1D6eBWMzT9L9z3+mQmfamrSSW1qdAlccVKvZNbzYj0x/njkI90TF1SmgpdElunftitI2kb2Mr76Y/yzDuFvD1rrd+pRHyhQpfEd+rF2E8/5KRMj1EZj/PP0e/x+AdLCEd08ZGkFhW6JIeWXbE7P6JB0xa8k/4nwp++yF1vzGF3qe79IqlDhS7Jo2k+dtdEvE5X8F+hN/n+yse4/rlJepSdpAwVuiSXjGy46S3o/Tv+IzCVF/c9yO//8RZvzFit+79I0lOhS/LxPOj9G7h1FPlZFYwM/YGNY//Mz16byfod+/1OJ1JrVOiSvNr3xRswk0Dnq/lNaBgPrBnA/U+/xuvTV+kLU0lKKnRJblk52A1vwA8G06XBLt71fkfgw4e55bkPmbFCV5dKclGhS/Izg643EPhlEXZuf24NTmLQ9juZ/upvuefVT/hy4y6/E4rEhfn1RVFBQYErKiryZd2S4jYtJjzhcQLLPmQbjRhccRXFp97EbX3PpiA/x+90IkdlZnOdcwWHnadCl5RVPJeKiY8TWjWZ/aQxqvJCZjf/Ief3uojvdW1JVlrQ74Qi36JCFzmaTYupnPEPbOF7BCLlLIi0Y6xdTLjLD+nTvTPntcshGNDRSakfVOgi1bF3C27Be+yb8xYNti0i7Iy5riMzAucSbn8FXbp+l14dcmmcEfI7qaQwFbrI8dq0hIqF77N/4Rga7/wCgI2uKZ9GurChaQHp+efR7vRudM9vRpOsNJ/DSipRoYvUxI61hL/6mB1LJpFRPJ0GldsB2O0yWRRpx7r0UyltejoZrb9DzildyM9rRduTsgjpMI3UAhW6SLw4B1uWUb52Dtu+moF9/RlN9qwg3ZUeWGSLa8wa14JtoRbsy2qNy84j2KQ1WSfl0Ti3DTm5LcnNzqJhehAz83FjJBHVuNDNrB/wdyAADHbO/eWQ+enAm8B3ga3ATc651Ud7TxW6JI1IBLavovTrRWxf9wWlG78isH0VWfu/pkllCUHCBy/ujG00YhuN2es1Zn8om7JQE8JpjXHpjbHMJgQyGuFlZhPMyiY9sxGhzEZkNGhMRoNGZGQ1JCMji6z0oP4KSEFHK/RjnpdlZgHgBeAyoBiYY2aFzrklVRb7ObDdOdfezG4GngBuqnl0kQTgeXDSaWScdBotz7z24HmRMOzeQOm2YnZsXsfeknVU7NpEZE8J3r4tNC7fQW7FBrL2f0nW3r1kUF6tVUacUUoau0ijnDTKLZ1ySyNsISosjYgXJOylEbY0IoE0IhYkEkjDeaEDL7wgBEIQCGEWgEAIFwhhXhALBA/8xAtiXiA2HvsZG/a8QHSaBfECATAvNi2I53l4XhDzPPA8Al4A8wJ4AQ+z6O955mGeh5mH53nR+Z4HZtH38TyM2HDAw8wO/I7neYBFf8/AM8OIXkeWqn/5VOdE23OB5c65lQBmNgy4Fqha6NcC/xUbHgE8b2bmdHs7SXVeALLzyMjOo0W7HsdevrIMt38H+/bsYP/unZTu2U7Zvt1Ulu6mcv9uwmV7iZTvw5Xvg4r9UFkKlaV44VK8cDleuIxQpIxApIJAZB8BV0GgvIIgFQRdBUEXJkhllVek9v8N6kDEGQ6IYEQwXLTaDww7ODDtm+Ho+P8OO8DZwctEHfk9qPIesUUPel8OWtc347DhtBvpeesf4/7vUJ1Cbw2sqzJeDJx3pGWcc5VmthM4CTjoZhlm1h/oD3DKKaecYGSRJBZMxxqdTINGJ9OgZR2sLxKBSCUuUkG4opzKygrC4UrCFRVUVpThwmEqw5VEqrxcuIJIOEwkUhn7GYbYsHMRXCSMi8SmuwguEp128HAYnMOFw4DDuQhEwjgcuEj0u4rYT+fC0RaM/Q7OHbSc4XCRCOD+9/diw+aiNQ/Ebp/8zbTYslV/4sDF6to5op9GqyxfZVk76LNq9PcOLBNd2YHaB7DY+1lsPC375FrZnXV6KZxzbhAwCKLH0Oty3SJyGJ4HXhpGGsG0BnVbCBJ31flGZT3Qpsp4XmzaYZcxsyCQTfTLURERqSPVKfQ5QAcza2dmacDNQOEhyxQCt8eGrwcm6fi5iEjdOuZfWLFj4vcC44metviqc26xmT0GFDnnCoFXgCFmthzYRrT0RUSkDlXrkJlzbhww7pBpj1QZLgVuiG80ERE5HroqQUQkSajQRUSShApdRCRJqNBFRJKEb3dbNLMSYM0J/nozDrkKNUWk4nan4jZDam53Km4zHP92t3XO5R5uhm+FXhNmVnSku40ls1Tc7lTcZkjN7U7FbYb4brcOuYiIJAkVuohIkkjUQh/kdwCfpOJ2p+I2Q2pudypuM8RxuxPyGLqIiHxbon5CFxGRQ6jQRUSSRMIVupn1M7OlZrbczP7T7zy1wczamNlkM1tiZovNbGBseo6ZfWxmy2I/m/qdtTaYWcDMPjOzD2Lj7cxsVmyfvxu7jXPSMLMmZjbCzL40sy/MrGcq7GszeyD23/ciM3vHzDKScV+b2atmttnMFlWZdtj9a1HPxrZ/gZl1P551JVShV3lg9ZVAZ+BHZtbZ31S1ohJ4yDnXGegBDIht538CE51zHYCJsfFkNBD4osr4E8DfnHPtge1EH0qeTP4O/Ms5dzpwFtFtT+p9bWatgfuAAufcd4jemvubB8wn275+Heh3yLQj7d8rgQ6xV3/gxeNZUUIVOlUeWO2cKwe+eWB1UnHObXDOzYsN7yb6P/DWRLf1jdhibwDX+ZOw9phZHnA1MDg2bsAlRB8+Dkm23WaWDVxE9JkCOOfKnXM7SIF9TfT23Zmxp5xlARtIwn3tnJtC9DkRVR1p/14LvOmiZgJNzKzaT5dNtEI/3AOrW/uUpU6YWT5wNjALONk5tyE2ayNQO0+a9dczwK/hwOPoTwJ2OOcqY+PJts/bASXAa7HDTIPNrAFJvq+dc+uBp4C1RIt8JzCX5N7XVR1p/9ao4xKt0FOKmTUERgL3O+d2VZ3nnKvyiPHkYGbfAzY75+b6naUOBYHuwIvOubOBvRxyeCVJ93VTop9G2wGtgAZ8+7BESojn/k20Qq/OA6uTgpmFiJb5W865UbHJm7758yv2c7Nf+WrJ+cA1Zraa6OG0S4geX24S+7Mckm+fFwPFzrlZsfERRAs+2ff1pcAq51yJc64CGEV0/yfzvq7qSPu3Rh2XaIVenQdWJ7zYceNXgC+cc09XmVX1Ydy3A/+s62y1yTn3W+dcnnMun+i+neScuwWYTPTh45Bk2+2c2wisM7NOsUl9gSUk+b4meqilh5llxf57/2a7k3ZfH+JI+7cQuC12tksPYGeVQzPH5pxLqBdwFfAVsAL4vd95amkbLyD6J9gCYH7sdRXR48kTgWXABCDH76y1+G/QG/ggNnwqMBtYDgwH0v3OF+dt7QYUxfb3aKBpKuxr4FHgS2ARMARIT8Z9DbxD9HuCCqJ/kf38SPsXMKJn8q0AFhI9C6ja69Kl/yIiSSLRDrmIiMgRqNBFRJKECvivjvcAAAAgSURBVF1EJEmo0EVEkoQKXUQkSajQRUSShApdRCRJ/H9lp61xXaUGowAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 15ms/step - loss: 0.0056 - accuracy: 1.0000\n",
            "loss= 0.005600697360932827\n",
            "acc= 1.0\n",
            "WARNING:tensorflow:5 out of the last 5 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7fa7ee3efc20> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAMs0lEQVR4nO3dX2id93nA8e9jy6ZyulaFiICcMPti9RQCw0UMr4Eylg53a6hzsYtsSaEZwzC2Nu26lMo3vWpTcCnNxQgYd4URs124wisj1B1svRjMpnKc4tVnhpKuif+Unl7YLcVDFnp6IbmW1aPoVX2O3qd+v58rWzqRHl6iL8c/nfM+kZlIkura1vYAkqS3Z6glqThDLUnFGWpJKs5QS1JxY6P4og8++GDu2bNnFF9aku5L586d+0lmTg763EhCvWfPHubn50fxpSXpvhQRP1zvcx59SFJxhlqSijPUklScoZak4gy1JBU3kld9SPejU+evcPT0Ja5ev8nUxDgvHNzHU/t3tz2WOsBQSw2cOn+F2bkLTC/2OLS9x5kb08zOLQAYa42coZYaOHr6EtOLPU7s/AI7WOQWYzyzcISjp3caao2cZ9RSA1ev3+TAth47WGQsltjBIge29bh6/Wbbo6kDDLXUwNTEOGeWprnFGIu5jVuMcWZpmqmJ8bZHUwd49CE18MLBfczOLfDMwhEObOtxZmma3tg0Lx7c1/Zo6gBDLTVw+xz66OmdvHz9vUxNjPOir/rQFjHUUkNP7d9tmNUKz6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqzlBLUnGGWpKKM9SSVFyj25xGxKeAvwISuAA8l5n/P8rBpNvc/q2u2/AZdUTsBj4BzGTmY8B24OlRDybBne3fD934Ln+9/V956MZ3mZ27wKnzV9oeTdoyTRcHjAHjEXEL2AVcHd1I0h1u/5YaPKPOzCvAl4A3gWvAjcz81trHRcThiJiPiPl+vz/8SdVJbv+Wmh19vAc4BOwFpoAHIuLZtY/LzGOZOZOZM5OTk8OfVJ3k9m+p2dHHB4EfZGYfICLmgPcDr4xyMAnc/i1Bs1C/CRyIiF3ATeAJYH6kU0kr3P4tNQh1Zp6NiJPAa8AicB44NurBpNvc/q2ua/Sqj8z8HPC5Ec8iSRrAdyZKUnGGWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1JxRlqSSrOUEtScYZakooz1JJUnKGWpOIMtSQV13RnojrI7d9SDYZaA93e/j292OPQ9h5nbkwzO7cAYKylLWaoNZDbv6U6PKPWQG7/luow1BrI7d9SHR59aCC3f0t1GGoN5PZvqQ5DrXW5/VuqwTNqSSrOUEtScYZakooz1JJUnKGWpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBXX6DanETEBHAceAxL4y8z871EO1mVu/5a0WtP7Ub8EfDMz/ywidgK7RjhTp7n9W9JaG4Y6It4NfAD4GEBmLgALox2ru9z+LWmtJmfUe4E+8LWIOB8RxyPigbUPiojDETEfEfP9fn/og3aF278lrdUk1GPA+4CXM3M/8HPgs2sflJnHMnMmM2cmJyeHPGZ3uP1b0lpNzqgvA5cz8+zK308yINQaDrd/S1prw1Bn5o8i4q2I2JeZl4AngIujH62b3P4taa2mr/r4OHBi5RUfbwDPjW4kuf1b0mqNQp2ZrwMzI55FkjSA70yUpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqrultTjvDDeCSqjHUq7gBXFJFhnoVN4BLqsgz6lXcAC6pIkO9ihvAJVXk0ccqbgCXVJGhXsUN4JIqMtRruAFcUjWeUUtScYZakooz1JJUnKGWpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpuMa3OY2I7cA8cCUznxz2IG7/lqTBNnM/6ueBHvCuYQ/h9m9JWl+jUEfEw8CHgc8DfzfsIdz+LUnra3pG/RXgM8DSeg+IiMMRMR8R8/1+f1NDuP1bkta3Yagj4kngx5l57u0el5nHMnMmM2cmJyc3NYTbvyVpfU2OPh4HPhIRfwq8A3hXRLySmc8Oawi3f0vS+jYMdWbOArMAEfGHwN8PM9Lg9m9JejtltpC7/VuSBttUqDPz28C3RzKJJGkg35koScUZakkqzlBLUnGGWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1JxRlqSSrOUEtScYZakooz1JJUXJnbnErSb6pT569w9PQlrl6/ydTEOC8M+X76hlqS7sGp81eYnbvA9GKPQ9t7nLkxzezcAsDQYm2oJekeHD19ienFHid2foEdLHKLMZ5ZOMLR0zuHFmrPqCXpHly9fpMD23rsYJGxWGIHixzY1uPq9ZtD+x6GWpLuwdTEOGeWprnFGIu5jVuMcWZpmqmJ8aF9D48+JOkevHBwH7NzCzyzcIQD23qcWZqmNzbNiwf3De17GGpJuge3z6GPnt7Jy9ffy9TEOC/6qg9JquWp/buHGua1PKOWpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqzlBLUnEb3uY0Ih4B/gl4CEjgWGa+NOrBJGkjo97+XUWT+1EvAp/OzNci4reAcxHx75l5ccSzSdK6tmL7dxUbhjozrwHXVv78s4joAbsBQy2pNVux/buKTZ1RR8QeYD9wdsDnDkfEfETM9/v94UwnSevYiu3fVTQOdUS8E/g68MnM/Onaz2fmscycycyZycnJYc4oSb9iK7Z/V9FoZ2JE7GA50icyc260I0nSxrZi+3cVTV71EcBXgV5mfnn0I0nSxrZi+3cVTZ5RPw58FLgQEa+vfOxIZr46urEkaWOj3v5dRZNXffwXEFswiyRpAN+ZKEnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqzlBLUnGGWpKKM9SSVFyjxQGStFZXNoBXYKglbVqXNoBXYKglbVqXNoBX4Bm1pE3r0gbwCgy1pE3r0gbwCjz6kLRpXdoAXoGhlrRpXdoAXoGhlvRr6coG8Ao8o5ak4gy1JBVnqCWpOEMtScUZakkqzlBLUnGGWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1JxRlqSSrOUEtScY1ucxoRHwJeArYDxzPziyOdStK63P7dPRuGOiK2A/8A/DFwGfhORHwjMy+OejhJd3P7dzc1eUb9+8D3M/MNgIj4F+AQYKilLeb2725qcka9G3hr1d8vr3zsLhFxOCLmI2K+3+8Paz5Jq7j9u5uG9svEzDyWmTOZOTM5OTmsLytpFbd/d1OTo48rwCOr/v7wysckbTG3f3dTk1B/B/idiNjLcqCfBv5ipFNJGsjt3920YagzczEi/hY4zfLL8/4xM7838skkDeT27+5p9DrqzHwVeHXEs0iSBvCdiZJUnKGWpOIMtSQVZ6glqbjIzOF/0Yg+8MNf8z9/EPjJEMf5Tea1uJvX425ejzvuh2vx25k58N2CIwn1vYiI+cycaXuOCrwWd/N63M3rccf9fi08+pCk4gy1JBVXMdTH2h6gEK/F3bwed/N63HFfX4tyZ9SSpLtVfEYtSVrFUEtScWVCHREfiohLEfH9iPhs2/O0KSIeiYj/jIiLEfG9iHi+7ZnaFhHbI+J8RPxb27O0LSImIuJkRPxvRPQi4g/anqlNEfGplZ+T/4mIf46Id7Q907CVCPWqBbp/AjwK/HlEPNruVK1aBD6dmY8CB4C/6fj1AHge6LU9RBEvAd/MzN8Ffo8OX5eI2A18ApjJzMdYvhXz0+1ONXwlQs2qBbqZuQDcXqDbSZl5LTNfW/nzz1j+QezsDYgj4mHgw8DxtmdpW0S8G/gA8FWAzFzIzOvtTtW6MWA8IsaAXcDVlucZuiqhbrRAt4siYg+wHzjb7iSt+grwGWCp7UEK2Av0ga+tHAUdj4gH2h6qLZl5BfgS8CZwDbiRmd9qd6rhqxJqDRAR7wS+DnwyM3/a9jxtiIgngR9n5rm2ZyliDHgf8HJm7gd+DnT2dzoR8R6W//W9F5gCHoiIZ9udaviqhNoFumtExA6WI30iM+fanqdFjwMfiYj/Y/lI7I8i4pV2R2rVZeByZt7+F9ZJlsPdVR8EfpCZ/cy8BcwB7295pqGrEupfLtCNiJ0s/zLgGy3P1JqICJbPIHuZ+eW252lTZs5m5sOZuYfl/y/+IzPvu2dMTWXmj4C3IuL22vEngIstjtS2N4EDEbFr5efmCe7DX6422pk4ai7Q/RWPAx8FLkTE6ysfO7Kyu1L6OHBi5UnNG8BzLc/Tmsw8GxEngddYfrXUee7Dt5P7FnJJKq7K0YckaR2GWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1Jxf0CnP++rG/VJ1oAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kDUU5YAZksVp"
      },
      "source": [
        "## CNN"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8fCNcgo4k0hA",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 945
        },
        "outputId": "0327df40-2156-4474-ff94-6cd160cc4f68"
      },
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras.layers import Dense, RepeatVector, Flatten\n",
        "from tensorflow.keras.layers import Bidirectional, LSTM, GRU\n",
        "from tensorflow.keras.layers import Conv2D, MaxPooling2D, Conv1D, MaxPooling1D, Input, Reshape\n",
        "\n",
        "input_shape = train_x.shape[1:]\n",
        "# input_shape = (5,10)\n",
        "\n",
        "model = keras.Sequential()\n",
        "model.add(Conv1D(32, 3, padding=\"same\", input_shape=input_shape))\n",
        "model.add(Conv1D(32, 3, padding=\"same\"))\n",
        "model.add(MaxPooling1D())\n",
        "model.add(Conv1D(32, 3, padding=\"same\"))\n",
        "model.add(Conv1D(32, 3, padding=\"same\"))\n",
        "model.add(MaxPooling1D())\n",
        "model.add(Dense(CATEGORY_NUM, activation=\"softmax\"))\n",
        "model.compile(loss=\"categorical_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n",
        "model.summary()\n",
        "\n",
        "history = model.fit(train_x, train_y, epochs=100, verbose=1, validation_split=0.1, batch_size=128)\n",
        "\n",
        "\n",
        "plt.plot(history.history['loss'])\n",
        "plt.plot(history.history['val_loss'])\n",
        "plt.show()\n",
        "\n",
        "\n",
        "\n",
        "loss, acc = model.evaluate(test_x, test_y)\n",
        "print(\"loss=\", loss)\n",
        "print(\"acc=\", acc)\n",
        "\n",
        "\n",
        "\n",
        "y_ = model.predict(test_x)\n",
        "predicted = np.argmax(y_, axis=-1)\n",
        "test_y_index = np.argmax(test_y, axis=-1)\n",
        "\n",
        "plt.plot(test_y_index, \"o\")\n",
        "plt.plot(predicted, \".\")\n",
        "plt.show()"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "conv1d (Conv1D)              (None, 5, 32)             992       \n",
            "_________________________________________________________________\n",
            "conv1d_1 (Conv1D)            (None, 5, 32)             3104      \n",
            "_________________________________________________________________\n",
            "max_pooling1d (MaxPooling1D) (None, 2, 32)             0         \n",
            "_________________________________________________________________\n",
            "conv1d_2 (Conv1D)            (None, 2, 32)             3104      \n",
            "_________________________________________________________________\n",
            "conv1d_3 (Conv1D)            (None, 2, 32)             3104      \n",
            "_________________________________________________________________\n",
            "max_pooling1d_1 (MaxPooling1 (None, 1, 32)             0         \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 1, 10)             330       \n",
            "=================================================================\n",
            "Total params: 10,634\n",
            "Trainable params: 10,634\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU1d3H8c9vJhs7QsK+h7C5sUSlimwigrjWWrW1WpdatYr7bq3Wp/axbri0+uBStbW2Fq1FRQHRVpBFgwqy76sICUtCAmSb8/xxBw3JBAIkczMz3/frlZcz95xkfter35zc5RxzziEiIrEv4HcBIiJSOxToIiJxQoEuIhInFOgiInFCgS4iEieS/Prg9PR016VLF78+XkQkJs2dOzfPOZcRqc23QO/SpQs5OTl+fbyISEwys7XVtemUi4hInFCgi4jECQW6iEicUKCLiMQJBbqISJxQoIuIxAkFuohInIi5QF+dV8QD7yyktDzkdykiIvVKzAX6qtxCJn+aw9tfbvS7FBGReiXmAn14yX/4b9rNfDxtEuUhLc4hIrJXzAW69RhJaaM23LPrYaZ8vtDvckRE6o2YC3QaHEHaT/5KhhWQPuU6QuXlflckIlIvxF6gA4H2/Vjc916OK/+SFW/e73c5IiL1QkwGOsBRZ45lStJQui96mtDyaX6XIyLiu5gN9GAwwO6Rj7A01JGyNy6Dbav9LklExFcxG+gAYwZ053dN7qa4tAz3j59CSZHfJYmI+CamAz0pGOCno4ZyXcl1sHkRTBwLTrcyikhiiulABxh1VBu2tR3M+KSfwIIJMOsZv0sSEfFFzAe6mXHraT35feHprG09AqbeBys/9rssEZGoi/lABxiclc7xXVtycd7PCaX3hAm6SCoiiScuAt3MuGNUT9YXBXit6+/BheDvukgqIoklLgIdYEDnFozo3Zo/zClh5xnjIXexLpKKSEKJm0AHuGNUT4pKynhidScYdrd3kfSz8X6XJSISFXEV6Fmtm/Dj7I78ZfYa1h95DfQYDZPvhnVz/C5NRKTOxVWgA9w4ogfBgPHo1OVw7nPQrAP881Io3OJ3aSIidSruAr1NszQuP6kr//7qGxZsM7jgr7B7O7x5BYQ0M6OIxK+4C3SAq4dm0qJRCr97bzGu9VEw5jFY/Qn85/d+lyYiUmcOGOhm1tHMPjazRWa20MxuiNDHzOwpM1thZvPNrH/dlFszTdOSuXFEFrNWbeXDxVug38XQ92L45BFYPtXP0kRE6kxNRuhlwC3OuT7AQOBXZtanUp/RQFb46yrg2Vqt8hBcdHwnMjMa8dCkxZSUheD0R6DVkfDWL2DHer/LExGpdQcMdOfcJufcF+HXO4HFQPtK3c4GXnWe2UBzM2tb69UehORggHvG9GZ1XhGvzVkLKQ3hx69CeZn3JGl5qZ/liYjUuoM6h25mXYB+QOX7ANsDFYe9G6ga+pjZVWaWY2Y5ubm5B1fpIRjWsxWDuqfz5LTl5O8qhfTucNaTsOFzmPbbOv98EZFoqnGgm1lj4E3gRudcwaF8mHNuvHMu2zmXnZGRcSg/4qCYGfeM6U3B7lKenLbc23jUeTDgMpj5FCybXOc1iIhES40C3cyS8cL8NefcWxG6bAQ6VnjfIbzNd73bNuWC4zry6qw1rMot9DaO+j20Phr+dTXk14syRUQOW03ucjHgRWCxc+7xarpNBC4J3+0yEMh3zm2qxToPy82n9iQtOchDkxZ7G5IbwPkvQ3kJvHml7k8XkbhQkxH6ScDPgOFm9lX463Qzu9rMrg73mQSsAlYAzwPX1k25hyajSSrXDe/Oh4u3MGN5nrcxvbt3f/q6mTD9MX8LFBGpBeZ8mo0wOzvb5eTkRO3zisvKOfXxT2iQHOS9sYNICoZ/l735C1jwJlz2PnQ6IWr1iIgcCjOb65zLjtQWl0+KRpKaFOSu0b1Yunknf/+8wg05Yx7z5nt580rYvcO/AkVEDlPCBDp4648O7NaCR6csZXtRibcxrSmc9yIUbIT3bva3QBGRw5BQgW5m3H/WkezcU8ZjU5d+39DxOBh6l3fq5esJ/hUoInIYEirQAXq1acrPBnbmtTnrWLAx//uGQTdBh+Ph3Zshf4N/BYqIHKKEC3SAm07tQYuGKfxm4kK+uygcTIIf/h+EyuDtayAU8rdIEZGDlJCB3qxBMneM6sXctdv515cVHixq0c176Gj1JzDH9/nFREQOSkIGOsCPBnTg2I7N+f37SygsLvu+of8l3tJ1034Lucv8K1BE5CAlbKAHAsb9Z/Yhd2cxT3+0/PsGMzjzSe9p0n9fq6dIRSRmJGygA/TrdAQ/GtCBl2as/n6eF4AmrWH0I96sjLOe8a9AEZGDkNCBDnD7qJ6kJgX5n/cW79tw9I+g1xnw0e8gd2nkbxYRqUcSPtBbNUnjhlOy+GjJFj5asvn7BjM44wlIaRS+60WnXkSkfkv4QAe49MQudMtoxIPvhper26txK2/puo1zYbbuehGR+k2BDqQkBfj1GX1YnVfEyzNX79t41HnQYxR89D+wdaU/BYqI1IACPWxYz1YM79WKp6atIHdn8fcNe0+9BJPhnRv0wJGI1FsK9AruHdOb4rJyHp1c6SJo03Yw8kFYMx2+eMWf4kREDkCBXkG3jMZcdlJX3pi7nq835O/b2P9S6DoYpt4HO7/1p0ARkf1QoFdy3fDutGyUwgPvVJjnBcKnXsZBWTFMvtu/AkVEqqFAr6RpWjK3juxJztrtTPq60ki8ZSacfLM3ze7Kj/wpUESkGgr0CM7P7kivNk343w8Ws6e00v3nJ93oTeL13q1QusefAkVEIlCgRxAMGPeO6cP6bbt5ZeaafRuT07xl67athE/H+VKfiEgkCvRqDMpK55RerXjmoxXkFRbv25g53Ls/ffrjujddROoNBfp+3D2mN7tLy3liaoRpdEf+DoIp8P4dUPHiqYiITxTo+5GZ0ZiLB3bm75+vZ8WWwn0bm7aFYXfBiqmw5D1/ChQRqUCBfgDXD+9Og+Qgf/hgSdXG438JrY6ED+6EkqLoFyciUoEC/QBaNk7lmqGZTFm0mc/XbNu3MZgEYx6F/PXwyaP+FCgiEqZAr4HLT+pK66apPDRp8b4PGwF0PhGOuRBmPq0l60TEVwr0GmiQEuTmU3vw5bodfLAgwmP/Ix+ElIbw3s26QCoivlGg19B5/TvQo3VjHv5gCaXllWZcbNwKTrnPm7zr6wn+FCgiCU+BXkNJwQB3jOrFmq27+Ptn66p2GHAZtOvvzfOye0f0CxSRhKdAPwjDe7Xi+K4teHLacoqKy/ZtDAThjMdhVx58/Dt/ChSRhKZAPwhmxl2je5FXWMLz01dV7dCuHxx3JXz2PGyaF/0CRSShKdAPUr9ORzD6qDaM/2TVvisb7TXsHmjYUk+QikjUKdAPwW2n9aS4LMRT05ZXbWzQ3LtAum6WN82uiEiUKNAPQbeMxlx0fEde/2wda/IiPCHa72JoeyxM+bWeIBWRqFGgH6Kxw7NIChpPfBjhYaJAEEb/AXZ+AzOeiH5xIpKQDhjoZvaSmW0xswXVtA81s3wz+yr8dV/tl1n/tGqaxmUndWXivG9YvKmgaodOA+Ho8+HTp2D7mqjXJyKJpyYj9JeBUQfoM9051zf89dvDLys2XD04kyapSTw6eWnkDiMeAAvAh/dHtS4RSUwHDHTn3CfAtgP1S0TNGibzyyGZTFuyhZzKE3cBNGsPJ90AC/8F6+ZEv0ARSSi1dQ79B2Y2z8zeN7Mjq+tkZleZWY6Z5eTm5tbSR/vrspO6kNEklT9MXlp14i6Ak8ZC4zYw+S4Ihaq2i4jUktoI9C+Azs65Y4Gngber6+icG++cy3bOZWdkZNTCR/uvYUoSY4d357PV2/jvsgi/pFIaebcxbpwLC9+KfoEikjAOO9CdcwXOucLw60lAspmlH3ZlMeSC4zrRsUUDHpm8lFAowij92Iu82xin/gZKd0e/QBFJCIcd6GbWxsws/Pr48M/cerg/N5akJAW4+dQeLPymgEkLNlXtEAh4a5AWbIDZz0a/QBFJCDW5bfF1YBbQ08w2mNkVZna1mV0d7vIjYIGZzQOeAi50EU8mx7ezjm1Pz9ZNeHzKMsoqT68L0PVk6DEKZoyDXbrGLCK1ryZ3uVzknGvrnEt2znVwzr3onHvOOfdcuP0Z59yRzrljnXMDnXMz677s+icYMG49rSer8oqYMHdD5E4j7oeSnVquTkTqhJ4UrUUjereiX6fmPDltOXtKy6t2aNUb+v4EPhuvh41EpNYp0GuRmXH7ab3YlL+Hv85eG7nT0LshkAQf/U90ixORuKdAr2U/yGzJyVnp/PHjFezcU1q1Q7P2MPAa+Pqf8M1X0S9QROKWAr0O3H5aL7bvKuX56asjdxh0IzQ4AqYlzCwJIhIFCvQ6cHSHZow5ui0vTF9FXmGERTDSmsGgm2HlNFgzI/oFikhcUqDXkZtH9qC4LMQzH62I3OH4X0CTdvDhA1rZSERqhQK9jmRmNObH2R3425x1rN+2q2qH5AYw5HbY8Bks+yD6BYpI3FGg16Gxp2SBEXmpOvBWNmrRDaY9qIm7ROSwKdDrUNtmDfjZwM68+cUGVmwprNohmOwtKr1lISyYEP0CRSSuKNDr2LVDM2mQHIy8VB3AkT+E1kfDxw9BeYTbHEVEakiBXsdaNk7likFdeW/+JhZszK/aIRCA4ffC9tXw1WvRL1BE4oYCPQquHNyNZg2SeXxqNaP0HqdBh+Pgv3+A0j3RLU5E4oYCPQqapiVz9ZBMPlqyhblrI8y0aAbDfw0FGyHnpegXKCJxQYEeJZee2Jn0xqk8NqWaUXq3IdB1MEx/DIojXEAVETkABXqUNExJ4pqhmcxcuZVZK6tZ/2P4fbArD+Y8F93iRCQuKNCj6KcndKJ101SemLos8oLSHY/zFsGY+TTsiXABVURkPxToUZSWHORXw7rz2ZptzFiRF7nT0Ltgzw6YrVG6iBwcBXqUXXBcR9o1S+OxKdWM0tv1hV5nwKw/wu7t0S9QRGKWAj3KUpOCXDc8i6/W7+DjpVsidxp6FxTne6EuIlJDCnQfnJ/dgQ5HNGDch8sjj9LbHAV9zoHZz2pBaRGpMQW6D5KDAa4b1p35G/L3M0q/E0qKYOZT0S1ORGKWAt0n5w04wCi9VW846ofw2fMapYtIjSjQfZIcDHD9cG+U/tGSakbpg2/3RumznolucSISkxToPvph/w50bLG/UXovOPIcmPN/GqWLyAEp0H2UHAxw/bAsvt6Yz7TF+xulF+qOFxE5IAW6z87t355OLRoyblo196W37gN9ztYoXUQOSIHus73n0hdsLODD6kbpQ+6Akp3ebYwiItVQoNcD5/ZrT+eWDRn3YXWj9COh95neKF1zvIhINRTo9UBSMMD1w7NY+E0BUxZtjtxp8G3e06OfjY9ucSISMxTo9cQ5fdvRpWVDnqzujpe2x0LWaTDrT5ovXUQiUqDXE3tH6Ys2FTB54X5G6bu3aVUjEYlIgV6PnN23Hd3SGzHuw2WEQtXMl95tqDdfeunuaJcnIvWcAr0eSQoGuP6U7iz5didTFn0budPg26BoC3zxanSLE5F6T4Fez5x1bPvwKH155FF6l0HQ6UT49EkoK4l+gSJSbynQ65lgwBh7ShZLvt3J5IXVjdJvgYKNMO/16BYnIvXaAQPdzF4ysy1mtqCadjOzp8xshZnNN7P+tV9mYjnz2HZkZuxnlJ55CrTrBzMeh/Ky6BcoIvVSTUboLwOj9tM+GsgKf10F6HHGw7R3lL50807eXxBhlG4GJ98K29fAwreiXp+I1E8HDHTn3CfA/iYRORt41XlmA83NrG1tFZiozjimHd1bNebJadXc8dLzdGjVB6Y/BqFQ9AsUkXqnNs6htwfWV3i/IbytCjO7ysxyzCwnNze3Fj46fu0dpS/bXBh5lB4IwMm3QO4SWPJu9AsUkXonqhdFnXPjnXPZzrnsjIyMaH50TBpzdNv9j9KPPBdaZML0RyHS06UiklBqI9A3Ah0rvO8Q3iaH6cCj9CAMugk2zYOV06JfoIjUK7UR6BOBS8J3uwwE8p1zm2rh5wo1GKUfcwE0bQ/TH49+cSJSr9TktsXXgVlATzPbYGZXmNnVZnZ1uMskYBWwAngeuLbOqk1AwYBxQ3iUPmlBhN+TSSlw4lhY+ymsnRX9AkWk3rCIM/tFQXZ2tsvJyfHls2NNecgxatwnAHxw42CCAdu3Q8kuGHe0d2/6xRN8qFBEosXM5jrnsiO16UnRGBAMGDeO6MHyLYW8O/+bqh1SGsIProUVU73z6SKSkBToMWL0UW3o1aYJ4z5cTll5hPvOj7sSUpt696WLSEJSoMeIQMC46dQerM4r4u2vIozS05rB8b+ARRMhd1n0CxQR3ynQY8jIPq05sl1Tnpq2nNJIo/SB10JyA2+OFxFJOAr0GGJm3HxqD9Zt28WEuRuqdmiUDgN+DvPf8OZ5EZGEokCPMcN7teLYjs15etpyisvKq3Y48XrvgaNPn4x+cSLiKwV6jDEzbh3Zg2/y9/D6nHVVOzRtB31/Al/+FQr0fJdIIlGgx6BB3dM5oWsLnvl4JbtLIozST7oRQuXe2qMikjAU6DHIzLj1tJ7kFRbzyqw1VTu06ApHnw9z/wxFedEuT0R8okCPUcd1acGQHhk899+VFOwprdrh5JuhdDfM/lP0ixMRXyjQY9itI3uyY1cpL05fXbUxoyf0ORvmjIfd26NfnIhEnQI9hh3doRmjjmzDizNWs62opGqHwbdCyU4v1EUk7inQY9wtI3uwq6SMZ/+zompjm6O9pepm/wmKd0a/OBGJKgV6jMtq3YRz+3XglVlr2ZS/u2qHk2+FPTvg8xejX5yIRJUCPQ7cOCIL5xxPTYswSu8wADKHw6xnvGl2RSRuKdDjQMcWDfnpCZ15I2c9a/KKqnYYfBsU5cLcl6Nem4hEjwI9TvxqWHdSggEenxphpsXOJ0KXk73pAEr3RL84EYkKBXqcyGiSyhWDujJx3jcs2JhftcOQ26HwW/jyL9EvTkSiQoEeR64a0o0jGibz8AdLqjZ2ORk6DoQZT0BZcfSLE5E6p0CPI03TkrlueBbTl+cxY3mlR/7NvFF6wUb46m/+FCgidUqBHmcuHtiJ9s0b8L8fLCYUqrQAeOZwaD/AWwCjPMJ0ASIS0xTocSY1Kcitp/VgwcYC3v260vS5ZjDkDtixTqN0kTikQI9DZx/bnt5tm/LI5CVVF8HIGumN0j95FMoiTBcgIjFLgR6HAgHjrtG9WL9tN3+ZtXbfRjMYehfkr4OvXvOnQBGpEwr0ODW4RwZDemTw1LTlbK88cVf3EdA+G6Y/plG6SBxRoMexe8b0prC4jCenLd+3wQyG3Q3563VfukgcUaDHsR6tm3DBcZ346+y1rMot3Lcxczh0PCE8Std96SLxQIEe524+tQepSQF+/36lh432jtILNsLnL/hTnIjUKgV6nMtoksq1w7ozddFmPl1R6WGjbkO9kfonj8DuHX6UJyK1SIGeAK4Y1JWOLRrwwDsLKS0P7ds44gEvzD8d509xIlJrFOgJIC05yL1j+rBscyF/nV3pNsa2x8AxP4bZz0L+Bn8KFJFaoUBPECP7tGZQ93SemLqMrYWVLoIOuwdcCD7+vT/FiUitUKAnCDPjN2f2oaiknEenVJoz/YjOcPxV3oNGmxf6U6CIHDYFegLJat2ES3/Qhb9/vo556ytdBD35FkhrBh/cBc5F/gEiUq8p0BPMjadmkd44lXvfXkB5xdkYG7bwbmNc/V9YOsm/AkXkkNUo0M1slJktNbMVZnZnhPafm1mumX0V/rqy9kuV2tA0LZlfn9GHrzfmV71Amn05pPeEyffoYSORGHTAQDezIPBHYDTQB7jIzPpE6PoP51zf8JeeVKnHzjymLSdnpfPo5KVsKaiwxmgwGUY9BNtXw5zn/CtQRA5JTUboxwMrnHOrnHMlwN+Bs+u2LKlLZsZvzz6K4vIQD763eN/G7iMg6zT47yNQuMWfAkXkkNQk0NsD6yu83xDeVtl5ZjbfzCaYWcdIP8jMrjKzHDPLyc3NPYRypbZ0TW/EtUMzeWfeN/xnaaXgPu0hKNsDU+/zpzgROSS1dVH0HaCLc+4YYCrwSqROzrnxzrls51x2RkZGLX20HKprhmaSmdGIe/61gKLisu8b0rvDidfDvNdhzaf+FSgiB6Umgb4RqDji7hDe9h3n3Fbn3N6raC8AA2qnPKlLqUlBHj7vGL7J380jk5fu2zj4NmjWCd67ReuPisSImgT650CWmXU1sxTgQmBixQ5m1rbC27OASidmpb7K7tKCSwZ25pVZa5i7dvv3DSkNYfTDkLtYF0hFYsQBA905VwZcB0zGC+o3nHMLzey3ZnZWuNtYM1toZvOAscDP66pgqX23jepF26Zp3Pnm/H3XIO11OvQY7U0JkL+x+h8gIvWCOZ+eCszOznY5OTm+fLZU9fHSLVz258/55eBu3HV67+8btq+BPw6EzGFw4d+8edRFxDdmNtc5lx2pTU+KCgDDerbiJyd0Yvz0VcxcWWHe9CO6eE+QLp0EC//lW30icmAKdPnOvWN607VlI255Yx75uypcCB14LbTtC+/fDru2+VegiOyXAl2+0zAliXEX9iV3ZzH3vP01352OCybB2c/A7u0w5V5/ixSRainQZR/HdGjOTaf24N35m/jn3AoLXrQ5Gk66wZtid8U0/woUkWop0KWKq4dkcmJmS+779wKWfFvwfcPg2yG9B0y8XmuQitRDCnSpIhgwxl3YlyZpyVz72hcU7n2KNDkNzn0Odn4LH1SZdFNEfKZAl4haNUnj6Yv6sSaviLveqnA+vf0AbzGMea/D4nf9LVJE9qFAl2oN7NaSW0b25J153/DKzDXfNwy+DdocA+/cAIWaZE2kvlCgy35dMySTEb1b8+B7i/l0Rfj+9KQUOPf/oLgA/n0thEL+FikigAJdDiAQPp+emdGIa1/7grVbi7yG1n28aXaXT4FZT/tbpIgACnSpgcapSTx/STZmcOUrOezcE37o6Lgroc/Z8OEDsG62v0WKiAJdaqZzy0b86Sf9WZVXxLWvfUFpecib1+Wsp6F5R5hwORRt9btMkYSmQJcaO7F7Og+dexTTl+dxx5vzvTtf0prB+S9DUS68ebnmThfxkQJdDsoFx3XiphE9eOuLjTw6JbwoRrt+MOZxWPUfeP8O8GkGT5FEl+R3ARJ7xp7SnW8L9vDHj1eS3jiVy07qCv1/BnnLYOZTkNETTvil32WKJBwFuhw0M+PBs49kW1ExD7yziKRggJ8N7Awj7oetK72nSFt0g6xT/S5VJKHolIsckqRggKcv6s+I3q349dsLeP2zdRAIwg/HQ+uj4I1LYN0cv8sUSSgKdDlkKUkB/vjT/gzrmcFdb33thXpqY7j4TWjSFl47HzbN97tMkYShQJfDkpoU5NmLBzA0HOrP/mclrlEGXPJvSG0CfzkX8pb7XaZIQlCgy2FLSw4y/mfZnHVsOx7+YAm/e28xoaYdvFA3g5fPgC2L/S5TJO4p0KVWpCQFGHdBX35+YhdemLGaG/7xFbubdoVL3/E6/Pl02PiFv0WKxDkFutSaQMD4zZl9uHN0L96d/w3nPTuT9Umd4fL3vXPrr5wFa2b4XaZI3FKgS60yM64ekslLPz+O9dt3cdYzM5ixtSlcPhmatoNXz4GcP/tdpkhcUqBLnRjWsxUTrxtEeuNUfvbSHB6akU/xpe9D18Hw7o0wcSyUFftdpkhcUaBLnema3oh/X3cSPzm+E+M/WcU5Ly1i2YiXYNDN8MUr8OfR3oNIIlIrFOhSpxqmJPG7c4/mxUuz2VKwhzHPzOTR8gspOe9l2LoCnhsEnz2v+V9EaoECXaLilN6tmXzTYM48ph3PfLyCEe83Z8bId3CdBsKkW+Ev5+h+dZHDpECXqElvnMrjF/Tlb784gaSgcfEbG7ig6DbW/uBB75bGP/0Apvwa9hT4XapITDLn05+62dnZLicnx5fPFv+VlIV4/bN1PP3RCvIKizknK5m7U9+g1Yp/QqMMOHEsZF/u3e4oIt8xs7nOueyIbQp08dOukjL+/OkaXpi+iu27Sjm31SbuTJlA67xZ0OAIOOEaGPBzaNLa71JF6gUFutR7e0rLeeuLjbw4YxUrc4s4MXU19zadRJ+dn+ICSVivMdD/Uug21JvVUSRBKdAlZoRCjtmrtzIhZwOTFmyibdlGrmzwX861/9CwvADXqBXW+0w48hzodCIENaW/JBYFusSknXtKmbpoM5MXfsusZRsZVJ7DWUlzGBb8ilRXTFlyE6zrEII9ToHOgyA9y5sMTCSOKdAl5u0uKWfmyjw+XbGVnOUbaJf3KUMC8xgSnE872wrAnuQj2N16AKmds2nYqR+0Pcabl10hL3FEgS5xZ2thMV+t38GXa7fz7eqvabIlhz6li+gfWE5mYNN3/XYFGrOjUVdKmmcSaNGVtFaZNGvbndT0ztColU7ZSMw57EA3s1HAk0AQeME597+V2lOBV4EBwFbgAufcmv39TAW61CbnHJsLilm8qYB1mzaze8M8UnIX0qxwFW3K1tPdNtLaduzzPeUEyA80pzC5JbtTWlKS2pJQw5bQ4AgCjVqS3LglyY2ak9K4BQ0aNyetcTMaNG5OMKWhRv3im/0F+gGHJ2YWBP4InApsAD43s4nOuUUVul0BbHfOdTezC4GHgQsOv3SRmjEz2jRLo02zNOjVCjj6u7bisnI2bt/N7K07KPh2JSV5q3H5GwkWbSJt92YalW6jyc5cWhYspyUFpFrpfj+r3Bm7LI09pFFsaZQEUim1NMoCqZQFUykPpBIKpBIKphAKpEAwhVAwBYLJuGAKFkiGYAoEk7BgMgSTCQSTIJCMBZMIBJOwCq8JBAkEgwQsCMEggUASFghi4e17XwcDQSwQwIJBAoEgZgECgQAEAuH35rVb4Ls227s93BYIBDDM6x8wwLBA+D3h78f7fWb6pVbv1OTvzeOBFc65VQBm9nfgbKBioJ8N3B9+PQF4xszM+XU+R6SC1KQg3TIa0y2jMfTqAAyJ2M85R1Lp2WgAAAYXSURBVGFxGVt3FlC0I5c9+XmU7tpBadF2ynfvwO0phOJCKNlJoGwXgdLdBMt3ESzfQ1L5HpJCe2hYuovkUDHJroQkV0oypaRQSpIrI4UyAhYf/0uEnBECHBb+8l6DF/J797JiO+HXe9v3vgbD2b79qfD97NP38F7Dvr+EHODMqrS5ffpE/n5H5F9orppfdBX7b8r8MQN/+puI/Q5HTQK9PbC+wvsNwAnV9XHOlZlZPtASyKvYycyuAq4C6NSp0yGWLFI3zIwmack0SWsJGS1r/wOcw4XKKCstobSkmLLSEsrLSigvLaGsrJRQWRnl5aWUl5XiyssIlXvvXXk5LlROKFT23WvnyqHc+6cL7d0WwoVCECrHOQcuhAuVhVNr7zaHcyHMlXuhFSr3tuG14ULhidLC3+/CcRwKhbeBY28fMBf6bt++i0Hnwq/Csf3dz3NV+4L3mRFeGxX7e9+x77Z9Y9erx+27mQjfW+F4VPzV893nuqo/t4oKfSxCHZW/0yqNbZPq6EG5qF4Rcs6NB8aDdw49mp8t4jszLJhMcjCZ5LRGflcjcagmk3NtBDpWeN8hvC1iHzNLAprhXRwVEZEoqUmgfw5kmVlXM0sBLgQmVuozEbg0/PpHwEc6fy4iEl0HPOUSPid+HTAZ77bFl5xzC83st0COc24i8CLwFzNbAWzDC30REYmiGp1Dd85NAiZV2nZfhdd7gPNrtzQRETkYWuBCRCROKNBFROKEAl1EJE4o0EVE4oRvsy2aWS6w9hC/PZ1KT6EmiETc70TcZ0jM/U7EfYaD3+/OzrmMSA2+BfrhMLOc6mYbi2eJuN+JuM+QmPudiPsMtbvfOuUiIhInFOgiInEiVgN9vN8F+CQR9zsR9xkSc78TcZ+hFvc7Js+hi4hIVbE6QhcRkUoU6CIicSLmAt3MRpnZUjNbYWZ3+l1PXTCzjmb2sZktMrOFZnZDeHsLM5tqZsvD/zzC71rrgpkFzexLM3s3/L6rmc0JH/N/hKdxjhtm1tzMJpjZEjNbbGY/SIRjbWY3hf/7XmBmr5tZWjweazN7ycy2mNmCCtsiHl/zPBXe//lm1v9gPiumAr3CgtWjgT7ARWbWx9+q6kQZcItzrg8wEPhVeD/vBKY557KAaeH38egGYHGF9w8DTzjnugPb8RYljydPAh8453oBx+Lte1wfazNrD4wFsp1zR+FNzb13gfl4O9YvA6Mqbavu+I4GssJfVwHPHswHxVSgU2HBaudcCbB3weq44pzb5Jz7Ivx6J97/4O3x9vWVcLdXgHP8qbDumFkHYAzwQvi9AcPxFh+HONtvM2sGDMZbUwDnXIlzbgcJcKzxpu9uEF7lrCGwiTg81s65T/DWiaiouuN7NvCq88wGmptZ25p+VqwFeqQFq9v7VEtUmFkXoB8wB2jtnNsUbvoWqJuVZv01Drgd2LtacEtgh3OuLPw+3o55VyAX+HP4NNMLZtaIOD/WzrmNwKPAOrwgzwfmEt/HuqLqju9hZVysBXpCMbPGwJvAjc65gopt4SX+4uqeUzM7A9jinJvrdy1RlAT0B551zvUDiqh0eiVOj/UReKPRrkA7oBFVT0skhNo8vrEW6DVZsDoumFkyXpi/5px7K7x5894/v8L/3OJXfXXkJOAsM1uDdzptON755ebhP8sh/o75BmCDc25O+P0EvICP92M9AljtnMt1zpUCb+Ed/3g+1hVVd3wPK+NiLdBrsmB1zAufN34RWOyce7xCU8XFuC8F/h3t2uqSc+4u51wH51wXvGP7kXPup8DHeIuPQ5ztt3PuW2C9mfUMbzoFWEScH2u8Uy0Dzaxh+L/3vfsdt8e6kuqO70TgkvDdLgOB/AqnZg7MORdTX8DpwDJgJXCP3/XU0T4OwvsTbD7wVfjrdLzzydOA5cCHQAu/a63DfwdDgXfDr7sBnwErgH8CqX7XV8v72hfICR/vt4EjEuFYAw8AS4AFwF+A1Hg81sDreNcJSvH+IruiuuMLGN6dfCuBr/HuAqrxZ+nRfxGROBFrp1xERKQaCnQRkTihQBcRiRMKdBGROKFAFxGJEwp0EZE4oUAXEYkT/w/9jVf6gc7M5gAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 44ms/step - loss: 0.0030 - accuracy: 1.0000\n",
            "loss= 0.0030273578595370054\n",
            "acc= 1.0\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAARPUlEQVR4nO3dcWikeX3H8fc3O0mdO4uRXpBmT7on9NKRA10JknpFWk8breLuH/3j7CrUUrYUq6fIiskfPSj0tiUiCi0Ly6ltcattz7gVsUZBRYTmMHs5u3rTgD31vOT0xj+ynpJ2MubbP5L1drfZy2ydyfPbzPsFy2Zmnsx8eHaeD8/+5vfMLzITSVK5hqoOIEl6bha1JBXOopakwlnUklQ4i1qSClfrx5PedttteeTIkX48tSQdSBcuXPhRZo7t9lhfivrIkSMsLS3146kl6UCKiO9d7zGHPiSpcBa1JBXOopakwlnUklQ4i1qSCtfVrI+IeA/wx0ACF4G3Z+Z/9zOYdNn55VXmFlZYW99gfLTOqekJjh89XHUsad/seUYdEYeBdwGTmXkXcAi4t9/BJNgu6Zn5i7zo0jf400P/yosufYOZ+YucX16tOpq0b7qdR10D6hGxCdwCrPUvkvSsuYUVGp0m50YeYJgOm9Q40Z5lbmHEs2oNjD3PqDNzFfgA8ATwFHApM79w7XYRcTIiliJiqdVq9T6pBtLa+gZTQ02G6VCLLYbpMDXUZG19o+po0r7pZujjhcAx4A5gHLg1It567XaZeTYzJzNzcmxs16sgpRs2PlpncavBJjU6OcQmNRa3GoyP1quOJu2bboY+Xgt8JzNbABExD7wK+Hg/g0kAp6YnmJlvc6I9y9RQk8WtBs1ag9PTE1VHk/ZNN0X9BDAVEbcAG8A9gF/koX1xeRx6bmGEM+t3Mj5a57SzPjRg9izqzHw4Ih4CHgE6wDJwtt/BpMuOHz1sMWugdTXrIzPvB+7vcxZJ0i68MlGSCmdRS1LhLGpJKpxFLUmFs6glqXAWtSQVzqKWpMJZ1JJUOItakgpnUUtS4SxqSSqcRS1JhbOoJalwFrUkFc6ilqTCWdSSVLg9Fw6IiAngn6646yXAn2fmh/qWSpJuIueXV5lbWGFtfYPx0TqnerxcXDdLca0ALweIiEPAKvDpniWQpJvY+eVVZuYv0ug0OXaoyeKlBjPzbYCelXVXS3Fd4R7gvzLzez15dUm6yc0trNDoNDk38gDDdNikxon2LHMLIz0r6hsdo74X+MRuD0TEyYhYioilVqv1iyeTpJvA2voGU0NNhulQiy2G6TA11GRtfaNnr9F1UUfECPBm4F92ezwzz2bmZGZOjo2N9SqfJBVtfLTO4laDTWp0cohNaixuNRgfrffsNW5k6OMNwCOZ+cOevbok3eROTU8wM9/mRHuWqaEmi1sNmrUGp6cnevYaN1LUb+E6wx6SNKguj0PPLYxwZv1OxkfrnN7vWR8AEXEr8DrgT3r2ypJ0QBw/erinxXytroo6M38K/ErfUkiSrssrEyWpcBa1JBXOopakwlnUklQ4i1qSCmdRS1LhLGpJKpxFLUmFs6glqXAWtSQVzqKWpMJZ1JJUOItakgpnUUtS4SxqSSqcRS1Jhet2hZdR4EHgLiCBP8rMf+9nMKk055dXmVtYYW19g/HROqd6vNySdD3drpn4YeDzmfn7O6uR39LHTFJxzi+vMjN/kUanybFDTRYvNZiZbwNY1uq7PYs6Il4AvBr4Q4DMbAPt/saSyjK3sEKj0+TcyAMM02GTGifas8wtjFjU6rtuxqjvAFrAxyJiOSIe3Fns9ioRcTIiliJiqdVq9TyoVKW19Q2mhpoM06EWWwzTYWqoydr6RtXRNAC6Keoa8ArgTGYeBX4KvP/ajTLzbGZOZubk2NhYj2NK1RofrbO41WCTGp0cYpMai1sNxkfrVUfTAOhmjPpJ4MnMfHjn9kPsUtTSQXZqeoKZ+TYn2rNMDTVZ3GrQrDU4PT1RdTQNgD2LOjN/EBHfj4iJzFwB7gEe6380qRyXx6HnFkY4s34n46N1TjvrQ/uk21kf7wTO7cz4eBx4e/8iSWU6fvSwxaxKdFXUmfkoMNnnLJKkXXhloiQVzqKWpMJZ1JJUOItakgpnUUtS4SxqSSqcRS1JhbOoJalwFrUkFc6ilqTCWdSSVDiLWpIKZ1FLUuEsakkqnEUtSYXr6vuoI+K7wDPAz4BOZvrd1H10fnmVuYUV1tY3GB+tc6qilURKyaEy+f7YP92u8ALwO5n5o74lEbD95p+Zv0ij0+TYoSaLlxrMzLcB9vUgKCWHyuT7Y3/dSFFrH8wtrNDoNDk38gDDdNikxon2LHMLI/t6AJSSQ2Xy/bG/uh2jTuALEXEhIk7utkFEnIyIpYhYarVavUs4YNbWN5gaajJMh1psMUyHqaEma+sbA5lDZfL9sb+6LerfysxXAG8A3hERr752g8w8m5mTmTk5NjbW05CDZHy0zuJWg01qdHKITWosbjUYH60PZA6VyffH/up2cdvVnb+fjohPA68EvtrPYIPq1PQEM/NtTrRnmRpqsrjVoFlrcHp6YiBzqEy+P/bXnkUdEbcCQ5n5zM7Pvwv8Rd+TDajL43tzCyOcWb+T8dE6pyv4NL2UHCqT74/9FZn53BtEvAT49M7NGvCPmfmXz/U7k5OTubS01JuEkjQAIuLC9aY+73lGnZmPAy/reSpJUle8MlGSCmdRS1LhLGpJKpxFLUmFs6glqXAWtSQVzqKWpMJZ1JJUOItakgpnUUtS4SxqSSqcRS1JhbOoJalwFrUkFc6ilqTCWdSSVLiuizoiDkXEckR8tp+BJElXu5Ez6vuAZr+CSJJ211VRR8TtwBuBB/sbR5J0rW7PqD8EvA/Yut4GEXEyIpYiYqnVavUknCSpi6KOiDcBT2fmhefaLjPPZuZkZk6OjY31LKAkDbpuzqjvBt4cEd8FPgm8JiI+3tdUkqSf27OoM3MmM2/PzCPAvcCXMvOtfU8mSQKcRy1JxavdyMaZ+RXgK31JIknalWfUklQ4i1qSCmdRS1LhLGpJKpxFLUmFs6glqXAWtSQVzqKWpMJZ1JJUOItakgpnUUtS4SxqSSqcRS1JhbOoJalwFrUkFc6ilqTC7blwQEQ8D/gq8Es72z+Umff3O5ik3Z1fXmVuYYW19Q3GR+ucmp7g+NHDVcdSH3Wzwsv/AK/JzJ9ExDDwtYj4t8xc7HM2Sdc4v7zKzPxFGp0mxw41WbzUYGa+DWBZH2B7FnVmJvCTnZvDO3+yn6Ek7W5uYYVGp8m5kQcYpsMmNU60Z5lbGLGoD7Cuxqgj4lBEPAo8DXwxMx/eZZuTEbEUEUutVqvXOSUBa+sbTA01GaZDLbYYpsPUUJO19Y2qo6mPuirqzPxZZr4cuB14ZUTctcs2ZzNzMjMnx8bGep1TEjA+Wmdxq8EmNTo5xCY1FrcajI/Wq46mPrrRVcjXI+LLwOuBb/YnkqTrOTU9wcx8mxPtWaaGmixuNWjWGpyenqg6mvqom1kfY8DmTknXgdcBf933ZJL+j8vj0HMLI5xZv5Px0TqnnfVx4HVzRv2rwN9HxCG2h0r+OTM/299Ykq7n+NHDFvOA6WbWx38AR/chiyRpF16ZKEmFs6glqXAWtSQVzqKWpMJZ1JJUOItakgpnUUtS4SxqSSqcRS1JhbOoJalwFrUkFc6ilqTCWdSSVDiLWpIKZ1FLUuH2LOqIeHFEfDkiHouIb0XEffsRTJK0rZsVXjrAezPzkYj4ZeBCRHwxMx/rczZJEl2cUWfmU5n5yM7PzwBNwHWAJGmf3NAYdUQcYXtZrod3eexkRCxFxFKr1epNOklS90UdEc8HPgW8OzN/fO3jmXk2Myczc3JsbKyXGSVpoHVV1BExzHZJn8vM+f5GkiRdqZtZHwF8BGhm5gf7H0mSdKVuzqjvBt4GvCYiHt3583t9ziVJ2rHn9LzM/BoQ+5BFkrQLr0yUpMJZ1JJUOItakgpnUUtS4SxqSSqcRS1JhbOoJalwFrUkFc6ilqTCWdSSVDiLWpIKZ1FLUuEsakkqnEUtSYWzqCWpcBa1JBVuz4UDIuKjwJuApzPzrv5Hqtb55VXmFlZYW99gfLTOqekJjh89XHUsSbsYlON1z6IG/g74G+Af+huleueXV5mZv0ij0+TYoSaLlxrMzLcBDuQ/vnQzG6TjtZuluL4aEUf6H6V6cwsrNDpNzo08wDAdNqlxoj3L3MLIgfuHl252g3S89myMOiJORsRSRCy1Wq1ePe2+WlvfYGqoyTAdarHFMB2mhpqsrW9UHU3SNQbpeO1ZUWfm2cyczMzJsbGxXj3tvhofrbO41WCTGp0cYpMai1sNxkfrVUeTdI1BOl67GaMeGKemJ5iZb3OiPcvUUJPFrQbNWoPT0xNVR5N0jUE6Xi3qK1we15pbGOHM+p2Mj9Y5fUA/RZZudoN0vEZmPvcGEZ8Afhu4DfghcH9mfuS5fmdycjKXlpZ6lVGSDryIuJCZk7s91s2sj7f0PpIkqVtemShJhbOoJalwFrUkFc6ilqTCWdSSVDiLWpIKZ1FLUuEsakkqnEUtSYWzqCWpcBa1JBXOopakwlnUklQ4i1qSCmdRS1LhLGpJKlxXS3FFxOuBDwOHgAcz8696HeT88ipzCyusrW8wPlrn1AFdUkeSbtSeRR0Rh4C/BV4HPAl8PSI+k5mP9SrE+eVVZuYv0ug0OXaoyeKlBjPzbQDLWtLA6+aM+pXAtzPzcYCI+CRwDOhZUc8trNDoNDk38gDDdNikxon2LHMLIxa1pIHXzRj1YeD7V9x+cue+q0TEyYhYioilVqt1QyHW1jeYGmoyTIdabDFMh6mhJmvrGzf0PJJ0EPXsw8TMPJuZk5k5OTY2dkO/Oz5aZ3GrwSY1OjnEJjUWtxqMj9Z7FU+SblrdDH2sAi++4vbtO/f1zKnpCWbm25xozzI11GRxq0Gz1uD09EQvX0aSbkrdFPXXgV+PiDvYLuh7gT/oZYjL49BzCyOcWb+T8dE6p531IUlAF0WdmZ2I+DNgge3peR/NzG/1Osjxo4ctZknaRVfzqDPzc8Dn+pxFkrQLr0yUpMJZ1JJUOItakgpnUUtS4SIze/+kES3ge//PX78N+FEP49zM3BdXc39czf3xrIOwL34tM3e9WrAvRf2LiIilzJysOkcJ3BdXc39czf3xrIO+Lxz6kKTCWdSSVLgSi/ps1QEK4r64mvvjau6PZx3ofVHcGLUk6WolnlFLkq5gUUtS4Yop6oh4fUSsRMS3I+L9VeepUkS8OCK+HBGPRcS3IuK+qjNVLSIORcRyRHy26ixVi4jRiHgoIv4zIpoR8ZtVZ6pSRLxn5zj5ZkR8IiKeV3WmXiuiqK9YQPcNwEuBt0TES6tNVakO8N7MfCkwBbxjwPcHwH1As+oQhfgw8PnM/A3gZQzwfomIw8C7gMnMvIvtr2K+t9pUvVdEUXPFArqZ2QYuL6A7kDLzqcx8ZOfnZ9g+EAf2y7oj4nbgjcCDVWepWkS8AHg18BGAzGxn5nq1qSpXA+oRUQNuAdYqztNzpRR1VwvoDqKIOAIcBR6uNkmlPgS8D9iqOkgB7gBawMd2hoIejIhbqw5VlcxcBT4APAE8BVzKzC9Um6r3Silq7SIing98Cnh3Zv646jxViIg3AU9n5oWqsxSiBrwCOJOZR4GfAgP7mU5EvJDt/33fAYwDt0bEW6tN1XulFHXfF9C92UTEMNslfS4z56vOU6G7gTdHxHfZHhJ7TUR8vNpIlXoSeDIzL/8P6yG2i3tQvRb4Tma2MnMTmAdeVXGmniulqH++gG5EjLD9YcBnKs5UmYgItscgm5n5warzVCkzZzLz9sw8wvb74kuZeeDOmLqVmT8Avh8REzt33QM8VmGkqj0BTEXELTvHzT0cwA9Xu1ozsd/2awHdm8jdwNuAixHx6M59sztrV0rvBM7tnNQ8Dry94jyVycyHI+Ih4BG2Z0stcwAvJ/cSckkqXClDH5Kk67CoJalwFrUkFc6ilqTCWdSSVDiLWpIKZ1FLUuH+FzLzmJjeDOPXAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FUjeezafksMt"
      },
      "source": [
        "## CNN + LSTM"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QSc7dlgXk1Y5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 891
        },
        "outputId": "412de08a-6453-4267-9de8-110e4adfa374"
      },
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras.layers import Dense, RepeatVector, Flatten\n",
        "from tensorflow.keras.layers import Bidirectional, LSTM, GRU\n",
        "from tensorflow.keras.layers import Conv2D, MaxPooling2D, Conv1D, MaxPooling1D, Input, Reshape\n",
        "\n",
        "input_shape = train_x.shape[1:]\n",
        "# input_shape = (5,10)\n",
        "\n",
        "model = keras.Sequential()\n",
        "model.add(Conv1D(32, 3, padding=\"same\", input_shape=input_shape))\n",
        "model.add(Conv1D(32, 3, padding=\"same\"))\n",
        "model.add(LSTM(128))\n",
        "model.add(RepeatVector(1))\n",
        "model.add(Dense(CATEGORY_NUM, activation=\"softmax\"))\n",
        "model.compile(loss=\"categorical_crossentropy\", optimizer=\"adam\", metrics=[\"accuracy\"])\n",
        "model.summary()\n",
        "\n",
        "history = model.fit(train_x, train_y, epochs=100, verbose=1, validation_split=0.1, batch_size=128)\n",
        "\n",
        "\n",
        "plt.plot(history.history['loss'])\n",
        "plt.plot(history.history['val_loss'])\n",
        "plt.show()\n",
        "\n",
        "\n",
        "\n",
        "loss, acc = model.evaluate(test_x, test_y)\n",
        "print(\"loss=\", loss)\n",
        "print(\"acc=\", acc)\n",
        "\n",
        "\n",
        "\n",
        "y_ = model.predict(test_x)\n",
        "predicted = np.argmax(y_, axis=-1)\n",
        "test_y_index = np.argmax(test_y, axis=-1)\n",
        "\n",
        "plt.plot(test_y_index, \"o\")\n",
        "plt.plot(predicted, \".\")\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_11\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "conv1d_22 (Conv1D)           (None, 5, 32)             992       \n",
            "_________________________________________________________________\n",
            "conv1d_23 (Conv1D)           (None, 5, 32)             3104      \n",
            "_________________________________________________________________\n",
            "lstm_5 (LSTM)                (None, 128)               82432     \n",
            "_________________________________________________________________\n",
            "repeat_vector_8 (RepeatVecto (None, 1, 128)            0         \n",
            "_________________________________________________________________\n",
            "dense_7 (Dense)              (None, 1, 10)             1290      \n",
            "=================================================================\n",
            "Total params: 87,818\n",
            "Trainable params: 87,818\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd5xU9f3v8ddnZrZTlrIsnaWDFAGXJkWMxqAxYv0FjC1qiO2qUX/GJPcm0cQkamJiRWmCvWBM0NgVEZAFFulFpIO0pS3LsnXme/+YwawKssjuninv5+Mxj5055+yc9/Ho27NnznyPOecQEZHY5/M6gIiI1AwVuohInFChi4jECRW6iEicUKGLiMSJgFcrbtq0qcvJyfFq9SIiMWnhwoW7nXNZR5rnWaHn5OSQn5/v1epFRGKSmW062jydchERiRMqdBGROKFCFxGJEyp0EZE4oUIXEYkTKnQRkTihQhcRiRMxV+i7DpTypzdXsaOw1OsoIiJRJeYKPW/9HqbNXsaw+z/kl9OWsr7goNeRRESiQswV+nnpK8hPv5lnWr5K/uJFnPHgTG587lNWbCv0OpqIiKdirtBp3AFfjwsYtOffvJ90K2+2mMyuNfP54cOzueqp+SzctM/rhCIinjCvbkGXm5vrTmgslwPbYN4TkP8UlB1gU6PB/KFwJO8f6szQTk255czO9M9pXHOBRUSigJktdM7lHnFezBb6YaWFsGAi5I2D4gK2ZZ7C3Qd+xDuHujC0U1NuP6sLfds2OvH1iIhEgfgu9MMqSuDTp2HWg3BwB9syc/nNgfOZcagDZ3Zvxh0/6Eq35g1qbn0iIh5IjEI/rKIEFk4JF3vxLtY3GcGte0axvCybS05pw21ndSG7QWrNr1dEpA4kVqEfVl4MeY/D7IdwFcXkNz2f67edTbGvAdeP6MjY4R1ITfLX3vpFRGpBYhb6YcW7YeZ9sGASwZQGvFz/Sn6zJZfWjevxux+dxBnds2s/g4hIDfm2Qo+9yxaPV0ZTOOcBuG4W/uY9GVPwD5a0eoAubOKaqfmMfTqfnQf0rVMRiX3xX+iHZfeAK1+HiyZRv+QLJpTezrRuM/hkzTbOfHAmLy3YjFd/rYiI1ITEKXQAM+h1Mdw4H+t5EbkbJ7Aw+8+c2XQfv3x1GVdMns8uHa2LSIxKrEI/LKMJXDgexrxESskuHtx/Cy/1XcGCjXsY+dAs3l+50+uEIiLHLTEL/bCuI+H6T7CcoQxcdS/5nabSvn6Ia5/O53f/Xk55ZcjrhCIi1ZbYhQ5QPxsufQXOupd6G9/jFf9vuDPXx9S5mxgzIU8fmIpIzFChA/h8cOpNcMW/8ZXs44Y1P+Pl0/ezavsBzn1kNvkb93qdUETkmFToVbUfBj+fCU06MiDvJmacto56KQEunTCP6Uu2eZ1ORORbqdC/rmFr+Omb0Pkssmf9mrd6zqBPm4bc/MIinpi5Tpc2ikjUUqEfSXIG/Pg5OOUqUvP+wQvNnua83tn85a3V/G76CkIhlbqIRJ+A1wGilj8A5/4D6rfA/9GfeahXkBbDbuHJWZs4VB7kvot64/eZ1ylFRL6kQv82ZjDiLvAFsA//wF29HOln3MbfP9hAZTDEXy85mYBff+SISHRQoVfH8DvADPvgHm7p6Qic9UseePdzKkOOh0b31ZG6iEQFFXp1Dbs9/PODe7hxQBMCI8fy57c/o35qgD9d0AszlbqIeOuY5wvMrI2ZzTCzlWa2wsxuOcIyZmYPm9laM1tqZv1qJ67Hht4Gg26E+U/y88Dr3HR6J16Yv4W/vvuZ18lERKp1hF4J3O6c+9TM6gMLzew959zKKsucDXSOPAYC4yI/44sZnPVHOLgD3v89t49qxt6BvXlsxjoapSdz7bAOXicUkQR2zCN059x259ynkedFwCqg1dcWGwU87cLygEwza1HjaaOBzwfnj4P2p2Gv38wf+hzgnF7N+eN/VvHOih1epxORBHZcl2iYWQ7QF5j3tVmtgC1VXm/lm6WPmY01s3wzyy8oKDi+pNEkkAL/8zQ0ysE/7UoeHNmUk9tk8ouXFrNq+wGv04lIgqp2oZtZPeBV4Fbn3HdqLefceOdcrnMuNysr67u8RfRIy4TRz0NFKanTrmDC6JOonxrg2qn57DlY5nU6EUlA1Sp0M0siXObPOef+eYRFvgDaVHndOjItvmV1DY+rvn0xzT7+FRMuP4XdB8u4/tlPqQhq6F0RqVvVucrFgEnAKufcg0dZbDpwReRql0FAoXNuew3mjF7dzoERv4IlL9B793+4/+LezN+4V1e+iEidq84R+hDgcuB7ZrY48jjHzK4zs+siy7wJrAfWAhOAG2onbpQa/r+QMwzevJNRrUu4dGBbnpy5nhmf7fI6mYgkEPNq9MDc3FyXn5/vybprReEX8MQQyGxH6ZVvc/4TC9hVVMZbtwwju0Gq1+lEJE6Y2ULnXO6R5mkgkprSsBWc9yhsX0zqx3/i0Uv7UlIe5JYXFxHU6IwiUgdU6DWp+7mQezV88jCdDi3lnlE9yFu/l0mz13udTEQSgAq9pp31R8hsB9P/Dxf3bsL3T8rmr++uYe2ug14nE5E4p0KvackZcN7DsHcd9vH93Ht+T9KS/Nw5bYlOvYhIrVKh14YOI6DvZTDnYZodXM3d5/Xg0837eWrOBq+TiUgcU6HXlrP+CBlNYfpNjOqVxZnds3ngnc/YsLvY62QiEqdU6LUlrRGc81fYsQxbMJE/XdCTZL+P3/57uW40LSK1QoVem7r/CDqeAR/9mWa+Im47qwuzPt+tURlFpFao0GuTGYz8C1Qcgg/u5vJB7ejWvD73vL6SQ+WVXqcTkTijQq9tWV1g0PWw6FkCOxbxh/N7sq2wlEc/XOt1MhGJMyr0ujD8TsjIgrd+Sf+2mVzUrzUTZq1nXYGuTReRmqNCrwupDeD7d8PWBbD8Ve46uxspAT9/eWu118lEJI6o0OtK79GQ3Qtm/JGsNOP6ER15b+VO5m/Y63UyEYkTKvS64vPBmb+DfRth4RSuHtKe5g1SuffNVbqMUURqhAq9LnU6E9oNhY/vJ82VcNtZXViyZT//WZYY9wIRkdqlQq9LZnDm76G4APIe56J+renWvD73v/0Z5ZW6ZZ2InBgVel1r0x+6nQtzHsZfsoe7zu7G5r2HeH7eJq+TiUiMU6F74YzfQvlB+OQRTuuSxYD2jXn8o3WUVgS9TiYiMUyF7oWsrtDzIpg/ATu0l1+c2YVdRWU8P2+z18lEJIap0L0y/I7wkAB5jzO4YxMGdWjMuJk6SheR706F7pVm3eGkUTDvSSjZxy/O7EJBURnP5ulcuoh8Nyp0Lw3/XygvgrwnGNihCad2bMITM9dTUq6jdBE5fip0LzXvGb7iZd44KC3kF9/vwu6DZTynK15E5DtQoXtt+P9CaSEsmEj/nMYM7tCECbPWU1apo3QROT4qdK+17AMdvwd5T0BFKdeP6MjOA2X8a9EXXicTkRijQo8GQ26B4l2w9CWGdW5Kj5YNeHLmeoIhjfEiItWnQo8G7U+D5r3hk0cw57hhRCfW7y7mXd2qTkSOgwo9GpiFj9L3fA5r3mJkz+bkNEln3Mx1GolRRKpNhR4tTjofGrYNj/HiM35+WkeWbi1kzto9XicTkRihQo8W/gAMvhG25MHmeVzYrxXN6qfw5MfrvE4mIjFChR5N+l0OqZkw91FSAn6uGNyOWZ/v5rMdRV4nE5EYoEKPJskZcMqVsPoN2L+ZSwe2IzXJx+TZG7xOJiIxQIUebfr/DDCYP4HGGclc2K81ry3+gt0Hy7xOJiJRToUebTLbQPdz4dOpUF7M1UPaU14Z0qBdInJMxyx0M5tsZrvMbPlR5o8ws0IzWxx5/LbmYyaYgdeHhwNY8iKdmtXj9K5ZPJu3SUPrisi3qs4R+hRg5DGWmeWc6xN53HPisRJc20HQok94aF3nuHZYB3YfLGf64m1eJxORKHbMQnfOfQzsrYMscpgZDLoedn8G6z7k1I5N6Na8PpPnbNAXjUTkqGrqHPpgM1tiZm+ZWY+jLWRmY80s38zyCwoKamjVcarHBZDRDOY9iZlx5ak5rN5RxPwN+n+riBxZTRT6p0A759zJwCPAv462oHNuvHMu1zmXm5WVVQOrjmOBFDjlKvj8Xdi3kfP7tKJhWhJT5270OJiIRKsTLnTn3AHn3MHI8zeBJDNresLJBHJ/CuaDBZNIS/bz4/5teGfFTrbtL/E6mYhEoRMudDNrbmYWeT4g8p4agKQmNGgZvoRx0TNQUcLlg9oRck53NBKRI6rOZYsvAHOBrma21cyuMbPrzOy6yCIXA8vNbAnwMDDa6ZO7mtP/Z1CyD5a/SpvG6ZzRLZsX5m/RJYwi8g2BYy3gnBtzjPmPAo/WWCL5qpyhkNUd5o+HPj/hqlNzeH/VTt5Yup2LT2ntdToRiSL6pmi0M4MB18L2JbA1nyGdmtCpWT2enrvR62QiEmVU6LGg92hIaQALJmBmXDG4HUu3FrJ4y36vk4lIFFGhx4KUenDyaFjxGhTv4YK+rchI9usoXUS+QoUeK3KvhmA5LHqG+qlJXNivNW8s3c7e4nKvk4lIlFChx4pm3aHdUMifDKEQlw9uR3lliJfzt3idTESihAo9lvS/GvZvgnUf0CW7PgPbN+bZvE0EQ7pKVERU6LGl24/C47ssmAjAFYNz2LqvhJlrdnkcTESigQo9lgSSod8VsOYd2LeJs3pk06x+Ck/P1TdHRUSFHntOuSp8bfrCKST5fYwZ0JaZawrYtKfY62Qi4jEVeqzJbANdzg6P71JZxqUD2+Iz47l5m71OJiIeU6HHov7XQHEBrHqd7Aap/KBHNi/na3wXkUSnQo9FHU6Hxh1g/gQALh+Uw/5DFbyxdLvHwUTESyr0WOTzQe41sCUPdixjUIfGdGpWj2fmbvQ6mYh4SIUeq/pcCoFUWDAJM+PyQe1YsrWQJRrfRSRhqdBjVXpj6HkxLH0ZSgu5sF8r0pP9PJOnSxhFEpUKPZb1vwYqimHJS9RPTeKCvq14fck29ml8F5GEpEKPZa36Qct+4W+OOscVg3Mo0/guIglLhR7rBoyF3Z/Bho/p2jw8vsszGt9FJCGp0GNdjwsgvUn4FnX8d3yXjz7T+C4iiUaFHuuSUsPju3z2Juzfwlk9sslukMJUje8iknBU6PEg9+rwz/xJJPl9/GRgOz5eU8D6goPe5hKROqVCjweZbaHrObBwKlSUMnpAG5L8xrN5Gt9FJJGo0OPFgJ9ByV5Y8U+a1U/l7J4teCV/C8VllV4nE5E6okKPF+1Pg6ZdYN6T4BxXnppDUVkl/1z0hdfJRKSOqNDjhVn4Esbti2HLfPq1zaR364ZMmbMB53QJo0giUKHHk5PHQEpDmDcOM+PKwTmsKyhm9trdXicTkTqgQo8nKfWg3+WwcjoUbuXck1vQtF4yUz/Z6HUyEakDKvR4M2As4GDBRFICfi4d0JYPVu/SLepEEoAKPd40ahe5hHEKlB/iJ4Pa4TfTjaRFEoAKPR4Nuh5K9sGyl8lukMo5vVrw8oItHNQljCJxTYUej9oNgea9IO8JcI6fDglfwvjqwq1eJxORWqRCj0dmMOgGKFgF6z6kb9tG9G2byVNzNhDSKIwicUuFHq96XgT1smHuowBcPaQ9G/cc4sPVGoVRJF6p0ONVICV8xcu6D2HnSs7u2ZyWDVOZPGeD18lEpJYcs9DNbLKZ7TKz5UeZb2b2sJmtNbOlZtav5mPKd5J7NQTSIO8xAn4fV5yawyfr9rBq+wGvk4lILajOEfoUYOS3zD8b6Bx5jAXGnXgsqRHpjaHvT8I3ki7ayej+bUhL8vOUjtJF4tIxC9059zGw91sWGQU87cLygEwza1FTAeUEDboBghWwYCKZ6clcdEor/rV4GwVFZV4nE5EaVhPn0FsBVe9KvDUy7RvMbKyZ5ZtZfkFBQQ2sWo6pScfwF40WTITyQ1w9pD3llSGezdMXjUTiTZ1+KOqcG++cy3XO5WZlZdXlqhPbqTeFx0pf/BwdsupxZvdmPJu3idKKoNfJRKQG1UShfwG0qfK6dWSaRIu2g6F1//AljMFKrhnagT3F5fxLY6WLxJWaKPTpwBWRq10GAYXOue018L5SU8xgyC2wbyOsms6gDo3p0bIBE2drrHSReFKdyxZfAOYCXc1sq5ldY2bXmdl1kUXeBNYDa4EJwA21lla+u67nQJNOMOchDLh2WHvW7jrIR2v0WYZIvAgcawHn3JhjzHfAjTWWSGqHzw+Db4I3boWNs/hhr6H85a3VTJq1gdO7NvM6nYjUAH1TNJGcPAYysmDOQyQHfFx5ag6z1+5m5TZ90UgkHqjQE0lSKgy8Dta+DzuW8ZMB7UhP9jNx1nqvk4lIDVChJ5r+10ByPZj9dxqmJ/Hj/m2YvmQb2wtLvE4mIidIhZ5o0hqFx3hZ8RrsWcfVQ9oTco4pczZ6nUxETpAKPRENvhF8SfDJw7RpnM45vVrw/LzNFJVWeJ1MRE6ACj0R1W8OfS6Fxc/Dge2MHd6BorJKXpy/5di/KyJRS4WeqIbcDKFKyHuM3q0zGdShMZPnbKAiGPI6mYh8Ryr0RNW4A/S4EPKfgkN7GTu8A9sLS3l9yTavk4nId6RCT2TDboPygzDvSUZ0aUaX7HqM/3i9hgMQiVEq9ESW3QO6nQvzxuErP8DPh3dk9Y4iPvpMwwGIxCIVeqIbfgeUFsL88fzo5Ja0aJjKEzPXeZ1KRL4DFXqia9kXOv8A5j5OcvAQ1wxtz7wNe1m0eZ/XyUTkOKnQBU67M3wDjPxJjBnQloZpSTw5U8MBiMQaFbpA61zocDp88ggZVs7lg9rxzsodrCs46HUyETkOKnQJO+2XUFwA+ZO4akgOyX4fT3ykc+kisUSFLmHtBoeP0mf/naZJFYwZ0JbXFn3B1n2HvE4mItWkQpf/+t7/hUN7YP6TjB3eAYAJH+tcukisUKHLf7XODV/xMudhWqaWc2G/Vry4YAsFRWVeJxORalChy1ed/mso3Q9547h+RCcqgiEmztZRukgsUKHLV7XsE/726NzHaJ9exg97t+TZuZsoPKShdUWinQpdvun030BZEcz+OzeM6EhxeZBJczZ4nUpEjkGFLt+UfVJ4vPR5T9I9bT8jezTnqdkbdJQuEuVU6HJkp/8azODDe7nlzM4UlVXqXLpIlFOhy5E1bA0Dr4OlL9GdDZzTqzmTZ29gX3G518lE5ChU6HJ0Q38BaZnw3u+45YwuHKoIMmGWjtJFopUKXY4uLROG3wnrZ9C1aB4/7NWCKZ9sZK+O0kWikgpdvl3/a8K3q3v7Lm49vR0lFUHGfbTW61QicgQqdPl2gRQ4+37Y8zmd1j/Lxf1aM/WTTWzZqzFeRKKNCl2OrfP3oes5MPN+7ji1Pmbw4HtrvE4lIl+jQpfq+cGfIFhBdt6fuHpoe15b9AXLvyj0OpWIVKFCl+pp3B6G3grLXuGmDjtolJ7En99ahXPO62QiEqFCl+obcitktiPj7du49bQ2zFm7h5lrCrxOJSIRKnSpvuR0+NFDsHcdl5W9SLsm6fzhjZWUV4a8TiYiqNDleHU8Hfpehn/uIzwwxLGuoJjJGrhLJCpUq9DNbKSZfWZma83sriPMv8rMCsxsceRxbc1Hlahx1r2QkcWAJb9lZPcmPPzB52zbX+J1KpGEd8xCNzM/8BhwNnASMMbMTjrCoi855/pEHhNrOKdEk7RM+OHfYOcy7st+n2DIce9/VnmdSiThVecIfQCw1jm33jlXDrwIjKrdWBL1up8LvS6h4bwH+UNuKf9Ztp1Zn+sDUhEvVafQWwFbqrzeGpn2dReZ2VIzm2ZmbY70RmY21szyzSy/oED/8ce8c/4KDVpy8aZ76N7Ex69fW0ZxWaXXqUQSVk19KPo6kOOc6w28B0w90kLOufHOuVznXG5WVlYNrVo8k5YJFzyJb98Gprb8J1v3lXD/26u9TiWSsKpT6F8AVY+4W0emfck5t8c5d/jW8BOBU2omnkS9nCEw7Daaff4y93Vfz9S5m5i7bo/XqUQSUnUKfQHQ2czam1kyMBqYXnUBM2tR5eV5gD4hSyQjfgUt+3HJF/cxuFEhd766RKdeRDxwzEJ3zlUCNwHvEC7ql51zK8zsHjM7L7LYzWa2wsyWADcDV9VWYIlC/iS4ZApmfiamPULBvkL+9Kb+ny5S18yrsThyc3Ndfn6+J+uWWrLmHXj+f1iUNYoLtvyYJy7rx8ieLY79eyJSbWa20DmXe6R5+qao1JwuP4Bht9O34N/c2nQBd05bytZ9GjddpK6o0KVmjfg15AzjlpLH6e7WcfMLi6gIaqwXkbqgQpea5Q/AJVOxetlMSfsHWzZv5G/v6mYYInVBhS41L6MJjHmetGAR0xo/zuSZq3lz2XavU4nEPRW61I7mveD8x2l3aDnjMp/ljlcWs3rHAa9TicQ1FbrUnh4XwPA7OaP0PW4IvM7Ypxey/1C516lE4pYKXWrX6b+GnhdzU+g5+hz4kJue14ekIrVFhS61ywxGPQZtBvH35Cc4tO4T/u9ry3UvUpFaoEKX2peUCqOfx9+wFc9m/J0FC+fx2Iy1XqcSiTsqdKkbGU3gsldJS0lmWsYDPPNuHq8t2up1KpG4okKXutOkI/aTaTTyFTOt3gPcO+0TZqze5XUqkbihQpe61bIPNuYFWrvtPJP2N25/djZ56zXcrkhNUKFL3Ws/HLtoEt2CnzMl5QFumjKLxVv2e51KJOap0MUbJ52HXTSBXqHVjA88wM8nzWLpVpW6yIlQoYt3el6EXTCevm4lj/vu49oJM1m4aa/XqURilgpdvNX7Euz8cfRzK5ji/yM3TfpAt7AT+Y5U6OK9k0dj//MM3W0TLwTu5s6n3ubt5RrMS+R4qdAlOnQ/F7tsGu0C+3g1+ff84/l/MXHWen2jVOQ4qNAlerQfjl31BlnpPv6Vcjfz3nqG301fobFfRKpJhS7RpWVfbOwMUlp0Y0LygzSY/w8uGz+XnQdKvU4mEvVU6BJ9GrTEfvoW9LqEO5Je4aYdv+KKh6bzybrdXicTiWoqdIlOSWlw4QT44YMMCazmxdAdTJj8JH9+axWlFUGv04lEJRW6RC8z6H8Nvp/PpGHTVjyVdD8d5/ySSx96U98sFTkCFbpEv2bd8Y2dAUN/wcVJc5h08AaeeeLP/L/XlrGvWHdAEjlMhS6xISkVzvw9vus+pn6rrvwt6QkuWnQldzzwGE/N2UB5pa6EEVGhS2zJ7kHgmndh1OP0qH+ISfyenLev5Mb7x/P8vM0qdklo5tUXN3Jzc11+fr4n65Y4UVGCmz+Bypl/Jam8kLxQd15JvpDuwy/kktx2NExP8jqhSI0zs4XOudwjzlOhS8wrO4j7dCplsx4h9dB2NoSyeZUzKO05hvNO7U2vVg0xM69TitQIFbokhmAFrHiN4rkTydg+jwrnZ2aoNwsyRtC43yh+0K8LOU0zvE4pckJU6JJ4CtZQtmAKlUtfJaN0B2UuiVmhnqzIGEhy95HknnwyfdpkkuTXx0gSW1TokrhCIdi6gIOfvkLos7doUBK+MfW6UAsW2knszRpAvc7D6NalGz1bZ5Ka5Pc4sMi3U6GLADgHe9ZSsuItilZ/QIOdC0gNFQOwy2WyxHViZ73uhJr1oEFOH9q170rn5g2olxLwOLjIf6nQRY4kFIQdSylaO5eidXmk7FxEk9LNX84ucmmscy3YHmhDcb0cXKP2pDbrSMOWnchu3ppWjdNV9lLnVOgi1VVWhNu5kn0bPuXg5mXYnrXUO7ieRpUFX13MJbHNNabA15SDSU0pS80imJGNr14WSQ2akZrZnPTMLOo3yiazYQMy01NIDuh8vZy4byv0ah1emNlI4CHAD0x0zv3la/NTgKeBU4A9wI+dcxtPJLSIJ1LqY20H0rjtQBpXnV5+CLd/Ewe2r6Vw21rK927G7d9K9qHttC9bQ4OiuaQUHXkYgjKXxH4yKCKdQ776lPozKA/UpzKpHqGkDFxyfUiphyVn4EvJwJ9Sj0BqBkmpGQRS65OSmk5yWsZ/f6alk5qcQpLfdDmmfMUxC93M/MBjwPeBrcACM5vunFtZZbFrgH3OuU5mNhq4D/hxbQQW8URyOtasOw2bdafhyUeY7xyUFhI8uJuivds4uHc7pYW7qSjaTah4L5Tux8oKSSk/QP3KgyRX7CS1rJg0V0IaZccdp9L5OEQS5SRTYQEqSKLCkqi0ZCotiaAl4XwBghYg5EsiZEmEfAGcLzz98E8iD+cLYL6k8Gu/H3wBzBfAfIef+8PP/QF8vgBEXpsvgPn9X84382H+AGZ+fH4/Pp8v8j4+zPyY34f5/PjMj89vYH58Pj/mM3yR9/KZRZaP/I7P8JkPn88PPt+X83w+Hz6fgfki88PzfJH5EJ6HWfiRAKpzhD4AWOucWw9gZi8Co4CqhT4K+H3k+TTgUTMzp/uHSaIwg7RM/GmZZGZ1IvN4fjcUhPKDlJUUUXqwiJLiQspLi6koKaaitIhgWQnB8hKC5YcIVZTgKkqhohQqyyBYji9YBsEyfMFyfKFyfKEKLFRBwFWQEizFV1mJ31UQoBK/q8TvgvipJEAQP0ECrhI/IZIsvoclDjnDAQ772iMyLVL6IXyEi+ury3DM54azb84j8v58OQ22dhzNwMvurvFtrE6htwK2VHm9FRh4tGWcc5VmVgg0Ab5yRwIzGwuMBWjbtu13jCwSZ3x+SG1ISmpDUhpBQ69yOEcwWEmwspLKynIqKipwwUoqg5WEKisJhSoJVlQSDFbgQkFCwWD4eTBIKBQkFKqEUIhQsBLnQuHpLogLBSEUxIVCX053zoEL4VwlOHDBSpwLhv/ScSFCoRDgwpeduhA4h3MhcEGMyPNQuIrD00OR3w3/fvhNv+U5gAuFK9a5KtPdf9/rcCV/+fy/y1iV+d+cdvh5+NVX1hF57WvYulZ2YZ1+RO+cGw+Mh/CHonW5bhE5BjP8gST8gSSSSfM6jXwH1fnY/QugTZXXrSPTjriMmQUIH2TsqYmAIiJSPdUp9AVAZzNrb2bJwGhg+teWmQ5cGXl+MfChzp+LiNStY55yiZwTvwl4h8wrJfkAAAQ4SURBVPBli5OdcyvM7B4g3zk3HZgEPGNma4G9hEtfRETqULXOoTvn3gTe/Nq031Z5XgpcUrPRRETkeOirayIicUKFLiISJ1ToIiJxQoUuIhInPBtt0cwKgE3f8deb8rVvoSaIRNzuRNxmSMztTsRthuPf7nbOuawjzfCs0E+EmeUfbfjIeJaI252I2wyJud2JuM1Qs9utUy4iInFChS4iEiditdDHex3AI4m43Ym4zZCY252I2ww1uN0xeQ5dRES+KVaP0EVE5GtU6CIicSLmCt3MRprZZ2a21szu8jpPbTCzNmY2w8xWmtkKM7slMr2xmb1nZp9HfjbyOmttMDO/mS0yszcir9ub2bzIPn8pMoxz3DCzTDObZmarzWyVmQ1OhH1tZr+I/Pu93MxeMLPUeNzXZjbZzHaZ2fIq0464fy3s4cj2LzWzfsezrpgq9Co3rD4bOAkYY2YneZuqVlQCtzvnTgIGATdGtvMu4APnXGfgg8jreHQLsKrK6/uAvzvnOgH7CN+UPJ48BLztnOsGnEx42+N6X5tZK+BmINc515Pw0NyHbzAfb/t6CjDya9OOtn/PBjpHHmOBccezopgqdKrcsNo5Vw4cvmF1XHHObXfOfRp5XkT4P/BWhLd1amSxqcD53iSsPWbWGvghMDHy2oDvEb75OMTZdptZQ2A44XsK4Jwrd87tJwH2NeHhu9MidzlLB7YTh/vaOfcx4ftEVHW0/TsKeNqF5QGZZtaiuuuKtUI/0g2rW3mUpU6YWQ7QF5gHZDvntkdm7QCyPYpVm/4B3AmEIq+bAPudc5WR1/G2z9sDBcBTkdNME80sgzjf1865L4C/ApsJF3khsJD43tdVHW3/nlDHxVqhJxQzqwe8CtzqnDtQdV7kFn9xdc2pmZ0L7HLOLfQ6Sx0KAP2Acc65vkAxXzu9Eqf7uhHho9H2QEsgg2+elkgINbl/Y63Qq3PD6rhgZkmEy/w559w/I5N3Hv7zK/Jzl1f5askQ4Dwz20j4dNr3CJ9fzoz8WQ7xt8+3Aludc/Mir6cRLvh439dnAhuccwXOuQrgn4T3fzzv66qOtn9PqONirdCrc8PqmBc5bzwJWOWce7DKrKo3474S+HddZ6tNzrlfOedaO+dyCO/bD51zPwFmEL75OMTZdjvndgBbzKxrZNIZwErifF8TPtUyyMzSI/++H97uuN3XX3O0/TsduCJytcsgoLDKqZljc87F1AM4B1gDrAN+43WeWtrGoYT/BFsKLI48ziF8PvkD4HPgfaCx11lr8Z/BCOCNyPMOwHxgLfAKkOJ1vhre1j5AfmR//wtolAj7GrgbWA0sB54BUuJxXwMvEP6coILwX2TXHG3/Akb4Sr51wDLCVwFVe1366r+ISJyItVMuIiJyFCp0EZE4oUIXEYkTKnQRkTihQhcRiRMqdBGROKFCFxGJE/8fnHBfD61r0UIAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "text": [
            "1/1 [==============================] - 0s 15ms/step - loss: 0.0061 - accuracy: 1.0000\n",
            "loss= 0.006077046971768141\n",
            "acc= 1.0\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAMs0lEQVR4nO3dX2id93nA8e9jy6ZyulaFiICcMPti9RQCw0UMr4Eylg53a6hzsYtsSaEZwzC2Nu26lMo3vWpTcCnNxQgYd4URs124wisj1B1svRjMpnKc4tVnhpKuif+Unl7YLcVDFnp6IbmW1aPoVX2O3qd+v58rWzqRHl6iL8c/nfM+kZlIkura1vYAkqS3Z6glqThDLUnFGWpJKs5QS1JxY6P4og8++GDu2bNnFF9aku5L586d+0lmTg763EhCvWfPHubn50fxpSXpvhQRP1zvcx59SFJxhlqSijPUklScoZak4gy1JBU3kld9SPejU+evcPT0Ja5ev8nUxDgvHNzHU/t3tz2WOsBQSw2cOn+F2bkLTC/2OLS9x5kb08zOLQAYa42coZYaOHr6EtOLPU7s/AI7WOQWYzyzcISjp3caao2cZ9RSA1ev3+TAth47WGQsltjBIge29bh6/Wbbo6kDDLXUwNTEOGeWprnFGIu5jVuMcWZpmqmJ8bZHUwd49CE18MLBfczOLfDMwhEObOtxZmma3tg0Lx7c1/Zo6gBDLTVw+xz66OmdvHz9vUxNjPOir/rQFjHUUkNP7d9tmNUKz6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqzlBLUnGGWpKKM9SSVFyj25xGxKeAvwISuAA8l5n/P8rBpNvc/q2u2/AZdUTsBj4BzGTmY8B24OlRDybBne3fD934Ln+9/V956MZ3mZ27wKnzV9oeTdoyTRcHjAHjEXEL2AVcHd1I0h1u/5YaPKPOzCvAl4A3gWvAjcz81trHRcThiJiPiPl+vz/8SdVJbv+Wmh19vAc4BOwFpoAHIuLZtY/LzGOZOZOZM5OTk8OfVJ3k9m+p2dHHB4EfZGYfICLmgPcDr4xyMAnc/i1Bs1C/CRyIiF3ATeAJYH6kU0kr3P4tNQh1Zp6NiJPAa8AicB44NurBpNvc/q2ua/Sqj8z8HPC5Ec8iSRrAdyZKUnGGWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1JxRlqSSrOUEtScYZakooz1JJUnKGWpOIMtSQV13RnojrI7d9SDYZaA93e/j292OPQ9h5nbkwzO7cAYKylLWaoNZDbv6U6PKPWQG7/luow1BrI7d9SHR59aCC3f0t1GGoN5PZvqQ5DrXW5/VuqwTNqSSrOUEtScYZakooz1JJUnKGWpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBXX6DanETEBHAceAxL4y8z871EO1mVu/5a0WtP7Ub8EfDMz/ywidgK7RjhTp7n9W9JaG4Y6It4NfAD4GEBmLgALox2ru9z+LWmtJmfUe4E+8LWIOB8RxyPigbUPiojDETEfEfP9fn/og3aF278lrdUk1GPA+4CXM3M/8HPgs2sflJnHMnMmM2cmJyeHPGZ3uP1b0lpNzqgvA5cz8+zK308yINQaDrd/S1prw1Bn5o8i4q2I2JeZl4AngIujH62b3P4taa2mr/r4OHBi5RUfbwDPjW4kuf1b0mqNQp2ZrwMzI55FkjSA70yUpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqrultTjvDDeCSqjHUq7gBXFJFhnoVN4BLqsgz6lXcAC6pIkO9ihvAJVXk0ccqbgCXVJGhXsUN4JIqMtRruAFcUjWeUUtScYZakooz1JJUnKGWpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpuMa3OY2I7cA8cCUznxz2IG7/lqTBNnM/6ueBHvCuYQ/h9m9JWl+jUEfEw8CHgc8DfzfsIdz+LUnra3pG/RXgM8DSeg+IiMMRMR8R8/1+f1NDuP1bkta3Yagj4kngx5l57u0el5nHMnMmM2cmJyc3NYTbvyVpfU2OPh4HPhIRfwq8A3hXRLySmc8Oawi3f0vS+jYMdWbOArMAEfGHwN8PM9Lg9m9JejtltpC7/VuSBttUqDPz28C3RzKJJGkg35koScUZakkqzlBLUnGGWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1JxRlqSSrOUEtScYZakooz1JJUXJnbnErSb6pT569w9PQlrl6/ydTEOC8M+X76hlqS7sGp81eYnbvA9GKPQ9t7nLkxzezcAsDQYm2oJekeHD19ienFHid2foEdLHKLMZ5ZOMLR0zuHFmrPqCXpHly9fpMD23rsYJGxWGIHixzY1uPq9ZtD+x6GWpLuwdTEOGeWprnFGIu5jVuMcWZpmqmJ8aF9D48+JOkevHBwH7NzCzyzcIQD23qcWZqmNzbNiwf3De17GGpJuge3z6GPnt7Jy9ffy9TEOC/6qg9JquWp/buHGua1PKOWpOIMtSQVZ6glqThDLUnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqzlBLUnEb3uY0Ih4B/gl4CEjgWGa+NOrBJGkjo97+XUWT+1EvAp/OzNci4reAcxHx75l5ccSzSdK6tmL7dxUbhjozrwHXVv78s4joAbsBQy2pNVux/buKTZ1RR8QeYD9wdsDnDkfEfETM9/v94UwnSevYiu3fVTQOdUS8E/g68MnM/Onaz2fmscycycyZycnJYc4oSb9iK7Z/V9FoZ2JE7GA50icyc260I0nSxrZi+3cVTV71EcBXgV5mfnn0I0nSxrZi+3cVTZ5RPw58FLgQEa+vfOxIZr46urEkaWOj3v5dRZNXffwXEFswiyRpAN+ZKEnFGWpJKs5QS1JxhlqSijPUklScoZak4gy1JBVnqCWpOEMtScUZakkqzlBLUnGGWpKKM9SSVFyjxQGStFZXNoBXYKglbVqXNoBXYKglbVqXNoBX4Bm1pE3r0gbwCgy1pE3r0gbwCjz6kLRpXdoAXoGhlrRpXdoAXoGhlvRr6coG8Ao8o5ak4gy1JBVnqCWpOEMtScUZakkqzlBLUnGGWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1JxRlqSSrOUEtScY1ucxoRHwJeArYDxzPziyOdStK63P7dPRuGOiK2A/8A/DFwGfhORHwjMy+OejhJd3P7dzc1eUb9+8D3M/MNgIj4F+AQYKilLeb2725qcka9G3hr1d8vr3zsLhFxOCLmI2K+3+8Paz5Jq7j9u5uG9svEzDyWmTOZOTM5OTmsLytpFbd/d1OTo48rwCOr/v7wysckbTG3f3dTk1B/B/idiNjLcqCfBv5ipFNJGsjt3920YagzczEi/hY4zfLL8/4xM7838skkDeT27+5p9DrqzHwVeHXEs0iSBvCdiZJUnKGWpOIMtSQVZ6glqbjIzOF/0Yg+8MNf8z9/EPjJEMf5Tea1uJvX425ejzvuh2vx25k58N2CIwn1vYiI+cycaXuOCrwWd/N63M3rccf9fi08+pCk4gy1JBVXMdTH2h6gEK/F3bwed/N63HFfX4tyZ9SSpLtVfEYtSVrFUEtScWVCHREfiohLEfH9iPhs2/O0KSIeiYj/jIiLEfG9iHi+7ZnaFhHbI+J8RPxb27O0LSImIuJkRPxvRPQi4g/anqlNEfGplZ+T/4mIf46Id7Q907CVCPWqBbp/AjwK/HlEPNruVK1aBD6dmY8CB4C/6fj1AHge6LU9RBEvAd/MzN8Ffo8OX5eI2A18ApjJzMdYvhXz0+1ONXwlQs2qBbqZuQDcXqDbSZl5LTNfW/nzz1j+QezsDYgj4mHgw8DxtmdpW0S8G/gA8FWAzFzIzOvtTtW6MWA8IsaAXcDVlucZuiqhbrRAt4siYg+wHzjb7iSt+grwGWCp7UEK2Av0ga+tHAUdj4gH2h6qLZl5BfgS8CZwDbiRmd9qd6rhqxJqDRAR7wS+DnwyM3/a9jxtiIgngR9n5rm2ZyliDHgf8HJm7gd+DnT2dzoR8R6W//W9F5gCHoiIZ9udaviqhNoFumtExA6WI30iM+fanqdFjwMfiYj/Y/lI7I8i4pV2R2rVZeByZt7+F9ZJlsPdVR8EfpCZ/cy8BcwB7295pqGrEupfLtCNiJ0s/zLgGy3P1JqICJbPIHuZ+eW252lTZs5m5sOZuYfl/y/+IzPvu2dMTWXmj4C3IuL22vEngIstjtS2N4EDEbFr5efmCe7DX6422pk4ai7Q/RWPAx8FLkTE6ysfO7Kyu1L6OHBi5UnNG8BzLc/Tmsw8GxEngddYfrXUee7Dt5P7FnJJKq7K0YckaR2GWpKKM9SSVJyhlqTiDLUkFWeoJak4Qy1Jxf0CnP++rG/VJ1oAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "emE8KhR9BF-Z"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}