Chap03-Getting_started_with_neural_networks/3.6-predicting_house_prices.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.6 주택 가격 예측: 회귀 문제\n",
    "\n",
    "이 노트북은 케라스 창시자에게 배우는 딥러닝 책의 3장 6절의 코드 예제입니다. 책에는 더 많은 내용과 그림이 있습니다. 이 노트북에는 소스 코드에 관련된 설명만 포함합니다. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "앞의 두 예제는 분류 문제입니다. 입력 데이터 포인트의 개별적인 레이블 하나를 예측하는 것이 목적입니다. 또 다른 종류의 머신 러닝 문제는 개별적인 레이블 대신에 **연속적인 값을 예측하는 회귀(regression)입니다**. 예를 들어 기상 데이터가 주어졌을 때 내일 기온을 예측하거나 소프트웨어 명세가 주어졌을 때 소프트웨어 프로젝트가 완료될 시간을 예측하는 것입니다.\n",
    "\n",
    "회귀와 로지스틱 회귀 알고리즘을 혼돈하지 마세요. 로지스틱 회귀는 회귀 알고리즘이 아니라 분류 알고리즘입니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.6.1 보스턴 주택 가격 데이터셋\n",
    "\n",
    "1970년 중반 보스턴 외곽 지역의 범죄율, 지방세율 등의 데이터가 주어졌을 때 주택 가격의 중간 값을 예측해 보겠습니다.\n",
    "\n",
    "여기서 사용할 데이터셋은 이전 두 개의 예제와 다릅니다. 데이터 포인트가 506개로 비교적 개수가 적고 404개는 훈련 샘플로 102개는 테스트 샘플로 나누어져 있습니다. 입력 데이터에 있는 각 특성(예를 들어 범죄율)은 스케일이 서로 다릅니다. 어떤 값은 0과 1 사이의 비율을 나타내고 어떤 것은 1과 12 사이의 값을 가지거나 1과 100 사이의 값을 가집니다.\n",
    "\n",
    "데이터를 살펴보겠습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'2.2.4'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import keras\n",
    "\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_data.shape : (404, 13)\n",
      "train_targets.shape : (404,)\n",
      "test_data.shape : (102, 13)\n",
      "test_targets.shape : (102,)\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import boston_housing\n",
    "\n",
    "(train_data, train_targets), (test_data, test_targets) =  boston_housing.load_data()\n",
    "\n",
    "print('train_data.shape : {}'.format(train_data.shape))\n",
    "print('train_targets.shape : {}'.format(train_targets.shape))\n",
    "print('test_data.shape : {}'.format(test_data.shape))\n",
    "print('test_targets.shape : {}'.format(test_targets.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "여기서 볼 수 있듯이 404개의 훈련 샘플과 102개의 테스트 샘플이 있고 모두 13개의 수치 특성을 가지고 있습니다. 13개의 특성은 다음과 같습니다:\n",
    "\n",
    "1. Per capita crime rate.\n",
    "2. Proportion of residential land zoned for lots over 25,000 square feet.\n",
    "3. roportion of non-retail business acres per town.\n",
    "4. Charles River dummy variable (= 1 if tract bounds river; 0 otherwise).\n",
    "5. Nitric oxides concentration (parts per 10 million).\n",
    "6. Average number of rooms per dwelling.\n",
    "7. Proportion of owner-occupied units built prior to 1940.\n",
    "8. Weighted distances to five Boston employment centres.\n",
    "9. Index of accessibility to radial highways.\n",
    "10. Full-value property-tax rate per $10,000.\n",
    "11. Pupil-teacher ratio by town.\n",
    "12. 1000 * (Bk - 0.63) ** 2 where Bk is the proportion of Black people by town.\n",
    "13. % lower status of the population.\n",
    "\n",
    "타깃은 주택의 중간 가격으로 천달러 단위입니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([15.2, 42.3, 50. , 21.1, 17.7, 18.5, 11.3, 15.6, 15.6, 14.4, 12.1,\n",
       "       17.9, 23.1, 19.9, 15.7,  8.8, 50. , 22.5, 24.1, 27.5, 10.9, 30.8,\n",
       "       32.9, 24. , 18.5, 13.3, 22.9, 34.7, 16.6, 17.5, 22.3, 16.1, 14.9,\n",
       "       23.1, 34.9, 25. , 13.9, 13.1, 20.4, 20. , 15.2, 24.7, 22.2, 16.7,\n",
       "       12.7, 15.6, 18.4, 21. , 30.1, 15.1, 18.7,  9.6, 31.5, 24.8, 19.1,\n",
       "       22. , 14.5, 11. , 32. , 29.4, 20.3, 24.4, 14.6, 19.5, 14.1, 14.3,\n",
       "       15.6, 10.5,  6.3, 19.3, 19.3, 13.4, 36.4, 17.8, 13.5, 16.5,  8.3,\n",
       "       14.3, 16. , 13.4, 28.6, 43.5, 20.2, 22. , 23. , 20.7, 12.5, 48.5,\n",
       "       14.6, 13.4, 23.7, 50. , 21.7, 39.8, 38.7, 22.2, 34.9, 22.5, 31.1,\n",
       "       28.7, 46. , 41.7, 21. , 26.6, 15. , 24.4, 13.3, 21.2, 11.7, 21.7,\n",
       "       19.4, 50. , 22.8, 19.7, 24.7, 36.2, 14.2, 18.9, 18.3, 20.6, 24.6,\n",
       "       18.2,  8.7, 44. , 10.4, 13.2, 21.2, 37. , 30.7, 22.9, 20. , 19.3,\n",
       "       31.7, 32. , 23.1, 18.8, 10.9, 50. , 19.6,  5. , 14.4, 19.8, 13.8,\n",
       "       19.6, 23.9, 24.5, 25. , 19.9, 17.2, 24.6, 13.5, 26.6, 21.4, 11.9,\n",
       "       22.6, 19.6,  8.5, 23.7, 23.1, 22.4, 20.5, 23.6, 18.4, 35.2, 23.1,\n",
       "       27.9, 20.6, 23.7, 28. , 13.6, 27.1, 23.6, 20.6, 18.2, 21.7, 17.1,\n",
       "        8.4, 25.3, 13.8, 22.2, 18.4, 20.7, 31.6, 30.5, 20.3,  8.8, 19.2,\n",
       "       19.4, 23.1, 23. , 14.8, 48.8, 22.6, 33.4, 21.1, 13.6, 32.2, 13.1,\n",
       "       23.4, 18.9, 23.9, 11.8, 23.3, 22.8, 19.6, 16.7, 13.4, 22.2, 20.4,\n",
       "       21.8, 26.4, 14.9, 24.1, 23.8, 12.3, 29.1, 21. , 19.5, 23.3, 23.8,\n",
       "       17.8, 11.5, 21.7, 19.9, 25. , 33.4, 28.5, 21.4, 24.3, 27.5, 33.1,\n",
       "       16.2, 23.3, 48.3, 22.9, 22.8, 13.1, 12.7, 22.6, 15. , 15.3, 10.5,\n",
       "       24. , 18.5, 21.7, 19.5, 33.2, 23.2,  5. , 19.1, 12.7, 22.3, 10.2,\n",
       "       13.9, 16.3, 17. , 20.1, 29.9, 17.2, 37.3, 45.4, 17.8, 23.2, 29. ,\n",
       "       22. , 18. , 17.4, 34.6, 20.1, 25. , 15.6, 24.8, 28.2, 21.2, 21.4,\n",
       "       23.8, 31. , 26.2, 17.4, 37.9, 17.5, 20. ,  8.3, 23.9,  8.4, 13.8,\n",
       "        7.2, 11.7, 17.1, 21.6, 50. , 16.1, 20.4, 20.6, 21.4, 20.6, 36.5,\n",
       "        8.5, 24.8, 10.8, 21.9, 17.3, 18.9, 36.2, 14.9, 18.2, 33.3, 21.8,\n",
       "       19.7, 31.6, 24.8, 19.4, 22.8,  7.5, 44.8, 16.8, 18.7, 50. , 50. ,\n",
       "       19.5, 20.1, 50. , 17.2, 20.8, 19.3, 41.3, 20.4, 20.5, 13.8, 16.5,\n",
       "       23.9, 20.6, 31.5, 23.3, 16.8, 14. , 33.8, 36.1, 12.8, 18.3, 18.7,\n",
       "       19.1, 29. , 30.1, 50. , 50. , 22. , 11.9, 37.6, 50. , 22.7, 20.8,\n",
       "       23.5, 27.9, 50. , 19.3, 23.9, 22.6, 15.2, 21.7, 19.2, 43.8, 20.3,\n",
       "       33.2, 19.9, 22.5, 32.7, 22. , 17.1, 19. , 15. , 16.1, 25.1, 23.7,\n",
       "       28.7, 37.2, 22.6, 16.4, 25. , 29.8, 22.1, 17.4, 18.1, 30.3, 17.5,\n",
       "       24.7, 12.6, 26.5, 28.7, 13.3, 10.4, 24.4, 23. , 20. , 17.8,  7. ,\n",
       "       11.8, 24.4, 13.8, 19.4, 25.2, 19.4, 19.4, 29.1])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_targets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "이 가격은 일반적으로 10,000달러에서 50,000달러 사이입니다. 저렴하게 느껴질텐데 1970년대 중반이라는 걸 기억하세요. 아직 인플레이션에 영향을 받지 않은 가격입니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.6.2 데이터 준비\n",
    "\n",
    "상이한 스케일을 가진 값을 신경망에 주입하면 문제가 됩니다. 네트워크가 이런 다양한 데이터에 자동으로 맞추려고 할 수 있지만 이는 확실히 학습을 더 어렵게 만듭니다. 이런 데이터를 다룰 때 대표적인 방법은 **특성별로 정규화**를 하는 것입니다. 입력 데이터에 있는 각 특성(입력 데이터 행렬의 열)에 대해서 특성의 평균을 빼고 표준 편차로 나눕니다. 특성의 중앙이 0 근처에 맞춰지고 표준 편차가 1이 됩니다. 넘파이를 사용하면 간단하게 할 수 있습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean = train_data.mean(axis=0)\n",
    "std = train_data.std(axis=0)\n",
    "train_data -= mean\n",
    "train_data /= std\n",
    "\n",
    "test_data -= mean\n",
    "test_data /= std"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "테스트 데이터를 정규화할 때 사용한 값이 **훈련 데이터에서 계산한 값임을 주목**하세요. 머신 러닝 작업 과정에서 절대로 **테스트 데이터에서 계산한 어떤 값도 사용해서는 안 됩니다**. 데이터 정규화처럼 간단한 작업조차도 그렇습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.6.3 모델 구성\n",
    "\n",
    "샘플의 개수가 적기 때문에 64개의 유닛을 가진 두 개의 은닉층으로 작은 네트워크를 구성하여 사용하겠습니다. 일반적으로 훈련 데이터의 개수가 적을수록 과대적합이 더 쉽게 일어나므로 작은 모델을 사용하는 것이 과대적합을 피하는 한 방법입니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_1 (Dense)              (None, 64)                896       \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 64)                4160      \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 1)                 65        \n",
      "=================================================================\n",
      "Total params: 5,121\n",
      "Trainable params: 5,121\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "from keras import models\n",
    "from keras import layers\n",
    "from keras import backend as K\n",
    "\n",
    "K.clear_session()\n",
    "\n",
    "def build_model():\n",
    "    # 동일한 모델을 여러 번 생성할 것이므로 함수를 만들어 사용\n",
    "    model = models.Sequential()\n",
    "    model.add(layers.Dense(64, activation='relu', \n",
    "                           input_shape=(train_data.shape[1], )))\n",
    "    model.add(layers.Dense(64, activation='relu'))\n",
    "    model.add(layers.Dense(1))\n",
    "    model.compile(optimizer='rmsprop', \n",
    "                  loss='mse', metrics=['mae'])  # MAE: Mean Absolute Error\n",
    "    return model\n",
    "\n",
    "build_model().summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 네트워크의 마지막 층은 하나의 유닛을 가지고 있고 활성화 함수가 없습니다(선형 층이라고 합니다). 이것이 전형적인 스칼라 회귀(하나의 연속적인 값을 예측하는 회귀)를 위한 구성입니다. 활성화 함수를 적용하면 출력 값의 범위를 제한하게 됩니다. 예를 들어 마지막 층에 sigmoid 활성화 함수를 적용하면 네트워크가 0과 1 사이의 값을 예측하도록 학습될 것입니다. 여기서는 마지막 층이 순수한 선형이므로 네트워크가 어떤 범위의 값이라도 예측하도록 자유롭게 학습됩니다.\n",
    "\n",
    "이 모델은 mse 손실 함수를 사용하여 컴파일합니다. 이 함수는 평균 제곱 오차(mean squared error)의 약자로 예측과 타깃 사이 거리의 제곱입니다. 회귀 문제에서 널리 사용되는 손실 함수입니다.\n",
    "\n",
    "훈련하는 동안 모니터링을 위해 새로운 지표인 **평균 절대 오차**(MAE, Mean Absolute Error)를 측정합니다. 이는 예측과 타깃 사이 거리의 절댓값입니다. 예를 들어 이 예제에서 MAE가 0.5이면 예측이 평균적으로 \\$500 정도 차이가 난다는 뜻입니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.6.4 K-Fold 검증을 사용한 훈련 검증\n",
    "\n",
    "(훈련에 사용할 에포크의 수 같은) 매개변수들을 조정하면서 모델을 평가하기 위해 이전 예제에서 했던 것처럼 데이터를 훈련 세트와 검증 세트로 나눕니다. 데이터 포인트가 많지 않기 때문에 검증 세트도 매우 작아집니다(약 100개의 샘플). 결국 검증 세트와 훈련 세트로 어떤 데이터 포인트가 선택됐는지에 따라 검증 점수가 크게 달라집니다. 검증 세트의 분할에 대한 검증 점수의 분산이 높습니다. 이렇게 되면 신뢰있는 모델 평가를 신뢰있게 할 수 없습니다.\n",
    "\n",
    "이런 상황에서 가장 좋은 방법은 K-Fold 교차 검증(Cross Validation)을 사용하는 것입니다. 데이터를 K개의 분할(즉, 폴드)로 나누고(일반적으로 K = 4 또는 5), K개의 모델을 각각 만들어 K - 1개의 분할에서 훈련하고 나머지 분할에서 평가하는 방법입니다. 모델의 검증 점수는 K 개의 검증 점수의 평균이 됩니다.\n",
    "\n",
    "![](./images/k-fold.PNG)\n",
    "\n",
    "코드로 보면 이해하기 쉽습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "처리중인 폴드 # 0\n",
      "처리중인 폴드 # 1\n",
      "처리중인 폴드 # 2\n",
      "처리중인 폴드 # 3\n",
      "CPU times: user 2min 34s, sys: 27.5 s, total: 3min 1s\n",
      "Wall time: 1min 19s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "import numpy as np\n",
    "\n",
    "k = 4\n",
    "num_val_samples = len(train_data) // k \n",
    "num_epochs = 100\n",
    "all_scores = []\n",
    "\n",
    "for i in range(k):\n",
    "    print('처리중인 폴드 #', i)\n",
    "    # 검증 데이터 준비 : k 번째 분할\n",
    "    val_data = train_data[i * num_val_samples : (i+1)*num_val_samples]\n",
    "    val_targets = train_targets[i * num_val_samples : (i+1)*num_val_samples]\n",
    "    \n",
    "    # 훈련 데이터 준비 : 다른 분할 전체\n",
    "    partial_train_data = np.concatenate(\n",
    "        [train_data[:i * num_val_samples], \n",
    "         train_data[(i+1)*num_val_samples:]], axis=0)\n",
    "    partial_train_targets = np.concatenate(\n",
    "        [train_targets[:i * num_val_samples], \n",
    "         train_targets[(i+1) * num_val_samples:]], axis=0)\n",
    "    \n",
    "    # Keras 모델 구성(compile 포함)\n",
    "    model = build_model()\n",
    "    # 모델 훈련(verbose=0 이므로 훈련 과정이 출력도지 않는다.)\n",
    "    model.fit(partial_train_data, partial_train_targets, \n",
    "              epochs=num_epochs, batch_size=1, verbose=0)\n",
    "    # 검증 세트로 모델 평가\n",
    "    val_mse, val_mae = model.evaluate(val_data, val_targets, verbose=0)\n",
    "    all_scores.append(val_mae)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[2.198098114221403, 2.1075122663290196, 2.9046623824846627, 2.453442376438934]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.415928784868505"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(all_scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "검증 세트가 다르므로 확실히 검증 점수가 2.0에서 2.8까지 변화가 큽니다. 평균값(2.4)이 각각의 점수보다 훨씬 신뢰할 만한 점수입니다. 이것이 K-겹 교차 검증의 핵심입니다. 이 예에서는 평균적으로 3,000달러 정도 차이가 납니다. 주택 가격의 범위가 10,000달러에서 50,000달러 사이인 것을 감안하면 비교적 큰 값입니다.\n",
    "\n",
    "신경망을 조금 더 오래 500 에포크 동안 훈련해 보죠. 각 에포크마다 모델이 얼마나 개선되는지 기록하기 위해 훈련 루프를 조금 수정해서 에포크의 검증 점수를 로그에 저장하겠습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import backend as K\n",
    "\n",
    "# 메모리 해제\n",
    "K.clear_session()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "처리중인 폴드 # 0\n",
      "처리중인 폴드 # 1\n",
      "처리중인 폴드 # 2\n",
      "처리중인 폴드 # 3\n",
      "CPU times: user 13min 20s, sys: 2min 17s, total: 15min 37s\n",
      "Wall time: 6min 56s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "import numpy as np\n",
    "\n",
    "k = 4\n",
    "num_val_samples = len(train_data) // k \n",
    "num_epochs = 500\n",
    "all_mae_histories = []\n",
    "\n",
    "for i in range(k):\n",
    "    print('처리중인 폴드 #', i)\n",
    "    # 검증 데이터 준비 : k 번째 분할\n",
    "    val_data = train_data[i * num_val_samples : (i+1)*num_val_samples]\n",
    "    val_targets = train_targets[i * num_val_samples : (i+1)*num_val_samples]\n",
    "    \n",
    "    # 훈련 데이터 준비 : 다른 분할 전체\n",
    "    partial_train_data = np.concatenate(\n",
    "        [train_data[:i * num_val_samples], \n",
    "         train_data[(i+1)*num_val_samples:]], axis=0)\n",
    "    partial_train_targets = np.concatenate(\n",
    "        [train_targets[:i * num_val_samples], \n",
    "         train_targets[(i+1) * num_val_samples:]], axis=0)\n",
    "    \n",
    "    # Keras 모델 구성(compile 포함)\n",
    "    model = build_model()\n",
    "    # 모델 훈련(verbose=0 이므로 훈련 과정이 출력도지 않는다.)\n",
    "    history = model.fit(partial_train_data, partial_train_targets,\n",
    "                        validation_data=(val_data, val_targets),\n",
    "                        epochs=num_epochs, batch_size=1, verbose=0)\n",
    "    mae_history = history.history['val_mean_absolute_error']\n",
    "    all_mae_histories.append(mae_history)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "그다음 모든 폴드에 대해 에포크의 MAE 점수의 평균을 계산합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "average_mae_history = [\n",
    "    np.mean([x[i] for x in all_mae_histories]) for i in range(num_epochs)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "그래프로 그려 보겠습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XecFOX9B/DPd3evcwcHHB05qoBU6UVFECSKJRG7xhaNxpbExBIT/VlCNCbYY4kmGntvWBApAiLlkN57hytc73f7/P6YsjO7s+WO2zuO/bxfr3vd7uzczjN7s8/36SNKKRAREQGAq6kTQERExw8GBSIiMjEoEBGRiUGBiIhMDApERGRiUCAiIhODAhERmRgUiIjIxKBAREQmT1MnoK7atm2rMjMzmzoZRETNysqVK3OVUhnh9mt2QSEzMxNZWVlNnQwiomZFRPZEsh+bj4iIyMSgQEREJgYFIiIyMSgQEZGJQYGIiEwMCkREZGJQICIiU8wEhS2HizHz2y3ILals6qQQER23YiYobM8uwTPztuNoaVVTJ4WI6LgVM0HBJdpvr1JNmxAiouNYzAQFES0qeL1NnBAiouNYDAUF7TdrCkREwcVMUHAZUYGIiIKKoaCg/WZNgYgouBgKCnqfAmMCEVFQMRMU2KdARBReDAUFLSooBgUioqBiJigYfQqMCUREwcVQUGCfAhFROFEPCiLiFpFVIjLL4bVrRSRHRFbrP7+KXjq03+xTICIKztMIx7gTwCYAaUFef08pdVu0EyEwagoMCkREwUS1piAiXQCcC+CVaB4nEkafAhgTiIiCinbz0VMA7gYQasWhi0RkrYh8KCJdnXYQkZtEJEtEsnJycuqVEJeLfQpEROFELSiIyDQA2UqplSF2+wJAplJqEIA5AF532kkp9bJSarhSanhGRka90sMZzURE4UWzpjAOwPkishvAuwAmisib1h2UUnlKKeOuN68AGBa95LBPgYgonKgFBaXUfUqpLkqpTACXAZinlLrKuo+IdLQ8PR9ah3RUmPMUonUAIqITQGOMPrIRkYcBZCmlPgdwh4icD6AGwFEA10bruC7OaCYiCqtRgoJSagGABfrjByzb7wNwX2OkwcWb7BARhRUzM5o5eY2IKLwYDApNmw4iouNZzAQF353XGBWIiIKJuaDAmgIRUXAxFBS03+xTICIKLmaCAvsUiIjCi6GgwHkKREThxExQ8E1ea+KEEBEdx2IoKGi/2adARBRczAQF3012mjghRETHsdgJCsaCeKwpEBEFFTNBwbjJDmMCEVFwsRMU2KdARBRWDAUF9ikQEYUTM0HBWPmINQUiouBiJygY8xSaOB1ERMezmAkKLo4+IiIKK4aCgnHnNQYFIqJgYiYocEE8IqLwYigosE+BiCicmAkK7FMgIgovhoKCMU+BQYGIKJiYCQrsUyAiCi9mggJrCkRE4cVMUPCtktq06SAiOp7FTFBw8XacRERhxVxQYJ8CEVFwMRMUuCAeEVF4sRMU2KdARBRWDAUFgQj7FIiIQol6UBARt4isEpFZDq8liMh7IrJdRJaJSGY00+ISYZ8CEVEIjVFTuBPApiCv3QAgXynVC8CTAB6PZkIE7FMgIgolqkFBRLoAOBfAK0F2uQDA6/rjDwFMEmPluihwiXBBPCKiEKJdU3gKwN0AvEFe7wxgHwAopWoAFAJoE63EiLCmQEQUStSCgohMA5CtlFrZAO91k4hkiUhWTk5Ovd/HJcLRR0REIUSzpjAOwPkishvAuwAmisibfvscANAVAETEA6AlgDz/N1JKvayUGq6UGp6RkVHvBInwzmtERKFELSgope5TSnVRSmUCuAzAPKXUVX67fQ7gGv3xdH2fqOXaHH1ERBSap7EPKCIPA8hSSn0O4FUAb4jIdgBHoQWPKB4bUOxqJiIKqlGCglJqAYAF+uMHLNsrAFzcGGkA2KdARBROzMxoBrRbcnL0ERFRcDEVFESEQYGIKISYCgou4YJ4REShxFRQEI4+IiIKKWhQEJG7LY8v9nttRjQTFS0urpJKRBRSqJqCdXjofX6vTY1CWqJOwD4FIqJQQgUFCfLY6XmzwD4FIqLQQgUFFeSx0/NmgX0KREShhZq8NlhEiqDVCpL0x9CfJ0Y9ZVHgcrFPgYgolKBBQSnlbsyENAb2KRARhVanIakikiIiV4nIl9FKUDRpM5qbOhVERMevsEFBROJF5Oci8gGAQwAmAXgx6imLAt55jYgotKDNRyIyBcDlAKYAmA/gfwBGKKWua6S0NTjeeY2IKLRQNYVvAPQAMF4pdZVS6gsEv61ms6CtksqgQEQUTKjRR6dCm8D2nYjshHb3tGbd+azdea2pU0FEdPwKWlNQSq1WSt2rlOoJ4EEAQwDEicjXInJTo6WwAWl9CqwpEBEFE9HoI6XUEqXU7QC6AHgSwOiopipKOHmNiCi0UB3NpwZ5KRfAc9FJTnRxQTwiotBC9SlkAVgPLQgA9vWOFICJ0UpUtAjnKRARhRQqKPwewHQA5dA6mT9RSpU0SqqihKOPiIhCC9XR/JRSajyA2wF0BTBXRN4XkSGNlroGxj4FIqLQwnY0K6V2AvgMwLcARgLoE+1ERYuLk9eIiEIK1dHcA9o8hQsA7IPWhDRDKVXeSGlrcALeT4GIKJRQfQrbAayFVksoAnASgFtEtP5mpdTMqKeugbmEq6QSEYUSKig8DN/NdFo0QlqiTutobupUEBEdv0LdT+H/GjEdjYIL4hERhVan+yk0d6wpEBGFFlNBgTUFIqLQYioo8CY7REShhepoBgCISAKAiwBkWvdXSj0cvWRFhwhQy9lrRERBhQ0K0IakFgJYCaAyusmJLreLQ1KJiEKJJCh0UUpNresbi0gigIUAEvTjfKiUetBvn2sBPAHggL7pOaXUK3U9VqTcIqwpEBGFEElQWCIiA5VS6+r43pUAJiqlSkQkDsBiEflaKbXUb7/3lFK31fG968XlYlAgIgolkqAwHsC1IrILWkavrRah1KBQf6S05UiNVVXj9J8mzZHdnNFMRBRSJEHhZ/V9cxFxQ+uL6AXgeaXUMofdLhKR0wFsBfA7pdS++h4vHDdrCkREIUWySuoeAK0AnKf/tNK3haWUqlVKDYF2G8+RIjLAb5cvAGTqtY45AF53eh8RuUlEskQkKycnJ5JDO2JQICIKLWxQEJE7AbwFoJ3+86aI3F6XgyilCgDMBzDVb3ueUsoY0fQKgGFB/v5lpdRwpdTwjIyMuhzaxu0S1LL5iIgoqEiaj24AMEopVQoAIvI4gB8BPBvqj0QkA0C1UqpARJIATAbwuN8+HZVSh/Sn5wPYVMf014lLBF5vNI9ARNS8RRIUBECt5Xkt7PdrDqYjgNf1fgUXgPeVUrNE5GEAWUqpzwHcISLnA6gBcBTAtXVJfF25XZy8RkQUSiRB4b8AlonIJ/rzCwG8Gu6PlFJrAQx12P6A5fF9AO6LLKnHjs1HREShhQ0KSqmZIrIA2tBUALhOKbUqqqmKEq35iEGBiCiYULfjTFNKFYlIawC79R/jtdZKqaPRT17DYk2BiCi0UDWFtwFMgzbPwJqTiv68RxTTFRUuLnNBRBRSqDuvTdN/d2+85ESXx8XmIyKiUCKZpzA3km3NgdslqGFQICIKKlSfQiKAZABtRSQdvmGoaQA6N0LaGpyLS2cTEYUUqk/h1wB+C6ATtH4FIygUAXguyumKCi6dTUQUWqg+hacBPC0ityulQs5ebi60mgKglIJIJPPviIhiSyTzFJ7VF7LrDyDRsv1/0UxYNLj1QOBVgJsxgYgoQCT3aH4QwARoQeEraEtpLwbQ/IKC3q1e61VwuxgViIj8hR19BGA6gEkADiulrgMwGEDLqKYqSlwuo6bAfgUiIieRBIVypZQXQI2IpAHIBtA1usmKDqP5iJ3NRETOIlkQL0tEWgH4N7RRSCXQls5udowmI85VICJyFklH82/0hy+KyDcA0vQVUJsdIyhwVjMRkbNQk9dODfWaUuqn6CQpeoygwEXxiIichaop/FP/nQhgOIA10CawDQKQBWBMdJPW8FzCmgIRUShBO5qVUmcqpc4EcAjAqfo9kodBu3HOgcZKYENiTYGIKLRIRh+drJRaZzxRSq0H0C96SYoejj4iIgotktFHa0XkFQBv6s+vBNAsO5rNeQreJk4IEdFxKpKgcB2AWwDcqT9fCOCFqKUoiswZzWw+IiJyFMmQ1AoAT+o/zZqLzUdERCGFGpL6vlLqEhFZB/vtOAEASqlBUU1ZFHhcWlWBQYGIyFmomoLRXDStMRLSGKwL4hERUaBQ91M4pP/e03jJiS5zngL7FIiIHIVqPiqGQ7MRtAlsSimVFrVURYk5T4E1BSIiR6FqCqmNmZDG4OLkNSKikCIZkgoAEJF2sN95bW9UUhRFbi5zQUQUUtgZzSJyvohsA7ALwPcAdgP4Osrpigo2HxERhRbJMhePABgNYKtSqju0u7AtjWqqosScp8DmIyIiR5EEhWqlVB4Al4i4lFLzoa2a2uy4ucwFEVFIkfQpFIhIC2jLW7wlItkASqObrOjw3XmNUYGIyEkkNYULAJQD+B2AbwDsAHBeuD8SkUQRWS4ia0Rkg4g85LBPgoi8JyLbRWSZiGTWLfl1Y9YU2HxEROQo1DyF5wG8rZT6wbL59Tq8dyWAiUqpEhGJA7BYRL5WSln7I24AkK+U6iUilwF4HMCldThGnfiWzo7WEYiImrdQNYWtAP4hIrtF5O8iMrQub6w0JfrTOP3Hv4h+AXyB5kMAk0T0nDsKXFzmgogopFB3XntaKTUGwBkA8gD8R0Q2i8iDItInkjcXEbeIrAaQDWCOUmqZ3y6dAezTj1cDoBBAm3qcR0TYfEREFFrYPgWl1B6l1ONKqaEALgdwIYBNkby5UqpWKTUEQBcAI0VkQH0SKSI3iUiWiGTl5OTU5y0A8M5rREThRDJ5zSMi54nIW9AmrW0B8Iu6HEQpVQBgPoCpfi8dANDVOA6AltBqJf5//7J+j+jhGRkZdTm0jYs1BSKikIIGBRGZLCL/AbAfwI0AvgTQUyl1mVLqs3BvLCIZItJKf5wEYDKAzX67fQ7gGv3xdADzlIpejs2aAhFRaKHmKdwH4G0Adyml8uvx3h0BvC4ibmjB532l1CwReRhAllLqcwCvAnhDRLYDOArgsnocJ2K+eQoMCkRETkKtkjrxWN5YKbUWQMCIJaXUA5bHFQAuPpbj1EWcfpedmloGBSIiJ5FMXjthJMW5AQBlVTVNnBIiouNTbAWFeC0olFfVNnFKiIiOTzEVFOI9LsS5BWXVDApERE5iKigAWhMSawpERM5iLigkx3vYp0BEFEQMBgU3ylhTICJyFHtBIYHNR0REwcReUIjzoJTNR0REjmIuKCTFu7F051EcLChv6qQQER13Yi4oJOtzFaY9u7iJU0JEdPyJuaBQrs9ROFpa1cQpISI6/sRcUDhcWAEAaJEQai1AIqLYFHtBoUgLCl3Sk5o4JUREx5+YCwo3n9ETAJCRmtDEKSEiOv7EZFAYkZnO5bOJiBzEXFAAtIXxqmq9TZ0MIqLjTmwGBbcLVTUMCkRE/mIyKMS5XahmTYGIKEBMBoV4D2sK9VVaWYNsfQQXEZ14YjYoVDIo1MvP//UDRs6Y29TJIKIoic2gwOajett6pKSpk0DNQG5JJTLv/RKLtuU0dVKojmIzKHD0EVFQe/JKj7mJcNXeAgDAaz/sboAUUWOKybUeOPro2FXXehHnjskyxQnvjCcWAAB2P3Zuvd/D+H7xGml+YvI/Fudh89GxqqiO7RsVPTlnK2Z8tampk3HcMr5fcZ7YyGKW7sxDbkllUyejQcTGf8yP1qeg4PVyVnN9lcdoUDhUWI4lO3Lx9NxteHnhzqZOTp1U1tSisLy6UY5lNM/GuaVRjtfULnt5Kaa/sKSpk9EgYjMo6KWXLUeKmzglzVdF1bHXtB6ZtRG3vv1TA6Sm8Ux7ZjGu+Pcy27Yr/r0Umfd+iateWYbaOhQ0Gru2de1/VmDwQ982yrGMmkJ8DDQfGee6O6+siVPSME78/5gD40L92dOLmjglzVeomsKCLdkoqghfIn118S58ufZQxMdcsCUbZU18K9U8v/twVNd6sWRHHgBg8fZcFEVYEv9xRx76/uUbLN91tEHT99y8bVi4NQcllTVQyh6gftyppdMIXD9sz8X+/OhkZEafQnwDNh8t2paDIQ9/i+IIrq368v/MInGiDW+PzaDgcKF+/NN+ZN77ZaNVr5sja3NbsKCQW1KJa/+7Are9vapBj709uwTX/ncFHvhsQ4O+r2HF7qM4VBj8Fq3PzN2GDQcLA7b736ypJsKawo87cvXfeRHtX1hWHdGIoH98uxW//M9yDHhwNv734x7HfQrKtDRf+coyTH0qOgUj4/qoa0dzUUU1pr+wBOv2B37Wz87djoKyaqx1eK0hrD9QiEEPfYuPVu4HAGQXVeBoaRUmPDE/ZPCuPMGaUmMyKFgvVKNE8+L3OwCA927WHSmqwL6j9lJkmeXiL69y/iIYpaaNB4siPpZROssvrcKnqw6Y271ehefmbUNBWZWZ+e7Mic48iYtf/BFTnlxoPt90qAiLt2kZd15JJWbO2YobXssK+LucYnvnYmVNhBmEaG3t3ghLppNmfh920qB/KXeRnn5/eaVVZpNHSaW95tVQ/Wyl+vu66tCl8N8fduGxrzcja08+Xly4I+D1Ni3iAcDWoVtWVYNtDdQM/NLCnSiuqMHyXUehlMLIGXMxeeb32J1Xhgc/DyyMrNqbj6tfXRbwGTaER2ZtxLvL9zb4+0YiRoOC70rNK9UuMGMpbU9druJmIK+kMiDjcrL+QCGmPPm9WTUfNWMuTvv7fNs+ZZaLP1h7uBEsaryhq9TWtncjkPzu/dX47XursSevFIDWHPOPb7figc82mMfzRKGN2shMiyt85/ezpxfhqle1voPt2Vogcqph+o84Mc5lf34ZMu/9Eiv3OJcwjcvMPwv+cUceVuwO/BvjOKH6LPybMTq1SrQ9N5pNc4srgzZzNdT8ndJK7f9VXYcl6h/6YiPeXqZlhF3TkwNeb52iBYWDBVqNyetVGP7od5j85MJ6Nfv426tfd0UV1WZGbzQX5pZU4kBBuXktAMDv31+DRdtyselQw/ZNllXV4PUlu/HtxiMN+r6RismgUGH58uQWa/9048tQVetFj/u+xJNztjZJ2hrasEe/w4i/fhd2v39+uwVbj5QEZEgHCsox7dlFyC6qsJWIgjUfGUGhusYLr1dh6lML8fmagwH75Zf5ml2MDMq4VaqRORul6PyyKhTo+9Sl43LZzjxc9coyTH9hCbKLK3D+c4sdm4gqqoNnhNuzi3Hpy0sBAB3SEgNezy2xNx9V1XixbGceZul9JR9k7Xd8X4EWFSpram0ZzeX/XoqLX/wRq/cVYOCDswNqrrM3HAYAzN10BP0f+AbDH/0O8zdnY39+mVk6Nyzelmur7aUkuLU0l1ahqMK5dFsZ4rOoCyMtxvcqXA3EP9iVVAYGLb1yhb36Oa07UIgy/XorrqxB1u6juOfDtREFiOpaL37ckYeB/zfb/IyM980prgz4v+YUV2LcY/Nw1szvkV9ahbKqGrPgYxQsj9W+o2UoLK/G6r0FqPEq5DXRENeoBQUR6Soi80Vko4hsEJE7HfaZICKFIrJa/3kgWumxKrRkSEYJzKgplFbWwquAp+dua4ykHDeMErj/pL5n527D+gNF+HjVAbP0B4SoKejbq70KpVU12Hy4GD/tyQ/Yz9oWn19WDa9XIUEviRtfdLdenK6pVWY7uKcOQxwvfXkpFm/PRdaefHyQtR9r9xfitSW7A/YLVf0/a6avSalFYuBcT/8+qKLyalz68lI89vVmAOHb1F/6fifOmvk99vqNXHl9yW4UV9bgq3WHbJncb976CUeKKnDD61koq6pFbkklrnttBcY/Ph8r/T7nnbmlOOMJX23PSH9eSWXQvrPK2vq3j+eVVOKcpxdhw0FfZl1d40V2cQV6/OkrfKi31WftPhrQWex/Pb25dC8enbXRtq1Mv/6MzvFVe33nm19ahekv/oj3svYhvyx8v+A9H67F5f9eiuKKGvzxwzUY87e55t9lF4euXQ99ZA6mv/CjmWccKmiYBSJP+/t8XPj8D1im91/4B6bGEs2aQg2Au5RS/QGMBnCriPR32G+RUmqI/vNwFNNjcrt8p238842of6JMQKkrowReVatspbYdeht+UpwbpVXhawrG6KCaWq8ZRHIcPtM8ywV/9lMLcd1rK8wMtLC8GvM3Z2N3rladr671Ir9U+8J6XC7c9/FanPuMr4P0m/WHzVqGwT/TMwLO7tzSgImL1hL2hyv3Y/ij9prV6X0yADgHD//j+I9OinO7UFBWhT5//hpLdvja+P0/v2W78myZv/E/WH+gEAV+mZw1M7Rt31cQsM1aAE/0aDWFvJKqoEHBWigIFvg/XLkfX6w5iC2H7c0maw8UYuOhItzxzip8uU6rKVXXerF0p5bJPTF7MxZvy8X0F3/Eg59vwJ8/XYfPVmt9SGUOfVSvLN5le258/rklVXh+/nb81xLg9+f7alTZxaEz6X/M3oKPLX1XS3cexSH9+uncKgnZxRVhm1w3Hioym8Yaoh/SqEntyi01a+s5xZW47e2fbNdNY4haUFBKHVJK/aQ/LgawCUDnaB2vLq4bl4k/TOkDwNeMYfyDT6SgUJd2VqOfpbK6FqNm+DLFHTlaxpzgcdkyT6eO5rKqGjwxewsALTMyvsQ5Rb7P9EBBOV5YsAOPfbPZ9rffb80x2+yPFFXgutdW4C/6SKNqrzL/TznFFXhn+T5sOFgEr1fhjaV7cPObK3Htf5fb3u9AvvMXdfaGI7jnw7W2bdZg96dP1gVcA/+7fiTO6tfe1ucAAKmJnoC2ef8qf5xHsHpfAapqvHhhwQ7kFFfiYEE5yv2G1i7bddSWUW/VO0+X7TqKhX6Lyjll/gDM2sawbunokZES8LoRDIsqqs1jGX0bhwrLUVRRbeuX6PuXb2xNW4Y/fLAGt7+zCmc/tdC23ciYjWtGO6Yya4pHiirNfpo5G4/gzaV7cee7q1FV4w06cMHKCByHC8vxxOwt2JNXhk4ttSa9rZbO5hV6SbugrAq3v7MK2UUVqLEUBJ6bv93x/S8f2RW/OLUzKqq9Ec2fMQqSBxyCQl37OKxDuFftLdCW4qn1YtbaQwHzYqKtUfoURCQTwFAATmc3RkTWiMjXInJKY6QnMc6NW8/sBREt49qT5ys9WksI32447Fha2ne0DM/M3dYgnVvrDxTWeR2mvJLKiIYnFpVHPirCaD4qrqixVVuNZp6yqlpbSdnpc3lm7nZssIw6MoKItaZw1/ur8fg3m7HGIWMzagrrD9iHHNbUes3mozWW4YhfrjuEv3y6HgBw2O/zMNreDdbM21pK1NLpOxe32JunpvRvD8A5ALRMigsolfpX+T0ugXGZiAhG/PU7jH1sHkr9MsEjRRW2YLRZL4UfKqzAne+uBgC8dPUw7ff3zjOpNx3SPvs/ndPPlslW1tRi8+EiM8Mvrqgxg4LxmY/52zxc8NwPAdfi28v2Yv7mbFTXerH1SHHAqLLKmloopfD8/O34fku2uT0l3o3e7VqgrEprAvNnDbDrDxairDr8tWpcf9bmoTP7tgMAs1YJAH/5bAOufGUpHvpiI75YcxCTn1yI/g/Oxqq9+bgtSGZ/7qCO+NsvBqFnRouw6fA/h4OWfqplO/Mwe8NhdL/vK2w5XOw4MkopFVC7sNYwy6trMapHa9vrf/crREVT1IOCiLQA8BGA3yql/Mcp/gSgm1JqMIBnAXwa5D1uEpEsEcnKyWmYpXhFBC3iPXh23nac8cQCsxSSbSnV3vTGSjzi164JAHd9sAYz52zFxz8dOKbAsHhbLqY9uxhvLnUeTx7MsEe/i+ieBtYOsHAzbY1RV8FqSqWVNbYq/tHSwOYH/xKTGRT0QKuUwqZDxWiR4LwOo5GG1X4Bo6ZWBTTLAPaMpbLai1Me+AbfbjiMf8zeEtAndLAwMIje/8k6LNqWY6sBuS2jzwZ0TsMzlw8FoAUF/2awtMQ4HCmyb/Ofy1BR7TU7W60D2/wn4RVX1CCn2H6OF53axfZ8cJdWAedgtVPPGFMS3LhgiK9SPuPLTZj61CKziaS4otrsV6us8Zpt9LtyS/FbPQAZ/vPDLlz32gq8vHAnpjy5EOc8Y5/XcO1/VmD+lmw8MXsLvtuUjcw2ybhlQk+8dv1IJMe7MX9LDrKDNMUYn/XK3fmOzUcGr1fhm/WHHSetjenZBkDgbOIftufhEz34F5ZXo6rGi5//a4k5AMBfj7ZazapdaoLj649cOACT9ABkML5T1qarS19eilcWaUH7oheWYPKTCwPSPXPOVox9bB6OWAoyeX6FidN6t7U9/9eCHZjx1aZGWbMtqkFBROKgBYS3lFIf+7+ulCpSSpXoj78CECcibR32e1kpNVwpNTwjI6PB0ufUcej/xd9yuDig9GSUwu76YA0+Wx04siZS7+jjkLdlF2Pd/sKAEvKxsnbmWjOh4opq3PfxOryxdA8y7/0SJZU15jkaJZjEOBd6tfOVmkqqaszMs3OrJOw96iuZGfxL0kf1jKeksgZlVTXIKdY6OCec7Pw/NPovNvu1VVd7vY7tttYRTOXVtSitqsVNb6x0bB7wb046XFiBt5btxdWvLrc1PVgrCkO7piMxTmuHb5HgsV0H04d1QcukuIC+jO82Zduel1XVmsHLGpjz/YJqSWVNwLV34dBOtuft0xIcR0D5S4n34I9nn4x7pvYFALzuN4ktu7jSNrFt/OO+zuhgS79YS+IAMLBzSwDaLOnrLfM3urZOxj1T+2JEZmuzFpKW6EGqXhD4w5Q+GNdLy8jH9myDti3isTO3BBX6d+rdm0bjo1vGmO9XUV2Lj37aj5vfXIkdOaW2YcGT+7fH2ad0QLzbZQ5jjtRov5K4EVzapfmCwi/HdDMfTxvYEZP6tQ94n+Hd0uFfLlyxW2suM2o26/Tv9XnPLsZd768x18zan1+GWq/CD9tzcclLP9reY+opHQOO9fLCnY2yCGM0Rx8JBvYHAAAZ2UlEQVQJgFcBbFJKzQyyTwd9P4jISD09kU3xbAApDiVW/w6mrD35uP61FbZtyfFu83FxHSeu1HoVPl9zEBXVtdijZ6zvLN+H855bjGnPLq7Te4Ub5mef5OMriX2QtR/vLN9rNr3sO1pmdnwapf2/XjgQHVv6MqCXvt9pjm7p3ykNy3cdxZGiCpRV1SC3pBK7c0vNYaOGdZYgl1NciW16+/SkfvYSlyHY2jElFTWO7bbhMoKpp3QwHxtLPBh+2O7rvPvb176qubWmkBjn+3qkJsaZj/958WA8MX0Q0pI8jumyKq/yNdVYh4f6p72kogbfWcalJ8W5A5oyRATf3z0h5PEALYC5XRIwT8Gwdn8hsosr0TIpzvF1Jwf9hvJeMryL437DuqWbj42gcHqfDHOm9/RhXZGerM036NWuBbq2TsZX6w7jile0luXkeDeGdWuNv/1iIACtYLPLEpC6tfbNX3jsFwMR53ahfcsE27UzpkcbPHfFUCTF+b6n/qYP62o+7pKehDE9tKCQker7zE6yHCsxzm0O6TW4XYIZejpDydqdjyNFFVh3oBAf/bTfbMY7VFiBJ+dsxZWv2FvVO6Ql4qQ2vmPfMqGn+fi6sd3DHu9YRbOmMA7A1QAmWoacniMiN4vIzfo+0wGsF5E1AJ4BcJlqiIb6CCU4TEZyGnWweLu999/anh7qwnPy9NxtuOOdVfhizcGQfQler3LsN5j4jwXmY2smvDs38MYoSyxLKBil/DX7CvCwX5PY0dIqX1DQS9SpiR7blwIAvl5/GElxbnRulYSiihqMmjEX/R+YjeGPfocJ/1gQ0E9gbfvOKa40mzdGdW9jbv/4N2MDzvGMPhm2zzW7uNJxLkG4BcjSU4JneksiWF4i0ZIGa62yf6c0iAjiPb7Xf316D8f3KK+uNWtQ1vRam7NaJHhwuKgCX607hAGd0wAAHVslolOrJCy6+0zb+yV43Jj929Nt2+6a3Ae3ndnLfJ6sZ15Du6YjlHMGdrA9n3nJYNvzXu1a4LzBnXBKp7SA5ThaJcdj1u3jzedGpmr8BnzDhzPbpODZy4dieLd0tEtNMINCn/apOKl1sq2D3fi/t22hldjfXLoH/1rgm91s1FAA32S2Uzr6tq1+YDLeuGEkpg3qhE2PTDW39++YhguGdMJEvQmoRYIHb94wCnPvOgOL7j4TetkUaZb/c5olaCZ4XEiJtxciB3VpaSs43T31ZDiZOWcrRjk09x4qqMDczdkB288brNUS3vrVKPztFwPNzvSz+rW3BYtoieboo8VKKVFKDbIMOf1KKfWiUupFfZ/nlFKnKKUGK6VGK6Uade1Zozp/x6Te5jan4ZP+rO3IRoDwL7UrpZBvab5ZsiMXb/y4G++t0JqMarwqYDSL1ePfbMbIGXNt71Fd6zUzVsBeE5jwjwUBM5Dnbso2m0OMmsJv37O3GQNahm28bmRWqYlxuPdnffHpreNs+6YkuNGzXeSdcdZj7DtahniPCx3SEvHIhQPw0S1jcepJ6Zjxc19p68yTM/D69SPN6rzVIxcOsM0435tXBo9LMKhLy4B9b5/oyyS7OXyRjLWHHr/IXtKzDv+0BoUJ+rDUPu1boF9HLePectjXRTb0JOcM+IftebZaQSdLJnLBkE5Y88AUXD8uE4BWsr5zkjYqbojef9C1dTKeu2KoLQM+uUMqLh2ulXTfvnEUbpvYyxbAE/RgdVKbZHzz29Mc0xXnFnRqmWQ+f+OGkTitt71Zb2Dnlnj28qF49MIB8K+UpiS40cFyLn+fPgh3TOyF4Zm+Zhmj0NMqOQ5n9W+PD28ZC5dLkJ5iBIUWAR37SXot/OT2qQBgCwgAcErnlnjp6mG4Y2IvMyMfaPn/t0yKc5z1/tEtY/H0ZUPNoJKeHIfxvduiZ0YL830A2B4bwQsAXC4xg63holO72PrHrh/X3dbkBGjNZZP6tsPITHtzFQD89atN5uAAQBv99MezT8ZdU7TgMq5XW1w+8iSzQOJUiI2GmJzRbDA6bXq1a4EdM84BEL5DttarkFNSiWv0f/6cjUeQee+X6PGnr8yFtADg12+sxNBH5mDToSKsP1CIK/69DH/5bIP5/iUVNY5BwQgyH/2kvdfQR+Zgvl6a8K/FGM+Nv7EOJ6yq8eJAQTlG6Bfj91u1Dvo0p36U4kpUVNf6NZd4kJoYhyFd7Z2bKQkeXDaiK7793en+b2Pzyi+H256vP1iIT1cdQJf0JLhcgqtHdzObGuz9HdpjaxOdYURmOqYP8zVbHC6qQPe2KbbSo8H4YgG+DAYAdsw4Bx6X4GBhBdqlJuCS4V3N0rk/a1Do2joZK+4/C+/e5Gvvvv9c37Qbd5DlUUoqa/Cppd/pxtN7IDXBgwGd03DD+O5omRxnlkg7tkrEWf3aYcbPB+KRCweYfzNtUCcM8DvHhy44Be/cOBpje7aFiJht4f5LcfTKaIHxvdqabfpGk1HX9GSU6J/7H88+Gaf1zkDbFvG2vzWaS4aelG67NgAgOd6D1pZMs2vrZPx+ysm2z8EY6tsq2f6+PTNSkJrgQe/2qRjv16GarJfGu7ZOQpsUXzOT4bxBHXH2KR3we8v/13qNil+QMQK4EWxun9gL/7t+JEb1CCx0GHpkpGBkZuuATmejptAqOQ5f3DYeV446yXa8BI8LD0zrj3X/N8XcNrZXW7x67Qi8f/MYbHz47KDHBIA/n9sft57Zy3bdAUB7vR/JqfATDTEdFIwMOj05Dm6XmJE4xSFDMuSWVKLWq9CtjTZawchsAeDuj3zj3411Sy5+8UdbX4FxzMLy6pCToazD7l7VJ/H4D7vcn1+Gz1YfMDtorYxOWKME+cTsLfB6lVktB4DXrhuBBI8L+/LLsD+/HGf08ZUU0xKdm14SPW7EuV3o0z4V/7l2uK0N2eqs/r5OObdL8Pz8HcgurnRsbjulk+9iNz4T4wto/Y53b5sSMEN4e06J2d5vNCf4s7adu11ilnAz26RARPD5rePNTlnbufplhBmpCbZjnNEnA5sfmYqHLzjFbJYIp2+HNKx6YDJm3X4aBum1AaO0mZoYBxHBFaNOcuzvsqfNbatNGenq39Ee4DxuF9781ShMG6x1WhvXdve2KWazlvH5WDO4EZnp+N1ZfcznD51vHy3eIsEDlx4A2gT53I0ZyK38+i7OH9wJy+6fhLTEOPx8aGdseMiXWRqFARHByO6t0allIr664zR8eus4PHXpELRz6Ggfnhm8mez9X4/GD/dOtH0ep/cJPVhl3l0T8P7NY2ydzoAvSMa7XRjYpWVAABIReNwuW/+TdWBAsqX56feT+2BEZrqt3yvY/3xsz7Z4/9djcONpzk2UDS2mg4IxYa1VknZRG00TZwQZHeP1Kry7fB8ArWTkXzg0MnxrU5J/xm+UhA85DJEEfEHBWmMxOqSP+P3NPR+tw53vrsZ7K/aZ2w4XVuCp77aatQjrwmLl1bWo1t833uPChJPboV1aAv734x4cLa2yZdjWNnTrl367JQBN7Nse/7zY3g5999ST8davRgHQRin5/71T1X5Mzzb4RO9bMD4fo2TX21JKTPC4zc/cSGtaYpxZU5oQ8GXXdm6VbM+UjA5co1nJ5RLHjC3RE76/KDHOjV+OyQxaU/DXJT0p4DMwPmunWlyk+nVMw7VjM/HcFUMdX0/XPwOjeXDKKe1x9ehMtEmJN+diWP3rymFoYylAXDriJGy2tNEbmfeSeydi7l1nOB7TV1Owf/4iYmaQImLLDK1NJH/7xUB8/JtxiPe4MKRrK1w41Hnua4LHjb4dUh2bCVMT48zrsK7apNiDgkdfCSEhLvJsMyPIENfbJ/bCBzePxYv63JNwRnZvbQbhaIvpoGBkvEZJyZhQdHrvwKBQXevFG0v34MnvtIXyOqQlBlTzjP1CjUgyRmEEmxrvv6QBAOw7qu3rX1MwrNqrdfB6XII73lmFp77bZs6CtVa9d+SUoKCsCh1bJmLBHyYAADIsX/w9ltExqZYM6tvfnY4v79DatP2b1/xLN6f3zsC4XlqTwOzfnY7lf5pkK3E9dekQx3Mw0mk0BRilMv+x+Ubp7I9nn4yPbhmLj38z1uxE9w/mRkHOv/nCaFJItwSCi4Z1wX+vHWFrMnD6/0bCf7jj8j9NMpuwrB2TBmMNnbQ6jAbyF+d24f/OPwVdHFYXBXzt4xcM6YQ/nn0yLjq1C/p3SsPKv0y2lb6Nzmf/WpK2zfd5GP/3Tq2SAj5fgzEpMNjrVsb/3Vr6bpUcb+u3CGXW7eMx764JEe0bKf+mOOM6/sMUe4fyr8Z3DzpU2L9m+8HNYzDj5wNt59kmJb5Bb0Z0rOpfNDkBDOuWji/XHQoYmudUHS0oq8anq30zYdu3TEBinDtg0k1ppXNfgb9gN3QpLK92nBA36Z8LbMsHWG3UO6tqlcLq/VqAyNLHSvftmIq7JvfBP+dsxfnP/QAAuHBIJ3TSS0/WksyRwgrceFp3/HvRLtvF3KZFAtq0SMB14zLRw2+YpP9EtDaWdukWCR60SPCgX4c0rD9QhOvHdUf3toHLLwBaie7rO08zS3vGHJ0u6cl4+rIhZkZufJcUfMMf757aF33ap2LaoE64893VZql9XM+2eHvZXoz2az82+iCsAcDtEpzZtx2W338WbnhtBeZuzq7TvQCsjKavkZmtsXz3UbRLS8Q7N43GrtxSx5qS8Rn6N/00pEtGdMWyXUfxp3P6mW3UTp68dAhuO7PU1gTiJFzzFhC8puDk7RtHRbSQXTDRWFLdX3K8B7sfOzdg+5+n9cefpzkt6xZoRGZrs5/P8MO9EwPmOjSlmA4KT1w8CLdM6ImWfhdt97aBo2senrXRLJEDQNuUBCQ6RPfiihrH0r4/pxm2AHDPR2tx8bDAMeDBAgLgK70r5RvxMU/vnG6bkoDe7e3nYy25WYPC81eeiqEnpds6UK0ePC9wFRL/EmW6Q6kwUw8E4e6x0M+SKRrDazu0TLDNzjWWnLYGzozUBNyoDwn9+0WDMFgvdZ47qCPG9Jwc0NdwzsAOePWa4bY+FCujRJjrMIs6Eo9PH4Qv1x7CL8d0M0uErVPig/Z5TOrXDi9fPcxxclRDaZkUh1euGR52vwSPG/07BQ9O6clxyC+rjmgotvEvimQ+RHK8x9bmfrzokZFSrxsPrbj/rLDXu6G+NdJoOf7+C40oOd4TMKoDcB5J8sWagxjcpaW59o7LJY7/zFLLZCXDS1cPQ3FFDf7wwRpzW7A5CkdLq/DSQue1beojLSnwy2YtubXTJ+tcOrxr0GGVoVirwTMvGez4mUzs2w5PzN4SdCazk9YOI0+042m/g5WsLhnR1fbcmhEbzToiEjIDvvG0Hli0LReT65hJnzOwA3q3S0XbFgm4ZmxmxH8nIphySofwOx4HPr11HFbuyY+oD+XFq4bhw5X76nxLzuPJ3N+fEdChHIlgfQnNQUwHBX8f3TLWLPm+eNWpuPlN++JZ90zti/YtE83lIxyDQmVgUPDvxGzbIh65JVrHrjFpzO2SsMNhrcb1aoMftuchIzUBOcWV6JCWaPY5vHT1MGTtPoqM1AS9I8+eTmsHYnqQ0mt9/OJU51mu/TqmYcujU83x85H4w9knY1yvthjWzV7VPq13W7y6eBdODTLqKZjtf/0ZXBF+uXtktMDieyaG39HPv66MrNOwOevWJsUceRfO1AEdMHVA8wh2wdQnIDR3DAoW1uGVUwcErj0yPLM14j0u9NQLvE6dccUVNSgotzc7tEj02NbPN1bSfOqyIXhyzlZsPlyM7X/9Gfr8+WtU1yqcO6gjfjW+O37+r8C5fIO6tMSVo07CJcO7asP29LuqdWuTjMNFFTirn7YezNmWkqd/TaGXpV/AGHEV6b2C66suAQHQAu6ZDsM8J5zcDhsfPrvOTQ2N0eZMdCJgUIjAIxcOQG2tN2CEgFFTuH5cd3RJT8LDszaitLI2oKaQEu9BSZyv8/n+c/ph4bYcTO7XHiMyW2NHTglEBK9eMwKvLN6Fpy8dErCOkKF1SjwuHXGS+dxYgfKuKSfjq3WHbLOzrce3svYxGENW/fsdjmfHY9sz0YmC364IXDK8i2NJ1wgKFw3rjJZJcXpQ0JqP4j0us98gNdGDogrf318/vrvZMap1QGpNJKf3yTAn1gSb0u7fBNK5VRIOFJRjeLd0jOweOJUeQMD0fOsEtvG92+Ldm0YHjIggotjEoBCC0bEc7GbxRvNRRmqCuY8xq9lo6we04XvW0RqRdNJF2tzyya1jUVpZG3Jii7WmcP85/QJe9x+yWVfv3DgalTX1v7cvER0/GBRC+N8No3AgvzxoZ1NinDbDtk1KQkCbfKukODMoxLlddW7yiIvwBvXtUhOB1ND7WPs+bgyymuexcFq8joiaJwaFEFomxYUcY901PRl92qfC7RK4Yc/E/f8uKcR6Sk6sgcjaFFUfsTiCgojqh0MyjsEdk3rblpa+ZUJPyzIN9nhrNB+l1mN9G//bANZXsNVAiYgMrCkcA7dL4Hb5agD3TO2LN5fuwep9BfAqhZeuHoZD+hpH8R4XHrlwAMb3CrjbaFj/vGQw4j2uY7r1509/mVznGwIRUexhUGhgxpr01bVe21wBALh6dDenPwkrOd6Du6f2xWerDwbczD1SwZZYICKyYlBoYC31ZbiNlS8bSudWSY6LcRERNSQGhQZmrDPfEItczbxkcJ1urk5EdKwYFBrYwM4tcfvEXrhi1Enhdw4j2FpCRETRwqDQwFwusd0fmIioOeGQVCIiMjEoEBGRiUGBiIhMDApERGRiUCAiIhODAhERmRgUiIjIxKBAREQmUVG+YXtDE5EcAHvq+edtAeQ2YHKaA55zbOA5x4ZjOeduSqmMcDs1u6BwLEQkSyk1vKnT0Zh4zrGB5xwbGuOc2XxEREQmBgUiIjLFWlB4uakT0AR4zrGB5xwbon7OMdWnQEREocVaTYGIiEKIiaAgIlNFZIuIbBeRe5s6PQ1FRP4jItkist6yrbWIzBGRbfrvdH27iMgz+mewVkRObbqU15+IdBWR+SKyUUQ2iMid+vYT9rxFJFFElovIGv2cH9K3dxeRZfq5vSci8fr2BP35dv31zKZM/7EQEbeIrBKRWfrzE/qcRWS3iKwTkdUikqVva9Rr+4QPCiLiBvA8gJ8B6A/gchHp37SpajCvAZjqt+1eAHOVUr0BzNWfA9r599Z/bgLwQiOlsaHVALhLKdUfwGgAt+r/zxP5vCsBTFRKDQYwBMBUERkN4HEATyqlegHIB3CDvv8NAPL17U/q+zVXdwLYZHkeC+d8plJqiGXoaeNe20qpE/oHwBgAsy3P7wNwX1OnqwHPLxPAesvzLQA66o87AtiiP34JwOVO+zXnHwCfAZgcK+cNIBnATwBGQZvE5NG3m9c5gNkAxuiPPfp+0tRpr8e5doGWCU4EMAuAxMA57wbQ1m9bo17bJ3xNAUBnAPssz/fr205U7ZVSh/THhwG01x+fcJ+D3kQwFMAynODnrTejrAaQDWAOgB0ACpRSNfou1vMyz1l/vRBAm8ZNcYN4CsDdALz68zY48c9ZAfhWRFaKyE36tka9tnmP5hOYUkqJyAk5vExEWgD4CMBvlVJFImK+diKet1KqFsAQEWkF4BMAfZs4SVElItMAZCulVorIhKZOTyMar5Q6ICLtAMwRkc3WFxvj2o6FmsIBAF0tz7vo205UR0SkIwDov7P17SfM5yAicdACwltKqY/1zSf8eQOAUqoAwHxoTSetRMQo2FnPyzxn/fWWAPIaOanHahyA80VkN4B3oTUhPY0T+5yhlDqg/86GFvxHopGv7VgICisA9NZHLcQDuAzA502cpmj6HMA1+uNroLW5G9t/qY9YGA2g0FIlbTZEqxK8CmCTUmqm5aUT9rxFJEOvIUBEkqD1oWyCFhym67v5n7PxWUwHME/pjc7NhVLqPqVUF6VUJrTv7Dyl1JU4gc9ZRFJEJNV4DGAKgPVo7Gu7qTtWGqnz5hwAW6G1w97f1OlpwPN6B8AhANXQ2hNvgNaOOhfANgDfAWit7yvQRmHtALAOwPCmTn89z3k8tHbXtQBW6z/nnMjnDWAQgFX6Oa8H8IC+vQeA5QC2A/gAQIK+PVF/vl1/vUdTn8Mxnv8EALNO9HPWz22N/rPByKsa+9rmjGYiIjLFQvMRERFFiEGBiIhMDApERGRiUCAiIhODAhERmRgUiHQiUquvTmn8NNiKuiKSKZbVbImOV1zmgsinXCk1pKkTQdSUWFMgCkNf4/7v+jr3y0Wkl749U0Tm6WvZzxWRk/Tt7UXkE/3+B2tEZKz+Vm4R+bd+T4Rv9dnJEJE7RLs/xFoRebeJTpMIAIMCkVWSX/PRpZbXCpVSAwE8B231TgB4FsDrSqlBAN4C8Iy+/RkA3yvt/genQpudCmjr3j+vlDoFQAGAi/Tt9wIYqr/PzdE6OaJIcEYzkU5ESpRSLRy274Z2k5ud+mJ8h5VSbUQkF9r69dX69kNKqbYikgOgi1Kq0vIemQDmKO1GKRCRewDEKaUeFZFvAJQA+BTAp0qpkiifKlFQrCkQRUYFeVwXlZbHtfD16Z0LbQ2bUwGssKwCStToGBSIInOp5feP+uMl0FbwBIArASzSH88FcAtg3hynZbA3FREXgK5KqfkA7oG25HNAbYWosbBEQuSTpN/dzPCNUsoYlpouImuhlfYv17fdDuC/IvJHADkArtO33wngZRG5AVqN4BZoq9k6cQN4Uw8cAuAZpd0zgahJsE+BKAy9T2G4Uiq3qdNCFG1sPiIiIhNrCkREZGJNgYiITAwKRERkYlAgIiITgwIREZkYFIiIyMSgQEREpv8HoemNddoHHikAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(1, len(average_mae_history) + 1), average_mae_history)\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Validation MAE')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 그래프는 범위가 크고 변동이 심하기 때문에 보기가 좀 어렵습니다. 다음처럼 해 보죠:\n",
    "\n",
    "- 곡선의 다른 부분과 스케일이 많이 다른 첫 10개 데이터 포인트를 제외시킵니다.\n",
    "- 부드러운 곡선을 얻기 위해 각 포인트를 이전 포인트의 **지수 이동 평균**으로 대체합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzsnXecXGW9/9/faTsz21t6NpUWSgohAgERBASxgKJiQUGviHBVrvyUi1evXtuFexXsBUEU5aoozUIVgoBATMBASAKkN5Js79Pn+f1xypyZnZ2d2Z2Z3c0+79drXpk5ZeaZ3ez5nG8XpRQajUaj0YwW13gvQKPRaDSTGy0kGo1GoxkTWkg0Go1GMya0kGg0Go1mTGgh0Wg0Gs2Y0EKi0Wg0mjGhhUSj0Wg0Y0ILiUaj0WjGhBYSjUaj0YwJz3gvoBw0NTWp+fPnj/cyNBqNZlLx/PPPtyulmkc6rmRCIiJzgTuA6YACblFKfTfjmDcB9wM7zU33KKW+mutcEfkK8HGgzTznC0qpB3KtZf78+axfv74YX0uj0WimDCKyO5/jSmmRxIFrlVIviEg18LyIPKqU2pxx3FNKqbcVeO7NSqlvlXDtGo1Go8mTksVIlFIHlFIvmM/7gC3A7FKfq9FoNJryUpZgu4jMB5YDa7PsPkVEXhSRB0Xk2DzP/VcReUlEfi4i9cVfsUaj0WjypeRCIiJVwN3ANUqp3ozdLwDzlFJLge8D9+Vx7o+BRcAy4ADw7WE+9woRWS8i69va2rIdotFoNJoiUFIhEREvhhDcqZS6J3O/UqpXKdVvPn8A8IpIU65zlVKHlFIJpVQS+BmwKttnK6VuUUqtVEqtbG4eMelAo9FoNKOkZEIiIgLcBmxRSt00zDEzzOMQkVXmejpynSsiMx0vLwJeLsX6NRqNRpMfpczaWg1cCmwUkQ3mti8ALQBKqZ8AFwOfFJE4EAIuUUopETkt27mm1fI/IrIMIy14F/CJEn4HjUaj0YyATIVRuytXrlS6jkQzHuztHGRbaz9nHj1tvJei0RSMiDyvlFo50nG6RYpGU0LO+vYTXP6LdeO9DE2JuHPtbl7vDo33MsYdLSQaTQmJJQyLPxJPjPNKNMWmayDKf9z7Mm/8nzXjvZRxRwuJRlMGekKx8V7CsOzvDvH79XvHexmTjq7BKADxpJrQv99yMCWaNmo044Hz4tIbijGt2s+aV1qpC3pZ3jIx6mif29HBJbc8B8B5x82g2u8d5xVNHrodv9+ugSi1gan7s9MWiUZTIvZ3pXznlqhc/ot1XPSjZ8ZrSUOwRASgtS8yjiuZfPQMpoSkJxTj+49tZe2OjnFc0fihhUSjKRGD0bj9fDK4Ptq0kBREdyhqP+8Jxfj2o6/xPocwTyW0kGg0JWIwmgqwTwYh0RZJYXQ7LJL9UzxzSwuJRlMiQjGHkAzGSCYnXs1W0OdmWnUFoC2SQnEKya72gXFcyfijhUSjKRGhNIskzoDD1TURiCWSDEYTfOANLXjdQmtfeLyXNKnoCcXwuY1L6M4sQpJMqgl581AKtJBoNCXCaZF0h6IMRCZWLYnlbqsLeJlW7ae1d/wskh1t/SQm2UW3JxRjem0FPreLXR1DheRTv/knC7+Qc3jrYYMWEo2mRFgxkoZKH219EZ6bYBk9tpAEfcypD7Cnc3Bc1rG7Y4Czvv03vvXIq+Py+aOlezBKfdBHTcDL9raUkFjFp3/ZeGC8llZ2tJBoNCUibFok8xuDvHqwj2t+t2GEM8qLJSS1AS8LmiqzumfKQXu/YQk9u31iCe1IdIdi1Aa8uIQ0a6prID2xYrJZWqNBC4lGUyIGo3HcLmF2fXDcLtK5sISkxhSSzoHouGSXmZMkmGyX255BQ0iuO+/otO2dA9G014MTLDZWCrSQaDQlIhRNEvS6mVFTQTzjrnQi3KVaBXW1AS/zmyqB8ck+EvPfydaJvDsUoy7o5d0nzuEnHzqRhc3GzzAUixONJ+3jnGnghytaSDSaEhGKxfH73Eyv8Q/ZF46N/8XFcik1VfmYVRsA4FBv+TO3rMaWyUkkJMmkonswSl3ABxjtZW589wkAPPTyQVZ87VH72P6Itkg0Gs0oCUUTBCe0kETxuoXagJf6SqNPlNWIsJxYwelkcoQDh+Hpre389wNbiriikemPxkkqqAum+msFvG4A7nh2d5p4DE6wbL1SoIVEoykRg9EEAa+bhkrfkH3h+CivmkWkoz9CY2UFImKvsWuw/DESyw00WovkQ7et5adP7iCWGPln+qtnd/HKwd5RfY4Tp1vQwm8KSSSe5C3HTue2jxjzoLRFotFoRk0oliDgc6fdtf7b2UcCpbdIesMjC0J7f4SmakNAAl43FR4XXQPjYZEYAjAaHdn8ekoURqrMjyeSfOn+TZz3nacK/6AMrKr2umDqJiHgc9vPT5xXT1OV0THgB2u28sM124CUO/FwQwuJRlMiQqZF4rzYNFQZz0spJGteaeWErzzCC3u6SCaVHcQORRMs/+oj3PPCPsBwbVkXOxGhPugbknFUDmzX1iiU5K3fS4nCwRHiO91FzEizXIDOm4SgNyUkTVUVVFYYUzr+vq2D/334VV7a183Kr/+V+/65n7vW7eXXz+0u2nrGm5LNIxGRucAdwHSMzL5blFLfzTjmTcD9wE5z0z1Kqa+a+84Dvgu4gVuVUjeY2xcAvwUageeBS5VS5f/fr9GMQH8kzpxgkDqH+6OxsvRCsm5XJ2AEfS+/fR3JpOJ9J83l1qeNP7PP3vUi71oxh/b+CEfNqLbPq6/0jU+MJGZaJFn2vXaoj/qgj2azH1guDvXkFhKnSCql7LTj0bC9rR+AOfUBe5vTIqn2e6mscKedY40V+NlTO9hkWlIfOnneqNcwkSilRRIHrlVKLQFOBq4WkSVZjntKKbXMfFgi4gZ+CJwPLAHe7zj3RuBmpdRioAv4WAm/g0Yzal7vDjGrzk/QcYGpD1pCUroYiRUruOXJHfSEYvRF4raIWPSGY7T3R9Iu0A2VXjtG0hOK8fzurpKt0Uk0kT1GopTi3Juf5J0/eDqv9xnJIunoTwnJoTG2g9mwt5vpNRXMrE0JSYUndTmt9nsI+tLv060anU0Od1w8j7jOZKBkQqKUOqCUesF83gdsAWbnefoqYJtSaodpbfwWeKcYtxBnAX8wj/slcGFxV67RjJ1DvWF6w3Fm1wXS7nyrTHdHKQv/8mlp/o8dncQSiiUza+xt9UEfB3vCJJOKD/zsOd7942fKcqGzLZIMk6TDtCBeH8bSUErhErj6zEX4PK4RhcRpbY107Ehs2NvNsrl1aducv+cav5dKX7pFYlkxTg6X9vNliZGIyHxgObA2y+5TRORFEXlQRI41t80GnEOk95nbGoFupVQ8Y7tGM+70hmNsa+0jEk/whm8+BsCc+mDaMUdMr8LvdfGPnZ0lW8e+rhCnH9GE2zW86+bpbe0ALJ2TuhietriJ/d0hfrNuj33XXI5Gk5kxkq2H+hiMxnntUF/O8waiCZLKuGg3BH0jJgp0OPZ3DozeIukaiLK7Y5ClGULipNrvweNOv7w6+3FZTLT+a6Ol5EIiIlXA3cA1SqnMvLsXgHlKqaXA94H7ivi5V4jIehFZ39bWVqy31WiG5dYnd3DRD59JszZmO3zoYKSIvmFBo30hLzZKKfZ2DjKnPojfM/yf99qdnVT7PcxtSK3vfSfNpbm6ghf3dtvb+iKlTwe2srbiCUUknuCcm5/kk79+gW2txh28z+Ni0+s9XPmr5+lzZKNZz6v9XmoD3hGtPKfQON1chbJhn/HzybRInNT4jbjY49eewUdXLwBgW2s/FR4XztDM1/+yxRbSyUxJhUREvBgicqdS6p7M/UqpXqVUv/n8AcArIk3AfmCu49A55rYOoE5EPBnbh6CUukUptVIptbK5ublo30mjGY7Wvgh9kXjarPaWBsMi8Tku6kfNqGZfV2k67R7sDdM1GOPoGdVU+Yfm0nzrPUsB2Nc1SEOlL80dIyK0NAR5eX/qfq8cNRBWHUkknuBAt+Fy+ttrbfzn/ZsA8Lld3PHMbh7adJDvP77NPq8vbKytJuChNuhNGzSVjc6BKJaRNpbstJf29iACJ8wZXkisn/3C5iouXz0fgD2dg8yo9VNtujdr/B76wnG2t068PmyFUjIhMeMZtwFblFI3DXPMDPM4RGSVuZ4OYB1whIgsEBEfcAnwR2XkMa4BLjbf4iMYWV8azbjTZ150d5gujE+ftdgu9Fv3hbNZ/8WzAeMCEo4lS3InaonAcbNruO0jJ6Xtu/rMRaxoMS5+feE4lb6hQjO3PsDmAykhsS7WpcSySMKxJK9niRn0R+LsNOd9OBMACrVIOgaizG0I4vO4RiUk4ViCjft62NUxwKzagB3vyobTrTiz1m/fSNQFvPZ5qxc3ARSlQHK8KaVFshq4FDhLRDaYj7eKyJUicqV5zMXAyyLyIvA94BJlEAf+FXgYI0h/l1Jqk3nOdcBnRWQbRszkthJ+BwAe2HiAPzy/r9Qfo5nkWBddK6i6rCV1x1ob9No1G1Y1tDN7p1hser0Hl8AxM2s4bnYt9151KmBYIp97y9HUOFKRM9NTAeY2pMd0+ssiJIaghmIJ9jmE5OSFDXzpbUayphVTcgpAb8hYW7XfQ23AS28erq2GSh+NlT46BqJcf89L/PRv2/Ne57cfeZW3/+Bpntrazuy6wMgnmHjcLo4206xrgz68pqgsmVmDz+3ilYO5Y0GTgZLVkSilnibV2HO4Y34A/GCYfQ8AQ8aLKaV2YGR1lY2r7nwBgItPnFPOj9VMMqw7ZMsiCXiz/3lZF/N3/egZHvzM6RzjyJwaK619ERoqfXbq6fKWep7597OYWWv0+6p2uLsqs9xRz8mI6fSVwbVlZW0lkoqX9/fY2z/z5iPpdmRa+TyutMpwq3q/xhSSkQoOOwaizK7zE40n6RyI8vgrrYDRcHFeY+WI69xqxmza+yOcfkRTnt/OoKUhyEv7ephRU2G7NSsrPMxrDLI7y3TFyYaubNdoioRlkexoNy44Qd/QO35IBWLBuPAXew3VjvcHmOVIQa7wuPF7jT/7bELi7B0FhkXy8v4e3vOTZ0pWRBlxpBj/+aUDBH1u/vtdx3PywgZaGlMW0hlHNtMXTrVot37e1X4vdQEvg9FEzn5bXQPGRMOGSl9al+Mz/vcJTv7mY7T2hXN2P26sTNXcDGeR/ObjJ3PLpScO2f7uFXNYNb+Bz55zFB7T7RXwuZlR6+fgCIWUkwEtJDm46ZFXefv38yuG0mgyLZJhhcRxsfbkSNEd7RqqswTZ0z7fFJqqLDGSTHHpC8f4z/tfZt2uLl7a1zPk+GIQiSWZZVpMnQNRzjt2Bu9f1YKIpNW5rF7UaB8D6RMea81WJcPFSZRSdA5EaagyRuPuyEjFPdgbZtU3HuMN33ws66yYZ7a1c/cLKff2jNqhHZ0BTlnUyLnHzhiy/cyjp3HXlacwo9aP22VcdgNeNzNr/WOuaZkIaCHJgcslvPx6af54NIcfVjzBGmLl92YXktpA6mJd7KFHhkWSW0gsqyObRZK5rT8Sx2Ne+OKj7fM+ApF4gmk1fmaY7fbPOy51IRYR/vNtS3j/qhZmmlaA5d7qGogS9Lnxe932dxouc2sgmiCaSNIQ9FEb8BIyratTFjby4pfPta00yN5K/wO3GiVwx86q4YNvaOHNx0wb9fe1bh78XjczagO09kXy6lw8kSlZjORwYPG0qlF1JNVMPRJJxUCGKOTj2ir2GNa+cIzmqqqcx8yo9bO1tZ+qLMH2zEykvnAcj9u48EVK1NblYE+YloYgP/vwSrYe6uMU0/Kw+OhpRh3G+l3pAffOwajdcsb6d7hsrE6zbqSh0pfWKv/fzz+a2oCXVQsaefI1o96svT9iJ0ZkEvS5+cZFx4/qe1q4Ha6tmbV+lDI6F88qIIA/0dAWSQ4WNaf/QU62UaCa8mFZI84728xeSxZO11axK8d7QyNbJFY34pEskhq/h95wzL7wlaKtSziWYEf7AMfMrKG5uoJTFzcN20xxWrVhsViuoO7BmN19d1qNceFv7Ut3EyVN67DNtGIaq3zUOCxCq9fY9y9ZzluOnQ7kLlY8UIR4hmWRVHhctovsQM/kbpWihSQHC5rSMzkiE2AYkWZiYmUQLZ6WuvmoGKay3OnyKoVFkhlsz6TGFBpflvU54yaz6gJ0DUTtC18+M04KZeuhfhJJlVfm2qw6P163sNOcK99ppvNCSmRaHc0YH3/lEAu/8ADbWvvtWfTzGivTEgoazbb+tUEvn3vL0eZ5rTy86aB9jNPt9NlzjhzV93RiCXNSKRabN6uf/s2GEVvCTGS0kOTA73WntTMIFdmfrTl8sHzzx8xIXRBdOQLp95j1HQ9sPMD7fvps1gBvvrzeHWLjvh6e2d7OQDSRdsedDcsiylZs6KwtmVbjp3MgageHe0owPfFV8+J59MzqEY406jFaGoLsMOt0ugejtnVVH/TidUtaFtyjmw8B8NTWNna09+NxGZX7Ttdihcc5Q8R4r9ue3sknfvW87YGwYjLfvOh43rVi7CUAlqswkVR2uvX+7hBfuu/lMb/3eKFjJCPgHPYTiiWoH+f1aCYmlkvlTUdN4/d5FK+uaKnH73Xxwh6jb1NfOJY2AKsQzvvOk/Q6RGEki2Sa6c7JJl7ORoONlT52tPVTa64rm2trrHM99nQO4hKYm9HccjgWNFXZFknXYIx607UlIjRXVaS5tmoDxrpb+yK8drCPlsYgXrfLFtLMppaZqc/7u0PsbB+wW/5Prxl5Jko+fP3C47nxwVc4aX5D2s+umDGSgUicoM89pt9NIWiLZAScQ4lCJR6Pqpm8WPMtlrcM338pE2eLksxAfSH0ZlgWI8VI3r+qhU+csZB/OX1BzuMaKo2bqEGzKDFTSPZ2DrLg+ge47emdfPo3/xyV62tPxwAzawNZ3WzZWDStkl3tgwxE4vSE0sW3ucafNm7XWs+Pn9jOY6+02m4kSzDqMoQj86J72o1ruPS2f/DxO9bjc7uKVji6oKmSn1x6ou3ivP/q1cBQYRstezsHOfbLD/Obf+wd+eAioYVkBGodozS1a0szHId6w4iQ1yQ/i6DDjTTaViTZrIrmYTKOLPxeN9eff8yIlktDpY/BaMJuv54pJFYrmK/9eTN/fPF17v9n1v6pOdnTOWg3tsyHlfMaiCaS3PToawBpdSYzairSguGZM9ytdis+0+py/m1bPHzNG1nz/940ZPvHTl9QsqyqpXPrOGJaFQNF6CLQORDl9P9ZA8Az20vTYTobWkhG4LhZtfbzUo5H1UxuWvsiNFb68LpdPP/Fs3n6ujNHPMdpkYy2y67Vafh9K1PNsk9a0DCq98rEGgu8t9No6fHEq21pLUwys752thfe0XhPZ6ggIVk1vwERI47RXF3B2Y56jgVNVezuGLCHcTnbqRw5vcruI7awuZIT5tRy47tPGPL+R82oZkFTJfdedSqfPmuxvX1xc+6U6rFSWeEpSqflRxxJAiPdKBQTLSQj8B8XHMNHTjHmKmvX1vgQSyT5+dM77dYYE5HW3rCdOdRYVTFkoFU2nHUmo72I7DL7NL1rxWxcYtSB5OpKWwhWRlQ8qXjLsdOJJpI8tTV1lxtPpFtD63Z1sqcjfzHZ1tpPe3+EY/IItFvUBr3MNAsX59QH0mI6i6dVEUso9pjC57RIxNH2z+9188d/PY2T5g8vuMtb6vnsuUfZrzN7kBWbqgpPUSyS3eZ3n10XoK2vfBXzWkhGwO918x7zbk+7tsaHXz+3m6/+eTN3PLtrvJcyLK19EbuWIV+cd/SjvYh0mJP+mqsreP6L5/DM9WeN6n2cPHTN6XYzSSv1t6UhyPSaCnvYFKRmrYORSrxxfw9v/N81ebdof2DjAUTg/ONnFrQ+60478457UbORrm+tsXMgarfNH2tVfmZX5GJTWeEeU01ROJbgHT94mh8/sZ059QEWTasqeh+3XGghyYOAeeeoLZLxwfqDKMfY19HirGnIF+fFabQxkq4BI25RH/RRX+lLS20dLUfPqOGYmTXMbQjyb2bdxKy6AIunVbGtNVXrEHH8PVxx+kL7+ZYD+bXH37i/h8XNVUyvyd63ajis9ObqDMtrkVnDs7N9gGRSMRhNsGyukWf56TcfUdBnWJyzxChSLHSNhTJW19am13vtXmihaIJp1RVsOdCbNlGylOj03zwImNkV2iLJj0tvW8u8xiBfv3BsrSTAqEy2OrJa+fcTka7BKA0Fpu9aMypg9K6t7sEoIunV8sXk6jMXc+Hy2TRV+djdMcgvntnFmd96gredMDMtY+rC5bPoCcX41XO72XKg1x7alIvdHQPMbxq5fXsmKYsk/fJV4/dS4/ewvztE2JxxMq2mgl03XFDwZ1h8//3LzVqa0v7fq6rwMDCG4lTn/59IPEltwEssoTjjf5/gRx9cwckLG3OcPXa0kOSBFRQdS4rmVKG1L8xTW9t5aitFEZJrf/8i95rZQKVo0VEMwrEEg9EE9QVaJM4gcz5C0jkQJeB12xYyGLUUtQFvSS90Vst0K0NqZ/sA3398W1rK7rzGSr524XE8tOkgWw6MXKGdTBqxjDOOLHwMtlWZny3NeXZ9kH1dIbsZ5nD9zvLF73WXpQdW5RhjJJZFe/5xM7h89QKSSrFuVydKwbzG0rrlQAtJXljzl8tlJk5mnnwtFYxNJNWYLnDdg1FbRCA9CwcgnkimBVvHC6tbbKGurWVzUzUn+VxEVnztUVa01HHPVavTPrt+lIWMhXL8nNq011byw/ffvxyv+Xs4YloVO9v7h5ybSWtfhHAsSUseA6UyCfgsIRlqhc2uC7C3c9D2HgSG6cA80aiq8BBLKCLxRFq1fb70R4xr05fetsQWvj/+62lFXWMuxv+vcBLgdglVFR57tKdmePodYjvWRnRdGS05Ovqj7GwfYGf7AM9sa2fxfzzIi3u7x/QZxcAKLhd6Qa8L+th1wwU0VflGtEgsobEq4S2cjQtLzRGOPmKrHCnGpzncWMbEv5Ezt6wpgYWk/lpYdYPZstNm1Fbw6qE+ezT2cI0zJxqVpuU02jig1e6maoRi1FKhhSRPqv0ebZHkQcyRErprFHUFTqy6nQ+fMo95jUE6BiKc+a0nOPNbT/DnjQcAeHHf+AuJFfAu1CKxqKrwZO175WTXMONYy2mReNwubrn0RB6+5o1p6bAV3nQXV8dAdMQqd8u6HKl4MhuWjZstZmbVfX33sa0ABHyT4xJnuUX/+4EtdOWZ9ebEuhGpHCfhLNlPWUTmisgaEdksIptE5DM5jj1JROIicrH5+kwR2eB4hEXkQnPfL0Rkp2PfslJ9Byc1fi+94Rgd/eVLqZuMOFNCN41xKJiVJXfm0dM4/YgmtremLqaWJZLZH2k86LRdW6Nby7Rqf84Rr4DdX8rZph7Ka5EAnHvsDI6aUZ1WJ+NzuBfnm/74kepJrGp5q/tuIeRqH/WelXM5cV6qI17AOzkskuNnGwL4++f38Y0HthR8fn/Y6K1V6qSA4SilXMeBa5VSS4CTgatFZEnmQSLiBm4EHrG2KaXWKKWWKaWWAWcBg879wOes/UqpDSX8DjbVfg9/e62NE7/+Vx56+UA5PnJSYrXcXtBUyT92do7pvSyLJOB184k3LrIzccBId4ShRXGReILjvvwwf3rx9TF9diF0jdK1ZTGj1j/inAvLXWTNDb/hwVf4xl82c7A3zKza8g9EmuUYNeuMU1kpzVZRoMXaHR3c/ved9mtr5sdofmZWcWG28UBul7B0Tir2NNZge7lwjqwYTTv5/ki8aIWoo6FkQqKUOqCUesF83gdsAWZnOfRTwN1A6zBvdTHwoFJqbH6SMVIT8NpdQB96+eAIR09dYokkHpfwhgUNrNvVaQ8WGg2WkPi9buZmtP+2GHTUMry8v4ejvvgQ/ZE4N//1tVF/bqF0DhgpuKO1jmbW+TnYE845OM2q0k6ax/zkb9v52VM7SSRV2gyUcjHcd7XqLZwW1u/X7+V9tzzHf/1ps/0dO/oj1Pg9eTdrdHLBCUYBY+YkRYsZtSl32WQREhHhX8xJkJte7y241KAvEh+3+AiUKUYiIvOB5cDajO2zgYuAH+c4/RLgNxnbviEiL4nIzSJSnN7OI+BMNWzPMUFtqhNLKLxuFyfNb6A3HLfnTYyGUNQQbivzxrp4LXVkO4UcufdrHRbQaIK4o6VrMEptwDvqDLJZtQGiiaTt7hnuMyB74eJ4CMlwdSsNQR8eV/pckC/dn5qz0dYf4ZJbnuXRzYdoHEV8BODkhY3suuECjpyevbWKs3gwMEmEBOCLb1vCzz68kkRSFewW7g/HhxRolpOSC4mIVGFYHNcopTJLXr8DXKeUytq/QERmAscDDzs2Xw8cDZwENADXDXPuFSKyXkTWt7W1jfFbZAqJjpMMRzSexOsWO6tn3a7Ru7ecri1IxQfOXTKdb79nKZCe5eKMUcysLW0lspPOgcKLEZ1Yaz3QPbx7y8pg64/Gh1h5C5sLT6EdK8NZJC6X0FxdwY+f2M4Tr7aSSCpiCWXHcR7edIjndnTyek/YbgpZbGY4hGSyZG1ZLDVTrDcUmI3YfzhbJCLixRCRO5VS92Q5ZCXwWxHZheHC+pEVVDd5L3CvUspOATFdZkopFQFuB1Zl+2yl1C1KqZVKqZXNzYUXPWXidKvk20toKhJNJPF5XMypD9BUVcHGfaMPuIds15bx39RlRlmr/R7efeIc/F5XWtuaQYc7IJYYvUutULoGowUXIzqZacY4Xs+RLt1tWiRKGWJiUeP3jMvFMpcbz6opuez2deztHCSRVJxquqEe23LIPm40gfZ8WOjo1DtZXFsW02r8zK4LsH5XV0Hn9YVjh2eMRIwpMbcBW5RSN2U7Rim1QCk1Xyk1H/gDcJVS6j7HIe8nw61lWinW+18IlGU+ZZ2eS5IXsXgSr9uFiDC7PjBiEDkXdozEvBhYGSnWH0ylz8O21n7+9+FXiCWS9u9lWnVFWX9HnQOxMaXgzqyzLJKhQtIzGON36/bYwWmAg46f6Z8/dfqoP3cs5GrJYtWIADy51fAGnGAGwJ94NeUduGRVS0nW5pyTUfE2AAAgAElEQVQJUzGKGMx4s3pxI89sb7fb4edD92CMukB50sCzUUoJWw1cCmwUESuz6gtAC4BS6ie5TjbjKnOBv2XsulNEmjHSyTcAVxZvycPjHAU6GEuMecTo4UoskbTvSGfV+scUI7GFxJNdSAI+N4+/0srjr7SyqLnKtkiMgUzlKx7tGohy/OzRT89rrPThc7uyiu4vn91lD3Fqrq6grS/CflNwbnz38bSUof1FNnL542fVBdhnzknZesiocj/BURX/6TcfwTuWzipLbGcy/o2+8chm7lq/jxf39aSlMg+HUoruUHnTwDMpmZAopZ4G8v4tKqUuy3i9iyxZXkqpsffJHgXOTq2JpCIST9qjMjUpjGC78WufWRvgiVfbRi26oVgCt0vs97NcW5aQON0Wj73SSktDEK9bqPF72bi/h90dA8wbRQuOzoEoP/3bdq4+a/GI3XSVUnSO0bUlIsOmADu/49z6gCEk5kW6GJ1+R4srR73Cbz5+Mvf9cz/ffvQ1XjlohEWdkwyvetOikv/t3PrhlfxjDPG58WT1oiZE4KmtbXkJSTiWJBpPZp34WC4mn903TmTe+Q1q91ZWok6LpM5PKJYYdbPFcCxJwOu2RciySKzsKGds4EB3iFA0YTc1bO+Pcsb/PpHX52x+vZe7zZYaAB+8dS0/fXIHz2wbeVTpYDRBNJ4cU7AdjID7/u7QkBbszliPNTjrhT2G/7ycE/AKYW5DkLctnQXAul1dzK4LUBf0ce9Vp/LCl84pyw3Y2Uum84W3HlPyzykF9ZU+TphTx5Ov5Zck1B0aWx1TMdBCkieZd3/ldJ1MJmKJpO2XtprH5dN7KRuhWCKtktvK8rEExXm33hOKMRiNE/C5CwqwfujWtbz1e09x7e9fZIc5g9y6mOeT5m332RpjBtKsugDP7+7i/O8+lSYmdnX/Uc1cc84R1Pg9PG1OKbTmckxEKh3z6FeYd9XLW+pH3UZmqnHa4kZe2NPN1/68mcQItVhWi566cezyoIWkAP7zbUu4cJlxp6Utkuw4YySrFjTgdQv3bdg/wlnZCccSaXevN7z7BK4772h76t1QIUkQ9HkK6vj6tMPq+MSvnk/rEZXPhDm78+8Y7wanOQLEzmK+SCxBhcfF7Zev4ugZNSydW2eva7wtkseuPYOnPp99Nr0zg2hpRtdgzcicstBohHnb0zt5aYR+cpZFol1bk4SPnraAd5hCUoz5yocjsbiyhaSpqoK3Hj+TPzy/b1QWXDiWSBOFhkofn3zTItvV5XRt9YRitmvL2e8rXMBUy62t/azdkfKrt2UIyTf+spnzvvNk2rZiWSRNjuI85x1oKJZIK6p7+wmz7OczSjy1byQWNVcNO4LW+XubOQ4tXCY7ztjI2hFaDfUMWhaJdm1NGqyLl04Bzk40kcTrSLn80Mnz6AvHeXBj4W1lQtFETn+6000SSyjaB6IEfO40a9ESg7vW7aU1ozGiM73Smk3+8TvWO85NP/5nT+3klYPpWWijnUWSSVN16vxuR/t8SxwtLloxm4tPnMMvP7pqQldtO5MrmkpUL3I4E/C5ee3r57OwuZLnd+euKek2Y5DjmbWlhaRArDbNjzoKqzQpYokkPkd7b6uB3sERuttmoz8ST/O1Z5J5gTrYEyLoc6dZi619YQ70hPj83S9x1Z0vpB3f62g3YtVyWEwzU22z4bRyOq0W8mN0bTktkq7BKPu7Q/QMxghnZAd63S6+9Z6lo5osOF446zo0+ePzuGhpCI7YGdrqSD6e8SctJAVi3QXe/vddo+rSebjjjJEAeN2CS4Z3Me1sH+BffrmONa8O7dnZ0R9Nu8BmkrnvUG+EgNedVu1+sCdi3+EfyrAweh3ZZDNr0t0vx82uHVZInD2xusx53tnGvhaC82LbE4qx+obHOefmv41olU0GmrSQjJrGygo6+qNE48lhg+4He8PUBb3j+v9kWCERkc87nr8nY983S7moiYzzDjmzhflUJJFU3PTIq3b/Matpo4WIUOFxDxGSwWgcpRS3Pb2Dv25p5ZFNQy28tv5IQUICRgD+9CNSE/v2dQ3aa/O60v+7O9OS6x19uo6fXcuCpkrbZTBkXQ6B6RyMUh/05qyryAfnd7GyxYxxtAkC3sl9vzeezQQnO01VPtr7Ixz5xQf57F3ZJ2Yc7ImMe7ws1//QSxzPr8/Yd14J1jIpcAZ4owW0MDhcWb+rk+89vo1/v3sjYDVtTP9v5fe6iMRTP6uDPWGW/OfD/Pq53fSY44t7QumptpF4gr5wPKd/PdudbrDCw2fPOYpnrz+L2oCXvQ4hyRz64xQS52S5/7n4BGoDXgajCXu+ipN2h5C09UXsGSFjwVkD8OrBVPpvbzg2oWMh+TAZq8snCo1VPvtv5/4N2WfsHOoNp3U8Hg9yCYkM8zzb6ymDM+U0UkBG0OGKdZFIWSRJfJ70/x5+b7pF8pt/7DH/3WvPeM9shGn1lsrVatwSGadALGisxO0SZtYGaGkIsqczRHtfdMhxQFqqr/NiXe332E0Je7NYJc7uz/u7QsyuH3tWktsl3HvVqSxoquTl/Skh2bi/x24RM9n44Bta0qxDTeHkc5NyoCdc1m7X2cglJGqY59leTxmcd9vOu+ypihWPsLLYMmMkYDTOs4aCAXZe/MHesB3wtoqqLH7xzC4gu/vKwtp35lGpwPPi6an+TS0NQfZ2DtqxrHiGj9lpkThFptrvtYv9nMdYNxHO+pL93SFm1RXnj3h5Sz1nHNmcZukqlWpaOdn4xkXH86uPvWG8lzGpyeyQHHVcc2KJJGteaaW9PzKhLZKlItIrIn3ACeZz6/XxZVrfhOTPnzoNKKxG4XClz7yrtwQlM0YCQy0SS1Q6B6J2aqM19xyMtNxbntwB5G417ve6efAzp/PdS5bb245wNAKcUx9gZ/sAvzfbnzjTasEIzls48/arK1IWiVNILCF67VAfT7zaStdAlJ5QjNl1xWuceM3ZR9jPLYurkAJLzeFF5o2Uc4TxXzcf4vJfrANgmWPY23gwrJAopdxKqRqlVLVSymM+t15PzCY/ZcJq26EtEugzLQpLSKLZLBKvO+1nFYolOP2IprQphl0DUXsMq7M1yYIRmi4eM7OGygoPV7xxIUDa/PLMtNPuwShf/dNm3vuTZwF4cW83R02v5u//fhbvXJbqD+pyScq1ZX6/RFLZd4N/fukAl92+jk/e+bzxmUWySADqgj7WfuHN3H7ZSZyzZDqA3bRSM/U4cno1l6+ez48+uAKA53enihOtv735jUHedNT4poMXlA4iIpUi8iER+UupFjQZqDB91lpIUqNf2/oiPLejY0gdCViuLadFYhTZOduIx5OKPrP+w8qb/9mHV+ZdMX79+Uez/ZtvTcuecpr7nz/vKOJJxc//vpN/mLPkX9zXzbK5dcyuGxrjcFokt/99Jyu//igAlQ4303NmFXyxK7en1/g58+hprF5sxBesVuyaqYfP4+LLbz+W84+bwcxaf9o8FyvGd//Vp417QsOIQiIiPhG5SER+DxwA3gzknCVyuFNhWiTatZVybQF856+voRRDZpf7vW7C8fS2JQGf2+4vNcsMFHY60l4Bptfknw0lIkOC6c7+VfMzLJvtbf10D8Y4bpg+UFaTzjue2cV//WmzPer22FlDjy/VZLrViwwhGWv7Fc3kR0Q4ZWFjWpW7ZZGM54hdi2FXICLnYkwoPBdYA9wBnKSUurxMa5uwaIskRV8kjsclTK/x2/UVmd13/R4XrQ7RDcUS+D1u+65/Wo2f13vC9h2WZZGMNYA4zXF+5lzzF80RwE7X2QXHz7TXYE0AXJ/RnqKlMThkzkWu6vuxUF/p4+5PnsqicZjJrpl4HDG9mnv+uZ/ecIwav5fecIzqCs+QG6jxIJdF8hCwEDhNKfUhpdSfAH3lJDW+MxLXFklfOM606gpOP6KJ7W0DAFRm3KFnxkjCsSQBn9s+zupzZbnJWvsiiKTaxo+WXBbJRjNzbJ5jzswPP7jCzjLye1NC94kzFrKwyTh/XpYmhaWs8zhxXj114zhnQjNxsBJJtrUars6+cHzMHRWKRS4hWQE8C/xVRB4VkY8BOn2ElJA4U1qnKn3hGFV+jz17BIa6evwZMZJQLEGF12VbLlZco9eOt4RprKwY4iIrFEuoZtT4h7SP+OWzuwFy5t9bQra4uYpjZhkT/iqyVJk7i1Q1mlJhxRT/uvkQr3eH6AvHxn2UgEWurK0NSql/V0otAr4MLAO8IvKgiFxRthVOQIy2Hy5tkWDdFXnTAtZDhMSR/ps0s58CXjdvPsbISnr/qrmA0aQRjMB9sRr9/elfT+NPZrr2rz62ihvelZ65nkus5ptWSEtDkE+fdQS1AS/nHzdzyHE6PVdTDuY2BKnxe/jRE9s556a/TRqLxEYp9YxS6lPAHOBm4OSRzhGRuSKyRkQ2i8gmEflMjmNPEpG4iFzs2JYQkQ3m44+O7QtEZK2IbBOR34nIuNj9FR4XkSlukcQSSV7e38OCpsq07rmZwT+/N1WQGDbFN+B1s6Cpkl03XMAZR04DUoH7tv5o0VqPHz+n1hal049o5o2Orrk/NlMqh+OGdx3P1Wcu4sR59Rw1o5oXv3xu1vkbE8FHrTn8cbvEHl41EE0YsZJxnIroJFfTxhWZDwyrpB34QR7vHQeuVUotwRCeq0VkSZbPcQM3Ao9k7AoppZaZj3c4tt8I3KyUWgx0AR/LYy1Fx/D7T22L5Pfr99EbjnP2MdPTWjlUZrh6KjzGz0opZQuK09VkWTBWjKS9L0Jzjor2sVDr+MM7//ih1oWTaTV+PveWo8fsYtNoisXHTzfqpUQmVowk1yrWAy9jCAek99dSwFm53lgpdQAjXRilVJ+IbAFmA5szDv0UcDdw0kiLFSNZ+izgA+amXwJfAX480rnFZqpbJNF4kv/60yZaGoxiqH7HDJDM/9x+r4ukgu1tA3Zg2ukO8nlcVHhc9EWMjsDt/ZGStR4vZJ77cNx39Wr+sbODbz7wShFWpNHkz6Unz2NPxyC3Pr2T3R2DvPGIiTGXJpeQfBa4GAgBvwXuVUqNqjJKROYDy4G1GdtnAxcBZzJUSPwish7DsrlBKXUf0Ah0K6Wsq9Y+DHHK9plXAFcAtLS0jGbZOfF73Ty06SBfj8anZLC1YyBCJJ7kyjMW4fe606rZM7O2rNZRH/3FOttllRm0rvZ76QvH6Y/EicSTJZuqJyJ88YJjWOFoiVIoy+bW2ZlmGk05ERFaHJmGHzl13jiuJkWuYPt3lFKnYVgMc4HHROQuEVlWyAeISBWGxXGNUqo3Y/d3gOuUUtlu7ecppVZiWB/fEZFFhXyuUuoWpdRKpdTK5ubiq/a21n4Gowm+99i2or/3ZMCqGbFSbJ1xgsy6itPMDrCJpOKFPUbabWaAutrvoT8St9uj5GrWOFb+5fSFrGgZvZDAUKtLoykXltv36BnVLJ5WPc6rMRjR+auU2gHcjxHDWAUcme+bi4gXQ0TuVErdk+WQlcBvRWQXhvXzIxG50Pzc/Y7PfwLDoukA6kTE+iueA+zPdz2lIDpFixItIcmWXVWR0fb8xHn1XHD8zLSYUmY6blWFh75wzG7RXkohKQYTJe1SM/WwrP85RRhfUCxyBdsXisgXRGQt8F/Ai8AxSqm78nljM55xG7BFKXVTtmOUUguUUvOVUvOBPwBXKaXuE5F6Eakw36cJWA1sVkZXvzUYogPwEQyRKzs+85eZra5gKpBLSLJRE/CmNWP0ZPTjqvZ76AvH7aFRE19ItEWiGR9WL27inctm8dV3HjfeS7HJdRXcBrwXo8L9WaAF+KSIfFZEPpvHe68GLgXOcqTxvlVErhSRK0c49xhgvYi8iCEcNyilrCD9dcBnRWQbRszktjzWUnSeuu5MIPvgo6mAJSS52rw7seZ7WGSOKbZavrdZFkn1xK7mzuxwrNGUi4DPzXcvWZ5WBDze5Lqt+iqpAVZVOY7LilLqaQqYpKiUuszx/BmGmXliurpWFbqeYjO9xs/Cpsq0eRVTibb+CHVBb5ob6/Frz6BrMJr1eGfa7aUnz+O0xemT85bOreOu9fv4555uRKBhkrQF0X2wNJocQqKU+koZ1zEpqQ547bYeU43uwRh1GcVQC5uHv99wCsm/nrU4rd07wNI5xmCev245REPQNylqN567/s0TovOqRjPeTPy/1glMbcA7ZV1boViCQAFpzzWO4HR9FmvjiOlVdpHVRI+PWMyo9ZeshbxGM5nQQjIGavyeKSskxnCq/P/7OC0Sn2foeRUeNzPNtu8TPT6i0WjS0UIyBmoCXnt+xVRjMJooqH26VVuyNMds6Tn1RqFVtomFGo1m4jKiXW6m4b4bmO88Xin11dIta3JguLaMth7jPeqy3ISiiawuquE4YU4dnzhjIf9y2sJhj7FE+dRFTcMeo9FoJh75WCT3A+/EaFUy4HhMeaoqPEQTySk5KdEal5svXreL688/JmfdyQnm2NtTFzeOeX0ajaZ85BMpnKOUOq/kK5mEWIHWwWhiSKX24c5gtLAYST585R3H8pFT5zOtemwjdjUaTXnJ50rwjIhkremY6lidZAciUy8FOBRLFL1ZZdDn4dhZtUV9T41GU3ryuRKcBlwmIjuBCEaRoVJKnVDSlU0CrC63A9GpKSRTzQrTaDTZyUdIzi/5KiYptpBEptaAq4RjXK5Go9Hk0/13N1AHvN181JnbpjyVpmtrcIpZJCFz/noxhkRpNJrJz4hCYs5avxOYZj5+LSKfKvXCJgNWjGCqxUhCUUNI/FpINBoN+bm2Pga8QSk1ACAiN2J0A/5+KRc2Gaiaoq4tS0i0a0uj0UB+WVsCOK+UCQro6ns4E6zQri2NRqPJxyK5HVgrIveary9knGaATDQqTddW/1SzSGLaItFoNClGFBKl1E0i8gRGGjDA5Uqpf5Z0VZMEv9eFS6aeRWLFhAqpbNdoNIcvwwqJiNQopXpFpAHYZT6sfQ1Kqc7SL29iIyJU+jxTLkbSMWAMr2qs1F16NRpNbovk/4C3Ac+TmpQIZkEiMHz3vSlEsMI95bK2JstcdY1GUx5yTUh8m/nvgvItZ/JR4596reQ7BiK4XZI2Y0Sj0Uxd8qkjeSyfbVmOmSsia0Rks4hsMutRhjv2JBGJi8jF5utlIvKsed5LIvI+x7G/EJGdIrLBfCwbaS2lpC7oHXZO+eFKe1+UxkrfkHG5Go1mapIrRuIHgkCTiNSTSvmtAWbn8d5x4Fql1AsiUg08LyKPKqU2Z3yOG7gReMSxeRD4sFJqq4jMMs99WCnVbe7/nFLqD/l8wVJTF/Sxt3NwvJdRVjoGIjRqt5ZGozHJZZF8AiM+crT5r/W4H/jBSG+slDqglHrBfN4HbCG7AH0KuBtodZz7mlJqq/n8dXNfcx7fp+zUBbx0D6a7ttr7I1z5q+fpGTw8XV5t/VGaqnSgXaPRGAwrJEqp75rxkf+nlFqolFpgPpYqpUYUEiciMh9YDqzN2D4buAj4cY5zVwE+YLtj8zdMl9fN5gTHbOddISLrRWR9W1tbIcstiPpKH92hdNfWDx7fxkObDvKHF/aV7HPHk86BiM7Y0mg0Nvk0bfy+iBwnIu8VkQ9bj3w/QESqMCyOa5RSvRm7vwNcp5TKOmJQRGYCv8KoXbGOuR7DSjoJaACuG2bdtyilViqlVjY3l86YqQt6CceShGOpFOB+M4urquLwrLPoC8ep0YF2jUZjks/M9i8DbwKWAA9gtJV/Grgjj3O9GCJyp1LqniyHrAR+a847bwLeKiJxpdR9IlID/AX4D6XUc9YJSqkD5tOIiNwO/L+R1lFK6gLGnXn3YIwZtYZw9Ietgr3iDn6aCCil6A/HqfYfft9No9GMjnx6bV0MvBk4qJS6HFgKjDjGTgx1uA3YopS6KdsxpqtsvlJqPvAH4CpTRHzAvcAdmUF100qx3v9C4OU8vkPJqA8ad+bOzC3LIoknDr9Z7uFYknhSUVWhLRKNRmOQz21lSCmVNNNzazAC33PzOG81cCmwUUQ2mNu+ALQAKKV+kuPc9wJvBBpF5DJz22VKqQ3AnSLSjJFFtgG4Mo+1lIy6oGGRdA0MFZJw7PATkr6IkUBQpS0SjUZjks/VYL2I1AE/w8ja6sdoI58TpdTTFNAlWCl1meP5r4FfD3PcWfm+ZzmYUesH4GBv2N6WEpLDr3WK5bar0UKi0WhM8mnaeJX59Cci8hBQo5R6qbTLmjzMqDGE5ECPQ0jMi204fvgJSV/YSiTQQqLRaAxyFSSuyLXPqhGZ6gR8buqCXg6aQhKNJ2nvN3pRHW6ure7BKO/84d8BLSQajSZFrqvBt81//RjZVS9iuKpOANYDp5R2aZOHGTV+2yLZ1TFAPGn0uDzcXFt/fPF1+7mOkWg0GotcBYlnKqXOBA4AK8yajBMxCgv3l2uBk4GZtX4O9oYAePVgn739cBOSJ15NFXbW+HXWlkajMcgn/fcopdRG64VS6mXgmNItafIxozZgu7a2t/UjYqQFH05CopTixb3d9mvt2tJoNBb5XA1eEpFbSWVRfRDQwXYHM2v9tPdHicQTdA/GqK7wUBvwHlYxkkO9EXugFUClFhKNRmOSz9XgcuCTgNUG/kly9MaailgpwK29EXrDMWoCXvxe92FlkWx6vQeAn3xoBR6XC58nH2NWo9FMBfJJ/w0DN5sPTRZm1qZSgHtDcWr8XrweF+H44WORdPQb1sjxc+qYXRcY59VoNJqJRK7037uUUu8VkY2kj9oFQCl1QklXNolICUmI3nDM7kMVjh4+FolVE1OhLRGNRpNBLovEcmW9rRwLmczMqDXu0Pd1hegLx5lTHyAaTx5WkxMjZrzH7z08OxprNJrRk2tm+wHz393lW87kpKrCw5HTq3ji1VZ6QzGqZ1YzEIkfVjES67toi0Sj0WSSy7XVRxaXFkZRolJK1ZRsVZOQdyydxbceeQ0waywUDEQmv5AopRARIvEkbpfgdWsh0Wg06eQqSKxWStVkeVRrERnK0TNSP5KagJeGSh8dA5FxXNHYuX/Dfo764kNsOdBLOJbAr60RjUaThbyLAURkGka7FACUUntKsqJJyrSa1MTfGr+HgNdNOJZkMBonOEkHXK3b1Uk0keSnf9tOld9DhY6PaDSaLIx4iyki7xCRrcBO4G/ALuDBEq9r0jGt2tZYqv0eGquMOSVW2uxkJJ4wPJudgzEisaS2SDQaTVbyuTJ8DTgZeE0ptQBjWuJzuU+ZejSZwgEwv7GSxkrj9Yd//g9ik3RSopV11huKEY4ntUWi0Wiyko+QxJRSHYBLRFxKqTUY3YA1DjyOIPSxs2tprDJcXTvbB9jTOTheyxoT3YPGNMS+cIxILKEztjQaTVbycd53i0gVRmuUO0WkFRgo7bImN1UVHtsigdSgq8mGJSS94TjheFLXkGg0mqzkc4v5TiAE/BvwELAdePtIJ4nIXBFZIyKbRWSTiHwmx7EnmTPhL3Zs+4iIbDUfH3FsP1FENorINhH5nojkPc631PzwAyv44QeMeWBNVang+2QtTOwOpVxb2iLRaDTDMeyVQUR+KCKrlVIDSqmEUiqulPqlUup7pqtrJOLAtUqpJRgxlqtFZEmWz3EDNwKPOLY1AF8G3gCsAr4sIvXm7h8DHweOMB/n5fVNy8AFJ8zkghNmAsbkxP9+1/EAtPZF2N8dGs+l5eSudXu59akdaduUUnSZFkkknqQvHNcWiUajyUquW8zXgG+JyC4R+R8RWV7IGyulDljjeJVSfcAWYHaWQz8F3A20Ora9BXhUKdWplOoCHgXOE5GZGDPjn1NKKeAO4MJC1lVOzl0yHYDP/+ElVt/w+DivJjvbWvv4/N0v8fW/bEnbHooliMaTzDL7iLX1R7RFotFospKrIPG7SqlTgDOADuDnIvKKiHxZRI4s5ENEZD7GZMW1GdtnAxcxtC39bGCv4/U+c9ts83nm9glJbSB9imAima1RwPhy9k1PZt3e1mcUUy5orrRfa4tEo9FkY8RbTKXUbqXUjUqp5cD7MSyALSOcZmMG6u8GrlFK9Wbs/g5wnVKq6PmxInKFiKwXkfVtbW0jn1ACPBntRCZT760dbUY+xQlz6uxt2iLRaDTZyKcg0SMibxeROzEKEV8F3pXPm4uIF0NE7lRK3ZPlkJXAb0VkF3Ax8CMRuRBjJvxcx3FzzG37zeeZ24eglLrFnDO/srm5OZ/llpzBCdZWPu6ob7Fa31vsaDeE5JSFjfa2gejkzD7TaDSlJVew/RwR+TmG++jjwF+ARUqpS5RS94/0xmY21W3AFqXUTdmOUUotUErNV0rNB/4AXKWUug94GDhXROrNIPu5wMNmR+JeETnZfP8PAyOuZaIw0SySqCkkbpcMWduOtn5qA15OP6KJG8ykgZf3ZxqUGo1Gk9siuR54BjhGKfUOpdT/KaUKqR9ZDVwKnCUiG8zHW0XkShG5MteJSqlOjIr6debjq+Y2gKuAW4FtGKnIE7pdy++vPIWPn74AMALYEwlrxkhdwEssodIq8He0DbCwuRIR4cLlRhhq9eLGrO+j0WimNrnmkZw1ljdWSj2N0XI+3+Mvy3j9c+DnWY5bDxw3lrWVk5PmN9AbivGzp3YSmmCuLWvqYW3QS8dAlHAsYbeJ39Hez2mLDZeg3+tm/RfPNtrjazQaTQY6eloGAj4j22kiWyQAYfP1QCTOod4IC82MLTAKLH062K7RaLKgrwxlIGCmzY7WIvn7tnbm//tfONBT3KLGSNwQjvqg0c7FipPsNAPtC5sqs5+o0Wg0DrSQlIGxWiR3rTdKap7a2l60NQFELNeWaZFY69vdYTSZnK+FRKPR5IEWkjIwVotkWrXRt8sqEiwWliurNmgKibk+q8eWs/GkRqPRDIcWkjJgC8koLRKXy8hZ2NdV3Hb0lkWS6drqDRn1IjUBHVzXaDQjo4WkDGDPPikAABNCSURBVNiurVFYJIPRONtbjZjF9rbidu+3g+3BdNdWbziGz+3SlewajSYvJucw8UmGfwwWyYduXcsLe7oBaC+ya8sKttfaWVuWRRKjJuBhAnXo12g0Exh9y1kGvG4XXreMSkgsEQHoGCjuXBNLOOpM11bKIolTrWtGNBpNnmghKRN+r3vMBYk9oVhR579bFolVRxKKGq97QzFq/NpY1Wg0+aGFpExU+jwMRAprevjxO9YP2VbMaYtWsH1mrZ/6oJc7nt1FMqnoDcd0oF2j0eSNFpIyUe330Ffg7PZHNx8asq2ziO4tyyKp9nu56k2LeeVgH12DUdMi0UKi0WjyQwtJmaj2e+gv0CJxct6xMwDo7C+ekFgxkgqPi2azVqU3HKc3HKcmoF1bGo0mP7SQlIkqv5e+cGxU5171pkV89lxjKGUxA+6ReBKf24XLJbZwtPdH6B6MUhvQxYgajSY/tJCUiUJdW9F4Kqge8LqZVRfAJfDaob6irSkcS9i1IpYr68nX2oglFMtb6nKdqtFoNDZaSMpEjd9DbwFC4szwiiaSVFV4WDq3rqj9tvrDcarM7CyrluShlw8iAicv0LNHNBpNfmghKRNVFR76I/m7tvodY22toPjqRU28tK+bcCzB2h0dYw6894ZTQXUrS2traz+zagN2/y2NRqMZCS0kZaLa7yUcS+ZdBzLoCMy/5djpACyaVklSGd1533fLc1x2+z/GtKbeUCqo7szSqtMiotFoCkALSZmoNl1I+cZJrAyv2y87iRPnNQAwtz4IwIv7jGr3scZLnBaJ3+vCZ05HtJo4ajQaTT5oISkTVsuR/jyFZNCMkVRWpNJwWxpMIdlrCElVxdhSdJ2FhyKpzC1tkWg0mkIomZCIyFwRWSMim0Vkk4h8Jssx7xSRl0Rkg4isF5HTzO1nmtusR1hELjT3/UJEdjr2LSvVdygm1kW/N88UYMsiCZqdgwGaqyuo8Lhsi6RyrEISiqe1QrFERVskGo2mEEppkcSBa5VSS4CTgatFZEnGMY8BS5VSy4CPArcCKKXWKKWWmdvPAgaBRxznfc7ar5TaUMLvUDRqCnRtDZrBdqfVISLMawzy8v5eAIK+3ELyu3V7+NbDrwKw+fVekkll70smFX0ZrVDmmK6zem2RaDSaAiiZkCilDiilXjCf9wFbgNkZx/QrpayrWyWgGMrFwINKqeJOdSoztmsrz+r2/ojh2gpWuNO2Hzm92n5e6Uvfl8l1d2/kB2u28dK+bt76vaf46ZM77H0D0ThJlR5kn1njB/RAK41GUxhliZGIyHxgObA2y76LROQV4C8YVkkmlwC/ydj2DdMldrOIVBR5uSWhyrZI8nNtWVlbmXGQY2bW2M8DIwiJxaFeY47J2p0d9jarpsXZCmVajdkmJTS6CnyNRjM1KbmQiEgVcDdwjVKqN3O/UupepdTRwIXA1zLOnQkcDzzs2Hw9cDRwEtAAXDfM515hxl3Wt7W1FeW7jIV8s7ZeO9RHTyjGQCSOSGpMr8UxM1MWSa5U4o7+1BCsHlMYwo55KJZYOOeOnLvE6Od18kJdjKjRaPKnpEIiIl4MEblTKXVPrmOVUk8CC0WkybH5vcC9SqmY47gDyiAC3A6sGub9blFKrVRKrWxubh7zdxkr1XlaJOfe/CTv/cmzDEQTVPqGTik848hpXH3mIiBVqJjJ7X/fyYlf/6v9ekdbPwDhWOp4qx29M7B+/Jxatn7jfE5d7PwVaDQaTW5KmbUlwG3AFqXUTcMcs9g8DhFZAVQAHY5D3k+GW8u0Uqz3vxB4ufirLz4VHjc+j4u+HDESq7/Wq4f6GIzG0zK2LNwu4XNvOZqzj5lOOJZk7Y6ONEsD4L/+tDnt9c52Y9Z7OJYgHEvQH4nTNWAIWkNleoaW160zwjUaTWGU8qqxGrgUOMuRqvtWEblSRK40j3k38LKIbAB+CLzPCr6bcZW5wN8y3vdOEdkIbASagK+X8DsUleqK7I0bn9nWznt/+izdjqFV/ZFEzjoRv9fFlgO9vO+W57jj2V1p+zLP29FmCEnXYJT33fIcx335YToti6RSB9Y1Gs3YKNnQCaXU04CMcMyNwI3D7NtFRpaXuf2sYqxvPBiuA/Cnf7uB9v4IO0zLAYxge2bGlhO/I3bidFmBMV/EESJhR7vh2jrUG7ED71sOGOGqOt0uXqPRjBHtxygj1X4v/VliJNbI23bH1b83HMtZJ2K1f4f0lvPhWIKOgSjHza7hd1ecDEAsMTSr+v/W7qG6woPPo/8LaDSasaGvImWkahjXlhU0P9gTtrftbB8YwbWVskic1fIHzPe4/NQFnDS/Ab/X+BUvnVM75D087pwGo0aj0eSFnqdaRqr9HnZ3DK2rjGYRkvb+aNZgu4XTkuhx1H0MWPUnfg8ul9DSEOS1Q/0cOb2a84+fSUd/hDOPmsYHbl1L16CuF9FoNGNHC0kZCfjchDIyrJwc6A2nvc5lkTjdWU4hsTK4LBGy+nEdM7OGj562AIBEMlsDAY1GoxkdWkjKiN/jtuMh2bAskoZKH50D0ZwxEmcXYWcluiVUViGjFYg/akaqkNHtEn53xclpxYgajUYzWnSMpIz4va4hGVZODvaEEYFTFxmV5ZU5sracPbucFok1oteKoVx68jwgvbUKwBsWNrJkVvo2jUajGQ1aSMpIhXeoRTLoGKl7sDdMdYWHVQuMQVZ7OofvU2kJyYwaf9os+FCGa+v9q+ay7RvnDyk81Gg0mmKhhaSM+D2GRZJqeAxrd3bazxNJRbXfy5lHTQNg6Zy6Yd/rK+84lgtOmMl5x83IapFYDR1FBI+uVtdoNCVEX2HKSIXpboomkmzY200yqfjThtfTsrNqAl7mNgT555fO4bJT5w/7XguaKvnhB1bQVOUjGk/alk5mjESj0WhKjRaSMmIVEW450MeFP/w7P1yzjXv+uZ/3r2qxZ4tYzR3rK324XCPXeVhZWQPm/JLBDItEo9FoSo0WkjJiWSS7O4xWKE9vawfgw6fMI2gKQk2BmVQpITHiJOFYApeAT7uzNBpNmdBXmzLiNy0SK83XKk5sqqqwLRLnDPV8sGpNrOB7KJog4HUPaT+v0Wg0pUILSRmxUnIPmoWHB3vDBLxuKis89njbQsfcZlokg7EEgRFmuWs0Gk0x0UJSRqwYySFHBXtjlZGWO7suAKRiJPlSZdaaWBZJOJog4NO/Vo1GUz70FaeM2BZJj1NIjDnpTea/lTnaomTDGWzf2T7APf/cj1u7tTQaTRnRQlJGUhZJql18s2mR1AcNl9ZAjgmK2aj0pVxbtzy5HYBdWRpDajQaTanQQlJGLIvkQE/I3tZYaVgis0zXliqwn6IVbL9vw36m1/iLsEqNRqMpDB2VLSMV5mwQZ/Pd6TWGkLz7xDm09UW43OzQmy+Wa+uZ7R22y+zWD68swmo1Go0mP7SQlBG/Z2iR4JyGIABet4tPvfmIgt/TOZekNxynLujl7CXTR79IjUajKZCSubZEZK6IrBGRzSKySUQ+k+WYd4rISyKyQUTWi8hpjn0Jc/sGEfmjY/sCEVkrIttE5HciMmm6EfqztC2ZUx8o2vv3hmJ2zESj0WjKRSljJHHgWqXUEuBk4GoRWZJxzGPAUqXUMuCjwK2OfSGl1DLz8Q7H9huBm5VSi4Eu4GOl+wrFpSLLfPS59cExv69VxBhNJAtOH9ZoNJqxUjIhUUodUEq9YD7vA7YAszOO6VepVriVQM5Qsxjl2mcBfzA3/RK4sJjrLiXBLPNFZtSOPUD+5OfPtJ8Xmj6s0Wg0Y6UsVx0RmQ8sB9Zm2XcR8N/ANOACxy6/iKzHsGxuUErdBzQC3UopK0d2HxniNJGpcMRIPnRyC/MbK/EWoSeWc5JirvG8Go1GUwpKftURkSrgbuAapVRv5n6l1L3AvSLyRuBrwNnmrnlKqf0ishB4XEQ2Aj0FfO4VwBUALS0tY/wWxeeMI6dxTpGC4j6PC69biCWUFhKNRlN2SlpHIiJeDBG5Uyl1T65jlVJPAgtFpMl8vd/8dwfwBIZF0wHUiYh1tZwD7B/m/W5RSq1USq1sbm4uxtcpClZzRqsAsVhY80e0kGg0mnJTyqwtAW4DtiilbhrmmMXmcYjICqAC6BCR+v/f3t3FWnGVYRz/P4XyIVRoD0iIoEdakhZtpYRUKL2oGA2isRclQdIoMcSGxihGo0BMTDReWC+sosRYUz8Sm9YY+xUu2lIgRlNTSi3QUxB72uAHoQIKGKxBiq8X8+7Dzuk5be3s2dOz5/klkz2zZvZkvZvhvHutmb2WpIlZPgNYBhzI+ym7gFV5irXAg1XFUIXWcPHTO5xIWvdGfI/EzLqtyhbJMuATwPK2x3hXSlovaX0eczMwIGkvsBVYncniKmCPpH0UieObEXEg37MR+IKkQYp7JndVGEPHfeDKYhrdaZM7+9TypKEWiSe0MrPuquzra0T8FnjV0QMj4naKx3mHlz8OXD3Ke14ArutEHevw9Zvew9rr+5l5ycSOnvfUS/8BLgy1YmbWLR5rq8smjL+Iq2a/tePnPfnSOQDmzZza8XObmb0aJ5IeM2/mlLqrYGYN40TSY/qmjJkRY8ysR/gRnx5xz6eX8Od//MtztZtZ1zmR9Iill/ex9PK+uqthZg3kri0zMyvFicTMzEpxIjEzs1KcSMzMrBQnEjMzK8WJxMzMSnEiMTOzUpxIzMysFF2YMr13SToO/OkNvn0GcKKD1RkrHHezNDVuaG7sryfud0bEa84M2IhEUoakPRGxuO56dJvjbpamxg3Njb2Tcbtry8zMSnEiMTOzUpxIXtuddVegJo67WZoaNzQ39o7F7XskZmZWilskZmZWihPJKCStkHRI0qCkTXXXp5Mk/VjSMUkDbWWXSdou6bl8vTTLJWlLfg77JS2qr+blSJoraZekA5KelbQhy5sQ+yRJuyXty9i/luXvkvRExvgLSROyfGJuD+b+/jrrX5akcZKelrQtt3s+bkmHJT0jaa+kPVlWybXuRDICSeOArcCHgQXAGkkL6q1VR/0UWDGsbBOwIyLmAztyG4rPYH4utwI/6FIdq/Ay8MWIWAAsAT6T/65NiP0ssDwi3gssBFZIWgLcDtwREVcAJ4F1efw64GSW35HHjWUbgINt202J+/0RsbDtMd9qrvWI8DJsAZYCj7RtbwY2112vDsfYDwy0bR8CZuf6bOBQrv8QWDPScWN9AR4EPti02IG3AL8H3kfxg7TxWT503QOPAEtzfXwep7rr/gbjnZN/NJcD2wA1JO7DwIxhZZVc626RjOztwF/atv+aZb1sVkQczfUXgVm53pOfRXZZXAs8QUNiz+6dvcAxYDvwPHAqIl7OQ9rjG4o9958Gxupczt8Bvgz8N7f7aEbcATwq6SlJt2ZZJde652y3V4iIkNSzj/NJmgr8Cvh8RPxT0tC+Xo49Is4DCyVNB+4Hrqy5SpWT9FHgWEQ8JenGuuvTZTdExBFJbwO2S/pD+85OXutukYzsCDC3bXtOlvWyv0maDZCvx7K8pz4LSRdTJJG7I+K+LG5E7C0RcQrYRdGlM11S6wtle3xDsef+acDfu1zVTlgGfEzSYeBeiu6t79L7cRMRR/L1GMUXh+uo6Fp3IhnZk8D8fLJjAvBx4KGa61S1h4C1ub6W4v5Bq/yT+VTHEuB0W9N4TFHR9LgLOBgR327b1YTYZ2ZLBEmTKe4NHaRIKKvysOGxtz6TVcDOyM7zsSQiNkfEnIjop/h/vDMibqHH45Y0RdIlrXXgQ8AAVV3rdd8QerMuwErgjxT9yF+puz4dju0e4ChwjqIvdB1FP/AO4DngMeCyPFYUT7A9DzwDLK67/iXivoGi33g/sDeXlQ2J/Rrg6Yx9APhqls8DdgODwC+BiVk+KbcHc/+8umPowGdwI7CtCXFnfPtyebb1N6yqa92/bDczs1LctWVmZqU4kZiZWSlOJGZmVooTiZmZleJEYmZmpTiRmJUg6XyOrtpaOjZStKR+tY3QbPZm5SFSzMr5d0QsrLsSZnVyi8SsAjkXxLdyPojdkq7I8n5JO3POhx2S3pHlsyTdn/OF7JN0fZ5qnKQf5Rwij+av0pH0ORXzquyXdG9NYZoBTiRmZU0e1rW1um3f6Yi4Gvg+xQi0AN8DfhYR1wB3A1uyfAvw6yjmC1lE8WtkKOaH2BoR7wZOATdn+Sbg2jzP+qqCM3s9/Mt2sxIknYmIqSOUH6aYSOqFHCjyxYjok3SCYp6Hc1l+NCJmSDoOzImIs23n6Ae2RzEJEZI2AhdHxDckPQycAR4AHoiIMxWHajYqt0jMqhOjrP8/zratn+fCfc2PUIyNtAh4sm0kW7OucyIxq87qttff5frjFKPQAtwC/CbXdwC3wdAEVNNGO6mki4C5EbEL2Egx1PkrWkVm3eJvMWblTM5ZB1sejojWI8CXStpP0apYk2WfBX4i6UvAceBTWb4BuFPSOoqWx20UIzSPZBzw80w2ArZEMceIWS18j8SsAnmPZHFEnKi7LmZVc9eWmZmV4haJmZmV4haJmZmV4kRiZmalOJGYmVkpTiRmZlaKE4mZmZXiRGJmZqX8DzAX5J04AZx3AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def smooth_curve(points, factor=0.9):\n",
    "    smoothed_points = []\n",
    "    for point in points:\n",
    "        if smoothed_points:\n",
    "            previous = smoothed_points[-1]\n",
    "            smoothed_points.append(previous * factor + point * (1 - factor))\n",
    "        else:\n",
    "            smoothed_points.append(point)\n",
    "    return smoothed_points\n",
    "\n",
    "smooth_mae_history = smooth_curve(average_mae_history[10:])\n",
    "\n",
    "plt.plot(range(1, len(smooth_mae_history) + 1), smooth_mae_history)\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Validation MAE')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이 그래프를 보면 검증 MAE가 80번째 에포크 이후에 줄어드는 것이 멈추었습니다. 이 지점 이후로는 과대적합이 시작됩니다.\n",
    "\n",
    "모델의 다른 매개변수에 대한 튜닝이 끝나면(에포크 수뿐만 아니라 은닉층의 크기도 조절할 수 있습니다) 모든 훈련 데이터를 사용하고 최상의 매개변수로 최종 실전에 투입될 모델을 훈련시킵니다. 그다음 테스트 데이터로 성능을 확인합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "102/102 [==============================] - 0s 357us/step\n"
     ]
    }
   ],
   "source": [
    "# 새롭게 컴파인된 모델을 얻습니다\n",
    "model = build_model()\n",
    "# 전체 데이터로 훈련시킵니다\n",
    "model.fit(train_data, train_targets,\n",
    "          epochs=80, batch_size=16, verbose=0)\n",
    "test_mse_score, test_mae_score = model.evaluate(test_data, test_targets)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.9856512032303155"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_mae_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "아직 2,985달러 정도 차이가 나네요."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.6.5 정리\n",
    "\n",
    "다음은 이 예제에서 배운 것들입니다.\n",
    "\n",
    "- 회귀는 분류에서 사용했던 것과는 다른 손실 함수를 사용합니다. 평균 제곱 오차(MSE)는 회귀에서 자주 사용되는 손실 함수입니다.\n",
    "- 비슷하게 회귀에서 사용되는 평가 지표는 분류와 다릅니다. 당연히 정확도 개념은 회귀에 적용되지 않습니다. 일반적인 회귀 지표는 평균 절대 오차(MAE)입니다.\n",
    "- 입력 데이터의 특성이 서로 다른 범위를 가지면 전처리 단계에서 각 특성을 개별적으로 스케일 조정해야 합니다.\n",
    "- 가용한 데이터가 적다면 K-겹 검증을 사용하는 것이 신뢰할 수 있는 모델 신뢰있게 평가 방법입니다.\n",
    "- 가용한 훈련 데이터가 적다면 과대적합을 피하기 위해 은닉층의 수를 줄인 모델이 좋습니다(일반적으로 하나 또는 두 개).\n",
    "\n",
    "세 개의 예제 시리즈를 마칩니다. 이제 벡터 데이터 입력을 받는 일반적인 문제를 다룰 수 있습니다:\n",
    "\n",
    "- 이진 분류\n",
    "- 단일 레이블 다중 분류\n",
    "- 스칼라 회귀\n",
    "\n",
    "다음 장에서 이 예제들에서 보았던 데이터 전처리, 모델 평가, 과대접과 같은 개념에 대해 조금 더 이론적인 설명을 보완하겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tensorflow",
   "language": "python",
   "name": "tensorflow"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}