Ising_CRBM.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Ising CRBM.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/danielalcalde/MCMC_CRBM/blob/master/Ising_CRBM.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "amXB-OcpSa1R"
      },
      "source": [
        "# Simulating the Ising model using a CRBM"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e0xxh6USSodO"
      },
      "source": [
        "## Abstract:\n",
        "Machine learning is becoming widely used in analyzing the thermodynamics of many-body condensed matter systems. Restricted Boltzmann Machine (RBM) aided Monte Carlo simulations have sparked interest recently, as they manage to speed up classical Monte Carlo simulations. Here we employ the Convolutional Restricted Boltzmann Machine (CRBM) method and show that its use helps to reduce the number of parameters to be learned drastically by taking advantage of translation invariance. Furthermore, we show that it is possible to train the CRBM at smaller lattice sizes, and apply it to larger lattice sizes. To demonstrate the efficiency of CRBM we apply it to the paradigmatic Ising and Kitaev models in two-dimensions.\n",
        "\n",
        "## Paper:\n",
        "https://arxiv.org/abs/2008.05401"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XaiBKOtWTBEV"
      },
      "source": [
        "## Introduction\n",
        "This Notebook is intended as an educational tool for the use of Convolutional Restricted Boltzmann Machines in classical Montecarlo simulations. The code for the paper was written with the library Theano, which will no longer be updated, that is why we have chosen to present the main concept in TensorFlow. Specifically, this notebook only tackles the Ising model. As there is only the nearest neighbor interaction, we can teach the CRBM using $3\\times3$ states, this has as a consequence that we can generate all possible states $2^{3\\times3}= 512$ and learn the energy function from them. So no Metropolis is necessary, and no sampling of the CRBM while training is necessary. Also contrary to the main work, for simplicity, no correction step or parallel tempering is introduced. Note that for the Ising model there exist cluster algorithms that would outperform the CRBM.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6t5AMJk__vdw"
      },
      "source": [
        "# Installing tensorflow"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mqVmIoHumhCe"
      },
      "source": [
        "\"\"\"\n",
        "### install necessary packages if in colab\n",
        "def run_subprocess_command(cmd):\n",
        "    process = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE)\n",
        "    for line in process.stdout:\n",
        "        print(line.decode().strip())\n",
        "\n",
        "\n",
        "import sys, subprocess\n",
        "\n",
        "IN_COLAB = \"google.colab\" in sys.modules\n",
        "colab_requirements = [\"pip install tensorflow-gpu==2.1.0\"]\n",
        "if IN_COLAB:\n",
        "    for i in colab_requirements:\n",
        "        run_subprocess_command(i)\n",
        "\"\"\";"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GzkLSnOF_4EG"
      },
      "source": [
        "# Importing necessary libraries"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jTbdHbRCoJzr"
      },
      "source": [
        "import numpy as np\n",
        "import tensorflow as tf\n",
        "from IPython import display\n",
        "import pandas as pd\n",
        "import time\n",
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xxU5e0nT5vzC"
      },
      "source": [
        "Set seed for reproducibility."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yr4p5UEh5kWk"
      },
      "source": [
        "np.random.seed(0)\n",
        "tf.random.set_seed(0)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "adjL1Ta-4yjG"
      },
      "source": [
        "Accuracy used for training and numerics."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5mYviS_74wT-"
      },
      "source": [
        "floatX = np.float32"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tdJQe7e7PmE_"
      },
      "source": [
        "t0 = time.time()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x6CGNL5T__7Q"
      },
      "source": [
        "# Helping functions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nMdm-89o52rP"
      },
      "source": [
        "Tensorflow has no implementation of the binomial function. With help of random.uniform the beahviour can be replicated."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HuZ3rWnuorPl"
      },
      "source": [
        "def binomial(means):\n",
        "    return (tf.sign(means - tf.random.uniform(tf.shape(means))) + 1 ) / 2"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZRYigXZ33_SS",
        "outputId": "6ba06760-2802-42d1-bcf3-1f9e948422ea",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "binomial(np.array([0.5, 0.5, 0.1, 0.9]))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4,), dtype=float32, numpy=array([1., 1., 0., 1.], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T8rR10bN6WBI"
      },
      "source": [
        "This functions creates periodic padding so that the convolution has periodic boundary conditions. When performing the transposed convolution step the padding is placed at the oposite side."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NWbNnXjOng1d",
        "outputId": "8a73258c-1357-4036-bd79-ee7349171395",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 192
        }
      },
      "source": [
        "def periodic_padding(x, kernel_size=2, deconv=False):\n",
        "    '''\n",
        "    x: shape (batch_size, d1, d2)\n",
        "    return x padded with periodic boundaries. i.e. torus or donut\n",
        "    '''\n",
        "    pad_r = kernel_size // 2\n",
        "    pad_l = kernel_size - pad_r - 1\n",
        "    \n",
        "    d1, d2 = x.shape[1: 3]\n",
        "    \n",
        "    # When deconvolving the pading should reverses\n",
        "    if deconv:\n",
        "        p = pad_r\n",
        "        pad_r = pad_l\n",
        "        pad_l = p   \n",
        "\n",
        "    top_left = x[:, d1 - pad_l:, d2 - pad_l:]\n",
        "    top_center = x[:, d1 - pad_l:, :]\n",
        "    top_right = x[:, d1 - pad_l:, :pad_r]\n",
        "\n",
        "    middle_left = x[:, :, d2 - pad_l:]\n",
        "    middle_center = x\n",
        "    middle_right = x[:, :, :pad_r]\n",
        "\n",
        "    bottom_left = x[:, :pad_r, d2 - pad_l:]\n",
        "    bottom_center = x[:, :pad_r, :]\n",
        "    bottom_right = x[:, :pad_r, :pad_r]\n",
        "    \n",
        "    top = tf.concat([top_left, top_center, top_right], axis=2)\n",
        "    middle = tf.concat([middle_left, middle_center, middle_right], axis=2)\n",
        "    bottom = tf.concat([bottom_left, bottom_center, bottom_right], axis=2)\n",
        "    padded_x = tf.concat([top, middle, bottom], axis=1)\n",
        "    return padded_x\n",
        "\n",
        "a = tf.Variable(np.arange(4 * 4).reshape(1, 4, 4))\n",
        "print(a[0].numpy())\n",
        "print()\n",
        "print(periodic_padding(a)[0].numpy())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[ 0  1  2  3]\n",
            " [ 4  5  6  7]\n",
            " [ 8  9 10 11]\n",
            " [12 13 14 15]]\n",
            "\n",
            "[[ 0  1  2  3  0]\n",
            " [ 4  5  6  7  4]\n",
            " [ 8  9 10 11  8]\n",
            " [12 13 14 15 12]\n",
            " [ 0  1  2  3  0]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9Ba-8XzQve8c"
      },
      "source": [
        "# Dataset Generation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "85mAzBWlt7p7"
      },
      "source": [
        "def to_int(x):\n",
        "    if x == '0' or x == '1': return int(x)\n",
        "    else: return 0\n",
        "\n",
        "#This will generate all posible LxL matrices with {0,1}^(LxL) don't use with more then L=4\n",
        "def bit_string(L):\n",
        "    maxim = 2 ** (L ** 2)\n",
        "    a = np.asarray([[to_int(x) for x in list(('{0:' + str(L ** 2) + 'b}').format(i))] for i in range(maxim)], dtype=floatX)\n",
        "    return a.reshape(2 ** (L ** 2), L, L)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m19Njkt2AJsT"
      },
      "source": [
        "Generate all posible $3\\times3$ states"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F5IEPtsTvUIa",
        "outputId": "04385db9-1682-4b02-a306-f57386aa9fc7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "states_train = bit_string(3)[:, :, :, None]\n",
        "states_train.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(512, 3, 3, 1)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wOS0L2tAATVe"
      },
      "source": [
        "Generate a random test set of lattice size $L=50$ to evaluate if the CRBM matches the Ising model at large lattice sizes."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e4t-Mm4E1ur4",
        "outputId": "8eb61288-4f5a-405a-8f65-5050cf5a1d6b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "L = 50\n",
        "states_test = np.asarray(np.random.binomial(size=(4 * 10 ** 3, L, L, 1), p=0.5, n=1), dtype=floatX)\n",
        "states_test.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(4000, 50, 50, 1)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k24jeiSitDR_"
      },
      "source": [
        "Define the energy of given state for the Ising model: $E(s)=-\\sum_{<ij>}s_is_j$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J1CTzY8LvrPI"
      },
      "source": [
        "def ising_energy(states):\n",
        "    states = 2 * states - 1\n",
        "\n",
        "    nb = np.roll(states, shift=-1, axis=1) + np.roll(states, shift=-1, axis=2)\n",
        "\n",
        "    return -np.sum(states * nb, axis=(1, 2, 3))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "31kyXyzxAteM"
      },
      "source": [
        "For now we will only consider the case $T=1$\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3DwA8L7N2XbB"
      },
      "source": [
        "temp = 1."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qoCi0HL2A16p"
      },
      "source": [
        "The physical energy is computed for train and test set."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6ny3qOL3vl3O",
        "outputId": "6f6c75f9-0647-47f1-8884-9b1139d633da",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "E_phys_train = ising_energy(states_train)\n",
        "E_phys_test = ising_energy(states_test)\n",
        "E_phys_train.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(512,)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aZ0t1lhZBB4c"
      },
      "source": [
        "The CRBM will be trained by batches of 64 states."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eSZRLeNDwxuQ",
        "outputId": "0e15dee5-f1a7-429d-9ab5-38f0bd9393bd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "TRAIN_BUF = E_phys_train.shape[0]\n",
        "BATCH_SIZE = 2 ** 6\n",
        "N_TRAIN_BATCHES = TRAIN_BUF// BATCH_SIZE\n",
        "\n",
        "train_dataset = tf.data.Dataset.from_tensor_slices((states_train, E_phys_train)).shuffle(TRAIN_BUF).batch(BATCH_SIZE)\n",
        "BATCH_SIZE"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9BzB92R0zgqs"
      },
      "source": [
        "# The Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dxSkWptE0hXI"
      },
      "source": [
        "Random Initializion: the kernel W with kernel_dims=(kernel_number, kernel_size) and both biases."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a0H-WBKOx1Lv",
        "outputId": "aad6c414-1074-47b3-a371-2c4e6407d1c6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 70
        }
      },
      "source": [
        "kernel_dims = (2, 2)\n",
        "kernel_shape = (kernel_dims[1], kernel_dims[1], 1, kernel_dims[0])\n",
        "\n",
        "multi = np.prod(kernel_shape)\n",
        "W_np = np.asarray(np.random.randn(*kernel_shape) * np.sqrt(2 / multi), dtype=floatX)\n",
        "\n",
        "W = tf.Variable(W_np * 2)\n",
        "\n",
        "vbias = tf.Variable(np.zeros(1, dtype=floatX))\n",
        "hbias = tf.Variable(np.zeros(kernel_dims[0], dtype=floatX))\n",
        "\n",
        "print(\"vbias.shape:\", vbias.shape)\n",
        "print(\"hbias.shape:\", hbias.shape)\n",
        "print(\"W.shape:\", W.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "vbias.shape: (1,)\n",
            "hbias.shape: (2,)\n",
            "W.shape: (2, 2, 1, 2)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nntZbmOaK4fR"
      },
      "source": [
        "Analytical mapping from the Ising model to the CRBM (see App. C in paper):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vz0-Cl8eK9dl"
      },
      "source": [
        "def get_param(beta):\n",
        "    e4b = np.exp(4 * beta)\n",
        "    e2b = np.exp(2 * beta)\n",
        "\n",
        "    W = np.log(-1 + 2 * e4b + 2 * e2b * np.sqrt(e4b - 1))\n",
        "\n",
        "    Wk = np.zeros((2, 2, 1, 2), dtype=floatX)\n",
        "    Wk[0, :, 0, 0] = W\n",
        "    Wk[:, 0, 0, 1] = W\n",
        "\n",
        "    vbias = np.asarray([-2 * W], dtype=floatX)\n",
        "    hbias = np.array([-W, -W], dtype=floatX)\n",
        "\n",
        "    return Wk, vbias, hbias\n",
        "\n",
        "W, vbias, hbias = [tf.Variable(param) for param in get_param(1 / temp)]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h6CTcpub1Dd8"
      },
      "source": [
        "The negative log likelihood also called free energy: \\\\\n",
        "$F(v) = -v_\\text{bias} \\sum_{i,j} v_{ij} -\\sum_{i,j,k} \\log(1+e^{(v*W^k)_{ij} +h^k_\\text{bias}})$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YYuxQcKzx5cd"
      },
      "source": [
        "def n_log_like(v):\n",
        "    input4D = periodic_padding(v, kernel_dims[1])\n",
        "    kernels4D = W\n",
        "\n",
        "    # Convolution\n",
        "    out = tf.nn.conv2d(input4D, kernels4D, strides=1, padding=\"VALID\")\n",
        "    out += hbias[None, None, None, :]\n",
        "\n",
        "    # Same as softplus(x)=log(1 + exp(x))\n",
        "    hidden_term_not = tf.math.softplus(out)\n",
        "    hidden_term = tf.reduce_sum(hidden_term_not, axis=(1, 2, 3))\n",
        "\n",
        "    visible_term = tf.reduce_sum(v, axis=(1, 2)) * vbias[None, :]\n",
        "    visible_term = tf.reduce_sum(visible_term, axis=1)\n",
        "\n",
        "    return -hidden_term - visible_term"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JXdCNfJ_1dGb"
      },
      "source": [
        "$P(h|x): x \\rightarrow h$  \n",
        "$P(h^k_{ij}=1|v)=\\sigma((W^k * v)_{ij} + h_\\text{bias}^k) ) $"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7EPUk1rvyAI3",
        "outputId": "bb6a5cae-3de5-46b9-d446-74d91dfd284c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "def prop_vis_to_hid(v):\n",
        "    input4D = periodic_padding(v, kernel_dims[1])\n",
        "    kernels4D = W\n",
        "\n",
        "    # Convolution\n",
        "    out = tf.nn.conv2d(input4D, kernels4D, strides=1, padding=\"VALID\")\n",
        "\n",
        "    out += hbias[None, None, None, :]\n",
        "\n",
        "    mean_activation = tf.math.sigmoid(out)\n",
        "    return binomial(mean_activation), mean_activation\n",
        "\n",
        "h, act = prop_vis_to_hid(states_train[100][None])\n",
        "print(\"visible_shape:\", states_train[100][None].shape)\n",
        "print(\"hidden_shape:\", h.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "visible_shape: (1, 3, 3, 1)\n",
            "hidden_shape: (1, 3, 3, 2)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OxXeic0n2Cft"
      },
      "source": [
        "$P(x|h): h \\rightarrow x$  \n",
        "$P(v_{ij}=1|h)=\\sigma((\\sum_k \\bar{W}^k * h^k)_{ij} +v_\\text{bias}))$\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_0s652fiyHR6"
      },
      "source": [
        "def prop_hid_to_vis(h):\n",
        "    input4D = periodic_padding(h, kernel_dims[1], deconv=True)\n",
        "    kernels4D = tf.transpose(W[::-1, ::-1], [0, 1, 3, 2])\n",
        "\n",
        "    # Convolution\n",
        "    out = tf.nn.conv2d(input4D, kernels4D, strides=1, padding=\"VALID\")\n",
        "\n",
        "    out += vbias[None, None, None, :]\n",
        "\n",
        "    mean_activation = tf.math.sigmoid(out)\n",
        "    return binomial(mean_activation), mean_activation"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vKCCahXU2Nj_"
      },
      "source": [
        "$P(x'|x): x \\rightarrow h\\rightarrow x'$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aDEsrO92yN3B"
      },
      "source": [
        "def gibbs(state):\n",
        "    h, h_act = prop_vis_to_hid(state)\n",
        "    new_state, x_act = prop_hid_to_vis(h)\n",
        "    return new_state, x_act, h, h_act"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hxzoSUrpHMrP"
      },
      "source": [
        "Plot the intermidiary states"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cXivV6u7_NEA"
      },
      "source": [
        "def arrow(ax):\n",
        "    ax.arrow(0., 0.5, 0.5, 0, head_width = 0.2, width = 0.05)\n",
        "    ax.axis('off')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Hjw_WiaXyTKP"
      },
      "source": [
        "def plot_sampling(x, x_act, h, h_act):\n",
        "    fig, axes = plt.subplots(nrows=3, ncols=9, figsize=(9 * 2, 3 * 2))\n",
        "    axes[1, 0].imshow(x_old[0, :, :, 0], vmin=0, vmax=1, cmap=\"gray\")\n",
        "    axes[1, 0].set_xlabel(\"x\", fontsize=20)\n",
        "    arrow(axes[1, 1])\n",
        "\n",
        "    # Remove unwanted plots\n",
        "    for ij in [(0,0), (2,0), (0,1), (2,1), (1,2), (1,3), (1, 4), (0,5), (2, 5), (0,6), (2, 6), (0, 7), (2, 7), (0, 8), (2, 8)]:\n",
        "        axes[ij[0], ij[1]].set_visible(False)\n",
        "\n",
        "    for i in range(2):\n",
        "        axes[2*i, 2].imshow(h_act[0, :, :, i], vmin=0, vmax=1, cmap=\"gray\")\n",
        "        axes[2*i, 2].set_xlabel(f'$P(h^{i}|x)$', fontsize=20)\n",
        "        \n",
        "        arrow(axes[2*i, 3])\n",
        "\n",
        "        axes[2*i, 4].imshow(h[0, :, :, i], vmin=0, vmax=1, cmap=\"gray\")\n",
        "        axes[2*i, 4].set_xlabel(f'$h^{i}$', fontsize=20)\n",
        "        \n",
        "        \n",
        "\n",
        "    arrow(axes[1, 5])\n",
        "    axes[1, 6].imshow(x_act[0, :, :, 0], vmin=0, vmax=1, cmap=\"gray\")\n",
        "    axes[1, 6].set_xlabel(\"P(x'|h)\", fontsize=20)\n",
        "\n",
        "    arrow(axes[1, 7])\n",
        "\n",
        "    axes[1, 8].imshow(x[0, :, :, 0], vmin=0, vmax=1, cmap=\"gray\")\n",
        "    axes[1, 8].set_xlabel(\"x'\", fontsize=20)\n",
        "    # Deactivate axes\n",
        "    for j in range(9):\n",
        "        for i in range(3):\n",
        "            axes[i, j].set_xticks([])\n",
        "            axes[i, j].set_yticks([])\n",
        "\n",
        "    plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f6pmhSnzMy4n"
      },
      "source": [
        "x_old = np.asarray(np.random.binomial(n=1, p=0.1, size=(1, 5, 5, 1)), dtype=floatX) * 0\n",
        "x_old[0, 2, 1, 0] = 1\n",
        "x_old[0, 2, 2, 0] = 1\n",
        "x_old[0, 2, 3, 0] = 1"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Lf3nroZRM8qY",
        "outputId": "e7a2e7a2-7937-43b0-d89c-736fc2bacba1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 386
        }
      },
      "source": [
        "#W, vbias, hbias = [tf.Variable(param) for param in get_param(1 / 100)]\n",
        "plot_sampling(*gibbs(x_old))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA/4AAAFxCAYAAADK5XPpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZRnd10n/Pe3qnqpXggSZBkw7ISwhSwgMojsjLghOuMMz6DMgCfjjKL4jPOIKGEZBZUZnUcfdZRAEtagQWHYhOwQIJCQQHe6kxCyEhJCCCGpffs8f9SvoU/TnXRXqutW3Xq9zulzu+/2e1dyblW97/K9raoCAAAA9NNQ1wEAAACAw0fxBwAAgB5T/AEAAKDHFH8AAADoMcUfAAAAekzxBwAAgB4b6ToAcOhaa97D2Z3bquqHuw7B6uF47E5Vta4zAMBaoPjDGtWa33e7UFXXd50BAAAOhVv9AQAAoMcUfwAAAOgxxR8AAAB6TPEHAACAHlP8AQAAoMcUfwAAAOgxxR8AAAB6TPEHAACAHlP8AQAAoMcUfwAAAOgxxR8AAAB6TPEHAACAHlP8AQAAoMcUfwAAAOgxxR8AAAB6TPEHAACAHlP8AQAAoMcUfwAAAOgxxR8AAAB6TPEHAACAHlP8AQAAoMcUfwAAAOgxxR8AAAB6TPEHAACAHlP8AQAAoMcUfwAAAOgxxR8AAAB6TPEHAACAHlP8AQAAoMcUfwAAAOgxxR8AAAB6TPEHAACAHlP8AQAAoMcUfwAAAOgxxR8AAAB6TPEH4JC01h4+vHnbR1obelVr7ciu8wAAcPdGug4AHLoHP/jBOemkk5ZlX69//euXZT/L7U1vetOy7euNb3zjsu2LJMmTh0bv87xNP/KEn5i89tK/HB7dfvHC1Pjbk/pQVX2n63BrWVV1HWG/WmtdRwAA7gXFH4BDNnLED08/4Bdef8TCzGQmv/bFfzm24+xjp274yt8Mj26/aHAS4MNV9d2ucwIAoPgDcC8MbRzN1mOela3HPGvbwvREJr/2hWeN7Tjr+KkbL/+74dHtFy5MjZ2S5P9U1V1dZwUAWK8UfwCWxdCmLdn6+Gdn6+OfvW1hejwTX/38c8d3nP3UqZt2v2N487YLFqbHT0ny0aoa6zorAMB6ovgDsOyGNm3Ntic+L9ue+Lzt81Njmbzqcy8Y23n206e/ceXG4c3bzlmYHn9Hko9V1UTXWQEA+k7xB+CwGt68Ldue/IJse/ILts9P3pmJqz73k+M7znrm9C1XbxjavO2sWjwJ8Imqmuw6KwBAHyn+AB0Z2rztPV1nWIo2svGhaUNLeh3s8Oh9sv3YF2X7sS/aPj/x3Uxc9dmfHvvKp35i5tZrNwxt3vbxmh4/Nck/V9X08qYGAFi/FH+Ajmw95lkv2/jAR3UdY0k2/8gT7vU+hrccke1P+clsf8pPbp8f+04mrrrw58e+ctbzZm67fmRo89aP1fTEqUk+VVUz9/rDAADWMcUfoCOjjzguWx77jK5jrArD234o24//6Ww//qfvM3fXtzNx5YW/OLbjrBfOffvGkaFNWz9cMxP/u6rO6zonAMBapPgDsKqMbD8y20/4mWw+6on3ufOiD86P7zrv37aNow9Mcl7X2QAA1iLFH4BVoaoye9v1Gd99wfz4zrMnF6bGJ1P17iTvrpnJS7vOBwCwVin+AB359if/auI7575ztuscS7HtuJ/cdMTTXrp5OfY1e9uNGb/igvmxHWdNLkzeNZ3Ue2t2+l1JLq6qWo7PAABYzxR/gI4sjN/xzIXc0XWMpXj21DUXn3xviv/s7TdlfPcFC+M7zhqfn7hjLmnvr9mpdyW5qKoWljErAMC6p/gDdKSq1uTt6621H1nKdrPfuTkTV3x6YWzHWePzd317IW3ojJqdfFeSzyr7AACHj+IPwGEze8ctmbjiMzW246yx+TtvTYaG/75mJk9P8pmqmu86HwDAeqD4A7Cs5u689Xtlf+6OW1qGhj9YM5OnJbmgqua6zgcAsN4o/gDca3N33ban7N81951vDGdo5EM1M3FqkvOqak0OYAgA0BeKPwBLMjd2eyauvDDjOz5158xtN460kQ0fqemJU5OcVTWt7AMArBKKPwCHamHqxl3bbvqbV062kY0fq+nxU5N8amFuZrrrYAAA/KDmFcmw9rTWqrXWdYx1qaouqaoTu87RpdbacJJnZvHVe1Nd5+laa80P0o5UlW+EAHAQXPEH4JAMRuM/v+scAAAcnKGuAwAAAACHj+IPAAAAPab4AwAAQI8p/gAAANBjij8AAAD0mOIPAAAAPab4AwAAQI8p/gAAANBjij8AAAD0mOIPAAAAPab4AwAAQI8p/gAAANBjij8AAAD0mOIPAAAAPab4AwAAQI8p/gAAANBjij8AAAD0mOIPAAAAPab4AwAAQI8p/gAAANBjij8AAAD0mOIPAAAAPab4AwAAQI8p/gAAANBjij8AAAD0mOIPAAAAPab4AwAAQI8p/gAAANBjij8AAAD0mOIPAAAAPab4AwAAQI8p/gAAANBjij8AAAD0mOIPAAAAPab4AwAAQI8p/gAAANBjI10HAJbktqq6vusQ69TDug7AqnNbEsfjynMsAsBBalXVdQYAAADgMHGrPwAAAPSY4g8AAAA9pvgDAABAjyn+AMD3tNb+c2vt2tbaVGvtktbaj3edCQC4dxR/YMUpFrA6tdZ+Kcn/SvJHSY5L8tkkH2+tHdVpMADgXlH8gRWlWMDKaa29prVWrbWXHeQmv53k1Kr6u6raXVW/keTmJL92+FICAIeb4g8clNbaaa21W1trW/eZf9iKRWvtDYN9P/xeZj9hsJ9X3Zv9wBp0wmD6pXtasbW2cbD+J/dZ9Mkkz1jmXADAClL8YR1qrX1yUIT3/nN7a+2LrbVXttbaPus/NcnLk7y1qsb32d2qLxZVdUmSf0ry5tbatsP1ObAKHZ9kLMlVB7Hu/ZMMJ/nmPvO/meRBy5wLAFhBI10HADpxQpKFJP89SWXxJOCjk/xikrcneViS1++1/h8muTPJX+9nX8tVLJ5/8PGX5C1JLkry6iw+ZgC9Nrg75+gkn6uqha7zAADdccUf1pnW2iOT3C/JFVV1clW9oapeX1UvS/KKwWon7bX+Y7NYyj9QVZP77GtPsfjyai8WVfWFJFckOam15nsf68FTsvhz/kuttce31t7dWrultTbWWvtsa+1H91n/tiTzSR64z/wHJrllBfICAIeJX35h/TlxMP3CfpadP5geude8/5ikJTljP+t3Viz2elzhF/aZ31prpw6WvXWfzd6f5KgkLziUz4I16vjB9GFJLk5yRJLTs3ic/1iSD7fWtu9ZuapmklySHzw+XpDFQTgBgDVK8Yf1Z0/xv2g/y44eTK/fa97zs1jWP7+f9bssFr+TxccV3txaG95r/tuS/EqSv62q391nmwv3+jzouz3jbzw9yTOr6meq6r9V1U8l+WCSB2Tx5N3e/meSV7TWXtVaO6a19r+S/Iskf7NiqQGAZecZf1h/9nvFv7V23yR/PPjn6YN5W7NYDHbvZ1C/5AeLxfcG+GutnZnkpYPtP73XNv8zybtaa1/IYhH/T1lCsaiqL7fW3pXFkv/yJKe21n4vi28N+ED2//qxLw6mzzqUz4I1as+JuVfsfWwO7B5MN+89s6rOaK0dmeT3kzw4yc4kL66q6wMArFmKP6wjg9H695SBl7bWfjaL3weOSvJTWXz2/6NJ9twi/5AsDsZ38wF22XWx+IMkv5Tk5MFo/X+Y5J+TvHx/Yw5U1Xdba1NZ/Hqht1prm5Mck+Saqvr4flZ55GD6tX0XVNVfJfmrwxgPAFhhij+sL4/J4u34SfK6wXQ2ybezeCv/u5KcUVU1WLbnWf/v7Luj1VAsqurG1tqfJ/ndJH+RxccFXjp4pOBAbs8PjjEAfXNsFn/G7/vqzD2OT/LdJNeuWCIAoDOKP6wve27z//Oqes1BrL9nFP/N+1m2WorFt/b6+yurauIe1h/N978u6Ks9d+Ncsu+Cwbgbj01y/l4n+QCAHjO4H6wve4r/pQe5/q2D6ZH7WXYwxeLSw1ksWmsvy+JgfnveCPCb97D+UJL75vtfF/TVnvE3fuD4THJcFt/Usb9lAEAPKf6wvuwp/pcd5Po3Z/GK+tH7WdZpsWitvTjJqVkcI+DJSa5M8qrW2v6y7nH0INfBfv2wVh2fZDqLx8e+9hy7+47LAQD01CHd6t9ac0tgR6qqdZ2BtW1wtfu4LJaBXQezTVVVa+2CJL/QWnt0VV291+LOikVr7ZlJ/iHJ15O8qKq+1Vr7/SR/n8U3E7zkAJs+fTA993DkgtWgtbYxyROTfLmqZvezyt2dtIM1y++pnbqtqn646xCsDo7FTh3wWHTFH9aPxyXZlmRnVc0dwnZnDqYv2jNjr2KxY6WLRWvtKUk+ksXxA15QVTcnSVX9Q5KLk/xca+3HD7D5C5PMJ/nQcueCVeRJSTbkwCfeTkgyluSqFUsE9J1XfsLqcMBjUfGH9eNQb/Pf48wsPhP/y3vN66RYtNYeneQTSSqLV/r3fWPAawfTP93Ptkdk8U6Aj1TVjcuZC1aTqrqkqlpVnXSA5cdU1XYD+wHA+mFUf1gnqur0JKcvYbuZwSvz/qi1dlxVXVpVl2TxWfkDbXPMvYh6d1muTvKgu1l+1t3k+uUsvp3gbYchGgAArFqu+AMH48+S3JDkTV0HWYrW2mgW7wY4s6o+03UeAABYSYo/cI+qairJy5Nc3Frb2nWeJXh4kr9N8l87zgEAACvOrf7AQamqC5Jc0HWOpaiq3Une0HUOAADoguIPrGbnDaZ3dBkCAADWMsUfWLWq6rx8v/wDAABL4Bl/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpspOsAAADAPWutLdu+qmrZ9gWsfq74AwAAQI8p/gAAANBjij8AAAD0mOIPAAAAPab4AwAAQI8p/gAAANBjij8AAAD0mOIPAAAAPab4AwAAQI8p/gAAANBjij8AAAD0mOIPAAAAPab4AwAAQI8p/gAAANBjij8AAAD0mOIPAAAAPab4AwAAQI8p/qwZw6Pb/20b2fD7rbVHdZ0FAFaDoc3b/msbGvovrbUHdp0F1rPW2pOGNo7+WWvtR1trres8sK9WVQe/cmsHv/I9OJTPXauW85ivqnX/DWR4yxGf23D/o06c+eY1c621Gxdmp96RhfkzqurarrMBQBeGNo5+Z9NDjtk0/fXLh9rIxh0LU+NvT+qDVfWtrrOttOX8PXU5f4c7+eSTl21fy+kNb3jDcu7ukqo6cTl3uNa01n5/wwMe8caFybsmFqbGplL17pqbfk8W/9v0v/jsZTmPxeW0nP8bVvG5nQMeiyMrnQTujSOe/q9HNj/8KSNTN+58zPjl5548ceWFJw9v3nbdXicBbug6IwCspCNf/JujQ5u3Z+qaS04cu/ycx09dc8n/Ozy6/UsLU+OnJPWPVfXtrjPCejD6qKe2+/74y7fNfuu6beO7z//18Z3n/OrCzOTY0IZN7665mfckuWy9nQRg9VD8WXPa0HBGH3ZsRh927OZ60a9n6oYdjxu//Nw3TVz12TcNj27/2sL0xCmphQ9U1U1dZwWAlTC0YVO2HP2MbDn6GVsWZqcy+bWLnz6+8+wnTV532V8Oj26/eHAnwIeq6jtdZ4U+a61l4wMekY0PeMTIfZ/1KyOzt16zdXzX+b85tvOc/1Rz03cObdh0es3NvDfJDicBWEmKP2taGx7J6COOy+gjjttc86/O1HWXPWF817l/OHHV598yPLr9yoXp8VNS9fdVdXPXWQFgJQxt2Jytj3tmtj7umVsXZiYzefUX/uXYznOOnbrhK38zPLr9osFJgA9X1Xe7zgp91lrLxgc+Khsf+KiR+z77P4zM3HL11vFd5//2+OXn/EbNz93eRjaelvnZ91bVrq6z0n+KP73Rhkcy+qgTM/qoE0drbjaT11365PHLz33L5NUX/cnw6PZdC9Pjb0/VmVX1za6zAsBKGNo4mq2P/4lsffxPbFuYnsjE1Rc9a3zn2cdP3Xj53w2Pbr9wYWrslCT/p6ru6jor9FlrLZse/JhsevBjNvzQc1+5YeYbV24Z333+74xfft5rhjZtva3mZ9+Z+dn3VdWVXWelnxR/eqmNbMiWRz8tWx79tC01N5PJa7903PjOc/508pqL/+fw6PYdC1PjfzcY/Oi2rrMCwEoY2rQl257wnGx7wnO2LUyNZeLqi547vuPsp07dtPsdw5u3nb8wPf6OJB+pqvGus0Kftday6SGPy6aHPG7jDz3vVzdO33TFUeO7zn/txK7z/tvQ5q231OzMO7Mw976qurrrrPSH4r/OtNZ+LckPd51jKdrIxocucbtseczTs+UxT9+yMDu9Z/CjYyavueQvBoMfvX0w+NHty50ZgNWvtfa7STZ2nWNJhoZHl7TZ5m3Z9sTnZdsTn7d9fmosk1d97oVjO8/+selvXLlxePO2cwYnAT5WVRPLnBjuVmvt9V1nWKKfWMpGrQ1l80Mfn80PffzG+73gpEx/fdcjxned97rx3Re8dnjztpv2GsD6muUOzPridX6H0Wp8nV8b2TB7xI/+4to84TM8kvuc+JIMbdy8LLtbmJnK5DUXZ3zn2eOT1102MrRh0xcHIyD/U1XdsSwfAsCqN7L9yKltT37hpq5zLMXQ6PZsP+Fn0trQsuxvfvLOTFz1uYzvOOuu6Vuu3tBGNn6qFk8CfKKqppblQw4Tr/M7NKv1dX73Of6na2h0+3LsasVtfeJzs+GH/sWy7KsW5jN94+UZ33Xe1PgVn05rQ9cPTgK8f7W/xcrr/Dp1wGNR8T+MVmPxH9q0deKo13xgSVcH+mww+FHGdp49NnXDjg1DGzZ9fq+TAJ57BOixTQ855o4Hv/xtR3SdY7WZH78jE1d9NmM7zrpr5tZrR9rIxk/U9Pg7s3gSYLbrfPtS/A/Nai3+Dznp72q5ynNf1MJ8pm7YkfHLz52auPLCtOGRawZvsXp/VX2j63z7Uvw7dcBjcXlOD8Mat2fwowf86zdue8AvnrxpaNPWZyV1eoY3/FzX2QCgC8Nb75vtx704D/r3f7r9yH/1G6NJXpLkwxkaflzX2WA9aUPDGX34U3L/n3rN5gf/yp9v3vCARxyTWvgfbcPmN3WdjbVjbd7yDcuoqjLzza9lYvf5s2M7z5mpuZk7Mz93WpL3Zn52Z9f5AGCl1cJ8pm7cmfFd501NXPGZpA1dV7NT70xyRs3PXd91PlhP5r57a8av+HSN7fjUXfPfvbVlaPjMJKfX7NQFXWdj7VD815mandrwzff/3pp8b28b3jh8v5989baRbfe71/uqqszeem3Gd58/N77znOmFmcmx1MLpNTfz3iRfrvXwLAoASZLZ267fvFZ/Ng5t+aGR+7/4t7a2kQ33el+1MJ/pr+/K+K7zp8evuGAhaV+v2al3Dp4pvnYZ4sI9uu0j/+OuoQ2bFrrOsRRHPPP/OmLzQ5+wLPuau/NbmbjiMzW246yxue/cPJThkX+smYnTkpxXVXPL8iGsK4r/elMLz566/itHdh1jKdqG0T+ZvfXao5da/Ksqs7ddn/HdF8yP7zh7cmF6fDJV76656fck+ZKyD7A+1czkS6au/8raHNV/eMP75p/zHzKy/f5L2rxqIdM3XZHxXefNTOw6f76Sm/cq+14lxoqb+caV/77rDEv0Sxv/xeP+3eaHPmHJD3/P3fXtTFx5YcZ2fOrOuW9/fSTDGz40KPvn1Nz0qhtXg7VF8V9nqurCrjMs1fCWI167lO1mb7sx41dcMD+246zJhcm7ppN6b81OvzvJF5V9AKrqE11nWKqhjaMzSbYcyjZVlZlvXJnxXefNjO86b65q4Vs1N3tq5mffV1VXHqaocFCq6sNdZ1iK1tqTl7Ld/Nh3MnHVhRn7yqfunLnthpE2suGjNT1xapKzam5mZnlTsp4p/vTS7O03ZXz3BQvjO84an5+4Yy5p76/ZqXcluaiq1uTtYwCwVFWVmVu+mvFd58+OX37ubM3P3V7zs6dlfva9VbWr63ywnnz/jRmfumvm1utG2sjGj9f0+KlJPrkwNzPddT76SfGnN2a/c3Mmrvj0wtiOs8bn7/r2QtrQGTU7+a4kn1X2AVhvvj947QWDwWun78z83Gk1P/veJDvd9QYrZ37iu5m46nMZ33nWndO3fG1jG9n4qZoef0eSTyzMzUx1nY/+U/xZ0+a++82M7/50je046675O29tGRr++5qZPD3JZ6pqvut8ALCSqiqz37o247svmBvfec7UwszkeBbm31VzM++JwWthRc1PjWXyqs9mbMfZd03ffNXGoZGN5yxMj5+S5OMLczMTXedjfVH8WXMWRzldLPtzd9zSMjT8wUHZv8AopwCsRzPfuj4Tuy+YH9t59uTC1JjBa6EjC9Pjmfjq5zO246y7pm+6YuPQhk3nL0yNnZLko/NzM+Nd52P9UvxZQyrf/tj/ml6YHpvL0Mg/DUY5PVfZB2DdqsrNp71momYmJw1eC90a+/I/z9/5hX+cH9qw6cKFqbG3J/nI/NzMXV3ngiRph/JzobXmh0hHqmrJrwbpi9basUkenOTsqvJKEwDWvdbac5NMxuC1fk/t1iVVdWLXIbrUWntAkucl+VhVfbfrPF1yLHbqgMei4r9GKP4AAAfm99ROrfviz/c5Fjt1wGNxaKWTAAAAACtH8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6LGRQ1z/tiTXH44g3K2HdR0AAGCV83tqd/yuyt4ci9054LHYqmolgwAAAAAryK3+AAAA0GOKPwAAAPSY4g8AAAA9pvgDAMC90Fo7rbV2a2tt62HY9ytaa9Vae/YB5r/iEPb14dba11prG5c7J7C6Kf4AAJBkUKT3/jPfWruttXZOa+1lB9jmqUlenuStVTV+Lz57vwV/mb0+ySOSvPowfgb0Smvt2Yd6km01OtTX+QEAQN+9cTDdkORxSX4uyXNaaydW1W/vs+4fJrkzyV+vYL4lqarLWmufSPK61tpfVdVE15mAleGKPwAA7KWq3jD487qq+oUkL0pSSX6rtfbwPeu11h6b5PlJPlBVk52EPXSnJblvkv3ewQD0U2+Lf2vtnwa3ZPzArUyttTcPlp3SRTYAANaOqjo7yRVJWpKn7rXoPw7mnbH3+q21ja21Lw5+3/zZfffXWjt9sOwPliNfa+05rbXzWmt3tdbubK19tLV2zAFW/1CSqSSvXI7PhrVoPXbF3hb/LH4jviHJn7TWjtszs7X2vCS/l2RXkt/oKBsAAGtLG0xrr3nPTzKf5PN7r1hVM0l+KYuPALyztfYj39tJa/8hi2MCnJ3FxwTurZ9O8snBZ/1Nkk8neXGS81tr99935aqaSnJJkqe21o5Yhs+HtWjddcXeFv+quj3Jv0synOSM1tq21toDk7w7yXSSf+O5JgAA7klr7flJjs5i6f/iYN7WJE9Jsnt/g/pV1TVJfjXJ/ZK8t7U2PLgK/5dJbk3y76tqYa/1T62qVlXnHWK8lyT5V1X1s1X1O1X1U0nemuSHs1hu9ueLWfwd+V8e4mdBLxxKV6yq8wbH5qmdBV4GvR7cr6o+O7iF6i1J/ncWvwE+KMmvVtXlnYYDAGBVaq29YfDXDVks/C/J4hX/P6uq6wfLHpLF0nDzgfZTVR9orT03yUlJ/jiLYwWMJvn5qrplmeK+f/Aowt7+NsnvJnnaAbbZ89lHLVMGWHPWW1fsdfEf+OMkz8n3BzB5X1W9vcM8AACsbicPppXkjizePn9KVb17r3WOHEy/cw/7+q0kz0jyfw/+/Zaq+uRyBU1y8X7m3TiY/tABtrl9MP2BRwFgnVk3XbG3t/rvUVWV5IN7zfrzrrIAALD6DW7rbVU1VFX3q6rn7FP6k2TPKP6b72FfU0k+OvjnXJL/b5nj3rGfz5wb/HX4ANuMDqZr5U0EcFisp67Y++LfWntMkrdl8WzsQpK3t9bu9hs0AADcg1sH0yPvbqXW2jOT/E6S27J4t+07Wmvt7rZZAXsy33q3a0HPraeu2Ovi31rblMXXq2zN4siqb0nypPT4TA4AACvi5iTfyuIYAPvVWjsyyfuSzCZ5bpL3JHlhkv9nJQLejccNppd1mgI6tN66Yq+LfxbP3hyX5E+q6lNZfF7rwiQntdb+dafJAABYswa3CF+Q5P6ttUcfYLV3JnloktdU1Y4kv5bk6iRvbq09Y2WS7tfTs3gHws4OM0DX1lVX7G3xb639fJJfT3JRkt9Pkqqaz+JrG27P4m0cj+wuIQAAa9yZg+mL9l3QWvutJD+T5Myq+pskqaq7snhlcSHJ+1prBxp877BprR2dxdH8Pzg4eQHrznrsir0s/q21o5KckuS7Sf7tXgOcpKpuzOI7Te+T5P2ttY3dpAQAYI07M4vPyf/y3jNbaydkcbTw65O8au9lVfWlLD7zf1QW7whYab8ymP51B58NnVuvXbE50QdrT2vNgdud26rqh7sOwerheOxOVXU9QBqktfbaJH+U5PiquvQw7P8VWTxB8JyqOu9e7mtTkmuS7K6q59/7dMBaMdJ1AGBpuh8QeH2qquu7zgDAqvJnSf5Tkjdl8db+1ezXkjwoyYu7DgKsrF7e6g8AACuhqqaSvDzJxa21rV3nuQfTSV5ZVV/uOgiwslzxBwCAe6GqLsjiCP+rWlV5rh/WKVf8AQBg9bosyRuTXNdxDmANc8UfAABWqaq6LIvlH2DJXPEHAACAHlP8AV+h+UAAAAxsSURBVAAAoMcUfwAAAOgxxR8AAAB6TPEHAACAHlP8AQAAoMcUfwAAAOgxxR8AAAB6TPEHAACAHlP8AQAAoMcUfwAAAOgxxR8AAAB6TPEHAACAHhvpOgDQrZNPPrnrCIfdG9/4xq4jwJpWVcu2r9basu0LADg4rvgDAABAjyn+AAAA0GOKPwAAAPSY4g8AAAA9pvgDAABAjyn+AAAA0GOKPwAAAPSY4g8AAAA9pvgDAABAjyn+AAAA0GOKPwAAAPSY4g8AAAA9pvgDAABAjyn+AAAA0GOKPwAAAPSY4g8AAAA9pvgDAABAjyn+AByS1toRbWjoN1trx3SdBQCAe6b4A3CofmJo07Y/Gdq87eKhzVuvbcMbTm6tPbbrUBw+rbVl+wMArDzFH4BDtvGBj5h86Kvfu+UBv3jyw7cd+8LXtk1bLxvavO3qNjzyutbao7rOBwDA9410HQCAtam1oWx+6BOy+aFP2HS/55+U6a/vetT4rvP+YHz3p183tHnb12t26pQszJ9RVdd1nRUAYD1T/AG419rQcDYf9aRsPupJm+73wv+cqRt3Pmb88nPfMHHFhW8Y3rztuoWZybenFj5QVTd2nRUAYL1R/AFYVm1oOKMPOzajDzt2c73o1zN1w47HjV9+7n+fuOqz/314dPvVC9MTp6QW/r6qbuo6KwDAeqD4A3DYtOGRjD7iuIw+4rjNNf/qTF132RPHdp37R5NXff6tw6Pbr1iYHj8lVX9fVbd0nRUAoK8Uf4COtNZe0XWGJTpuKRu14ZGMPurEjD7qxNGam83kdZceO375uW+dvPqiPx0e3X75wtT425M6s6puXe7AAADrmeIP0JFNDznmr4e333++6xxLsfWYH990b7ZvIxuy5dFPy5ZHP21Lzc1k8tovHT++85y3TV5z8Z8Nj27/ysLU+N8l9Y9VddtyZQYAWK8Uf4CO3OdpP795y2Of0XWMzrWRjdnymKdny2OevmVhdjpT11zy1LGdZz9+6rpL/2J4dPuXBncC/FNV3d51VgCAtUjxB2DVGNqwKVuOfka2HP2MrQtTY7n97Lf/2PjOs36sjWz8jSzxEQMAgPVO8Qdg1aj5uUxd/+WMX37u5MRXP9fa8IavprW319zMP3SdDQBgrVL8ATpy15c+Njt5zZfW5DP+Wx77YxtHH3nC0HLsqxbmM3XDjoxffu7UxJUXpg2NXLMwM3FKauEDCzNTX1+OzwAAWM8Uf4COTF1/2W/n+q5TLMmT5+74xr8ZfeQJRyx1B7Uwn+kbL8/YrnOnJq74TFobun5hduodWZg/Y6Fqbf5XAQBYpRR/gI5U1V92nWEpWms/m+TfHOp2tTCf6Zt2Z3zXedPjuy9YSNrXa3bqnVmYf/9C1bWHISoAAFH8ATiMqhYyfdMVGd91/szErvPmK7m5ZqdPzcLc+6vqq13nAwBYDxR/AJZVVWXmG1dmfPf5M+OXnzdXNf+tmps9NfOz76+qK7rOBwCw3ij+ANxrVZWZW76a8V0XzI5ffs5szc/dXvOzp2V+9n1VdXnX+QAA1jPFH4AlqarMfPNrmdh9wdzYznOma276zizMn15zM+9JsrOqquuMAAAo/gAswcy3rh+96a9+ZXxhZnIsC/PvrrmZdyf5srIPALD6NL+jwdrTWqvWWtcx1qWquqSqTuw6R5daa/fL0Mh/ycLcx5Ncst7LfmttXX/9Xaoq3wgB4CAo/rAGKf7dUfzZl+LfHcUfAA7OUNcBAAAAgMNH8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6DHFHwAAAHpM8QcAAIAeU/wBAACgxxR/AAAA6LGRrgMAS3JbVV3fdYh16mFdB2DVuS2J43HlORYB4CC1quo6AwAAAHCYuNUfAAAAekzxBwAAgB5T/AEAAKDHFH8AIEnSWntWa+3DrbWbWmvVWntF15kAgHtP8QcA9tiWZGeS30wy2XEWAGCZKP7AinJFEVZOa+01g+PsZQezflV9rKp+r6r+IcnCYY4HAKwQxR9Yaa4owso5YTD9UqcpAIBOKf7AQWmtndZau7W1tnWf+YftimJr7Q2DfT98ycEX93PCYD+vujf7gTXo+CRjSa7qOggA0B3FH9ah1tonB0V47z+3t9a+2Fp7ZWut7bP+U5O8PMlbq2p8n92t+iuKVXVJkn9K8ubW2rau88BKGJykOzrJl6vKbfsAsI4p/rA+nZDFq+1vSvLGJG9O8okkxyZ5+2De3v4wyZ1J/no/+1orVxTfkuRBSV7ddRBYIU/J4s/5L7XWHt9ae3dr7ZbW2lhr7bOttR/tOiAAsDIUf1hnWmuPTHK/JFdU1clV9Yaqen1VvSzJKwarnbTX+o9N8vwkH6iqyX32tWauKFbVF5JckeSk1prvfawHxw+mD0tycZIjkpye5PwkP5bkw6217R1lAwBWkF9+Yf05cTD9wn6WnT+YHrnXvP+YpCU5Yz/rd3ZFca/HFX5hn/mttXbqYNlb99ns/UmOSvKCw5ULVpE9j+E8Pckzq+pnquq/VdVPJflgkgdk8Rj+ntbattbaU1pre47towb/PmpFkwMAy0rxh/VnT/G/aD/Ljh5Mr99r3vOTzCf5/H7W7/KK4u9k8XGFN7fWhvea/7Ykv5Lkb6vqd/fZ5sLBVPFnPdhzfL6iqvYdg2P3YLp5n/knJrl08Gc0i4/9XJrFx4IAgDVqpOsAwIrb7xX/1tp9k/zx4J+nD+ZtzeIVwd37GdQv+cErit8rF621M5O8dLD9p/eavy3Jowf//N4VxSS3V9UNB/tFVNWXW2vvymLJf3mSU1trv5fkt5N8IMmv7WezLw6mzzrYz4G1qLW2OckxSa6pqo/vZ5VHDqZf23tmVZ2XxTt8AIAeUfxhHRmM1r/nKuBLW2s/m8XvA0cl+aksPvv/0SR7bpF/SJLhJDcfYJdLvaJ47l7/fuPgz2n5/hgDB+sPkvxSkpMHJxT+MMk/J3n5/sYcqKrvttamsvj1Qp8dm8Vj+5MHWH58ku8muXbFEgEAnVH8YX15TBZvx0+S1w2ms0m+ncVb+d+V5IyqqsGyPc/6f2ffHa2GK4pVdWNr7c+T/G6Sv0jy2SQvraqZu9ns9iQPXI7Ph1Vsz0m5S/ZdMHj85rFJzt/rWAcAekzxh/Vlz23+f15VrzmI9feM4r/vVftk9VxR/NZef39lVU3cw/qj+f7XBX215zGcHyj+SY7L4sm3/S0DAHrI4H6wvuwp/pce5Pq3DqZH7mfZwVxRvPRwXlFsrb0si4P53TKY9Zv3sP5Qkvvm+18X9NXxSaaT7NzPsj0nBfZ9PAcA6CnFH9aXPcX/soNc/+YsXlE/ej/LOr2i2Fp7cZJTs1hsnpzkyiSvaq3tL+seRw9yHezXD2tOa21jkicm2VFVs/tZ5e6OXQCghxR/WCcGV7uPy+JVwF0Hs83gav0FSe7fWnv0Pos7u6LYWntmkn9I8vUkL6qqbyX5/Sw+evDHd7Pp0wfTc+9mHVjrnpRkQw58/J2QZCzJVSuWCADolOIP68fjkmxLsrOq5g5huzMH0xftmdHlFcXBq/8+ksXxA15QVTcnSVX9Q5KLk/xca+3HD7D5C5PMJ/nQcueC1aKqLqmqVlUnHWD5MVW13cB+ALB+KP6wfhzqbf57nJnFZ+J/ea95nVxRHNx18IkklcUr/V/bZ5XXDqZ/up9tj0jykiQfqaoblzMXAACsZkb1h3Wiqk5PcvoStpsZvDLvj1prx1XVpVV1Se7mlXxVdcy9iHp3Wa5O8qC7WX7W3eT65Sy+neBthyEaAACsWq74Awfjz5LckORNXQdZitbaaBbvBjizqj7TdR4AAFhJij9wj6pqKsnLk1zcWtvadZ4leHiSv03yXzvOAQAAK86t/sBBqaoLsjjC/5pTVbuTvKHrHAAA0AXFH1jNzhtM7+gyBAAArGXN23wAAACgvzzjDwAAAD2m+AMAAECPKf4AAADQY4o/AAAA9JjiDwAAAD2m+AMAAECPKf4AAADQY/8/iUrhFcPT0oMAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 1296x432 with 27 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dvZzNjW_2W2_"
      },
      "source": [
        "Computing the loss function:  \n",
        "$\\text{diff}(x) = E(x) - F(x)$  \n",
        "$C = \\frac{1}{M} \\sum_x \\text{diff}(x)$  \n",
        "$\\text{loss} = \\sum_x (\\text{diff}(x) - C)^2$  \n",
        "If the parameters where initialized with the analytical solution the loss will only differ slightly from zero because of numerical errors"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "expppT3Qyc48",
        "outputId": "0a225edd-932f-412a-d33b-f69802d557bd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "@tf.function\n",
        "def compute_loss(x, nll_phys):\n",
        "    nll_crbm = n_log_like(x)\n",
        "\n",
        "    diff = nll_phys - nll_crbm\n",
        "    C = tf.reduce_mean(diff)\n",
        "\n",
        "    loss = tf.reduce_mean((diff - C) ** 2)\n",
        "    return loss\n",
        "\n",
        "print(\"Train_loss:\", compute_loss(states_train, E_phys_train / temp).numpy())\n",
        "print(\"Test_loss:\", compute_loss(states_test, E_phys_test / temp).numpy())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train_loss: 5.9401373e-12\n",
            "Test_loss: 6.770492e-07\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1tazcvUJ3b4D"
      },
      "source": [
        "Initialize the adam optimizer, compute the gradients and apply them."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5ctvr0eLymUe",
        "outputId": "42cc66cf-e7fd-4610-c057-7b04d0e74198",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "opt = tf.keras.optimizers.Adam()\n",
        "params = [W, hbias, vbias]\n",
        "def compute_grad(x, nll_phys):\n",
        "    ### pass through network\n",
        "    with tf.GradientTape() as tape:\n",
        "        loss = compute_loss(x, nll_phys)\n",
        "\n",
        "    # compute the grad\n",
        "    grad = tape.gradient(loss, params)\n",
        "    return grad, loss\n",
        "\n",
        "@tf.function\n",
        "def train(x, nll_phys):\n",
        "    grad, loss = compute_grad(x, nll_phys)\n",
        "\n",
        "    opt.apply_gradients(zip(grad, params))\n",
        "    return loss\n",
        "\n",
        "train(states_train, E_phys_train / temp)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=5.9401373e-12>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-2mNJF1ixnN4"
      },
      "source": [
        "Everything combined in one class."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Hu2rPLnOmrjh"
      },
      "source": [
        "class CRBM:\n",
        "    def __init__(self, kernel_dims=(2, 2), opt=None):\n",
        "        \n",
        "        assert len(kernel_dims) == 2\n",
        "        if opt is None:\n",
        "            opt = tf.keras.optimizers.Adam()\n",
        "            \n",
        "        self.opt = opt\n",
        "        \n",
        "        self.kernel_shape = (kernel_dims[1], kernel_dims[1], 1, kernel_dims[0])\n",
        "        self.kernel_dims = kernel_dims\n",
        "\n",
        "\n",
        "        multi = np.prod(self.kernel_shape)\n",
        "        W_np = np.asarray(np.random.randn(*self.kernel_shape) * np.sqrt(2 / multi), dtype=floatX)\n",
        "\n",
        "        W = tf.Variable(W_np)\n",
        "\n",
        "        vbias = tf.Variable(np.zeros(1, dtype=floatX))\n",
        "        hbias = tf.Variable(np.zeros(kernel_dims[0], dtype=floatX))\n",
        "\n",
        "        self.W = W\n",
        "        self.hbias = hbias\n",
        "        self.vbias = vbias\n",
        "\n",
        "\n",
        "        # Learning parameters\n",
        "        self.params = [self.W, self.vbias, self.hbias]\n",
        "\n",
        "\n",
        "    def n_log_like(self, v):\n",
        "        input4D = periodic_padding(v, self.kernel_dims[1])\n",
        "        kernels4D = self.W\n",
        "\n",
        "        # Convolution\n",
        "        out = tf.nn.conv2d(input4D, kernels4D, strides=1, padding=\"VALID\")\n",
        "        out += self.hbias[None, None, None, :]\n",
        "\n",
        "        # Same as softplus(x)=log(1 + exp(x))\n",
        "        hidden_term_not = tf.math.softplus(out)\n",
        "        hidden_term = tf.reduce_sum(hidden_term_not, axis=(1, 2, 3))\n",
        "\n",
        "        visible_term = tf.reduce_sum(v, axis=(1, 2)) * self.vbias[None, :]\n",
        "        visible_term = tf.reduce_sum(visible_term, axis=1)\n",
        "\n",
        "        return -hidden_term - visible_term\n",
        "\n",
        "    def set_analytical_param(self, beta):\n",
        "        W, vbias, hbias = get_param(beta)\n",
        "        self.W.assign(W)\n",
        "        self.hbias.assign(hbias)\n",
        "        self.vbias.assign(vbias)\n",
        "\n",
        "    def prop_vis_to_hid(self, v):\n",
        "        input4D = periodic_padding(v, self.kernel_dims[1])\n",
        "        kernels4D = self.W\n",
        "        \n",
        "        # Convolution\n",
        "        out = tf.nn.conv2d(input4D, kernels4D, strides=1, padding=\"VALID\")\n",
        "        \n",
        "        out += self.hbias[None, None, None, :]\n",
        "        \n",
        "        mean_activation = tf.math.sigmoid(out)\n",
        "        return binomial(mean_activation)\n",
        "        \n",
        "    \n",
        "    def prop_hid_to_vis(self, h):\n",
        "        input4D = periodic_padding(h, self.kernel_dims[1], deconv=True)\n",
        "        kernels4D = tf.transpose(self.W[::-1, ::-1], [0, 1, 3, 2])\n",
        "\n",
        "        # Convolution\n",
        "        out = tf.nn.conv2d(input4D, kernels4D, strides=1, padding=\"VALID\")\n",
        "        \n",
        "        out += self.vbias[None, None, None, :]\n",
        "        \n",
        "        mean_activation = tf.math.sigmoid(out)\n",
        "        return binomial(mean_activation)\n",
        "    \n",
        "    @tf.function\n",
        "    def gibbs(self, state):\n",
        "        return self.prop_hid_to_vis(self.prop_vis_to_hid(state))\n",
        "    \n",
        "    \n",
        "    @tf.function\n",
        "    def gibbs_k(self, state, k=1):\n",
        "        \n",
        "        i = tf.constant(0)\n",
        "        cond = lambda i, state: tf.less(i, k)\n",
        "\n",
        "        def operation(i, state):\n",
        "            i += 1\n",
        "            state = self.gibbs(state)\n",
        "            return i, state\n",
        "\n",
        "        _, state = tf.while_loop(cond, operation, [i, state])\n",
        "        \n",
        "        return state\n",
        "    \n",
        "    @tf.function\n",
        "    def compute_loss(self, x, nll_phys):\n",
        "        nll_crbm = self.n_log_like(x)\n",
        "\n",
        "        diff = nll_phys - nll_crbm\n",
        "        C = tf.reduce_mean(diff)\n",
        "        \n",
        "        loss = tf.reduce_mean((diff - C) ** 2)\n",
        "        return loss\n",
        "    \n",
        "    def compute_grad(self, x, nll_phys):\n",
        "        # compute the grad\n",
        "\n",
        "        ### pass through network\n",
        "        with tf.GradientTape() as tape:\n",
        "            loss = self.compute_loss(x, nll_phys)\n",
        "\n",
        "        grad = tape.gradient(loss, self.params)\n",
        "        return grad, loss\n",
        "\n",
        "\n",
        "    @tf.function\n",
        "    def train(self, x, nll_phys):\n",
        "        grad, loss = self.compute_grad(x, nll_phys)\n",
        "\n",
        "        self.opt.apply_gradients(zip(grad, self.params))\n",
        "        return loss"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nMFUHQ7hyNXR"
      },
      "source": [
        "# Training"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5ylnigpuBSB9"
      },
      "source": [
        "We create an CRBM object and train it. One can see that both the train_loss and the test_loss decrease simulaniously. The expected difference between the two losses should be around $\\frac{t_{test}}{t_{train}}=\\frac{N_{test}^2}{N^2_{train}}=\\frac{50^2}{3^2}=277.77$. Note that our objective is that $F_{RBM}(x)=\\beta E(x)+C$."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Rpvy_HS-vrMf"
      },
      "source": [
        "crbm = CRBM(kernel_dims=(2, 2))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "74Luj353ygYP"
      },
      "source": [
        "loss_train = []\n",
        "loss_test = []\n",
        "epoch = 0"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dmVULw63xiGi",
        "outputId": "a8e40893-b4fc-4129-889b-1446fb067e86",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 317
        }
      },
      "source": [
        "%%time\n",
        "l_train = 10 ** 9\n",
        "l_test = 10 ** 9\n",
        "\n",
        "while l_train > 10 ** -4:\n",
        "    # train\n",
        "    l_train = 0\n",
        "    \n",
        "    for train_s, train_E in train_dataset:\n",
        "        l_train += crbm.train(train_s, train_E / temp)\n",
        "    \n",
        "    l_train = l_train / N_TRAIN_BATCHES\n",
        "        \n",
        "    # plot results    \n",
        "    if epoch % 100 == 0:\n",
        "        l_test = crbm.compute_loss(states_test, E_phys_test / temp)\n",
        "        \n",
        "        loss_train.append(l_train) \n",
        "        loss_test.append(l_test)\n",
        "        \n",
        "        display.clear_output(wait=True)\n",
        "\n",
        "        print(\n",
        "            f\"Epoch: {epoch} | loss test: {loss_test[-1]}| loss train: {loss_train[-1]}| ltest/ltrain: {loss_test[-1]/loss_train[-1]}\"\n",
        "        )\n",
        "        plt.plot(loss_train)\n",
        "        plt.plot(loss_test)\n",
        "        plt.yscale(\"log\")\n",
        "        plt.show()\n",
        "    epoch += 1"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch: 6100 | loss test: 0.029209081083536148| loss train: 9.948405931936577e-05| ltest/ltrain: 293.60565185546875\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3yW5b3H8c+VQQgrBEIgZJAQEBI2CSBiERRliYMh4qjVqsXReVpPbU+PHccOa1ttrVts3SLiwoEgCooyEjYBQlgZhAzIgOxxnT/uoEhFgYz7Gd/368WLPHeS5/nd+vDNneu+rt9lrLWIiIjvC3C7ABERaRsKfBERP6HAFxHxEwp8ERE/ocAXEfETQW4X8HUiIiJsfHy822WIiHiV9PT0Ymttj5OPe3Tgx8fHk5aW5nYZIiJexRhz4KuOa0hHRMRPKPBFRPyEAl9ExE8o8EVE/IQCX0TETyjwRUT8hAJfRMRP+Gbgb10EW14BtX4WEfmcbwb+5pdg8c3w7BVweI/b1YiIeATfDPxrXoZp90PeBnh4LKy8D+pr3K5KRMRVvhn4AYEw+ha4cz0MnA4f3guPjIM9KzTMIyJ+yzcD/7jOvWDO03Dtq9BQC89eCU9Pg32r3K5MRKTN+XbgH9d/EtyxDqb+GUr2wb9nwNPTYd/HblcmItJm/CPwAYLbw5hb4QebYMqf4HAW/PtSeOYKKM5yuzoRkVbnP4F/XHB7OHc+/HATTP495KXDI2Nhxb1QV+V2dSIircb/Av+44FAYewfcmQbJl8Oq+5wZPbuXu12ZiEir8N/AP65zT5j1JHz7DWd2z/Oz4JXvwLFCtysTEWlRCvzj+k6A2z6Fib+EnW/DQ6Ng4/OaxikiPkOBf6KgELjgLpi/GiKT4I3bndW6R/a5XZmISLMp8L9Kj3PgO+/A9L9Abroztv/JA1Bf63ZlIiJnTYF/KgEBMOpmuGMNJE6E5fc4s3l2L3O7MhGRs6LA/yZhMTDvRbimqfvm87PhhblqyiYiXkeBf7rOuQRuXwMX/w72r4aHz4Wlv4SKYrcrExE5LQr8MxHUDsb9AL6fDkOugjUPwwNDYdk9UHHY7epERL5WmwW+MSbJGPOoMWaRMea2tnrdVtG5J1zxT6c/z8BpsPpBeHAofPBbqDzidnUiIl+pWYFvjFlgjCk0xmw76fgUY8wuY0yWMebnANbaHdba+cBVwLjmvK7HiOjvLNq6fQ30vwQ+/iv8cwxkvu92ZSIi/6G5V/j/AqaceMAYEwj8E5gKJAPzjDHJTZ+7DHgbeKeZr+tZIgc6bZi/txI69oAX5sCSH0NthduViYh8rlmBb61dBZw8hjEayLLW7rXW1gIvAZc3ff2b1tqpwLWnek5jzK3GmDRjTFpRUVFzymt7UcPglhUw9k5IexoeG+/M4xcR8QCtMYYfDeSc8DgXiDbGTDDG/N0Y8xhfc4VvrX3cWptqrU3t0aNHK5TXyoLbw+R74YY3oa4anroY1jzidlUiIgS11QtZaz8CPmqr13Ndwni4bTW8cQe893MI7gApN7hdlYj4sda4ws8DYk94HNN0zP+EdoU5/4J+k2DJj2DHErcrEhE/1hqBvx7ob4xJMMa0A64G3myF1/EOgcFw1TPQeyQsuslZtCUi4oLmTst8EfgMGGCMyTXGfNdaWw/cCSwFdgALrbXbm1+qF2vXEa59BcLj4cWr4dBWtysSET9krAf3e09NTbVpaWlul9FyynLhqUugsR5uWgrdEtyuSER8kDEm3VqbevJxtVZoS2ExcN1iaKiFZ6+EowVuVyQifkSB39YiBzqdN48VwHOzoKrU7YpExE8o8N0QOwrmPgtFO+HFeVBX5XZFIuIHFPhu6TcJZj4G2Z/BKzdCQ53bFYmIj1Pgu2nwLJh+P2S+C2/cCY2NblckIj6szVbayimMutlpqfzhvdC+C0y9D4xxuyoR8UEKfE8w/mdQXQafPeRM2Zz2F2dPXRGRFqTA9wTGwCX/BwGBzmYqDbUw4+/OYxGRFqLA9xTGwKTfQGAIrLrPuYl7+cMQqP9FItIylCaexBi48JfO3rkr/s+50p/5hNOPR0SkmRT4nmj8z5wr/WW/gtpKZxvF9l3crkpEvJzuDHqqcT+A6X+FrOWwYDKU7He7IhHxcgp8Tzbqu3Ddq1CeB09cCAc+dbsiEfFiCnxPlzgRbl4BoeHw78tgw7NuVyQiXkqB7w0i+sHNyyHhW/DmnfD2T539ckVEzoAC31uEhjtdNsfeCeufcIZ4CjLcrkpEvIgC35sEBsHke+HaRVBRCI9PgLWPgwdvYiMinkOB7436Xwy3fQp9L4B3fwYvzIVjRW5XJSIeToHvrTpFwjULnWZrez+Cf46GLQt1tS8ip6TA92bGwJjvwfdWQfdEWHwLPD8HSnPcrkxEPJAC3xdEDnQ2RZ/yJziwGh4+F9Y9of76IvIlCnxfERAI586H29dA7Gh456ew4BLI3+J2ZSLiIRT4via8D1y3GK54FI7sg8cvgHf/2+m3LyJ+TYHvi4yB4fPg+2mQciOsfQweGqWbuiJ+ToHvy0LD4dK/wi0roEu0c1P3X9Ph0Fa3KxMRFyjw/UH0SKc1w6V/g8Id8Nh4WPITqDjsdmUi0oYU+P4iIBBSb4Lvp8OoWyD9X/CPkc5K3YZ6t6sTkTagwPc3HbrBtPtg/icQNcxZqfvo+U7ffRHxaQp8f9UzGb79Bsx9Duqr4LlZzqKtoky3KxORVqLA92fGQNIMuGMdXPw7yF7jLNp65y6oPOJ2dSLSwhT4AkEhzpaK398AKTc47Zf/Phw+/QfU17hdnYi0EAW+fKFTD2cmz/zVEDMa3v8fZ/7+tsWavy/iAxT48p96JsN1i+D61yCkMyy6EZ662BnyERGvpcCXU0u80OnEefk/oSwXFkyGF6+Bwp1uVyYiZ0GBL18vIBBGXOeM71/4K9j/MTwyFt64E8ry3K5ORM6ATwb+ip0FrMzUDlAtql0HGP9T+MEmGHMbbHnZWbi17H81o0fES/hc4FtreWLVPm5YsI67F2/haHWd2yX5lo7dYcrv4c40SL4CVv8dHhwGK++DmqNuVyciX8PnAt8Yw9M3jmL+BYm8vD6HKQ98zCe7i90uy/eE94GZjzl76yaMhw/vdYL/04egrsrt6kTkKxjrwdPtUlNTbVpa2ll//4bsEn76ymb2FlVwzZg4fjEtiU4hQS1YoXwuNx1W/A72fgido+D8H8PIGyC4vduVifgdY0y6tTb1P477cuADVNc18Jf3d/HkJ/uI6BTC98b35doxfQhtF9hCVcqX7PsYPvqDs9Vi597wrZ/AyG87i7tEpE34beAftyG7hPuX7uLTPYe/CP5z4+jQTlf8Lc5a2LfKCf7sz5xe/Of/2JntExzqdnUiPs/1wDfG9AV+CYRZa2efzve0ZOAft37/ER5cvptPsorp3rEdt01I5Ibz4gkO9LnbGe6zFvathI/+6AR/x0g4706nTXNIZ7erE/FZrRL4xpgFwKVAobV28AnHpwAPAoHAk9baP57wuUVuBv5x6QeO8LdlTvD3i+zEr2cM4vz+Ea3yWn7PWmeIZ9X9zhh/+65w7m0w+lanXbOItKjWCvzxwDHgmeOBb4wJBDKBi4FcYD0wz1qb0fR5jwh8cKZwfrCjkN8uySD7SCVTB/fil9OTiAnv0Gqv6fdy0+Hj+2HXOxDc0WnWNvYOCItxuzIRn3GqwG/WOIa1dhVw8qqb0UCWtXavtbYWeAm4/AwKvdUYk2aMSSsqat3FU8YYJiX35P0fj+enl5zDh7sKmfTXlTy0Yjd1DY2t+tp+KyYF5r3oTOdMmuFssP7gMHhtvrP9ooi0mtYYuI4Gck54nAtEG2O6G2MeBUYYY+4+1Tdbax+31qZaa1N79OjRCuX9p/bBgdx5YX8++K8JTBwQyf3vZ3L5Q6vZfrCsTV7fL/Uc5Mzj/+EmGHUzZLzh9OJ/fg7s+VDdOUVaQZvdqbTWHrbWzrfWJlpr/9BWr3smoruG8sh1KTx6XQqFR2u4/KHV/OX9XdTUN7hdmu/qGgdT/wQ/3g4TfgEHN8KzV8Aj42DDs1BX7XaFIj6jNQI/D4g94XFM0zGvMWVwL5b/ZDyXDe/NP1ZkMeMfn7Ahu8Ttsnxbh24w4b/hR9uc7pwAb94JDwyGFfdCeb679Yn4gGZPyzTGxANLTrhpG4Rz0/YinKBfD1xjrd1+ps/d2jdtT8eHOwu5e/FWDpVXM2tkDP89ZQCRXbR6tNUdn9K55hHIXOp07Uy6DMbMh9jRzvaMIvKVWmuWzovABCACKADusdY+ZYyZBjyAMy1zgbX23rN5fk8IfIBjNfX8Y8VuFnyyj3aBAXz/ov7cOC6ekCCt1m0TR/bCuidh43NQUwZRw5wpnYNmOl08ReRLXF94dTY8JfCP21dcwb1vZ7B8RyHx3Tvwi2lJXJzcE6OrzbZRc8xpy7zuCSja4cznH3Gds5Cre6Lb1Yl4DAV+C1qZWcRv39rOnqIKRsWH8/OpSaT0CXe7LP9xfCHX+idhx1vQWA+JF0HqjXDOFAgMdrtCEVcp8FtYXUMjL6/P4YHluyk+VsPkQT25a8pAEnt0crs0/3L0EGx4BtKehqMHoVNPGH4NjLheV/3itxT4raSipp6nPtnHYyv3UF3fyPXn9uGuKQPUlK2tNdRD1jIn/DOXgm2A+G85wZ90KbTr6HaFIm1Ggd/Kio/V8MDyTJ5bk02f7h24f84wRsWrT4wryvNh03POPP7SA9Cuk7M71/B5EHceBKhRnvg2BX4bWbP3MD9btJnckipuGpfAzyYPoH2wZvO4orERsj+FTS9CxutQe8xZ6DX8Whg2z9m1S8QHKfDbUEVNPb9/ZwfPr82mb0RHHrx6BENiwtwuy7/VVsCOJbDpeadXPxYSLnBm+STNUJ9+8SkKfBd8vLuIuxZt4VhNPa/MH8vAXl3cLkkASrOdq/5NzztDPiFhkHwZDJkD8ec7i7xEvJgC3yV5pVXMfHg1AK/edp5aL3uSxkZneuem553pnbXHoFMvGDwTBs+G6JFa0SteSYHvol2HjjLn0U+J6BzCovnn0a1jO7dLkpPVVUHme7B1Eex+HxpqITwBBs+CIbMhMsntCkVOmwLfZev2HeG6p9aSHNWFF24Zo2mbnqyq1Lni37bIGe+3jRA5qOnKfyZ06+t2hSJfS4HvAZZuP8Rtz6VzwTk9ePzbqdpH1xscLXB69W9bBDlrnWNRw53gT75CM33EIynwPcQLa7P5xWtbmTUyhj/PHkpAgMaIvUZpNmx/Hba/Bgc3OMeiU51hn0FXQpcod+sTaaLA9yAPLt/N35ZnctO4BH51aZKar3mjI/ucuf3bFsOhLYBxZvgMme20cdbm7OIiBb4Hsdby2yUZPL16Pz+5+Bx+cFF/t0uS5ije7dzs3bYIDmdBQBD0m+TM9BkwFULUX0na1qkCX3cOXWCM4VfTkymrquOvyzIJCw3mhvPi3S5LzlZEf5h4N0z4uXO1v/UV58o/8z0I7uCE/pA5TkfPIM3QEvco8F0SEGC4b9ZQyqvquefN7YSFBnPFiGi3y5LmMMbZnCVqGEz6LeSsccJ/++uw7VUIDXfG+odcBbFj1NNH2pyGdFxWXdfAjU+vZ93+Izx6XQoXJ/d0uyRpaQ11sGcFbFkIO9+G+ioIi4Ohc2DoXOgxwO0KxcdoDN+DHaup59on1pCRX87D1yr0fVrNMSf0ty50fgjYRuc3gqFzndk+nXu5XaH4AAW+hyurrOPbC9aSkV/OP+aNZMpg/cP3eccKnaGeLS/DwY1gAqDvBCf8B16qm71y1hT4XqC8uo4bFqxja24Zf583gmlDNK/bbxRlOlf9W1525vsHd3Q2bhl6FSRMgEDdbpPTp8D3Eker6/jO0+vZlFPKA3OHM2NYb7dLkrbU2Ojc7N3ysrPAq7rM2bZx8GwYNhd6DVVDN/lGCnwvcqymnhufXkf6gRLumz2M2Skxbpckbqirht1LYfPLTkO3xjroMdAZ8hkyB7rGul2heCgFvpepqKnnlmfS+HTPYW75VgL/PWUgQeq9478qjzhX/FsWOr8BAPQZ56zsTb5CK3vlSxT4XqiuoZH/W5LBvz87wLh+3Xlo3kjC1VpZjuxzVvZuXQjFmc7K3sSLnKv+gdOhnfZc8HcKfC+2MC2H/3ltG5FdQnj8+lSSe2vnLAGshUNbm1b2vgrledCus9PJc8T1EJOq8X4/pcD3cptySpn/bDqlVbX8ceZQrcqVL/t8964XnKZudZUQMQBGXOtc+XfRzX9/osD3AUVHa7jjhQ2s23eEK0dE85vLB9GlfbDbZYmnqS53xvs3Pd/Uw9844/3He/h37O52hdLKFPg+or6hkYc/2sODH+wmKqw9D8wdTmq8btjJKRRnOcM92xZ9Md7fdyKMuM4Z7w/UBYMvUuD7mPQDJfzo5Y3klVRx54X9+cGF/TSLR07t+Hj/tledG77luc78/hHXQ8p3NMXTxyjwfdDR6jrueXM7izfkMTQmjD/NGkpSlG7oyjdobIDdyyBtgTO/3xjof4kT/udM1lW/D1Dg+7C3t+Tzv29so6yqju9d0JfvX9if9sGBbpcl3qA0G9L/DRufhWMF0CHCWdg14lroOcjt6uQsKfB9XElFLb97O4PFG/Lo26Mjf5w5lNEJGtuX09RQD1nLYdNzsOs9Z1Vv1HBnrH/wLC3s8jIKfD+xMrOIXyzeSl5pFdeMieOuyQPo2kGLteQMVBx25vZvfA4KtkJgO6d754hrnRu+Afrt0dMp8P1IRU09f12Wyb8+3U9YaDA/nzqQ2SNjCAjQIhw5Q/mbYePzzqreqhLoEu3s2jXoSohO0cIuD6XA90MZB8v51RvbSD9QQkqfcH53+WCt0pWzU18Du96FzS86G7c01EJYLCRf7sztj07Rlo0eRIHvpxobLYs25PLHd3dSVlXH9ef24ceTziGsg2ZiyFmqKnXCP+P1L8K/Y6Qz0+ecyZA4EUI6u12lX1Pg+7nSylr+vHQXL6zLpmtoMD+dPICrR8URqGEeaY6qUshc6rRxzlru9O8PCIb4cTBsnnP1H9ze7Sr9jgJfANh+sIzfvJXBun1HSIrqwq9nJDOmr5baSwtoqHNaOWQuhR1vQck+CA2H4ddC6k3QPdHtCv2GAl8+Z63l7a35/P7tHRwsq+bSoVH8z/RkeoXpSkxaSGMj7F8F659yNm23Dc5+vSOud1o6BIe6XaFPU+DLf6iqbeCRlXt4dOUeggMMP5p0Dt8ZF0+wWjRISyrPd6Z4bvg3lOVASBgMusK58o8drZk+rUCBL6d04HAFv35zOx/uKqJ/ZCd+e/lgxiZqmEdaWGMj7P/YaeG8402nhXO3RGeKZ/Jl2q+3Bbke+MaYvsAvgTBr7ezT+R4Fftux1rJ8RyG/eWs7uSVVTB8SxX9dcg59e3RyuzTxRTVHIeMN2PyS08ffNkJ4PCRd5kz17D1S0zyboVmBb4xZAFwKFFprB59wfArwIBAIPGmt/eNpPNciBb7nqq5r4JGP9vDEx3upqW/kqtRYfnhRf43vS+upKHbG+TPegH0robHe6enT7yLoN8nZvlE9/M9IcwN/PHAMeOZ44BtjAoFM4GIgF1gPzMMJ/z+c9BQ3WWsLm75Pge8Fio7W8NCK3bywLpsAY7hxXAK3XZCo+fvSuqpKIPN9Z4rnng+g8jBgIHqkc/U/6EoI7+N2lR6v2UM6xph4YMkJgT8W+LW1dnLT47sBrLUnh/3Jz/O1gW+MuRW4FSAuLi7lwIEDp1WftI7sw5X8bXkmr2/Ko3NIEPMnJHLjeQmEtlM/FWlljY2QvxGyPnAWeh3c4ByPGQWDZjo3frV141dqjcCfDUyx1t7c9Ph6YIy19s5TfH934F6c3wie/KYfDKArfE+yI7+cPy/dxYqdhUR2DuGHk/pzVWqsZvRI2ynZ72zduO1VZzMXcDp69r/YWeUbnaLGbk1cD/yzocD3POv2HeG+93aSdqCEhIiO/GzyAKYO7oXR7AppS8VZTmuH3csgd51z0zc0HBIvhLixznTPyEEQGOR2pa7wmCGdM6HA90zWWlbsLOS+93axq+AoI+O68svpyaT0CXe7NPFHlUdg74dO+O9Z4WzkAhDc0Rn7jx0DA6Y5H/vJhUlrBH4Qzk3bi4A8nJu211hrt7dQzQp8D9fQaHk1PZf7399F4dEapg3pxV2TBxIf0dHt0sRfWQulByBnvXPln7POGf6xDU53z6TLnLH/6FSfnvbZ3Fk6LwITgAigALjHWvuUMWYa8ADOzJwF1tp7W7JoBb53qKyt54lV+3hs1R7qGhq5elQct01IpHdXLZ8XD1BV4uzidWJ3z85R0Oc85+o/ZhT0GuJTe/m6vvDqbCjwvUtheTUPfLCbhetzCDCGOakx3D6xH9EKfvEU1WVOc7dd7zhX/+V5zvGgUGfIJ+5ciDsPYkdB+zB3a20GBb60mdySSh75aA8L03IAmJ0Syx0TE4kJ7+ByZSInKct1gj93PWSvcXb4sg1gApxN3GNGQadezp6+oeEQ2tXp/d9zkEfPCFLgS5s7WFrFIx/t4eX1OVgs147pw+0TE4nsrFW74qFqKyA3DbI/c/4c3Oj8VnCy0G5fTAftd5Hzw8CDKPDFNQdLq/jHiiwWpuXQLjCA74yL53vj+2pzdfEODXXORi9VJVB1BEqznZXAu5c5j01g072AVGdYqPdI6Brn6owgBb64bn9xBX9bnsmbmw/SKSSI75wXz/Xn9iGyi674xQs1Nji/DexeCns+hIJtzg1hgA7dIWqYE/xhMdAlxvk7PB66xrZ6aQp88Rg7D5Xzt2WZvJ9RQFCAYcaw3nz3/AQG9fbem2Qi1NdAYQbkbXCGgg5tde4RVBZ/+eu694cBU521AbGjW+VegAJfPM7+4gr+9el+FqblUFnbwJiEbtw4LoFJSZEEqWWD+Iq6Kig/6Gz+UrgTMt+D/Z9AY53zm0C/SRCZDBH9nR8G3RKaPUVUgS8eq6yqjpfXZ/PvTw+QV1pFVFh7rhkdx9Wj4+jROcTt8kRaXnXZF03h9q38YnUwOPcEuiXAvJecHwJnQYEvHq++oZEVOwt5ds0BPt5dTHCgYcrgKG4cF8/IOM+aBSHSoqpK4fAeKM6Ew7uheDdc9veznv2jwBevsrfoGM+tyeaV9ByOVteT2iecW8b3ZVJSTwID/KMfisjZUuCLV6qoqWdhWg5PfbKP3JIqEiI68t3zE5g1MkY9+UVOQYEvXq2+oZH3th/i8VV72ZJbRueQIGYM783Vo2IZEh2m9swiJ1Dgi0+w1rJ+fwkvrcvmnW35VNc1MrBXZ+aOiuXKEdFazCWCAl98UHl1HW9uOsjL63PYmldGu8AALhnUk7mjYhmXGEGAxvrFTynwxadlHCxnYVoOr2/Ko7SyjuiuocxOiWFOaoyatonfUeCLX6iua2BZRgEL03L4JMtZ4Xh+vwjmjorl4uSehATpRq/4PgW++J3ckkpeSctlUXoueaVVhHcI5ooR0cxJiSW5dxe3yxNpNQp88VsNjZbVWcW8nJbDsu0F1DY4N3pnjozm8uHR9FTzNvExCnwRoKSiliVbDrJ4Yx4bs0sJMDCuXwRXj4rjkkE9CVYPH/EBCnyRk+wtOsZrG/NYvCGPvNIqIjuHcPXoOOaNjiUqTNsyivdS4IucQkOjZWVmIc9+doCPMosIMIZJSZHMGNab8/tFaG6/eJ1TBX6QG8WIeJLAAMOFA3ty4cCe5Byp5Pm12SxMy2Hp9gICDAyN6coF5/Rg/Dk9GBHbVfP7xWvpCl/kK9Q3NLI5t4xVmUWszCxiS24pjZbP5/fPTokhtpvm94tn0pCOSDOUVtayMrOIRem5fJJVjLUwrl93rkp15vd3aKdflsVzKPBFWkheaRWL0nJ5JT2H3JIq2gUFcH6/CCYl9eSipEhN8xTXKfBFWlhjo2XNvsMsyyhgWUYBuSVVAAyLCWPGsN5cPjxaO3aJKxT4Iq3IWktmwTGW7yhg6fZDbMktIzDAcME5PZidEsNFSZFq6yBtRoEv0oayCo+yKD2P1zbmUlBeQ1hoMNOG9OLSob0Zk9BNm7RLq1Lgi7igodHySVYxizfksjyjgIraBiI6tWPKYCf8R8d30zRPaXEKfBGXVdc18OHOQpZsyeeDnQVU1zUS160D80bHMSc1hohOGu+XlqHAF/EglbX1LMso4IW12azdd4TgQMPkQb24dkwfzu3bTVs2SrMo8EU8VFbhUZ5fm82r6bmUV9cT160DM0dGM3NEDHHdtbhLzpwCX8TDVdU28M7WfF7dkMtnew9jLYyKD2fmyBimDu6lnj5y2hT4Il4kr7SK1zfm8eqGXPYWVRAcaDi/XwTTh/bm4uSehIUGu12ieDAFvogXstayNa+Mt7fks2RLPnmlVQQHGsb378GslBgmJfWkXZCmeMqXKfBFvJy1ls25Zby95SBLtuSTX1ZNRKd2zBoZw9xRsfTt0cntEsVDKPBFfEhDo2XV7iJeWpfN8h2FNDRaxiR0Y+bIaKYMiiKsg4Z8/JkCX8RHFR6tZlF6LgvX57D/cCXBgU5LhxnDejMpqScdQ9TJ098o8EV83PHx/rc2fzHk0z44gOlDenPNmDhGxnXV/H4/ocAX8SONjZb07BJe25jHGxvzqKhtYGCvzlwzJo4rRkTTpb2GfHyZAl/ET1XU1PPm5oM8v/YA2/LKCQ0OZNqQKK5KjWF0glb1+iIFvoiwJbeUF9dl89bmfI7V1BPfvQNzUmOZNTKGXmHauMVXuB74xpgk4IdABPCBtfaRb/oeBb5I66isrefdrYdYmJbD2n1HCDAwrl8Es0bGcMkgbdno7ZoV+MaYBcClQKG1dvAJx6cADwKBwJPW2j+exnMFAM9Ya6/7pq9V4Iu0vv3FFSxKz+W1jXnklVbRsV0gkwf3YtbIGMb27a72zV6ouYE/HjiGE9SDm44FApnAxUAusB6YhxP+fzjpKW6y1hYaYy4DbgOetda+8E2vq8AXaTuNjZb1+4+weEMe72zN52hNPb3D2jMrJYZZI2OIj+jodolympo9pGOMiQeWnBD4Y4FfW2snNz2+G8Bae3LYf9VzvW2tnX6Kz8HxisUAAAmcSURBVN0K3AoQFxeXcuDAgdOqT0RaTnVdA8syCliUnsvHu4totDA6vhuzU2KYNjSKTprb79FaI/BnA1OstTc3Pb4eGGOtvfMU3z8BmAmEAFustf/8ptfUFb6I+w6VVfPqhlxeTc9lb3EFocGBTB3ci1kpGvLxVKcK/Db7MW2t/Qj4qK1eT0RaRq+w9twxsR+3T0hkQ3YJi9LzWLLlIIs35hHdNZQrR0QzdUgvkqO6aIqnh2tO4OcBsSc8jmk6JiI+yBhDSp9upPTpxj0zknk/o4BX03N5+KMsHvowi95h7ZmU3JNJST05t293dfH0QM0J/PVAf2NMAk7QXw1c0yJViYhHax8cyGXDenPZsN4UH6thxc5ClmUUsDAth2c+O0CnkCAuHRrF3FGxDI9VSwdPcbqzdF4EJuDMoS8A7rHWPmWMmQY8gDMzZ4G19t6WLE5j+CLepbqugdVZxby77RBvb8mnqq6BAT07M3dULFeOiCa8o3btaguuL7w6Gwp8Ee91tLqOtzbn8/L6bDbnltEuMICJA3tw2bBoLkqKpH1woNsl+iwFvoi4Zkd+Oa+k5fLWloMUHa2hU0gQlyT3ZMbw3ozt213h38IU+CLiuoZGy5q9h3lz00He2ZbP0ep62gUFkNonnPMSuzM2MYKhMWEEB+qGb3Mo8EXEo9TUN/Bp1mFWZxXz6Z7DZOSXA3x+9X/FiGjOS+xOkML/jLk+D19E5EQhQYFMHBjJxIGRABypqGXN3sN8tKuQd7cdYvHGPCI6hTBjWBRXjohmSHSYZvs0k67wRcTjVNc18NGuQl7feJAVOwupbWgkrlsHpg7pxbTBUQyNUfh/HQ3piIhXKqusY+n2Q7y9NZ/VWcXUN1qiu4YybUgvLkrqSUqfcI35n0SBLyJer7SylmUZBbyzNZ9Psoqpa7B0Dgni/P4RTBwQyYQBPYjsoo1cFPgi4lOOVtexOuswKzML+XBnEYfKqwFI7RPOpUOjmDYkym/DX4EvIj7LWsvOQ0f5YEcBS7bks/PQUYyBMQndmD60N+MSu5MQ0dFvxv0V+CLiN7IKj/LW5nyWbDnInqIKALp3bEdqfDij4ruR0iecpKguPrvgS4EvIn7HWsueogrW7z/C+v1HSNtfQvaRSgACAwx9IzqS3LsLSVFdGNS7CyPiwn1icxcFvogIUFBezYYDJezILycjv5yMg+UcLHPG/wMDDENjwhjbtztjE7uT2qcboe2877cABb6IyCmUVtayNa+MNXsP89mew2zOLaOh0WIMdGkfTNcOwXTt0I6uocGEdwgmskt7IjuHENmlPT07hxDTrQPRXUPdPo3PaaWtiMgpdO3Qjm/178G3+vcA4FhNPWn7j7App5SSilpKq+oorayjtLKWvcXHKCyvoaa+8UvP8a3+Ecy/IJHzErt77M1hXeGLiJwhay3l1fUUlldTUF7D5txSnl69n+JjNQyNCWP+BYlMHtSLQJf2+9WQjohIK6qua2DxhjweX7WH/Ycrie4aSlJUF2LCQ4nuGkpMeCix3TowoFfnVl8ZrCEdEZFW1D44kGvGxDF3VCzvbTvEaxvzyDlSyZq9hzlWU3/C1wUwLKYrqfHhpPQJZ2RcOF07tM1OYAp8EZEWFBhgmD40iulDo4Cm4Z+qenJLK9lbVMGG7BLSD5Tw6Mq9NDQ6IywDe3VmTEI3xvTtzuiEbkR0CmmV2jSkIyLigsraejbnlJG2/whr9x0h/UAJVXUNAPSL7MSj16XQL7LTWT23hnRERDxIh3ZBjE105vt/H6itb2RrXhlr9x1m3b4jRIW1fB8gBb6IiAdoFxRASh9nXP/2Ca3zGmoiLSLiJxT4IiJ+QoEvIuInFPgiIn5CgS8i4icU+CIifkKBLyLiJxT4IiJ+wqNbKxhjioADZ/ntEUBxC5bjFp2HZ9F5eBZfOI/WOIc+1toeJx/06MBvDmNM2lf1kvA2Og/PovPwLL5wHm15DhrSERHxEwp8ERE/4cuB/7jbBbQQnYdn0Xl4Fl84jzY7B58dwxcRkS/z5St8ERE5gQJfRMRP+GTgG2OmGGN2GWOyjDE/d7ue02WMWWCMKTTGbDvhWDdjzDJjzO6mv8PdrPGbGGNijTEfGmMyjDHbjTE/bDrubefR3hizzhizuek8ftN0PMEYs7bpvfWyMaZtdp9uJmNMoDFmozFmSdNjrzsPY8x+Y8xWY8wmY0xa0zGvel8BGGO6GmMWGWN2GmN2GGPGttV5+FzgG2MCgX8CU4FkYJ4xJtndqk7bv4ApJx37OfCBtbY/8EHTY09WD/yXtTYZOBe4o+m/v7edRw1wobV2GDAcmGKMORf4E/A3a20/oAT4ros1nokfAjtOeOyt5zHRWjv8hHnr3va+AngQeM9aOxAYhvP/pW3Ow1rrU3+AscDSEx7fDdztdl1nUH88sO2Ex7uAqKaPo4Bdbtd4hufzBnCxN58H0AHYAIzBWREZ1HT8S+81T/0DxDSFyIXAEsB46XnsByJOOuZV7ysgDNhH04SZtj4Pn7vCB6KBnBMe5zYd81Y9rbX5TR8fAnq6WcyZMMbEAyOAtXjheTQNg2wCCoFlwB6g1Fpb3/Ql3vLeegC4C2hsetwd7zwPC7xvjEk3xtzadMzb3lcJQBHwdNMQ25PGmI600Xn4YuD7LOv8+PeKebTGmE7Aq8CPrLXlJ37OW87DWttgrR2Oc4U8GhjocklnzBhzKVBorU13u5YWcL61diTOcO0dxpjxJ37SS95XQcBI4BFr7QiggpOGb1rzPHwx8POA2BMexzQd81YFxpgogKa/C12u5xsZY4Jxwv55a+3ipsNedx7HWWtLgQ9xhj66GmOCmj7lDe+tccBlxpj9wEs4wzoP4n3ngbU2r+nvQuA1nB/C3va+ygVyrbVrmx4vwvkB0Cbn4YuBvx7o3zQLoR1wNfCmyzU1x5vADU0f34AzJu6xjDEGeArYYa396wmf8rbz6GGM6dr0cSjOfYgdOME/u+nLPP48rLV3W2tjrLXxOP8WVlhrr8XLzsMY09EY0/n4x8AlwDa87H1lrT0E5BhjBjQdugjIoK3Ow+2bGK10Y2QakIkz5vpLt+s5g7pfBPKBOpwrge/ijLd+AOwGlgPd3K7zG87hfJxfR7cAm5r+TPPC8xgKbGw6j23A/zYd7wusA7KAV4AQt2s9g3OaACzxxvNoqndz05/tx/9de9v7qqnm4UBa03vrdSC8rc5DrRVERPyELw7piIjIV1Dgi4j4CQW+iIifUOCLiPgJBb6IiJ9Q4IuI+AkFvoiIn/h/vqN4r3JWbIsAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "text": [
            "CPU times: user 2min 37s, sys: 11.2 s, total: 2min 49s\n",
            "Wall time: 2min 30s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WBaalL7QRmEn"
      },
      "source": [
        "The learned convolutional kernels show the effective interaction between nearest neighbours."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dBc3oN6q41eB",
        "outputId": "501251e5-b28c-48a3-f1ab-08799e7b6f2f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 208
        }
      },
      "source": [
        "W = crbm.W.numpy()\n",
        "vmax = np.max(abs(W))\n",
        "\n",
        "fig, axes = plt.subplots(nrows=1, ncols=2)\n",
        "\n",
        "axes[0].imshow(W[:, :, 0, 0], vmin =-vmax, vmax=vmax, cmap=\"seismic\")\n",
        "axes[1].imshow(W[:, :, 0, 1], vmin =-vmax, vmax=vmax, cmap=\"seismic\")\n",
        "\n",
        "# Deactivate axes\n",
        "for j in range(2):\n",
        "    axes[j].set_xticks([])\n",
        "    axes[j].set_yticks([])\n",
        "    axes[j].set_xlabel(f'$W^{j}$', fontsize=20)\n",
        "\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAADACAYAAAA6CrkWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAGrUlEQVR4nO3cT6jlZR3H8c/jDGE65thY1ix0BLeCOdIiqEUhRNCqpKVSJNIMRISBEIKbiIqwjdYmrNwELqKgyE22KFCUwAKhP2gDkyNJzkyKJerT4p7R621unNucez9zx9cLhnvv+T1zznfx8Ob5nXu4Y84ZAHbeRe0BAN6uBBigRIABSgQYoESAAUoEGKBk71YWX3nZZfPQgQPbNcuF5+KL2xPsKs+cOJHnT54cO/26+/dfOQ8ePLTTL7trXTJfao+wqzzz7LOb7ustBfjQgQN5/O67VzPV28F117Un2FVuuv32yusePHgoDz74eOW1d6MbX32sPcKuctOtt256zVsQACUCDFAiwAAlAgxQIsAAJQIMUCLAACUCDFAiwAAlAgxQIsAAJQIMUCLAACUCDFAiwAAlAgxQIsAAJQIMUCLAACUCDFAiwAAlAgxQIsAAJQIMUCLAACUCDFAiwAAlAgxQIsAAJQIMUCLAACUCDFAiwAAlAgxQIsAAJQIMUCLAACUCDFAiwAAlAgxQIsAAJQIMUCLAACUCDFAiwAAlAgxQIsAAJQIMUCLAACUCDFAiwAAlAgxQIsAAJQIMUCLAACUCDFAiwAAlAgxQIsAAJQIMUCLAACUCDFAiwAAlAgxQIsAAJQIMUCLAACUCDFAiwAAlAgxQIsAAJQIMUCLAACV7t7T62LHkyJFtGuXC8/pLL7dH2F327au87FNP/TmHD3+y8tq70V13/aw9wq5y/OSlm15zAgYoEWCAEgEGKBFggBIBBigRYIASAQYoEWCAEgEGKBFggBIBBigRYIASAQYoEWCAEgEGKBFggBIBBigRYIASAQYoEWCAEgEGKBFggBIBBigRYIASAQYoEWCAEgEGKBFggBIBBigRYIASAQYoEWCAEgEGKBFggBIBBigRYIASAQYoEWCAEgEGKBFggBIBBigRYIASAQYoEWCAEgEGKBFggBIBBigRYIASAQYoEWCAEgEGKBFggBIBBigRYIASAQYoEWCAEgEGKBFggBIBBigRYIASAQYoEWCAEgEGKBFggBIBBigRYIASAQYoEWCAEgEGKBFggBIBBigRYIASAQYoEWCAEgEGKBFggJIx51x+8Rh/T/LX7RuHt7lr5pzv2ekXta/ZZpvu6y0FGIDV8RYEQIkAA5QIcNEY4wtjjKfHGP8aYzwxxvhweyY4V2OMj4wxfjrGOD7GmGOM29ozna8EuGSM8Zkk30nytSQfSPLbJL8YY1xdHQzO3b4kf0jyxSQvl2c5r/klXMkY49EkT845P7/usT8leWjOeVdvMlidMcaLSY7OOR9oz3I+cgJewhhj3+JW6ulNrt+wuD7HGB/bZM2Ti+vXjzHekeRwkoc3LHs4yYdWOz1sbtV7e3unvfAI8BLmnC8meTXJ5Zss+cq67/9rzRjj40muT/LzOefvk1yZZE+S5zYsfS7J+855YFjSNuxttkCAl3cyybs2PjjGuCbJLUmOLR7af5b/e+fi6ze2ZzQ4J/Z2iQAv74Uke8YY+zY8/uUke5Pcs/j5LaeEMcaNST6a5NE5568XDz+f5LUkV214rquSnFjl0LCEVe5ttkCAl3dy8fWNTTjGOJDkc0keyZvv5248JZy5hXvjhDDnfCXJE0lu3rD25qx9GgJ20sr2Nluztz3ALvLC4uvlSY4vvj+a5JIkX09yat31JMkY49okn07yxyQ/2fB8307yozHGY0l+k+SOJAeTfHc7hof/YaV7e3GSvm7x40VJrh5j3JDkH3POY+ENAry89Zs0Y4x3Zm2T/m7O+csxxkjyet56SvhS1n7Z9q055+vrn2zO+ePFKeOrSd6ftc9NfmLO6Y/CsNNWureT3JTkV+t+vmfx7wdJblv59LuYAC9v423aZ7P2aYajSTLnnGOMf+bNTfzuxZoTSX54tiecc96X5L5tnBmWsdK9Ped8JMnY3pEvDN4DXt6ZU8L+McaerP2C4i9JHlq35lTePCUcSXJpknvnnP/esSlh6+ztEifg5a0/JdyS5Nokd8w5X1u35lSSy8cYF2ft9HA63tPl/GdvlzgBL2/9+2R3Zu3264ENa04vrt+a5L1JvjfnPBU4v9nbJQK8vDOnhE8luTFnv/06leSKrN3CvZLk3p0bD/5v9naJtyCWd+aU8MGsbcb7z7LmzCa9Isn355x/26HZ4FzY2yVOwMt7Yd339885T59lzZlbspnkm9s/EqyEvV3iz1EClDgBA5QIMECJAAOUCDBAiQADlAgwQIkAA5QIMECJAAOUCDBAiQADlPwHDlZhY6BDUMgAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-mZoKqjr7uVT"
      },
      "source": [
        "# Sampling"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uVHIsquKZoh1"
      },
      "source": [
        "A state with $L=40$ is updated $5\\times10 ^4$ times using gibbs sampling.  Because the temperature T=1, is well bellow the critical temperature TC=2.269 it converges to the ground state."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "30z8c8PT7JLd"
      },
      "source": [
        "L = 200\n",
        "states = tf.random.uniform((1, L, L, 1))\n",
        "crbm.set_analytical_param(1 / temp)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rsODKWM66otO",
        "outputId": "15665bbd-d32e-4a95-f78d-d23a8a77b4c4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 342
        }
      },
      "source": [
        "epochs = 5 * 10 ** 4\n",
        "Es = []\n",
        "for i in range(epochs):\n",
        "    states = crbm.gibbs(states)\n",
        "    Es.append(ising_energy(states)[0])\n",
        "    \n",
        "    if i % 1000 == 0:\n",
        "        display.clear_output(wait=True)\n",
        "        fig, axes = plt.subplots(nrows=1, ncols=2, figsize=(15, 5))\n",
        "\n",
        "        axes[0].imshow(states.numpy()[0, :, :, 0], vmin=0, vmax=1, cmap=\"gray\")\n",
        "        axes[0].set_xticks([])\n",
        "        axes[0].set_yticks([])\n",
        "        axes[0].set_xlabel('state', fontsize=20)\n",
        "\n",
        "\n",
        "        axes[1].plot(Es)\n",
        "        axes[1].plot([0, len(Es)], [-2 * L ** 2, -2 * L ** 2])\n",
        "        axes[1].set_xlabel(f'steps', fontsize=20)\n",
        "        axes[1].set_ylabel(f'E', fontsize=20)\n",
        "        display.clear_output(wait=True)\n",
        "        plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAyYAAAFGCAYAAACfXItBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5xddX3v/9dnZjIJCSQhEAOSxASNWvSnUQLS1gsKheAt2IMWz68SLQ9pK57qOcdTsR4rVjk/9LSiHq0WhYpUBQ62JVX4Re7nyClyEZCLtymiJEIIuXHJZTIzn/PH+k6yGWdiLjOzste8no/Heuy1P2vttb97kc3Me9b3u76RmUiSJElSnTrqboAkSZIkGUwkSZIk1c5gIkmSJKl2BhNJkiRJtTOYSJIkSaqdwUSSJElS7br2ZOdDDz00FyxYMEZNkdRUhx56KCtXrlyZmUvrbovq5c8RSZoY7rzzzsczc/aevGaPgsmCBQu444479qxVkgRExKF1t0H18+eIJE0MEfGLPX2NXbkkSZIk1c5gIkmSJKl2BhNJkiRJtTOYSJIkSaqdwUSSJElS7QwmkiRJkmpnMJEkSZJUO4OJJEmSpNoZTNQomcmqVavYvHlz3U2RJEnSHjCYqFG2bNnCC1/4Qr797W/X3RRJY+CuX27gitsfrrsZkqQxYDBRo0yZMoWbbrqJE088se6mSBoD//99j/KXK+6ruxmSpDHQVXcDpNHU0dHBkiVL6m6GJEmS9pBXTCRJkiTVzmAiSQ0TEf89In4cET+MiH+KiJkt2z4UET0R8ZOIOLmlvrTUeiLinJb6woj4fqlfHhHdpT65PO8p2xeM52eUJDWPwUSSmuda4MWZ+RLgp8CHACLiKOB04EXAUuBvI6IzIjqBLwCnAEcBby/7AnwSuCAznwdsAM4s9TOBDaV+QdlvXGSO1ztJksaTwUSSGiYzv5uZfeXprcDcsr4MuCwzt2Xmz4Ee4Niy9GTmg5nZC1wGLIuIAF4HXFlefwlwasuxLinrVwInlP3H1ti/gySpJgYTSWq2PwKuKetHAK332l1VaiPVDwE2toScwfozjlW2byr7S5K0V7wrlyS1oYi4DjhsmE0fzsyryj4fBvqAr49n24aKiLOAswDmz59fZ1MkSfsxg4kktaHM3OVkPRHxTuCNwAmZO0ZlrAbmtew2t9QYob4OmBkRXeWqSOv+g8daFRFdwIyy/3BtvRC4EGDJkiX7PELEISaS1Ex25ZKkhomIpcCfA2/OzM0tm1YAp5c7ai0EFgG3AbcDi8oduLqpBsivKIHmRuC08vrlwFUtx1pe1k8DbmgJQGMmHGQiSY3lFRNJap7PA5OBa8t49Fsz808y8/6IuAJ4gKqL19mZ2Q8QEe8FVgKdwMWZeX851geByyLiE8BdwEWlfhFwaUT0AOupwowkSXvNYCJJDVNu4TvStvOA84apXw1cPUz9Qaq7dg2tbwXeum8tlSRpJ7tySZLai4NMJKmRDCaSpLYxDjOlSJJqYjCRJEmSVDuDiSRJkqTaGUwkSZIk1c5g0iC9vb2sWbOGcZhKQJJqk45+l6RGMpg0yC233ML8+fN5/PHH626KJI0Jx75LUnMZTBrkZS97GStXrmTmzJl1N0WSJEnaI06w2CAzZ87k+OOPr7sZkiRJ0h7ziokkqa04jE6SmslgIklqG06wKEnNZTCRJEmSVDuDiSRJkqTaGUwkSZIk1c5gIklqK459l6RmMphIktpGOMWiJDWWwUSSJElS7QwmkiRJkmpnMJEktZV0hkVJaiSDiSSpbTjBoiQ1l8FEkiRJUu0MJpIkSZJqZzCRJEmSVDuDiSSprTj0XZKayWAiSWobjn2XpOYymEiSJEmqncFEkiRJUu0MJpKktuL8ipLUTAYTSVL7cIZFSWosg4kkSZKk2hlMJEmSJNXOYCJJDRMRH4+IH0bE3RHx3Yh4dqlHRHwuInrK9pe3vGZ5RPysLMtb6kdHxL3lNZ+LqPpSRcSsiLi27H9tRBw8/p9UktQkBhNJap7/npkvyczFwLeBvyz1U4BFZTkL+CJUIQP4KPAK4Fjgoy1B44vAu1tet7TUzwGuz8xFwPXluSRJe81gIkkNk5lPtDydxs7J0pcBX8vKrcDMiDgcOBm4NjPXZ+YG4Fpgadk2PTNvzcwEvgac2nKsS8r6JS31MeXQd0lqrq66GyBJGn0RcR5wBrAJeG0pHwE83LLbqlLbVX3VMHWAOZn5SFl/FJgzmu2XJE08XjGRpDYUEddFxH3DLMsAMvPDmTkP+Drw3rFsS7maMuLsIhFxVkTcERF3rF27diybIklqY14xkaQ2lJkn7uauXweuphpDshqY17JtbqmtBo4fUr+p1OcOsz/Amog4PDMfKV2+HttFWy8ELgRYsmTJqEyPmJmEc5pIUqN4xUSSGiYiFrU8XQb8uKyvAM4od+c6DthUumOtBE6KiIPLoPeTgJVl2xMRcVy5G9cZwFUtxxq8e9fylvqYMotIUnN5xUSSmuf8iHgBMAD8AviTUr8aeD3QA2wG3gWQmesj4uPA7WW/v8rM9WX9PcBXgQOAa8oCcD5wRUScWd7jbWP5gSRJzWcwkaSGycx/N0I9gbNH2HYxcPEw9TuAFw9TXwecsG8tlSRpJ7tySZIkSaqdwUSS1HZyVIbQS5L2JwYTSVLbCKdYlKTGMphIkiRJqp3BRJIkSVLtDCaSpLbjEBNJah6DSZvp7e3lhhtuYMOGDXU3RZLGnRMsSlJzGUzazMaNGznllFP44Q9/WHdTJEmSpFHjBIttZvbs2axatYqZM2fW3RRJkiRp1BhM2kxEMHv27LqbIUm1qiaxt1+XJDWJXbkkSW3DKCJJzWUwkSRJklQ7g4kkSZKk2hlMJEmSJNXOYCJJajtOsChJzWMwkSS1DSdYlKTmMphIkiRJqp3BRJIkSVLtDCaSpLaTDjKRpMYxmEiS2kY4yESSGstgIkmSJKl2BhNJkiRJtTOYSJIkSaqdwUSS1HbSKRYlqXEMJpIkSZJqZzCRJEmSVDuDiSRJkqTaGUwkSW3HCRYlqXkMJpKktuH8ipLUXAYTSZIkSbUzmEiSJEmqncFEkhoqIv5zRGREHFqeR0R8LiJ6IuKHEfHyln2XR8TPyrK8pX50RNxbXvO5iKozVUTMiohry/7XRsTB4/8JJUlNYjCRpAaKiHnAScAvW8qnAIvKchbwxbLvLOCjwCuAY4GPtgSNLwLvbnnd0lI/B7g+MxcB15fnkiTtNYNJmxkYGGDNmjVs37697qZI2r9dAPw5PGOK9GXA17JyKzAzIg4HTgauzcz1mbkBuBZYWrZNz8xbMzOBrwGnthzrkrJ+SUt9TAWOfpekpjKYtJm1a9cyf/58brnllrqbImk/FRHLgNWZec+QTUcAD7c8X1Vqu6qvGqYOMCczHynrjwJzRqf1kqSJqqvuBmjPzJo1i5UrV7J48eK6myKpRhFxHXDYMJs+DPwFVTeucZGZGREjziwSEWdRdR1j/vz549UsSVKbMZi0mUmTJnH88cfX3QxJNcvME4erR8T/AywE7inj1OcCP4iIY4HVwLyW3eeW2mrg+CH1m0p97jD7A6yJiMMz85HS5euxXbT1QuBCgCVLlozK1IhOsChJzWNXLklqkMy8NzOflZkLMnMBVferl2fmo8AK4Ixyd67jgE2lO9ZK4KSIOLgMej8JWFm2PRERx5W7cZ0BXFXeagUwePeu5S31MeUEi5LUXF4xkaSJ42rg9UAPsBl4F0Bmro+IjwO3l/3+KjPXl/X3AF8FDgCuKQvA+cAVEXEm8AvgbePxASRJzWUwkaQGK1dNBtcTOHuE/S4GLh6mfgfw4mHq64ATRq2hkqQJz65ckiRJkmpnMJEktZ3E0e+S1DQGE0lS23DsuyQ1l8FEkiRJUu0MJpIkSZJqZzCRJLUdJ1iUpOYxmEiS2oYTLEpScxlMJEmSJNXOYCJJkiSpdgYTSVLbcYiJJDWPwUSS1DbCmUwkqbEMJpIkSZJqZzCRJEmSVDuDiSRJkqTaGUwkSW0nnWFRkhrHYCJJahtOsChJzWUwkSRJklQ7g4kkSZKk2hlMJEltxxEmktQ8BhNJkiRJtTOYSJIkSaqdwUSSJElS7QwmkiRJkmpnMJEktR3nV5Sk5jGYSJLaRjjDoiQ1lsFEkiRJUu0MJpIkSZJqZzCRJLUfx5hIUuMYTCRJbcMRJpLUXAYTSRpDEfHqiJi/B/u/JCLOGMs2SZK0PzKYSNLYuhF4Z2shIj4YEetG2P8twN+PdaMkSdrfGEwkaWwN1/toCjBzzN4w4tyIWB0Rd5fl9S3bPhQRPRHxk4g4uaW+tNR6IuKclvrCiPh+qV8eEd2lPrk87ynbF4zV55EkTQwGE0lqpgsyc3FZrgaIiKOA04EXAUuBv42IzojoBL4AnAIcBby97AvwyXKs5wEbgDNL/UxgQ6lfUPYbN+nod0lqHIOJJE0cy4DLMnNbZv4c6AGOLUtPZj6Ymb3AZcCyqGYzfB1wZXn9JcCpLce6pKxfCZwQ4zD7ofMrSlJzGUwkqZneGxE/jIiLI+LgUjsCeLhln1WlNlL9EGBjZvYNqT/jWGX7prK/JEl7xWAiSW0oIq6LiPuGWZYBXwSeCywGHgH+pua2nhURd0TEHWvXrq2zKZKk/VhX3Q2QpAlg1AdEZOaJu7NfRHwZ+HZ5uhqY17J5bqkxQn0dMDMiuspVkdb9B4+1KiK6gBll/+HaeiFwIcCSJUtG5VykQ0wkqXG8YiJJY+/ciOgfXIC/BGitDd22LyLi8JanbwHuK+srgNPLHbUWAouA24DbgUXlDlzdVAPkV2RmUt3u+LTy+uXAVS3HWl7WTwNuKPuPKYeYSFJzecVEksbenv4+va+/4H8qIhaX4zwE/DFAZt4fEVcADwB9wNmZ2Q8QEe8FVgKdwMWZeX851geByyLiE8BdwEWlfhFwaUT0AOupwowkSXvNYCJJYygzx/3KdGa+YxfbzgPOG6Z+NXD1MPUHqe7aNbS+FXjrvrVUkqSd7MolSZIkqXYGE0lS23HsuyQ1j8FEktQ2xmEOR0lSTQwmkiRJkmpnMJEkSZJUO4OJJKntjMOUKZKkcWYwkSS1DYeYSFJzGUwkSZIk1c5gIkmSJKl2BhNJkiRJtTOYSJLajkPfJal5DCaSpLbh2HdJai6DiSRJkqTaGUwkSZIk1c5gIklqO86vKEnNYzCRJLUPZ1iUpMYymEiSJEmqncFEkiRJUu0MJpKktpPOZCJJjWMwkSS1DUeYSFJzGUwkSZIk1c5gIkmSJKl2BhNJkiRJtTOYSJLaj2PfJalxDCaSpLbh/IqS1FwGE0mSJEm1M5hIkiRJqp3BRJLUdhxiIknNYzCRJLWNjjLIJE0mktQ4BhNJUtvoKIPfB0wmktQ4BhNJUtuIcsXEYCJJzWMwkaQGioj/EBE/joj7I+JTLfUPRURPRPwkIk5uqS8ttZ6IOKelvjAivl/ql0dEd6lPLs97yvYF4/G57MolSc1lMJGkhomI1wLLgJdm5ouAvy71o4DTgRcBS4G/jYjOiOgEvgCcAhwFvL3sC/BJ4ILMfB6wATiz1M8ENpT6BWW/MWdXLklqLoOJJDXPnwLnZ+Y2gMx8rNSXAZdl5rbM/DnQAxxblp7MfDAze4HLgGVR9Zt6HXBlef0lwKktx7qkrF8JnBAx9tMfduzoyjXW7yRJGm8GE0lqnucDrypdrG6OiGNK/Qjg4Zb9VpXaSPVDgI2Z2Tek/oxjle2byv5jKrxiIkmN1VV3AyRJey4irgMOG2bTh6n+3z4LOA44BrgiIo4cx+Y9Q0ScBZwFMH/+/H061s4xJgYTSWoag4kktaHMPHGkbRHxp8A/ZvXb+20RMQAcCqwG5rXsOrfUGKG+DpgZEV3lqkjr/oPHWhURXcCMsv9wbb0QuBBgyZIl+5QoHPwuSc1lVy5Jap5/Bl4LEBHPB7qBx4EVwOnljloLgUXAbcDtwKJyB65uqgHyK0qwuRE4rRx3OXBVWV9RnlO235DjcBlj5+D3sX4nSdJ484qJJDXPxcDFEXEf0AssL6Hh/oi4AngA6APOzsx+gIh4L7AS6AQuzsz7y7E+CFwWEZ8A7gIuKvWLgEsjogdYTxVmxpzzmEhScxlMJKlhyp21/nCEbecB5w1Tvxq4epj6g1R37Rpa3wq8dZ8bu4e8XbAkNZddufYTmzZtore3t+5mSNJ+zTEmktRcBpP9QG9vL4sWLeIb3/hG3U2RpP1aR/mp5RUTSWoeu3LtB7q6uviHf/gHjjrqqN+8syRNYOEEi5LUWAaT/UBHRwcnnXRS3c2QpP1eh4PfJamx7MolSWobg4PfnWBRkprHYCJJahsdduWSpMYymEiS2kYM3i7YZCJJjWMwkSS1Da+YSFJzGUwkSW1j5zwmJhNJahqDiSSpbeyc+b3edkiSRp/BRJLUNsLbBUtSYxlMJEltY+cVE4OJJDWNwUSS1DZ2jjGpuSGSpFFnMJEktQ1nfpek5jKYSJLaRjj4XZIay2AiSWobg8Gk32QiSY1jMJEktY3uzurHlsFEkprHYCJJahtdJZhs7x+ouSWSpNFmMJEktY1JnVVfrl6DiSQ1jsFEktQ2JpUrJn39duWSpKYxmEiS2sYku3JJUmMZTCRJbWOwK5fBRJKax2AiSWobO6+Y2JVLkprGYCJJaht25ZKk5jKYSJLaRmdHEAF9BhNJahyDiSSprUzq7KDXrlyS1DgGE0lSW5nUEXblkqQGMphIktrKpK4Ou3JJUgMZTCRJbaWrw65cktREBhNJUlvp7rQrlyQ1kcFEktRWpkzqZOv2/rqbIUkaZQYTSVJbmTq5k829BhNJahqDiSQ1TERcHhF3l+WhiLi7ZduHIqInIn4SESe31JeWWk9EnNNSXxgR3y/1yyOiu9Qnl+c9ZfuC8fp8U7u7eHpb33i9nSRpnBhMJKlhMvMPMnNxZi4GvgX8I0BEHAWcDrwIWAr8bUR0RkQn8AXgFOAo4O1lX4BPAhdk5vOADcCZpX4msKHULyj7jYtp3V4xkaQmMphIUkNFRABvA75ZSsuAyzJzW2b+HOgBji1LT2Y+mJm9wGXAsvL61wFXltdfApzacqxLyvqVwAll/zE3bXIXT/d6xUSSmsZgIknN9SpgTWb+rDw/Ani4ZfuqUhupfgiwMTP7htSfcayyfVPZ/9dExFkRcUdE3LF27dp9/lDTurvYvM0rJpLUNF11N0CStOci4jrgsGE2fTgzryrrb2fn1ZLaZOaFwIUAS5Ys2ecJSKZO7nSMiSQ1kMFEktpQZp64q+0R0QX8PnB0S3k1MK/l+dxSY4T6OmBmRHSVqyKt+w8ea1V5rxll/zE3rbvqypWZjFPvMUnSOLArlyQ104nAjzNzVUttBXB6uaPWQmARcBtwO7Co3IGrm2qA/IrMTOBG4LTy+uXAVS3HWl7WTwNuKPuPuRkHTGIg4UmvmkhSo3jFRJKa6XSGdOPKzPsj4grgAaAPODsz+wEi4r3ASqATuDgz7y8v+yBwWUR8ArgLuKjULwIujYgeYH15v3Fx8LRuADY+vZ3pUyaN19tKksaYwUSSGigz3zlC/TzgvGHqVwNXD1N/kOquXUPrW4G37nND98Kjm7YAcNtD65l/yNQ6miBJGgN25ZIktZUZU6srJrf0PF5zSyRJo8lgIklqK699wWwA5h18QM0tkSSNJoOJJKmtHD7jADoCxmWkvSRp3BhMJEltpbMjOOTAyax5YmvdTZEkjSKDiSSp7TzroMn8+NEn626GJGkUeVcuSVLbuf9XT9TdBEnSKPOKiSSp7Sxb/GwAevsGam6JJGm0GEwkSW3nVYuqO3P9dI3duSSpKQwmkqS28/hT2wB44//4Xs0tkSSNFoOJJKntvOt3F9TdBEnSKDOY7Ae2b9/Om970Jq655pq6mzIhvOMd7+Ciiy6quxmS9sHkrs4d65nOaCJJTWAw2Q9EBHPmzOGAA5zFeDzMnj2bgw46qO5mSBolb/q83bkkqQm8XfB+oKuri6985St1N2PC+PSnP113EySNgkvPPJZ3XHQb96321sGS1AReMZEktaXBO3MB3PPwxhpbIkkaDQYTSVLbmtxV/Rhb9oVbam6JJGlfGUwkSW3r3nNP3rHuIHhJam8GE0lS2+ru2vljbOGHrq6xJZKkfWUwkSS1tUv+6Ngd66s3bqmxJZKkfWEwkSS1tdc8f+cg+N89/4YaWyJJ2hcGE0lS2+s575Qd6wvO+U6NLZEk7S2DiSSp7XV1dvDSeTN3PDecSFL7MZhIkhrhqrN/9xnPF5zzHZ7e1ldTayRJe8pgIklqjIfOf8Mznr/ooytZtWFzTa2RJO0Jg4kkqVEeOv8NnPnKhTuev/KTN7LgnO84O7wk7ecMJpKkxvnIG4/i/o+d/Izasi/cwoJzvsN373+0plZJknbFYCJJaqRpk7t46Pw38KU/PPoZ9bMuvZMF53yHh9fbxUuS9idddTdAkqSxtPTFh/HQ+W/gf/10LWdcfNuO+qs+deNvfO0xCw7msBkH8O9efgQJzJ15AIvmHDSGrZWkictgIkmaEF79/Nk8dP4bWHHPr/izb961W6+5/aENwAb+5Z5f7fH7/f27jmHGAZN47uwD6e7sYHJXBxEQEXt8LEmaCAwmkqQJ5c0vfTZvfumzGRhIBjL5zHU/4/M39oz6+7zr728f9WPuiwWHTOV3nnco82dN5bDpU5h/yFQ2bdnO0c85mK6OYFJnB09u7WPWtO66myppgjKYSJImpI6OoIPgAye/gA+c/IJd7puZRASZyU/WPMm196/hgUee4Jr72mcg/UPrNvPQul/W3YxhHTSli+lTJvFfTn4Bs6Z10zcwwJGHHshhM6aw5omtTO3uYta0bjo7qqtNg/89JDWLwUQArFmzhttvv52lS5fS1eU/C0lqNfhLcETwwsOm88LDpu/2a/sHks6OKtQ8ua2Pp7f1MW1yFwMDyc0/Xcv7Lrub+bOm8ssJPBj/ya19PLm1j/dffnfdTdltRx46jdUbt/DGlzyb3ztqDrMPmszaJ7dy7MJDeOzJrXRGMGfGFA7s7rILn7Sb/A1UANx2222cdtppPPbYY8yYMaPu5kjaBxGxGPgSMAXoA96TmbdF9ZvRZ4HXA5uBd2bmD8prlgP/tRziE5l5SakfDXwVOAC4GnhfZmZEzAIuBxYADwFvy8wN4/IB28zgX/kjgulTJjF9yqQd25YtPoJli4+oq2nAM68+ZCb9A8m6p3tZtWEzK+7+FQdO6eJ5zzqQ7z+4nh89+iT3PLyRVyycxb2rN7G5t7/WttfpwcefBuBbP1jFt36walzf+9iFs5gzfQqL583kObOmctSzp3P4jClkQt9A0t3lTVfVngwmktQ8nwI+lpnXRMTry/PjgVOARWV5BfBF4BUlZHwUWAIkcGdErChB44vAu4HvUwWTpcA1wDnA9Zl5fkScU55/cPw+okZL61/yI4KuzmDO9CnMmT6Fo58za8e2t7xsbh3NA+CJrds5YFInHRH85NEn6e0f4I6H1jO1u4vNvX08va2fO3+5gY2be3n8yW08/lQvZ736SK6+95EdAaJJbvv5eoC9uikDwKJnHcg3zzqOQ6Z188SWPqZN7qSrs4P+gSQz2do3wLTuTrZuHyACJnV20DcwQBBs2NzLIaVb3fb+Ksg+ta2PKZOqMUoRcPDUbro6gk1bttPd1cGBk7t2XDnc1jfA2ie3cfiMKQA73rezI3h4/WYOmzGFLdv7mT5lEk9v69vx/qs2bGHGAZOYPqWLzo5gc28/U7s76e0fYFLHziC2act2Dp7Wzfqne5k1rZut26vwnAkDmUzt7iQiGBhInu7t46CWPxRs3d5PRwTdXR1kJlu297O9LzlwShfl7ws7vi+DY9QGa5nVZ+jtH+CxJ7YxZ/oUOqIKihEQVNv6B5Ipkzro7qza/MTWvh3P1z3dW52v7i6e7u3jgEmdPL2tn97+Abb3D7C5t4/nzj6Qp7b1sb0/OXBy144Q+sTW7UydVP137OsfYN3TvcyZPmXHZ8vS1ie29DFjavWZ+/oHiAg6oqp3dgZPbt3OL9dt5kVHzODAyeMfExodTNavX88HPvABPvKRj7Bw4cLf/IIJbPHixXz5y19m6tSpdTdF0r5LYLCv0Qxg8LeXZcDXsvoJdWtEzIyIw6lCy7WZuR4gIq4FlkbETcD0zLy11L8GnEoVTJaV1wFcAtzEeASTa86BR+8d87fR/qW149xR5XHx0J0CmFYWgF/BBw4BDhnbtv0mA1n9Ajv4C+32/gG2ba9+0Vy/uZeujmDtU9sIYscvumNuI/R8Cvb2lg8//w3bf7Ebx1i9h3WANbtx3P3Frj7Hvnh8D/b9Tf+dduWKgefwR5+4fB+OsHcaHUy2b9/Oj370I7Zs2VJ3U/Z78+bN44wzzqi7GZJGx/uBlRHx11QT6f5OqR8BPNyy36pS21V91TB1gDmZ+UhZfxSYM1JjIuIs4CyA+fPn78XHkdpXRwQdLVelOrs6mdLVCcChB04G4MhDDxyV9xogCWBzbz+PP1VdOdrePzAqx9bEs71/gEmd49stsNHBZM6cOfzrv/5r3c2QpFEXEdcBhw2z6cPACcB/zMxvRcTbgIuAE8eqLWXMyYh/6s3MC4ELAZYsWbJvfxI+5fx9ernUZIO/Qg5eOHrOLvYdOrZodwfnZ0v3pb013Pu13vmutV27eq+hr9nbzzScgdIFK7O6g9+u2j4wkM/YZ1D/QNJRbnww0j77Ymhb+voHdoxp23nTDYZ938F9hztHW3r7Oa67c1TbursaHUwkqakyc8SgUbpcva88/Z/AV8r6amBey65zS201O7tlDdZvKvW5w+wPsCYiDs/MR0p3sMf26oNIqsXQsUV787rReO+htT1p19DX7O1nGk7HjptWDP+ew+07VGdLfbRDyXBt6Wq5utHVOXz7h9t3qANqCiWwM1xLkprjV8BryvrrgJ+V9RXAGVE5DthUumOtBE6KiIMj4mDgJGBl2fZERBxX7uh1BnBVy7GWl/XlLXVJkvaKV0wkqXneDXw2IrqArZTxHVR31Xo91ZjXzcC7ADJzfUR8HBicqvyvBgfCA7bGov8AAAzzSURBVO9h5+2CrykLwPnAFRFxJtVY17eN5QeSJDWfwUSSGiYzvwccPUw9gbNHeM3FwMXD1O8AXjxMfR3VWBZJkkaFXbkkSZIk1c5gIkmSJKl2BhNJkiRJtTOYSJIkSaqdwUSSJElS7QwmkiRJkmpnMJEkSZJUu6hua7+bO0espZpIS5L2xOMAmbm07oaoXqP0c+RQyr8p/RrPzfA8LyPz3IzMczO83T0vz8nM2Xty4D0KJpIk1S0i7sjMJXW3Y3/kuRme52VknpuReW6GN5bnxa5ckiRJkmpnMJEkSZJUO4OJJKndXFh3A/ZjnpvheV5G5rkZmedmeGN2XhxjomeIiJuA12RmjNLxzgU+Crw2M28ajWNKkiSpebxiIkmSJKl2BhNJUtuIiKUR8ZOI6ImIc+puz1iIiIsj4rGIuK+lNisiro2In5XHg0s9IuJz5Xz8MCJe3vKa5WX/n0XE8pb60RFxb3nN5yJiVK6Qj7WImBcRN0bEAxFxf0S8r9Q9NxFTIuK2iLinnJuPlfrCiPh++TyXR0R3qU8uz3vK9gUtx/pQqf8kIk5uqbftdy8iOiPiroj4dnnueQEi4qHy7/3uiLij1Or9PmWmywRZgDcD1wOPANuAXwE3A+8BFgA5wnJTyzFeS9W38AHgCWALcB9Vd60pQ97voZGOOWS/qcCHgLuBp4GngH8F3l73OXNxcdl/FqAT+DfgSKAbuAc4qu52jcHnfDXwcuC+ltqngHPK+jnAJ8v664FrgACOA75f6rOAB8vjwWX94LLttrJvlNeeUvdn3s3zcjjw8rJ+EPBT4CjPTVLae2BZnwR8v3yOK4DTS/1LwJ+W9fcAXyrrpwOXl/WjyvdqMrCwfN862/27B/wn4BvAt8tzz0vu+D3t0CG1Wr9PXjGZICLiLOAqqi/XvwB/A1wNHAC8C9gIfIydE599rGX5asuhPgicRBUi/g74CtALnAtcExGdLft+hir4AFwy5JiD7ZoJfA/4b0A/cHHZdzbwjYj4xD5+dEnNcSzQk5kPZmYvcBmwrOY2jbrM/F/A+iHlZVT/b6Q8ntpS/1pWbgVmRsThwMnAtZm5PjM3ANcCS8u26Zl5a1a/OXyt5Vj7tcx8JDN/UNafBH4EHIHnhvIZnypPJ5UlgdcBV5b60HMzeM6uBE4of81eBlyWmdsy8+dAD9X3rm2/exExF3gD1e8rlM854c/LLtT6feoa5Q+j/dcfUwWIl2bmY60bIuLQzNwInBsRx1PN1HnuCMd5D/Dz8o+s9RgfB/4rcBpwOUBmfqYEj9cAX83hB79/BngZ8MHM/FTL8aYA/wz8RURcmZl37+HnldQ8RwAPtzxfBbyipraMtzmZ+UhZfxSYU9aHOydH/Ib6qmHqbaV0sXkZ1ZUBzw1VdyXgTuB5wBeo/pK/MTP7yi6tn2fHOcjMvojYBBxS6re2HLb1Ne363fsM8OdUV9mg+pyel0oC342IBP4uMy+k5u+TV0wmlj5g+9BiZj6+uwcofxUY7lZuF5THk4fZNqyIOAT4Q+CO1lBS3mcr1dWZAP797h5Tkpqu/D94wt5SMyIOBL4FvD8zn2jdNpHPTWb2Z+ZiYC7VX/JfWHOTahcRbwQey8w7627LfuqVmfly4BTg7Ih4devGOr5PXjGZOL5O1X3rgYi4jKqL1S2ZuXZPDhIR04D3AW8Bnk/1F4jWwUx78telY6j6Z2ZUtxUealJ5/K09aaOkxloNzGt5PrfUJoI1EXF4Zj5SukgMXvke6ZysBo4fUr+p1OcOs39biIhJVKHk65n5j6XsuWmRmRsj4kbgt6m623SVqwOtn2fw3KyKiC5gBrCOXX/H2vG797vAmyPi9cAUYDrwWTwvAGTm6vL4WET8E1WgrfX75BWTCSIzPw0spxpD8mfAP1H947sxIpbszjHKD4QbgPOovuCXA/8fzxw3MnkPmnVIeTyGavD80OUvyvYD9+CYkprrdmBRuaNON9Xg1BU1t2m8rKD6fzjl8aqW+hnljjnHAZtKN4yVwEkRcXC5q85JwMqy7YmIOK70nT+j5Vj7tdLei4AflZ9pgzw3EbNL12ki4gDg96jG4NxI1cUafv3cDJ6z04Abyl/HVwCnl7tTLQQWUQ1gbsvvXmZ+KDPnZuYCqjbfkJn/LxP8vED1h+aIOGhwnep7cB91f59+0+h4l+YtwEyquyt8mWrA+Tpgdtl2E0PumtXyutOoLun9/TDbDmfIHbxK/dxSP36Y17yxbPt03efExcWlPZby/66fUvWf/3Dd7Rmjz/hNqrsnbqfql30m1R9yrgd+BlwHzCr7BjvHE9wLLGk5zh9RDdLtAd7VUl9SfgH5N+DzlMmW9/cFeGX5mfFDqhuw3F3+PXhu4CXAXeXc3Af8ZakfSfULdA/wP4HJpT6lPO8p249sOdaHy+f/CS13UWr37x7VX/UH78o14c9LOQf3lOX+wbbX/X1y5vcJLiIuovoHdVpmfisirqe6W0VXZvYP2fccqiskb87Mfxmy7XSqH6Y3Z+bxLfWPAH8FnJiZ1w95zbOofvj+IDOPGfUPJ0mSpLZhV64JIiJeO8LENs8qj5vL47ryOH+YfR8qj8cPOfaRwCdHeOsRj5fV3cG+DiyJiI8MudXw4LGfWy6bSpIkqcG8YjJBRMRGqokLb6UKGAG8imp8x53Ab2fm9oj4Y6rJhu6mmudkC/CLzLy09EG8m+pWhNdSXTaeT9Ul6zvAH/DrV0x+i+oy3lqqELIBIDM/UbZPp+qfeBzVZcPvAWuAZ1MNej+GaqLFy8bgtEiSJGk/YTCZICLiT6hu5ftS4DBgK9VA+G8CX8xqsqrB+6B/nGoA1zyqO7ftCBsRMQ84n+qqyeBsn5cAn6bqD/2MYFJe84fAB4AXUPXfJDOjZXs3cBbVbYFfVPZZQxVU/gW4NDPXIUmSpMYymEiSJEmqnWNMJEmSJNXOYCJJkiSpdgYTSZKkXYiImyLCvu/SGDOYSJIkSaqdwUSSJElS7QwmkiRpwoqIN0fE9RHxSERsi4hfRcTNEfGeiFhQunC9puybLctNQ44zNyI+HxEPluOsi4gVEXHMMO95bjnG8RGxPCLuiogtEfFYRFwcEYcN85ojI+LCiOgp+66PiHsj4ksRcchYnR9pPHm7YEmSNCFFxFnA3wGPUs2b9TjwLOAlVBMR/x7wfuCdwHOAj7W8/KHM/Go5zsuB71LN77USuB84FDgVOAB4S2Ze3fK+5wIfBVYAJwGXA48AryzLz4FXZObasv/hVJMVT6ea/PjHVHN+LQROKPveN0qnRaqNwUSSJE1IEXEn8GJgXmY+NmTboZn5eFm/CXhN6+TALft1UQWFucDJmXlzy7ZnA7dT9VBZkJnbSv1cqmCynSpU3NXymguowtDFmXlmqf0H4HPA+zPzs0PefxowkJlb9uFUSPsFu3JJkqSJrI8qIDzDYCjZDW8Angv8j9ZQUo7xK+BTwGFUVzaGurQ1lBTnApuAfx8Rk4ds+7XwkZlPG0rUFF11N0CSJKkmXwf+BnggIi4DbgZuGexCtZt+uzw+p1wJGWpRefwtqm5YrW4e8pzM3BQRd1ONa/kt4G6qLl//DfhCRJxM1V3sFuCBtOuLGsRgIkmSJqTM/HREPA68B/gzqi5UGRE3A/8lM+/YjcMMDjx/62/Y78BhamtG2PfR8jijtPMXEXEs1dWUpcDvl+0PR8RfZ+bndqOd0n7PrlySJGnCysyvZeZxVAHjDcBFwKuBlRExezcOsak8LsvM2MXysWFeO2eEYw7elWvw2GTmjzLzD0o7lwDnUP0e99mIOHM32int9wwmkiRpwsvMjZl5dWa+G/gq1R22Xl029wNEROcwL721PL5qL972NUMLETEDWAxsBX40TDv7MvPOzPwk8PZSPnUv3lva7xhMJEnShBQRr42IX7vTFtUtgwE2l8d15XH+MPteBfwbcHZEvH6E9/ntiJg6zKZ3RMTLhtTOperC9c2Wu3gdXQLLUINXXDYPs01qO44xkSRJE9U/AU9FxK3AQ1Rzl7wKOAa4E7iu7Hc91RiSf4yIq6nujvWLzLw0M7dHxO9TDUj/TkT8H6oB65uBeeVYRwKH8+sB4hrgloi4gmfOY/IQVVetQe8A/jgivkcVgjZQ3QnsTcA24DOjcTKkujmPiSRJmpAi4k+Ak4GXUo3r2Ar8Avgm8MXMfLLs1wl8HDidKmx0ATdn5vEtx3oW8J+AN1IFkQGqsHEP8M/AZZnZV/Y9l2oek9cCC6gG3b8AeAr4NvAXmflIy7FfQTXJ4++U9z8AWA38b+BvnFxRTWEwkSRJGketwSQzb6q3NdL+wzEmkiRJkmpnMJEkSZJUO4OJJEmSpNo5xkSSJElS7bxiIkmSJKl2BhNJkiRJtTOYSJIkSaqdwUSSJElS7QwmkiRJkmpnMJEkSZJUu/8Lh/N7rPa8bYoAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 1080x360 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GJbMS_sd--vo"
      },
      "source": [
        "The minimal energy is $2L^2$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Kj4_-LChLMin"
      },
      "source": [
        "# Thermodynamic constants at different temperatures"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uLK2kJOWwLwR"
      },
      "source": [
        "Until now we only trained as CRBM at a $T=1$. Train differnt CRBM at different $T$s.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "D45rMFAJLR8A"
      },
      "source": [
        "def train(T, crbm=None):\n",
        "    if crbm is None:\n",
        "        crbm = CRBM(kernel_dims=(2, 2))\n",
        "    \n",
        "    l_train = 10 ** 9\n",
        "    l_test = 10 ** 9\n",
        "    epoch = 0\n",
        "    while l_train > 10 ** -4:\n",
        "        # train\n",
        "        l_train = 0\n",
        "\n",
        "        for train_s, train_E in train_dataset:\n",
        "            l_train += crbm.train(train_s, train_E / T)\n",
        "        \n",
        "        l_train = l_train / N_TRAIN_BATCHES\n",
        "\n",
        "        # plot results\n",
        "\n",
        "        if epoch % 100 == 0:\n",
        "            l_test = crbm.compute_loss(states_test, E_phys_test / T)\n",
        "\n",
        "            display.clear_output(wait=True)\n",
        "\n",
        "            print(\n",
        "                f\"Temp: {T} | Epoch: {epoch} | loss test: {l_test}| loss train: {l_train}\"\n",
        "            )\n",
        "        epoch += 1\n",
        "   \n",
        "    return crbm   "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6fVXXJsbxRhz"
      },
      "source": [
        "Ts = np.linspace(1.5, 3.5, 25)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jKA6BFdb9n32"
      },
      "source": [
        "Train different CRBM's at different temperatures."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-X4-M258w5kI"
      },
      "source": [
        "#%%time\n",
        "#crbms = [train(T) for T in Ts]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4EpzESjt8tYj"
      },
      "source": [
        "Faster. Train a CRBM at a temperature, save it, and then modify it's temperature and retrain it at the new temperature."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ou7tzktTThL5"
      },
      "source": [
        "\"\"\"\n",
        "%%time\n",
        "crbm = train(Ts[0])\n",
        "crbms = [crbm]\n",
        "T_old = Ts[0]\n",
        "for T in Ts[1:]:\n",
        "    crbm_new = CRBM(kernel_dims=(2, 2))\n",
        "    \n",
        "    crbm_new.W.assign(crbm.W / T * T_old )\n",
        "    crbm_new.vbias.assign(crbm.vbias / T * T_old)\n",
        "    crbm_new.hbias.assign(crbm.hbias / T * T_old)\n",
        "    \n",
        "    W_old = crbm_new.W.numpy()\n",
        "    crbm = train(T, crbm_new)\n",
        "    W_new = crbm.W.numpy()\n",
        "    \n",
        "    crbms.append(crbm)\n",
        "\"\"\";"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KPt3v5JMQxNr"
      },
      "source": [
        "Alternatively, the CRBM can also be initialized directly with the analytical solution so that no further training is necessary."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iZUxauUUTpe1"
      },
      "source": [
        "crbms = []\n",
        "for T in Ts:\n",
        "    crbm = CRBM(kernel_dims=(2, 2))\n",
        "    crbm.set_analytical_param(1 / T)\n",
        "    crbms.append(crbm)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T-8wCYOXEl6e"
      },
      "source": [
        "# Montecarlo Simulations"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aon4yyLvEsDM"
      },
      "source": [
        "The functions performs gibbs steps with a CRBM and saves the energy and magnetization in arrays."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GIz4HrfXyCX0"
      },
      "source": [
        "def MC(crbm, states, steps, hidden_steps=10):\n",
        "    Es = np.empty(steps)\n",
        "    Ms = np.empty(steps)\n",
        "    \n",
        "    for i in range(steps):\n",
        "        states = crbm.gibbs_k(states, k=hidden_steps)\n",
        "        \n",
        "        # Get the state out of the gpu\n",
        "        state_np = states.numpy()\n",
        "        Es[i] = ising_energy(state_np)[0]\n",
        "        Ms[i] = (2 * state_np - 1).sum()\n",
        "              \n",
        "    return Es, Ms, states"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uiAgoIsXEXa3"
      },
      "source": [
        "This function will plot the thermodynamic constants"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7WxXnDVh06Ia"
      },
      "source": [
        "def plot_thermodynamics(Temp, Es, Cvs, mag, susceptibility):\n",
        "    # ----------------------------------------------------------------------\n",
        "    # Plot the Energy, Magnetization, Specific Heat and Susceptibility\n",
        "    # ----------------------------------------------------------------------\n",
        "\n",
        "    f = plt.figure(figsize=(18, 10), dpi=80, facecolor='w', edgecolor='k')\n",
        "    xlabel = \"T\"\n",
        "    \n",
        "    sp = f.add_subplot(2, 2, 1)\n",
        "    plt.xlabel(xlabel, fontsize=20)\n",
        "    plt.ylabel(\"Energy \", fontsize=20)\n",
        "    plt.scatter(Temp, Es)\n",
        "\n",
        "    sp = f.add_subplot(2, 2, 2)\n",
        "    plt.xlabel(xlabel, fontsize=20)\n",
        "    plt.ylabel(\"Magnetization \", fontsize=20)\n",
        "    plt.scatter(Temp, abs(np.array(mag)))\n",
        "    plt.axvline(x=2.269)\n",
        "    \n",
        "    sp = f.add_subplot(2, 2, 3)\n",
        "    plt.xlabel(xlabel, fontsize=20)\n",
        "    plt.ylabel(\"Specific Heat \", fontsize=20)\n",
        "\n",
        "    plt.scatter(Temp, Cvs)\n",
        "    plt.axvline(x=2.269)\n",
        "\n",
        "    sp = f.add_subplot(2, 2, 4);\n",
        "    plt.xlabel(xlabel, fontsize=20);\n",
        "    plt.ylabel(\"Susceptibility\", fontsize=20);\n",
        "    \n",
        "    plt.scatter(Temp, susceptibility)\n",
        "    plt.axvline(x=2.269)\n",
        "\n",
        "    plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SH35rZQhFSI2"
      },
      "source": [
        "Initialize the step that will be used during the MC. There is an oportunity for parallelization in the Gpu if severall states are updated in the simulation. For simplicity only one states is used.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O2FSWc2ImNI0"
      },
      "source": [
        "L = 25\n",
        "states = tf.random.uniform((1, L, L, 1))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eTbe0Ya6Fa6Q"
      },
      "source": [
        "We start with the highest temperature and slowly lower it. First there is a warmup phase and then thermodynamic constants are recorded."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_krDMaEex-MW",
        "outputId": "da22f5f1-65f5-4d2e-a84b-43abd46ebd9e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 982
        }
      },
      "source": [
        "%%time\n",
        "\n",
        "energy = []\n",
        "magnetization = []\n",
        "CV = []\n",
        "susceptibility = []\n",
        "\n",
        "e, m, states = MC(crbms[-1], states, 2 * 10 ** 4)\n",
        "for i, crbm in enumerate(crbms[::-1]):\n",
        "    # Warmup\n",
        "    e, m, states = MC(crbm, states, 10 ** 3)\n",
        "    \n",
        "    # Mc Simulation\n",
        "    e, m, states = MC(crbm, states,  2 * 10 ** 4)\n",
        "    \n",
        "    # Computing expectaion values\n",
        "    e1 = e.mean()\n",
        "    e2 = (e ** 2).mean()\n",
        "    \n",
        "    m1 = m.mean()\n",
        "    absm1= abs(m).mean()\n",
        "    m2 = (m ** 2).mean()\n",
        "    \n",
        "    \n",
        "    energy.append(e1 / L ** 2)\n",
        "    magnetization.append(m1 / L ** 2)\n",
        "    CV.append( (e2 - e1 ** 2) / Ts[-i - 1] ** 2 / L ** 2 )\n",
        "    susceptibility.append( (m2 - absm1 ** 2) / Ts[-i - 1] / L ** 2 )\n",
        "    \n",
        "    \n",
        "    display.clear_output(wait=True)\n",
        "    print(\"Temp: \", Ts[-i -1])\n",
        "    plt.imshow(states.numpy()[0, :, :, 0], vmin=0, vmax=1, cmap=\"gray\")\n",
        "    plt.show()\n",
        "    \n",
        "    # Plot thermodynamics\n",
        "    plot_thermodynamics(Ts[-i - 1:], energy[::-1], CV[::-1], magnetization[::-1], susceptibility[::-1])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Temp:  1.5\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAKDklEQVR4nO3dT4gcBRqG8ffdRC/qIZJmCDG740pYmMtGaYKwQSLuSuIlehFzWHIQxkMEBS/Bi14WvKh7ESFiSA7+QVDXHMKuIQjuwiK2EjQaJEFGTBgzHXIwN0ny7WEqSydOp3u6qruq8z0/CN1d1Zn6UuShurtqZhwRAnDz+03dAwCYDGIHkiB2IAliB5IgdiCJtZPc2Pr162N2dnaSmwRSWVhY0Pnz573SuonGPjs7q06nM8lNAqm02+2+60q9jLe9w/Z3tk/b3lfmawEYr5Fjt71G0muSdkqak7Tb9lxVgwGoVpkj+1ZJpyPi+4j4RdK7knZVMxaAqpWJfaOkH3senymWXcP2vO2O7U632y2xOQBljP3UW0Tsj4h2RLRbrda4NwegjzKxn5W0qefxXcUyAA1UJvbPJW22fbftWyU9IelwNWMBqNrI59kj4pLtpyX9S9IaSQci4pvKJgNQqVIX1UTEEUlHKpoFwBhxbTyQBLEDSRA7kASxA0kQO5AEsQNJEDuQBLEDSRA7kASxA0kQO5AEsQNJEDuQBLEDSRA7kMREf0kEpp+94i8buUZETGASrBZHdiAJYgeSIHYgCWIHkiB2IAliB5IgdiAJYgeS4KIarAoXzEwvjuxAEsQOJEHsQBLEDiRB7EASxA4kQexAEsQOJEHsQBKlrqCzvSDpoqTLki5FRLuKoQBUr4rLZR+MiPMVfB0AY8TLeCCJsrGHpI9tf2F7fqUn2J633bHd6Xa7JTcHYFRlY98WEfdJ2ilpr+0Hrn9CROyPiHZEtFutVsnNARhVqdgj4mxxuyTpQ0lbqxgKQPVGjt32bbbvuHpf0sOSTlQ1GIBqlfk0fkbSh8VvCFkr6e2I+GclUwGo3MixR8T3kv5Y4SwAxohTb0ASxA4kQexAEsQOJEHsQBLEDiRB7EASxA4kQexAEsQOJEHsQBLEDiRB7EASxA4kQexAEsQOJEHsQBLEDiRB7EASxA4kQexAEsQOJEHsQBLEDiRB7EASxA4kQexAEsQOJEHsQBLEDiRB7EASxA4kQexAEsQOJEHsQBIDY7d9wPaS7RM9y+60fdT2qeJ23XjHBFDWMEf2g5J2XLdsn6RjEbFZ0rHiMYAGGxh7RHwq6cJ1i3dJOlTcPyTp0YrnAlCxUd+zz0TEYnH/J0kz/Z5oe952x3an2+2OuDkAZZX+gC4iQlLcYP3+iGhHRLvVapXdHIARjRr7OdsbJKm4XapuJADjMGrshyXtKe7vkfRRNeMAGJdhTr29I+m/kv5g+4ztJyW9JOkvtk9J+nPxGECDrR30hIjY3WfVQxXPAmCMuIIOSILYgSSIHUiC2IEkiB1IgtiBJIgdSGLgeXbkYrv011j+dombxzD7ZBr+zRzZgSSIHUiC2IEkiB1IgtiBJIgdSILYgSQ4zz5m03aOtkmzNMXNsk84sgNJEDuQBLEDSRA7kASxA0kQO5AEsQNJEDuQBBfVjNnNckEGph9HdiAJYgeSIHYgCWIHkiB2IAliB5IgdiAJYgeSIHYgiYGx2z5ge8n2iZ5lL9o+a/t48eeR8Y4JoKxhjuwHJe1YYfmrEbGl+HOk2rEAVG1g7BHxqaQLE5gFwBiVec/+tO2vipf56/o9yfa87Y7tTrfbLbE5AGWMGvvrku6RtEXSoqSX+z0xIvZHRDsi2q1Wa8TNAShrpNgj4lxEXI6IK5LekLS12rEAVG2k2G1v6Hn4mKQT/Z4LoBkG/vAK2+9I2i5pve0zkl6QtN32FkkhaUHSU2OcEUAFBsYeEbtXWPzmGGYBMEZcQQckQexAEsQOJEHsQBLEDiRB7EASxA4kQexAEsQOJEHsQBLEDiRB7EASxA4kQexAEsQOJEHsQBIDf3gFcrF9w/URMaFJUDWO7EASxA4kQexAEsQOJEHsQBLEDiRB7EASxA4kwUU1uAYXzdy8OLIDSRA7kASxA0kQO5AEsQNJEDuQBLEDSRA7kASxA0kMjN32Jtuf2P7W9je2nymW32n7qO1Txe268Y8LYFTDHNkvSXouIuYk3S9pr+05SfskHYuIzZKOFY8BNNTA2CNiMSK+LO5flHRS0kZJuyQdKp52SNKj4xoSQHmres9ue1bSvZI+kzQTEYvFqp8kzfT5O/O2O7Y73W63xKgAyhg6dtu3S3pf0rMR8XPvulj+VqkVv10qIvZHRDsi2q1Wq9SwAEY3VOy2b9Fy6G9FxAfF4nO2NxTrN0haGs+IAKowzKfxlvSmpJMR8UrPqsOS9hT390j6qPrxAFRlmB9e8SdJf5X0te3jxbLnJb0k6T3bT0r6QdLj4xkRQBUGxh4R/5HU73cCPVTtOADGhSvogCSIHUiC2IEkiB1IgtiBJIgdSILYgSSIHUiC2IEkiB1IgtiBJIgdSILYgSSIHUiC2IEkiB1IgtiBJIgdSILYgSSIHUiC2IEkiB1IgtiBJIgdSILYgSSIHUiC2IEkiB1IgtiBJIgdSILYgSSIHUiC2IEkHBGT25jdlfRDz6L1ks5PbIDypmneaZpVmq55mzzr7yKitdKKicb+q43bnYho1zbAKk3TvNM0qzRd807TrL14GQ8kQexAEnXHvr/m7a/WNM07TbNK0zXvNM36f7W+ZwcwOXUf2QFMCLEDSdQWu+0dtr+zfdr2vrrmGIbtBdtf2z5uu1P3PNezfcD2ku0TPcvutH3U9qnidl2dM/bqM++Lts8W+/i47UfqnPEq25tsf2L7W9vf2H6mWN7Y/dtPLbHbXiPpNUk7Jc1J2m17ro5ZVuHBiNjS0POrByXtuG7ZPknHImKzpGPF46Y4qF/PK0mvFvt4S0QcmfBM/VyS9FxEzEm6X9Le4v9qk/fviuo6sm+VdDoivo+IXyS9K2lXTbNMvYj4VNKF6xbvknSouH9I0qMTHeoG+szbSBGxGBFfFvcvSjopaaMavH/7qSv2jZJ+7Hl8pljWVCHpY9tf2J6ve5ghzUTEYnH/J0kzdQ4zpKdtf1W8zG/cy2Lbs5LulfSZpnD/8gHdcLZFxH1aftux1/YDdQ+0GrF8frXp51hfl3SPpC2SFiW9XO8417J9u6T3JT0bET/3rpuS/Vtb7Gclbep5fFexrJEi4mxxuyTpQy2/DWm6c7Y3SFJxu1TzPDcUEeci4nJEXJH0hhq0j23fouXQ34qID4rFU7V/pfpi/1zSZtt3275V0hOSDtc0yw3Zvs32HVfvS3pY0okb/61GOCxpT3F/j6SPapxloKvhFB5TQ/axbUt6U9LJiHilZ9VU7V+pxivoilMrf5e0RtKBiPhbLYMMYPv3Wj6aS9JaSW83bVbb70jaruVvvTwn6QVJ/5D0nqTfavnbih+PiEZ8KNZn3u1afgkfkhYkPdXznrg2trdJ+rekryVdKRY/r+X37Y3cv/1wuSyQBB/QAUkQO5AEsQNJEDuQBLEDSRA7kASxA0n8D1k5bCqq4vBkAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABKcAAAKZCAYAAABtIEiaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAMTQAADE0B0s6tTgAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXRTdf7/8ddN2oKtyjQUkS9pGhDK9kVAW7ZRARlcRqkcEUVEqLXCoI7j1I1RwZ1BnW9HvjoqAsXpF3UcFrWOCwoKgoAlTmUEdVgkacuidnGQhqVN7u8Pf2TstLRNmyZdno9zek5v7qc3r5ymSfq+n/v+GKZpmgIAAAAAAAAiwBLpAAAAAAAAAGi/KE4BAAAAAAAgYihOAQAAAAAAIGIoTgEAAAAAACBiKE4BAAAAAAAgYihOAQAAAAAAIGIoTgEAAAAAACBiWk1xyu/369e//rXOOuss9erVS88888xJxx47dky33nqrevfurYEDB2rq1KlhTAoAAAAAAICGiop0gIZatmyZvvjiC+3cuVP/+te/NGTIEI0ZM0YDBgyoMXb27NkyDEM7d+6UYRg6ePBgBBIDAAAAAACgPoZpmmakQzTEZZddpuuvv16TJ0+WJN19992KiYnRo48+Wm1cRUWFunXrpuLiYp1++umRiAoAAAAAAIAGajUzpwoLC5WUlBTYdjqd2rJlS41xe/bskc1m07x587RmzRqdcsopevDBBzV27Nh676NDhw7q0qVLSHMDAICW5bvvvtOxY8ciHQM/wWcwoHYlh398rUo4tUOEkwBA09T3+avFFKdGjBihXbt21bqvoKCgwcepqqqSx+NR//79NX/+fBUUFGjcuHHasWOHunbtWm1sdna2srOzA9unnHKKiouLG/cAAABAq2C32yMdAf+hS5cufAYDajEue70k6f2sURFOAgBNU9/nrxZTnNq8eXOd+x0Ohzwej0aMGCFJcrvdcjgctY6zWCy67rrrJElDhgxRjx499Pnnn9coTmVlZSkrKyuwzYdVAAAAAACA8Go1q/VNmjRJixYtks/nU1lZmV599VVdc801NcYlJCRo7NixWr16tSRp79692rt3r/r16xfuyAAAAAAAAKhHqylOXX/99erbt6969+6t1NRUZWVlaeDAgZKkvLw8ZWZmBsY+//zzevLJJzVw4EBNmDBBCxcuVPfu3SMVHQAAAAAAACfRYi7rq4/VatWf/vSnWvelpaUpLS0tsN2zZ099+OGH4YoGAADQpt12223Ky8uTx+NRQUGBBg8eXOu4JUuWaP78+fL7/brwwgv17LPPKjo6OsxpAQBAa9NqZk4BAAAgMq666ipt3Lix2srJ/2nv3r2aM2eONmzYoN27d+ubb77RCy+8EMaUAACgtaI4BQAAgDpdcMEF9S4cs2LFCqWlpenMM8+UYRj61a9+pVdeeSVMCQEAQGvWai7rAwAAbZ9pmnJ5yuUuqZAzIU4pSfEyDCPSsdAAhYWF1WZWOZ1OFRYWRiwPzyUAAFoPilMAAKBFKC73alpOvorKvIq2WlTp8yvRFqvcjKGyx8dGOh5CKDs7W9nZ2YHtw4cPh/T4PJcAAGhduKwPAABEnGmampaTL0+pV5U+U97jPlX6THlKvZqeky/TNCMdEfVwOBzyeDyBbbfbLYfDUevYrKwsFRcXB75OPfXUkOVojueSaZra6i7TcleRtrrLeD4CABBizJwCAAAR5/KUq7jsiHz+6v/0+/ymCsu8cnnKleq0RSgdGmLixIk677zz9OCDD6pr1656/vnnNXny5LDnCPVziVlYAAA0P2ZOAQCARgvVjBJ3SYWirLX3A4q2WuQuqWhKTDTRzJkzZbfbVVxcrIsvvli9evWSJGVmZiovL0+S1LNnTz300EP6+c9/rl69eqlLly6aOXNm2LOG8rnUXDP6mIkFAEB1zJwCAACNEsoZJc6EOFX6/LXuq/T55UyIC0VkNNLChQtrvX3x4sXVtm+66SbddNNN4Yh0UqF8LjXHjD5mYgEAUBMzpwAAQNBCPaMkJSleibZYWS3VZ7xYLYYctlilJMWHMj7asFA+l0I9o49+WAAA1I7iFAAACFpDZpQEwzAM5WYMVVLnWEVbDcXGWBVtNeTsHKvcG4fJMGovEAD/KZTPpVDP6Av1301xuVdjs9dryqIteiBvh6Ys2qKx2etVXO4N6jgAAEQal/UBAICgnZhRctxXc9+JGSXBXu5kj4/V2qxRcnnK5S6pkDMhTilJ8RSmELRQPZdOzMLylHqrFZQaO6MvlH83P52F5fObqvT9eNATs7DWZI3ibwcA0GpQnAIAoJ0xTbPJ/7Q3V48owzCU6rSxMh+aLBTPpROzsP6zR5TD1rgZfS29H1YoXhsAAGgMilMAALQjoWrGHOoZJUBLFcoZfaH8uwn17EUatQMAIomeUwAAtBOhbMZMjyi0JydmYU1KSVSq09bo53dL7YfVHI3aAQAIBjOnAABoJ0J9GRA9ooDgtcR+WM1xiSAAAMFg5hQAAO3EicuAanPiMqBghWpGCdCehOLvJpSzsJrjtQEAgGAwcwoAgFagJTcxBxAZoZqF1RyvDTRXBwAEg+IUAAAtHE3MAZxMKFYlDPVrA83VAQDB4rI+AABaMJqYA2huoXxtoLk6AKAxmDkFAEALRhNzAOEQqtcGmqsDABqD4hQAAC3YiUbFx301951oVBzsP3qhuAwIQNsTiteG5njNAgC0fRSnAABoJjQxB9De8JoFAGgMilMAADQDmpgDaI+a4zWLlf8AoO2jIToAACFGE3MA7VWoX7OKy70am71eUxZt0QN5OzRl0RaNzV6v4nJvMz0CAEAkMHMKAIAQo4k5gPYsVK9ZPy30+/ymKn0/NrI6UehfkzWK10EAaCMoTgEAEGI0MQfQ3oXiNYuV/wCg/eCyPgAA/j/TNLXVXablriJtdZcFdfndT9EQGACa7kShvzYnCv0AgLaBmVMAACh0DcwlmpgDQChQ6AeA9oOZUwCAdi+UDcwlmpgDQCicKPRbLdVfMyn0A0Dbw8wpAEC71xx9TWhiDgBNc6LQ/5+zWh02Cv0A0Na0muKU3+/Xb37zG7399tsyDEO33367br311lrHvv3227r//vvl9/tVVVWlu+66S9OnTw9zYgBAa9EcDcwlmpgDQFM1R6HfNE1OHABAC9NqilPLli3TF198oZ07d+pf//qXhgwZojFjxmjAgAHVxpmmqalTp2rdunU6++yz5Xa71bdvX1155ZU67bTTIpQeANCS0dcEAFquUBb6Q9lfEAAQOq2m59Srr76qm266SVarVTabTddcc41eeeWVWscahqHvv/9eknTo0CF17txZHTp0CGdcAECYhGKFPfqaAEDbF+r+ggCA0Gk1M6cKCwuVlJQU2HY6ndqyZUuNcYZh6NVXX9WVV16puLg4lZeXa9WqVYqJiakxNjs7W9nZ2YHtw4cPN094AECzCNUZcPqaAEDb1xz9BQEAodFiilMjRozQrl27at1XUFDQ4ONUVVXp0Ucf1apVq3TBBRdo69atSktL0+eff66EhIRqY7OyspSVlRXYttvtjQsPAAi7n54B9/lNVfp+bBh14gz4mqxRQRWVaGAOAG1bc/UXBAA0XYspTm3evLnO/Q6HQx6PRyNGjJAkud1uORyOGuM+++wz7d+/XxdccIEkKTU1VXa7XQUFBRo3blzogwMAIqI5zoDTwBwA2i76CwJAy9Vqek5NmjRJixYtks/nU1lZmV599VVdc801NcYlJibqwIED+vLLLyVJu3fv1p49e9SnT59wRwYANKMTZ8Brc+IMOAAAJ9BfEABarhYzc6o+119/vbZu3arevXvLMAxlZWVp4MCBkqS8vDzl5eVp8eLF6tq1q1544QVdffXVslgs8vv9euaZZ2qdZQUAaL04Aw4ACAb9BQGg5TJMlqUIsNvtKi4ujnQMAEADmKapsdnrAz2nTrBaDDk7xwbdcwrtB+/3LQ+/E4STaZqtpr/guOz1kqT3s0ZFOAkANE197/Wt5rI+AEDbYZqmtrrLtNxVpK3uskYt333iDHhS51hFWw3FxlgVbf2xMMUZcADAyZzoLzgpJVGpThvvFwDQArSay/oAAG1Dcbm3xiUVibZY5WYMlT0+NqhjscIeAAAA0PoxcwoAEDamaWpaTr48pV5V+kx5j/tU6TPlKfVqek5+o2dQcQYcAAAAaL0oTgEAwsblKVdx2ZFqPaIkyec3VVjmlctTHqFkAAAAACKF4hQAIGzcJRWKstY+synaapG7pCLMiQAAAABEGsUpAEDYOBPiVOnz17qv0ueXMyEuzIkAAAAARBrFKQBA2KQkxSvRFiurpfrsKavFkMMWq5Sk+AglAwAgeKFYfRYAwGp9AIAwMgxDuRlDa6zW57DFKvfGYTQzBwC0GqFcfRYA2juKUwCABjFNUy5PudwlFXImxCklKb5RxSR7fKzWZo0KybEAAIiEn64+6/ObqvT5JCmw+uyarFG8rwFAEChOAQDqFeqzw4ZhKNVpU6rT1gxpAQBoXg1ZfZb3OABoOHpOAQDq9NOzw5U+U97jPlX6zMDZYfprAADaG1afBYDQojgFAKhTQ84OA2jbdu3apZEjRyo5OVmpqanasWNHjTF+v19ZWVnq37+/zj77bI0ZM0a7d++OQFqg+bH6LACEFsUpAECdODsMYObMmZoxY4Z27type+65R+np6TXG5OXl6eOPP9a2bdv0j3/8Q2PHjtW9994b/rBAGLD6LACEFsUpAECdODsMtG/ffvutXC6Xpk6dKkmaOHGiioqKasyKMgxDx44d09GjR2Wapg4dOiS73R6JyECzO7H6bFLnWEVbDcXGWBVtNeTszOqzANAYNEQHANTpxNnhEysSncDZYaB9KCoqUrdu3RQV9ePHRsMw5HA4VFhYqF69egXGjR8/Xh9++KHOPPNMnXbaaerevbvWr19f6zGzs7OVnZ0d2D58+HDzPgigGbD6LACEDjOnAAB14uwwgIZwuVzavn279u3bp/3792vs2LH61a9+VevYrKwsFRcXB75OPfXUMKcFQuPE6rOTUhKV6rTxnggAjcTMKQBow0zTDMkZXc4OA+1XYmKiDhw4oKqqKkVFRck0TRUWFsrhcFQbl5ubqwsvvFA/+9nPJEnTp0/XRRddFInIAACglaE4BQBtVHG5V9Ny8lVU5lW01aJKn1+JtljlZgyVPT426OOdODuc6rQ1Q1oALdUZZ5yhc845R8uWLVN6erpWrlwpu91e7ZI+SerZs6fefvtt3XnnnYqJidHf/vY3/fd//3eEUgMAgNaEy/oAoA0yTVPTcvLlKfWq0mfKe9ynSp8pT6lX03PyZZpm/QcBgP9v4cKFWrhwoZKTkzV//nwtXbpUkpSZmam8vDxJ0i233KIePXpo0KBBOvvss7V27Vo999xzkYwNAABaCWZOAUAb5PKUq7jsSLUG5pLk85sqLPPK5SlnBhSABuvTp482b95c4/bFixcHvu/QoYMWLVoUzlgAAKCNoDgFAG2Qu6RCUVZDx30190VbLXKXVFCcAtqgH374Qc8++6zWrFmjffv26ejRo7WOMwxDe/bsCXM6AACA2lGcAoA2yJkQp0qfv9Z9lT6/nAlxYU4EoLnt379f5513njweT72X7rKYAQAAaEkoTgFAG5SSFK9EW6w8pd5ql/ZZLYYctlilJMVHMB2A5nDvvffK7XZr8ODBmj17tvr166fTTz890rEAAADqRXEKANogwzCUmzG0xmp9Dluscm8cxqwJoA1avXq1unbtqg8//FCdOnWKdBwAAIAGozgFAG2UPT5Wa7NGyeUpl7ukQs6EOKUkxVOYAtqo8vJy/fKXv6QwBQAAWh2KUwDQhhmGoVSnjebnQDuQmJgov7/2XnMAAAAtmSXSAQAA/2aapra6y7TcVaSt7rJ6mxoDwAlXXXWVNmzYoIqKikhHAQAACAozpwCghSgu99boEZVoi1VuxlDZ42MjHQ9ACzdnzhy98847uvrqq7V06VKdccYZkY4EAADQIBSnAKAFME1T03LyA6vrVfp8kiRPqVfTc/K1JmsUvaIA1OnWW2/VWWedpddee029evVSSkqKHA6HLJaaE+UNw9CSJUsikBIAAKCmVlOceuuttzR37lxt375ds2bN0lNPPXXSsbt27dL06dNVUlKiTp066cUXX9SAAQPCmBYAguPylKu47Ih8/uqX8fn8pgrLvHJ5yukbBaBOL774YqCIffjwYa1bt+6kYylOAQCAlqTVFKd69+6tnJwcLV++XIcPH65z7MyZMzVjxgylp6drxYoVSk9P19atW8OUFACC5y6pUJTV0HFfzX3RVovcJRUUpwDUaenSpZGOAAAA0CitpjiVnJwsSXrttdfqHPftt9/K5XLpvffekyRNnDhRt956q3bv3q1evXo1e04AaAxnQpwqfbWvslXp88uZEBfmRABam+nTp0c6AgAAQKO0udX6ioqK1K1bN0VF/Vh3MwxDDodDhYWFEU4GACeXkhSvRFusrJbqfaWsFkMOW6xSkuIjlAwAAAAAmleLmTk1YsQI7dq1q9Z9BQUFSkxMDPl9ZmdnKzs7O7Bd3+WCANBcDMNQbsbQGqv1OWyxyr1xGM3QAQTl+PHj+vTTT7Vv3z5JUvfu3XXuuecqJiYmwskAAABqajHFqc2bN4fkOImJiTpw4ICqqqoUFRUl0zRVWFgoh8NRY2xWVpaysrIC23a7PSQZAKAx7PGxWps1Si5PudwlFXImxCklKZ7CFIAGq6qq0kMPPaSnn35aP/zwQ7V9p512mm677TbNnTs3MMMcAACgJWhzn0zOOOMMnXPOOVq2bJnS09O1cuVK2e12+k0BaBUMw1Cq00bzcwBB8/v9SktL0+rVq2WapuLj49WjRw9J0t69e1VeXq7HHntMn376qd58801ZLG2uuwMAAGilWs2nkrVr18putys7O1tLliyR3W5XXl6eJCkvL0+ZmZmBsQsXLtTChQuVnJys+fPns3oNAABo8xYvXqx3331XSUlJWrFihUpLS+VyueRyuVRaWqqVK1cqKSlJ7777rpYsWRLpuAAAAAGGaZpmpEO0FHa7XcXFxZGOAaCVMU2TS/GAVqStvt+fd955Kigo0I4dO+R0Omsds3fvXg0YMEDnnHOONm7cGN6AdWirvxOgqcZlr5ckvZ81KsJJAKBp6nuvb3OX9QFAOBWXe2s0MU+0xSo3Y6js8bGRjgegHdm+fbtGjx590sKUJPXo0UMXXnhhiypMAQAAtJrL+gCgpTFNU9Ny8uUp9arSZ8p73KdKnylPqVfTc/LFxFQA4XTs2DF16tSp3nGnnXaajh07FoZEAAAADUNxCgAayeUpV3HZEfn81YtQPr+pwjKvXJ7yCCUD0B4lJiZq8+bN8vl8Jx3j8/m0ZcsWVigGAAAtCsUpAGgkd0mFoqy195aKtlrkLqkIcyIA7dnFF1+swsJC/eY3v1FlZWWN/cePH9dtt92mwsJCXXrppRFICAAAUDt6TgFAIzkT4lTp89e6r9LnlzMhLsyJALRns2fP1ssvv6znnntOb7zxhiZPnqwePXpIkr7++mu9+uqr2r9/v2w2m+65554IpwUAAPg3ilMA0EgpSfFKtMXKU+qtdmmf1WLIYYtVSlJ8BNMBaG+6d++ud999V5MmTVJhYaGys7Or7TdNUw6HQytWrFD37t0jlBIAAKAmilMA0EiGYSg3Y2iN1foctljl3jhMhlH7JX8A0FxSU1O1c+dOLV++XOvWrdO+ffsk/Vi4Gj16tCZNmqSYmJgIpwQAAKiO4hQANIE9PlZrs0bJ5SmXu6RCzoQ4pSTFU5gCEDExMTG67rrrdN1110U6CgAAQINQnAKAJjIMQ6lOm1KdtkhHAQAAAIBWh9X6AAAAAAAAEDHMnAIAAGiFevbsKcMwtGbNGvXo0UM9e/Zs8M8ahqE9e/Y0YzoAAICGozgFAADQCrndbhmGocrKysB2Q9EXDwAAtCQUpwC0S6Zp0sQcQKu2d+9eST+uxPfTbQAAgNaG4hSAdqe43KtpOfkqKvMq2mpRpc+vRFuscjOGyh4fG+l4ANAgSUlJdW4DAAC0FjREB9CumKapaTn58pR6Vekz5T3uU6XPlKfUq+k5+TJNM9IRAaBRPvroI+3cubPecbt27dJHH30UhkQAAAANQ3EKQLvi8pSruOyIfP7qRSif31RhmVcuT3mEkgFA04wePVqPP/54veOeeOIJjRkzJgyJAAAAGobiFIB2xV1SoShr7b2loq0WuUsqwpwIAEKH2Z8AAKA1ojgFoF1xJsSp0uevdV+lzy9nQlyYEwFAeJWXl6tjx46RjgEAABBAQ3QA7UpKUrwSbbHylHqrXdpntRhy2GKVkhQfwXQAEJzCwsJq24cPH65x2wlVVVXasWOH3nvvPZ111lnhiAcAANAgFKcAtCuGYSg3Y2iN1foctljl3jhMhlH7JX8A0BI5nc5qr1srV67UypUr6/wZ0zQ1derU5o4GAADQYEEVp8rLyxUfz6wCAK2bPT5Wa7NGyeUpl7ukQs6EOKUkxVOYAtDqOByOwGtXYWGhYmNjlZCQUOvYmJgY2e12TZw4UbNmzQpnTAAAgDoFVZyy2+2aMmWKbrnlFg0ePLi5MgFAszMMQ6lOm1KdtkhHAYBGc7vdge8tFosmTZqknJycyAUCAABohKAaoh8/flxLlizRueeeq/PPP1+vvvqqqqqqmisbAAAAGmjp0qW68cYbIx0DAAAgaEEVpwoLCzVnzhx17dpVH3/8saZMmSKHw6GHHnpIBw8ebK6MAAAAqMf06dP185//vFmOvWvXLo0cOVLJyclKTU3Vjh07ah33+eefa/To0erXr5/69eunVatWNUseAADQtgRVnOrWrZseeughFRYW6uWXX9bIkSN18OBBPfzww0pKStK1116rjRs3NldWAAAARMDMmTM1Y8YM7dy5U/fcc4/S09NrjPF6vbriiiv06KOP6ssvv9T27dt1/vnnhz8sAABodYIqTp0QFRWlyZMna8OGDdq2bZsyMzMVExOjV199VaNGjdKQIUO0ZMkSHT16NNR5AQAAcBKVlZX6n//5Hw0fPlzx8fGyWq21fkVFNbzt6LfffiuXyxVY4W/ixIkqKirS7t27q417+eWXNXz4cJ133nmSJKvVqi5duoTuwQEAgDarUcWpnxo4cKAWLlyo4uJi3XnnnTJNU//4xz80Y8YMde/eXXPmzNGhQ4dCkRUAAAAncezYMY0ZM0Z333238vPz9a9//Uumadb65ff7G3zcoqIidevWLVDQMgxDDodDhYWF1cZ98cUX6tChgy6//HINHjxY06ZN03fffRfSxwgAANqmJhenJGnDhg2aMWOGFixYIOnHpYqHDh2q77//XvPmzVP//v1P2psAAAAATbdgwQJt2rRJF110kf75z39q2rRpMgxDx44d0/bt23XPPfeoQ4cOmjNnTlDFqYaqqqrSmjVrtHDhQhUUFKh79+6aNWtWrWOzs7Nlt9sDX4cPHw55HgAA0Ho0ujh15MgRvfDCCxo0aJBGjx6t5cuXKyEhQQ8//LAKCwu1efNmffXVV5o0aZL279+vO+64I5S5AQAA8BPLly/Xaaedpr/85S/q3bu3DMOQJEVHR6t///76/e9/r7/+9a969NFHtWLFigYfNzExUQcOHAis0GyapgoLC+VwOKqNczgcGjNmjLp37y7DMDR16lRt2bKl1mNmZWWpuLg48HXqqac28lEDAIC2IOji1O7du/Xb3/42cDbs888/V2pqqpYtWyaPx6P7778/0F+gd+/e+stf/qJhw4ad9MMJAAAAmm7nzp0aNmyYOnXqJEmB4pTP5wuMGT9+vIYMGaKnn366wcc944wzdM4552jZsmWSpJUrV8put6tXr17Vxl199dXaunVroJ3D22+/rUGDBjXpMQEAgPYhqOLUpZdeqr59+2rBggWqqKjQ5MmTtXnzZm3ZskVTpkw5aXPN/v3764cffmhS0LfeekvnnnuuOnTooNtvv/2k444ePaoJEyYoOTlZgwYN0rhx42o07ATQOpmmqa3uMi13FWmru0ymaUY6EgC0GJWVldUakJ9yyimSVKP3Z58+ffT5558HdeyFCxdq4cKFSk5O1vz587V06VJJUmZmpvLy8iT9OHPq3nvv1ciRI3X22Wfrgw8+0PPPP9+UhwQAANqJhi/VImn16tXq0qWLZsyYoZtvvlndunVr0M9NmDChxtTvYPXu3Vs5OTlavnx5vX0JZsyYoUsvvVSGYeiZZ55RZmam1q1b16T7BxBZxeVeTcvJV1GZV9FWiyp9fiXaYpWbMVT2+NhIxwOAiDvzzDN14MCBwPaJz2lffvmlRo4cGbh9//791WZTNUSfPn20efPmGrcvXry42vb111+v66+/PqhjAwAABDVzKicnR0VFRXrkkUcaXJiSfpxC/sADDwQd7qdOzISqb+njjh076pe//GVgKvvw4cPldrubdN8AIss0TU3LyZen1KtKnynvcZ8qfaY8pV5Nz8lnBhUASOrXr1+12eIjR46UaZp64oknAg3Q169frw0bNqhPnz6RigkAAFBDUMWp9PR0xcTENFeWZrFgwQJdccUVkY4BoAlcnnIVlx2Rz1+9COXzmyos88rlKY9QMgBoOS6++GIVFxcrPz9fkjR69Gj1799fb775prp3765zzz1X48aNk2mauvnmmyOcFgAA4N+CuqyvOY0YMUK7du2qdV9BQYESExODPua8efO0e/durV27ttb92dnZys7ODmyzjDHQMrlLKhRlNXS8lqtQoq0WuUsqlOq0hT8YALQgU6ZMUefOnQMN0S0Wi15//XVNnDhRn3/+ub755htZrVbddtttSk9Pj2xYAACAnwiqOHXhhRc2aFxMTIwSEhKUkpKia6+9Vl27dq33Z2rrY9AUf/jDH7Rq1SqtWbNGsbG196PJyspSVlZWYNtut4c0A4DQcCbEqdLnr3Vfpc8vZ0JcmBMBQMuTkJCg6667rtptvXr10rZt2/TPf/5TZWVlSk5OVufOnSOUEAAAoHZBFadONBU/0c+ptj4vhmEEbn/llVd033336bnnntO0aaEoFYwAACAASURBVNOaGLXhsrOz9corr2jNmjX62c9+Frb7BdA8UpLilWiLlafUW+3SPqvFkMMWq5Sk+AimA4CWjx5TAACgJQuq59SHH36oO+64Q6ZpKiUlRX/84x/12muv6fXXX9dTTz2loUOHyjRNZWVlKTc3V9OnT9fRo0eVmZkZ6H/QWGvXrpXdbld2draWLFkiu90eWLo4Ly9PmZmZkqTi4mLdcccd+v777zVmzBgNHjxYw4YNa9J9A4gswzCUmzFUSZ1jFW01FBtjVbTVkLNzrHJvHBYomANAe5abm6uXXnpJx48fr3Pcpk2blJubG6ZUAAAA9TPMIJa52rx5s0aNGqUnnnhCt99+e61jFixYoLvuukvr1q3TyJEjtWjRIs2cOVPXXnutXnrppZAFbw52u13FxcWRjgHgJEzTlMtTLndJhZwJcUpJiqcwBSBobfX93mKxyDAMnX/++Xr99ddPOnv8hhtuUG5urny+Whr5RUhb/Z0ATTUue70k6f2sURFOAgBNU997fVAzpx555BH17dv3pIUpSfrNb36jvn376tFHH5UkZWZmyul0auPGjcHcFQDUYBiGUp02TUpJVKrTRmEKAP6DYRj66KOPdN5556moqCjScQAAABokqOJUfn6+Bg4cWO+4gQMH6pNPPpH044ek/v3769tvv21cQgAAADTI5MmTdd111+mLL77Q8OHD9dlnn0U6EgAAQL2CKk4dOXJEBw4cqHfcgQMHdPTo0cB2XFycoqKC6r0OAACAIMXExOj//u//NHv2bB04cECjRo3Se++9F+lYAAAAdQqqONWvXz9t2LAhMCuqNp988ok2bNig/v37B27bt2+fEhISGp8SAAAADTZv3jw999xz8nq9Gj9+vJYuXRrpSAAAACcVVHHq5ptvls/n00UXXaQ5c+boyy+/1JEjR3TkyBF99dVXmjt3ri6++GL5/X7NmjVLkuT1elVQUKBzzz23WR4AAAAAapo5c6Zef/11RUdHKzMzUw899FCkIwEAANQqqGvtMjIy5HK59Pzzz2vevHmaN29ejTGmaWrmzJnKyMiQJLndbl199dWaPHlyaBIDAACgQS677DKtW7dOl19+uR5++GF5PB5VVlZGOhYAAEA1QTeCevbZZ3XJJZdowYIF2rx5c6C3VIcOHTRixAjddtttmjBhQmB8//79mUoOAAAQISkpKdq8ebMuvfRS/fnPf5bFEtTEeQAAgGbXqC7laWlpSktLk8/nU0lJiSSpc+fOND0HAABogXr06KHNmzdr/Pjx2rRpkwzDiHQkAACAgKCqST179lRycrLeffddSZLValXXrl2bJRgAAAAabu/evTr11FNPuj8+Pl5r167V8uXL5fP5wpgMAACgbkEVp7755hsNHz68ubIAAACgkZKSkuod06FDB02dOjUMaQAAABouqKYDSUlJOnToUHNlAQAAAAAAQDsT1Mypq666Sk899ZS+++47denSpbkyAQAAoB4PP/ywJOnWW2+VzWYLbDfU3LlzmyMWAABA0AzTNM2GDj5y5IhGjx6t48eP609/+pNGjhzZnNnCzm63q7i4ONIxAABAM2or7/cWi0WGYejLL79UcnJyYLuuj3Yn9huG0aL6TrWV3wkQauOy10uS3s8aFeEkANA09b3XBzVz6rLLLpPVatW2bdt0/vnn64wzzpDT6dQpp5xSY6xhGFq7dm3wiQEAAFCvuXPnyjAMJSQkVNsGAABobYIqTq1bty7wvWma+uabb/TNN9/UOpYPRwAAAM3nwQcfrHMbAACgtQiqOPXhhx82Vw4AbZRpmnJ5yuUuqZAzIU4pSfEUrwEAAAAAAUEVp0aN4lpnAA1XXO7VtJx8FZV5FW21qNLnV6ItVrkZQ2WPj410PABoU3r27KlJkybp8ccfr3Pc7373O/31r3/Vnj17wpQMAACgbpZIBwDQNpmmqWk5+fKUelXpM+U97lOlz5Sn1KvpOfl1NuwFAATP7Xbru+++q3dcSUmJ3G538wcCAABooKBmTp1gmqbeeecdbdq0Sd99952GDRumjIwMSdJ3332n8vJynXXWWbJarSENC6D1cHnKVVx2RD5/9SKUz2+qsMwrl6dcqU5bhNIBQPt19OhRRUU16iMgAABAswh65tS2bdvUr18/jR8/XvPmzdPixYu1cePGwP73339f/fr109tvvx3SoABaF3dJhaKstfeWirZa5C6pCHMiAIDP55PL5VKXLl0iHQUAACAgqNNmxcXF+sUvfqHS0lL98pe/1OjRo3X33XdXGzNhwgRFR0frjTfe0Pjx40MaFkDr4UyIU6XPX+u+Sp9fzoS4MCcCgLbnwgsvrLb97rvv1rjthKqqKu3atUvffvutpkyZEo54AAAADRJUcWrevHkqLS3VU089pdtuu02SahSnYmNjNWjQIG3dujV0KQG0OilJ8Uq0xcpT6q12aZ/VYshhi1VKUnwE0wFA27Bu3brA94Zh6ODBgzp48GCdP5OSkqLf//73zZwMAACg4YIqTr377rvq27dvoDB1Mk6nUx9++GGTggFo3QzDUG7G0Bqr9Tlsscq9cZgMo/ZL/gAADXfi85Zpmrrwwgt1ySWX6J577ql1bExMjOx2uxITE8MZEQAAoF5BFaf279+vK664ot5xhmHo0KFDjQ4FoG2wx8dqbdYouTzlcpdUyJkQp5SkeApTABAio0aNqvb96NGjq90GAADQGgRVnIqLi2vQEsV79+6VzcYqXAB+LFanOm2szAcAzYxZ6wAAoLUKqjg1cOBAffrppyopKVFCQkKtYzwej7Zt26Zx48aFJCAAAACC88UXX2jTpk367rvvNGDAAKWlpUmS/H6/qqqqFBMTE+GEAAAA/2YJZvDUqVP1ww8/KDMzU16vt8b+48eP6+abb1ZlZaWmTp0aspAAAACoX1FRkX7xi19o4MCBmjlzpu6//369/vrrgf2LFi3SKaecorVr10YwJQAAQHVBFaduuOEGjRo1Snl5eerbt69mzJghSdq2bZtuu+02JScn65133tHYsWN1zTXXNEtgAAAA1FRWVqZRo0bpgw8+0IABAzRr1iyZplltzNVXXy2LxaK8vLwIpQQAAKgpqOKU1WrVm2++qWuvvVb79u3T4sWLJUkFBQV65plnVFhYqIkTJ2rVqlXNEhYAAAC1e/zxx+V2u3XnnXdq27ZteuaZZ2qMiY+P18CBA7Vx48YIJAQAAKhdUD2nJOnUU0/VSy+9pDlz5ujtt9/W119/Lb/fr8TERF166aUaPHhwc+QEAABAHd544w05nU7Nnz+/zlVRe/bsqY8//jiMyQAAAOoWdHHqhL59+6pv376hzFKnt956S3PnztX27ds1a9YsPfXUU/X+zNKlS5WRkaHXXntNEyZMCENKAACAyPB4PLrssstksdQ9MT4mJkZlZWVhSgUAAFC/Rhenwq13797KycnR8uXLdfjw4XrHu91uLVq0SMOHDw9DOgAAgMjq2LGjfvjhh3rHFRYWqlOnTmFIBAAA0DCNLk7t27dP+/bt09GjR0865oILLmjs4WtITk6WJL322mv1jvX7/crMzNTTTz+tO+64I2QZAAAAWqq+ffvq73//uyoqKhQXF1frmJKSEm3btk3Dhg0LczoAAICTC7o49cYbb2j27NnauXNnneMMw1BVVVWjgzVFdna2fv7zn+vcc8+td1x2dnZguyEzsgAAAFqiq666SnfddZeysrL03HPP1Xp531133SWv18uqygAAoEUJqjj1zjvvaOLEifL7/erUqZN69uyp008/PSRBRowYoV27dtW6r6CgQImJiQ06zvbt27Vy5Up99NFH9Y7NyspSVlZWYNtutzcsLAAAQAtzyy236M9//rMWL16sTz/9VFdeeaUkac+ePcrOztby5cuVn5+vwYMHKz09PbJhAQAAfiKo4tRjjz0mv9+vBx98ULNnz1ZMTEzIgmzevDkkx9mwYYPcbrd69+4tSTp48KBmzJihAwcOaNasWSG5DwAAgJamY8eOWr16tSZNmqRNmzapoKBAkrRx40Zt3LhRpmkqNTVVr7/+uqKjo4M69q5duzR9+nSVlJSoU6dOevHFFzVgwIBax5qmqbFjx+rvf/+7vv/++yY/LgAA0PYFVZz67LPPNHjwYM2dO7e58jTZrFmzqhWhRo8erdtvv53V+gAAQJvXrVs3bdy4UatXr9Zbb72lr7/+Wn6/X4mJibr00kt1xRVXyDCMoI87c+ZMzZgxQ+np6VqxYoXS09O1devWWsf+8Y9/1FlnnaW///3vTX04AACgnQiqOGW1WtW3b9/mylKntWvXavr06Tp06JBM09SKFSv07LPPKi0tTXl5ecrLy9PixYsjkg0AAKAlufjii3XxxReH5FjffvutXC6X3nvvPUnSxIkTdeutt2r37t3q1atXtbE7duzQ66+/rqVLl2r58uUhuX8AAND2BVWcOvvss1VcXNxcWeo0duzYk953Wlqa0tLSat23bt26ZkwFAADQthUVFalbt26KivrxY6NhGHI4HCosLKxWnKqsrNRNN92kJUuWyGq1RiouAABohYIqTt1+++26+uqr5XK5lJKS0lyZAAAA0AQ+n0+lpaU6evToScc4HI6Q3udDDz2kK6+8Uv369ZPb7a5zLCsmAwCAnwqqODVx4kTNmTNHF198sR555BFdfvnlIf9gAwAAgMbZunWr5s6dq/Xr1+vYsWMnHWcYhqqqqhp0zMTERB04cEBVVVWKioqSaZoqLCys8Rlw/fr1Kiws1DPPPKOqqiodOnRITqdTW7duVZcuXaqNZcVkAADwU0H3nDrh17/+tX7961+fdGwwH3oAAADQNFu2bNGFF14YmC0VHx+v008/vcnHPeOMM3TOOedo2bJlSk9P18qVK2W322v0m9qwYUPge7fbrcGDB9c7gwoAAEAKsjhlmmazjAUAAEDTPPDAAzp69KgyMjL02GOPqWvXriE79sKFC5Wenq558+bp9NNP19KlSyVJmZmZdfb+BAAAaIigilN+v7+5cgAAAKAJPvnkE/Xp00eLFi2SYRghPXafPn20efPmGrefbKVkp9Op77//PqQZAABA2xVUcQpA22eaplyecrlLKuRMiFNKUnzI/8kBAIReVVWVBg8ezGs2AABodShOAQgoLvdqWk6+isq8irZaVOnzK9EWq9yMobLHx0Y6HgCgDn379lVJSUmkYwAAAATNUtfO3Nxcbdq0qdZ9hw4dOunyxK+88kq1FVgAtHymaWpaTr48pV5V+kx5j/tU6TPlKfVqek4+feQAoIWbMWOGNmzYoD179kQ6CgAAQFDqLE6lp6eftJdAfHy8brnlllr3vffee1qwYEHT0wEIG5enXMVlR+TzVy9C+fymCsu8cnnKI5QMANAQM2bM0LXXXqtx48bp7bffls/ni3QkAACABmn0ZX2maTKTAmhD3CUVirIaOl7L/zLRVovcJRVKddrCHwwA0CA9e/aUJLndbo0fP15RUVHq1q2bLJaa5yINw2CGFQAAaDHoOQVAkuRMiFOlr/YVOSt9fjkT4sKcCAAQDLfbHfjeNE1VVlaqsLCw1rE0TQcAAC0JxSkAkqSUpHgl2mLlKfVWu7TPajHksMUqJSk+gukAAPXZu3dvpCMAAAA0CsUpAJJ+PIuemzG0xmp9Dluscm8cxll2AGjhkpKSIh0BAACgUShOAQiwx8dqbdYouTzlcpdUyJkQp5SkeApTAAAAAIBmQ3EKQDWGYSjVaaP5OQAAAAAgLOotTu3evVu5ublB7du9e3fTkwEAAKDBTqzWV5+YmBglJCQoNTVV06ZN05AhQ5o5GQAAQN0M0zTNk+20WCyNupzHNE0ZhiGfr5Y16Vswu92u4uLiSMcAAADNqK2+31ssFkk/zoA92ce7/9xnsVj04IMP6v777w9LxpNpq78ToKnGZa+XJL2fNSrCSQCgaep7r69z5pTD4aDXDAAAQCuwd+9ePffcc/rDH/6giRMn6rrrrpPT6ZRhGHK73XrppZe0atUq/fa3v1VaWpo++OADzZ8/Xw888ICGDRumcePGRfohAACAdqrO4pTb7Q5TDAAAADTFjh079OSTT+qvf/2rJk6cWG3f2WefrbS0NK1atUqTJk3SBRdcoLlz52rw4MGaMGGCnn32WYpTAAAgYiyRDgAAAICme+KJJ5SamlqjMPVTV155pVJTU/Xkk09KktLS0tSnTx/l5+eHKyYAAEANFKcAAADagM8++0y9evWqd1yvXr20bdu2wHafPn1UWlranNEAAADqRHEKAACgDfD7/fr666/rHff111/L7/cHtqOjo9WxY8fmjAYAAFAnilMAAABtwKBBg/TJJ58oLy/vpGPy8vK0ZcsWDR48OHBbUVGRunTpEo6IAAAAtaI4BQAA0AbceeedMk1TV111la6//nq98847+vLLL/XVV1/p3Xff1bRp03TVVVfJMAzdcccdkqTvv/9eBQUFGj58eITTAwCA9qzO1foAAADQOlxxxRWaP3++7rvvPr388st6+eWXq+03TVMWi0WPPfaYrrjiCklSSUmJ7r33Xl1yySWRiAwAACCJ4hQAAECbcffdd+uiiy7S008/rY8++kjFxcWSpO7du+uCCy7QrbfeqnPOOScwvlevXnrggQciFRcAAEASxSmgTTBNUy5PudwlFXImxCklKV6GYUQ6FgAgAgYPHqwlS5ZEOgYAAECDUZwCWrnicq+m5eSrqMyraKtFlT6/Em2xys0YKnt8bKTjAQAAAABQJxqiA62YaZqalpMvT6lXlT5T3uM+VfpMeUq9mp6TL9M0Ix0RAAAAAIA6tZqZU2+99Zbmzp2r7du3a9asWXrqqadOOvbYsWO64447tHr1anXs2FGDBg3SsmXLwpgWCA+Xp1zFZUfk81cvQvn8pgrLvHJ5ypXqtEUoHQAgElwul1asWKF//vOfOnToUK0nKgzD0Nq1ayOQDgAAoKZWU5zq3bu3cnJytHz5ch0+fLjOsbNnz5ZhGNq5c6cMw9DBgwfDlBIIL3dJhaKsho77au6LtlrkLqmgOAUA7cidd96pP/7xj4GClGEY1YpTJ7bpSwgAAFqSVnNZX3JysgYNGqSoqLrraRUVFVqyZIkee+yxwAevM888MxwRgbBzJsSp0uevdV+lzy9nQlyYEwEAImX58uXKzs5W9+7dtXDhQl100UWSpNWrV+uZZ57RiBEjZJqmZs+erQ8++CDCaQEAAP6t1RSnGmrPnj2y2WyaN2+eUlJSdP755zNtHW1WSlK8Em2xslqqnwG3Wgw5bLFKSYqPUDIAQLi98MILslqtWrt2rW666SZ169ZNkjRu3DjdfPPN+vjjj3XfffcpOztbnTp1inBaAACAf2sxxakRI0YoISGh1q+ioqIGH6eqqkoej0f9+/eXy+XS//7v/+qaa67RN998U2Nsdna27HZ74Ku+ywWBlsYwDOVmDFVS51hFWw3FxlgVbTXk7Byr3BuHcdkGALQjBQUFGjZsmHr37n3SMQ899JC6deumRx99NIzJAAAA6tZiek5t3rw5JMdxOByyWCy67rrrJElDhgxRjx499Pnnn6tr167VxmZlZSkrKyuwbbfbQ5IBCCd7fKzWZo2Sy1Mud0mFnAlxSkmKpzAFAO3MDz/8IIfDEdiOiYmRJB0+fFinnnqqJMlisWjYsGFav359RDICAADUpsXMnAqVhIQEjR07VqtXr5Yk7d27V3v37lW/fv0inAxoPoZhKNVp06SURKU6bRSmAKAd6tKli77//vvAdkJCgiTJ7XZXG1dRUaFDhw6FMxoAAECdWk1xau3atbLb7crOztaSJUtkt9uVl5cnScrLy1NmZmZg7PPPP68nn3xSAwcO1IQJE7Rw4UJ17949UtEBAACandPplMfjCWwPGTJEpmnq5ZdfDtx28OBBrV+/XklJSZGICAAAUKsWc1lffcaOHavi4uJa96WlpSktLS2w3bNnT3344YfhigYAABBxY8eO1aOPPiq32y2n06lLL71UNptNjz/+uHbt2iWHw6EVK1aooqJCEydOjHRcAACAgFZTnAIAAMDJTZ48Wfv371dRUZGcTqfi4uK0dOlSTZ48WStXrgyMO/fcc/W73/0ugkkBAACqozgFAADQBvTr10+LFi2qdtv48eO1a9cuvfnmmyorK1O/fv00fvx4Wa3WCKUEAACoieIUAABAG/Zf//VfmjlzZqRjAAAAnFSraYgOAAAAAACAtofiFAAAAAAAACKGy/oAAABaoZiYmEb/rGEYOnbsWAjTAAAANB7FKSBCTNOUy1Mud0mFnAlxSkmKl2EYkY4FAGglqqqqIh0BAAAgJChOARFQXO7VtJx8FZV5FW21qNLnV6ItVrkZQ2WPj410PABAK2EYhlJTU5WRkaGLLrqIkxwAAKBVojgFhJlpmpqWky9PqVc+v6lKn0+S5Cn1anpOvtZkjeKfCwBAvR5//HEtXbpU+fn52rp1qxITEzV9+nTdcMMNcjqdkY4HAADQYDREB8LM5SlXcdkR+fxmtdt9flOFZV65POURSgYAaE3uuusuffHFF9q4caPS09NVVlamRx55RL169dIvfvELvfzyyyHrK7Vr1y6NHDlSycnJSk1N1Y4dO2qM+eCDDzR06FD1799fAwYM0N133y2/3x+S+wcAAG0bxSkgzNwlFYqy1j4zKtpqkbukIsyJAACt2ciRI7VkyRIdOHBAixcv1vDhw/XBBx/o+uuv15lnnqmbb75ZW7dubdJ9zJw5UzNmzNDOnTt1zz33KD09vcaY+Ph4/eUvf9EXX3yhTz/9VJs2bVJubm6T7hcAALQPFKeAMHMmxKnSV/uZ5EqfX86EuDAnAgC0BXFxccrIyNDGjRv11Vdf6c4771THjh31/PPPa/jw4TrvvPMaddxvv/1WLpdLU6dOlSRNnDhRRUVF2r17d7VxQ4YMUc+ePSVJHTt21ODBg+V2u5v0mAAAQPtAcQoIs5SkeCXaYmW1VJ89ZbUYcthilZIUH6FkAIC2Ijk5WY8//ri+/PJLjR8/XqZpaufOnY06VlFRkbp166aoqB9blRqGIYfDocLCwpP+zMGDB7VixQpdfvnlte7Pzs6W3W4PfB0+fLhR2QAAQNtAcQoIM8MwlJsxVEmdYxVtNRQbY1W01ZCzc6xybxxGM3QAQJNt2LBBN9xwgxITE/W3v/1NFotFF1xwQVju+9ChQxo/frzuvvtupaSk1DomKytLxcXFga9TTz01LNkAAEDLxGp9QATY42O1NmuUXJ5yuUsq5EyIU0pSPIUpAECjHThwQC+++KJefPFF7d69W6ZpqkePHkpPT1d6eroSExMbddzExEQdOHBAVVVVioqKkmmaKiwslMPhqDH2hx9+0CWXXKIrrrhCWVlZTX1IAACgnaA4BUSIYRhKddqU6rRFOgoAoJWqqqrSG2+8oZycHL333nvy+Xw65ZRTNGXKFGVkZGjMmDFNvo8zzjhD55xzjpYtW6b09HStXLlSdrtdvXr1qjbu8OHDuuSSS3TJJZfo/vvvb/L9AgCA9oPiFAAAQCv029/+Vi+99JJKS0tlmqZSUlKUkZGhKVOm6PTTTw/pfS1cuFDp6emaN2+eTj/9dC1dulSSlJmZqbS0NKWlpWnBggXKz89XRUWFVq1aJUmaNGmS7rvvvpBmAQAAbY9hmqYZ6RAthd1uV3FxcaRjAACAZtRW3u8tFosMwwgUpQYOHBjUz48cObKZkgWvrfxOgFAbl71ekvR+1qgIJwGApqnvvZ6ZUwAAAK2Yy+WSy+UK6mcMw1BVVVUzJQIAAAgOxSkgCKZp0sQcANAiOBwO3oMAAECbQHEKaKDicq+m5eSrqMyraKtFlT6/Em2xys0YKnt8bKTjAQDaGbfbHekIAAAAIWGJdACgNTBNU9Ny8uUp9arSZ8p73KdKnylPqVfTc/JF6zYAAAAAABqH4hTQAC5PuYrLjsjnr16E8vlNFZZ55fKURygZAAAAAACtG8UpoAHcJRWKstbe1yPaapG7pCLMiQAAAAAAaBsoTgEN4EyIU6XPX+u+Sp9fzoS4MCcCAAAAAKBtoDgFNEBKUrwSbbGyWv4fe3ceH1V973/8fTIJSAJqQhCQSTIEUDbDUgJSERCKUCuL4NqLIbKIF3vx/qKitSr1cblYq421UgsKaHNdqgaupSqghsWFKkkFLrUuhDKTjBAlC4YkLknm/P6gmZJmJzNzZnk9H495PDhzvvmez3fOMOc7n/me77fx6ClblKHkhFiNSYm3KDIAAAAAAEIbySmgHQzDUM7CsUrpGasYm6HYLjbF2Aw5esYqZ9E4lvIGAAAAAOAMRVsdABAq7PGxysuapAJXhZyl1XIkxmlMSjyJKQAAAAAAOoHkFNABhmEo3ZGgdEeC1aEAAAAAABAWSE4hrJmmyUgnAAAAAACCWMjMOfXaa6/pe9/7nrp27ar//M//bLXs66+/rtGjR2vkyJEaPny4fv/73wcoSgQTd0WNpmbv1o+fel8rt3ykHz/1vqZm75a7osbq0AAAAAAAwD+EzMipQYMGaePGjXr55ZdVVVXVYjnTNDV//nzt2rVLaWlpcjqdGjx4sObOnasePXoEMGJYyTRNZWzcK1dZjeo9pmrr6yVJrrIaLdi4V29lTWIEFQAAAAAAQSBkRk5dcMEFGjFihKKj286nGYahEydOSJIqKyvVs2dPde3a1d8hIogUuCrkLv9a9R6z0fP1HlNF5TUqcFVYFBkAAAAAADhdyIycai/DMPTiiy9q7ty5iouLU0VFhTZv3qwuXbpYHRoCyFlarWiboe/qm+6LsUXJWVrNpOYAAAAAAASBoBk5NX78eCUmJjb7KC4ubnc9dXV1WrVqlTZv3iyXy6W8vDzdeOONKi0tbVI2Oztbdrvd+2jtdkGEFkdinGrrPc3uq633yJEYF+CIAAAAAABAc4Jm5NSf//xn+b5ZpQAAIABJREFUn9Szf/9+HT16VBMnTpQkpaeny263a9++fZo2bVqjsllZWcrKyvJu2+12n8QA641JiVdSQqx3zqkGtihDyQmxGpMSb2F0AAAAAACgQdCMnPKVpKQkHTt2TB9//LEkqbCwUIcPH9aFF15ocWQIJMMwlLNwrFJ6xirGZii2i00xNkOOnrHKWTSOydABAAAAAAgSQTNyqi15eXlasGCBKisrZZqmcnNz9cQTT2jWrFnasmWLtmzZovXr16t379568sknde211yoqKkoej0dr1qxRcnKy1U1AgNnjY5WXNUkFrgo5S6vlSIzTmJR4ElMAAAAAAAQRwzRNs+1ikcFut8vtdlsdRsQzTZOEEgDAb7jeBx/OCdC8adm7JUlvZk2yOBIA6Jy2rvUhM3IKkcFdUaOMjXtVXF6jGFuUaus9SkqIVc7CsbLHx1odHgAAAAAA8LGwm3MKocs0TWVs3CtXWY1q603VfFev2npTrrIaLdi4VwzyAwAAAAAg/JCcQtAocFXIXf51o9X1JKneY6qovEYFrgqLIgMAAAAAAP5CcgpBw1larWhb83NLxdii5CytDnBEAAAAQGCYpql8Z7leLihWvrOcuwYARBTmnELQcCTGqbbe0+y+2nqPHIlxAY4IAAAA8L+W5l31eEzF2BhPACD88UmHoDEmJV5JCbGyRTUePWWLMpScEKsxKfEWRQYAAAD4R2vzrn5+4murwwOAgCA5BZ/wxTBkwzCUs3CsUnrGKsZmKLaLTTE2Q46escpZNE6G0fwtfwAAAECoam3e1dp6U1/X1lsUGQAEDrf1odNaGoacs3Cs7PGxHarLHh+rvKxJKnBVyFlaLUdinMakxJOYAgAAQFhqmHf1u2ZyUIahFqe9aI1pmvSnAYQUklPolNOHIZ/6defUVdVVVqMFG/fqraxJHb4QGoahdEeC0h0J/ggZAAAACBqtzbtqmurwnFO+/OEYAAKF2/rQKa0NQy4qr1GBq8KiyAAAAIDg19q8qzE2Q91ibO2uq7X5qxZs3MsKgACCFskpdErDMOTmxNii5CytDnBEAAAAQOhobd7Vfh0c6cQPxwBCFbf1oVNaG4ZcW++RIzEuwBEBAAAAoaWleVcvf/TtDtXT2vxVDT8cd3TqDOavAhAIJKcimC8uNA3DkBvmnGpgizKUnBCrMSnxvg4bAAAACDu+mHfV1z8cM38VgEAhORWhfHWhaRiG/K91JSfEKmfROH5VAQAAAALElz8c+2PhIwBoCcmpCOTrC01Lw5C5WAEAAACB48sfjtszfxWrawPwFZJTEcgfFxpfDEMGAAAA0Dm++uHYH/NXAUBLSE6FGF/ME8WFBgAAAAhfwTh/FQC0huRUCPHVPFFcaAAAAAC0hoWPAARSlNUBoH1Onyeqtt5UzXf1qq03vfNEmabZdiX/0HChsUU1HnHFhQYAAACA9M/5q1J6xirGZii2i00xNkOOnp1b+Mg0TeU7y/VyQbHyneUd+h7jz7oAWIuRUyHCl/NEscIeAAAAgLb4euEjX90J4uu6fM0XU7H4sz5f8WVcwVqXrwVrO4PhNSM55We+Osm+nieKFfYAAEB7HTp0SAsWLFBpaanOOeccPfPMMxo2bFiTchs2bNAvfvELeTweTZkyRU888YRiYmIsiBiAr/hq4SNfrhju69XHG+r0xXcjXyfNfF1fMLYzWOtqwGsWGIbJ2Ecvu90ut9vts/p8eZLzneX68VPvq7a+6emKsRl6fsnFTGIOAEA7+Pp6HwmmTJmijIwMZWZmKjc3Vw899JDy8/MblTly5IguueQSffjhh+rdu7dmz56t6dOn69Zbb22zfs4J0Lxp2bslSW9mTbI4ks7Ld5br3576QN81M/dtR7/P+LIuyXff20zT1NTs3c3O0+XoGdvhpJmv6wvGdgZrXQ14zTr+mrWkrWs9c075iS/niJKYJwoAAFjjyy+/VEFBgebPny9JmjdvnoqLi1VYWNioXG5urmbNmqU+ffrIMAzdcssteuGFF6wIGUAQargTpDkNd4JYUZcvv7e1ZyqWjvBlfcHazmCtS+I1O5PXrDNITvmJr0+yvyYkBAAAaE1xcbH69u2r6OhTs0EYhqHk5GQVFRU1KldUVKSUlBTvtsPhaFIGQOTy5YrhvqzLl9/bfJk083V9wdrOYK1L4jU7k9esM5hzyk98PUeUxDxRAAAgPGRnZys7O9u7XVVVZWE0AAKh4U6Q5m4f6uidIL6sy5ff23yZNPN1fcHazmCtS+I1O5PXrDMYOeUn/jrJDRMSXjMmSemOBBJTAADAr5KSknTs2DHV1dVJOnWbQ1FRkZKTkxuVS05Olsvl8m47nc4mZRpkZWXJ7XZ7H927d/dfAwAEBV/eCeLLunz5vc3XU7H4sr5gbWew1iXxmgV6+iCSU34STCcZAADgTJ133nkaPXq0nn32WUnSpk2bZLfbNXDgwEbl5s2bpy1btqikpESmaWrt2rW6/vrrrQgZQJBquBPk+SUX64FZw/T8kov1VtYk9Tu3m2V1+fJ7m6+nYvFlfcHazmCtS+I1C/T0QazWd5pArNaXnHDqJJ/JBzAAAOg8VobruE8//VSZmZkqKyvT2WefraeffloXXXSRFi9erFmzZmnWrFmSpKeeekq/+MUvJEmTJ0/W2rVrFRMT02b9nBOgeeG0Wl8w8/X3NtM0fToVi6/qC+Z2BmtdvGa+09a1nuTUafzRMQrESQYAAO1HIiT4cE6A5pGcCpxI+d4WKe30JV4z32jrWs+E6H7WMEdURyc/BwAAAAAERqR8b4uUdvoSr1lghMycU7/5zW80fPhwXXTRRUpLS/POe9CcL7/8UjNmzNCgQYM0fPhwvf322wGMFAAAAAAAAO0VMiOnhg0bpvfee0/nnHOOiouLNWrUKI0fP14DBgxoUvbuu+/WxRdfrG3btik/P19XXXWVjhw50q45DwAAAAAAABA4ITNyaurUqTrnnHMknVrSuE+fPiouLm627EsvvaRbbrlFkpSenq7zzz9fu3fvDlisAAAAAAAAaJ+QSU6d7q233lJFRYXS09Ob7CsrK1Ntba369Onjfc7hcKioqKhJ2ezsbNntdu+jqqrKr3EDAAAAAACgsaBJTo0fP16JiYnNPk4fIXXw4EHddNNNevHFFxUXF9epY2ZlZcntdnsf3bt372wzAAAAAAAA0AFBM+fUn//85zbL/O1vf9OVV16pjRs3asKECc2W6dmzp6Kjo1VSUuIdPeV0OpWcnOzTeAEAAAAAANB5hmmaptVBtMfHH3+sH/7wh1q3bp2mT5/eatnMzEw5HA79/Oc/V35+vubMmSOn09nmhOhdu3ZVr169fBm2V1VVVUSMzKKd4SMS2ijRznASCW2UaKcvHD9+XN9++61f6saZoQ/WeZHQzkhoo0Q7w0kktFGineHEyv5XyCSnpk2bpoKCAqWkpHife+ihhzR9+nQVFBTo/vvv1+uvvy5J+uKLL3TjjTfqyJEj6tKli9asWaPLLrvMqtAlSXa7XW6329IYAoF2ho9IaKNEO8NJJLRRop1AR0XKeykS2hkJbZRoZziJhDZKtDOcWNnGoLmtry1vvvlmi/vGjBnjTUxJUu/evfXGG28EIiwAAAAAAAB0QtBMiA4AAAAAAIDIY/v5z3/+c6uDiBTjx4+3OoSAoJ3hIxLaKNHOcBIJbZRoJ9BRkfJeioR2RkIbJdoZTiKhjRLtDCdWtTFk5pwCAAAAAABA+OG2PgAAAAAAAFiG5BQAAAAAAAAsQ3KqE5YvXy6HwyHDMLR///5my+zatUvdunXTyJEjvY+vv/7au3/Dhg0aNGiQBgwYoCVLlqi2tjZQ4bdbe9r59NNPN2pjYmKi5s6dK0lyOp2y2WyN9h8+fDiQTWjTN998ozlz5uiCCy7QiBEjNG3aNBUWFjZb9tVXX9XgwYM1aNAgzZ07V5WVle3aFwza286DBw9q4sSJGjx4sIYPH66FCxc2et8ahqGLLrrIez7feeedQDajVe1tY1vvy3A5l9u3b2/UxvPPP1+jR4/27g/mc9ng8ssvV1pamkaOHKlLL71U+/bta7Zca5+nwf5Z25427tixQ2PHjtXQoUM1bNgwrVixQh6PR1JofM5K7WtnOFw34X/0wf6JPljb+4IBfbB/og92SjCfywb0wU6hD3aK38+liTO2e/dus7i42ExJSTH37dvXbJmdO3eaI0aMaHbf3//+d7Nv377msWPHTI/HY86cOdNcs2aNP0M+I+1p578aNmyYmZuba5qmaR45csQ855xz/Blip3399dfma6+9Zno8HtM0TfPxxx83J02a1KTcyZMnzfPOO8/8+OOPTdM0zVtvvdW844472twXLNrbzs8++8w8cOCAaZqmWVdXZ1577bXmypUrvfslmRUVFYEIucPa28bW3pfhdC7/1Y9+9CPzkUce8W4H87lscHp8mzdvNtPS0pqUae3zNBQ+a9vTxg8//NA8fPiwaZqnzv8ll1xiPv3006ZphsbnrGm2r53hcN2E/9EHaxl9sNC+btMHC59z+a/ogwXnZy19sH+y+rrJyKlOmDhxoux2+xn/fW5urmbNmqU+ffrIMAzdcssteuGFF3wYoW90tJ0ffPCBvvzyS82aNcuPUfnWWWedpSuuuEKGYUiSLr74Yjmdzibltm7dqlGjRmnw4MGSpGXLlnnPWWv7gkV72zlo0CClpaVJkmw2m9LT05stF4za28bWhNO5PN3Ro0eVl5enG2+8MQAR+s65557r/fdXX33lbfPpWvs8DYXP2va0cdSoUUpNTZV06vyPHDkyZP5fNmhPO1sTCucSgUEfrHn0wUL/uk0fLHzO5enogwXvZy19sPYJxLmM9mltaNbhw4c1evRo2Ww23XTTTVq2bJkkqaioSCkpKd5yDodDRUVFVoXpMxs2bNCNN96omJgY73PV1dVKT09XfX295syZo5/97Gey2WwWRtm6xx57TLNnz27yfHPn7NixY6qrq2t1X3R0cP5Xa6mdp6uurtb69ev14IMPNnp+6tSpqqur09SpU/Vf//VfiouL82eoZ6y1Nrb0vgzXc/nMM8/oiiuu0Hnnndfo+VA4lxkZGdq5c6ck6fXXX2+yv7XP01D5rG2rjacrKSlRbm6uXn31Ve9zofI52552Rtp1E/4Tae8l+mBN94XydZs+WPicS/pgTfcFE/pg/2TldZORU342evRoud1uffjhh/rf//1frV27Vi+99JLVYflNdXW1/vCHP2jRokXe5/r27avPP/9c+fn5euutt/TOO+/oV7/6lYVRtm716tUqLCxs0hEIN+1p53fffafrrrtOl19+ua666irv8y6XS3/5y1+0Z88eHT9+XHfeeWcgQu6w1toYau/L1rTnXJqmqY0bNzb6vymFzrnMyclRcXGxVq1apbvuusvqcPyivW2srKzUzJkztWLFCo0ZM0ZSaL2f22pnpF034T+R9l6iDxY66IOF1vuyNfTBwgN9sFOsvm6SnPKzs88+W+ecc44kyW6364YbbvBOdpecnCyXy+Ut63Q6lZycbEmcvvLyyy9r2LBhGjp0qPe5rl27en8lSEhI0MKFC4Nywj9JeuSRR7R582Zt3bpVsbGxTfY3d8769u2r6OjoVvcFm7baKUm1tbW67rrr1LdvXz322GON9jW8T+Pi4rRs2bKgPJ9ttbG192W4nUtJ2r17t7755htNnz690fOhcC5Pt2DBAu3cuVNlZWWNnm/t8zTUPmtbaqMknTx5UjNmzNDs2bOVlZXlfT6UPmcbtNTOSLtuwn8i7b1EHyx8rtv0wcLnXEr0wULps5Y+mLXXTZJTfnbs2DHvTP4nT57Uq6++qlGjRkmS5s2bpy1btqikpESmaWrt2rW6/vrrrQy30zZs2NDkV4Evv/zSO5P/t99+q82bN3tfg2CSnZ2tF154QW+++Waje3JPN2PGDH344Yf65JNPJElPPPGE95y1ti+YtKeddXV1uv7665WQkKAnn3yy0T3JFRUVqqmpkSR5PB69+OKLQXc+29PG1t6X4XQuG2zYsEGZmZmNhhiHwrk8ceKEjh496t1+5ZVX1LNnTyUkJDQq19rnabB/1ra3jVVVVZoxY4ZmzJihe++9t9G+UPicbW87I+26Cf+JtPcSfbDwuG7TBwufc9mAPljwftbSBwuyPphPp1ePMDfffLPZr18/02azmeedd545YMAA0zRNc9GiReYf//hH0zRPrd4wdOhQMy0tzRw6dKi5cuVK78oOpmmaTz75pJmammqmpqaaCxcuNL/77jtL2tKa9rTTNE3zk08+Mbt3725WVlY2+vtNmzaZw4YN874GP/nJT8xvvvkmoG1oS3FxsSnJTE1NNUeMGGGOGDHCHDt2rGmapnnfffeZv/vd77xl//jHP5oXXnihOWDAAHP27NnmiRMn2rUvGLS3nc8++6wpyUxLS/OWW7ZsmWmaprlnzx7zoosu8p7P+fPnm2VlZZa16V+1t41tvS/D5VyapmmeOHHCjI2N9a4y0iDYz6VpmqbT6TTT09PN4cOHm2lpaebUqVO9K1b962dQa5+nwfxZ2942rlq1yoyOjvae7xEjRpirVq0yTTM0Pmfb285wuG7C/+iD0QejDxZ81236YPTB6IMF5+dsqPTBDNM0Td+muwAAAAAAAID24bY+AAAAAAAAWIbkFAAAAAAAACxDcgoAAAAAAACWITkFAAAAAAAAy5CcAgAAAAAAgGVITgEAAAAAAMAyJKcAhD3DMDr8mDx5stVhAwAAhDT6YADaK9rqAADA3xYsWNDkuZKSEm3fvr3F/YMHD/Z7XAAAAOGMPhiA9jJM0zStDgIAAm3Xrl267LLLJEl8DAIAAAQGfTAAzeG2PgAAAAAAAFiG5BQAAAAAAAAsQ3IKAAAAAAAAliE5BQAAAAAAAMuQnAIAAAAAAIBlSE4BAAAAAADAMiSnAAAAAAAAYBmSUwAAAAAAALAMySkAAAAAAABYhuQUAAAAAAAALENyCgAAAAAAAJYhOQUAAAAAAADLkJwCAAAAAACAZQzTNE2rgwAAAAAAAEBkYuQUAAAAAAAALENyCgAAAAAAAJYhOQUAAAAAAADLkJwCAAAAAACAZUhOAQAAAAAAwDIkpwAAAAAAAGAZklMAAAAAAACwDMkpAAAAAAAAWIbkFAAAAAAAACxDcgoAAAAAAACWITkFAAAAAAAAy5CcAgAAAAAAgGVITgEAAAAAAMAyIZWcWr58uRwOhwzD0P79+5st4/F4lJWVpaFDhyotLU2XXXaZCgsLAxwpAAAAAAAA2iOkklNXX3213n33XaWkpLRYZsuWLXrvvfd04MAB/d///Z+mTp2qe+65J4BRAgAAAAAAoL2irQ6gIyZOnNhmGcMw9O233+qbb75RdHS0KisrZbfb21V/165d1atXr86GCYSl0qpvJUmJ3btaHAkAdM7x48f17bffWh0GTkMfDGge/S8A4aKt/ldIJafaY+bMmdq5c6f69OmjHj16qF+/ftq9e3ezZbOzs5Wdne3d7tatm9xud6BCBULKtOxT/4/ezJpkcSQA0Dnt/dEKgdOrVy/6YEAz6H8BCBdt9b9C6ra+9igoKNBf//pXff755zp69KimTp2qW265pdmyWVlZcrvd3kf37t0DHC0AAAAAAEBkC7vkVE5OjqZMmaJzzz1XUVFRWrBggXbu3Gl1WAAAAAAAAGhG2CWnUlNTtWPHDn333XeSpFdffVXDhw+3OCoAAAAAAAA0J6SSU0uXLpXdbpfb7db06dM1cOBASdLixYu1ZcsWSdKtt96q/v37a8SIEUpLS1NeXp5+97vfWRk2AAAAAAAAWhBSE6KvW7eu2efXr1/v/XfXrl311FNPBSokAAAAAAAAdEJIjZwCAAAAAABAeCE5BQAAAAAAAMuE1G19AEKPaZoqcFXIWVotR2KcxqTEyzAMq8MCAAAIW/S/AIQaklMA/MZdUaOMjXtVXF6jGFuUaus9SkqIVc7CsbLHx1odHgAAQNih/wUgFHFbHwC/ME1TGRv3ylVWo9p6UzXf1au23pSrrEYLNu6VaZpWhwgAABBW6H8BCFUkpwD4RYGrQu7yr1XvadwJqveYKiqvUYGrwqLIAAAAwhP9LwChiuQUAL9wllYr2tb83AYxtig5S6sDHBEAAEB4o/8FIFSRnALgF47EONXWe5rdV1vvkSMxLsARAQAAhDf6XwBCFckpAH4xJiVeSQmxskU1/vXOFmUoOSFWY1LiLYoMAAAgPNH/AhCqSE4B8AvDMJSzcKxSesYqxmYototNMTZDjp6xylk0juWMAQAAfIz+F4BQFW11AADClz0+VnlZk1TgqpCztFqOxDiNSYmnYwQAAOAn9L8AhCKSUwD8yjAMpTsSlO5IsDoUAACAiED/C0Co4bY+AAAAAAAAWIbkFAAAAAAAACxDcgoAAAAAAACWITkFAAAAAAAAy5CcAgAAAAAAgGVITgEAAAAAAMAyJKcAAAAAAABgGZJTAAAAAAAAsAzJKQAAAAAAAFiG5BQAAAAAAAAsQ3IKAAAAAAAAlgmp5NTy5cvlcDhkGIb279/fYrmDBw9q8uTJGjJkiIYMGaLNmzcHMEoAAAAAAAC0V0glp66++mq9++67SklJabFMTU2NZs+erVWrVunjjz/WX//6V1166aUBjBIAACB8Nfdj4TfffKM5c+boggsu0IgRIzRt2jQVFhZaHCkAAAgVIZWcmjhxoux2e6tlnn/+eV188cWaMGGCJMlms6lXr16BCA8AACDstfRj4c0336xPP/1UBw4c0OzZs7V48WKLIgQAAKEmpJJT7fG3v/1NXbt21ZVXXqmRI0cqIyNDx48ftzosAACAsNDcj4VnnXWWrrjiChmGIUm6+OKL5XQ6LYgOAACEorBLTtXV1emtt97SunXrtG/fPvXr10///u//3mzZ7Oxs2e1276OqqirA0QIAAISfxx57TLNnz25xP30wAABwurBLTiUnJ+uyyy5Tv379ZBiG5s+fr/fff7/ZsllZWXK73d5H9+7dAxwtAABAeFm9erUKCwv14IMPtliGPhgAADhd2CWnrr32WuXn56uyslKS9Prrr2vEiBEWRwUAABD+HnnkEW3evFlbt25VbGys1eEAAIAQEVLJqaVLl8put8vtdmv69OkaOHCgJGnx4sXasmWLpFMjp+655x59//vfV1pamnbs2KG1a9daGTYQUkzTVL6zXC8XFCvfWS7TNK0OCQAQArKzs/XCCy/ozTff1Lnnnmt1OAAAIIQYJt88vRoSX0CkclfUKGPjXhWX1yjGFqXaeo+SEmKVs3Csbno6X5L0ZtYki6MEgM7het85S5cu1WuvvaaSkhL17NlTPXr00K5du5SUlKTU1FT16NFDktS1a1d98MEH7aqTcwI0b1r2bkn0vwCEvrau9dEBjAVAEDNNUxkb98pVVqN6j6na+npJkqusRgs27lXUP1ZgAgBEtnXr1jX7PL93AgCAMxVSt/UB8J8CV4Xc5V+r3tP4y0W9x1RReY2+rq23KDIAAAAAQDgjOQVAkuQsrVa0rfnRUQ23+AEAAAAA4GskpwBIkhyJcS0moGrrPYqx8XEBAAAAAPA9vm0CkCSNSYlXUkKsbFGNR0/ZogwlJ8SqW4zNosgAAAAAAOGM5BQASZJhGMpZOFYpPWMVYzMU28WmGJshR89Y5SwaZ3V4AAAAsIBpmsp3luvlgmLlO8tZ/ACAX7BaHwAve3ys8rImqcBVIWdptRyJcRqTEi+DlfoAAAAijruiRhkb96q4vMY7B2lSQqxyFo6VPT7W6vAAhBFGTgFoxDAMpTsSdM2YJKU7EkhMAQAARCDTNJWxca9cZTWqrTdV8129autNucpqtGDjXkZQAfApklMAAAAAgEYKXBVyl3+tek/jJFS9x1RReY0KXBUWRQYgHJGcAgAAAAA04iytVrSt+RH0MbYoOUurAxwRgHBGcgoAAAAA0IgjMU619Z5m99XWe+RIjAtwRADCGckpAAAAAEAjY1LilZQQK1tU49FTtihDyQmxGpMSb1FkAMIRySkAAAAAQCOGYShn4Vil9IxVjM1QbBebYmyGHD1jlbNoHIvmAPCpaKsDAAAAAAAEH3t8rPKyJqnAVSFnabUciXEakxJPYgqAz5GcAgAAAAA0yzAMpTsSlO5IsDoUAGGM2/oAAAAAAABgGZJTAAAAAAAAsAzJKQAAAAAAAFiG5BQAAAAAAAAsQ3IKAAAAAAAAliE5BQAAAAAAAMuQnAIAAAAAAIBlSE4BAAAAAADAMiSnAAAAAAAAYJmQSk4tX75cDodDhmFo//79rZY1TVNTpkzRueeeG6DoAAAAAAAA0FEhlZy6+uqr9e677yolJaXNso8++qgGDBgQgKgAAAAAAABwpkIqOTVx4kTZ7fY2y3300Ud65ZVXdPfddwcgKgAAAAAAAJypkEpOtUdtba2WLFmidevWyWaztVo2Oztbdrvd+6iqqgpQlAAAAAAAAJDCMDn1wAMPaO7cuRoyZEibZbOysuR2u72P7t27ByBCAACAwLj44ov13HPPqba21upQAAAAWhR2yandu3fr8ccfl8Ph0IQJE1RZWSmHw6Hjx49bHRoAAEBA7d27VxkZGUpKStJ9992nzz//3OqQAAAAmgi75NQ777wjl8slp9Opd999V2effbacTqd69epldWgAAAAB9Yc//EETJkzQl19+qdWrV6t///665pprtHv37jOus6XVkw8dOqTvf//7uuCCC5Senq6PPvrIF00AAAARwOfJKZvNpkWLFrVZbsmSJYqOju5Q3UuXLpXdbpfb7db06dM1cOBASdLixYu1ZcuWM4oXAAAgXF177bXavXu3Dh48qCVLlqhr167atGmTpkyZorS0NK1bt041NTUdqrOl1ZOXLl2qm2++WZ999pnuuusuZWZm+rAlAAAgnPk8OWWapkz9V9DsAAAgAElEQVTTbHfZjli3bp3cbrfq6ur0xRdfqLCwUJK0fv16zZo1q0l5h8OhEydOdOgYAAAA4WbYsGFau3atPv/8c/3617/WoEGD9Ne//lXLli1Tv3799P/+3//ToUOH2lVXc6snf/nllyooKND8+fMlSfPmzVNxcbG3rwYAANAay27rq6mpUUxMjFWHBwAAiDhnn322li9frk8++URvvPGGZs6cqa+++kq/+c1vNGTIEF1xxRXKy8vrcL3FxcXq27evd1S8YRhKTk5WUVFRs+VZMRkAAJzOkuTUiRMn9O6776pv375WHB4AACCiVVdX69ChQ96RTaZpyjAMbdu2TZdffrmuvPJKffXVV347PismAwCA03Vs0qcWpKamNtrOzc3Vrl27mi1bV1enkpIS1dfXa+nSpb44PAAAANrhs88+05o1a5STk6OTJ0/KNE1NmTJFy5cv19SpU/XCCy/owQcf1NatW3X77bdr/fr17ao3KSlJx44dU11dnaKjo2WapoqKipScnOznFgEAgHDgk+SU0+n0/tswDFVVVbU6PLtLly6aM2eOVq9e7YvDAwAAoAWmaWrLli367W9/q7y8PJmmqW7dumnRokVavny5hg8f7i27ePFi3XDDDRoxYoT+9Kc/tfsY5513nkaPHq1nn31WmZmZ2rRpk+x2u3fxGgAAgNb4JDl15MgRSac6P6mpqbr66qv18MMPN1u2S5cu6tWrV4dX6gMAAEDHPPTQQ1q7dq2KiopkmqaSkpK0bNkyLVmyRAkJCc3+TVxcnCZMmKD/+Z//aXb/0qVL9dprr6mkpETTp09Xjx49VFhYqHXr1ikzM1OrV6/W2WefraefftqfTQMAAGHEJxmi05cSXrBggS699NImywsDAAAgsH76059Kki655BLddtttuuqqq2Sz2dr8u+HDh2vixInN7lu3bl2zz1944YX685//fObBAgCAiOXz4Uv8SgYAABAcMjIydNttt2nUqFEd+rs77rhDd9xxh5+iAgAAaIx76wAAAMLUM888Y3UIAAAAbfJbcqqgoEC5ubn69NNPVVlZKdM0m5QxDEN5eXn+CgEAACCi2Ww2ZWZmasOGDa2WW7JkiZ5++mnV1dUFKDIAAIB/8kty6o477tCjjz7qTUgZhtEoOdWwbRiGPw4PAAAAnVqsprkfCFsqCwAAYIUoX1f48ssvKzs7W/369dO6det0+eWXS5K2b9+uNWvWaPz48TJNU3fffbd27Njh68MDCGOmaSrfWa6XC4qV7yznixQA+EhNTY1iYmKsDgMAAEQon4+cevLJJ2Wz2ZSXl6dBgwZpz549kqRp06Zp2rRpWrZsme677z49/PDDuvbaa319eABhyl1Ro4yNe1VcXqMYW5Rq6z1KSohVzsKxssfHWh0eAISsEydO6N1331Xfvn2tDgUAAEQonyen9u3bp3HjxmnQoEEtlnnggQf07LPPatWqVcrNzfV1CADCjGmayti4V66yGtV7TNXW10uSXGU1WrBxr97KmsRtwgDwD6mpqY22c3NztWvXrmbL1tXVqaSkRPX19Vq6dGkAogMAAGjK58mpkydPKjk52bvdpUsXSVJVVZW6d+8uSYqKitK4ceO0e/duXx8eQBgqcFXIXf616j2Nb+Or95gqKq9RgatC6Y4Ei6IDgODidDq9/zYMQ1VVVaqqqmqxfJcuXTRnzhytXr06ANEBAAA05fPkVK9evXTixAnvdmJioqRTHaXhw4d7n6+urlZlZaWvDw8gDDlLqxVtM/RdfdN9MbYoOUurSU4BwD8cOXJE0qlRp6mpqbr66qv18MMPN1u2S5cu6tWrl6Kj/baAMwAAQJt83hNxOBxyuVze7VGjRsk0TT3//PPeX+RKSkq0e/dupaSk+PrwAMKQIzFOtfWeZvfV1nvkSIwLcEQAELxO718tWLBAl156KX0uAAAQ1HyenJo6dapWrVolp9Mph8OhH/7wh0pISNBDDz2kQ4cOKTk5Wbm5uaqurta8efN8fXgAYWhMSrySEmK9c041sEUZSk6I1ZiUeAujA4Dg9fTTT1sdAgAAQJt8npy6/vrrdfToURUXF8vhcCguLk5PP/20rr/+em3atMlb7nvf+55++tOf+vrwAMKQYRjKWTi2yWp9yQmxylk0jsnQAQAAACCE+Tw5NWTIED311FONnps5c6YOHTqkP/3pTyovL9eQIUM0c+ZM2Ww2Xx8eiEimaarAVSFnabUciXEakxIfdgkbe3ys8rImhX07AaAzFi5cKMMwtHr1avXu3VsLFy5s998ahqENGzb4MToAAIDmBWz2y/PPP58ligE/cFfUNBlRlJQQq5yFY2WPj7U6PJ8yDEPpjgQmPweAFjzzzDMyDEN33XWXevfurWeeeabdf0tyCgAAWIWlWYAQZpqmMjbu9c7FVFt/ajk7V1mNFmzcq7eyJjGyCAAiSMMcU3379m20DQAAEMz8lpx655139Pjjj2vPnj06fvy45s+f7/017s0339TOnTu1fPly9enTx18hAGGvwFUhd/nXjSYJl6R6j6mi8hoVuCoYZQQAEWTBggWtbgMAAASjKH9UumrVKk2ePFm5ubk6evSoamtrZZr//PJ8zjnn6KGHHtLmzZv9cXggYjhLqxVta35kVIwtSs7S6gBHBAAAAABAx/g8ObV161bdf//96tevn1566SV98cUXTcqMHTtWvXr10quvvtqhupcvXy6HwyHDMLR///5my+zYsUNjx47V0KFDNWzYMK1YsUIej+eM2gIEO0dinGrrm39/19Z75EiMC3BEAAAAAAB0jM9v63vsscfUtWtXbd26VcOGDWux3IgRI3To0KEO1X311VdrxYoVmjBhQotl4uPj9Yc//EGpqan65ptv9IMf/EA5OTnKzMzs0LGAUDAmJV5JCbHeOaca2KIMJSfEakxKvIXRAQACLScnp1N/n5GR4aNIAAAA2s/nyan8/HyNHTu21cSUJPXq1Ut79uzpUN0TJ05ss8yoUaO8/z7rrLM0cuRIOZ3ODh0HCBWGYShn4dgmq/UlJ8QqZ9E4JkMHgAiTmZnZqc9+klMAAMAKPk9OVVdXt2uS86+++srvt9uVlJQoNze3w7cPAqHEHh+rvKxJKnBVyFlaLUdinMakxJOYAoAIlJGRwec/AAAIOT5PTvXu3VuFhYVtlvv000+VlJTk68N7VVZWaubMmVqxYoXGjBnTbJns7GxlZ2d7t6uqqvwWD+BPhmEo3ZHAynwAEOGeeeYZq0MAAADoMJ9PiD5hwgTt379f7733XotlXn31VRUWFuqyyy7z9eElSSdPntSMGTM0e/ZsZWVltVguKytLbrfb++jevbtf4gEAAAAAAEDzfJ6cuv3222UYhubOnatXXnlFdXV1jfZv27ZNixcvVkxMjP7jP/7D14dXVVWVZsyYoRkzZujee+/1ef0AAAAAAADwHZ/f1jd69Gj96le/UlZWlubNm6du3brJMAxt2rRJmzdv1smTJ2Wapn7zm99o6NChHap76dKleu2111RSUqLp06erR48eKiws1OLFizVr1izNmjVLjz32mPbu3avq6mpt3rxZknTNNdfoZz/7ma+bCgAAEFSKiookSf369ZPNZvNut1dycrI/wgIAAGiVYZqm2Xaxjtu+fbtWrlyp/Px8nX6ItLQ0rVq1SldeeaU/Dtspdrtdbrfb6jCAoDQte7ck6c2sSRZHAgCdE87X+6ioKEVFRelvf/ubLrjgAkVFRbV7gnTDMJqMeA+UcD4nQGfQ/wIQLtq61vt85FSD6dOna/r06SorK9ORI0fk8XiUlJSkvn37+uuQAAAAES05OVmGYSgmJqbRNgAAQDDzW3KqQc+ePdWzZ09/HwYAACDiOZ3OVrcBAACCkc8nRAcAAAAAAADaq9Mjp3Jycjr19xkZGZ0NAQAAAEHi9ddf17333iuPx6O6ujrdeeedWrBggdVhAQCAINbp5FRmZuYZz2VgGAbJKQAAAD+rra3Vpk2btGvXLrndbpmmKbvdrsmTJ2vevHnq0qWLT45jmqbmz5+vXbt2KS0tTU6nU4MHD9bcuXPVo0cPnxwDAACEn04np1qbaNPlcikuLk6JiYmdPQwAAADOwAcffKAbbrhBLpdL/7pI8/r163XPPffo+eef1/jx431yPMMwdOLECUlSZWWlevbsqa5du/qkbgAAEJ46nZxqbaLNqKgoXXPNNdq4cWNnDwMAAIAO+vTTTzVt2jRVVVWpX79+uu6669S/f39Jp/pwL774olwul6ZPn669e/dq8ODBnTqeYRh68cUXNXfuXMXFxamiokKbN29uMjIrOztb2dnZ3u2qqqpOHRcAAIQ2JkQHAAAIU/fff7+qqqq0fPlyHT58WI888ohuvfVW3XrrrXr44Yd1+PBh3XbbbaqqqtLKlSs7fby6ujqtWrVKmzdvlsvlUl5enm688UaVlpY2KpeVlSW32+19dO/evdPHBgAAoYvkFAAAQJjasWOHBgwYoEcffbTZeaViYmKUnZ2tAQMGaMeOHZ0+3v79+3X06FFNnDhRkpSeni673a59+/Z1um4AABC+SE4BAACEqerqao0dO7bVxWsMw1B6erqqq6s7fbykpCQdO3ZMH3/8sSSpsLBQhw8f1oUXXtjpugEAQPjq9JxTAAAACE6pqalNbqlrTllZmXcuqs7o3bu3nnzySV177bWKioqSx+PRmjVrlJyc3Om6AQBA+GLkFAAAQJjKyMjQ7t279dFHH7VY5qOPPtKuXbt04403+uSYN9xwgw4ePKgDBw7o4MGD+vGPf+yTegEAQPgiOQUAABCmbr/9dl155ZWaMmWK1qxZo8rKSu++yspK/fa3v9XUqVM1c+ZMrVixwsJIAQBAJOv0bX1vv/12q/tLSkpaLdMwYSYAAAA6JzU1tdnnjx8/rttuu0233Xabzj33XEnSiRMnvPv/8pe/aNCgQTp8+HBA4gQAADhdp5NTkydPbnGSTcMwtH37dm3fvr3F/XV1dZ0NAQAAAJKcTmeL+0zTlCRVVFQ02edyuVqdNB0AfMU0TRW4KuQsrZYjMU5jUuL5/AHgmwnRGzo7gfo7AAAANHXkyBGrQwCAFrkrapSxca+Ky2sUY4tSbb1HSQmxylk4Vvb4WKvDA2ChTienPB6PL+IAAABAJ6WkpFgdAgA0yzRNZWzcK1dZjeo9pmrr6yVJrrIaLdi4V29lTWIEFRDBmBAdAAAAAOBXBa4Kucu/Vr2n8d0z9R5TReU1KnA1veUYQOQgOQUAAAAA8CtnabWibc2PjIqxRclZWh3giAAEE5/MOQUAAADrTZkyRYZh6Pe//73sdrumTJnS7r81DEN5eXl+jA5AJHMkxqm2vvkpYWrrPXIkxgU4IgDBhOQUAABAmNi1a5cMw1BNTY13u72Y6wWAP41JiVdSQqx3zqkGtihDyQmxGpMSb2F0AKxGcgoAACBM7Ny5U5KUnJzcaBsArGYYhnIWjm2yWl9yQqxyFo0jQQ5EOJJTAAAAYWLSpEmtbgOAlezxscrLmqQCV4WcpdVyJMZpTEo8iSkAJKcAAADCVVFRkbp3766EhIRWy1VUVOjkyZPeEVcA4C+GYSjdkaB0R+ufSwAiS0it1rd8+XI5HA4ZhqH9+/e3WG7Dhg0aNGiQBgwYoCVLlqi2tjaAUQIAAASH/v37684772yz3IoVK5SamhqAiAAAAJoKqeTU1VdfrXfffVcpKSktljly5Ijuu+8+vfPOOyosLNQXX3yhJ598MoBRAgAABAfTNGWaZtsF/1EWAADACj5PThUXFysnJ0effvppi2U++eQT5eTkyO12d6juiRMnym63t1omNzdXs2bNUp8+fWQYhm655Ra98MILHToOAABAJDl58qS6dOlidRgAACBC+Tw59fjjj+umm25q9dc30zSVmZmpJ554wteHV1FRUaORVQ6HQ0VFRT4/DgAAQKjzeDw6ePCgduzYwXxTAADAMj5PTr3xxhsaMmSIBg8e3GKZIUOGaOjQodq2bZuvD98h2dnZstvt3kdVVZWl8QAAAHSWzWbzPiTp97//faPnTn/ExMRo5MiRKisr09y5cy2OHAAARCq/3NY3cODANssNHDhQxcXFvj68kpOT5XK5vNtOp7PFXwKzsrLkdru9j+7du/s8HgAAgEBqmGfKNE0ZhtFo+18f0dHRcjgcuv322/XAAw9YHToAAIhQPk9O1dTUqFu3bm2W69atm06ePOnrw2vevHnasmWLSkpKZJqm1q5dq+uvv97nxwEAAAhGHo/H+2iYSuH0505/fPvttzp8+LB++ctfMucUAACwjM+TU3379tX+/fvbLHfgwAGdd955Hap76dKlstvtcrvdmj59uneE1uLFi7VlyxZJUmpqqh544AFdcsklGjhwoHr16qWlS5d2vCEAAAAhbuXKlZozZ47VYQAAALQq2tcVXnrppXr22We1adMmzZs3r9kymzdv1ieffKIf//jHHap73bp1zT6/fv36RttLlizRkiVLOlQ3gMhimqYKXBVyllbLkRinMSnxMgzD6rAAwKdWrlxpdQgAAABt8nly6rbbbtNzzz2njIwMud1uLVy4UD169JB0apnijRs36mc/+5mioqK0fPlyXx8eANrkrqhRxsa9Ki6vUYwtSrX1HiUlxCpn4VjZ42OtDg8A/OL999/Xrl275Ha7ZZqm7Ha7Jk+erPHjx1sdGgAAiHA+T06NHj1aDz74oO666y5lZWXpjjvuUN++fSVJx44d885/sHr1ao0dO9bXhweAVpmmqYyNe+Uqq1G9x1Rtfb0kyVVWowUb9+qtrEmMoAIQVg4fPqyMjAy9//77kk59DkryftaNGzdOOTk57VrQBgAAwB98npySpDvvvFMXXnihVq5cqQMHDsjtdnv3jRgxgvkPAFimwFUhd/nXqveYjZ6v95gqKq9RgatC6Y4Ei6IDAN86evSoLr30UpWUlCg2NlYzZsxQ//79JZ1a0Xjbtm16//33NXHiROXn56tfv34WRwwAACKRX5JTkjRr1izNmjVLX3zxhYqKiiRJycnJ6t27t78OCQBtcpZWK9pm6Lv6pvtibFFyllaTnAIQNu6//36VlJRo3rx5euKJJ9SrV69G+0tLS7Vs2TLl5uZq5cqVTebxBAAACAS/Jaca9O7dm4QUgKDhSIxTbb2n2X219R45EuMCHBEA+M/WrVt1/vnn67nnnlOXLl2a7E9MTNSzzz6rPXv26PXXX7cgQgAAACnK6gCASGWapvKd5Xq5oFj5znLvHCDwrzEp8UpKiJUtqvG8UrYoQ8kJsRqTEm9RZADge+Xl5ZowYUKziakGXbp00YQJE1RRURHAyAAAAP6p0yOncnJyJElXXXWVevTo4d1ur4yMjM6GAIQcVouzjmEYylk4tsnrn5wQq5xF45gMHUBYsdvtqq6ubrNcTU0N800BAADLdDo5lZmZKcMwdPHFF6tHjx7e7fYiOYVIw2px1rPHxyova5IKXBVyllbLkRinMSnxvO4Aws4111yjxx9/XJ9//nmLyafPP/9cO3bs0E9+8pMARwcAAHBKp5NTGRkZMgxD55xzTqNtAM1jtbjgYBiG0h0JvNYAwtp9992nXbt2acqUKfrVr36lK6+8stH+1157TbfffrvS0tK0cuVKnxzz22+/1e23367t27frrLPO0ogRI/Tss8/6pG4AABCeOp2ceuaZZ1rdBtAYq8UBAALlRz/6kaKionTo0CHNnj1b5557rhwOhyTJ6XTqxIkTkqTx48frRz/6UaO/NQxDeXl5HT7m3XffLcMw9Nlnn8kwDJWUlHS6HQAAILx1Ojm1cOFCTZgwQQsXLpQkFRUVqXv37kpI4Ms10BxWiwMABMquXbu8/zZNUxUVFc1OfL5nz54mz53JSPjq6mpt2LBBbrfb+/d9+vTpcD0AACCy+GzkVENyqn///srMzNSGDRs6WzUQlhpWi2uYc6oBq8UBAHxt586dAT3e4cOHlZCQoNWrV+utt95St27d9POf/1xTp04NaBxAKDJNs8l8mAAQKTqdnLLZbKqtrfVum6Yp0zRb+QsgsrFaHAAgUCZNmhTQ49XV1cnlcmno0KH6xS9+oX379mnatGn66KOP1Lt3b2+57OxsZWdne7erqqoCGicQbFpaydnjMRVji7I6PADwu04np8477zzt379fpmnypRpoJ1aLAwCEo+TkZEVFRenf/u3fJEmjRo1S//79dfDgwUbJqaysLGVlZXm37XZ7wGMFgkVrKzlHGZKjJ1M+AAh/nU5OTZkyRc8995xSU1PVv39/SdK2bds0ZcqUNv/2TCfaBMIBq8UBAALFNE1t3bpVe/bs0fHjxzVu3DjvlAzHjx9XRUWFBgwYIJvN1qnjJCYmaurUqdq+fbuuuOIKHTlyREeOHNGQIUN80QwgLLW2knO9pK9rm1lFBwDCTKeTUw899JA+++wz5efny+VySZJKSkratTILo0QAAAD868CBA7ruuut06NAh70j32tpab3LqzTff1I033qhXXnlFM2fO7PTx1q5dq0WLFumuu+5SVFSU1q1bp379+nW6XiBctbaSs2GoxYV0ACCcdDo5df755+uDDz6Qy+WSy+XS5MmTNWPGDN11112+iA8AAABnyO126wc/+IHKysp0xRVXaPLkyVqxYkWjMnPmzFFMTIz++Mc/+iQ5lZqaGvCJ2IFQ1tpKzqYp5pwCEBE6nZxqkJKSopSUFEmnlgwO9AScAAAAaGz16tUqKyvTr3/9ay1fvlySmiSnYmNjNWLECOXn51sRIhDxWlvJOcqQusV07nZbAAgFPktONThy5Ii6d+/u62oBAADQQdu2bdPgwYO9iamWOBwORjsBFmltJed6FkEHECF8npxqGD0FAAAAax09elSzZ89us5xhGKqsrAxARACa09JKzpc/+rbVoQFAQHQ6OfX226c+MMeOHauzzjrLu91eEydO7GwIAAAAaEZcXJyOHz/eZrkjR44oIYHVYwErsZIzgEjW6eTU5MmTZRiGPv74Y11wwQXe7fYwDEN1dXWdDQEAAADNuOiii/SXv/xFpaWlSkxMbLaMy+XSgQMHNG3atABHBwAAcEqnk1MTJ06UYRiKjY1ttA0AAABrzZ8/X2+//bYWL16s559/3ttfa/Ddd99p2bJlqq2t1fz58y2KEgAARLpOJ6d27drV6jYAAACscdNNN+m5557Tli1bNHjwYM2YMUOSdODAAS1fvlxbtmxRUVGRfvCDH+i6666zOFoAABCpoqwOoKMOHTqk73//+7rggguUnp6ujz76qEkZj8ejrKwsDR06VGlpabrssstUWFhoQbQAAADWsdls+tOf/qQbbrhBn3/+udavXy9J2rdvn9asWaOioiLNmzdPmzdvtjhSAAAQyXy+Wp+/LV26VDfffLMyMzOVm5urzMxM5efnNyqzZcsWvffeezpw4IBiYmK0atUq3XPPPXrppZcsihoAAMAa3bt313PPPaf77rtPr7/+uv7+97/L4/EoKSlJP/zhDzVy5EirQwQAABHO58mpbdu26Ze//KXuvfdeTZkypdkyeXl5+u///m/99Kc/7dDkm19++aUKCgr0xhtvSJLmzZunn/zkJyosLNTAgQO95QzD0LfffqtvvvlG0dHRqqyslN1u71zDAAAAQtjgwYM1ePBgq8MAAAD/v717D4+ivvc4/hk2gAZUiJBDZBNikIugJKQEqBeuFaiUSxG8cEByMGKlViWt2NPLYy/U+liaU46e00aEKOKlR4hKUY4KAscqSiJgxStoc8NQSAhCEmluv/MHzZY1IWzI7M7s7vv1PHkeszOZ+f2c3dkv3/nOd9CC7bf15eXlaefOncrIyDjtOiNHjtTbb7+txx57rF3bLi0tVUJCgmJiTubULMtSUlKSSkpK/NabNm2axo0bpz59+ighIUFbtmzRL37xixbby8nJkdfr9f1UV1e3azwAAAAAAADoGNuTU4WFhUpLS9N555132nXOO+88DR8+XDt37rR7974x7N27VwcOHNDnn3+uiRMn6jvf+U6L9bKzs1VWVub76d69e1DGAwAA4IT//d//1YQJE/Taa6+ddp0tW7ZowoQJevXVV0M4MgDoOGOMCoqO6NnCUhUUHZExxukhAThLtt/WV15erlGjRp1xvcTERO3Zs6dd205MTFR5ebkaGhoUExMjY4xKSkqUlJTkt96aNWs0YcIE9ejRQ5K0YMECTZo0qV37AgAACHftrWhvT7sFAHBSWVWtbl69U6VHatXZ00n1jU1KjIvVmoUj5e0Z6/TwALST7ZVTXbp00fHjx8+4XnV1tTp1at/u4+PjlZ6errVr10qS1q9fL6/X69dvSpJSUlL02muvqa6uTpK0ceNGXXbZZe3aFwAAQLhzQ0U7ANjNGKObV+9UcWWt6huNausaVd9oVFxZqwWrd1JBBYQh25NTAwYM0BtvvKHa2trTrlNbW6s33nhDKSkp7d5+bm6ucnNzNXDgQD3wwAPKy8uTJGVlZWnDhg2SpO9+97u6+OKLlZqaqmHDhmnLli36/e9/f3YTAgAACFPl5eUtKsxb01ydDgDhoLC4SmVHvlRjk38SqrHJqORIrQqLqxwaGYCzZfttfdOmTdPPfvYz3XHHHVq1apUsy/JbbozR9773PX3xxReaMWNGu7c/aNAg7dixo8Xrjz76qO+/u3btqpUrV7Z/8ABwFowxKiyuUlFFjZJ7ddOIfj1bnPsAwAnBrGgHAKcUVdQoxmOprrHlss6eTiqqqFFGclzoBwbgrNmenLrzzjv1yCOP6PHHH9df/vIXLVy40PfY4o8++kirV6/W7t271adPH91111127x4AQop+BwDc7NSK9tjY1s9JHaloBwAnJPfqpvrGplaX1Tc2KblXtxCPCEBH2Z6c6tGjh1588UVNmzZNu3bt0u7du/2WG2Pk9Xq1YcMGxcWRzRoNdzgAACAASURBVEZ4oUIGpzq130Fjk1F948nLd839DjZnj+X9AcBRwa5oBwAnjOjXU4lxsb4YrJmnk6WkuFiN6NfTwdEBOBu2J6ckKTU1VR999JFWrlypl19+WcXFxZKkpKQkTZkyRVlZWerWjWw2wgsVMviqQPodUFIOwElUtAOIRJZlac3CkS1i86S4WK25ZRQXB4EwFJTklCTFxsbqrrvuItBBRKBCBq2h3wEAt6OiHUCk8vaM1ZbssdzVAESIoCWngEhChQxaQ78DAOGAinYAkcqyLGUkxxGHAxEgaMmpY8eOae3atXrzzTd1+PBhTZw4UUuXLpUkffLJJyoqKtKYMWN0zjnnBGsIgG2okEFr6HcAIFxQ0Q4AANwsKMmpV155RXPnzlVVVZWMMbIsS3379vUt//jjjzVz5kw9/fTTuv7664MxBMBWVMigNfQ7AAAAAICOsz059eGHH+rb3/626urqdPvtt2vs2LG64YYb/NaZPHmyYmNj9cILL5CcQligQganQ78DAG7W2NiompoaxcbGKibmn2Hfl19+qQcffFB79uxRcnKy7rnnHl100UUOjhQAAEQz25NT999/v06cOKFnn31Ws2bNkqQWyakuXbooLS1N7777rt27B4KCChm0hX4HANzqF7/4hZYtW6Zt27bp6quvlnSyCfq4ceNUWFjoq3DPz8/Xnj171LMnF1sAAEDo2Z6c2rp1q1JTU32JqdPxer364IMP7N49EDRUyAAAws2WLVvUp08fX2JKkv70pz+poKBAAwcO1OLFi7Vp0ya98sorWrlypa8/KAAAQCh1snuDhw8f1sCBA8+4XkNDg2pqauzePRBUzRUyc0YkKiM5jsQUAMDV/vrXv2rw4MF+r73wwguyLEtPPvmk7rzzTv3pT39S7969tW7dOodGCQAAop3tyakLLrhABw4cOON6n332meLj4+3ePQAAAP6hsrJSffr08XvtjTfeUN++ffW1r31NkhQTE6PRo0erpKTEiSECAADYn5xKT0/XO++802aAs3fvXr377rsaNWqU3bsHAADAP8TExPhVqldVVWnfvn268sor/dY777zz9MUXX4R6eAAAAJKCkJzKysrSiRMndNNNN+ngwYMtlldUVCgrK0vGGGVlZdm9ewAAAPxDSkqK3nrrLTU1NUmSNm7cKGOMrrrqKr/1Dh06pN69e9u677y8PFmWpeeff97W7QIAgMhje3Jq9uzZmjNnjnbs2KH+/ftr0qRJkk6WkE+fPl0pKSnauXOn5s6dq8mTJ9u9ewAAAPzD9OnTdejQIc2YMUMrVqzQvffeK4/Ho2nTpvnWMcZo9+7duvjii23bb1FRkVauXKnRo0fbtk0AABC5bE9OSdJTTz2lf//3f5ckbd68WZK0b98+bdy4UXV1dfr+97+vxx57LBi7BgAAwD8sXbpUQ4cO1YsvvqglS5bo4MGDuueee5SUlORb589//rMqKipaVFOdraamJmVlZemhhx5S165dbdkmAACIbDHB2KjH49GvfvUr/eAHP9DWrVv12WefqampSYmJiZo4cSKN0AEAAELg/PPP186dO7Vu3Tr97W9/U0ZGhsaOHeu3TmVlpe666y7dcMMNtuwzJydHV155pa/hOgAAwJkEJTnVrGfPnpo1a1YwdwEAAIA2nHvuuZo/f/5pl8+cOVMzZ860ZV979+7V+vXr9X//939trpeTk6OcnBzf79XV1bbsHwAAhKegJqcAAO1jjFFhcZWKKmqU3KubRvTrKcuynB4WAATk9ddfV1FRkQYMGCBJOnjwoBYtWqTy8nLdfvvtvvWys7OVnZ3t+93r9YZ8rAAAwD2Clpz64IMP9J//+Z/atm2bysrKZIyR1+vV+PHjdccdd+iyyy4L1q4BICyVVdXq5tU7VXqkVp09nVTf2KTEuFitWThS3p6xTg8PQBg6UwXTV40ZM6ZD+7v99tv9klDjxo3T3XffbVtlFgAAiExBSU7913/9l7Kzs9XQ0CBjjO/1ffv2ad++fcrLy9NvfvMb3XnnncHYPQCEHWOMbl69U8WVtWpsMqpvbJQkFVfWasHqndqcPZYKKgDtNm7cuIDPHZZlqaGhIcgjAgAAaMn25NSmTZv0ve99T5ZladasWVqwYIHv0cRFRUV6/PHHlZ+fryVLlmjAgAH65je/afcQACDsFBZXqezIl2psMn6vNzYZlRypVWFxlTKS4xwaHYBwNWbMmFaTU01NTSouLlZpaakk6etf/7o6d+5s+/63bdtm+zYBAEDksT059eCDD8qyLD3zzDOaM2eO37KhQ4dq6tSpWrduna6//no9+OCDJKcQVPTvQbgoqqhRjMdSXWPLZZ09nVRUUUNyCkC7nSk59Je//EWZmZnq1q2bXnrppdAMCgAA4Cs62b3Bd955RyNHjmyRmDrV7NmzNWrUKL3zzjvt3v6+fft0xRVXaODAgcrIyND777/f6nrvvfeexo0bp0svvVSXXnqp8vPz270vhLeyqlpNzNmuuSvf0n0b3tfclW9pYs52lVXVOj00oIXkXt1U39jU6rL6xiYl9+oW4hEBiAbDhg1Tfn6+/vznP+s3v/mN08MBAMcYY1RQdETPFpaqoOiIX3saAMFne3LKsiz179//jOv179//rCpYbrvtNi1atEiffPKJ7r33XmVmZrZYp7a2VjNmzNCyZcv04Ycfau/evbr66qvbvS+Er1P799Q3GtXWNaq+0fj69/BlA7cZ0a+nEuNi5enkf170dLKUFBerEf16OjQyAJEuOTlZGRkZWrNmjdNDAQBHcFEbcJ7tyalhw4Zp3759Z1xv3759uvzyy9u17UOHDqmwsFDz5s2TJF133XUqLS3V/v37/dZ76qmnNHr0aF111VWSJI/Ho969e7drXwhvgfTvAdzEsiytWThS/S6MVWePpdguHnX2WEq+MFZrbhnF7agAgqp3794qKipyehgAEHJc1AbcwfbkVHZ2tgoKCvTMM8+cdp0//vGPKigo0JIlS9q17dLSUiUkJCgm5mSrLMuylJSUpJKSEr/1PvjgA3Xt2lXf+ta3lJaWpptvvlmHDx9u/2QQtpr797SmuX8P4DbenrHakj1WT906Wj+fPlRP3Tpam7PHqm+Pc50eGoAIVldXp4KCAsXGxjo9FAAIOS5qA+5ge3Lqa1/7mpYsWaJ58+Zp9uzZ2rBhg9577z2999572rBhg+bMmaN//dd/1ZIlS5SRkaGSkhK/Hzs0NDRo8+bNys3N1e7du9W3b1/dfvvtLdbLycmR1+v1/VRXV9uyfziP/j0IV5ZlKSM5TnNGJCojOY6KKQBBU1NTo8LCQl8l+vjx450eEgCEHBe1AXew/Wl9F198saST5ZHPPfecnnvuuRbrGGP0u9/9Tr/73e/8XrcsSw0NDafddmJiosrLy9XQ0KCYmBgZY1RSUqKkpCS/9ZKSkjR+/Hj17dtXkjRv3jxNnjy5xfays7OVnZ3t+93r9QY+Ubhac/+e4spav6sg9O8BAEQTj8dzxnWMMerRo4eWLVsWghEBgLtwURtwB9uTU4mJiUG70h8fH6/09HStXbtWmZmZWr9+vbxery655BK/9a6//nqtWrVKx44d0/nnn6+XXnpJqampQRkT3Km5f8/Nq3eq9EitOns6qb6xSUlx9O8BAESPtnqldO7cWX379tU3vvEN/ehHP1JycnLoBgYALsFFbcAdbE9OBbuZZm5urjIzM3X//ffr/PPPV15eniQpKytL06dP1/Tp05WUlKQf/ehHuuKKK9SpUyf17dtXjzzySFDHBfdp7t9TWFyloooaJffqphH9epKYAgBEjaam1qsBAAAncVEbcAfbk1PBNmjQIO3YsaPF648++qjf7/Pnz9f8+fNDNSy4VHP/nozkOKeHAjjCGEOCFgAAoA1c1AacF3bJKQBAYMqqaltcBUyMi9WahSPl7clTuYBo1NDQoBUrVuj5559XRUWFvF6vbrrpJi1cuNDpoQGAo7ioDTjL9qf1fVVDQ4N++9vf6uqrr9all16qa665RqtXrw72bgEgqhljdPPqnSqurFV9o1FtXaPqG42KK2u1YPXONvvQAAhf+fn5io+P149//OMWy5qamjR16lQtXbpUb7zxhj7++GNt2bJFt956qzIzM0M/WAAAgH/ocHKKIAgA3KewuEplR770a+wpSY1NRiVHalVYXOXQyAAE09atW1VZWanZs2e3WLZy5Uq9+uqrMsZo+vTpevjhh7V06VKde+65euKJJ/TKK684MGIAAAAbbusLJAiSpOnTp2vSpEkqKSnRww8/rCeeeEJz587VpEmTOjoEAMBXFFXUKMZjqa6x5bLOnk4qqqihbB2IQG+//bYSEhI0fPjwFstyc3NlWZZuvPFGPfnkk77XR44cqdmzZ+uJJ54gLgMAAI7ocHKKIAgA3Ce5VzfVN7b+lK76xiYl9+oW4hEBCIXy8nKlpaW1eL2iokJ79uyRZVm65557/JbNmjVLycnJevvtt0M1TAAAAD8dvq3vTEGQJIIgAAixEf16KjEuVp5O/k+Z8XSylBQXqxH9ejo0MgDBVFFRoZ49W36+CwoKJEm9e/duNW4bMmSIPv/886CPDwAAoDUdTk4RBAGA+1iWpTULR6rfhbHq7LEU28Wjzh5LyRfGas0to3g0MhChPB6PDh8+3OL1Xbt2SZLS09Nb/bsePXqooaEhqGMDgGhhjFFB0RE9W1iqgqIjPIgGCECHb+sjCAIAd/L2jNWW7LEqLK5SUUWNknt104h+PUlMARGsX79+2rVrl+rq6tSlSxff61u2bJFlWRo1alSrf1dRUaF/+Zd/CdUwASBilVXV6ubVO1V6pFadPZ1U39ikxLhYrVk4Ut6esU4PD3CtDldOnRoEnYogCACcZ1mWMpLjNGdEojKS40hMARFu/Pjxqqys1E9/+lPfa1u3btX27dslSVOnTm3173bv3q2LLrooJGMEgEhljNHNq3equLJW9Y1GtXWNqm80Kq6s1YLVO6mgAtrQ4eQUQRDsRhks4E58NgH3u/vuu9WlSxctX75ciYmJSk9P1+TJkyVJo0aN0ogRI1r8zY4dO3T48OHTXlAEAASmsLhKZUe+VGOTf4zU2GRUcqRWhcVVDo0McL8O39Z39913a9WqVVq+fLmeeuop9e7dW3v37pV05iDopptu6ujuEWEogwXcic8mEB4uueQSPfnkk8rMzNSBAwd04MABSVLfvn31+OOPt/o3ubm5kqSJEyeGbJwAEImKKmoU47FU19hyWWdPJxVV1CgjOS70AwPCQIeTUwRBsMupZbCNTUb1jSfP6s1lsJuzx3JLEuAAPptAeJk1a5auuuoqbdy4UX/729+UlJSkmTNnqlu3bq2uP3LkSA0fPlwTJkwI8UgBILIk9+qm+samVpfVNzYpuVfr52EANiSnJIIg2COQMliuNAChx2cTCD/x8fFauHBhQOsuXrw4yKMBgOgwol9PJcbF+i7oNfN0spQUF6sR/Vo+5R7ASbYkpySCIHQcZbCAO/HZBACgdcYYnooLH8uytGbhyBatEJLiYrXmllG8N4A22JacAjqKMljAnfhsAgDQEv0Y0Rpvz1htyR5L0hJopw4/rQ+wS3MZrKeT/4mbMljAWXw2AQDwd2o/xvpGo9q6RtU3Gl8/Rp5oG90sy1JGcpzmjEhURnIciSkgACSn4BrNZbD9LoxVZ4+l2C4edfZYSr6QMljASXw2AQDwF0g/RgBA4LitD65CGSzgTnZ/NunRAQAIZ/RjBAB7kZyC6zSXwfKFDriLXZ9NenQAAMId/RgBwF7c1gcACBl6dACR7cSJE5o5c6YGDhyo1NRUXXPNNdq/f7/TwwJsRz9GhJIxRgVFR/RsYakKio4QLyEiUTkFAAiZQHp0UDUJhLdFixbpm9/8pizL0sMPP6ysrCxt27bN6WEBtmrux/jVSuCkOPoxwl5UnCNakJwCAIQMPTqAyHbOOefo2muv9f0+evRoLV++3MERAcFDr1QE26kV541NRvWNJwOo5orzzdljeb8hYpCcAgCEDD06gOiyYsUKzZgxw+lhAEFDr1QEExXniCb0nAIAhAw9OoDocf/992v//v369a9/3WJZTk6OvF6v76e6utqBEQKAuzVXnLemueIciBRhl5zat2+frrjiCg0cOFAZGRl6//33T7uuMUYTJkxQjx49QjjC6ESTPgCBaO7R0e/CWHX2WIrt4lFnj6XkC8++RwfnH8B9li9frvz8fG3atEmxsS17omRnZ6usrMz30717dwdGCQDuRsU5oknY3dZ32223adGiRcrMzNS6deuUmZmpgoKCVtf9j//4D/Xv31+7du0K8SijC036ALSHnT06OP8A7pOTk6Onn35amzdv5gIhAHRAc8V5c8+pZh2pODfG0CcNrhRWlVOHDh1SYWGh5s2bJ0m67rrrVFpa2uojit9//309//zz+uEPfxjqYUYVHgsP4Gw09+iYMyJRGclxZ10xxfkHcJeysjJ9//vf19GjRzV+/HilpaVp1KhRTg8LAMKS3RXnZVW1mpizXXNXvqX7NryvuSvf0sSc7Sqrqg3SDIDAhVXlVGlpqRISEhQTc3LYlmUpKSlJJSUluuSSS3zr1dfX69Zbb9WqVavk8XhOu72cnBzl5OT4fqffQfvRpA+AUzj/AO7j9XpJDAOAjeyqOOfJf3C7sKqcCtTPf/5zzZo1S5deemmb69HvoONo0gfAKZx/AABANLCj4jyQi3qAk8IqOZWYmKjy8nI1NDRIOpn9LSkpUVJSkt9627dv10MPPaTk5GRdddVVOnbsmJKTk3X48GEnhh3RaNIHwCnBOP/QXB0AAEQiLurB7cLqtr74+Hilp6dr7dq1yszM1Pr16+X1ev1u6ZOk119/3fffRUVFSktLU1FRUYhHGx2C0aQPAAJh9/mH5uoAACBSUVQAtwuryilJys3NVW5urgYOHKgHHnhAeXl5kqSsrCxt2LDB4dFFn2A8Fh4AAmHn+Yfm6gAAIJI1X9TzdPKPjzr65D8qzmGXsKqckqRBgwZpx44dLV5/9NFHW10/OTlZR48eDfawopqdj4UHgPaw6/xDc3UAABDJmi/qfbVKPCnu7J/8R8U57BR2ySm4U3OTPv7xBiDU7Dj/NPdhqGtsuay5DwPnNwAAEM548h/cjOQUACDq0YcBAABEAzsu6lFxjmAIu55TAADYLRh9GCR6MQAAgMgTjCf/ETOByikAQNSzuw+DRC8GAAAQmeyuOCdmgkRyKqoZY2hiDgD/YOfDHejFAAAAIlVzxXlznNPsbCrOiZnQjORUlCI7DQAt2fVwh2D0YuCCAgAAcAM7K86JmdCM5FQUIjsNAMFl99P/uKAAAADcxK6Kc2ImNKMhehQKJDsNADh7dvZiOPWCQn2jUW1do+obje+CAg1DAQCAE5orzueMSFRGctxZFTiEQ8xEs/bQoHIqCtmdnQYA+LOzFwOPawYAAJHK7TETlVihQ+VUFLL76QoAAH/NvRj6XRirzh5LsV086uyxlHxh+3sx8LhmAAAQqdwcMwWjEosY7PSonIpCdmanAQCts6sXA49rBgB70SwZcBe3xkx2V2IRg7WN5FQUsvPpCgCA07Pj6X88rhkA7MM/DgF3clvMJNnbDicYMVikJdpJTkUpu7LTAIDg4nHNAGAPEvRAZLO7CMPOSiy3V2G5IZ4jORXF7MhOAwCCj8c1A0DH8YAJIPLZWYRhZyWWm6uw3BLP0RA9zNBADQCiU7Q8rhkAgiUYD5gA4D52xEzN27GrWXuoq7AC5aZ4jsqpMOKWjCYAIDy5/XHNkjvKygFEJp5YDaC97KrEcmsVlpsqSqmcChNuymgCAMKTmx/XLJ28CDMxZ7vmrnxL9214X3NXvqWJOdtVVlXb7m0BiBx23TnQ/I9DTyf/cxdPrAbQFjsqsdxaheWmilIqp8KEmzKaAIDw5dbHNdOoGEBr7LxzgCdWA3CSG6uw3FRRSnIqTNjdxBYAEL3c+LhmLsIA+KpgJK15YjUAJ9kRg9mZaLc7nusIklNhwk0ZTQAA7K5A4CIMEFns6B8XrKQ1T6wGEO7sSrS7qaKU5FSYcFNGEwAAyd4KBC7CAJHDrlvxSFoDwOnZlWh3S0UpDdHDhJ0N1AAAsItdj2umUTHgPDsaj9v5EB+S1gAQGnbFcx1B5VQYcUtGEwAAu7mprBwIJ3bcPifZV+1k56143DkAANEj7JJT+/bt04IFC1RRUaELLrhAjz32mIYOHeq3zmuvvaYf/vCHqq6ulmVZmjp1qh544AF16hT6QjG7AoZm3CMPAIhUXISBneyMweyO59yWULKz8bidt+KRtAaA6BF2yanbbrtNixYtUmZmptatW6fMzEwVFBT4rdOzZ08988wzSklJ0YkTJ/SNb3xDa9asUWZmZkjHauejbwEAiAZchAl/gVxIDDY7YzC74zk3JpTsrHay+1Y8ktYAEB3CqufUoUOHVFhYqHnz5kmSrrvuOpWWlmr//v1+6w0fPlwpKSmSpHPOOUdpaWkqKioK6VjtvN8eAAAgXDRfSPzkk0907733hvzioJ0xmN3xnJ3bCyShFKjmaqfWNFc7BSoY/ePc0AsFABBcYZWcKi0tVUJCgmJiThZ8WZalpKQklZSUnPZvDh48qHXr1ulb3/pWqIYpyd6AAQAAIBwEeiExmOyMweyO59yaULKz2omH+AAAzkbY3dbXHseOHdO0adO0dOlSjRgxosXynJwc5eTk+H6vrq62bd88+hYAAESbti4kXnLJJSEZg50xmN3xnJ3bszOhZHfjcW7FAwC0V1hVTiUmJqq8vFwNDQ2STpZGl5SUKCkpqcW6x48f15QpUzRjxgxlZ2e3ur3s7GyVlZX5frp3727bWHn0LQAAQOtycnLk9Xp9P3ZeILQzBrM7ngtGQsmO2+eCUe3ErXgAgPYIq+RUfHy80tPTtXbtWknS+vXr5fV6W1yJq66u1pQpUzRlyhT95Cc/cWKoQbnfHgAAwM0CvZAYzAuEdsZgdsdzbk4oNVc7PXXraP18+lA9detobc4eq749zm3XdgAAOBthlZySpNzcXOXm5mrgwIF64IEHlJeXJ0nKysrShg0bJEkrVqzQzp07lZ+fr7S0NKWlpelXv/pVSMfJ/fYAACDaBHohMZjsjMHsjufcnlCi2gkA4BTL8Ng4H6/Xq7KyMlu3aYzhfntEhGtytkuSXs0e6/BIAKBjgvF9j3/6+OOPlZmZqcrKSp1//vnKy8vT5Zdf3ubfuD0GszueIz5EoIi/AESKM33XR3RDdDdovgJF83MAABANBg0apB07djg9DFtjMLvjOeJDAAD8hd1tfQAAAAAAAIgcJKcAAAAAAADgGJJTAAAAAAAAcAzJKQAAAAAAADiG5BQAAAAAAAAcQ3IKAAAAAAAAjiE5BQAAAAAAAMdYxhjj9CDcomvXrurdu3dQtl1dXa3u3bsHZdtuwjwjRzTMUWKekSQa5igxTzscPnxYf//734OybZwdYrCOi4Z5RsMcJeYZSaJhjhLzjCROxl8kp0LE6/WqrKzM6WEEHfOMHNEwR4l5RpJomKPEPIH2ipb3UjTMMxrmKDHPSBINc5SYZyRxco7c1gcAAAAAAADHkJwCAAAAAACAYzw/+9nPfub0IKLF17/+daeHEBLMM3JEwxwl5hlJomGOEvME2ita3kvRMM9omKPEPCNJNMxRYp6RxKk50nMKAAAAAAAAjuG2PgAAAAAAADiG5BQAAAAAAAAcQ3KqA+68804lJyfLsizt2bOn1XW2bdumc889V2lpab6fL7/80rd81apVGjBggPr3769bb71V9fX1oRp+wAKZZ15ent8ce/XqpVmzZkmSioqK5PF4/JZ/+umnoZzCGZ04cUIzZ87UwIEDlZqaqmuuuUb79+9vdd2NGzdq8ODBGjBggGbNmqVjx44FtMwNAp3ne++9pzFjxmjw4MG67LLLtHDhQr/3rWVZuvzyy33H8/XXXw/lNNoU6BzP9L6MlGP58ssv+83xoosuUnp6um+5m49ls0mTJmnYsGFKS0vT1Vdfrd27d7e6XlvnU7efawOZ42uvvaaRI0dqyJAhGjp0qJYuXaqmpiZJ4XGelQKbZyR8byL4iMH+iRjszMvcgBjsn4jBTnLzsWxGDHYSMdhJQT+WBmdt+/btprS01PTr18/s3r271XW2bt1qUlNTW1322WefmYSEBFNeXm6amprMtGnTzMMPPxzMIZ+VQOb5VUOHDjXr1q0zxhjz17/+1VxwwQXBHGKHffnll+bFF180TU1NxhhjHnroITN27NgW6x0/ftzEx8ebDz/80BhjzHe/+13zgx/84IzL3CLQeX7yySfm3XffNcYY09DQYK6//npz3333+ZZLMlVVVaEYcrsFOse23peRdCy/aurUqWb58uW+3918LJudOr78/HwzbNiwFuu0dT4Nh3NtIHPctWuX+fTTT40xJ4//lVdeafLy8owx4XGeNSaweUbC9yaCjxjs9IjBwvt7mxgsco7lVxGDufNcSwz2T05/b1I51QFjxoyR1+s9679ft26dpk+frj59+siyLH3nO9/R008/beMI7dHeeb799ts6dOiQpk+fHsRR2eucc87RtddeK8uyJEmjR49WUVFRi/U2bdqk4cOHa/DgwZKkxYsX+45ZW8vcItB5DhgwQMOGDZMkeTweZWRktLqeGwU6x7ZE0rE81eeff64tW7Zo/vz5IRihfXr06OH77y+++MI351O1dT4Nh3NtIHMcPny4UlJSJJ08/mlpaWHzuWwWyDzbEg7HEqFBDNY6YrDw/94mBoucY3kqYjD3nmuJwQITimMZY+vW0KpPP/1U6enp8ng8+rd/+zctXrxYklRSUqJ+/fr51ktOTlZJSYlTw7TNqlWrNH/+fHXu3Nn3Wk1NjTIyMtTY2KiZM2fqxz/+sTwej4OjbNuKq5muLAAABnJJREFUFSs0Y8aMFq+3dszKy8vV0NDQ5rKYGHd+1E43z1PV1NTo0Ucf1a9//Wu/1ydOnKiGhgZNnDhRv/zlL9WtW7dgDvWstTXH070vI/VYPvbYY7r22msVHx/v93o4HMubb75ZW7dulSS99NJLLZa3dT4Nl3PtmeZ4qoMHD2rdunXauHGj77VwOc8GMs9o+95E8ETbe4kYrOWycP7eJgaLnGNJDNZymZsQg/2Tk9+bVE4FWXp6usrKyrRr1y4999xz+sMf/qD/+Z//cXpYQVNTU6NnnnlGt9xyi++1hIQEHThwQAUFBdq8ebNef/11/fa3v3VwlG27//77tX///haBQKQJZJ51dXW64YYbNGnSJH3729/2vV5cXKx33nlHb775pg4fPqx77rknFENut7bmGG7vy7YEciyNMVq9erXfZ1MKn2O5Zs0alZaWatmyZbr33nudHk5QBDrHY8eOadq0aVq6dKlGjBghKbzez2eaZ7R9byJ4ou29RAwWPojBwut92RZisMhADHaS09+bJKeC7Pzzz9cFF1wgSfJ6vbrpppt8ze6SkpJUXFzsW7eoqEhJSUmOjNMuzz77rIYOHaohQ4b4XuvatavvKkFcXJwWLlzoyoZ/krR8+XLl5+dr06ZNio2NbbG8tWOWkJCgmJiYNpe5zZnmKUn19fW64YYblJCQoBUrVvgta36fduvWTYsXL3bl8TzTHNt6X0basZSk7du368SJE5o8ebLf6+FwLE+1YMECbd26VZWVlX6vt3U+Dbdz7enmKEnHjx/XlClTNGPGDGVnZ/teD6fzbLPTzTPavjcRPNH2XiIGi5zvbWKwyDmWEjFYOJ1ricGc/d4kORVk5eXlvk7+x48f18aNGzV8+HBJ0nXXXacNGzbo4MGDMsboD3/4g2688UYnh9thq1atanFV4NChQ75O/n//+9+Vn5/v+3/gJjk5OXr66af16quv+t2Te6opU6Zo165d+uijjyRJ//3f/+07Zm0tc5NA5tnQ0KAbb7xRcXFxeuSRR/zuSa6qqlJtba0kqampSX/84x9ddzwDmWNb78tIOpbNVq1apczMTL8S43A4lkePHtXnn3/u+/3555/XhRdeqLi4OL/12jqfuv1cG+gcq6urNWXKFE2ZMkU/+clP/JaFw3k20HlG2/cmgifa3kvEYJHxvU0MFjnHshkxmHvPtcRgLovBbG2vHmUWLVpk+vbtazwej4mPjzf9+/c3xhhzyy23mBdeeMEYc/LpDUOGDDHDhg0zQ4YMMffdd5/vyQ7GGPPII4+YlJQUk5KSYhYuXGjq6uocmUtbApmnMcZ89NFHpnv37ubYsWN+f79+/XozdOhQ3/+DO+64w5w4cSKkcziT0tJSI8mkpKSY1NRUk5qaakaOHGmMMeanP/2p+f3vf+9b94UXXjCDBg0y/fv3NzNmzDBHjx4NaJkbBDrPtWvXGklm2LBhvvUWL15sjDHmzTffNJdffrnveM6bN89UVlY6NqevCnSOZ3pfRsqxNMaYo0ePmtjYWN9TRpq5/VgaY0xRUZHJyMgwl112mRk2bJiZOHGi74lVXz0HtXU+dfO5NtA5Llu2zMTExPiOd2pqqlm2bJkxJjzOs4HOMxK+NxF8xGDEYMRg7vveJgYjBiMGc+d5NlxiMMsYY+xNdwEAAAAAAACB4bY+AAAAAAAAOIbkFAAAAAAAABxDcgoAAAAAAACOITkFAAAAAAAAx5CcAgAAAAAAgGNITgGIeJZltftn3LhxTg8bAAAgrBGDAQhUjNMDAIBgW7BgQYvXDh48qJdffvm0ywcPHhz0cQEAAEQyYjAAgbKMMcbpQQBAqG3btk3jx4+XJHEaBAAACA1iMACt4bY+AAAAAAAAOIbkFAAAAAAAABxDcgoAAAAAAACOITkFAAAAAAAAx5CcAgAAAAAAgGNITgEAAAAAAMAxJKcAAAAAAADgGJJTAAAAAAAAcAzJKQAAAAAAADiG5BQAAAAAAAAcQ3IKAAAAAAAAjiE5BQAAAAAAAMeQnAIAAAAAAIBjLGOMcXoQAAAAAAAAiE5UTgEAAAAAAMAxJKcAAAAAAADgGJJTAAAAAAAAcAzJKQAAAAAAADiG5BQAAAAAAAAcQ3IKAAAAAAAAjiE5BQAAAAAAAMeQnAIAAAAAAIBjSE4BAAAAAADAMSSnAAAAAAAA4BiSUwAAAAAAAHDM/wOdh7aQB0rZhAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 1440x800 with 4 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "CPU times: user 47min 27s, sys: 1min 5s, total: 48min 32s\n",
            "Wall time: 47min 55s\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HmlMWCl2eC13"
      },
      "source": [
        "Above you can see that the Magnetization colapses to 0 after the critical temperature $T_c=2.269$ and that the specific heat is maximal around $T_c$ as expected."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oEcPyQpDPtQL",
        "outputId": "ffa6af13-09b0-4588-d6c1-98d6ad201e45",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "t1 = time.time()\n",
        "(t1 - t0) / 60"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "52.53264275789261"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 45
        }
      ]
    }
  ]
}