Chapter 12 - Bayesian Inference.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 12. Bayesian Inference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 12.1 Bayesian Philosophy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Postulates of **frequentist (or classical)** inference:\n",
    "\n",
    "- Probabilty refers to limiting relative frequencies.  Probabilities are objective properties of the real world.\n",
    "- Parameters are fixed, usually unknown constants.  Because they are not fluctuating, no probability statements can be made about parameters.\n",
    "- Statistical procedures should be designed to have well defined long run frequency properties.  For example, a 95% confidence interval should trap the true value of the parameter with limiting frequency at least 95%."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Postulates of **Bayesian** inference:\n",
    "\n",
    "- Probability indicates degrees of belief, not limiting frequency.  As such, we can make probability statements about lots of things, not just data which are subject to random variation.  For example, I might say that 'the probability that Albert Einstein drank a cup of tea on August 1 1948 is 35%'.  This does not refer to limiting frequency.  It reflects my strength of belief that the proposition is true.\n",
    "- We can make probability statements about parameters, even though they are fixed constants.\n",
    "- We can make inferences about a parameter $\\theta$, by producing a probability distribution for $\\theta$.  Inferences, such as point estimates and interval estimates, may then be extracted from this distribution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 12.2 The Bayesian Method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bayesian inference is usually carried out in the following way:\n",
    "\n",
    "1. Choose a probability density $f(\\theta)$ -- called the **prior distribution** -- that expresses our degrees of belief about a parameter $\\theta$ before we see any data.\n",
    "\n",
    "2. We choose a statistical model $f(x | \\theta)$ that reflects our beliefs about $x$ given $\\theta$.\n",
    "\n",
    "3. After observing data $X_1, \\dots, X_n$, we update our beliefs and form the **posterior distribution** $f(\\theta | X_1, \\dots, X_n)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bayesian update:\n",
    "\n",
    "$$f(\\theta | x^n) = \\frac{f(x^n | \\theta) f(\\theta)}{\\int f(x^n | \\theta) f(\\theta) d\\theta} \n",
    "= \\frac{\\mathcal{L}_n(\\theta) f(\\theta)}{\\int \\mathcal{L}_n f(\\theta) d\\theta}\n",
    "\\propto \\mathcal{L}_n(\\theta) f(\\theta)$$\n",
    "\n",
    "We throw away the denominator, which is a constant that does not depend on $\\theta$, called a **normalizing constant**.  We can summarize this by writing:\n",
    "\n",
    "$$ \\text{\"posterior is proportional to likelihood times prior\"} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can obtain a **point estimate** using the posterior mean:\n",
    "\n",
    "$$ \\overline{\\theta} = \\int \\theta f(\\theta | x^n) d\\theta = \\frac{\\int \\theta \\mathcal{L}_n(\\theta) f(\\theta)}{\\int \\mathcal{L}_n(\\theta) f(\\theta) d\\theta}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also obtain a Bayesian interval estimate. Define $a$ and $b$ by $ \\int_{-\\infty}^a f(\\theta | x^n) d\\theta = \\int_b^\\infty f(\\theta | x^n) d\\theta = \\alpha / 2$.  Let $C = (a, b)$.  Then\n",
    "\n",
    "$$\\mathbb{P}(\\theta \\in C | x^n) = \\int_a^b f(\\theta | x^n) d\\theta = 1 - \\alpha$$\n",
    "\n",
    "so **$C$ is a $1 - \\alpha$ posterior interval**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When the prior and the posterior are in the same family, we say the prior is **conjugate**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 12.3 Functions of Parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The posterior CDF for $\\tau = g(\\theta)$ is\n",
    "\n",
    "$$ H(\\tau | x^n) = \\mathbb{P}(g(\\theta) \\leq \\tau) = \\int_A f(\\theta | x^n) d\\theta$$\n",
    "\n",
    "where $A = \\{ \\theta : g(\\theta) \\leq \\tau \\}$.  The posterior density is $h(\\tau | x^n) = H'(\\tau | x^n)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 12.4 Simulation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The posterior can often be approximated by simulation.  Suppose we draw $\\theta_1, \\dots, \\theta_B \\sim p(\\theta | x^n)$.  Then a histogram of $\\theta_1, \\dots, \\theta_n$ approximates the posterior density $p(\\theta | x^n)$.  An approximation to the posterior mean $\\overline{\\theta}_n = \\mathbb{E}(\\theta | x^n)$ is $B^{-1} \\sum_{j=1}^B \\theta_j$.  The posterior $1 - \\alpha$ interval can be approximated by $(\\theta_{\\alpha/2}, \\theta_{1 - \\alpha/2})$ where $\\theta_{\\alpha/2}$ is the $\\alpha/2$ sample quantile of $\\theta_1, \\dots, \\theta_B$.\n",
    "\n",
    "Once we have a sample $\\theta_1, \\dots, \\theta_B$ from $f(\\theta | x^n)$, let $\\tau_i = g(\\theta_i)$.  Then $\\tau_1, \\dots, \\tau_B$ is a sample from $f(\\tau | x^n)$.  This avoids the need to do analytical calculations.  Simulation is discussed in more detail later in the book."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 12.5 Large Sample Properties for Bayes' Procedures"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Theorem 12.5**. Under appropriate regularity conditions, we have that the posterior is approximately $N(\\hat{\\theta}, \\hat{\\text{se}}^2)$ where $\\hat{\\theta}_n$ is the MLE and $\\hat{\\text{se}} = 1 / \\sqrt{nI(\\hat{\\theta}_n)}$.  Hence, $\\overline{\\theta}_n \\approx \\hat{\\theta}_n$.  Also, if $C_n = (\\hat{\\theta}_n - z_{\\alpha/2} \\hat{\\text{se}}, \\hat{\\theta}_n + z_{\\alpha/2} \\hat{\\text{se}})$ is the asymptotic frequentist $1 - \\alpha$ confidence interval, then $C_n$ is also an approximate $1 - \\alpha$ Bayesian posterior interval:\n",
    "\n",
    "$$\\mathbb{P}(\\theta \\in C_n | X^n) \\rightarrow 1 - \\alpha$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is also a **Bayesian delta method**.  Let $\\tau = g(\\theta)$.  Then\n",
    "\n",
    "$$ \\tau | X^n \\approx N(\\hat{\\tau}, \\overline{\\text{se}}^2)$$\n",
    "\n",
    "where $\\hat{\\tau} = g(\\hat{\\theta})$ and $\\overline{\\text{se}} = \\text{se} | g'(\\hat{\\theta}) |$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 12.6 Flat Priors, Improper Priors and \"Noninformative\" Priors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Improper priors**.  We can adopt a flat prior $f(\\theta) \\propto c$ where $c > 0$ is a constant.  Note that $\\int f(\\theta) d\\theta = \\infty$ so this is not a real probability density function in a proper sense. We call such prior an improper prior.  In general, improper priors are not a problem as long as the resulting posterior is a well defined probability distribution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Flat priors are not invariant**.  The notion of a flat prior is not well-defined because a flat prior on a parameter does not imply a flat prior ron a transformed version of a parameter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Jeffreys' prior**.  Take $f(\\theta) \\propto I(\\theta)^{1/2}$ where $I(\\theta)$ is the Fisher information function.  This rule turns out to be transformation invariant.\n",
    "\n",
    "In a multiparameter problem, the Jeffreys' prior is defined to be $f(\\theta) \\propto \\sqrt{\\text{det} I(\\theta)}$, where $\\text{det}(A)$ denotes the determinant of a matrix $A$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 12.7 Multiparameter Problems"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Suppose that $\\theta = (\\theta_1, \\dots, \\theta_p)$.  The posterior density is still given by\n",
    "\n",
    "$$p(\\theta | x^n) \\propto \\mathcal{L}(\\theta) f(\\theta)$$\n",
    "\n",
    "Suppose we want to make inferences for $\\theta_1$.  The marginal posterior for $\\theta_1$ is\n",
    "\n",
    "$$ f(\\theta_1 | x^n) = \\int \\cdots \\int f(\\theta_1, \\dots, \\theta_p) d\\theta_2 \\dots d\\theta_p$$\n",
    "\n",
    "In practice, it might not be feasible to do this integral.  Simulation can help.  Draw randomly from the posterior:\n",
    "\n",
    "$$\\theta^1, \\dots, \\theta^B \\sim f(\\theta | x^n)$$\n",
    "\n",
    "where the superscripts index the different draws; each $\\theta^j$ is a vector $\\theta^j = (\\theta^j_1, \\dots, \\theta^j_p)$.  Now collect together the first component of each draw:\n",
    "\n",
    "$$\\theta^1_1, \\dots, \\theta^B_1$$\n",
    "\n",
    "These form a sample from $f(\\theta_1 | x^n)$ and we have avoided doing any integrals."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 12.8 Strenghts and Weaknesses of Bayesian Inference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Frequentist and Bayesian methods are answering different questions:\n",
    "\n",
    "- To combine prior beliefs with data in a principled way, use bayesian inference.\n",
    "- To construct procedures with guaranteed long run performance, such as confidence intervals, use frequentist methods."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 12.9 Appendix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Proof of Theorem 12.5**.\n",
    "\n",
    "It can be shown that the effect of the prior diminishes as $n$ increases so that $f(\\theta | X^n) \\propto \\mathcal{L}_n(\\theta)f(\\theta) \\approx \\mathcal{L}_n(\\theta)$.  Hence, $\\log f(\\theta | X^n) \\approx \\ell(\\theta)$.  Now, $\\ell(\\theta) \\approx \\ell(\\hat{\\theta}) + (\\theta - \\hat{\\theta})\\ell'(\\hat{\\theta}) + [(\\theta - \\hat{\\theta})^2/2]\\ell''(\\hat{\\theta}) = \\ell(\\hat{\\theta}) + [(\\theta - \\hat{\\theta})^2/2] \\ell''(\\hat{\\theta})$ since $\\ell'(\\hat{\\theta}) = 0$.  Exponentiating, we get approximately that\n",
    "\n",
    "$$ f(\\theta | X^n) \\propto \\exp \\left\\{ - \\frac{1}{2} \\frac{(\\theta - \\hat{\\theta})^2}{\\sigma_n^2} \\right\\} $$\n",
    "\n",
    "where $\\sigma_n^2 = 1 / \\ell''(\\hat{\\theta}_n)$.  So the posterior of $\\theta$ is approximately Normal with mean $\\hat{\\theta}$ and variance $\\sigma^2_n$.  Let $\\ell_i = \\log f(X_i | \\theta)$, then\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "\\sigma_n^{-2} &= -\\ell''(\\hat{\\theta}_n) = \\sum_i -\\ell''_i(\\hat{\\theta}_n) \\\\\n",
    "&= n \\left( \\frac{1}{n} \\right) \\sum_i -\\ell''_i(\\hat{\\theta}_n) \\approx n \\mathbb{E}\\left[-\\ell''_i(\\hat{\\theta}_n)\\right] \\\\\n",
    "&= n I(\\hat{\\theta}_n)\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "and hence $\\sigma_n \\approx \\text{se}(\\hat{\\theta})$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 12.11 Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise 12.11.1**.  Verify (12.5).\n",
    "\n",
    "*Let $X_1, \\dots, X_n \\sim N(\\theta, \\sigma^2)$.  For simplicity, let us assume that $\\sigma$ is known.  Suppose that we take as a prior $\\theta \\sim N(a, b^2)$.  In problem 1 of the homework, it is shown that the posterior for $\\theta$ is* \n",
    "\n",
    "$$\\theta | X^n \\sim N(\\overline{\\theta}, \\tau^2)$$\n",
    "\n",
    "*where*\n",
    "\n",
    "$$\\overline{\\theta} = w \\overline{X} + (1 - w) a$$\n",
    "\n",
    "*where*\n",
    "\n",
    "$$w = \\frac{\\frac{1}{\\text{se}^2}}{\\frac{1}{\\text{se}^2} + \\frac{1}{\\text{b}^2}} \\quad \\text{and} \\quad \\frac{1}{\\tau^2} = \\frac{1}{\\text{se}^2} + \\frac{1}{b^2}$$\n",
    "\n",
    "*and $\\text{se} = \\sigma / \\sqrt{n}$ is the standard error of the MLE $\\overline{X}$.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Solution**.\n",
    "\n",
    "The posterior is proportional to the likelihood times the prior:\n",
    "\n",
    "$$ p(\\theta | x^n) \\propto \\mathcal{L}_n(\\theta) f(\\theta)$$\n",
    "\n",
    "The likelihood is the product of $n$ Normal PDFs evaluated at different data points:\n",
    "\n",
    "$$ \n",
    "\\mathcal{L}_n(\\theta) = \\prod_{i=1}^n f_X(X_i, \\theta)\n",
    "$$\n",
    "\n",
    "Looking at the case $n = 2$:\n",
    "\n",
    "$$\n",
    "\\begin{align} \n",
    "f(X_1, \\theta) f(X_2, \\theta) &= \n",
    "\\frac{1}{\\sigma \\sqrt{2 \\pi}} \\exp \\left\\{-\\frac{1}{2} \\left(\\frac{X_1 - \\mu}{\\sigma} \\right)^2 \\right\\}\n",
    "\\frac{1}{\\sigma \\sqrt{2 \\pi}} \\exp \\left\\{-\\frac{1}{2} \\left(\\frac{X_2 - \\mu}{\\sigma} \\right)^2 \\right\\} \\\\\n",
    "&= \\left( \\frac{1}{\\sigma \\sqrt{2 \\pi}} \\right)^2 \\exp \\left\\{-\\frac{1}{2\\sigma^2} \\left((X_1 - \\mu)^2 + (X_2 - \\mu)^2 \\right) \\right\\} \\\\\n",
    "&= \\left( \\frac{1}{\\sigma \\sqrt{2 \\pi}} \\right)^2 \\exp \\left\\{-\\frac{1}{2\\sigma^2} \\left(\n",
    "X_1^2 + 2 X_1 X_2 + X_2^2 - 2 (X_1 + X_2)\\mu + 2\\mu^2 - 2 X_1 X_2\n",
    "\\right) \\right\\} \\\\\n",
    "&= \\left( \\frac{1}{\\sigma \\sqrt{2 \\pi}} \\right)^2 \\exp \\left\\{-\\frac{1}{\\sigma^2}\\left(\n",
    "\\left(\\mu - \\frac{X_1 + X_2}{2}\\right)^2 + \\left(\\frac{X_1 + X_2}{2}\\right)^2 - X_1 X_2\n",
    "\\right) \\right\\} \\\\\n",
    "&= \\left( \\frac{1}{\\sigma \\sqrt{2 \\pi}} \\right)^2 \\exp \\left\\{-\\frac{1}{2} \\left(\n",
    "\\frac{\\left(\\mu - \\frac{X_1 + X_2}{2}\\right)^2 + \\left(\\frac{X_1 + X_2}{2}\\right)^2 - X_1 X_2}{\\sigma^2 / 2}\n",
    "\\right) \\right\\} \\\\\n",
    "&= \\left( \\frac{1}{\\sigma \\sqrt{2 \\pi}} \\right)^2 \\exp \\left\\{-\\frac{1}{2} \\left(\n",
    "\\frac{\\left(\\mu - \\frac{X_1 + X_2}{2}\\right)^2}{\\sigma^2 / 2} + C\n",
    "\\right) \\right\\} \\\\\n",
    "&\\propto \\exp \\left\\{-\\frac{1}{2}\n",
    "\\left(\\frac{\\mu - \\frac{X_1 + X_2}{2}}{\\sigma / \\sqrt{2}}\\right)^2\n",
    "\\right\\}\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "The exponent is a quadratic form in $\\mu$, and that makes this expression is proportional to the PDF of a Normal with mean $\\mu$ and standard error $\\sigma / \\sqrt{2}$, observed at the mean, $X = (X_1 + X_2)/2$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This strongly suggests that, for the general case, the product is proportional to the PDF of a Normal with mean $\\mu$ and standard error $\\sigma / \\sqrt{n}$, observed at the mean, $X = n^{-1} \\sum_{i=1}^n X_i$.  We can prove it by induction, going through similar steps:\n",
    "\n",
    "$$\n",
    "\\begin{align} \n",
    "f((X_1, \\dots, X_n), \\theta_n) f(X_{n+1}, \\theta) &= \n",
    "\\frac{1}{\\sigma_n \\sqrt{2 \\pi}} \\exp \\left\\{-\\frac{1}{2} \\left(\\frac{\\overline{X}_n - \\mu}{\\sigma_n} \\right)^2 \\right\\}\n",
    "\\frac{1}{\\sigma \\sqrt{2 \\pi}} \\exp \\left\\{-\\frac{1}{2} \\left(\\frac{X_{n+1} - \\mu}{\\sigma} \\right)^2 \\right\\} \\\\\n",
    "&\\propto \\exp \\left\\{-\\frac{1}{2} \\left( \\left(\\frac{\\overline{X}_n - \\mu}{\\sigma_n} \\right)^2 + \\left(\\frac{X_{n+1} - \\mu}{\\sigma} \\right)^2 \\right) \\right\\} \\\\\n",
    "&= \\exp \\left\\{-\\frac{1}{2} \\left( n \\left(\\frac{\\overline{X}_n - \\mu}{\\sigma} \\right)^2 + \\left(\\frac{X_{n+1} - \\mu}{\\sigma} \\right)^2 \\right) \\right\\} \\\\\n",
    "&= \\exp \\left\\{-\\frac{1}{2 \\sigma^2} \\left( n \\left(\\overline{X}_n - \\mu\\right)^2 + \\left(X_{n+1} - \\mu \\right)^2 \\right) \\right\\} \\\\\n",
    "&\\propto \\exp \\left\\{-\\frac{1}{2} \\left( \\frac{\\mu - (n\\overline{X}_n + X_{n+1})}{\\sigma / \\sqrt{n+1}} \\right)^2 \\right\\}\n",
    "= \\exp \\left\\{-\\frac{1}{2}\n",
    "\\left(\\frac{\\mu - \\overline{X}_{n+1}}{\\sigma / \\sqrt{n+1}}\\right)^2\n",
    "\\right\\}\n",
    "\\end{align}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we just need to prove one final result -- that the product of the likelihood and the prior, both of which are PDFs of normals, is proportional to yet another normal, with the parameters given.   The steps are again similar -- except now we are completing the squares in $\\theta$, rather than $\\mu$, as each PDF has distinct mean ($\\overline{X}$ and $a$) and standard error ($\\text{se}$ and $b$).\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "\\mathcal{L}_n(\\theta) f(\\theta) \n",
    "&= \\frac{1}{\\text{se} \\sqrt{2 \\pi}} \\exp \\left\\{-\\frac{1}{2} \\left(\\frac{\\overline{X} - \\theta}{\\text{se}}\\right)^2 \\right\\} \\frac{1}{b \\sqrt{2 \\pi}} \\exp \\left\\{-\\frac{1}{2} \\left( \\frac{a - \\theta}{b} \\right)^2 \\right\\} \\\\\n",
    "& \\propto \\exp \\left\\{-\\frac{1}{2} \\left( \\left(\\frac{\\overline{X} - \\theta}{\\text{se}}\\right)^2 + \\left( \\frac{a - \\theta}{b} \\right)^2 \\right) \\right\\} \\\\\n",
    "& \\propto \\exp \\left\\{-\\frac{1}{2} \\left( \\frac{\\overline{\\theta} - \\theta}{\\tau} \\right)^2 \\right\\}\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "which is the desired result.\n",
    "\n",
    "An **excruciatingly complete** derivation of the results used here -- products of Normal PDFs are proportional to Normal PDFs -- and more general versions of it can be seen in Bromiley, Paul. \"Products and convolutions of Gaussian probability density functions.\" Tina-Vision Memo 3.4 (2003): 1, which is available online."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise 12.11.2**.  Let $X_1, \\dots, X_n \\sim N(\\mu, 1)$.\n",
    "\n",
    "**(a)** Simulate a dataset (using $\\mu = 5$) consisting of $n = 100$ observations.\n",
    "\n",
    "**(b)** Take $f(\\mu) = 1$ and find the posterior density.  Plot the density.\n",
    "\n",
    "**(c)** Simulate 1000 draws from the posterior.  Plot a histogram of the simulated values and compare the histogram to the answer in (b).\n",
    "\n",
    "**(d)** Let $\\theta = e^\\mu$.  Find the posterior density for $\\theta$ analytically and by simulation.\n",
    "\n",
    "**(e)** Find a 95% posterior interval for $\\theta$.\n",
    "\n",
    "**(f)** Find a 95% confidence interval for $\\theta$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Solution**.\n",
    "\n",
    "**(a)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from scipy.stats import norm\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "n = 100\n",
    "mu = 5\n",
    "sigma = 1\n",
    "X = norm.rvs(loc=mu, scale=sigma, size=n)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(b)**  The posterior is proportional to likelihood times the prior:\n",
    "\n",
    "$$f(\\mu | X^n) \\propto \\mathcal{L}_n(\\mu) f(\\mu) = \\mathcal{L}_n(\\mu)$$\n",
    "\n",
    "But the likelihood is the product of the PDFs for each data point:\n",
    "\n",
    "$$\\mathcal{L}_n(\\mu) = \\prod_{i=1}^n f(X_i; \\mu)$$\n",
    "\n",
    "We can plot this with \"brute force\" -- calculating this product for each sample point -- or use the analytic result that the likelihood is proportional to a random variable following a Normal distribution in $\\mu$ with mean $\\hat{\\mu} = \\overline{X}$ and standard error $\\sigma / \\sqrt{n}$:\n",
    "\n",
    "$$\\mathcal{L}_n(\\mu) \\propto N(\\overline{X}, \\sigma^2 / n)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# posterior is proportional to likelihood times f(\\mu)\n",
    "# likelihood is \\prod_i f(X_i; \\mu)\n",
    "# pdf is f(x; (\\mu, \\sigma^2)), the density of the normal function\n",
    "\n",
    "mu_hat = X.mean()\n",
    "mu_values = np.linspace(4, 6, 100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dfXRU933n8fdXo2cJkITEg5FAYGTH2PEDkaljJ2maJik4bWhP2i3udt34tKVu7Gyz29Os0z192odz2iZnN/HWa+Ik7tZtGm82ibc0S4L7lCZuYgf8hA0YI4sHCQkQSCD0hJ6++8fMkPFYQlcwM/fe0ed1jo5n7v3dme+MLx9d/e7v/q65OyIiUrxKwi5ARETyS0EvIlLkFPQiIkVOQS8iUuQU9CIiRa407AJm0tjY6K2trWGXISISG88///wZd2+aaV0kg761tZW9e/eGXYaISGyY2bHZ1qnrRkSkyCnoRUSKnIJeRKTIKehFRIqcgl5EpMgp6EVEilygoDezzWZ2yMw6zOyhGdabmT2cWr/PzDZmrPt3ZrbfzF41s6+YWWUuP4CIiFzenEFvZgngEWALsAG4x8w2ZDXbArSlfrYDj6a2XQX8W6Dd3W8CEsC2nFUvcpUO9AzyrVd6wy5DJK+CHNFvAjrcvdPdx4Enga1ZbbYCT3jSs0Cdma1MrSsFqsysFKgGenJUu8gVm5p2HvmnDj78Z8/wm19+gX3d58IuSSRvggT9KqAr43l3atmcbdz9BPAZ4DjQC5x396dnehMz225me81sb19fX9D6Reat59wov/j5H/Dp3Yf4qRtX0FBTzqd3Hwq7LJG8CRL0NsOy7NtSzdjGzOpJHu2vBa4Baszsl2d6E3d/zN3b3b29qWnG6RpEcuI//e0BDvQO8t9/8Rb+7Jdu42PvvZbvHT7D9zvOhF2aSF4ECfpuoCXjeTNv7X6Zrc37gSPu3ufuE8A3gDuvvFyRqzM2McU/v97HRzY283O3NWNm/PIda1i5pJI/2X0I3VpTilGQoN8DtJnZWjMrJ3kydWdWm53AvanRN3eQ7KLpJdllc4eZVZuZAT8JHMxh/SLz8szhM4xOTPHBG5dfWlZZluAT72/j5a5zPH3gVIjVieTHnEHv7pPAg8BukiH9VXffb2b3m9n9qWa7gE6gA/gC8LHUts8BXwNeAF5Jvd9juf4QIkE9feAkiypL+bG1S9+0/CMbm1nXVMNndh9ielpH9VJcAk1T7O67SIZ55rIdGY8deGCWbf8A+IOrqFEkJ6amnb8/eJr3vW0Z5aVvPsYpTZTw6+9ex6e+8QqdZ4ZYv2xRSFWK5J6ujJUF4/ljA/QPj/PBDStmXH/HuuRR/p6jA4UsSyTvFPSyYDy9/yTliRJ+/PqZR3W1Lq2msbacPUf7C1yZSH4p6GVBcHeePnCKO9cvpbZi5h5LM6N9TQN7dUQvRUZBLwvC66eGON4/Mmu3TVp7az3H+0c4NThWoMpE8k9BLwvC0/tPYgbv37Dssu1ub20AUPeNFBUFvSwI//LGGW66ZgnLFl1+8tQbr1lMdXlC3TdSVBT0UvTcnQM9g7y9ecmcbUsTJdy2uk5H9FJUFPRS9E6cG2VwbJINKxcHat++poGDvYNcGJvIc2UihaGgl6J3oGcQgA3XBAv621sbmHZ48bimLpbioKCXonegdxAzeNuKYFe73rq6jkSJsVfdN1IkFPRS9A70DLK2sYbq8kAzflBbUcqGlYt1hawUDQW9FL0DvYOB++fT2lvrebFrgImp6TxVJVI4CnopaudHJ+geGA3cP5922+p6xiam6Tg9lKfKRApHQS9F7WBv6kTsPI/o25bVAijopSgo6KWozXfETdraxhpKDA4r6KUIKOilqB3oHaSxtmLOK2KzVZYlWLO0hjcU9FIEAgW9mW02s0Nm1mFmD82w3szs4dT6fWa2MbX8ejN7KeNn0Mw+kesPITKbAz2D8z6aT7u2qZbDpy/kuCKRwpsz6M0sATwCbAE2APeY2YasZluAttTPduBRAHc/5O63uvutwDuAEeCp3JUvMrvxyWkOn74w7/75tLbltRw5M6yRNxJ7QY7oNwEd7t7p7uPAk8DWrDZbgSc86VmgzsxWZrX5SeANdz921VWLBNBxeoiJKb/iI/q2ZbVMTDnHzo7kuDKRwgoS9KuArozn3all822zDfjKbG9iZtvNbK+Z7e3r6wtQlsjlHbjCETdpban7xmrkjcRdkKC3GZb5fNqYWTnwYeD/zPYm7v6Yu7e7e3tT08y3ehOZj4O9g1SWlbC2seaKtr92WXK7DvXTS8wFCfpuoCXjeTPQM882W4AX3P3UlRQpciUO9Axy/YrFJEpmOg6ZW3V5KavqqjTEUmIvSNDvAdrMbG3qyHwbsDOrzU7g3tTomzuA8+7em7H+Hi7TbSOSD51nhljfVHtVr9G2vJbDpxT0Em9zzvLk7pNm9iCwG0gAj7v7fjO7P7V+B7ALuBvoIDmy5r709mZWDXwA+I3cly8ys5HxSU4NXmRtY/VVvc76plp+8MZZpqb9iv8yEAlboOn83H0XyTDPXLYj47EDD8yy7Qiw9CpqFJm3o2eSI2Var7B/Pq1teS0XJ6c5MTDK6qVX90tDJCy6MlaK0tGzwwC0Lr26oF+fGnmjC6ckzhT0UpSOnEkF/VUe0a9PTW6mE7ISZwp6KUpHzwyzbFEFtRXBbjYymyVVZSxbVKGx9BJrCnopSkfODF/10Xxa2/JaHdFLrCnopSgdPTvM2qvsn09b31RLx6kLJMcciMSPgl6KzoWxCc4MjefsiH798kUMj0/Re34sJ68nUmgKeik66aGVVzr1QbZrU6+TPsErEjcKeik6R1JDK3MV9Onx88f7NYulxJOCXorOkb5k0K/J0QVOK5dUUZYwTVcssaWgl6Jz9Oww1yyppLIskZPXS5QYLfXVHO9X143Ek4Jeis6RM8OsbcpNt01aS0O1juglthT0UnSOnh2+6qkPsq1ZWs3xsyMaYimxpKCXojIwPM65kYmcnYhNW91QzYWLkwyMTOT0dUUKQUEvReVIjiYzy7Ym9XoaeSNxpKCXonI0R5OZZUuP4Dl2VidkJX4CBb2ZbTazQ2bWYWYPzbDezOzh1Pp9ZrYxY12dmX3NzF4zs4Nm9s5cfgCRTEfPDFNiya6WXEq/3nGdkJUYmjPozSwBPELyvq8bgHvMbENWsy1AW+pnO/BoxrrPAd9297cBtwAHc1C3yIyOnB2hub6a8tLc/rFaWZZg+eIKjqnrRmIoyL+GTUCHu3e6+zjwJLA1q81W4AlPehaoM7OVZrYYeA/wJQB3H3f3czmsX+RNjuZw1spsqxuqdUQvsRQk6FcBXRnPu1PLgrRZB/QBf25mL5rZF80sP/8KRUj2oa/JcbdN2uqGGp2MlVgKEvQz3RE5ezDxbG1KgY3Ao+5+GzAMvKWPH8DMtpvZXjPb29fXF6AskTc7PzrB4NgkLQ1VeXn9NUurOTk4xtjEVF5eXyRfggR9N9CS8bwZ6AnYphvodvfnUsu/RjL438LdH3P3dndvb2pqClK7yJt0pY62W+rzc0SfHnnTpaN6iZkgQb8HaDOztWZWDmwDdma12Qncmxp9cwdw3t173f0k0GVm16fa/SRwIFfFi2TqHkgFfd66btJDLBX0Ei9z3lDT3SfN7EFgN5AAHnf3/WZ2f2r9DmAXcDfQAYwA92W8xMeBL6d+SXRmrRPJmXT/eb6CPn3RlEbeSNwEunOyu+8iGeaZy3ZkPHbggVm2fQlov4oaRQLp6h9lcWUpS6rK8vL69dVlLKoo5bgumpKY0ZWxUjS6BkbydjQPYGa0NFRr5I3EjoJeikZX/0jOr4jNtmZptbpuJHYU9FIUpqedroHRvB7RQ/K2gt39o0xNa7piiQ8FvRSFvqGLjE9O01KfnzH0aWsaahifmubk4Fhe30cklxT0UhTSY9ubC9B1A5rFUuJFQS9FoSs1hj7fffTpi7G6B0bz+j4iuaSgl6Jw/GwyeFfV5bfrZmVdJSUG3TohKzGioJei0DUwwvLFFVSWJfL6PmWJElYuqaJLR/QSIwp6KQpd/SN5m+MmW3N91aXpFkTiQEEvRaEQY+jTWhqq6erXEb3Eh4JeYm98cprewbG8j7hJa66v4tSFMS5OarpiiQcFvcRez7lR3Mn7GPq0lvpq3KHnnMbSSzwo6CX2uvI8PXG29PtoXnqJCwW9xF56krFC9dE3p/5y6NIJWYkJBb3EXlf/KGUJY/niyoK83/LFlZQlTBdNSWwo6CX2ugZGWFVXRaJkplsX516ixLimrkpdNxIbgYLezDab2SEz6zCzt9zcO3ULwYdT6/eZ2caMdUfN7BUze8nM9uayeBFIXqVaqP75tJb6al00JbExZ9CbWQJ4BNgCbADuMbMNWc22AG2pn+3Ao1nrf8Ldb3V33WlKcu54/wjNBbpYKq2loYoT6qOXmAhyRL8J6HD3TncfB54Etma12Qo84UnPAnVmtjLHtYq8xdDFSQZGJmhpKMzQyrTm+mrODI0zMj5Z0PcVuRJBgn4V0JXxvDu1LGgbB542s+fNbPtsb2Jm281sr5nt7evrC1CWCJemIijU9Adp6ZE3OiErcRAk6Gc6w5V9e53LtbnL3TeS7N55wMzeM9ObuPtj7t7u7u1NTU0ByhLh0lQEBe+jb0hPV6zuG4m+IEHfDbRkPG8GeoK2cff0f08DT5HsChLJiXTQNhfoqti0S2PpNeeNxECQoN8DtJnZWjMrB7YBO7Pa7ATuTY2+uQM47+69ZlZjZosAzKwG+CDwag7rlwWuq3+UqrIES2vKC/q+TbUVVJaVaIilxELpXA3cfdLMHgR2AwngcXffb2b3p9bvAHYBdwMdwAhwX2rz5cBTZpZ+r79292/n/FPIgtU1MEJLQxWpfaxgzIzm+mr10UsszBn0AO6+i2SYZy7bkfHYgQdm2K4TuOUqaxSZVSHnoc/WXF+laRAkFnRlrMSWu3NiYLTg/fNpLfXV6rqRWFDQS2ydH53gwsXJgo+4SWtpqGJwbJLzoxOhvL9IUAp6ia30iJdCXxWbln5fDbGUqFPQS2yFNbQyLX1uQEMsJeoU9BJbhb7hSLb0tAs6opeoU9BLbHX1j7K4spQlVWWhvP+SqjIWVZTqhKxEnoJeYqt7oPCzVmYyM1oaqi/d4UokqhT0EltdA6MFn7UyW0tDleall8hT0EssuTvdA+FdLJXWUl9N98AIyWsGRaJJQS+x1Dd0kbGJ6dBG3KStXlrN2MQ0fUMXQ61D5HIU9BJL6Tlmwhpxk/ajIZbqp5foUtBLLKWDNfSgb9B0xRJ9CnqJpfQRfdhdN806opcYUNBLLHUPjLC0ppzq8kATsOZNZVmCZYsqNMRSIk1BL7HU1T9Kc8jdNmktDdWarlgiTUEvsdQ1MEJLyN02aS31Veqjl0gLFPRmttnMDplZh5k9NMN6M7OHU+v3mdnGrPUJM3vRzL6Zq8Jl4ZqadnrOjYZ6VWym1Q3V9J4fZWJqOuxSRGY0Z9CbWQJ4BNgCbADuMbMNWc22AG2pn+3Ao1nrfws4eNXVikAqVJ01S6MR9M0N1Uw79JzTUb1EU5Aj+k1Ah7t3uvs48CSwNavNVuAJT3oWqDOzlQBm1gx8CPhiDuuWBez42WR/+Oqo9NFrumKJuCBBvwroynjenVoWtM1ngU8Cl/271sy2m9leM9vb19cXoCxZqI71RyvoV6f+stAJWYmqIEFvMyzLnthjxjZm9tPAaXd/fq43cffH3L3d3dubmpoClCUL1fH+EcoSxjV10TgZu2JxJWUJ0xBLiawgQd8NtGQ8bwZ6Ara5C/iwmR0l2eXzPjP7qyuuVoRk101zfTWJkpmOLwovUZL8paOLpiSqggT9HqDNzNaaWTmwDdiZ1WYncG9q9M0dwHl373X3T7l7s7u3prb7R3f/5Vx+AFl4jvUPhz71QbbVDdWarlgia86gd/dJ4EFgN8mRM1919/1mdr+Z3Z9qtgvoBDqALwAfy1O9Ihw/O8KaiAV9c321juglsgJdP+7uu0iGeeayHRmPHXhgjtf4DvCdeVcokuHcyDiDY5ORGVqZ1tJQRf/wOMMXJ6mpCHdaBpFsujJWYuVYxIZWpqXr0cgbiSIFvcTKpaGVUTuiT42lT4/xF4kSBb3ESlfExtCnpevREEuJIgW9xMqxs8M0LaoIfXribHXVZSyuLL3UtSQSJQp6iZVjZ0cidzQPYGasbazh6NnhsEsReQsFvcRKV3/0hlamrVmqoJdoUtBLbFycnKJ3cCxyJ2LTWhtrODEwyvikpiuWaFHQS2x09Y/iTuTG0Ke1Lk1OV6wTshI1CnqJjeP9yW6RKPbRQ/KIHpInjEWiREEvsfGjeehrQq5kZq1Lk3UdOaOgl2hR0EtsHOsfobo8QWNtedilzKheQywlohT0EhvHU0MrzaIxPXE2DbGUqFLQS2wc74/mGPpMa5bWqOtGIkdBL7EwPe0c7x+J7IibtNbGGnrOjXJxcirsUkQuUdBLLJwcHOPi5DRrlkbzRGxaeoilbhQuURIo6M1ss5kdMrMOM3tohvVmZg+n1u8zs42p5ZVm9kMze9nM9pvZH+X6A8jC0NmX7A5Z1xTxoNcQS4mgOYPezBLAI8AWYANwj5ltyGq2BWhL/WwHHk0tvwi8z91vAW4FNqduNSgyL2/0DQGwvqk25EouT0MsJYqCHNFvAjrcvdPdx0ne5HtrVputwBOe9CxQZ2YrU8+HUm3KUj+eq+Jl4ejsG6K2opSmRRVhl3JZ6SGWGnkjURIk6FcBXRnPu1PLArUxs4SZvQScBv7O3Z+b6U3MbLuZ7TWzvX19fUHrlwXijb5hrm2qiezQyjQzo7WxRmPpJVKCBP1M/7Kyj8pnbePuU+5+K9AMbDKzm2Z6E3d/zN3b3b29qakpQFmykHT2DbEu4t02aa0aYikREyTou4GWjOfNQM9827j7OZI3B9887yplQRsZn6Tn/BjrGqN9IjatdWm1hlhKpAQJ+j1Am5mtNbNyYBuwM6vNTuDe1OibO4Dz7t5rZk1mVgdgZlXA+4HXcli/LADpETfXLovJEX1jjYZYSqTMeT82d580sweB3UACeNzd95vZ/an1O4BdwN1ABzAC3JfafCXwF6mROyXAV939m7n/GFLM0iNuoj60Mi1ziOX6mPxykuIW6Mab7r6LZJhnLtuR8diBB2bYbh9w21XWKAtcZ98wZj8auhh1GmIpUaMrYyXy3ugborm+isqyRNilBFJfXUZ9ddmlv0REwqagl8jr7Bvm2piMuIHkEMu25Yt4/ZSCXqJBQS+RNj3tdJ4ZYl1jfIIeoG1ZLa+fukCyV1MkXAp6ibTewTHGJqa5dlk8+ufTrlu+iAtjk5wavBh2KSIKeom2zvSIm7gd0S9P1nv49IWQKxFR0EvEvXE6GfRxPKIH1E8vkaCgl0jrPDPMoopSmmqjPZlZtsbaChpqyjl8Skf0Ej4FvUTaG31DrFtWG/nJzGayflkth0/riF7Cp6CXSOvsG+bamMxxk+265Rp5I9GgoJfIGr44Se/5sdjMcZNNI28kKhT0Elnpbo84XSyVqW1Z+oSs+uklXAp6iawDPYMA3HjN4pAruTI/GmKpfnoJl4JeIutg7yCLKkpprq8Ku5QropE3EhUKeomsA72D3LBycSxH3KSlp0IQCZOCXiJpetp5rXeQG1YuCruUq3Ld8kUcPj2kkTcSKgW9RNLx/hGGx6fYENP++bTrltdq5I2ELlDQm9lmMztkZh1m9tAM683MHk6t32dmG1PLW8zsn8zsoJntN7PfyvUHkOJ0sDd5InbDyiUhV3J11mvkjUTAnEGfug3gI8AWYANwj5ltyGq2BWhL/WwHHk0tnwR+291vAO4AHphhW5G3ONA7SKLELo1ciavrUvUr6CVMQY7oNwEd7t7p7uPAk8DWrDZbgSc86VmgzsxWunuvu78A4O4XgIPAqhzWL0XqQM8g1zbVxOauUrNZWltBY20FB3sV9BKeIEG/CujKeN7NW8N6zjZm1kry/rHPzfQmZrbdzPaa2d6+vr4AZUkxO5gacVMMbm5ewisnzoVdhixgQYJ+prFt2UMILtvGzGqBrwOfcPfBmd7E3R9z93Z3b29qagpQlhSrcyPj9JwfY0ORBP3bVy2h4/QQwxcnwy5FFqggQd8NtGQ8bwZ6grYxszKSIf9ld//GlZcqC8WB9InYmI+4SbulZQnTDq+eOB92KbJABQn6PUCbma01s3JgG7Azq81O4N7U6Js7gPPu3mvJK12+BBx09/+W08qlaKWnPiierps6APZ1K+glHKVzNXD3STN7ENgNJIDH3X2/md2fWr8D2AXcDXQAI8B9qc3vAv4N8IqZvZRa9rvuviu3H0OKycHeCyxblDyJWQwaaytYVVfFy93qp5dwzBn0AKlg3pW1bEfGYwcemGG7Z5i5/15kVgeK6ERsWvKErI7oJRy6MlYiZXxymo7TF4qmfz7t5uY6jp0d4dzIeNilyAKkoJdIOXz6AhNTXpRH9KB+egmHgl4i5fljAwDc1lIXciW5ddOqdNCrn14KT0EvkfLDI/2sXFIZ2znoZ7Okqox1jTW8rCN6CYGCXiLD3dlztJ/bWxtiPQf9bG5uXqIjegmFgl4io3tglFODF7m9tT7sUvLi5uY6Tg1e5NTgWNilyAKjoJfI+OGRfgBuX9sQciX5cUtLsp/+5S4d1UthKeglMvYc7WdxZSnXLYv3XaVms2HlEhIlppE3UnAKeomMPUf7aW9toKSk+PrnAarKE9ywchE/PNofdimywCjoJRLODl3kjb5h2ou0fz7trvWNvHh8QDNZSkEp6CUS9hxNjp/f1Fqc/fNp717fxMSU89yRs2GXIguIgl4iYe/RfspLS3h7c7zvETuX9tZ6KkpL+N7hM2GXIguIgl4iYc/Rfm5trqOiNN63DpxLZVmCTWsbeEZBLwWkoJfQDV+c5NWeQW5fW9z982nvWt/I4dNDnDyv8fRSGAp6Cd3zxwaYmnbai7x/Pu3dbclbZT7ToaN6KYxAQW9mm83skJl1mNlDM6w3M3s4tX6fmW3MWPe4mZ02s1dzWbgUj787cIrKshLuWLs07FIK4m0rFtFYW84zh/vCLkUWiDmD3swSwCPAFmADcI+ZbchqtgVoS/1sBx7NWPe/gM25KFaKz/S08/SBk/z4dU1UlRd3/3xaSYlx1/pGnuk4S/KePSL5FeSIfhPQ4e6d7j4OPAlszWqzFXjCk54F6sxsJYC7fxfQFSIyo5e6z3Fq8CKbb1oRdikF9a71jZwZushrJy+EXYosAEGCfhXQlfG8O7Vsvm0uy8y2m9leM9vb16c/aReK3ftPUlpivO/65WGXUlCX+uk1+kYKIEjQz3Q9evbfm0HaXJa7P+bu7e7e3tTUNJ9NJabcnd2vnuSd1y5lSXVZ2OUU1IollbQtq+UfXjsVdimyAAQJ+m6gJeN5M9BzBW1E3uT1U0McPTvCT924sLpt0j5080qeO9LPiXOjYZciRS5I0O8B2sxsrZmVA9uAnVltdgL3pkbf3AGcd/feHNcqRWb3/pOYwQc3LKxum7SPbGzGHZ56oTvsUqTIzRn07j4JPAjsBg4CX3X3/WZ2v5ndn2q2C+gEOoAvAB9Lb29mXwF+AFxvZt1m9qs5/gwSU7v3n2Tj6nqWLa4Mu5RQtDRUs2ltA19/4YRG30helQZp5O67SIZ55rIdGY8deGCWbe+5mgKlOHX1j7C/Z5D/ePcNYZcSqp/f2Mwnv76PF46f4x1rFsaVwVJ4ujJWQvH1VHfFQhtWmW3L21dQWVZy6fsQyQcFvRTc2MQUf/mDY7zvbctoaagOu5xQLaosY/ONK/jmyz2MTUyFXY4UKQW9FNxTL57g7PA4v/butWGXEgkfeUczg2OT/P1BDbWU/FDQS0FNTztf/F4nN16zmHeuWxhz28zlzmsbWbG4kv+9p2vuxiJXQEEvBfWd10/zRt8wv/7udZgV571h5ytRYvzKna187/AZfnhEs4VI7inopaC+8N0jrFxSyYduXhl2KZHy0TtbWb64gj/+1kENtZScU9BLwbx64jw/6DzLR+9spSyhXS9TVXmCT7z/Ol44fo6nD6ivXnJL/9qkIKamnd/7m1epqy5j26bVYZcTSb/wjmbWNdXw6d2HmJyaDrscKSIKeimILz3TyYvHz/FHH76RJVULawKzoEoTJfzOB6+n4/QQ33jhRNjlSBFR0EvedZwe4jNPv84HNyznw7dcE3Y5kbb5phXc0lLHn+5+jd7zmuxMckNBL3k1Ne188msvU12e4L/83E0aaTMHM+MzP38zo+NT/MZfPq+LqCQnFPSSN+7On+5+jRdSXTbLFi3Mycvmq235Ij677Tb2dZ/nU994RaNw5Kop6CUv3J3/+v8O8vl/7uSeTavVZTNPH9iwnN/+wHU89eIJPv/dzrDLkZgLNHulyHxMTzt/sHM/f/nsMX7lnWv4g5+5UV02V+DB963n4MlB/vhbr3Hy/Bi/e/cNlJfq2EzmT0EvOXXs7DB/uHM//3Soj994zzoe2vI2hfwVMjM+t+02Vix+jcf/5QivnDjPI7+0kRVL1AUm82NR7P9rb2/3vXv3hl2GzMPI+CQ7vvMGO77bSVmJ8dsfvJ777mpVyOfIN/f18B++to/SRAm/cmcrH72zlYaa8rDLkggxs+fdvX2mdYGO6M1sM/A5IAF80d3/OGu9pdbfDYwAH3X3F4JsK/E1MTXNMx1n+NuXeti9/yTD41P87K3X8Km7b2D5Ar1rVL789M3X8LYVi/mTb7/Gw/9wmMe++wYf2djM+29Yzqa1DdRU6I9zmd2cR/RmlgBeBz5A8ibge4B73P1ARpu7gY+TDPofAz7n7j8WZNuZ6Ig+Gi5OTjF8cYqhsUn6hi5yanCMk+fH6OgbYv+J8xw8eYHxyWkWV5ay5aaV/KvbW3SXpALoOH2Bz/9zJ3/zcg/jk9OUJYxbmutoW76ItY3VrFlaQ2NtBQ015dRXl1FdXkpZwvTXVZG72iP6TUCHu3emXuxJYCuQGdZbgSdStxR81szqzGwl0Bpg25z5mf/xzIIcdzzbr+rMX+Ke8cBT6xyYdhTWbIcAAAYDSURBVGd6OjnefcqdialpxieTP5PTM7/y4spSblq1hI/e2crtrQ2857pGKkoTOfxEcjnrly3i079wC//5Z29i79EBvtfRx96jA+zef5L+4fEZt0mUGJWlJZSVllBaUkJZwigxo6QESswwkucEDCD1+yC9LAj9CsmN+upyvnr/O3P+ukGCfhWQOVF2N8mj9rnarAq4LQBmth3YDrB69ZXNhXJtUw3jC3SOEJvtn5q99WH6H3SJJR+XmJEoSYZBWaKE8kQJ5aUl1FSUUltRSk1FKUtrylm+uJIVSyqpry7T0WEEVJYleFdbI+9qa7y07NzIOMf7Rzg7PM65kXH6hycYm5hidHyKkfEpJqenmZhyJqemmfbkL/wpdzzjAABSBwYBT9950IYyp8WV+ZkeJEjQz/QvOvv/7GxtgmybXOj+GPAYJLtuAtT1Fp/ddtuVbCZSNOqqy6mr1klaebMgQd8NtGQ8bwZ6ArYpD7CtiIjkUZCrL/YAbWa21szKgW3Azqw2O4F7LekO4Ly79wbcVkRE8mjOI3p3nzSzB4HdJIdIPu7u+83s/tT6HcAukiNuOkgOr7zvctvm5ZOIiMiMdMGUiEgRuNzwSk2cISJS5BT0IiJFTkEvIlLkFPQiIkUukidjzawPOHaFmzcCZ3JYTq6orvlRXfOjuuanGOta4+5NM62IZNBfDTPbO9uZ5zCprvlRXfOjuuZnodWlrhsRkSKnoBcRKXLFGPSPhV3ALFTX/Kiu+VFd87Og6iq6PnoREXmzYjyiFxGRDAp6EZEiF6ugN7OEmb1oZt+cYZ2Z2cNm1mFm+8xsY8a6zWZ2KLXuoQLX9a9T9ewzs++b2S0Z646a2Stm9pKZ5XwWtznqeq+ZnU+990tm9vsZ68L8vn4no6ZXzWzKzBpS6/L9fV329cPaxwLUFco+FqCuUPaxAHWFso9Z8harXzOz18zsoJm9M2t9/vYvd4/ND/Dvgb8GvjnDuruBb5G8q9UdwHOp5QngDWAdyRuhvAxsKGBddwL1qcdb0nWlnh8FGkP6vt47y/JQv6+sdj8D/GMBv6/Lvn5Y+1iAukLZxwLUFco+Np/PXMh9DPgL4NdSj8uBukLtX7E5ojezZuBDwBdnaXLpBuXu/iyQvkH5pZubu/s4kL5BeUHqcvfvu/tA6umzJO+ylXcBvq/ZhPp9ZbkH+Equ3jsHQtnH5hLWPnYVQv2+shRkHzOzxcB7gC8BuPu4u5/Lapa3/Ss2QQ98FvgkMNvdv+dzg/JVBawr06+S/I2d5sDTZva8JW+OnktB6nqnmb1sZt8ysxtTyyLxfZlZNbAZ+HrG4nx+X0FeP6x9bD6fu5D7WJDXDmMfC/SZC7yPrQP6gD9PdVt+0cxqstrkbf8Kcs/Y0JnZTwOn3f15M3vvbM1mWDavG5Tnqa50258g+Y/wXRmL73L3HjNbBvydmb3m7t8tUF0vkJwbY8jM7gb+L9BGRL4vkn9S/4u792csy8v3NY/XL/g+FrCuZHEF3McCvnbB97GAdaUVch8rBTYCH3f358zsc8BDwO9ltMnb/hWXI/q7gA+b2VGSf7a8z8z+KqvNbDcoD3Jz83zWhZndTLKrYqu7n00vd/ee1H9PA0+R/BOtIHW5+6C7D6Ue7wLKzKyRCHxfKdvI+pM6j99X0NcPYx8L9LlD2MfmfO2Q9rH5fOZC7mPdQLe7P5d6/jWSwZ/dJj/7Vz5OOuTzh9lP8HyIN5/I+GFqeSnQCazlRycybixgXatJ3kv3zqzlNcCijMffBzYXsK4V/OiCuU3A8dR3F+r3lVq3BOgHagr1fQV5/TD2sYB1FXwfC1hXwfexoJ85pH3se8D1qcd/CHy6UPtXLLpuZmMRvUF5Vl2/DywF/qeZAUx6cna65cBTqWWlwF+7+7cLWNfPA79pZpPAKLDNk3tV2N8XwM8BT7v7cEazfH9fM75+BPaxIHWFsY8FqSuMfSxIXRDOPvZx4MtmVk4yuO8r1P6lKRBERIpcXProRUTkCinoRUSKnIJeRKTIKehFRIqcgl5EpMgp6EVEipyCXkSkyP1/pwEegJHL07oAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Brute-force solution: explicitly compute the PDF for each sample, and multiply everything \n",
    "# (or take logs, add, and exponentiate):\n",
    "likelihood = np.vectorize(lambda mu_hat: np.exp(np.log(norm.pdf(X, loc=mu_hat, scale=sigma)).sum()))\n",
    "L_i = likelihood(mu_values)\n",
    "\n",
    "plt.plot(mu_values, L_i / L_i.sum());"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dfXRU933n8fdXo2cJkITEg5FAYGTH2PEDkaljJ2maJik4bWhP2i3udt34tKVu7Gyz29Os0z192odz2iZnN/HWa+Ik7tZtGm82ibc0S4L7lCZuYgf8hA0YI4sHCQkQSCD0hJ6++8fMkPFYQlcwM/fe0ed1jo5n7v3dme+MLx9d/e7v/q65OyIiUrxKwi5ARETyS0EvIlLkFPQiIkVOQS8iUuQU9CIiRa407AJm0tjY6K2trWGXISISG88///wZd2+aaV0kg761tZW9e/eGXYaISGyY2bHZ1qnrRkSkyCnoRUSKnIJeRKTIKehFRIqcgl5EpMgp6EVEilygoDezzWZ2yMw6zOyhGdabmT2cWr/PzDZmrPt3ZrbfzF41s6+YWWUuP4CIiFzenEFvZgngEWALsAG4x8w2ZDXbArSlfrYDj6a2XQX8W6Dd3W8CEsC2nFUvcpUO9AzyrVd6wy5DJK+CHNFvAjrcvdPdx4Enga1ZbbYCT3jSs0Cdma1MrSsFqsysFKgGenJUu8gVm5p2HvmnDj78Z8/wm19+gX3d58IuSSRvggT9KqAr43l3atmcbdz9BPAZ4DjQC5x396dnehMz225me81sb19fX9D6Reat59wov/j5H/Dp3Yf4qRtX0FBTzqd3Hwq7LJG8CRL0NsOy7NtSzdjGzOpJHu2vBa4Baszsl2d6E3d/zN3b3b29qWnG6RpEcuI//e0BDvQO8t9/8Rb+7Jdu42PvvZbvHT7D9zvOhF2aSF4ECfpuoCXjeTNv7X6Zrc37gSPu3ufuE8A3gDuvvFyRqzM2McU/v97HRzY283O3NWNm/PIda1i5pJI/2X0I3VpTilGQoN8DtJnZWjMrJ3kydWdWm53AvanRN3eQ7KLpJdllc4eZVZuZAT8JHMxh/SLz8szhM4xOTPHBG5dfWlZZluAT72/j5a5zPH3gVIjVieTHnEHv7pPAg8BukiH9VXffb2b3m9n9qWa7gE6gA/gC8LHUts8BXwNeAF5Jvd9juf4QIkE9feAkiypL+bG1S9+0/CMbm1nXVMNndh9ielpH9VJcAk1T7O67SIZ55rIdGY8deGCWbf8A+IOrqFEkJ6amnb8/eJr3vW0Z5aVvPsYpTZTw6+9ex6e+8QqdZ4ZYv2xRSFWK5J6ujJUF4/ljA/QPj/PBDStmXH/HuuRR/p6jA4UsSyTvFPSyYDy9/yTliRJ+/PqZR3W1Lq2msbacPUf7C1yZSH4p6GVBcHeePnCKO9cvpbZi5h5LM6N9TQN7dUQvRUZBLwvC66eGON4/Mmu3TVp7az3H+0c4NThWoMpE8k9BLwvC0/tPYgbv37Dssu1ub20AUPeNFBUFvSwI//LGGW66ZgnLFl1+8tQbr1lMdXlC3TdSVBT0UvTcnQM9g7y9ecmcbUsTJdy2uk5H9FJUFPRS9E6cG2VwbJINKxcHat++poGDvYNcGJvIc2UihaGgl6J3oGcQgA3XBAv621sbmHZ48bimLpbioKCXonegdxAzeNuKYFe73rq6jkSJsVfdN1IkFPRS9A70DLK2sYbq8kAzflBbUcqGlYt1hawUDQW9FL0DvYOB++fT2lvrebFrgImp6TxVJVI4CnopaudHJ+geGA3cP5922+p6xiam6Tg9lKfKRApHQS9F7WBv6kTsPI/o25bVAijopSgo6KWozXfETdraxhpKDA4r6KUIKOilqB3oHaSxtmLOK2KzVZYlWLO0hjcU9FIEAgW9mW02s0Nm1mFmD82w3szs4dT6fWa2MbX8ejN7KeNn0Mw+kesPITKbAz2D8z6aT7u2qZbDpy/kuCKRwpsz6M0sATwCbAE2APeY2YasZluAttTPduBRAHc/5O63uvutwDuAEeCp3JUvMrvxyWkOn74w7/75tLbltRw5M6yRNxJ7QY7oNwEd7t7p7uPAk8DWrDZbgSc86VmgzsxWZrX5SeANdz921VWLBNBxeoiJKb/iI/q2ZbVMTDnHzo7kuDKRwgoS9KuArozn3all822zDfjKbG9iZtvNbK+Z7e3r6wtQlsjlHbjCETdpban7xmrkjcRdkKC3GZb5fNqYWTnwYeD/zPYm7v6Yu7e7e3tT08y3ehOZj4O9g1SWlbC2seaKtr92WXK7DvXTS8wFCfpuoCXjeTPQM882W4AX3P3UlRQpciUO9Axy/YrFJEpmOg6ZW3V5KavqqjTEUmIvSNDvAdrMbG3qyHwbsDOrzU7g3tTomzuA8+7em7H+Hi7TbSOSD51nhljfVHtVr9G2vJbDpxT0Em9zzvLk7pNm9iCwG0gAj7v7fjO7P7V+B7ALuBvoIDmy5r709mZWDXwA+I3cly8ys5HxSU4NXmRtY/VVvc76plp+8MZZpqb9iv8yEAlboOn83H0XyTDPXLYj47EDD8yy7Qiw9CpqFJm3o2eSI2Var7B/Pq1teS0XJ6c5MTDK6qVX90tDJCy6MlaK0tGzwwC0Lr26oF+fGnmjC6ckzhT0UpSOnEkF/VUe0a9PTW6mE7ISZwp6KUpHzwyzbFEFtRXBbjYymyVVZSxbVKGx9BJrCnopSkfODF/10Xxa2/JaHdFLrCnopSgdPTvM2qvsn09b31RLx6kLJMcciMSPgl6KzoWxCc4MjefsiH798kUMj0/Re34sJ68nUmgKeik66aGVVzr1QbZrU6+TPsErEjcKeik6R1JDK3MV9Onx88f7NYulxJOCXorOkb5k0K/J0QVOK5dUUZYwTVcssaWgl6Jz9Oww1yyppLIskZPXS5QYLfXVHO9X143Ek4Jeis6RM8OsbcpNt01aS0O1juglthT0UnSOnh2+6qkPsq1ZWs3xsyMaYimxpKCXojIwPM65kYmcnYhNW91QzYWLkwyMTOT0dUUKQUEvReVIjiYzy7Ym9XoaeSNxpKCXonI0R5OZZUuP4Dl2VidkJX4CBb2ZbTazQ2bWYWYPzbDezOzh1Pp9ZrYxY12dmX3NzF4zs4Nm9s5cfgCRTEfPDFNiya6WXEq/3nGdkJUYmjPozSwBPELyvq8bgHvMbENWsy1AW+pnO/BoxrrPAd9297cBtwAHc1C3yIyOnB2hub6a8tLc/rFaWZZg+eIKjqnrRmIoyL+GTUCHu3e6+zjwJLA1q81W4AlPehaoM7OVZrYYeA/wJQB3H3f3czmsX+RNjuZw1spsqxuqdUQvsRQk6FcBXRnPu1PLgrRZB/QBf25mL5rZF80sP/8KRUj2oa/JcbdN2uqGGp2MlVgKEvQz3RE5ezDxbG1KgY3Ao+5+GzAMvKWPH8DMtpvZXjPb29fXF6AskTc7PzrB4NgkLQ1VeXn9NUurOTk4xtjEVF5eXyRfggR9N9CS8bwZ6AnYphvodvfnUsu/RjL438LdH3P3dndvb2pqClK7yJt0pY62W+rzc0SfHnnTpaN6iZkgQb8HaDOztWZWDmwDdma12Qncmxp9cwdw3t173f0k0GVm16fa/SRwIFfFi2TqHkgFfd66btJDLBX0Ei9z3lDT3SfN7EFgN5AAHnf3/WZ2f2r9DmAXcDfQAYwA92W8xMeBL6d+SXRmrRPJmXT/eb6CPn3RlEbeSNwEunOyu+8iGeaZy3ZkPHbggVm2fQlov4oaRQLp6h9lcWUpS6rK8vL69dVlLKoo5bgumpKY0ZWxUjS6BkbydjQPYGa0NFRr5I3EjoJeikZX/0jOr4jNtmZptbpuJHYU9FIUpqedroHRvB7RQ/K2gt39o0xNa7piiQ8FvRSFvqGLjE9O01KfnzH0aWsaahifmubk4Fhe30cklxT0UhTSY9ubC9B1A5rFUuJFQS9FoSs1hj7fffTpi7G6B0bz+j4iuaSgl6Jw/GwyeFfV5bfrZmVdJSUG3TohKzGioJei0DUwwvLFFVSWJfL6PmWJElYuqaJLR/QSIwp6KQpd/SN5m+MmW3N91aXpFkTiQEEvRaEQY+jTWhqq6erXEb3Eh4JeYm98cprewbG8j7hJa66v4tSFMS5OarpiiQcFvcRez7lR3Mn7GPq0lvpq3KHnnMbSSzwo6CX2uvI8PXG29PtoXnqJCwW9xF56krFC9dE3p/5y6NIJWYkJBb3EXlf/KGUJY/niyoK83/LFlZQlTBdNSWwo6CX2ugZGWFVXRaJkplsX516ixLimrkpdNxIbgYLezDab2SEz6zCzt9zcO3ULwYdT6/eZ2caMdUfN7BUze8nM9uayeBFIXqVaqP75tJb6al00JbExZ9CbWQJ4BNgCbADuMbMNWc22AG2pn+3Ao1nrf8Ldb3V33WlKcu54/wjNBbpYKq2loYoT6qOXmAhyRL8J6HD3TncfB54Etma12Qo84UnPAnVmtjLHtYq8xdDFSQZGJmhpKMzQyrTm+mrODI0zMj5Z0PcVuRJBgn4V0JXxvDu1LGgbB542s+fNbPtsb2Jm281sr5nt7evrC1CWCJemIijU9Adp6ZE3OiErcRAk6Gc6w5V9e53LtbnL3TeS7N55wMzeM9ObuPtj7t7u7u1NTU0ByhLh0lQEBe+jb0hPV6zuG4m+IEHfDbRkPG8GeoK2cff0f08DT5HsChLJiXTQNhfoqti0S2PpNeeNxECQoN8DtJnZWjMrB7YBO7Pa7ATuTY2+uQM47+69ZlZjZosAzKwG+CDwag7rlwWuq3+UqrIES2vKC/q+TbUVVJaVaIilxELpXA3cfdLMHgR2AwngcXffb2b3p9bvAHYBdwMdwAhwX2rz5cBTZpZ+r79292/n/FPIgtU1MEJLQxWpfaxgzIzm+mr10UsszBn0AO6+i2SYZy7bkfHYgQdm2K4TuOUqaxSZVSHnoc/WXF+laRAkFnRlrMSWu3NiYLTg/fNpLfXV6rqRWFDQS2ydH53gwsXJgo+4SWtpqGJwbJLzoxOhvL9IUAp6ia30iJdCXxWbln5fDbGUqFPQS2yFNbQyLX1uQEMsJeoU9BJbhb7hSLb0tAs6opeoU9BLbHX1j7K4spQlVWWhvP+SqjIWVZTqhKxEnoJeYqt7oPCzVmYyM1oaqi/d4UokqhT0EltdA6MFn7UyW0tDleall8hT0EssuTvdA+FdLJXWUl9N98AIyWsGRaJJQS+x1Dd0kbGJ6dBG3KStXlrN2MQ0fUMXQ61D5HIU9BJL6Tlmwhpxk/ajIZbqp5foUtBLLKWDNfSgb9B0xRJ9CnqJpfQRfdhdN806opcYUNBLLHUPjLC0ppzq8kATsOZNZVmCZYsqNMRSIk1BL7HU1T9Kc8jdNmktDdWarlgiTUEvsdQ1MEJLyN02aS31Veqjl0gLFPRmttnMDplZh5k9NMN6M7OHU+v3mdnGrPUJM3vRzL6Zq8Jl4ZqadnrOjYZ6VWym1Q3V9J4fZWJqOuxSRGY0Z9CbWQJ4BNgCbADuMbMNWc22AG2pn+3Ao1nrfws4eNXVikAqVJ01S6MR9M0N1Uw79JzTUb1EU5Aj+k1Ah7t3uvs48CSwNavNVuAJT3oWqDOzlQBm1gx8CPhiDuuWBez42WR/+Oqo9NFrumKJuCBBvwroynjenVoWtM1ngU8Cl/271sy2m9leM9vb19cXoCxZqI71RyvoV6f+stAJWYmqIEFvMyzLnthjxjZm9tPAaXd/fq43cffH3L3d3dubmpoClCUL1fH+EcoSxjV10TgZu2JxJWUJ0xBLiawgQd8NtGQ8bwZ6Ara5C/iwmR0l2eXzPjP7qyuuVoRk101zfTWJkpmOLwovUZL8paOLpiSqggT9HqDNzNaaWTmwDdiZ1WYncG9q9M0dwHl373X3T7l7s7u3prb7R3f/5Vx+AFl4jvUPhz71QbbVDdWarlgia86gd/dJ4EFgN8mRM1919/1mdr+Z3Z9qtgvoBDqALwAfy1O9Ihw/O8KaiAV9c321juglsgJdP+7uu0iGeeayHRmPHXhgjtf4DvCdeVcokuHcyDiDY5ORGVqZ1tJQRf/wOMMXJ6mpCHdaBpFsujJWYuVYxIZWpqXr0cgbiSIFvcTKpaGVUTuiT42lT4/xF4kSBb3ESlfExtCnpevREEuJIgW9xMqxs8M0LaoIfXribHXVZSyuLL3UtSQSJQp6iZVjZ0cidzQPYGasbazh6NnhsEsReQsFvcRKV3/0hlamrVmqoJdoUtBLbFycnKJ3cCxyJ2LTWhtrODEwyvikpiuWaFHQS2x09Y/iTuTG0Ke1Lk1OV6wTshI1CnqJjeP9yW6RKPbRQ/KIHpInjEWiREEvsfGjeehrQq5kZq1Lk3UdOaOgl2hR0EtsHOsfobo8QWNtedilzKheQywlohT0EhvHU0MrzaIxPXE2DbGUqFLQS2wc74/mGPpMa5bWqOtGIkdBL7EwPe0c7x+J7IibtNbGGnrOjXJxcirsUkQuUdBLLJwcHOPi5DRrlkbzRGxaeoilbhQuURIo6M1ss5kdMrMOM3tohvVmZg+n1u8zs42p5ZVm9kMze9nM9pvZH+X6A8jC0NmX7A5Z1xTxoNcQS4mgOYPezBLAI8AWYANwj5ltyGq2BWhL/WwHHk0tvwi8z91vAW4FNqduNSgyL2/0DQGwvqk25EouT0MsJYqCHNFvAjrcvdPdx0ne5HtrVputwBOe9CxQZ2YrU8+HUm3KUj+eq+Jl4ejsG6K2opSmRRVhl3JZ6SGWGnkjURIk6FcBXRnPu1PLArUxs4SZvQScBv7O3Z+b6U3MbLuZ7TWzvX19fUHrlwXijb5hrm2qiezQyjQzo7WxRmPpJVKCBP1M/7Kyj8pnbePuU+5+K9AMbDKzm2Z6E3d/zN3b3b29qakpQFmykHT2DbEu4t02aa0aYikREyTou4GWjOfNQM9827j7OZI3B9887yplQRsZn6Tn/BjrGqN9IjatdWm1hlhKpAQJ+j1Am5mtNbNyYBuwM6vNTuDe1OibO4Dz7t5rZk1mVgdgZlXA+4HXcli/LADpETfXLovJEX1jjYZYSqTMeT82d580sweB3UACeNzd95vZ/an1O4BdwN1ABzAC3JfafCXwF6mROyXAV939m7n/GFLM0iNuoj60Mi1ziOX6mPxykuIW6Mab7r6LZJhnLtuR8diBB2bYbh9w21XWKAtcZ98wZj8auhh1GmIpUaMrYyXy3ugborm+isqyRNilBFJfXUZ9ddmlv0REwqagl8jr7Bvm2piMuIHkEMu25Yt4/ZSCXqJBQS+RNj3tdJ4ZYl1jfIIeoG1ZLa+fukCyV1MkXAp6ibTewTHGJqa5dlk8+ufTrlu+iAtjk5wavBh2KSIKeom2zvSIm7gd0S9P1nv49IWQKxFR0EvEvXE6GfRxPKIH1E8vkaCgl0jrPDPMoopSmmqjPZlZtsbaChpqyjl8Skf0Ej4FvUTaG31DrFtWG/nJzGayflkth0/riF7Cp6CXSOvsG+bamMxxk+265Rp5I9GgoJfIGr44Se/5sdjMcZNNI28kKhT0Elnpbo84XSyVqW1Z+oSs+uklXAp6iawDPYMA3HjN4pAruTI/GmKpfnoJl4JeIutg7yCLKkpprq8Ku5QropE3EhUKeomsA72D3LBycSxH3KSlp0IQCZOCXiJpetp5rXeQG1YuCruUq3Ld8kUcPj2kkTcSKgW9RNLx/hGGx6fYENP++bTrltdq5I2ELlDQm9lmMztkZh1m9tAM683MHk6t32dmG1PLW8zsn8zsoJntN7PfyvUHkOJ0sDd5InbDyiUhV3J11mvkjUTAnEGfug3gI8AWYANwj5ltyGq2BWhL/WwHHk0tnwR+291vAO4AHphhW5G3ONA7SKLELo1ciavrUvUr6CVMQY7oNwEd7t7p7uPAk8DWrDZbgSc86VmgzsxWunuvu78A4O4XgIPAqhzWL0XqQM8g1zbVxOauUrNZWltBY20FB3sV9BKeIEG/CujKeN7NW8N6zjZm1kry/rHPzfQmZrbdzPaa2d6+vr4AZUkxO5gacVMMbm5ewisnzoVdhixgQYJ+prFt2UMILtvGzGqBrwOfcPfBmd7E3R9z93Z3b29qagpQlhSrcyPj9JwfY0ORBP3bVy2h4/QQwxcnwy5FFqggQd8NtGQ8bwZ6grYxszKSIf9ld//GlZcqC8WB9InYmI+4SbulZQnTDq+eOB92KbJABQn6PUCbma01s3JgG7Azq81O4N7U6Js7gPPu3mvJK12+BBx09/+W08qlaKWnPiierps6APZ1K+glHKVzNXD3STN7ENgNJIDH3X2/md2fWr8D2AXcDXQAI8B9qc3vAv4N8IqZvZRa9rvuviu3H0OKycHeCyxblDyJWQwaaytYVVfFy93qp5dwzBn0AKlg3pW1bEfGYwcemGG7Z5i5/15kVgeK6ERsWvKErI7oJRy6MlYiZXxymo7TF4qmfz7t5uY6jp0d4dzIeNilyAKkoJdIOXz6AhNTXpRH9KB+egmHgl4i5fljAwDc1lIXciW5ddOqdNCrn14KT0EvkfLDI/2sXFIZ2znoZ7Okqox1jTW8rCN6CYGCXiLD3dlztJ/bWxtiPQf9bG5uXqIjegmFgl4io3tglFODF7m9tT7sUvLi5uY6Tg1e5NTgWNilyAKjoJfI+OGRfgBuX9sQciX5cUtLsp/+5S4d1UthKeglMvYc7WdxZSnXLYv3XaVms2HlEhIlppE3UnAKeomMPUf7aW9toKSk+PrnAarKE9ywchE/PNofdimywCjoJRLODl3kjb5h2ou0fz7trvWNvHh8QDNZSkEp6CUS9hxNjp/f1Fqc/fNp717fxMSU89yRs2GXIguIgl4iYe/RfspLS3h7c7zvETuX9tZ6KkpL+N7hM2GXIguIgl4iYc/Rfm5trqOiNN63DpxLZVmCTWsbeEZBLwWkoJfQDV+c5NWeQW5fW9z982nvWt/I4dNDnDyv8fRSGAp6Cd3zxwaYmnbai7x/Pu3dbclbZT7ToaN6KYxAQW9mm83skJl1mNlDM6w3M3s4tX6fmW3MWPe4mZ02s1dzWbgUj787cIrKshLuWLs07FIK4m0rFtFYW84zh/vCLkUWiDmD3swSwCPAFmADcI+ZbchqtgVoS/1sBx7NWPe/gM25KFaKz/S08/SBk/z4dU1UlRd3/3xaSYlx1/pGnuk4S/KePSL5FeSIfhPQ4e6d7j4OPAlszWqzFXjCk54F6sxsJYC7fxfQFSIyo5e6z3Fq8CKbb1oRdikF9a71jZwZushrJy+EXYosAEGCfhXQlfG8O7Vsvm0uy8y2m9leM9vb16c/aReK3ftPUlpivO/65WGXUlCX+uk1+kYKIEjQz3Q9evbfm0HaXJa7P+bu7e7e3tTUNJ9NJabcnd2vnuSd1y5lSXVZ2OUU1IollbQtq+UfXjsVdimyAAQJ+m6gJeN5M9BzBW1E3uT1U0McPTvCT924sLpt0j5080qeO9LPiXOjYZciRS5I0O8B2sxsrZmVA9uAnVltdgL3pkbf3AGcd/feHNcqRWb3/pOYwQc3LKxum7SPbGzGHZ56oTvsUqTIzRn07j4JPAjsBg4CX3X3/WZ2v5ndn2q2C+gEOoAvAB9Lb29mXwF+AFxvZt1m9qs5/gwSU7v3n2Tj6nqWLa4Mu5RQtDRUs2ltA19/4YRG30helQZp5O67SIZ55rIdGY8deGCWbe+5mgKlOHX1j7C/Z5D/ePcNYZcSqp/f2Mwnv76PF46f4x1rFsaVwVJ4ujJWQvH1VHfFQhtWmW3L21dQWVZy6fsQyQcFvRTc2MQUf/mDY7zvbctoaagOu5xQLaosY/ONK/jmyz2MTUyFXY4UKQW9FNxTL57g7PA4v/butWGXEgkfeUczg2OT/P1BDbWU/FDQS0FNTztf/F4nN16zmHeuWxhz28zlzmsbWbG4kv+9p2vuxiJXQEEvBfWd10/zRt8wv/7udZgV571h5ytRYvzKna187/AZfnhEs4VI7inopaC+8N0jrFxSyYduXhl2KZHy0TtbWb64gj/+1kENtZScU9BLwbx64jw/6DzLR+9spSyhXS9TVXmCT7z/Ol44fo6nD6ivXnJL/9qkIKamnd/7m1epqy5j26bVYZcTSb/wjmbWNdXw6d2HmJyaDrscKSIKeimILz3TyYvHz/FHH76RJVULawKzoEoTJfzOB6+n4/QQ33jhRNjlSBFR0EvedZwe4jNPv84HNyznw7dcE3Y5kbb5phXc0lLHn+5+jd7zmuxMckNBL3k1Ne188msvU12e4L/83E0aaTMHM+MzP38zo+NT/MZfPq+LqCQnFPSSN+7On+5+jRdSXTbLFi3Mycvmq235Ij677Tb2dZ/nU994RaNw5Kop6CUv3J3/+v8O8vl/7uSeTavVZTNPH9iwnN/+wHU89eIJPv/dzrDLkZgLNHulyHxMTzt/sHM/f/nsMX7lnWv4g5+5UV02V+DB963n4MlB/vhbr3Hy/Bi/e/cNlJfq2EzmT0EvOXXs7DB/uHM//3Soj994zzoe2vI2hfwVMjM+t+02Vix+jcf/5QivnDjPI7+0kRVL1AUm82NR7P9rb2/3vXv3hl2GzMPI+CQ7vvMGO77bSVmJ8dsfvJ777mpVyOfIN/f18B++to/SRAm/cmcrH72zlYaa8rDLkggxs+fdvX2mdYGO6M1sM/A5IAF80d3/OGu9pdbfDYwAH3X3F4JsK/E1MTXNMx1n+NuXeti9/yTD41P87K3X8Km7b2D5Ar1rVL789M3X8LYVi/mTb7/Gw/9wmMe++wYf2djM+29Yzqa1DdRU6I9zmd2cR/RmlgBeBz5A8ibge4B73P1ARpu7gY+TDPofAz7n7j8WZNuZ6Ig+Gi5OTjF8cYqhsUn6hi5yanCMk+fH6OgbYv+J8xw8eYHxyWkWV5ay5aaV/KvbW3SXpALoOH2Bz/9zJ3/zcg/jk9OUJYxbmutoW76ItY3VrFlaQ2NtBQ015dRXl1FdXkpZwvTXVZG72iP6TUCHu3emXuxJYCuQGdZbgSdStxR81szqzGwl0Bpg25z5mf/xzIIcdzzbr+rMX+Ke8cBT6xyYdhTWbIcAAAYDSURBVGd6OjnefcqdialpxieTP5PTM7/y4spSblq1hI/e2crtrQ2857pGKkoTOfxEcjnrly3i079wC//5Z29i79EBvtfRx96jA+zef5L+4fEZt0mUGJWlJZSVllBaUkJZwigxo6QESswwkucEDCD1+yC9LAj9CsmN+upyvnr/O3P+ukGCfhWQOVF2N8mj9rnarAq4LQBmth3YDrB69ZXNhXJtUw3jC3SOEJvtn5q99WH6H3SJJR+XmJEoSYZBWaKE8kQJ5aUl1FSUUltRSk1FKUtrylm+uJIVSyqpry7T0WEEVJYleFdbI+9qa7y07NzIOMf7Rzg7PM65kXH6hycYm5hidHyKkfEpJqenmZhyJqemmfbkL/wpdzzjAABSBwYBT9950IYyp8WV+ZkeJEjQz/QvOvv/7GxtgmybXOj+GPAYJLtuAtT1Fp/ddtuVbCZSNOqqy6mr1klaebMgQd8NtGQ8bwZ6ArYpD7CtiIjkUZCrL/YAbWa21szKgW3Azqw2O4F7LekO4Ly79wbcVkRE8mjOI3p3nzSzB4HdJIdIPu7u+83s/tT6HcAukiNuOkgOr7zvctvm5ZOIiMiMdMGUiEgRuNzwSk2cISJS5BT0IiJFTkEvIlLkFPQiIkUukidjzawPOHaFmzcCZ3JYTq6orvlRXfOjuuanGOta4+5NM62IZNBfDTPbO9uZ5zCprvlRXfOjuuZnodWlrhsRkSKnoBcRKXLFGPSPhV3ALFTX/Kiu+VFd87Og6iq6PnoREXmzYjyiFxGRDAp6EZEiF6ugN7OEmb1oZt+cYZ2Z2cNm1mFm+8xsY8a6zWZ2KLXuoQLX9a9T9ewzs++b2S0Z646a2Stm9pKZ5XwWtznqeq+ZnU+990tm9vsZ68L8vn4no6ZXzWzKzBpS6/L9fV329cPaxwLUFco+FqCuUPaxAHWFso9Z8harXzOz18zsoJm9M2t9/vYvd4/ND/Dvgb8GvjnDuruBb5G8q9UdwHOp5QngDWAdyRuhvAxsKGBddwL1qcdb0nWlnh8FGkP6vt47y/JQv6+sdj8D/GMBv6/Lvn5Y+1iAukLZxwLUFco+Np/PXMh9DPgL4NdSj8uBukLtX7E5ojezZuBDwBdnaXLpBuXu/iyQvkH5pZubu/s4kL5BeUHqcvfvu/tA6umzJO+ylXcBvq/ZhPp9ZbkH+Equ3jsHQtnH5hLWPnYVQv2+shRkHzOzxcB7gC8BuPu4u5/Lapa3/Ss2QQ98FvgkMNvdv+dzg/JVBawr06+S/I2d5sDTZva8JW+OnktB6nqnmb1sZt8ysxtTyyLxfZlZNbAZ+HrG4nx+X0FeP6x9bD6fu5D7WJDXDmMfC/SZC7yPrQP6gD9PdVt+0cxqstrkbf8Kcs/Y0JnZTwOn3f15M3vvbM1mWDavG5Tnqa50258g+Y/wXRmL73L3HjNbBvydmb3m7t8tUF0vkJwbY8jM7gb+L9BGRL4vkn9S/4u792csy8v3NY/XL/g+FrCuZHEF3McCvnbB97GAdaUVch8rBTYCH3f358zsc8BDwO9ltMnb/hWXI/q7gA+b2VGSf7a8z8z+KqvNbDcoD3Jz83zWhZndTLKrYqu7n00vd/ee1H9PA0+R/BOtIHW5+6C7D6Ue7wLKzKyRCHxfKdvI+pM6j99X0NcPYx8L9LlD2MfmfO2Q9rH5fOZC7mPdQLe7P5d6/jWSwZ/dJj/7Vz5OOuTzh9lP8HyIN5/I+GFqeSnQCazlRycybixgXatJ3kv3zqzlNcCijMffBzYXsK4V/OiCuU3A8dR3F+r3lVq3BOgHagr1fQV5/TD2sYB1FXwfC1hXwfexoJ85pH3se8D1qcd/CHy6UPtXLLpuZmMRvUF5Vl2/DywF/qeZAUx6cna65cBTqWWlwF+7+7cLWNfPA79pZpPAKLDNk3tV2N8XwM8BT7v7cEazfH9fM75+BPaxIHWFsY8FqSuMfSxIXRDOPvZx4MtmVk4yuO8r1P6lKRBERIpcXProRUTkCinoRUSKnIJeRKTIKehFRIqcgl5EpMgp6EVEipyCXkSkyP1/pwEegJHL07oAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Alternatively, use the analytic solution, \\mathcal{L}_n(\\mu) \\sim N(\\mu_hat, \\sigma^2/n)\n",
    "\n",
    "L_i2 = norm.pdf(mu_values, loc=mu_hat, scale=sigma/np.sqrt(n))\n",
    "\n",
    "plt.plot(mu_values, L_i2 / L_i2.sum());"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(c)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAd90lEQVR4nO3deZhU9Z3v8fe3d2hAlm4Wm4YGaTY31BYZcQEZEgUTkpvEq5nExJm5PN5otslz5zp3njuT55nJcx2zmPjEJBdNMiYxMU7UCdeoaAy4BrBBQLBpaFmbtRtkbej1e/+oai3Lbrqqu6tOVZ/P63n6oeqc3zn1reJ0ffr8zu+cY+6OiIiET07QBYiISDAUACIiIaUAEBEJKQWAiEhIKQBEREIqL+gCklFSUuIVFRVBlyEiklXWrVvX6O6l8dOzKgAqKiqorq4OugwRkaxiZru7mq4uIBGRkFIAiIiEVFZ1AYmkWktbB6t3HOG5LQfZuPcY/+3ayXzisrKgyxJJCQWASNSW/cf5/E/XcvR0C4MLchk7rIiv/XYDu4808ZUFUzCzoEsU6VcKABHgbGs7X//tBvJyjIdvr+KayhJyzPiHJ9/i/j9uY/eR09z7qUsoyFOvqQwcCgAR4P4XtrHt0Cn+/Y4rmTdt9HvTv/OZS6gYNZjvvrCNshGD+MZHpgVYpUj/SujPGTO70cxqzazOzO7pYr6Z2QPR+ZvM7PLo9GlmtiHm54SZfS0675tmti9m3qL+fWsiiVm78yjLXtnBZ6+a8IEvfwAz48sLKll88Th+9upOjpxqDqhKkf7XYwCYWS7wIHATMBO4zcxmxjW7CaiM/iwFfgzg7rXuPsvdZwFXAE3AUzHL3d85392f6fO7EUnS6eY2vvEfGygfMZh/XDSj23ZfX1jJmdZ2/u/LO9JYnUhqJbIHMBuoc/cd7t4CPAYsiWuzBPiFR6wGhpvZuLg2C4B33L3LExJEgvDomt3sPXqG73zmUooLu+8RnTJ6KJ+4rIxHXt/F4RNn01ihSOokEgBlwN6Y5/XRacm2uRX4Tdy0u6NdRj8zsxFdvbiZLTWzajOrbmhoSKBckcR0dDiPrtnDlRUjmD1pZI/tv7qgkvYO58GVdWmoTiT1EgmArsa+xd9G7JxtzKwA+DjwHzHzfwxcAMwCDgDf7erF3X2Zu1e5e1Vp6YcuZSHSa6/UNbL7SBOfmzMxofYTRxXzmapyfr12D/XvNqW4OpHUSyQA6oHymOfjgf1JtrkJWO/uhzonuPshd2939w7gISJdTSJp86vVuxlVXMCNF41NeJkv3zAFgJ+/titFVYmkTyIB8AZQaWaTon/J3wosj2uzHLg9OhpoDnDc3Q/EzL+NuO6fuGMEnwQ2J129SC/tO3aGF2sOccuV5RTm5Sa83PnDBzF/2miWb9xPe4fupy3ZrccAcPc24G5gBVADPO7uW8zsTjO7M9rsGWAHUEfkr/kvdS5vZoOBhcCTcau+z8zeMrNNwHzg6319MyKJemztHhz47OwJSS/7ycvKaDjZzOvvNPZ/YSJplNCJYNEhms/ETftJzGMH7upm2SZgVBfTP59UpSL9pKWtg9+s3csN00ZTPnJw0svPnz6aoUV5PPXmPq6t1HEpyV46r11C58WaQzSeak744G+8ovxcFl88jhWbD9LU0tbP1YmkjwJAQucPbx2gZEgB103t/V/vS2aVcbqlnRfePtRzY5EMpQCQUDnb2s7KrYdZOHMsuTm9v7rnVZNGMu68In6/IX5AnEj2UABIqLxW18jplnY+euGYPq0nJ8dYMquMl7Y16PpAkrUUABIqz20+yNDCPK6+oKTP6/rkZWW0dzhPbzrQc2ORDKQAkNBoa+/ghZpDLJgxul+u6z9t7FAqRw9hxZaD/VCdSPopACQ01u48yrGm1qTO/O3JghljWLvzKCfOtvbbOkXSRQEgofHcloMU5ef0afRPvL+cMZq2DuelWl2oULKPAkBCoaPDWbHlINdPLWVwQf/dCO+yCSMYMTifP2093G/rFEkXBYCEwsb6Yxw60dyv3T8AuTnG/GmjWVl7mLb2jn5dt0iqKQAkFF54+xC5OcYN0/o2/LMrC2aM4VhTK+v3HOv3dYukkgJAQmFlbQNXTBzBeYPz+33d100tIS/HeHGrzgqW7KIAkAHv0Imz1Bw4wbxpqblw29CifK6aPJIXa3QcQLKLAkAGvM4ROvOnjU7ZayyYPoa6w6fYfeR0yl5DpL8pAGTAW7XtMGOHFTF97NCUvcaCGZFw0V6AZBMFgAxobe0dvLK9keunlmLW+4u/9WTiqGKmjB7CyloFgGQPBYAMaOv3HOPk2baU9f/Hun5qKWt2HuVMS3vKX0ukPygAZEBbWXuYvBxjbmXfL/7Wk+unltLS1sHqnUdS/loi/UEBIAPaqujwz2FF/T/8M97sSSMpys/RZSEkayQUAGZ2o5nVmlmdmd3TxXwzswei8zeZ2eUx83ZFb/6+wcyqY6aPNLMXzGx79N8R/fOWRCLeH/6ZutE/sYryc7lq0ihe3qYAkOzQYwCYWS7wIHATMBO4zcxmxjW7CaiM/iwFfhw3f767z3L3qphp9wAvunsl8GL0uUi/6fxLPB39/52un1rKjsbT7D3alLbXFOmtRPYAZgN17r7D3VuAx4AlcW2WAL/wiNXAcDMb18N6lwCPRB8/AnwiibpFevTStoaUD/+Md300bF7SXoBkgUQCoAzYG/O8Pjot0TYOPG9m68xsaUybMe5+ACD6b5f76Wa21Myqzay6oUG/VJKYyPDPhpQP/4w3uaSY8SMGKQAkKyQSAF399ngSbea6++VEuonuMrPrkqgPd1/m7lXuXlVamr5decluG+uPceJsW79e+z8RZsZ1U0t5va6RljZdHVQyWyIBUA+UxzwfD+xPtI27d/57GHiKSJcSwKHObqLovzqDRvrNS7UN5BhcMyX1wz/jXT+1lNMt7azb/W7aX1skGYkEwBtApZlNMrMC4FZgeVyb5cDt0dFAc4Dj7n7AzIrNbCiAmRUDHwE2xyzzhejjLwC/7+N7EXnPS9sauGxCaq7+2ZOrLxhFXo7x8nZ1A0lm6zEA3L0NuBtYAdQAj7v7FjO708zujDZ7BtgB1AEPAV+KTh8DvGpmG4G1wB/c/bnovHuBhWa2HVgYfS7SZ0dONbNp33GuT3P3T6ehRflcMXEEq3Q+gGS4hO6N5+7PEPmSj532k5jHDtzVxXI7gEu7WecRYEEyxYok4tW6RtwJLAAA5k0bzb89t5WDx88y9ryiwOoQORedCSwDzku1DYwsLuDisvMCq2Hee8NBdWhLMpcCQAaUjg7n5e0NXFtZQk5O+oZ/xps+dihjhxWpG0gymgJABpS3D5yg8VRLoN0/EBkOOm9aKa9ub6RVN4uXDKUAkAGl8wSsayuDP2dk3rRSTja3aTioZCwFgAwoK7ce5qKyYZQOLQy6FOZOidwsXt1AkqkUADJgHD3dwvo977Jg+pigSwEiw0GrKkawSncJkwylAJABY1XtYTr8/fvzZoJ500az9eBJDh4/G3QpIh+iAJAB48Waw5QOLeSi84Mb/hlvfvReBNoLkEykAJABoaWtg5e3NbBg+uhAh3/GmzpmCOPOK9LN4iUjKQBkQKjedZSTzW3cMD1zun8gMhz0humjeWV7I2dbdbN4ySwKABkQ/lhzmIK8HK5Jw83fk/WRC8fS1NLOa3WNQZci8gEKAMl67s6LWw9x9QWjGFyQ0OWt0uovJo9iaGEez285FHQpIh+gAJCst6PxNLuPNLEgw7p/OhXk5TB/+mj+WHOI9o74eymJBEcBIFnvxZrIX9Y3zMiM8f9d+ciFYzhyukVnBUtGUQBI1nt+yyGmjx1K2fBBQZfSrXnTRlOQm8OKLQeDLkXkPQoAyWr7j52heve7LL54XNClnNOQwjzmThnF828fJHL7DJHgKQAkqz3z1gEAbr70/IAr6dlHLxzL3qNnqDlwMuhSRAAFgGS5/7fpABeeP4xJJcVBl9KjBTPGYAbPv61uIMkMCgDJWnuPNrFx7zFuviTz//oHKB1ayBUTRvDcZgWAZIaEAsDMbjSzWjOrM7N7uphvZvZAdP4mM7s8Or3czFaaWY2ZbTGzr8Ys800z22dmG6I/i/rvbUkY/KGz++eSzO7/j/WxS89n68GT1Bw4EXQpIj0HgJnlAg8CNwEzgdvMbGZcs5uAyujPUuDH0eltwDfcfQYwB7grbtn73X1W9OcDN50X6cnTm/Zz6fjzKB85OOhSEvaxS88nL8d46s19QZciktAewGygzt13uHsL8BiwJK7NEuAXHrEaGG5m49z9gLuvB3D3k0ANUNaP9UtI7Wo8zeZ9J7Km+6fTyOIC5k8fzVNv7qNNt4qUgCUSAGXA3pjn9Xz4S7zHNmZWAVwGrImZfHe0y+hnZjaiqxc3s6VmVm1m1Q0NurOSRDy9aT8Ai7Oo+6fTpy4vo+FkM6+9cyToUiTkEgmArq6tGz+Q+ZxtzGwI8ATwNXfv7Pz8MXABMAs4AHy3qxd392XuXuXuVaWlwd/nVYLn7jz15j6umDiC8zP45K/uzJ8+mvMG5fPk+vqgS5GQSyQA6oHymOfjgf2JtjGzfCJf/o+6+5OdDdz9kLu3u3sH8BCRriaRHq3ecZR3Gk5z65XlPTfOQIV5uXzs0nGs2HKQk2dbgy5HQiyRAHgDqDSzSWZWANwKLI9rsxy4PToaaA5w3N0PmJkBPwVq3P17sQuYWey++yeBzb1+FxIqj67ZzbCiPD6WBSd/dedTl4/nbGsHz2pIqASoxwBw9zbgbmAFkYO4j7v7FjO708zujDZ7BtgB1BH5a/5L0elzgc8DN3Qx3PM+M3vLzDYB84Gv99u7kgGr4WQzK7Yc5NNXlFOUnxt0Ob02q3w4k0uKeWKduoEkOAldPD06RPOZuGk/iXnswF1dLPcqXR8fwN0/n1SlIsDj1XtpbXf+as6EoEvpEzPj01Xjue+5WrYePMH0scOCLklCSGcCS9Zo73B+vWYPV18wigtKhwRdTp99dvYEBhfk8tDLO4MuRUJKASBZ46Vth9l37AyfmzMx6FL6xfDBBdxSVc7yjfs4ePxs0OVICCkAJGv88s+7KR1ayMKZmXvjl2T9zTWTaO9wfv669gIk/RQAkhU27D3GytoGbp8zkfzcgbPZlo8czKKLx/Hr1Xs0JFTSbuD8JsmA9t3naxlZXMAd10wKupR+t/S6yZxsbuO3b+ztubFIP1IASMb78ztHeGV7I1+adwFDChMauJZVLhk/nKsmjeSnr+7kbGt70OVIiCgAJKO5O995vpYxwwoHzMHfrnx1QSUHjp/loZd3BF2KhIgCQDLaytrDrNv9Ll9ZUJnVJ3715OopJSy6eCwPrqqj/t2moMuRkFAASMZqbe/gvudqmTByMLdUZed1f5Lxj4sjt8r41h9qAq5EwkIBIBnrB3/cztaDJ/lfi2YMqJE/3SkbPoi750/h2c0HeXV7Y9DlSAgM/N8qyUrrdh/lR6vq+MwV47nxorFBl5M2f3vtZCaOGsw/L9+sA8KScgoAyTinmtv42m83UDZiEP/88QuDLietivJz+ZclF7Gj8TT3PLGJyGW2RFJDASAZxd355vIt7Hv3DPffMmtADvvsyXVTS/nGwqn854b9LNOoIEkhBYBkDHfn2ytq+d26eu6aP4WqipFBlxSYu+ZPYfEl47j3ua2s3Ho46HJkgFIASMa4/4Vt/GjVO9w2ewJf/8upQZcTKDPj25++hBljh/GV37zJ2p1Hgy5JBiAFgATO3fneC9t44E91/Neqcr71iYvIyenyNhKhMrggj4e+UEXp0EI+9/Aa3UNY+p0CQAJ18PhZvvjzN3jgxe18+orx/J//crG+/GOUDR/Ek1+6mismjuDvHt/Id1bU0t6hA8PSPxQAEoiODuepN+v5yP0vsXbnUf5lyYXc96lL9OXfheGDC3jkr2dzS9V4friyjsUPvMLL2xqCLksGAMumYWZVVVVeXV0ddBnSB6eb23hifT0/f20XOxtPc/mE4Xz3lllMKikOurSM5+48u/kg9z67lT1Hm7h+ailfnFvBNVNKQnGinPSema1z96oPTU8kAMzsRuAHQC7wsLvfGzffovMXAU3AF919/bmWNbORwG+BCmAXcIu7v3uuOhQA2cfd2X2kiVfrGnl1eyOv1TVysrmNS8uH89dzK7j5kvPJ1V/9SWlua+eXf97ND1fWcayplZHFBSy6eCxzLyhh1oThjB1WRORXUiSi1wFgZrnANmAhUA+8Adzm7m/HtFkEfJlIAFwF/MDdrzrXsmZ2H3DU3e81s3uAEe7+P89VS28DoLmtnbb27NnTSafYT6VzW/D3nkemdTh0uNPR4bS709butLZ30NruNLe1c6q5jabmdk42t9J4soXG080cOn6WdxpO807DKZpaIme0nn9eEddWlnLLleO5fMIIfUn1UXNbOy9va+T3G/bxx5pDnG3tAKB0aCGTS4opGzGI8cMHUTK0kPMG5TNsUD7FBXkU5edQlJ9LQW4OuTlGfm4OOTmQa0aOGWZgGBjRx++L/T/T/156FeblkNfLPb3uAiCRs2xmA3XuviO6oseAJcDbMW2WAL/wyDfIajMbbmbjiPx1392yS4B50eUfAVYB5wyA3vrXp2v45erdqVi1dKEgNyfyJVRazC1V5VSOGcKcyaOYXFKsL/1+VJiXy8KZY1g4cwzNbe3UHDjJxr3H2Fh/jD1HmvjzO0c4eOIsWdTLK+fw73dcybxpo/t1nYkEQBkQe6uieiJ/5ffUpqyHZce4+wEAdz9gZl2+MzNbCiwFmDBhQgLlfthHLxxL+chBvVo2DCzmb7n47+fcnMjcnJzIX4e5OZGfwrwc8nNzKMzLYXBBHsWFuRQX5lEypJBhRXn6ok+zwrxcZpUPZ1b58A9Mb23v4PiZVo41tXL8TCtnWto529rO2bb29/bi2toje3Ye3cvr8MheYHzvQOxTR6mSbheUDun3dSYSAF39Jsf/73fXJpFlz8ndlwHLINIFlMyyna6pLOGaypLeLCqS1fJzcygZUkjJkMKgS5EMlEiHUj0QezH28cD+BNuca9lD0W4iov/qfHcRkTRKJADeACrNbJKZFQC3Asvj2iwHbreIOcDxaPfOuZZdDnwh+vgLwO/7+F5ERCQJiQ4DXQR8n8hQzp+5+7fM7E4Ad/9JdBjoD4EbiQwDvcPdq7tbNjp9FPA4MAHYA3zG3c95wRMzawB6ezS3BMjEu2yoruSoruSoruRkal3Qt9omuntp/MSsOhGsL8ysuqthUEFTXclRXclRXcnJ1LogNbXp9EERkZBSAIiIhFSYAmBZ0AV0Q3UlR3UlR3UlJ1PrghTUFppjACIi8kFh2gMQEZEYCgARkZBSAIiIhJQCQEQkpBQAIiIhpQAQEQkpBYCISEgpAEREQkoBICISUgoAEZGQUgCIiISUAkBEJKQUACIiIaUAEBEJqbygC0hGSUmJV1RUBF2GiEhWWbduXWNX9wTOqgCoqKiguro66DJERLKKme3uarq6gEREQkoBICISUgoAEZGQyqpjACLpVnHPH957vOvexQFWItL/tAcgIhJSCgARkZBSAIiIhJQCQEQkpBQAIiIhFfgoIDPLBaqBfe5+c9D1iMSO/BEZyAIPAOCrQA0wLOhCJDw0vFMk4C4gMxsPLAYeDrIOEZEwCnoP4PvA3wNDu2tgZkuBpQATJkxIU1ki56Y9CBkIAtsDMLObgcPuvu5c7dx9mbtXuXtVaemHrmYqIiK9FGQX0Fzg42a2C3gMuMHMfhVgPSIioRJYALj7P7j7eHevAG4F/uTunwuqHhGRsNF5ACIiIRX0QWAA3H0VsCrgMkREQkV7ACIiIaUAEBEJKQWAiEhIKQBEREJKASAiElIKABGRkFIAiIiEVEacByASJF3/X8JKewAiIiGlABARCSkFgIhISOkYgEiCdKxABhrtAYiIhJQCQEQkpBQAIiIhpQAQEQkpHQSW0NBBXJEP0h6AiEhIKQBEREJKXUAy4MR29ey6d3GAlYhkNgWADAjq3xdJnrqARERCSgEgIhJSCgARkZBSAIiIhJQOAsuApoPDIt3THoCISEgpAEREQkpdQCJ9pBPPJFspAET6kcJAsom6gEREQkoBICISUgoAEZGQCiwAzKzczFaaWY2ZbTGzrwZVi4hIGAV5ELgN+Ia7rzezocA6M3vB3d8OsCYRkdAIbA/A3Q+4+/ro45NADVAWVD0iImGTEccAzKwCuAxYE2wlIiLhEXgAmNkQ4Anga+5+oov5S82s2syqGxoa0l+giMgAFeiJYGaWT+TL/1F3f7KrNu6+DFgGUFVV5WksT6RPdFKYZLogRwEZ8FOgxt2/F1QdIiJhFWQX0Fzg88ANZrYh+rMowHpEREIlsC4gd38VsKBeX0Qk7AI/CCwiIsFQAIiIhJQCQEQkpHQ/AJE00JBQyUTaAxARCSkFgIhISCkARERCSgEgIhJSCgARkZBSAIiIhJQCQEQkpBQAIiIhpRPBRNJMJ4VJplAASNaK/SIVkeSpC0hEJKQUACIiIaUAEBEJKQWAiEhIKQBEREJKASAiElIKABGRkFIAiIiElE4EEwmQzgqWIGkPQEQkpLQHIBlPl3wQSQ0FgGQkfemLpJ66gEREQkoBICISUuoCEskQGhEk6aYAkEDpS08kOOoCEhEJKQWAiEhIqQtIMoaGfr5PXWOSDgoAkQx3rmBUOEhfBNoFZGY3mlmtmdWZ2T1B1iIiEjaB7QGYWS7wILAQqAfeMLPl7v52UDVJeqirp/+oq0j6IsguoNlAnbvvADCzx4AlgAJggNEXfnp0FwYKCelOkAFQBuyNeV4PXBXfyMyWAkujT0+ZWW0vX68EaOzlsqmkupKjuhJg//beww/UFTM9aBn1ecXI1Lqgb7VN7GpikAFgXUzzD01wXwYs6/OLmVW7e1Vf19PfVFdyVFdyVFdyMrUuSE1tQR4ErgfKY56PB/YHVIuISOgEGQBvAJVmNsnMCoBbgeUB1iMiEiqBdQG5e5uZ3Q2sAHKBn7n7lhS+ZJ+7kVJEdSVHdSVHdSUnU+uCFNRm7h/qdhcRkRDQtYBEREJKASAiElIDIgDMLNfM3jSzp7uYZ2b2QPRyE5vM7PKYeSm9FEUPdf1VtJ5NZva6mV0aM2+Xmb1lZhvMrLq/60qgtnlmdjz6+hvM7J9i5gX5mf2PmJo2m1m7mY2MzkvZZ9bTuoPaxhKoK5BtLIG6Atm+EqgrqO1ruJn9zsy2mlmNmf1F3PzUbV/unvU/wN8Bvwae7mLeIuBZIucdzAHWRKfnAu8Ak4ECYCMwM411XQ2MiD6+qbOu6PNdQEmAn9m8bqYH+pnFtfsY8Kd0fGY9rTuobSyBugLZxhKoK5DtK5n3nObt6xHgb6OPC4Dh6dq+sn4PwMzGA4uBh7tpsgT4hUesBoab2ThiLkXh7i1A56Uo0lKXu7/u7u9Gn64mch5EWiTwmXUn0M8szm3Ab/rrtfsokG2sJ0FuY70U6OcVJy3bl5kNA64Dfgrg7i3ufiyuWcq2r6wPAOD7wN8DHd3M7+qSE2XnmJ6uumL9DZGE7+TA82a2ziKXwuhvidT2F2a20cyeNbMLo9My4jMzs8HAjcATMZNT+Zn1tO6gtrFk3nM6t7FE1h3E9pXQe07z9jUZaAB+Hu36fNjMiuPapGz7yur7AZjZzcBhd19nZvO6a9bFND/H9HTV1dl2PpFfzmtiJs919/1mNhp4wcy2uvvLaaxtPTDR3U+Z2SLgP4FKMuQzI7J7/pq7H42ZlrLPLIF1p30bS7CuSHFp3sYSWHfat68E6+qUzu0rD7gc+LK7rzGzHwD3AP87pk3Ktq9s3wOYC3zczHYR2f25wcx+Fdemu0tOpPJSFInUhZldQqS7Y4m7H+mc7u77o/8eBp4isqvXX3qszd1PuPup6ONngHwzKyEDPrOoW4nbPU/lZ5bAuoPYxhJ6z0FsYz2tO6DtK5n3nM7tqx6od/c10ee/IxII8W1Ss32l4qBGED90f2BpMR88gLI2Oj0P2AFM4v0DKBemsa4JQB1wddz0YmBozOPXgRvT/JmN5f2TBGcDe6KfX6CfWXTeecBRoDgdn1ki6w5iG0uwrrRvYwnWlfbtK9H3nO7tK7rOV4Bp0cffBL6dru0rq7uAumNmdwK4+0+AZ4gcRa8DmoA7ovPSfSmK+Lr+CRgF/MjMANo8cqW/McBT0Wl5wK/d/blU1tVFbZ8G/ruZtQFngFs9ssUF/ZkBfBJ43t1PxzRL5WfW5bozYBtLpK4gtrFE6gpi+0qkLkj/9gXwZeBRi1wTbQdwR7q2L10KQkQkpLL9GICIiPSSAkBEJKQUACIiIaUAEBEJKQWAiEhIKQBEREJKASAiElL/H7epyJ/sG2F5AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, (ax1,ax2) = plt.subplots(2, 1, sharex='col')\n",
    "ax1.plot(mu_values, L_i2 / L_i2.sum())\n",
    "\n",
    "posterior_samples = norm.rvs(loc=mu_hat, scale=sigma/np.sqrt(n), size=1000)\n",
    "ax2.hist(posterior_samples, density=True, bins=mu_values);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(d)**\n",
    "\n",
    "Let $Y = e^X$.  Analytically:\n",
    "\n",
    "- The CDF is $F_Y(z) = \\mathbb{P}_\\theta(e^X \\leq z) = \\mathbb{P}_\\mu(X \\leq \\log z) = \\mathbb{P}_\\mu \\left( \\frac{X - \\mu}{\\sigma} \\leq \\frac{\\log z - \\mu}{\\sigma} \\right) = \\mathbb{P}(Z \\leq \\log z - \\mu) = \\Phi(\\log z - \\mu)$, where $\\Phi$ is the CDF of a standard normal distribution.\n",
    "- The PDF is $f_Y(z) = F'_Y(z) = \\partial \\Phi(\\log z - \\mu) / \\partial z = \\phi(\\log z - \\mu) / z$, where $\\phi = \\Phi'$ is the PDF of a standard normal function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXhdVb3/8fc3UzM3STM0TdKmQ0obSktjaEsBQZlaBAoiICCjWqug3usEij/1ivAoXi+KIoiACogoIlplBoGitHRu6dw0ndIxaYZmbNJk/f7ISQ0hw0mTdCdnf17Pc57k7LNXzneFks/Za+29tjnnEBER/wnzugAREfGGAkBExKcUACIiPqUAEBHxKQWAiIhPKQBERHwqqAAwszlmttnMiszsjk5eNzO7P/D6WjMr6KmtmX3PzPaY2erA46L+6ZKIiASjxwAws3DgAWAukA9cY2b5HXabC+QFHvOBB4Nse59z7tTA44W+dkZERIIXzBHADKDIOVfsnGsEngbmddhnHvC4a7UESDKzzCDbioiIByKC2CcL2N3ueQkwM4h9soJoe5uZ3QAsB77qnKvorpDU1FSXm5sbRMkiItJmxYoVZc65tI7bgwkA62Rbx/Ujutqnu7YPAncFnt8F/AS45QNvbjaf1mElRo8ezfLly4MoWURE2pjZzs62BzMEVALktHueDewNcp8u2zrnDjjnmp1zLcCvaR0u+gDn3MPOuULnXGFa2gcCTEREjlMwAbAMyDOzsWYWBXwSWNhhn4XADYGzgWYBVc65fd21DcwRtLkcWNfHvoiISC/0OATknDtqZrcBLwPhwGPOufVmtiDw+kPAC8BFQBFQB9zcXdvAj77XzE6ldQhoB/C5/uyYiIh0z4bSctCFhYVOcwAiIr1jZiucc4Udt+tKYBERn1IAiIj4lAJARMSngrkOQI5TVV0TL63fR5gZZ+WlMXJ4tNcliYgcowAYAMt2lPPbd3bw6oYDNB5tObb9pIwErj4th5tm5xIW1tk1ciIiJ44CoJ8t3V7Otb9eQkJ0BNfOGM0VBdmEhxmLtpby2oYDfP8fG/hXURk/uXIayXFRXpcrIj6m00D70d7Kei79xb9IjI7kuVvPYHhM5Pted87x+OKd/OD5DaTFD+OB6wqYPjrZo2pFxC90GugAa2hq5nNPrKChqYWHb/jQB/74A5gZN87O5dnPzyYszPjUI++ytqTSg2pFRBQA/ebbf13He3uquO/qU5mQntDtvlOzk3j287NJjovi5t8sY3tZ7QmqUkTkPxQA/WDT/sP8eUUJnz9nPOfnZwTVJiMxmsdvmYEDrn/0XQ4ebhjYIkVEOlAA9IMnFu9kWEQY888a16t249Li+c1Np1Fe28jNv11GQ1PzAFUoIvJBCoA+qm5o4rlVe7hk2qjjOqtnWk4SP79mOuv3Hubu5zcOQIUiIp1TAPTRX1buoa6xmetnjTnun3Hu5Aw+e9ZYnliykxff29eP1YmIdE0B0AfOOZ5YspNp2cOZlpPUp5/19QsnMS0niW88u5bd5XX9VKGISNcUAH2wuPgQRQdruP703D7/rKiIMH5xzXQAvvT0Ko42t/TQQkSkbxQAffDE4p0kxUZy8dTMnncOQk5KLHdffgqrdlXy6L+298vPFBHpigLgOFXUNvLKhgNcVZhDdGR4v/3cS6ZmckF+Bv/36haKS2v67eeKiHSkADhO724/RHOL44Igz/sPlpnxg8umMCwijNufXUtLy9BZqkNEhhYFwHF6Z9shYiLDmZrdt8nfzqQnRvOdS05m2Y4Knliys99/vogIKACO2+JthyjMTSYqYmB+hVcUZHH2xDR+9NIm9lTWD8h7iIi/KQCOQ2n1EbYerGH2+NQBe4+2oaAW57jr7xsG7H1ExL8UAMdhSfEhAE4fP2JA3ycnJZYvfjSPl9bv541NBwf0vUTEfxQAx+GdbYeIHxbBlFGJA/5enz1rHOPT4vjuwvVaK0hE+pUC4DgsKT7EzLEpRIQP/K8vKiKMu+ZNYVd5Hb98c9uAv5+I+IcCoJf2VdWzvax2wId/2ps9IZV5p47ioTe3sUP3DhCRfqIA6KXF207M+H9Hd140mchw4wdaMVRE+okCoJcWbztEUmwkk0cO/Ph/e+mJ0dz20Txe23iAt7aUntD3FpHQpADopcWB8f+wMDvh733Lmbnkjojl+39fT5MWixORPlIA9MKeynpKKuo5fdyJHf5pMywinG9/LJ9tpbX87p0dntQgIqFDAdALm/YdBuCU7OGe1XDu5HTOnpjGz17bSlnNEc/qEJGhTwHQC1sOtK7OOSE9wbMazIz/d3E+9U3N/OSVLZ7VISJDnwKgF7YcqCZzeDTDYyI9rWNCejzXnz6GPy7bxYa9hz2tRUSGLgVAL2w5UE1ehnef/tv78rl5JMZEctc/NuCclowWkd4LKgDMbI6ZbTazIjO7o5PXzczuD7y+1swKetH2a2bmzGzgVlbrB80tjqKDNZyUEe91KQAkxUbxlfMnsrj4EK9sOOB1OSIyBPUYAGYWDjwAzAXygWvMLL/DbnOBvMBjPvBgMG3NLAc4H9jV554MsF3ldRw52jJojgAArp0xmrz0eO55YSNHjmqdIBHpnWCOAGYARc65YudcI/A0MK/DPvOAx12rJUCSmWUG0fY+4BvAoB/D2Ly/GoCTBlEARISH8e2L89l5qE6nhYpIrwUTAFnA7nbPSwLbgtmny7Zmdimwxzm3prs3N7P5ZrbczJaXlnp3BezWA60BMCF9cAwBtTl7YhrnnJTGz18v4pBOCxWRXggmADq75LXjJ/au9ul0u5nFAncC3+npzZ1zDzvnCp1zhWlpaT0WO1A2H6gmJyWGuGERntXQlW9/bDJ1Tc3c95pOCxWR4AUTACVATrvn2cDeIPfpavt4YCywxsx2BLavNLORvSn+RNp6oIaJHp7/350J6QlcN3M0T727iy2BIxURkZ4EEwDLgDwzG2tmUcAngYUd9lkI3BA4G2gWUOWc29dVW+fce865dOdcrnMul9agKHDO7e+vjvWnpuYWistqmDhycAYAwH+dN5G4YRFaLVREgtZjADjnjgK3AS8DG4E/OefWm9kCM1sQ2O0FoBgoAn4NfKG7tv3eiwG2o6yWpmbHxEFyCmhnUuKi+PK5eSzaUsobm3X7SBHpWVAD2s65F2j9I99+20PtvnfArcG27WSf3GDq8MrmwLDKxEF0BlBnbjg9l9+/u4u7n9/ImRNSiTwBdywTkaFLfyGCsOVADWEG49MG7xEAtN4+8lsXTaboYA1PvTvoL60QEY8pAIKwZX81uSPiiI4M97qUHp03OZ3Z40dw32tbqKpr8rocERnEFABB2HKwetAP/7RpWy30cH0TP3t9q9fliMggpgDoQUNTMzvKagf1BHBHkzMTufq00Ty+eAfbSmu8LkdEBikFQA+2l9XS4hhUawAF46sXTCQ6Mpy7dVqoiHRBAdCDXeV1AOSOiPO4kt5JjR/Gl86dwD83HeRNnRYqIp1QAPRgT0U9AKOSoj2upPdumj2W3BGx3PWPDbqJvIh8gAKgB3sr64mODCMlLsrrUnotKiLs2E3kn1yy0+tyRGSQUQD0YE9lPVlJMZh1tq7d4Hfu5HTOykvlvle3UF7b6HU5IjKIKAB6sKeynqzkWK/LOG5tp4XWNjbzk1c2e12OiAwiCoAe7KmoJ2sIjv+3NzEjgetnjeEPS3exfm+V1+WIyCChAOhGfWMzh2obyUqK8bqUPvvv8yaSFBvF/yzUTeRFpJUCoBt7q1rPAMpKHvoBMDw2kq9feBJLd5Tz97X7vC5HRAYBBUA32k4BzUoaunMA7V1VmMOUrETueX4jdY1HvS5HRDymAOjGnsqhew1AZ8LDjO9dcjL7DzfwwBtFXpcjIh5TAHRjT0U94WHGyMTQCACAwtwUPj49i18v2k6x1gkS8TUFQDf2VtYzMjGaiBC7scodF01iWEQY3124XhPCIj4WWn/Z+llJ4CKwUJOeEM1XLpjI21vLeHn9oLwNs4icAAqAbuypqA+Z8f+Orp81hkkjE/j+3zdoQljEpxQAXTja3ML+ww0hcQpoZyLCw/j+vCnsrWrgF//UhLCIHykAunCw+gjNLS5kTgHtzIyxKVxRkM2v3y5ma+DG9yLiHwqALrSdAhqqRwBtvnXRJGKjIrjzr+s0ISziMwqALvznIrDQnANoMyJ+GHfMncTS7eU8u3KP1+WIyAmkAOjCfy4CC+0jAICrC3MoGJ3EPS9spEJLRov4hgKgCyUV9aTERREbFeF1KQMuLMy4+/JTqKpv4ocvbvK6HBE5QRQAXdgbotcAdGVyZiKfOXMsf1y+myXFh7wuR0ROAAVAF/ZUhu41AF35r/MmkpMSw7f+8h4NTc1elyMiA0wB0AnnXOBGMKF7CmhnYqLCuefyUyguq9VicSI+oADoREVdE/VNzSF/CmhnzspL4+PTs3jwzW1s3q9rA0RCmQKgE3vbrgHw0RxAe9++OJ/EmEhuf3YtzS26NkAkVCkAOrE3xO4D0FspcVF895J8Vu+u5Df/3u51OSIyQBQAnSiraT0XPi1hmMeVeOfSaaM4b3I6//vKZnaU1XpdjogMgKACwMzmmNlmMysyszs6ed3M7P7A62vNrKCntmZ2V2Df1Wb2ipmN6p8u9V1ZzREARsT5NwDMjB9cdgqR4WF849m1tGgoSCTk9BgAZhYOPADMBfKBa8wsv8Nuc4G8wGM+8GAQbX/snJvqnDsV+Afwnb53p3+U1RxheEwkURH+PkAaOTya//exfJZuL+fJd3d6XY6I9LNg/sLNAIqcc8XOuUbgaWBeh33mAY+7VkuAJDPL7K6tc+5wu/ZxwKD5iFlWc4TU+CivyxgUrizM5qy8VH744iZ2HarzuhwR6UfBBEAWsLvd85LAtmD26batmd1tZruB6xhMRwDVjaTG+3f4pz0z40dXTCXcjK89s0ZnBYmEkGACwDrZ1vGvQFf7dNvWOXency4H+D1wW6dvbjbfzJab2fLS0tIgyu27spojpPp4ArijUUkxfPfSk1m6o1xnBYmEkGACoATIafc8G9gb5D7BtAV4Criiszd3zj3snCt0zhWmpaUFUW7fldYcIU1HAO9zRUEW503O4N6XN1N0UBeIiYSCYAJgGZBnZmPNLAr4JLCwwz4LgRsCZwPNAqqcc/u6a2tmee3aXwoMimUoG5qaqW44qjmADsyMez4+hbiocL7ypzU0Nbd4XZKI9FGPAeCcO0rr8MzLwEbgT8659Wa2wMwWBHZ7ASgGioBfA1/orm2gzQ/NbJ2ZrQUuAL7cf906fm2ngGoO4IPSE6K5+/JTWFtSxf2vb/W6HBHpo6AWu3fOvUDrH/n22x5q970Dbg22bWB7p0M+Xmu7CEwB0LmLTsnkioJsHnijiLMnplGYm+J1SSJynPx9onsnyqoDRwCaBO7S9y7NJys5hv/+02qqG5q8LkdEjpMCoIP/DAFpDqArCdGR3HfVqeypqOd7Czd4XY6IHCcFQAeaAwhOYW4Kt31kAs+uLOFvq3UzeZGhSAHQQVlNIwnDIoiODPe6lEHvS+fmUTgmmTufW8fOQ1owTmSoUQB0UKqLwIIWER7Gz66ZTpjBF/+wisajOjVUZChRAHRQVq11gHojKymGez8xjbUlVfzvK5u9LkdEekEB0EHrQnA6AuiNOVNGcv2sMTy8qJjXNhzwuhwRCZICoIOyGi0Edzzu/NhkTh6VyFefWcPucq0aKjIUKADaaTzaQlV9kwLgOERHhvPL6wpocY7bnlrJkaPNXpckIj1QALRzqLbtIjDNARyPMSPi+PEnprGmpIp7nt/odTki0gMFQDtl1VoGoq/mTBnJZ84cy+8W7+Svq3R9gMhgpgBoRxeB9Y/b505ixtgU7vjLWtbvrfK6HBHpggKgndJAAOheAH0TGR7GA9cWkBQTxYInV1BZ1+h1SSLSCQVAO8eOADQH0GdpCcN48FMFHKg6wpeeXq1bSYoMQgqAdsqqG4mNCic2KqhVsqUH00cn871LT2bRllLufWlQ3O9HRNrRX7p2dBFY/7t25mg27KviV4uKOWlkAh8vyPa6JBEJ0BFAO60BoOGf/vbdS07m9HEjuOMv77FqV4XX5YhIgAKgnbKaI6RpIbh+Fxkexi+vK2BkYjTzn1jBvqp6r0sSERQA76NlIAZOclwUj9xYSH1jM7f8djk1R456XZKI7ykAAo42t1BRpwAYSBMzEnjgugK2HKjmi0+t5Gizlo8W8ZICIKC8thHndC/ggXb2xDTumjeFNzaX8j9/34BzOj1UxCs6CyjgYHXbRWCaBB5o184czc5DtfxqUTE5KTHM//B4r0sS8SUFQICWgTixbp8ziZKKeu55YRMZidHMOzXL65JEfEcBEHCopnW5ghEKgBMiLMz4yVXTKKs5wteeWUNq/DDOmJDqdVkivqI5gICKwHo1KbEaAjpRoiPDefiGQsalxvO5J1Zo4TiRE0wBEFBZ10SYQUK0DopOpOExkfz2ltNIjI7gxseWUlxa43VJIr6hAAioqGskKTaKsDDzuhTfyRwewxOfmYlzcP2jS9lbqQvFRE4EBUBAZV0TSbGRXpfhW+PT4vndLTM4XN/E9Y++y6HApLyIDBwFQEBFXSPJGv/31JSs4Tx602mUVNRz/aNLdR8BkQGmAAioqGsiKUZHAF6bMTaFh28opOhgDTc+tpTDDU1elyQSshQAAZWBOQDx3tkT0/jldQWs33uYm3+zTOsGiQwQBUBA6xCQjgAGi/PyM/j5NdNZvbuSm3+zVCEgMgCCCgAzm2Nmm82syMzu6OR1M7P7A6+vNbOCntqa2Y/NbFNg/+fMLKl/utR7DU3NNDS1kBynI4DBZO4pmfz06lNZuauSmx5bSrWGg0T6VY8BYGbhwAPAXCAfuMbM8jvsNhfICzzmAw8G0fZVYIpzbiqwBfhmn3tznCrrWv+w6CygweeSaaOOHQloTkCkfwVzBDADKHLOFTvnGoGngXkd9pkHPO5aLQGSzCyzu7bOuVecc23H9UsAz+4V2HYVsM4CGpwuOiWTX1xbwNqSKq5/5F2dHSTST4IJgCxgd7vnJYFtwewTTFuAW4AXg6hlQLQFgI4ABq85U0by0Kc+xMb91Vz9qyUcrG7wuiSRIS+YAOjs0tiOi7h3tU+Pbc3sTuAo8PtO39xsvpktN7PlpaWlQZTbe21DQDoCGNzOy8/gNzedxu6KOq7+1RL26IphkT4JJgBKgJx2z7OBvUHu021bM7sRuBi4znVxZxDn3MPOuULnXGFaWloQ5faehoCGjjMmpPLEp2dSVnOETzz4DlsPVHtdksiQFUwALAPyzGysmUUBnwQWdthnIXBD4GygWUCVc25fd23NbA5wO3Cpc66un/pzXDQJPLR8aEwyf5x/OkdbHFf+ajErdlZ4XZLIkNRjAAQmam8DXgY2An9yzq03swVmtiCw2wtAMVAE/Br4QndtA21+ASQAr5rZajN7qP+61TsVtY3ERIYTHRnuVQnSS/mjEnl2wWySYiK57pEl/HPTAa9LEhlybCjdk7WwsNAtX76833/uV/+0hsXbynjnm+f2+8+WgVVWc4Sbf7OMDfsOc9e8KVw7c7TXJYkMOma2wjlX2HG7rgRGy0AMZanxw3h6/izOykvlW8+9x70vbaKlZeh8qBHxkgKAwDIQcRr/H6rihkXwyA2FXDtzNL98cxtf/uNqGpqavS5LZNDT7a9onQTOTIrxugzpg4jwMO6+bAo5ybH86KVN7C6v4+EbPkR6QrTXpYkMWjoCQAvBhQoz4/PnjOehTxWweX81l/3i32zYe9jrskQGLd8HQEuLo6q+SdcAhJA5UzJ5ZsHpOOCKB9/h+bX7vC5JZFDyfQAcbmiixbXenFxCx5Ss4fzt1jPIH5XIrU+t5EcvbaJZk8Mi7+P7AKjQMhAhKz0xmj98dhbXzhzNg29u45bfLqOiVgvJibRRALQtA6GzgEJSVEQY91x+CvdcfgqLtx3i4p//i7UllV6XJTIo+D4Aqo4tA6EjgFB27czRPLPgdAA+8eBinlyyk6F0EaTIQPB9AGghOP+YlpPEP754JrMnjODbf13HF/+wSjeYEV9TABybA9AQkB8kx0Xx2I2n8Y05J/Hiuv1cfL+GhMS/fB8AlXWNhBkkRisA/CIszPjCORP40+dm0dziuOLBd3jorW1aQkJ8x/cBUFHXyPCYSMLCOrt3jYSyD41J4fkvncl5kzP44YubuO6Rd9mrm8yIjygA6nQRmJ8lxUbxy+sKuPeKqawpqWTOTxfxt9V7NEEsvuD7AGhdCVTDP35mZlx1Wg4vfOksxqfH8+WnV3PrUys5VHPE69JEBpTvA6CiVkcA0io3NY4/L5jN7XMm8dqGg1z400W8tE7LSEjo8n0A6F4A0l54WOuCcgu/eAYZidEseHIlt/5+JaXVOhqQ0OP7AGidA9AQkLzfpJGJ/PXWM/j6hSfx6oYDnH/fW/x5RYnmBiSk+DoAGpqaqW9qJjlORwDyQZHhYdz6kQm88OUzGZcax9eeWcN1j7zL9rJar0sT6Re+DoDKY8tA6AhAujYhPYE/L5jNDy6bwnt7qrjwp4v46WtbdNcxGfJ8HQBaBkKCFRZmfGrWGF7/ytlckJ/BT1/bygX3LeKNTQe9Lk3kuCkA0BGABC89MZpfXFvAk5+eSUS4cfNvl/GZ3y1jh4aFZAjydQBU6l4AcpzOzEvlpS9/mNvnTGLxtkNccN8ifvjiJmqOHPW6NJGg+ToAdAQgfREVEcbnzxnPP792DpdMG8VDb23jnB+/yVPv7uJoc4vX5Yn0yNcBoCMA6Q8ZidH85Kpp/PXWM8gdEcu3nnuPi+5/mzc2HdRpozKo+ToAKmobiY0KJzoy3OtSJAScmpPEMwtO56FPFdB4tIWbf7uMqx9ewspdFV6XJtIpXwdAeV2jPv1LvzIz5kzJ5JX/PpvvzzuZ4tIaPv7Ld/js48vZuO+w1+WJvI+vA6CitlH3ApYBERURxg2n5/LW1z/CV86fyJJth5j7s7e59amVFB2s9ro8EcDvAaCloGWAxQ2L4Evn5vGv2z/KbR+ZwBubDnL+fYv40h9WKQjEcz4PAA0ByYkxPDaSr114Em9/4yN87sPjeW3jAc6/bxG3PrWSDXs1NCTeiPC6AC+V1zaSonWA5AQaET+MO+ZOYv6Hx/HI28U8vngnz6/dx0cnpfOFc8ZTmJvidYniI749AmhqbqG64aiOAMQTKXFRfGPOJP59+0f56vkTWbWrgk88tJhPPPgOr6zfr/sTywnh2wA4dg2AJoHFQ8NjI/niuXn8+46P8r1L8tlX1cD8J1Zw3v+9xZNLdlLfqAXnZOAEFQBmNsfMNptZkZnd0cnrZmb3B15fa2YFPbU1syvNbL2ZtZhZYf90J3iVWghOBpHYqAhuOmMsb339HO6/ZjpxwyL49l/XcfoPX+felzbpZvUyIHqcAzCzcOAB4HygBFhmZgudcxva7TYXyAs8ZgIPAjN7aLsO+Djwq37sT9DKa1sDQHMAMphEhIdx6bRRXDI1k+U7K3j07e089NY2frWomAvyM7hxdi4zx6ZgZl6XKiEgmEngGUCRc64YwMyeBuYB7QNgHvC4a73ufYmZJZlZJpDbVVvn3MbAtv7qS69oHSAZzMyM03JTOC03hd3ldTy5ZCdPL9vNi+v2MzEjnutmjuHygiwSo/XvV45fMENAWcDuds9LAtuC2SeYtp6oCMwB6AhABruclFi+edFklnzzXO69YirRkeF8d+F6Zt79Ol9/Zg0rdlZozSE5LsEcAXT2Eb3jv7au9gmmbfdvbjYfmA8wevTo3jTtVtsQkOYAZKiIiQrnqtNyuOq0HNaWVPLUu7tYuGYvz6woYWJGPFcV5nD59CxGxA/zulQZIoIJgBIgp93zbGBvkPtEBdG2W865h4GHAQoLC/vtY05FbSMxkVoIToamqdlJTM1O4tsX5/P3NXt5eukufvD8Rn744ibOnZzOFQXZnHNSOlERvj3RT4IQTAAsA/LMbCywB/gkcG2HfRYCtwXG+GcCVc65fWZWGkRbT1TUNWn4R4a8+GERXDNjNNfMGM2WA9U8s3w3z63aw8vrD5AcG8kl00Zx2fQspuckaeJYPqDHAHDOHTWz24CXgXDgMefcejNbEHj9IeAF4CKgCKgDbu6uLYCZXQ78HEgDnjez1c65C/u7g12pqGvUBLCElIkZCdz5sXxunzOJt7eW8ezKEv64bDePL97J6JRY5p06ikunjSIvI8HrUmWQsKE0eVRYWOiWL1/eLz/rsgf+TUJ0BE98ema//DyRwehwQxMvr9vPwjV7+XdRGS0OTspI4OKpmXxsaibj0uK9LlFOADNb4Zz7wPVWvl0LqLKukdEpsV6XITKgEqMjubIwhysLczhY3cCL7+3nH2v38pNXt/CTV7cwaWQCc6dkMveUkeSlx2uYyGd8GwDltY0kawhIfCQ9IZobZ+dy4+xc9lXV8+J7+3lx3T5++voW7nttC2NT47jg5AwuPHkkp2YnERamMAh1vgyAo80tHG44SrImgcWnMofHcMuZY7nlzLEcONzAKxsO8Mr6/Tz69nZ+9VYxqfHDOG9yOudOzuCMCSOIjfLln4qQ58v/qpX1ughMpE1GYjTXzxrD9bPGUFXXxJtbDvLqhgM8v3YfTy/bTVREGKePG8FHJ6VzzklpjBkR53XJ0k98GQAVtW3LQCgARNobHhvJvFOzmHdqFo1HW1i2o5zXNx7kjc0H+e7C9QCMTY3j7IlpfHhiKrPG6ehgKPPlf7ljC8EpAES6FBURxhkTUjljQirfuSSfHWW1vLn5IG9uKeUPS3fx23d2EBUexofGJHNmXut+p2QNJ1xzB0OGLwOgQvcCEOm13NQ4bkody01njKWhqZnlOyp4e2spi7aW8eOXN/PjlzeTGB3BzHEjmD1+BLPHp5KXHq/J5EHMpwGgdYBE+iI6Mpwz81I5My+VbwJlNUd4Z9sh/r21jMXFh3h1wwGgdZ5tRm4Ks8alMGPsCE4amaAjhEHElwGgheBE+ldq/DAundZ6pTFASUUdi7cdYklxOe9uP8RL6/cDkBAdQeGYZPVzuLIAAAnfSURBVApzUygck8y0nCStx+UhXwZAZV3rQnAxUfqHJzIQspNjubIwlisLW9eCLKmoY9mOcpZur2DZjnLe2LwZgMhw4+RRwykYnUzBmCQKRieTOTxaF6SdIL4MgPLaJl0EJnICZSfHkp0cy+XTs4HWM/FW7Kxg+c4KVu6s4Pfv7uSxf28HID1hGNNHJ3FqTjLTsodzSvZwEnTjmwHhywCoqGvURWAiHkqOi+K8/AzOy88AoPFoCxv3HWb17kpW765k1a4KXl7fOo9gBuNS45iancQpWa2BkJ+ZSNwwX/756le+/A1W1DXqIjCRQSQqIoxpOUlMy0nixsC2yrpG1pZUsXp3JWtLqnhnWxnPrdoDtIbC2NQ4powazsmjEskflcjJo4br/+te8mcA1DaSnayF4EQGs6TYKD48MY0PT0w7tu3A4QbW7ali3Z7DvLenihU7K1i45j/3mMpIHMbkzEQmZyYyaWQCk0YmMi4tjshw3RinM74MgPLaRlI0ByAy5GQkRpORGM25kzOObauobWTDvsNs2HuYjfsOs2HfYf61tYyjLa1L3UeGG+PT4pmYkcBJIxPIS2/9Picl1venpPouALQQnEhoSY6LOnbFcpvGoy0Ul9WwaV81G/cfZsv+6g8cLQyLCGNcWjx56fFMCDzGp8WTmxrLsAh/nCHouwBoWwhO1wCIhK6oiDAmjUxk0shELiPr2PbqhiaKDtaw9UANWw5UU1Raw8pd7w+GMGs9a2lcWhzjUuMDX+PITY1jZGJ0SF3Z7LsAaFsITkcAIv6TEB3J9NHJTB+d/L7tdY1HKS6tZVtpDdsO1rCtrJbi0lqWFB+ioanl2H7RkWGMSYkjNzWW3BFxjBkRx5gRsYxOiWVUUsyQG1LyXwAE1gHSQnAi0iY2KoIpWcOZkjX8fdtbWhz7Dzewo6yW4rJadpTVsuNQLUUHa3hjUymNzf8Jh8hwIysphpyU1kBo+5qdHENOcixJsZGD7gI33wVA+bGloDUJLCLdCwszRiXFMCophtnt5hgAmgPhsLOslp3ldewqr2PXoTp2V9Tx/Hv7qAx82GwTFxVOdnIsWckxZCXFHPs6Kqn1a1rCsBN+BOG7AKgMLASn84VFpC/Cw1o/8WclxTC7k9cPNzRRUl7P7oo6Sirq2V1ex57KevZU1LN8RzmHG46+b/+IMCMjMZpRSdFkDo8hMymazMRoRg6PIXN4NBPS4/v94jffBUC5VgIVkRMgMTqS/FGR5I9K7PT16oYm9lY2sKeyjr2VDeytrG99VDWwancFL65roKnZHdv/Nzedxkcmpfdrjb4LgKKDNaTGR2khOBHxVEJ0JCeNjOSkkQmdvt7S4iiva2R/VQP7qhqYmj280/36wncBsGpX5QfOABARGWzCwozU+GGkxg/7wOR0v73HgPzUQaq8tpHtZbVMH53kdSkiIp7zVQCs3l0BQIGOAERE/BUAK3dWEh5mAzKWJiIy1PgrAHZVMDkzgdgo3019iIh8gG8CoLnFsWZ3JdNzNPwjIgI+CoAtB6qpbWymYIwmgEVEwEcBsHKXJoBFRNrzTwDsrGREXBSjU3QnMBERCDIAzGyOmW02syIzu6OT183M7g+8vtbMCnpqa2YpZvaqmW0NfB3Qj+ardlUwfXTSoFuNT0TEKz0GgJmFAw8Ac4F84Bozy++w21wgL/CYDzwYRNs7gNedc3nA64HnA6KitpHislpdASwi0k4wRwAzgCLnXLFzrhF4GpjXYZ95wOOu1RIgycwye2g7D/hd4PvfAZf1sS9dWr27EtD4v4hIe8EEQBawu93zksC2YPbprm2Gc24fQOBr/y5z187KXRWEGUzL0QVgIiJtggmAzgbNXZD7BNO2+zc3m29my81seWlpaW+aHpOdHMMnPpStC8BERNoJJgBKgJx2z7OBvUHu013bA4FhIgJfD3b25s65h51zhc65wrS0tCDK/aCrTxvNvZ+YdlxtRURCVTABsAzIM7OxZhYFfBJY2GGfhcANgbOBZgFVgWGd7touBG4MfH8j8Lc+9kVERHqhxzER59xRM7sNeBkIBx5zzq03swWB1x8CXgAuAoqAOuDm7toGfvQPgT+Z2aeBXcCV/dozERHpljnXqyF5TxUWFrrly5d7XYaIyJBiZiucc4Udt/vmSmAREXk/BYCIiE8pAEREfEoBICLiUwoAERGfGlJnAZlZKbDzOJunAmX9WM5QoD77g/rsD33p8xjn3AeupB1SAdAXZra8s9OgQpn67A/qsz8MRJ81BCQi4lMKABERn/JTADzsdQEeUJ/9QX32h37vs2/mAERE5P38dAQgIiLt+CIAerqp/VBlZo+Z2UEzW9duW4qZvWpmWwNfk9u99s3A72CzmV3oTdXHz8xyzOwNM9toZuvN7MuB7aHc52gzW2pmawJ9/p/A9pDtcxszCzezVWb2j8DzkO6zme0ws/fMbLWZLQ9sG9g+O+dC+kHrMtTbgHFAFLAGyPe6rn7q24eBAmBdu233AncEvr8D+FHg+/xA34cBYwO/k3Cv+9DL/mYCBYHvE4AtgX6Fcp8NiA98Hwm8C8wK5T636/tXgKeAfwSeh3SfgR1AaodtA9pnPxwBBHNT+yHJObcIKO+weR7wu8D3vwMua7f9aefcEefcdlrv3TDjhBTaT5xz+5xzKwPfVwMbab3HdCj32TnnagJPIwMPRwj3GcDMsoGPAY+02xzSfe7CgPbZDwEQzE3tQ0mGa70bG4Gv6YHtIfV7MLNcYDqtn4hDus+BoZDVtN429VXnXMj3Gfgp8A2gpd22UO+zA14xsxVmNj+wbUD77Ie7pPf5xvQhImR+D2YWDzwL/Jdz7rBZZ11r3bWTbUOuz865ZuBUM0sCnjOzKd3sPuT7bGYXAwedcyvM7JxgmnSybUj1OeAM59xeM0sHXjWzTd3s2y999sMRQDA3tQ8lB8wsEyDw9WBge0j8HswsktY//r93zv0lsDmk+9zGOVcJvAnMIbT7fAZwqZntoHXI9qNm9iSh3Wecc3sDXw8Cz9E6pDOgffZDAARzU/tQshC4MfD9jcDf2m3/pJkNM7OxQB6w1IP6jpu1ftR/FNjonPu/di+Fcp/TAp/8MbMY4DxgEyHcZ+fcN51z2c65XFr/f/2nc+5ThHCfzSzOzBLavgcuANYx0H32eub7BM2uX0TrGSPbgDu9rqcf+/UHYB/QROsngk8DI4DXga2Brynt9r8z8DvYDMz1uv7j6O+ZtB7mrgVWBx4XhXifpwKrAn1eB3wnsD1k+9yh/+fwn7OAQrbPtJ6luCbwWN/2d2qg+6wrgUVEfMoPQ0AiItIJBYCIiE8pAEREfEoBICLiUwoAERGfUgCIiPiUAkBExKcUACIiPvX/AUzBAlWoPkhpAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def posterior_density(z):\n",
    "    # Suppress warnings about log(z) when z < 0 and division by zero \n",
    "    # np.where will filter out invalid values\n",
    "    with np.errstate(divide='ignore', invalid='ignore'):\n",
    "        return np.where(z > 0, norm.pdf(np.log(z) - mu_hat) / z, 0)\n",
    "    \n",
    "z_values = np.linspace(0, 500, 100)\n",
    "f_values = posterior_density(z_values)\n",
    "\n",
    "plt.plot(z_values, f_values);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAaGUlEQVR4nO3de3Bc5X3/8fdXq7uwLduSwXc5xNCalItRDPzSSWhaWpukdtPbwKSF0ItLG37zazudxAxNZvpHW6eZXkJDcV1KG9o0NNOGRNNCiQvln7ZOLJurASfC8UW2MbINMtjWZXe/vz/OWXst766OpJVWu8/nNaNo95zn0T5fHPuj55znnGPujoiIhKuu0gMQEZHKUhCIiAROQSAiEjgFgYhI4BQEIiKBq6/0ACaio6PDu7q6Kj0MEZGqsnv37hPu3llsf1UFQVdXF729vZUehohIVTGzg6X269CQiEjgFAQiIoELLgieff04v/5YL7qiWkQkElwQfOcHp9jx6nEGz41WeigiIrNCcEEwPJoF4Mg75yo8EhGR2SG8IEhnADj2zlCFRyIiMjsEFwRD8Yzg2KBmBCIiEGQQRDOCo4OaEYiIQMhBoHMEIiJAgEEwnI4PDekcgYgIEGAQXDg0pBmBiAgEGQTRjODNwSEyWV1UJiISXhDEy0fTWefEe8MVHo2ISOUlCgIzW29m+8ysz8y2FNhvZvZgvP8lM1s7gb6/Z2ZuZh1TKyWZ4dEs7a0NgE4Yi4hAgiAwsxTwELABWAPcaWZrxjTbAKyOvzYDDyfpa2bLgduAQ1OuJKHhdIZVHW0AHNMSUhGRRDOCdUCfu+939xHgcWDTmDabgMc8shNoN7PFCfr+OfAZYMYO1g+NZs8HgWYEIiLJgmApcDjvfX+8LUmbon3NbCNwxN1fLPXhZrbZzHrNrHdgYCDBcEsbGs1wxdxmWhpSHNUSUhGRREFgBbaN/Q2+WJuC282sFXgA+Px4H+7u29292927OzuLPmktkXQmSzrrNDekWNzerNtMiIiQLAj6geV575cBRxO2Kbb9SmAV8KKZHYi37zGzKyYy+InKXUzWVF/H0vYW3WZCRIRkQbALWG1mq8ysEbgD6BnTpge4K149dDMw6O7HivV195fdfZG7d7l7F1FgrHX3N8tVWCG5i8maG1IsntescwQiIiR4eL27p83sPuBpIAU86u57zezeeP824EngdqAPOAvcU6rvtFSSwFA8I2huqGPxvBZOvDfMSDpLY31wl1OIiJw3bhAAuPuTRP/Y52/blvfagU8n7VugTVeScUzVcN6MYGl7C+5w/PQQyxe0zsTHi4jMSkH9Kpy7vURTfR2L25sBPalMRCSsIIhvL9HUkGLxvBZAD6gREQkrCHKHhupTLIlnBLqWQERCF1QQDOedLG5trKe9tUEzAhEJXlhBEM8ImupTACye16IZgYgEL6ggyJ0sbm6Iyl6iawlEREILggvLRwGWtLfoDqQiErygguDCOYL40FB7M4PnRjkznK7ksEREKiqoIBg6f44gKntFfCHZoVNnKzYmEZFKCywILp4RrFwQPZfg4MkzFRuTiEilhRUE6QwNKSNVF90de8XCaEZw8KRmBCISrqCCYHg0S3O8dBRgXksDC9oaOaAgEJGABRUEQ+kMTQ2pi7atXNiqQ0MiErSwgmA0c/5Ecc7KBa06NCQiQQsqCIZHs+cvJstZubCNo4PnGI5vSCciEpqwgiCdOb9iKKeroxV3OHxKVxiLSJiCCoKh0ewlQbAiXkJ66JTOE4hImAILgkvPEXTFS0gPnNB5AhEJU1hBUODQ0IK2RuY01WvlkIgEK6ggKHSy2MxYsbCVg7rNhIgEKqggGEpnLrqgLKdrYZuWkIpIsMIKgtEsTQ2XlrxyYSuHT50lnclWYFQiIpUVWBBkzj+dLF/XwjbSWdfTykQkSEEFwXD60uWjkHfzOS0hFZEABRME2awzkr70ZDFEMwJAN58TkSAFEwS5p5MVOjS0aE4TzQ11HDyhGYGIhCeYILjwvOJLS66rM1Ys0BJSEQlTMEEw9nnFY61c2KaLykQkSMEEQakZAUS3mjh48izZrM/ksEREKi5REJjZejPbZ2Z9ZralwH4zswfj/S+Z2drx+prZF83s9bj9E2bWXp6SChtK5x5cX3xGMJzO8uZpLSEVkbCMGwRmlgIeAjYAa4A7zWzNmGYbgNXx12bg4QR9dwAfcPdrge8B90+5mhIuPLi+cMlrlswF4KX+d6ZzGCIis06SGcE6oM/d97v7CPA4sGlMm03AYx7ZCbSb2eJSfd392+6ejvvvBJaVoZ6ihnOHhorMCK5ZMpfG+jp2H3x7OochIjLrJAmCpcDhvPf98bYkbZL0BfgV4KkEY5m0odzy0SIni5vqU/zI0nnsOaQZgYiEJUkQWIFtY8+oFmszbl8zewBIA18t+OFmm82s18x6BwYGEgy3sNzJ4rHPI8h348r5vNw/qMdWikhQkgRBP7A87/0y4GjCNiX7mtndwMeBT7p7weU67r7d3bvdvbuzszPBcAu7sGqo8IwAYO2K+Yxksrxy5PSkP0dEpNokCYJdwGozW2VmjcAdQM+YNj3AXfHqoZuBQXc/Vqqvma0HPgtsdPdpv5LrwnUExUteuzJauPT8IZ0nEJFwjBsE8Qnd+4CngdeAr7v7XjO718zujZs9CewH+oC/AX6rVN+4z5eBOcAOM3vBzLaVr6xLDSeYESya08zyBS06YSwiQalP0sjdnyT6xz5/27a81w58OmnfePv7JzTSKcotHy11jgDgxhXz+e83TuLumBU6xSEiUlsCvLK4+IwAohPGA+8O0//2uZkYlohIxQUTBMPpLKk6oyFVuuQbVswHYI/OE4hIIIIJgqHRDM3jHBYC+KEr5tDamGKPzhOISCDCCYJ0ZtzDQgD1qTquX97Obs0IRCQQ4QTBaHbcE8U5N66cz2vH3uXsSHr8xiIiVS6YICj2vOJCursWkMk6//vGyWkelYhI5QUTBEOjmaL3GRrrlvctZH5rA088f2SaRyUiUnlBBUGpq4rzNdbX8dPXLWHHq8c5PTQ6zSMTEamsYIJgeALnCAA+ccNShtNZ/uPlN6dxVCIilRdOECRcNZRz/fJ2VnW08Y3n+6dxVCIilRdMEAyNZos+lKYQM+MTNyxl5/5THHlHVxmLSO0KJwjSyc8R5PzM9dEzdL6pk8YiUsPCCYLRTNEH1xezYmEr3Svn88TzRyjyuAQRkaoXTBBE1xFMvNxPrF1K31vv8cJhPcJSRGpTMEEQLR+d2IwAYON1S1jQ1sjWp17XrEBEalIQQeDu0S0mJhEEc5ob+J3bruI7PzjF03uPT8PoREQqK4ggyD2mciLXEeS784PLueryy/jjp17Tg+1FpOYEFQSTOTQE0R1JH/jYGg6ePMtj/3OwnEMTEam4MILg/NPJJl/uR67q5CNXdfLgs9/n5HvD5RqaiEjFBREEuecVT+SCskJ+/2M/zNBohv/7tecZiWcZIiLVLowgiI/rN01hRgCw+vI5bP3Za/mfN07yuW++olVEIlIT6is9gJkwXKYZAcDP3biMAyfP8JfP9tHV0cZv3nrllH+miEglBREEuRnBZE8Wj/W7t13FgZNn+cJ/vM7clno+edPKsvxcEZFKCCMIynCyOJ+Z8cWfv5bT50Z54IlXeO3YaT7/8WtonOTyVBGRSgriX67cyeKJ3muolOaGFI9+6oP8xkfexz/uPMQnH9nJm4NDZfv5IiIzJZAgKO+MICdVZ9y/4Yd58M4bePnIIB/90+d48Jnvc25EF52JSPUIIgimekHZeDZet4Rv//ZHuPXqTv5sx/f46J8+xz/87wHe1WMuRaQKBBEEuRnBVJePlrJiYSt/9ckb+fpv3MLlc5v53Lf2ctMfPcP933iZ3QdPkclqqamIzE5BnSwu5zmCYtatWsATv/V/eLF/kK/uPMg39vTzte8eYkFbIz929SI+fFUHa1fMZ9n8Fsxs2scjIjKeIILgwqGhmZkAmRnXL2/n+uXtfO6n1/DcvgGefe04//nacf51T/QM5M45TVy3bB5XXT6Hq6+Yw5Wdl7FiYStzmxtmZIwiIjmJgsDM1gNfAlLAI+6+dcx+i/ffDpwFPuXue0r1NbMFwD8DXcAB4Bfd/e2pl3Sp4dEMZtCYmvkjYXObG9h43RI2XreEdCbLvuPvsufg2+w59A57jw7y3L4B0nmHjea1NLBsfgtXzG3m8nnNLJrTxMK2Rua3NbKgtZG5LQ3Ma2lgbksDlzXVk6rTrEJEpmbcIDCzFPAQcBvQD+wysx53fzWv2QZgdfx1E/AwcNM4fbcAz7j7VjPbEr//bPlKu2AoHT24vtKHYupTdVyzZB7XLJnHL98SbRtJZ/nBiTPsH3iPw2+f5dCpsxx5+xzHBod44fA7nDwzUvJntjSkaGuqp60pRUtDitbGFM0Nua86GlN1NNWnaKyvo7G+joZUHY0poz5VR33KaKiLvtfXGan4dcqM+pRRZ0aq7sL3VF0020lZtK2ujui7GXUW7TOLthnx9/g/+fntBkauPUBuG+f/fCxub1zon/sZ+X+Glrc9/kkXtx3bsECbAk0uUujzLm1TZHvRHiV+WIKfO4UfOSmV/nszWdU56uKa6uuon6ZfZpPMCNYBfe6+H8DMHgc2AflBsAl4zKOb7+w0s3YzW0z0236xvpuAW+P+XwGeY7qCYHTiD66fKY31dVx9RXR4qJDRTJZ3zo5y6swIp86MMHhulNPnRhk8N8p7w2nODKc5M5Lm7EiGsyMZzo1kGBrNcHpolKHRLCPpLMPpDMPpLOmMM5LJ6oZ5IlXo7+/5ILdevWhafnaSIFgKHM5730/0W/94bZaO0/dydz8G4O7HzKxghWa2GdgMsGLFigTDvdRPXXMF71902aT6VlpDqo7OOU10zmkq2890dzJZJ511RjNZMllnNOOks9HrbJbzrzNx22wWsh69z2adrEfvs1nHiffFr4n3ee474PGOrEev/fxrj8cUj42oX9Qmb//5/7nQ7uJ++fVdun9sm/wdxdZzXdS3yA0Gk/RN2ufi/rNjldksGcaEeaL/ytXlys7p+zcsSRAUmmGN/a9crE2SviW5+3ZgO0B3d/ek/nQ/9P4OPvT+jsl0rUkWH/qpT03ftRUiUj2SHC/pB5bnvV8GHE3YplTf4/HhI+LvbyUftoiIlEuSINgFrDazVWbWCNwB9Ixp0wPcZZGbgcH4sE+pvj3A3fHru4FvTbEWERGZBEtyLNLMbgf+gmgJ6KPu/odmdi+Au2+Ll49+GVhPtHz0HnfvLdY33r4Q+DqwAjgE/IK7nxpnHAPAZB8a3AGcmGTf2aiW6lEts1ct1VNLtcDE6lnp7p3FdiYKglpgZr3u3l3pcZRLLdWjWmavWqqnlmqB8tYzO9dUiojIjFEQiIgELqQg2F7pAZRZLdWjWmavWqqnlmqBMtYTzDkCEREpLKQZgYiIFKAgEBEJnIJARCRwCgIRkcApCEREAqcgEBEJnIJARCRwCgIRkcApCEREAqcgEBEJnIJARCRwCgIRkcApCEREAqcgEBEJXH2SRma2HvgS0XOHH3H3rWP2W7z/dqJnFn/K3fck7Pt7wBeBTncv+fzNjo4O7+rqSjJkERGJ7d69+0SpZxaPGwRmlgIeAm4D+oFdZtbj7q/mNdsArI6/bgIeBm4ar6+ZLY/3HUpSTFdXF729vUmaiohIzMwOltqf5NDQOqDP3fe7+wjwOLBpTJtNwGMe2Qm0m9niBH3/HPgMoKfjiIhUSJIgWAocznvfH29L0qZoXzPbCBxx9xdLfbiZbTazXjPrHRgYSDBcERGZiCRBYAW2jf0NvlibgtvNrBV4APj8eB/u7tvdvdvduzs7ix7iEhGRSUpysrgfWJ73fhlwNGGbxiLbrwRWAS9G55lZBuwxs3Xu/uZECpiMri3/fv71ga0fm+6PExGZ1ZLMCHYBq81slZk1AncAPWPa9AB3WeRmYNDdjxXr6+4vu/sid+9y9y6iIFk7EyEgIiIXG3dG4O5pM7sPeJpoCeij7r7XzO6N928DniRaOtpHtHz0nlJ9p6USERGZlETXEbj7k0T/2Odv25b32oFPJ+1boE1XknGIiEj56cpiEZHAKQhERAKnIBARCZyCQEQkcAoCEZHAKQhERAKXaPloLci/mlhERC7QjEBEJHAKAhGRwCkIREQCpyAQEQmcgkBEJHAKAhGRwCkIREQCpyAQEQmcgkBEJHAKAhGRwCkIREQCpyAQEQmcgkBEJHAKAhGRwCkIREQCpyAQEQmcgkBEJHAKAhGRwCkIREQCpyAQEQlcMA+vLyb/ofYHtn6sgiMREakMzQhERAKnIBARCZyCQEQkcAoCEZHAKQhERAKnIBARCZyCQEQkcAoCEZHAKQhERAKnIBARCZyCQEQkcImCwMzWm9k+M+szsy0F9puZPRjvf8nM1o7X18y+aGavx+2fMLP28pQkIiITMW4QmFkKeAjYAKwB7jSzNWOabQBWx1+bgYcT9N0BfMDdrwW+B9w/5WpERGTCkswI1gF97r7f3UeAx4FNY9psAh7zyE6g3cwWl+rr7t9293TcfyewrAz1iIjIBCUJgqXA4bz3/fG2JG2S9AX4FeCpQh9uZpvNrNfMegcGBhIMV0REJiJJEFiBbZ6wzbh9zewBIA18tdCHu/t2d+929+7Ozs4EwxURkYlI8mCafmB53vtlwNGEbRpL9TWzu4GPAz/u7mPDRUREZkCSGcEuYLWZrTKzRuAOoGdMmx7grnj10M3AoLsfK9XXzNYDnwU2uvvZMtUjIiITNO6MwN3TZnYf8DSQAh51971mdm+8fxvwJHA70AecBe4p1Tf+0V8GmoAdZgaw093vLWdxIiIyPqumIzLd3d3e29s7qb75zyZOQs8vFpFaYWa73b272H5dWSwiEjgFgYhI4BQEIiKBUxCIiAROQSAiEjgFgYhI4BQEIiKBUxCIiAROQSAiEjgFgYhI4BQEIiKBUxCIiAROQSAiEjgFgYhI4JI8oSxIY29brdtSi0it0oxARCRwCgIRkcApCEREAqcgEBEJnIJARCRwCgIRkcBp+WhC+ctJtZRURGqJZgQiIoFTEIiIBE5BICISOAWBiEjgFAQiIoHTqqFJ0AoiEaklmhGIiAROQSAiEjgFgYhI4HSOYIp0vkBEqp1mBCIigVMQiIgEToeGykiHiUSkGikIpolCQUSqhYJgBigURGQ20zkCEZHAKQhERAKX6NCQma0HvgSkgEfcfeuY/Rbvvx04C3zK3feU6mtmC4B/BrqAA8AvuvvbUy9pdss/TFSMDh+JyEwaNwjMLAU8BNwG9AO7zKzH3V/Na7YBWB1/3QQ8DNw0Tt8twDPuvtXMtsTvP1u+0qqXwkJEZlKSGcE6oM/d9wOY2ePAJiA/CDYBj7m7AzvNrN3MFhP9tl+s7ybg1rj/V4DnUBAkliQsJiM/YHSSWyQMSYJgKXA4730/0W/947VZOk7fy939GIC7HzOzRYU+3Mw2A5vjt++Z2b4EYy6kAzgxyb6z0bTUY1+Y2PYyqaU/m1qqBWqrnlqqBSZWz8pSO5MEgRXY5gnbJOlbkrtvB7ZPpE8hZtbr7t1T/TmzRS3Vo1pmr1qqp5ZqgfLWk2TVUD+wPO/9MuBowjal+h6PDx8Rf38r+bBFRKRckgTBLmC1ma0ys0bgDqBnTJse4C6L3AwMxod9SvXtAe6OX98NfGuKtYiIyCSMe2jI3dNmdh/wNNES0Efdfa+Z3Rvv3wY8SbR0tI9o+eg9pfrGP3or8HUz+1XgEPALZa3sUlM+vDTL1FI9qmX2qqV6aqkWKGM9Fi30ERGRUOnKYhGRwCkIREQCF0QQmNl6M9tnZn3xVcyzjpk9amZvmdkredsWmNkOM/t+/H1+3r7743r2mdlP5W2/0cxejvc9GN/+Y6ZrWW5m/2Vmr5nZXjP7f9Vaj5k1m9l3zezFuJY/qNZa8saRMrPnzezfaqCWA/E4XjCz3hqop93M/sXMXo///twyI/W4e01/EZ2kfgN4H9AIvAisqfS4Cozzw8Ba4JW8bX8CbIlfbwG+EL9eE9fRBKyK60vF+74L3EJ0DcdTwIYK1LIYWBu/ngN8Lx5z1dUTf+5l8esG4DvAzdVYS15Nvwv8E/Bv1fz/s3gcB4COMduquZ6vAL8Wv24E2meinhkvtAL/YW8Bns57fz9wf6XHVWSsXVwcBPuAxfHrxcC+QjUQrcq6JW7zet72O4G/ngV1fYvoflNVXQ/QCuwhujq+KmshupbnGeCjXAiCqqwl/uwDXBoEVVkPMBf4AfEinpmsJ4RDQ8Vuf1ENLroNB5C7DUepW3r0F9heMWbWBdxA9Jt0VdYTH0p5geiixx3uXrW1AH8BfAbI5m2r1logulPBt81st0W3o4Hqred9wADwd/Ghu0fMrI0ZqCeEIJjybS5moWm7pUc5mdllwL8Cv+3up0s1LbBt1tTj7hl3v57ot+l1ZvaBEs1nbS1m9nHgLXffnbRLgW2zopY8H3L3tUR3QP60mX24RNvZXk890eHhh939BuAM0aGgYspWTwhBkOQWGbNVsdtwlLqlx7IC22ecmTUQhcBX3f0b8eaqrQfA3d8hukvueqqzlg8BG83sAPA48FEz+0eqsxYA3P1o/P0t4AmiuyVXaz39QH884wT4F6JgmPZ6QgiCJLfImK2K3YajB7jDzJrMbBXRcyC+G08b3zWzm+NVAndRgVt3xJ/9t8Br7v5nebuqrh4z6zSz9vh1C/ATwOtUYS3ufr+7L3P3LqK/B8+6+y9VYy0AZtZmZnNyr4GfBF6hSutx9zeBw2Z2dbzpx4lu2T/99VTiBE8FTsLcTrRy5Q3ggUqPp8gYvwYcA0aJEv1XgYVEJ/a+H39fkNf+gbiefeStCAC6if4yvAF8mTEnnmaolh8lmoq+BLwQf91ejfUA1wLPx7W8Anw+3l51tYyp61YunCyuylqIjqm/GH/tzf3drtZ64nFcD/TG/3/7JjB/JurRLSZERAIXwqEhEREpQUEgIhI4BYGISOAUBCIigVMQiIgETkEgIhI4BYGISOD+P9XGvhlecU5dAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# By simulation:\n",
    "\n",
    "# Resample from the estimated parametric distribution for X, and calculate Y = exp(X)\n",
    "Y = np.exp(norm.rvs(loc=mu_hat, scale=sigma, size=10000))\n",
    "\n",
    "# Recompute ranges for plot based on observed Y values\n",
    "z_values = np.linspace(0, max(Y), 100)\n",
    "f_values = posterior_density(z_values)\n",
    "\n",
    "fig, (ax1,ax2) = plt.subplots(2, 1, sharex='col')\n",
    "ax1.plot(z_values, f_values)\n",
    "ax2.hist(Y, density=True, bins=z_values);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(e)**\n",
    "\n",
    "Analytic solution:  $Y = g(X) = e^X$ and $X \\sim N(\\mu, 1)$.  Since $g$ is a monotonically increasing function, the quantiles of $Y$ are the exponentials of the quantiles of $X$; that is,\n",
    "\n",
    "$$F_Y^{-1}(q) = g(F_X^{-1}(q)) = e^{F_X^{-1}(q)}$$\n",
    "\n",
    "But $X$ follows a Normal distribution -- we can plug in the MLE for $\\mu$, $\\hat{\\mu} = \\overline{X}$, obtain the quantiles for $X$, and then obtain the quantiles for $Y$.\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "F_X^{-1}(q) &= \\Phi^{-1}(q) / \\sigma + \\mu \\\\\n",
    "F_Y^{-1}(q) &= e^{\\Phi^{-1}(q) / \\sigma + \\mu}\n",
    "\\end{align}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "95% posterior interval (analytic):  19.388, 977.081\n"
     ]
    }
   ],
   "source": [
    "from scipy.stats import norm\n",
    "\n",
    "z_025 = norm.ppf(0.025)\n",
    "z_975 = norm.ppf(0.975)\n",
    "\n",
    "posterior_interval_analytic = (np.exp(z_025 + mu_hat), np.exp(z_975 + mu_hat))\n",
    "print(\"95%% posterior interval (analytic):  %.3f, %.3f\" % posterior_interval_analytic)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, we can just sample from our simulation draws to get an approximation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "95% posterior interval (simulation):  19.021, 985.730\n"
     ]
    }
   ],
   "source": [
    "# Find percentile from simulated draws\n",
    "\n",
    "posterior_interval_simulation = (\n",
    "    np.quantile(Y, 0.025),\n",
    "    np.quantile(Y, 0.975)\n",
    ")\n",
    "\n",
    "print(\"95%% posterior interval (simulation):  %.3f, %.3f\" % posterior_interval_simulation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(f)**\n",
    "\n",
    "For the Bayesian interval estimate, we need to find $C = (a, b)$ such that\n",
    "\n",
    "$$ \\int_{-\\infty}^a f_Y(\\theta | X^n) d\\theta = \\int_b^\\infty f_Y(\\theta | X^n) d\\theta = \\frac{\\alpha}{2}$$\n",
    "\n",
    "or, using the cumulative density functions,\n",
    "\n",
    "$$ F_Y(a) = 1 - F_Y(b) = \\frac{\\alpha}{2}$$\n",
    "\n",
    "$$\\Phi(\\log a - \\mu) = 1 - \\Phi(\\log b - \\mu) = \\frac{\\alpha}{2}$$\n",
    "\n",
    "Solving for $a$ and $b$,\n",
    "\n",
    "$$\n",
    "a = e^{\\mu + \\Phi^{-1}(\\alpha / 2)}\n",
    "\\quad \\text{and} \\quad\n",
    "b = e^{\\mu + \\Phi^{-1}(1 - \\alpha / 2)}\n",
    "$$\n",
    "\n",
    "This is exactly the same calculation as the analytic solution for the posterior interval."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "95% confidence interval (analytic):  19.388, 977.081\n"
     ]
    }
   ],
   "source": [
    "from scipy.stats import norm\n",
    "\n",
    "z_025 = norm.ppf(0.025)\n",
    "z_975 = norm.ppf(0.975)\n",
    "\n",
    "confidence_interval_analytic = (np.exp(z_025 + mu_hat), np.exp(z_975 + mu_hat))\n",
    "print(\"95%% confidence interval (analytic):  %.3f, %.3f\" % confidence_interval_analytic)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise 12.11.3**.  Let $X_1, \\dots, X_n \\sim \\text{Uniform}(0, \\theta)$.  Let $f(\\theta) \\propto 1/\\theta$.  Find the posterior density."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Solution**.  The posterior density is proportional to the likelihood times the prior:\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "f(\\theta | X^n) &\\propto \\mathcal{L}_n(\\theta) f(\\theta) \\\\\n",
    "&= \\frac{1}{\\theta} \\prod_{i=1}^n f(X_i; \\theta)  \\\\\n",
    "&= \\frac{1}{\\theta} \\prod_{i=1}^n \\frac{I(X_i \\leq \\theta)}{\\theta} \\\\\n",
    "&= \\theta^{-(n+1)} I( \\max \\{ X_1, \\dots, X_n \\} \\leq \\theta \\} ) \\\\\n",
    "&= \\begin{cases}\n",
    "\\theta^{-(n+1)} & \\text{if } \\theta \\geq \\max \\{ X_1, \\dots, X_n \\} \\\\\n",
    "0 & \\text{otherwise}\n",
    "\\end{cases}\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "Now we just need to normalize the posterior density so it integrates to 1.  Let $m = \\max \\{ X_1, \\dots, X_n \\}$.  Then:\n",
    "\n",
    "$$ 1 = \\int_{-\\infty}^\\infty f(\\theta) d\\theta  = \\int_m^\\infty c \\theta^{-(n+1)} d\\theta = c m^{-n} / n$$\n",
    "\n",
    "Solving this we get $c = n m^n$, so the posterior density is:\n",
    "\n",
    "$$\n",
    "f(\\theta) = \\begin{cases}\n",
    "\\frac{n}{\\theta} \\left(\\frac{m}{\\theta}\\right)^n & \\text{if } \\theta \\geq m \\\\\n",
    "0 & \\text{otherwise}\n",
    "\\end{cases}\n",
    "$$\n",
    "\n",
    "where $m = \\max \\{ X_1, \\dots, X_n \\}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise 12.11.4**.  Suppose that 50 people are given a placebo and 50 are given a new treatment.  30 placebo patients show improvement while 40 treated patients show improvement.  Let $\\tau = p_2 - p_1$ where $p_2$ is the probability of improving under treatment and $p_1$ is the probability of improving under placebo.\n",
    "\n",
    "**(a)** Find the MLE of $\\tau$.  Find the standard error and 90% confidence interval using the delta method.\n",
    "\n",
    "**(b)** Find the standard error and 90% confidence interval using the parametric bootstrap.\n",
    "\n",
    "**(c)** Use the prior $f(p_1, p_2) = 1$.  Use simulation to find the posterior mean and posterior 90% interval for $\\tau$.\n",
    "\n",
    "**(d)** Let\n",
    "\n",
    "$$ \\psi = \\log \\left( \\left( \\frac{p_1}{1 - p_1} \\div \\frac{p2}{1 - p2} \\right) \\right) $$\n",
    "\n",
    "be the log-odds ratio.  Note that $\\psi = 0$  if $p_1 = p_2$.  Find the MLE of $\\psi$.  Use the delta method to find a 90% confidence interval for $\\psi$.\n",
    "\n",
    "**(e)** Use simulation to find the posterior mean and posterior 90% interval for $\\psi$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Solution**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(a)**\n",
    "\n",
    "We have two random variables, $X_1 \\sim \\text{Binomial}(n_1, p_1)$ and $X_2 \\sim \\text{Binomial}(n_2, p_2)$.  We are given $n_1 = n_2 = 50$, and the measurements $X_1 = 30$ and $X_2 = 40$.\n",
    "\n",
    "We wish to estimate $\\tau = g(p_1, p_2) = p_2 - p_1$.  The MLE is simply $\\hat{\\tau} = g(\\hat{p}_1, \\hat{p}_2) = \\hat{p}_2 - \\hat{p}_1 = X_2 / n_2 - X_1 / n_1$.  \n",
    "\n",
    "Using the multiparameter delta method, the gradient of $g$ is\n",
    "\n",
    "$$ \\nabla g = \n",
    "\\begin{pmatrix} \\partial g / \\partial p_1 \\\\ \\partial g / \\partial p_2 \\end{pmatrix} \n",
    "= \\begin{pmatrix} -1 \\\\ 1 \\end{pmatrix} \n",
    "$$\n",
    "\n",
    "The log likelihood is:\n",
    "\n",
    "$$ \n",
    "\\begin{align}\n",
    "\\ell(p_1, p_2) &= \\log f_{X_1}(X_1; p_1) + \\log f_{X_2}(X_2; p_2) \\\\\n",
    "& = \\log \\binom{n_1}{X_1} p_1^{X_1} (1 - p_1)^{n_1 - X_1}\n",
    "+ \\log \\binom{n_2}{X_2} p_2^{X_2} (1 - p_2)^{n_2 - X_2} \\\\\n",
    "&= \\log \\binom{n_1}{X_1} + X_1 \\log p_1 + (n_1 - X_1) \\log (1 - p_1)\n",
    "+ \\log \\binom{n_2}{X_2} + X_2 \\log p_2 + (n_2 - X_2) \\log (1 - p_2) \\\\\n",
    "&= X_1 \\log p_1 + (n_1 - X_1) \\log (1 - p_1) + X_2 \\log p_2 + (n_2 - X_2) \\log (1 - p_2) + C\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "where $C$ does not depend on $p_1$ or $p_2$.\n",
    "\n",
    "The partial derivatives of the log likelihood are:\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "\\frac{\\partial^2 \\ell}{\\partial p_1^2} & = - \\frac{X_1}{p_1^2} - \\frac{n_1 - X_1}{(1 - p_1)^2} \\\\\n",
    "\\frac{\\partial^2 \\ell}{\\partial p_2^2} & = - \\frac{X_2}{p_2^2} - \\frac{n_2 - X_2}{(1 - p_2)^2} \\\\\n",
    "\\frac{\\partial^2 \\ell}{\\partial p_1 \\partial p_2} & = 0\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "The Fisher Information Matrix is\n",
    "\n",
    "$$ I_n(p_1, p_2) = -\\begin{pmatrix}\n",
    "\\mathbb{E}\\left[ \\frac{\\partial^2 \\ell}{\\partial p_1^2} \\right]\n",
    "& \\mathbb{E}\\left[ \\frac{\\partial^2 \\ell}{\\partial p_1 \\partial p_2} \\right]  \\\\\n",
    "\\mathbb{E}\\left[ \\frac{\\partial^2 \\ell}{\\partial p_1 \\partial p_2} \\right]\n",
    "& \\mathbb{E}\\left[ \\frac{\\partial^2 \\ell}{\\partial p_2^2} \\right]\n",
    "\\end{pmatrix} = \\begin{pmatrix}\n",
    "\\frac{n_1}{p_1(1 - p_1)} & 0 \\\\\n",
    "0 & \\frac{n_2}{p_2(1 - p_2)}\n",
    "\\end{pmatrix}$$\n",
    "\n",
    "and its inverse is \n",
    "\n",
    "$$J_n(p_1, p_2) = I_n^{-1}(p_1, p_2) = \\begin{pmatrix} \n",
    "\\frac{p_1(1 - p_1)}{n_1} & 0 \\\\\n",
    "0 & \\frac{p_2(1 - p_2)}{n_2}\n",
    "\\end{pmatrix}$$\n",
    "\n",
    "Then,\n",
    "\n",
    "$$\\hat{\\text{se}}(\\hat{\\tau}) = \\sqrt{(\\hat{\\nabla}g)^T \\hat{J}_n (\\hat{\\nabla}g)}\n",
    "= \\sqrt{\\frac{\\hat{p}_1(1 - \\hat{p}_1)}{n_1} + \\frac{\\hat{p}_2(1 - \\hat{p}_2)}{n_2}}\n",
    "$$\n",
    "\n",
    "And we get the confidence interval:\n",
    "\n",
    "$$ \\hat{\\tau} \\pm z_{1 - \\alpha/2}\\hat{\\text{se}}(\\hat{\\tau})$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Estimated difference of means: \t\t 0.200\n",
      "Estimated 90% confidence interval:\t 0.053, 0.347\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from scipy.stats import norm\n",
    "\n",
    "n1 = 50\n",
    "n2 = 50\n",
    "\n",
    "X1 = 30\n",
    "X2 = 40\n",
    "\n",
    "p1_hat = X1 / n1\n",
    "p2_hat = X2 / n2\n",
    "\n",
    "tau_hat = p2_hat - p1_hat\n",
    "se_hat = np.sqrt((p1_hat * (1 - p1_hat) / n1) + (p2_hat * (1 - p2_hat) / n2))\n",
    "\n",
    "z_90 = norm.ppf(0.95)\n",
    "\n",
    "confidence_interval = (tau_hat - z_90 * se_hat, tau_hat + z_90 * se_hat)\n",
    "\n",
    "print('Estimated difference of means: \\t\\t %.3f' % tau_hat)\n",
    "print('Estimated 90%% confidence interval:\\t %.3f, %.3f' % confidence_interval)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(b)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Estimated difference of means: \t\t 0.200\n",
      "Estimated 90% confidence interval:\t 0.060, 0.340\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from scipy.stats import binom\n",
    "\n",
    "B = 100000\n",
    "\n",
    "n1 = 50\n",
    "n2 = 50\n",
    "\n",
    "X1 = 30\n",
    "X2 = 40\n",
    "\n",
    "p1_hat = X1 / n1\n",
    "p2_hat = X2 / n2\n",
    "\n",
    "XX1 = binom.rvs(n1, p1_hat, size=B)\n",
    "XX2 = binom.rvs(n2, p2_hat, size=B)\n",
    "tau_boot = XX2 / n2 - XX1 / n1\n",
    "\n",
    "tau_boot_hat = tau_boot.mean()\n",
    "q_05 = np.quantile(tau_boot, 0.05)\n",
    "q_95 = np.quantile(tau_boot, 0.95)\n",
    "\n",
    "boot_confidence_interval = (q_05, q_95)\n",
    "\n",
    "print('Estimated difference of means: \\t\\t %.3f' % tau_boot_hat)\n",
    "print('Estimated 90%% confidence interval:\\t %.3f, %.3f' % boot_confidence_interval)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(c)**  The posterior density is proportional to the likelihood times the prior:\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "f((p_1, p_2) | X^2) &\\propto \\mathcal{L}(p_1, p_2 | X^2) f(p_1, p_2) \\\\\n",
    "&= f_{X_1}(X_1 | p_1) f_{X_2}(X_2 | p_1) \\cdot 1 \\\\\n",
    "&= \\binom{n_1}{X_1} p_1^{X_1} (1 - p_1)^{n_1 - X_1} \\binom{n_2}{X_2} p_2^{X_2} (1 - p_2)^{n_2 - X_2} \\\\\n",
    "&\\propto p_1^{X_1} (1 - p_1)^{n_1 - X_1} p_2^{X_2} (1 - p_2)^{n_2 - X_2} \\\\\n",
    "&\\propto f(p_1 | X_1) f(p_2 | X_2)\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "where \n",
    "\n",
    "$$p_1 | X_1 \\sim \\text{Beta}(X_1 + 1, n_1 - X_1 + 1) \n",
    "\\quad \\text{and} \\quad\n",
    "p_2 | X_2 \\sim \\text{Beta}(X_2 + 1, n_2 - X_2 + 1)$$\n",
    "\n",
    "We can now sample from the Beta distributions to sample $p_1$, $p_2$, and for each pair of samples, compute the sample $\\tau = p_2 - p_1$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Estimated posterior mean: \t\t 0.193\n",
      "Estimated 90% confidence interval:\t 0.047, 0.335\n"
     ]
    }
   ],
   "source": [
    "from scipy.stats import beta\n",
    "\n",
    "B = 100000\n",
    "\n",
    "n1 = 50\n",
    "n2 = 50\n",
    "\n",
    "X1 = 30\n",
    "X2 = 40\n",
    "\n",
    "p1_boot = beta.rvs(X1 + 1, n1 - X1 + 1, size=B)\n",
    "p2_boot = beta.rvs(X2 + 1, n2 - X2 + 1, size=B)\n",
    "tau_boot = p2_boot - p1_boot\n",
    "\n",
    "tau_boot_hat = tau_boot.mean()\n",
    "q_05 = np.quantile(tau_boot, 0.05)\n",
    "q_95 = np.quantile(tau_boot, 0.95)\n",
    "\n",
    "boot_confidence_interval = (q_05, q_95)\n",
    "\n",
    "print('Estimated posterior mean: \\t\\t %.3f' % tau_boot_hat)\n",
    "print('Estimated 90%% confidence interval:\\t %.3f, %.3f' % boot_confidence_interval)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(d)**\n",
    "\n",
    "Let $\\psi = h(p_1, p_2) = \\log p_1 - \\log (1 - p_1) - \\log p_2 + \\log (1 - p_2)$.  The MLE is just $\\hat{\\psi} = h(\\hat{p_1}, \\hat{p_2})$.\n",
    "\n",
    "Using the multiparameter delta method, the gradient of $h$ is:\n",
    "\n",
    "$$\\nabla h = \n",
    "\\begin{pmatrix}\n",
    "\\frac{\\partial h}{\\partial p_1} \\\\ \n",
    "\\frac{\\partial h}{\\partial p_2} \\end{pmatrix}\n",
    "= \\begin{pmatrix}\n",
    "\\frac{1}{p_1(1 - p_1)} \\\\\n",
    "-\\frac{1}{p_2(1 - p_2)}\n",
    "\\end{pmatrix}\n",
    "$$\n",
    "\n",
    "The inverse Fisher Information Matrix is still:\n",
    "\n",
    "$$J_n(p_1, p_2) = \\begin{pmatrix}\n",
    "\\frac{p_1(1 - p_1)}{n_1} & 0 \\\\\n",
    "0 & \\frac{p_2(1 - p_2)}{n_2}\n",
    "\\end{pmatrix}$$\n",
    "\n",
    "and so the estimated standard error is:\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "\\hat{\\text{se}}(\\hat{\\psi}) &= \\sqrt{(\\hat{\\nabla}h)^T \\hat{J}_n (\\hat{\\nabla}h)} \\\\\n",
    "&= \n",
    "\\sqrt{\\begin{pmatrix}\n",
    "\\frac{1}{\\hat{p}_1(1 - \\hat{p}_1)} &\n",
    "-\\frac{1}{\\hat{p}_2(1 - \\hat{p}_2)}\n",
    "\\end{pmatrix}\n",
    "\\begin{pmatrix}\n",
    "\\frac{\\hat{p}_1(1 - \\hat{p}_1)}{n_1} & 0 \\\\\n",
    "0 & \\frac{\\hat{p}_2(1 - \\hat{p}_2)}{n_2}\n",
    "\\end{pmatrix}\n",
    "\\begin{pmatrix}\n",
    "\\frac{1}{\\hat{p}_1(1 - \\hat{p}_1)} \\\\\n",
    "-\\frac{1}{\\hat{p}_2(1 - \\hat{p}_2)}\n",
    "\\end{pmatrix}} \\\\\n",
    "&= \\sqrt{\\frac{1}{n_1 \\hat{p}_1(1 - \\hat{p}_1)} + \\frac{1}{n_2 \\hat{p}_2(1 - \\hat{p}_2)}}\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "and we get the confidence interval:\n",
    "\n",
    "$$\\hat{\\psi} \\pm z_{1 - \\alpha/2} \\hat{\\text{se}}(\\hat{\\psi})$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Estimated log-odds ratio: \t\t -0.981\n",
      "Estimated 90% confidence interval:\t -1.732, -0.230\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from scipy.stats import norm\n",
    "\n",
    "n1 = 50\n",
    "n2 = 50\n",
    "\n",
    "X1 = 30\n",
    "X2 = 40\n",
    "\n",
    "p1_hat = X1 / n1\n",
    "p2_hat = X2 / n2\n",
    "\n",
    "psi_hat = np.log((p1_hat / (1 - p1_hat)) / (p2_hat / (1 - p2_hat)))\n",
    "se_hat = np.sqrt(1/(n1 * p1_hat * (1 - p1_hat)) + 1/(n2 * p2_hat * (1 - p2_hat)))\n",
    "\n",
    "z_90 = norm.ppf(0.95)\n",
    "\n",
    "confidence_interval = (psi_hat - z_90 * se_hat, psi_hat + z_90 * se_hat)\n",
    "\n",
    "print('Estimated log-odds ratio: \\t\\t %.3f' % psi_hat)\n",
    "print('Estimated 90%% confidence interval:\\t %.3f, %.3f' % confidence_interval)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(e)**  The probability distributions for $p_1 | X_1$ and $p_2 | X_2$ are still the same -- we are just computing $\\psi = h(p_1, p_2)$ for each simulation sample now, rather than $\\tau = g(p_1, p_2)$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Estimated posterior mean: \t\t -0.953\n",
      "Estimated 90% confidence interval:\t -1.699, -0.224\n"
     ]
    }
   ],
   "source": [
    "from scipy.stats import beta\n",
    "\n",
    "B = 100000\n",
    "\n",
    "n1 = 50\n",
    "n2 = 50\n",
    "\n",
    "X1 = 30\n",
    "X2 = 40\n",
    "\n",
    "p1_boot = beta.rvs(X1 + 1, n1 - X1 + 1, size=B)\n",
    "p2_boot = beta.rvs(X2 + 1, n2 - X2 + 1, size=B)\n",
    "psi_boot = np.log((p1_boot / (1 - p1_boot)) / (p2_boot / (1 - p2_boot)))\n",
    "\n",
    "psi_boot_hat = psi_boot.mean()\n",
    "q_05 = np.quantile(psi_boot, 0.05)\n",
    "q_95 = np.quantile(psi_boot, 0.95)\n",
    "\n",
    "boot_confidence_interval = (q_05, q_95)\n",
    "\n",
    "print('Estimated posterior mean: \\t\\t %.3f' % psi_boot_hat)\n",
    "print('Estimated 90%% confidence interval:\\t %.3f, %.3f' % boot_confidence_interval)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise 12.11.5**.  Consider the $\\text{Bernoulli}(p)$ observations\n",
    "\n",
    "$$ 0\\; 1\\; 0\\; 1\\; 0\\; 0\\; 0\\; 0\\; 0\\; 0 $$\n",
    "\n",
    "Plot the posterior for $p$ using these priors: $\\text{Beta}(1/2, 1/2)$, $\\text{Beta}(1, 1)$, $\\text{Beta}(10, 10)$, $\\text{Beta}(100, 100)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Solution**.\n",
    "\n",
    "The observations include $n = 10$ samples, of which $k = 2$ samples with observed value 1.  Assume a prior of the form $\\text{Beta}(\\alpha, \\beta)$.\n",
    "\n",
    "The posterior is proportional to the likelihood times the prior.\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "f(p | X^n) & \\propto \\mathcal{L}(p | x^n) f(p) \\\\\n",
    "&= \\binom{n}{k} p^k (1 - p)^{n-k} \\frac{p^{\\alpha - 1}(1-p)^{\\beta - 1}}{B(\\alpha, \\beta)}\\\\\n",
    "&\\propto p^{k + \\alpha - 1} (1 - p)^{n - k + \\beta - 1}\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "This density is proportional to a Beta density function in p, so\n",
    "\n",
    "$$p | X^n \\sim \\text{Beta}(k + \\alpha - 1, n - k + \\beta - 1)$$\n",
    "\n",
    "Now, plotting the posterior becomes just a matter of plotting these beta distributions.\n",
    "\n",
    "When $\\alpha$ and $\\beta$ are integers, we can also reinterpret this posterior as equivalent to starting with a flat prior and observing extra $\\alpha + \\beta$ events, with $\\alpha$ extra events producing outcome 1 and $\\beta$ extra events producing outcome 0.  This is shown on the plots below -- the larger the number of \"extra observations\", the less impact the actual observations make on shifting the belief about the true parameter value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAr8AAAHUCAYAAAAzwkDqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzde3iU9Zn/8feTmQkJmPOJYIBwiqGSkGCo0KDIguABo7VQWmKBxVq1Xa1WWcS2sqXtiktLu5ZiSxWlNqVb+xOxtmIUaPFARZCoKIcAhpAASSacDyEzk+f3RzJDICGZZCaZJPN5XReXZPIc7uTC8OE73+e+DdM0EREREREJBiGBLkBEREREpLMo/IqIiIhI0FD4FREREZGgofArIiIiIkFD4VdEREREgobCr4iIiIgEjVbDr2EYKw3DqDQMY0ej15YYhrHLMIyPDcNYYxhGdMeWKSIiIiLiO29Wfl8AbrrktTeBEaZpZgJ7gAV+rktERERExO+srR1gmuYmwzBSL3mtsNGH/wKmeXOz+Ph4MzU1tdXjRERERER8sW3bNrtpmgmXvt5q+PXCXOD/vDkwNTWVrVu3+uGWIiIiIiKXZxjGgeZe9+mBN8Mwvg84gYIWjvmWYRhbDcPYWlVV5cvtRERERER80u7waxjGbGAqkG+apnm540zTXGGaZo5pmjkJCU1WnkVEREREOk27tj0YhnETMB8Yb5rmWf+WJCIiIiLSMVoNv4ZhrAZuAOINwygDFlLf3aEX8KZhGAD/Mk3zvg6sU0RERHoAh8NBWVkZNTU1gS5FeoiwsDBSUlKw2WxeHe9Nt4evN/Pyc20tTERERKSsrIyIiAhSU1NpWEATaTfTNKmurqasrIxBgwZ5dY4mvImIiEinqampIS4uTsFX/MIwDOLi4tr0ToLCr4iIiHQqBV/xp7b+eVL4FREREWnGE088wVtvvdVp97NYLGRlZTFy5EhGjRrFe++91+Lxx48fZ/ny5V5d+9y5c4wfPx6XywXAqlWrGDZsGMOGDWPVqlXNnvPCCy+QkJBAVlYWWVlZPPvss63eZ9u2bWRkZDB06FAefPBBmmsIVlJSQnh4uOe699134bGxSZMmcezYMa++pvbyx5ALERERkR7F5XKxaNGiNp9jsVjafc/w8HCKiooAeOONN1iwYAH//Oc/L3u8O/x++9vfbvXaK1eu5M4778RisXD06FF+9KMfsXXrVgzD4JprriEvL4+YmJgm582YMYNly5Z5/TXcf//9rFixgjFjxnDLLbewbt06br755ibHDRkyxPO1NvaNb3yD5cuX8/3vf9/re7aVVn5FREQkaJSUlJCens7s2bPJzMxk2rRpnD1b37U1NTWVRYsWMW7cOF566SXmzJnDX/7yFwDWr19PdnY2GRkZzJ07l/Pnzzd7jr+cPHnyojC6ZMkSRo8eTWZmJgsXLgTgscceY9++fWRlZTFv3jxOnz7NxIkTGTVqFBkZGaxdu9ZzfkFBAbfffjtQH6xvvPFGYmNjiYmJ4cYbb2TdunU+13z48GFOnjzJ2LFjMQyDWbNm8corr7TpGnl5eaxevdrnWlqilV8REREJiB/99VM+O3TSr9f8Qr9IFt52dYvH7N69m+eee47c3Fzmzp3L8uXLefTRR4H6tlnvvPMOgCcQ1tTUMGfOHNavX09aWhqzZs3imWee4aGHHmpyTmMFBQUsWbKkyetDhw71hOrGzp07R1ZWFjU1NRw+fJgNGzYAUFhYSHFxMVu2bME0TfLy8ti0aROLFy9mx44dnhVUp9PJmjVriIyMxG63M2bMGPLy8nA4HOzfv5/U1FQAysvL6d+/v+e+KSkplJeXN/u9+n//7/+xadMm0tLS+MUvfnHReZcqLy8nJSXFq+t+/vnnZGdnExkZyU9+8hOuu+46AGJiYjh//jzV1dXExcVd9l6+0MqviIiIBJX+/fuTm5sLwF133XVRcJ0xY0aT43fv3s2gQYNIS0sDYPbs2WzatKnFcwDy8/MpKipq8qu54AsXtj3s2rWLdevWMWvWLEzTpLCwkMLCQrKzsxk1ahS7du2iuLi4yfmmafL444+TmZnJpEmTKC8vp6KiArvdTnR09EXHXaq5h8Zuu+02SkpK+Pjjj5k0aRKzZ89utu62Xjc5OZnS0lK2b9/O0qVLmTlzJidPXvhHUGJiIocOHWrxXr7Qyq+IiIgERGsrtB3l0kDW+OM+ffo0Ob65UNdYc+dA21d+Gxs7dix2u52qqipM02TBggXce++9Fx1TUlLS5H5VVVVs27YNm81GamoqNTU1REVFXdQKLCUlhX/84x+ej8vKyrjhhhua1NB45fWee+5h/vz5LdackpJCWVnZRdft169fk+N69epFr169ALjmmmsYMmQIe/bsIScnB6hfaQ8PD2/xXr7Qyq+IiIgEldLSUjZv3gzA6tWrGTduXIvHp6enU1JSwt69ewF48cUXGT9+fKv3aevKb2O7du3C5XIRFxfHlClTWLlyJadPnwbqtxdUVlYSERHBqVOnPOecOHGCxMREbDYbGzdu5MCBA0D9VgKXy+UJwFOmTKGwsJBjx45x7NgxCgsLmTJlSpMaDh8+7Pn9q6++yvDhwy/6nlwqOTmZiIgI/vWvf2GaJr///e89+4wbq6qq8nSd2L9/P8XFxQwePBio/4fGkSNHPFs0OoJWfkVERCSoDB8+nFWrVnHvvfcybNgw7r///haPDwsL4/nnn2f69Ok4nU5Gjx59UXsuf3Hv+YX6ELhq1SosFguTJ09m586djB07FoArrriCP/zhDwwZMoTc3FxGjBjBzTffzPz587ntttvIyckhKyvrooA6efJk3nnnHSZNmkRsbCw//OEPGT16NFDf0i02Ntbz+5ycHPLy8nj66ad59dVXsVqtxMbG8sILLwBgt9svuxr+zDPPMGfOHM6dO8fNN9/s6fTw6quvsnXrVhYtWsSmTZt44oknsFqtWCwWfvOb33juv23bNsaMGYPV2nER1WhtKd+fcnJyzK1bt3ba/URERKRr2blz50UriJ2tpKSEqVOnsmPHjoDVEAju/bUvvviiz9d67bXX2L9/Pw8++KAfKrvYd7/7XfLy8pg4cWKbzmvuz5VhGNtM08y59Fit/IqIBJBZVweAEaJdaCLScbKzs5kwYYLPvYgBpk6d6qeqmhoxYkSbg29bKfyKiHQy1+nTnHqjkDPvvsuZ997DlpJC6kt/1shXkU6QmpoadKu+bnPnzg10Ca265557OvweCr8iIp2s/Hvf48ymt7EkxNNr2DDOfvABZ9/fQp8x1wa6NBGRHk/vs4mIdCJHZSVn3n6HuG/ezbBNm+j/uxVYoqI49sc/Bro0EZGgoPArItKJTr1RCKZJ1B13YBgGIWFhRE37CqfWr8dx5EigyxMR6fEUfkVEOtHJ11+nV1oavYYO9bwW8/WvQ10dx//85wBWJiISHBR+RUQ6iePwYc59+CGRt9xy0euhKSlccf31HPvzS5i1tQGqTkQu9cQTT/DWW2912v0sFgtZWVmMHDmSUaNG8d5777V4/PHjx1m+fLlX1z537hzjx4/3DJe46aabiI6O9rpzw6ZNmxg1ahRWq/WiIR1VVVXcdNNNXl2jq1D4FRHpJCdfXwdA5C03N/lcTP5MXHY7J998s7PLEpFmuFwuFi1axKRJk9p0ji/Cw8MpKirio48+4sknn2TBggUtHt+W8Lty5UruvPNOT5uzefPmtann74ABA3jhhReYOXPmRa8nJCSQnJzMu+++6/W1Ak3hV0Skk5z8+98JGzGC0AEDmnyuz7hx2AYM4NgfVwegMpHgUVJSQnp6OrNnzyYzM5Np06Zx9uxZoL4N2qJFixg3bhwvvfQSc+bM8axyrl+/nuzsbDIyMpg7dy7nz59v9hx/OXnyJDExMZ6PlyxZwujRo8nMzGThwoUAPPbYY+zbt4+srCzmzZvH6dOnmThxIqNGjSIjI4O1a9d6zi8oKLho1PDEiROJiIjwup7U1FQyMzMJaaYn+R133EFBQUF7vsyAUKszEZFOUFtaSs2OHSTOm9fs542QECJvvYXq366g7uxZQnr37uQKRQLg9cfgyCf+vWbfDLh5cYuH7N69m+eee47c3Fzmzp3L8uXLefTRR4H6UcbvvPMOAOvW1b9bU1NTw5w5c1i/fj1paWnMmjWLZ555hoceeqjJOY0VFBSwZMmSJq8PHTr0oq0Dbu7xxjU1NRw+fJgNGzYAUFhYSHFxMVu2bME0TfLy8ti0aROLFy9mx44dFBUVAeB0OlmzZg2RkZHY7XbGjBlDXl4eDoeD/fv3k5qa6uU3sW1ycnL4wQ9+0CHX7ggKvyIineDk318HIPLmy++NC8/IhLo6anbtoveoUZ1VmkjQ6d+/P7m5uQDcddddPP30057wO2PGjCbH7969m0GDBpGWlgbA7Nmz+fWvf+0Jv82dA5Cfn09+fr7Xdbm3PQBs3ryZWbNmsWPHDgoLCyksLCQ7OxuA06dPU1xczIBL3kUyTZPHH3+cTZs2ERISQnl5ORUVFdTV1REdHe11HW2VmJjIoUOHOuz6/qbwKyLSCU5v3EjYyExs/fpd9piwq68GoGbHDoVfCQ6trNB2lEunKTb+uE+fPk2ON02zxes1dw60feW3sbFjx2K326mqqsI0TRYsWMC999570TElJSVN7ldVVcW2bduw2WykpqZSU1NDVFQUNTU1Ld7PFzU1NYSHh3fY9f1Ne35FRDqY6XJRs3s3vbOyWjzOlpSINTGRc0E6elWks5SWlrJ582YAVq9ezbhx41o8Pj09nZKSEvbu3QvAiy++yPjx41u9T35+PkVFRU1+tRZ8AXbt2oXL5SIuLo4pU6awcuVKTp8+DUB5eTmVlZVERERw6tQpzzknTpwgMTERm83Gxo0bOXDgAAAxMTG4XC6vAvCCBQtYs2ZNq8c1tmfPHkaMGNGmcwJJ4VdEpIPVHjiAWVNDr6vSWz02bMQIanZ82glViQSv4cOHs2rVKjIzMzl69Cj3339/i8eHhYXx/PPPM336dDIyMggJCeG+++7ze13uPb9ZWVnMmDGDVatWYbFYmDx5MjNnzmTs2LFkZGQwbdo0Tp06RVxcHLm5uYwYMYJ58+aRn5/P1q1bycnJoaCggPT0Cz9zJk+efNG+5Ouuu47p06ezfv16UlJSeOONNwD45JNP6Nu3b5PaPvjgA1JSUnjppZe49957ubrhnSqAjRs3cuutt/r9+9FRjNaW8v0pJyfH3Lp1a6fdT0SkKzj5979T/r1HGLTmZcKGD2/x2Krly7H/ahlpH2zBcsUVnVShSOfZuXMnw1v5/6AjlZSUMHXqVHYE2Tss27dvZ+nSpa22N5syZYonCHvr+uuvZ+3atRd1p+hszf25Mgxjm2maOZceq5VfEZEOVrNrN1ithA4Z0uqx4SNGgGlS89lnnVCZiASL7OxsJkyY0Gov4rYG36qqKr73ve8FNPi2lcKviEgHq9m1k16DBxMSGtrqsZ6H3j4JrlUpkc6SmpoadKu+bnPnzvUMufCXhIQE7rjjDr9es6Mp/IqIdLDzu3YTNrz1/b4A1rg4rP2Sqfk0OP9yFhHpaAq/IiIdyHn0KM7KSq8ednMLv3oE5/TQm4hIh1D4FRHpQOd37QIgLP0qr88JGzECR2kprhMnOqosEZGgpfArItKBanbtBqBXehtWfjPq+2XWfKrVXxERf1P4FRHpQDW7dmJNTMQaG+v1Oe6H3rT1QSSwnnjiCd56661Ou5/FYiErK4uRI0cyatQo3nvvvRaPP378OMuXL/fq2ufOnWP8+PGebg833XQT0dHRTJ069aLjPv/8c6699lqGDRvGjBkzqK2tbfXabb3Wa6+9xsKFC72quyMo/IqIdKDzu3bTqw1bHgAsUVHYBgygJkifSBfpClwuF4sWLWLSpEltOscX4eHhFBUV8dFHH/Hkk0+yYMGCFo9vS/hduXIld955p6fbw7x585rt+Tt//nwefvhhiouLiYmJ4bnnnmv12m291q233sqrr77K2bNnvard3xR+RUQ6SF1tLef37ycsve0N/cNHXM25HZ90QFUiwa2kpIT09HRmz55NZmYm06ZN84Sw1NRUFi1axLhx43jppZeYM2eOZxTx+vXryc7OJiMjg7lz53L+/Plmz/GXkydPXtQ7d8mSJYwePZrMzEzPquljjz3Gvn37yMrKYt68eZw+fZqJEycyatQoMjIyWLt2ref8goICbr/9ds/HEydOJCIi4qJ7mqbJhg0bmDZtGgCzZ8/mlVdeabXWtl7LMAxuuOEGXnvttbZ8S/zGGpC7iogEgdq9e8HpbNPDbm690q7i5N9fp+7MGUL69OmA6kQC76ktT7Hr6C6/XjM9Np35X5zf4jG7d+/mueeeIzc3l7lz57J8+XIeffRRoH6UsXsM8Lp16wCoqalhzpw5rF+/nrS0NGbNmsUzzzzDQw891OScxgoKCliyZEmT14cOHeoJ1Y25xxvX1NRw+PBhNmzYAEBhYSHFxcVs2bIF0zTJy8tj06ZNLF68mB07dlBUVASA0+lkzZo1REZGYrfbGTNmDHl5eTgcDvbv309qamqL35fq6mqio6OxWuvjYUpKCuXl5S2e095r5eTk8Pbbb/PVr361Xdf3hcKviEgHac/Dbm6hAwcAUHvwIGHtOF9ELq9///7k5uYCcNddd/H00097wu+MGTOaHL97924GDRpEWloaUL+K+etf/9oTfps7ByA/P5/8/Hyv63JvewDYvHkzs2bNYseOHRQWFlJYWEh2djYAp0+fpri4mAEDBlx0vmmaPP7442zatImQkBDKy8upqKigrq6O6OjoVu9vmmaT1wzD8Lr+tlwrMTGRQ4cOtevavlL4FRHpIOd378IICyN04MA2n2tr+EuttrRU4Vd6rNZWaDvKpYGu8cd9mnmnpbkg11hz50DbV34bGzt2LHa7naqqKkzTZMGCBdx7770XHVNSUtLkflVVVWzbtg2bzUZqaio1NTVERUVRU1PT4v0A4uPjOX78OE6nE6vVSllZGf369Wv1vPZcq6amhvDw8HZd21fa8ysi0kFqdu6iV1oaRjvGiYY2hF9Haam/yxIJeqWlpWzevBmA1atXM27cuBaPT09Pp6SkhL179wLw4osvMn78+Fbvk5+fT1FRUZNfrQVfgF27duFyuYiLi2PKlCmsXLmS06dPA1BeXk5lZSURERGcOnXKc86JEydITEzEZrOxceNGDhw4AEBMTAwul6vVAGwYBhMmTPDUt2rVKs8+4S1btjBr1qxW6/bmWgB79uxhxIgRXl/PnxR+RUQ6SO3nn9NryJB2nWuJiMASG0vtAYVfEX8bPnw4q1atIjMzk6NHj3L//fe3eHxYWBjPP/8806dPJyMjg5CQEO677z6/1+Xe85uVlcWMGTNYtWoVFouFyZMnM3PmTMaOHUtGRgbTpk3j1KlTxMXFkZuby4gRI5g3bx75+fls3bqVnJwcCgoKSG/0rtHkyZMv2pd83XXXMX36dNavX09KSgpvvPEGAE899RRLly5l6NChVFdXc/fddwP1/2C43EptW68FsHHjRm699Va/fw+9YbS2lO9POTk55tatWzvtfiIigVJXU8PurGziH3yAhG9/u13XKPna1zF69WLgqhf8W5xIAO3cuZPhw9veAcVfSkpKmDp1KjuCrJXg9u3bWbp0abMtybwxb948vvGNb5CZmelzLRUVFcycOZP169f7fC235v5cGYaxzTTNnEuP1Z5fEZEO4Gh4qjm0f/92XyN04ADObPnAXyWJSBDLzs5mwoQJuFwuT6/ftmhu73J7lZaW8vOf/9xv12srhV8RkQ5Qe/AgALaUlHZfwzZgAM61r1JXU0NIWJi/ShMJaqmpqUG36us2d+7cQJcAwOjRowN6f+35FRHpAI6DZYCPK78D6rtEOMrK/FKTiIgo/IqIdAhH2UGM8HAscXHtvkbogPrgXKuODyIifqPwKyLSAWoPlhGacmW7G8TDxb1+RUTEPxR+RUQ6gKOsDFtK+7c8AFiiowmJjFSvXxERP1L4FRHxM9M0qS0rw9a//Q+7QX2T+NABA9TrVyRAnnjiCd56661Ou5/FYiErK4uRI0cyatQo3nvvvRaPP378OMuXL/fq2ufOnWP8+PG4XC4AbrrpJqKjo5k6depFx33++edce+21DBs2jBkzZlBbWwvU/1x78MEHGTp0KJmZmXz44Yet3nPZsmUMHToUwzCw2+2e11u61rp167jqqqsYOnQoixcv9rz+6KOPsmHDBq++1tYo/IqI+Jnr6FHMs2cJ9XHlF+onvWnbg0jnc7lcLFq0iEmTJrXpHF+Eh4dTVFTERx99xJNPPsmCBQtaPL4t4XflypXceeednjZn8+bNa7bn7/z583n44YcpLi4mJiaG5557DoDXX3+d4uJiiouLWbFiRauDQQByc3N56623GHjJiPfLXcvlcvGd73yH119/nc8++4zVq1fz2WefAfDAAw9cFIZ9ofArIuJnDnebMx9XfgFsAwfgKC/HbFh9ERHflJSUkJ6ezuzZs8nMzGTatGmcPXsWqG+DtmjRIsaNG8dLL73EnDlzPON5169fT3Z2NhkZGcydO5fz5883e46/nDx5kpiYGM/HS5YsYfTo0WRmZrJw4UIAHnvsMfbt20dWVhbz5s3j9OnTTJw4kVGjRpGRkcHatWs95xcUFFw0XnjixIlERERcdE/TNNmwYQPTpk0DYPbs2bzyyisArF27llmzZmEYBmPGjOH48eMcPny4xa8hOzub1NTUJq9f7lpbtmxh6NChDB48mNDQUL72ta95voaBAwdSXV3NkSNH2vBdbJ76/IqI+FmtH9qcuYUOGAh1dTgOHSK0mb9ERLqzI//935zfucuv1+w1PJ2+jz/e4jG7d+/mueeeIzc3l7lz57J8+XIeffRRoH6UsXsM8Lp16wCoqalhzpw5rF+/nrS0NGbNmsUzzzzDQw891OScxgoKCpodDjF06FBPqG7MPd64pqaGw4cPe97mLywspLi4mC1btmCaJnl5eWzatInFixezY8cOioqKAHA6naxZs4bIyEjsdjtjxowhLy8Ph8PB/v37mw2ijVVXVxMdHY3VWh8PU1JSKG8Y2FNeXk7/Rj/T3J9LTk5u8ZrNudy1mnv9/fff93w8atQo3n33Xb7yla+0+Z6NKfyKiPiZo6xh5ffKK32+VujACx0fFH5F/KN///7k5uYCcNddd/H00097wu+MGTOaHL97924GDRpEWloaUL8i+utf/9oTfps7ByA/P5/8/Hyv63JvewDYvHkzs2bNYseOHRQWFlJYWEh2djYAp0+fpri4mAENHWHcTNPk8ccfZ9OmTYSEhFBeXk5FRQV1dXVER0e3en/TNJu85u5Y09Ln2upy12rtHomJiRw6dKhd92xM4VdExM9qy8qwJMQTEh7u87VCPe3ODvp8LZGuprUV2o5yaWhr/HGfPn2aHN9cKGusuXOg7Su/jY0dOxa73U5VVRWmabJgwQLuvffei44pKSlpcr+qqiq2bduGzWYjNTWVmpoaoqKiqKmpafF+APHx8Rw/fhyn04nVaqWsrIx+/foB9auwBw9e+DnU+HNtdblr1dbWtniPmpoawv3wc1V7fkVE/MxxsMwvD7sBWOLiMHr3prb0gF+uJyJQWlrK5s2bAVi9ejXjxo1r8fj09HRKSkrYu3cvAC+++CLjx49v9T75+fkUFRU1+dVa8AXYtWsXLpeLuLg4pkyZwsqVKzl9+jRQv22gsrKSiIgITp065TnnxIkTJCYmYrPZ2LhxIwcO1P/ciImJweVytRqADcNgwoQJnvpWrVrl2Secl5fH73//e0zT5F//+hdRUVGeLQ8TJ070bI/wxuWuNXr0aIqLi/n888+pra3lT3/6E3l5eZ7z9uzZw4gRI7y+z+Uo/IqI+Flt2UG/POwGF9qdOdTuTMRvhg8fzqpVq8jMzOTo0aOtdi4ICwvj+eefZ/r06WRkZBASEsJ9993n97rce36zsrKYMWMGq1atwmKxMHnyZGbOnMnYsWPJyMhg2rRpnDp1iri4OHJzcxkxYgTz5s0jPz+frVu3kpOTQ0FBAenp6Z5rT548+aJ9yddddx3Tp09n/fr1pKSk8MYbbwDw1FNPsXTpUoYOHUp1dTV33303ALfccguDBw9m6NCh3HPPPZ4uE3V1dezdu5fY2NgmX8/TTz9NSkoKZWVlZGZm8s1vfrPFa1mtVpYtW8aUKVMYPnw4X/3qV7n66qsBcDgc7N27l5ycHJ+/z0ZrS/n+lJOTY27durXT7ici0tnM2lp2jcwi/v77SXjwAb9cs+zB73K+uJghr//dL9cTCaSdO3cyfPjwgN2/pKSEqVOnsmPHjoDVEAjbt29n6dKlzbY388WOHTtYuXIlS5cu9et1L7VmzRo+/PBDfvzjHzf7+eb+XBmGsc00zSZpWSu/IiJ+5Dh0CEwTmx86PbiFDhxAbVkZpo89REUkeGVnZzNhwgSfexFfasSIER0efKG+k8Ujjzzil2vpgTcRET/ytDlL8b3Tg5vtyivB4cBpt2NLSvLbdUWCUWpqatCt+rrNnTs30CW02/Tp0/12La38ioj4kaO8Pvz6c+XX2hB4nX5o7i4iEuwUfkVE/Kj24EEMmw1rYqLfrmlreKLacVjhV3qGznzeSHq+tv55ajX8Goax0jCMSsMwdjR6LdYwjDcNwyhu+G9MS9cQEQkWjoNl2FJSMEL8t7Zg69u3/tpHWh4lKtIdhIWFUV1drQAsfmGaJtXV1YSFhXl9jjd7fl8AlgG/b/TaY8B60zQXG4bxWMPH89tQq4hIj+TPNmduIVFRGGFhOI9U+PW6IoHgbn1VVVUV6FKkhwgLCyMlxfufu62GX9M0NxmGkXrJy7cDNzT8fhXwDxR+RURwlh8iPDPTr9c0DANb3744tOdXegCbzcagQYMCXYYEsfa+L5dkmuZhgIb/+m9zm4hIN1VXU4PrxAnPNgV/sib3xXlY2x5ERHzV4Q+8GYbxLcMwthqGsVVvcYhIT+asqN+WYE3yf/i1JfXFUaFtDyIivmpv+K0wDCMZoOG/lZc70DTNFaZp5pimmZOQkIE1ycoAACAASURBVNDO24mIdH2Ohj25tr7+78VrTe6Ls7IS0+n0+7VFRIJJe8Pvq8Dsht/PBtb6pxwRke7LWele+fV/+LUl9YW6Opx2u9+vLSISTLxpdbYa2AxcZRhGmWEYdwOLgRsNwygGbmz4WEQkqLkfSLMmdkD4TW5od6Z9vyIiPvGm28PXL/OpiX6uRUSkW3NWVBJyxRVYrujj92tb+9YPutCUNxER32jCm4iInzgrjmDtgP2+cGEfsUO9fkVEfKLwKyLiJ46KSmwdsOUBICQyEqN3b5ya8iYi4hOFXxERP3EeOYK1A3r8QqNBF4e17UFExBcKvyIifmA6nTjtdqxJHTfzx9Y3CUeFwq+IiC8UfkVE/MBpt0NdXX1Lsg5i7ZuMUyu/IiI+UfgVEfEDdxeGjl75dVZVYTocHXYPEZGeTuFXRMQPHBX1gy5tHbTnF6jfT2yaODUqXkSk3RR+RUT8wNmwF7ejHngDsCXX9/p1qNeviEi7KfyKiPiBo6ICIzQUS3R0h93DPTZZgy5ERNpP4VdExA+cRyqwJiVhGEaH3cOz8quH3kRE2k3hV0TED5wVFR36sBuAJSKCkD59tO1BRMQHCr8iIn7gqKjo0DZnbta+fbXtQUTEBwq/IiI+Mk2zYeW3Y0YbN2br21crvyIiPlD4FRHxkev4cczaWmx9Oz78WpP74jhyuMPvIyLSUyn8ioj4yFlRAYA1sRNWfpP64rJXY9bWdvi9RER6IoVfEREfubchdMrKb98kDboQEfGBwq+IiI+cDdPdOnLAhZs1IaH+ngq/IiLtovArIuIjZ8URCAnBGh/f4fdyh1+Hwq+ISLso/IqI+MhRUYE1Ph7Dau3we2nlV0TENwq/IiI+ck936wzWuDgICVH4FRFpJ4VfEREfOSsrOuVhNwDDYsESF4vLbu+U+4mI9DQKvyIiPnIcqeiUNmdu1oQEnJVa+RURaQ+FXxERH9TV1FB36pRnL25nsCYkaNuDiEg7KfyKiPjA2bD9QOFXRKR7UPgVEfGBO4RaEzq+zZmbNT4eZ3U1psvVafcUEekpFH5FRHzgWfnthB6/btaEBKirw3X0aKfdU0Skp1D4FRHxwYWV387d9tD43iIi4j2FXxERH7jsdggJwRIb22n3VPgVEWk/hV8RER84q+xYYmMxLJZOu6c1IbHh3gq/IiJtpfArIuIDZ1VVp255gAsP1yn8ioi0ncKviIgPnHZ7pz7sBhDSqxchUVEKvyIi7aDwKyLig0CEX6hf/VX4FRFpO4VfEZF2Muvq6sNvJ297AI04FhFpL4VfEZF2cp04AU5ngFZ+NeVNRKQ9FH5FRNopENPd3Nzh1zTNTr+3iEh3pvArItJOgRhw4WZNSMB0OKg7caLT7y0i0p0p/IqItJMrAKON3TyDLhpqEBER7yj8ioi0kzt4BmrlF9TrV0SkrRR+RUTayVlZhdG7NyF9+nT6vRV+RUTaR+FXRKSdAtXjFzTiWESkvRR+RUTaKVA9fgEsV/TB6N1bvX5FRNpI4VdEpJ2cVVUBW/kFTXkTEWkPhV8RkXYK5LYH0KALEZH2UPgVEWmHuvPnqTt5MmDbHkDhV0SkPRR+RUTawVnlbnOmlV8Rke5E4VdEpB1c9obpboHc9hCfQN2ZM9SdPRuwGkREuhuFXxGRdgjkgAs39foVEWk7hV8RkXZwB05LQFd+6+/trK4OWA0iIt2Nwq+ISDs4q+xgGFhjYwNWgzU+rr6WhlVoERFpncKviEg7OO12LHFxGFZrwGrwrPwq/IqIeE3hV0SkHQI94ALAEhMDhoHLrm0PIiLeUvgVEWmHQA+4ADCsViwxMdrzKyLSBgq/IiLt4LTbA9rpwc0aF4ezWtseRES8pfArItJGpmk2rPzGBboUrAnxuKoUfkVEvKXwKyLSRnUnToDDEfBtDwCWuHhtexARaQOFXxGRNnKHTUtc4MNv/bYHhV8REW8p/IqItJGzobtCl9j2EB+Hee4cdWfOBLoUEZFuQeFXRKSNXA0PmFnjAh9+Ler1KyLSJgq/IiJt5F75DeRoYzdrnEYci4i0hcKviEgbOaurwWLBEhUV6FI04lhEpI18Cr+GYTxsGManhmHsMAxjtWEYYf4qTESkq3IdrcYSG4MREvj1A0vD1guXVn5FRLzS7p/chmFcCTwI5JimOQKwAF/zV2EiIl2V017t2W4QaNbYWDAMnOr1KyLiFV+XLaxAuGEYVqA3cMj3kkREujZndXWXeNgNNOJYRKSt2h1+TdMsB34GlAKHgROmaRb6qzARka7K1UWmu7lpxLGIiPd82fYQA9wODAL6AX0Mw7irmeO+ZRjGVsMwtlZVVbW/UhGRLsA0TZzV1V1iwIWbJT5OI45FRLzky7aHScDnpmlWmabpAF4GvnTpQaZprjBNM8c0zZyEhAQfbiciEnh1Z85gnj+PNS420KV4WOMTtO1BRMRLvoTfUmCMYRi9DcMwgInATv+UJSLSNbkaWopZusieX9CIYxGRtvBlz+/7wF+AD4FPGq61wk91iYh0Sc6jRwG6TLcH0IhjEZG2sPpysmmaC4GFfqpFRKTLcw+T6EoPvDUecRzap0+AqxER6doC36FdRKQbcQ+T6FrbHjTiWETEWwq/IiJt4LTXB0xrbFd64E0jjkVEvKXwKyLSBs5qO5boaAyrT7vG/EojjkVEvKfwKyLSBq7qaixdaL8vaMSxiEhbKPyKiLSB017dpTo9gEYci4i0hcKviEgbOKursXahh93cNOJYRMQ7Cr8iIm3QFbc9QMOIY7tWfkVEWqPwKyLipbqaGurOnOly2x6gYcSxuj2IiLRK4VdExEueNmddcOVXI45FRLyj8Csi4iVXw55aSxfq8eumEcciIt5R+BUR8ZJ7ZdUa3/W2PVg05U1ExCsKvyIiXnLvqe2S3R4aArn2/YqItEzhV0TES+4JapYuGX414lhExBsKvyIiXnLaqwmJiCCkV69Al9KERhyLiHhH4VdExEvOo11zwAU0GnGsXr8iIi1S+BUR8ZLLXt0ltzxAoxHH2vYgItIihV8RES911dHGbhpxLCLSOoVfEREvOauru+SACzeNOBYRaZ3Cr4iIF8zaWupOnOiy2x4ArHHx6vMrItIKhV8RES84jx4F6gNmV2WNj9eeXxGRVij8ioh44cJ0ty688qsRxyIirVL4FRHxgmfARWzXDb8acSwi0jqFXxERL7j753b1lV9AvX5FRFqg8Csi4gV3C7Eu3eosvmHl114V4EpERLouhV8RES+47NUY4eGE9OkT6FIuSyOORURap/ArIuIF59GjXXrVFzTiWETEGwq/IiJecFXbscTFBrqMFhlWK5boaE15ExFpgcKviIgXnPbqLt3j1029fkVEWqbwKyLiBWd1dZff9gAacSwi0hqFXxGRVpguF65jx7B04TZnbhpxLCLSMoVfEZFWuI4fh7o6rF14wIWbNS5O2x5ERFqg8Csi0oruMODCzZoQrxHHIiItUPgVEWmFq6F7gqU77PnViGMRkRYp/IqItMIdJN0T1LoyjTgWEWmZwq+ISCs84bcbrPy6a9SIYxGR5in8ioi0wlVdDTYbIZGRgS6lVZaG1WmNOBYRaZ7Cr4hIK5z2aqyxsRiGEehSWqURxyIiLVP4FRFphbPa3i22PIBGHIuItEbhV0SkFS57dbcYcOGmEcciIpen8Csi0grn0aNY47p+pwc3jTgWEbk8hV8RkRaYpomruhprXGygS/GaRhyLiFyewq+ISAvqTp7EdDg8wyO6A2tcnMKviMhlKPyKiLTgwoCLbrTnNyEe8+xZjTgWEWmGwq+ISAvcD451l24PoBHHIiItUfgVEWmB6+hRACzdKPxqxLGIyOUp/IqItMAdILvTyq9nxLF6/YqINKHwKyLSAme1HUJCsMTEBLoUr3lGHKvXr4hIEwq/IiItcNmrscTEYFgsgS7FaxpxLCJyeQq/IiItcFZXd6stD6ARxyIiLVH4FRFpgau6Gks3GnDhZo2Pw6VuDyIiTSj8ioi0oH7lt/sMuHCzxMfjrNLKr4jIpRR+RURa0B23PYBGHIuIXI7Cr4jIZdSdOYN57hyWbjTdzU0jjkVEmqfwKyJyGc6GARfdc9tDnEYci4g0Q+FXROQyLow27o4PvCUAGnEsInIphV8Rkctwd0uwdMOVX404FhFpnsKviMhleEYbd9M9v6ARxyIil1L4FRG5DHdwtMZ2v20P7tVq9foVEbmYwq+IyGW4qo8SEhmJERoa6FLazBobUz/iWL1+RUQuovArInIZ3bXHL4Bhs2nEsYhIM3wKv4ZhRBuG8RfDMHYZhrHTMIyx/ipMRCTQnNX2bht+QSOORUSa4+vK7/8C60zTTAdGAjt9L0lEpGtw2auxxHe/Tg9ulrh4dXsQEblEu8OvYRiRwPXAcwCmadaapnncX4WJiARad972AGCNj/f0KhYRkXq+rPwOBqqA5w3D2G4YxrOGYfS59CDDML5lGMZWwzC2VlVV+XA7EZHOU1dbS92pU1i64YALN404FhFpypfwawVGAc+YppkNnAEeu/Qg0zRXmKaZY5pmTkJCgg+3ExHpPO69st1xtLGbRhyLiDTlS/gtA8pM03y/4eO/UB+GRUS6ve484MLNHdy1+isickG7w69pmkeAg4ZhXNXw0kTgM79UJSISYJ4BF915z29CQ/jVQ28iIh5WH89/ACgwDCMU2A/8u+8liYgEnqv6KEC37vagEcciIk35FH5N0ywCcvxUi4hIl+HeKtAdRxu7acSxiEhTmvAmItIMV7Udo3dvQnr3DnQp7WaNjQHQiGMRkUYUfkVEmuG0d+8ev9Aw4jgmRtseREQaUfgVEWlGdx9w4aYRxyIiF1P4FRFphqu6GksPCL8acSwicjGFXxGRZvSYld+4OI04FhFpROFXROQSptOJ69ixbj3gws0aH68hFyIijSj8iohcwnXsGJhmz9j24B5xfPZsoEsREekSFH5FRC7h6fEb130HXLhpxLGIyMUUfkVELnEh/HbfARdunhHH6vUrIgIo/IqINOFuDWbpESu/GnEsItKYwq+IyCXcrcF6wgNvGnEsInIxhV8RkUs4q+0YNhshERGBLsVnnhHH6vUrIgIo/IqINOGyV2OJj8cwjECX4jPPiGN7VaBLERHpEhR+RUQu4Tx6FGts93/YzU0jjkVELlD4FRG5hLPajqUH7Pd104hjEZELFH5FRC7hslf3iB6/bta4OPX5FRFpoPArItKIWVdXv+2hB0x3c7PGx+O0q9WZiAgo/IqIXKTu5ElwOrH0gAEXbhpxLCJygcKviEgjPWm0sZtGHIuIXKDwKyLSiHsMsHsscE/gHtahrQ8iIgq/IiIXcVbV98O1JiQEuBL/scY3rPwq/IqIKPyKiDTWE8OvRhyLiFyg8Csi0ojTbscIDe0Ro43dNOJYROQChV8RkUacVVVYExJ6xGhjN8NmwxIdjbNa2x5ERBR+RUQacdqrPHtkexJrQjwu7fkVEVH4FRFpzFlVhTWx5+z3ddOIYxGRegq/IiKNuKrsPephNzeNOBYRqafwKyLSoK62FteJE1h64raH+DhtexARQeFXRMTD1QPbnLlZ4uOp04hjERGFXxERN/cQiB75wJtGHIuIAAq/IiIeFwZcJAa4Ev/TiGMRkXoKvyIiDTwrvwk9b+XXEqfwKyICCr8iIh7OyiowDKwNQbEnscbX72PWiGMRCXYKvyIiDZx2O5bYWAyrNdCl+J1GHIuI1FP4FRFp4KzqmdPdQCOORUTcFH5FRBo4q6p6ZJszN4t6/YqIKPyKiLg57fYeu/IL9ft+te1BRIKdwq+ICGDW1dWH3x688qsRxyIiCr8iIgC4TpwAp7Nnh19texARUfgVEYGGNmf0zB6/bpY4jTgWEVH4FREBnHZ3+O3JK78acSwiovArIkKj0cY9+oE3TXkTEVH4FRGhUfjtwSu/7hHHmvImIsFM4VdEBHDZ7YT07k1Inz6BLqXDuEccu4O+iEgwUvgVEaE+EFp68MNu0LDtISQER2VloEsREQkYhV8REcBZ1bN7/AIYFgvW+HicFQq/IhK8FH5FRGgYbRzfs8MvgDUpCadWfkUkiCn8iohAj5/u5mZNTMRZURHoMkREAkbhV0SCXt25c9SdPh0U4deWlKiVXxEJagq/IhL0gqHHr5s1MRHXiRPU1dQEuhQRkYBQ+BWRoOce+hAMK7/WxCRA7c5EJHgp/IpI0HPvgbUmJga4ko7n/hq171dEgpU10AWISHAxTZO9x/ey7/g+Sk+VcvDUQc44zng+FxEawZVXXMmVEVeSFpPG0OihhBgd++90R0MQtCX1/PDr/hq171dEgpXCr4h0uFpXLRtKN7CpbBObD2/Gfs7u+VxCeAIRoREYGBiGwfHzxy/6fGxYLKP7jub6lOuZNGASvW29/V6fs6ISo1cvQqKi/H7trsa98utQr18RCVIKvyLSYQ6eOshLe17ileJXOHb+GNG9ohmbPJax/cZydfzVpFyR0myYrXHWUH66nB32Hbx/+H3eP/w+b5S8wX/b/pubB93MV9O+yvC44X6r01lRgTUpCcMw/HbNriokMhIjLEzbHkQkaCn8iojfVZ6tZHnRctbsXYOBwYT+E5h+1XSu7XstlhBLq+eHWcMYEj2EIdFDuH3o7ZimyYeVH/Jy8cu8tu81/rLnL0zoP4HvZH2Hq2Kv8rleR2UFtiDY7wtgGAZWtTsTkSCm8CsifnPOeY7fffw7XvzsRZymk5npM5lz9RyS+iT5dF3DMLgm6RquSbqG+V+cT8HOAl789EWm/XUaN6fezLzR80jo3f5ODc6KSsIzM32qsTuxJSTiqNTKr4gEJ4VfEfGLosoifvDuDzhw8gC3DLqFB7IfICUixe/3iQyN5P6R9zMzfSarPl3Fqk9X8Xb52zw06iGmXzW9zQ/Hmabp2fYQLKxJSZz75JNAlyEiEhBqdSYiPnG4HPx868+Z9fosHC4Hz05+lqeuf6pDgm9jUb2ieHDUg7x8+8tcHXc1P3n/J8xZN4dDpw+16Tqu48cxa2uDotODm3vEsWmagS5FRKTTKfyKSLtVnq1k7htzeeHTF/hK2ld4+faXuTb52k6tYWDkQH43+Xf8JPcn7Dm2h2l/ncb60vVen+/e+xpcK7+JmOfPU3fyZKBLERHpdD6HX8MwLIZhbDcM4zV/FCQi3cP2yu3MeG0Gu4/t5mfjf8bCsQvpY+sTkFoMw+D2obfz0tSX6B/Rn4c2PsRTW57CUedo9dwLAy6CJ/zaPO3OtO9XRIKPP1Z+vwvs9MN1RKSb+Nv+vzH3jbmEW8MpuKWAKalTAl0SAP0j+/PizS9y1/C7+MPOP/Cdt77DqdpTLZ7jGXDRN3jCr3uV21mpEcciEnx8euDNMIwU4Fbgp8D3/FJRkKg+fZ6tB46x7cAxiitOUXX6PPZTtZw576SXLYReVgsRYVYGxPYmNb4PQxL6cM3AGIYkXBEUvUil6yrYWcDiLYvJScrhlxN+SVSvrjUYItQSyvwvzictJo1Fmxfxjb9/g2UTl112D7KzohIMA2tC+7tFdDcacSwiwczXbg+/BP4TiLjcAYZhfAv4FsCAAQN8vF33dvq8k79+dIg/fXCQjw4eByDUEsLQxCtIiuzF8L6RXBFmpdZZR42jjhPnavncfoZ/7Kmi1lkHQGyfUL6YGsuNX0hi0vAkonrbAvklSRAxTZNlRctY8fEK/q3/v/E/4/+HXpZegS7rsr487MtcecWVPPyPh8n/ez6/mfSbZgdjOCsqsMTFYdiC5/8lT/hVuzMRCULtDr+GYUwFKk3T3GYYxg2XO840zRXACoCcnJygfLT4xFkHT28o5k9bSjlT6yIt6QrmTbmKawfFMuLKKMJsLTf9d9WZlFSfYWvJUbZ8fox399pZ9+kRrCEGXxoaz/RrUph8dRK9rK0PDxBpD9M0WbptKS98+gJ3DruTH475IdaQrt8p8YvJX+QPt/yBb735Le5+426WT1pOVmLWRccE04ALt5BevbBEReHQoAsRCUK+/O2VC+QZhnELEAZEGobxB9M07/JPad2fq87kTx+U8vPCPRw/W8sd2VeSf+1ARg2IbtPWBUuIwZCEKxiScAUzRg+grs7k4/ITvL7jMK99dJgHVm8ntk8o065JYdbYgaTENB0XK+KLZUXLeOHTF5hx1Qy+f+33u9XWm0FRg/j9Tb/nm4Xf5Ftvfotf/duvLupI4ayoxJacHMAKA8OalFS/5UNEJMgY/ujz2LDy+6hpmlNbOi4nJ8fcunWrz/frDo6cqOHbBdv4sPQ4XxwUy8LbvsDV/fy/N7KuzuTtvXb+tKWUNz+rwATyRvbj3vGDSe8b6ff7SfD57Ue/ZVnRMr4y7Cs8MfaJNg+R6Crs5+zcU3gPpSdLWTZxGWP7jQVgz9gvETFlMsn/9V+BLbCTlX7zHlzHjzPoLy8FuhQRkQ5hGMY20zRzLn29679v2Q1tLTnK/QUfcua8k1/MGMkdWVd22EpZSIjB+LQExqclcOj4OZ5753NWbyllzfZybrq6L9+bnEZa0mW3ZIu06I87/8iyomXcNvg2fjjmh902+ALEh8fz/JTnubvwbr678bs8M+kZsmMycB07hi2Ievy6WZMSOb97d6DLEBHpdH75m8w0zX+0tuobLP60pZSvrfgXfUItvPKdXL6cndJpbxH3iw7nh1O/wObHJvLQpGG8s9fOlF9u4uH/K+Lg0bOdUoP0HOsPrGfxlsVM6D+BRbmLsIR0/z3l0WHRrLhxBUm9k/jO+u/w6c63geDq8etmS0rCWV2N6XQGuhQRkU7VfZdxuqA/bSnlsZc/4UtD41n7H+MCtuIa1dvGQ5PSePs/J/Ct6wfz+o7DTFz6T5a8sYsz5/UXnbTuo6qPmP/2fDLiM3jq+qe6xcNt3ooLj+PZyc8S0yuGn73+GBBc093crImJUFeHs7o60KWIiHQqhV8/WVtUzoI1nzA+LYFnZ+UQFR74tkkxfUJZcPNwNj56A7eM6MuvN+5jws/+wdqicvyx11t6ptKTpTyw/gESeyfyq4m/ItwaHuiS/C6pTxLPTnmWxDP1of5EZPD9KHSvdjvV8UFEgkzw/cTvAG9+VsH3/vwRo1Nj+c1d1xBq7Vrf1uSocH75tWxe/vaX6BsVxnf/VMS/v/ABZce0FUIudsZxhgc3PEgddTwz6Rliw2IDXVKHufKKK7k76Q4AHv3sSU7WngxwRZ1Lgy5EJFh1rZTWDe2tPMWDq7czol8kz83OITy06+6LHDUghjXfzuWJqV9gy+dHuXHpJp59ez+uOq0CC9SZdTz+9uOUnCzhZ+N/xsDIgYEuqcPFnDIxw0LZdb6UhzY+RK2rNtAldRpbUn34dSj8ikiQUfj1QY3DxX/8cTu9Qy2smJVDRFjgtzq0xhJiMHfcIAofvp4xg2P5yd928uXl7/LpoROBLk0C7Lcf/ZYNBzfwSM4jjEkeE+hyOoWjsoJeScksGvdjPjjyAYs2LwqaLUGWuDiw2XAeORLoUkREOpXCrw9+/Npn7Dpyip9/dSRJkWGBLqdNUmJ6s3LOaH719WwOHT9H3rJ3+XnhbhyuukCXJgGwsXQjyz9aTt6QPO4aHjxzapwVlViTkrhtyG3cN/I+1u5by6pPVwW6rE5hhIRg69sXx6HDgS5FRKRTKfy2098+PkzB+6Xce/1gbriqe45GNQyD20b2463vjeeOrCv51Ya9fOWZ99hXdTrQpUknKj9dzvff/T5fiPsCT4x9oltNb/OVs6LC0+nh/pH3c+PAG1m6bSmbyjYFuLLOYUtOxqGVXxEJMgq/7VB9+jwLXv6YrP7RPDrlqkCX47Po3qH8/KsjeSZ/FKVHz3Lr02/zh38dCJq3f4OZw+Vg3j/nYZomPxv/M3pZegW6pE5jmibOykrP3tcQI4Sfjvsp6bHp/Oem/2Tf8X0BrrDj2ZL74jh8KNBliIh0KoXfdvj5m3s4U+tiybRMbJae8y28OSOZNx66ntGpsfzglR3cvWorVafOB7os6UBLty3lE/snLMpdRP+I/oEup1O5jh3DdDguGnARbg3n6X97mjBLGA//42HOOM4EsMKOZ01OxllRqUEXIhJUek5y6ySfHjrB6i2lzBo7kGHdfWywywknyuDgB/DZq1D0R5L2rGbV1dv5Y9YO4va9zFO/+B8+3PQaHCsBZ/A8CR8M1peu5w87/8DM9JncOPDGQJfT6dwtvi4dcNG3T1+WjF/CgZMH+K/3/qtHvwNiS+4HLhfOqqpAlyIi0ml6ztimTmCaJj/662fE9A7loYlpgS6nbUwTqvfB/o1wqAiOfAxVu6CZ1k4hwJeAL1mAOmBD/S8TAyOqPyR9ARK/AMkjYcBYiAi+6VjdXeXZSha+t5DhscN5JOeRQJcTEO4WX+5tD42N7juaB7Mf5Jcf/pKsxCzyh+d3dnmdwtYvGQDH4cPYkpMDXI2ISOdQ+G2Dv31ymC2fH+WnXx5BVO+u39YM04SDW+CTP0Pxm3D8QP3rveMhORMG3wexgyGyH0QkQ1gkWHqBJRTqnOA4w/mzpyjYsI2du3dyTfRZ8vqeofex3bD3rfpjAGIGQeo4GHYjDJ5Qfx3psurMOn7wzg847zzP4usXE2oJDXRJAeE8XN/lwJrcr9nPzx0xl6KqIn72wc+4Ou5qshKzOrO8TuEOvI5Dh2FUgIsREekkCr9eqnXW8eTfdzE8OZKvjR4Q6HJadvYobHsethfA0X1g6w2Db4AvPQBDJ9aHVS+f6O8VC3NnjWRtUTmPv/wJTxWHsHRGFhOGRMORT6B0c/2vna/C9hchxAoDc+HqO2D47dAnrkO/VGm7gp0FbD68mR+O+SGDowYHupyAcRw6hGGzYU2Ib/bzhmHw03E/ZcZfZ/DIPx/hz1P/TFx4z/rzbO3rXvnVQ28iEjwUfr20tqic8uPn+OmXR2AJovE+cgAAIABJREFU6aKtoE4egveWwbYXwHEGBo6D6x6BL+RBL9/2J9+edSUZV0bx7YIP+ffnP+D+G4bwyI3ZWFOugS/9R/3+4bItsOcN2PUavPYw/O1RGDIBsvIh/VawBk8nga5qz7E9/GLbL7ih/w1MT5se6HICylF+CGtyMkbI5R99iAyN5BcTfsFdf7+L+W/P57eTfoslpOtOcWwryxV9CImK8qyCi4gEAz3w5oW6OpMVm/aT3jeC8WkJgS6nqfOn4M0n4H9Hwvu/geG3wbf/Bf/+N8jO9zn4ug1OuIJXvpPL1784gGf+sY+Zv3v/QjcIixUGfglu/BH8x1a47x3I/S5U7oK//Dv8PB3WLYCjn/ulFmk7h8vB428/TmRoJD/60o+Cqp9vc7zd55oem873r/0+7x9+n+UfLe+EyjqXLTlZgy5EJKgo/HrhH3sqKa48zb3jB3etwFBXB0V/hF9dA+/+L4yYBg9uhzt/C4nDO+SWYTYLT96ZwS9nZPFx+XFu+9U7bC89dvFBhgF9M2DSQnjoY7jrZRg8Hrb8Dn41Cv7vG/V7kaVTrfhkBbuP7Wbh2IXEhsUGupyAcxw6hK1f8/t9L/XlYV/my0O/zIqPV/B22dsdXFnnsiUn49DKr4gEEYVfL/zmn/vpF/X/2bvvsCrLN4Dj3zPZe4mIgDhRceHEbWY2NBtmw/pZmqZpqWVlZqVppWVZOctRWZalllqae++9B4qADFmyx1nv749XbWky3sMBzvO5rnNBcHieG3PcPOd+7tuR+yNL9g9lhchJgu8ehl9eAI9gGLwZ+s0Br5AK2f7BFkGseCEanVbFY/P2snR//K2fqNbIdcaPLpYT4eiXIHYbLOgJX90Fp34Bi7lCYrZnpzNO8+XxL+kT3odutbvZOhybkwwGecBFCZNfgPFtx1Pfqz4Tdk0gvTDditFVLJH8CoJgb0TyewdHE7LYH5vJsx3DKs9Ai5MrYHZ7iN8L930Mz22AWq0qPIyImu6sfrEjbet488aKE7yx4gTFpv9IZN1rwl3vwOjT0Hs65KfBT8/Ip8FHlsh1w4LiDGYDb+58Ex9HH8a1HmfrcCoF49WrIEk3W32VhKPWkQ87fUi+MZ+3dr1Vbfr/6moGYsnJwZwnxpoLgmAfKkk2V3nN334Rd0ctA9pUgg4PJgOsGSPX0PqEw9Ad0How/MeFHWvzdNazeFAbhncNZ+n+eAbM30tKdtF/f5GDK7R9HkYehseWgKMn/DoCZreDk8vlcg5BMXOPzSUmK4a3O7yNh4OHrcOpFG7UuJbm5BegrlddxkaNZWfiTr4/+701Qqtw2ut1z+LSmyAI9kIkv/8hKauQtSdTeLJdCK4ONm6MkZ8O3z4IBxdAh1Hw7HrwrWvbmK7TqFWMu6chc55syfmUXO7/fCf7YzPv/IVqjXw57/mtchKs1sLPz8K8znBurdynWCiXc5nnWHRyEX3C+9C5Vmdbh1NpGJPk1l6lTX4BBjQYQJdaXZhxcAbnr51XOrQKp7ve51iUPgiCYC9E8vsffj2ahCTB47bu63v1NMzvBomH4KEv4e7JcneFSqZ300B+GRGNm6OWJ77cy9e7L5fspWGVSk6CX9glf3+GPFg6QK4Jjt9r/cCrKbPFzLt73sXdwZ1Xo161dTiVijEpEfjz1LM0VCoV73Z4Fze9G69tf40i0x1e6ajkbk55Ex0fBEGwEyL5vQ1Jklh55AqtQryo7eNsu0ASD8Hie+UxxIPWQmR/28VSAvUC3Pj1xWi6NvDj7VWneOWn4xQZS3ihTa2Rv78XD8ADn8mX+hb2gp+fg+wr1g28Gvrh3A+cSD/Ba61fw9PR09bhVCrG5GQ0fr6o9WWbbufj5MOUjlOIyYphxqEZCkdXsbR+fqDRYEwRya8gCPZBJL+3cSoph/NX8+jXIsh2QVzeCV/3AQd3eO4PCKoa80fdHXXMHxjF6Lvqs/zwFQbM30tqTilOxzQ6aPUMjDwIncfJQzM+j4KtH4ChwHqBVyPJecnMPDyT6KBoeof1tnU4lY6pFG3Obic6KJqnGj3F0rNL2X5lu0KRVTyVRoMuIEDU/AqCYDdE8nsbvxxJRKdRcX9k6V8WVcTFzbDkYXAPgmfXgVeobeIoI7VaxUt31WPuU604fzWXPl/s4viVrNItoneB7m/KJ8EN7oGt78MXreVLcaIe+LYkSWLKvikAvNXurcrVm7qSMCaWP/kFeLnVy9T3qs9bu96q0u3PtDXFoAtBEOyHSH5vwWS28OuxJLo18MfTuWwvi5ZLwn744UnwqSeXOrhXov7CpXRPkxosf6EDGrWKR+fuYdWxpNIv4llb7hP8v9/B2Uu+FPdNH0iPUTze6mB93Hq2XdnGiOYjCHK14SsXlZRksVyf7lb+P1cOGgemdZ5GvjGfibsmVtn2Z7rAmuLCmyAIdkMkv7ew62IGabnFPNTSBonD1VPw3SPgVgMGrgAXn4qPQWGNAt1Z9WI0zWp5MmrpEab/cRaLpQxJQmg0PL9N7m2cdAzmdIBt08BUrHzQVVR2cTbv73ufCJ8Inmz0pK3DqZTMmZlIBoMiJ78A4Z7hjG41mh2JO/gl5hdF1qxouho1MF69imQWA2cEQaj+RPJ7CysPX8HdUUu3hv4Vu3FmLHzbD3QuMPAXcK3g/a3Ix9WBJYPb8nibYGZtucjQJYfIKy7DUAu1Ru5t/OJ+aHgfbJkCc6Ll+miBTw59QlZxFu+0fwetuvJ1BKkMytPm7HYeb/g4rWu05sMDH5KYl6jYuhVFVzMQjEZM6Rm2DkUQBMHqRPL7DwUGE3+cusp9kTVx0GoqbuOiHPj+Mbmrw8CVFTamuCLptWqm9mvKOw9EsPlsKg/P3k1CZhkvsLnVgEcXwZPL5V+zxffBLyOgoAT9haupAykHWH5hOU9HPE0jn0a2DqfS+jP5Va6eX61SMzl6MpIkMXHXRCxS1RrU8uegizKUJQmCIFQxIvn9h90xGRQazRV70c1ihhVDICMG+n8D/g0rbu8KplKp+F90GF8PakNKThF9vtjJnovlOG2qdxcM3wsdx8DxH2BWWzi9SrmAqwij2cjkvZMJcg3iheYv2DqcSs2YqPzJL0CQaxDjWo9jf8p+lp5dquja1iYGXQiCYE9E8vsP286n4azXEBXqVXGbbn4Pzq+D3h9CmH1M4epYz5dfRkTj7aJn4IJ9LNkbV/bF9M5w19tyPbB7ICwbCD/9T56KZye+O/MdsdmxjG87Hietk63DqdSMycmoXV3RuLsrvvZD9R6iY1BHPj30KZezLyu+vrX8OehCnPwKglD9ieT3LyRJYuv5VDqE+1RcycPJ5bBzBrT6n1zLakfCfF1YOSKaTvV8mfDLSd765SRGczleLq7RBAZvgu4T4MwamNUGTq6o9m3RUgtSmXNsDl1qdREjjEvAmJSErgyT3UrixvQ3vUbPm7vexGypGhfING5uqN3dMSZWvXplQRCE0hLJ719cziggIbOQLvX9KmbDzEuw6iUIbgu9p8tjfu2Mu6OOr55pzdDOdfh2bxxPL9jPtXxD2RfU6KDzqzB0u9wi7edB8klwXqpyQVcyMw7NwGQx8Vrr12wdSpVgVGDAxX/xd/ZnfNvxHE87zuJTi622j9L0wcEY4hNsHYYgCILVieT3L7adkxOkLvUroMuCySCP7VWr4eEFoLVBP+FKQqNW8ca9jfj40WYcirtG31m7OH81t3yLBkTAcxvhrnfg/HqY3U4+Da5mDqYc5LdLvzGoySCC3YNtHU6VYExORhdk3d7Z94bdS8+Qnsw6Oovz185bdS+l6IKDMSTE2zoMQRAEqxPJ719sv5BOqI8ztX2crb/Zlvcg6TD0+QI8RdIC8HCrWvwwtB2FRjMPzd7NpjNXy7egRgsdR8unwB614Mcn5Y4QRTnKBGxjJouJqfunEugSyHNNn7N1OFWCOS8fS3a2VU9+QS5/mNBuAm56NybsnIDRbLTqfkrQBwdjTEwSvX4FQaj2RPJ7XZHRzJ6LGRVT8nBxM+yaCVHPQkQf6+9XhbSs7cWqF6MJ83Vh8DcHmb01pvxTs/wbyqfAnV6BY9/D3Gi4vEuZgG3ox3M/cuHaBca1HicuuZWQMUmuadVaqeb3r7wdvZnYfiJnMs8w/8R8q+9XXrrawWAyYUxOsXUogiAIViWS3+sOXr5GodFMlwZWTn6LsuXTR7+G0GuqdfeqogI9nFg2tD33R9Zk2rpzvPTDUQoN5TyN0uqhx1vw7B+g0sh9gde/VWWnw2UUZjDryCzaB7anR+0etg6nyrDGgIv/0qN2D/qE9+HL419yKv1UhexZVvpg+RUooyh9EAShmhPJ73Xbzqei16hpV8fK44Q3TIS8FHhwNujEad3tOOk1fDagOa/2asDq40k8Om83SVmF5V84uA0M2yl319j9GczvBikny79uBZt5eCaF5kJeb/s6Kju8KFlWxivyya8uqOJGl7/W5jV8nHwYv3M8xebK+8PWjeTXkCAuvQmCUL2J5Pe6befTaBPmjbPeiiNhY7fDocXQfgQEtbLePtWESqViRLe6fDkwisvpBfT5YheH4hSY4ObgCg98Ck8sg/w0+LKbXIZSRdpSHU87zsqYlQyMGEgdjzq2DqdKMcTHoXJ2RutXQR1dAHe9O5M7TOZS9iU+P/x5he1bWtoaNUCnwyiSX0EQqjmR/AIp2UWcv5pn3XpfQz6sGgnedaDreOvtUw3dFRHAyuEdcHXQMGD+XpYdUOgf5/q9YPge+e2GifD1A5B9RZm1rcRsMTNl3xT8nfwZGjnU1uFUOca4ePTBwRV+Wt4hqAP96/fnm9PfcCT1SIXuXVIqjQZ9UJBodyYIQrUnkl/gUNw1ANrW8bbeJlumwrXLcncHfQV0k6hm6gW48cuIaNqG+TBu+XHeWXUKU3kGYtzg4gv9v4UH50DyMZgTDad/Lf+6VrIiZgWnM04zNmosLjoXW4dT5Rji49HXrm2TvcdEjaGma00m7JxAoUmBEh4rEO3OBEGwByL5BY5dyUKvVdOwhvLjTgFIPQN758h1pqHR1tnDDng661k8qDXPRoexePdl/rfoAFkF5RiIcYNKBc2fkFuiedeBZU/DqlHyaX0lklWUxczDM2kV0IreYb1tHU6VI5nNGK9cQR9im+TXRefCpA6TiM+N57PDn9kkhjvRBwdjjE8of4cVQRCESkwkv8DRhCwa13RHr7XCL4ckwdrXwMENuk9Ufn07o9WomfhABNMeiWR/bKYyAzFu8AmH59bLvYEPfwPzusinwZXEF0e/IM+Qxxtt3hCX3MrAlJKCZDSis9HJL0CbwDY80fAJlpxZwoGUAzaL43Z0tYOx5OVhzsqydSiCIAhWY/fJr8ls4cSVbJrV8rTOBmdWQew26D4BXKzcScKO9I8KZunz7cgvNtNv1i42ni7nQIwbNDp5KtzTv4IhD766C/bMAosCJRblcDrjNMvOLWNAwwE08G5g01iqKkO8/HK+vnaITeN4qeVLBLsFM3HXRAqMBTaN5Z/+bHcm6n4FQai+7D75jUnLo9Boplmwh/KLGwrgjzchoAm0GqT8+nauVYgXq0dGU8fPlSHfHmTWFgUGYtxQpwu8sBvq9oQ/xsN3j0CuQgl2KVkkC1P3TcXL0YvhzYfbJIbqwBB3Pfm1UdnDDc46ZyZHTyYxL5EZh2bYNJZ/0ol2Z4Ig2AG7T36PJcgv71nl5HfXTMhOgN4fyqN2BcUFejjx07D29GlWk+l/nGPk0iPlH4hxg7M3DPgO7psBcbvkyXDn1yuzdimsubSGY2nHGN1qNO56K9Wl2wFDfDwqvR5tQICtQ6FVQCueiniKH8/9yN7kvbYO5yZx8isIgj0Qye+VbNwdtYT6KHxzPjdFHqLQuB+EdlR2beFvHHUaPn2sOa/3bshvJ5J5ZO5uEpUYiAHyZbjWz8Hz28A1AL5/FNa+DsYiZda/g1xDLjMOziDSL5I+4WIUdnkY4uPQ1Q5Gpa4cf+2NbDGSUPdQJu6aSJ4hz9bhAKB2ckLj5yvanQmCUK1Vjn8FbOhYQhbNgj1RqxW+QLR9OpgN0P0tZdcVbkmlUjGsSzgLnokiPqOAvl/s5MBlBQZi3ODfEAZvgrbDYN8c+KoHpJ5Vbv3bmH10NplFmYxvOx61yu7/uJaLMS7e5vW+f+WkdWJy9GSuFlzl40Mf2zqcm/TBtcXJryAI1Zpd/2taZDRzNiVX+ZKHjIvyJLeWz8gdBIQK071hACtHdMDNUcfj8/fy9e7LytUB6xzlEpYnlskn+/O7wsGFckcPK7hw7QJLzy7lkfqP0NinsVX2sBeSxYIhIcFmPX5vp7l/c56JeIafz//M7sTdtg4HkEsfRM2vIAjVmV0nv6eSsjFbJJoFK5z8bpkKah10GafsukKJ1PWXB2J0beDH26tOMXbZMeXqgEGeCPfCbghpD2tGw49PQYGCp8yAJEm8v/99XPWujGoxStG17ZEpLQ2pqMjml91uZUSLEYR5hDFx90RyDQq17SsHXe1gTFevYikutnUogiAIVmHXye/RhGwAmtVSsNND8nE4+TO0ewHcaii3rlAqHk465g+MYkzP+qw8msjDc3aTkKlgWym3AHhyOdw9Bc7/AXM6wKVtii3/x+U/OJBygFEtRuHpaKU2fHbEEBcHYNMev7fjoHFgSvQU0grTmH5guq3DkS+9SRLGK5V71LcgCEJZ2XXyeywhi5oejvi7Oyq36ObJ4OgJ0S8pt6ZQJmq1ilE96rHwmdZcuVbAA1/sZPv5NCU3gA4vwuCNoHeFb/rCxnfAbCzXsgXGAqYfnE4j70Y8XO9hZWK1c8YbPX5DKk/N71819WvKc02eY2XMSrZf2W7TWES7M0EQqjv7Tn6vZClb8pB4GC6sh+hR4CRO6yqLbg39WfViR2q4O/LMov3K9gMGqNkchm6Dlk/Dzk9gwd1y3XcZzTs+j9SCVMa3HY9GrVEuTjtmiIsHnQ5djcr7asywZsOo61mXd3e/S3Zxts3iuFEXbRQdHwRBqKbsNvm9lm8gLqNA2eR3x8fg6AGthyi3pqCIUF8XVgzvwP2Rcj/gYUsOkVtUvhPav9G7QJ/PoP83kHkJ5nWGo9+X+jJcbHYs35z+hr7hfWnu31y5+OycIT4efVAQKm3l7bet1+iZ0nEKGUUZfLj/Q5vFofH2Ru3sfHMiniAIQnVjt8nvqaQcAJoGKVTvm3ISzq6Bti+AoxhEUBk567V8NqA5E+5rxMYzqfSdtYuYVIUvGEX0hRd2QWBz+OUFWP4cFGaV6EslSeKD/R/gqHHk5VYvKxuXnTPEx6OrhJfd/inCJ4IhkUNYfWk1m+M32yQGlUqFvk4dDJfK/uqFIAhCZWa3yW9sutxUPtzPVZkFd3ws1322HarMeoJVqFQqBneqw5Ln2pJdYKTPF7v49Wiispt41IJnVsk9nk/9AnM7Qfy+O37Z5oTN7E7azYjmI/B18lU2JjsmSRLGuLhK1eP3vzzf9HkaeDVg0p5JZBWV7AcnpTmEh1N88ZJN9hYEQbA2O05+C3DSaQhwdyj/YukX4NRKaD1YHokrVHrtw334bVQnIgLdeemHo0z45QRFRgXboak10PkVeG69fDFu0T2w9QMwm2759CJTEdP2T6OuZ10GNBygXBwC5owMLAUFla7H7+3oNDqmdJxCdnE2U/dNtUkM+vBwTFevYs61fes1QRAEpdlt8ns5I58QH2dUKgUmu+2YAVpHaP9i+dcSKkwND0eWPt+OoV3qsGRvPI/M3U18hoLt0ABqRcHQHdC0P2x9HxbfB1n/rqVceHIhSflJjG87Hq268talVkWGm50eqkbyC9DAuwFDmw1l7eW1rItdV+H7O9SVh/MYLonTX0EQqh/7TX7T8wnzdSn/QjlJcGKZfNPf1a/86wkVSqdR80bvRnz5tDwW+b7Pd7DuZIqymzi6w0Pz4KEv4eopmNMRTi6/+emE3AQWnFhA77DetK7RWtm9BbnTA3+28KoqBjcdTKRvJJP2TuJq/tUK3duhTh0AimNE3a8gCNWPXSa/JrOF+MwCQpVIfvfPB8kiD7UQqqyeEQH8NqoTYb4uDFtyiPfWnMZotii7SWR/GLYD/OrDz8/CL8OhOJdpB6ahUWsY22qssvsJAPLFLZ0Ofa1atg6lVLRqLVM7TcVkMfHWrrewSAr/fvwPulq1UOl0FItLb4IgVEN2mfwmZhViskiE+ZQz+TXkw8FF0PA+8A5TJjjBZoK9nflpWHv+1yGUr3bG8ti8PSRlFSq7iXcYDFoLncfBsaVsX9CRrQlbGRo5lACXAGX3EgAoPn8Bh9BQVDqdrUMptRD3EF6JeoU9yXtYenZphe2r0mrRh4VhEJfeBEGohsqc/KpUqmCVSrVFpVKdUalUp1QqVZUZaRabng9Q/pPfY0uhKAvajVAgKqEycNBqeKdPY754ogXnr+Zx32c72HIuVdlNNDro/iZFT61gqqOZOgYjT2emg0XBC3fCTcUxMTjUq2vrMMrs0fqP0imoE58c+oRLWRWXjOrD61B8UZz8CoJQ/ZTn5NcEjJUkqRHQDhihUqkilAnLui7fTH6dy76IxQJ750DNllC7nUKRCZXF/ZE1WfViNAHujgxadID31pym2KRscrog9zSJGhVvujVGt2UqLLoXrl1WdA97Z8nPx5iYiEO9erYOpcxUKhWToifhpHXi9R2vYyzn+OyScgivi/HKFSxFRRWynyAIQkUpc/IrSVKyJEmHr7+fC5wBgpQKzJouZxTgotfg51qONmcX1kNGDLQfAUp0jBAqnTp+rvwyIpqn24fw1c5YHp6zm0tpeYqsHZ8Tz8ITC+kd1ps2j/0M/eZD6mmYEw1HlpR6Mpxwa8XXuxXo61bdk18AXydf3mn/DmcyzzD3+NwK2dMhvA5IEobY2ArZTxAEoaIoUvOrUqlCgRbAvzr5q1Sq51Uq1UGVSnUwLS1Nie3KLTY9n1Bfl/K1Ods7C9yD5IleQrXlqNMwqW8Tvnw6isRrhdz/+U6WHUxAKkdyKkkSU/dPRafR8WrUq/IPT80egxd2Q80W8OsI+PEpyM9Q8DuxT8XnLwDgUMWTX4AeIT14sO6DfHXiK46mHrX6fvpwud2ZGHYhCEJ1U+7kV6VSuQLLgZclScr55+clSZovSVKUJElRfn6VoxXY5Yz88tX7pp2D2O3yUAtN1btEI5Rez4gA1r7UmWa1PBn383FeXHqE7MKyvfy8KX4TuxJ3MaL5CPyc//JnwjMYnl4FPSfLryzMbgfn1yv0Hdin4pgYVHp9lRlwcSevtX6NQJdAxu8cT74x36p76UNDQa2m+GKMVfcRBEGoaOVKflUqlQ458f1OkqQVyoRkXUazhSvXCsvX6eHQYlDroMVAxeISKr8aHo4sGdyWV3s1YN3JFO6duYNDcZmlWqPAWMAH+z+gvld9Hm/4+L+foFZD9CgYsgVcfOH7R2HNGLmziFBqxTEx6MPDUWk0tg5FEa56V6Z2nEpiXiJT9k6x6l7q6z80iI4PgiBUN+Xp9qACFgBnJEmaoVxI1pWQWYDZIpX95NdYCEe/g4g+YqiFHdKoVYzoVpefh7VHrYb+8/Yyc+MFzJaSlUHMPT6XqwVXmdBuwn9PcqvRRE6A278IBxfCvM5w5ZBC34X9KI6JqRYlD3/VMqAlwyKHsfrSalZfXG3VvfTh4aLjgyAI1U55Tn6jgYFAd5VKdfT6416F4rKayxnyCVpYWTs9nFoJRdkQ9ayCUQlVTYvaXvw+qhMPRAbyycbzPD5/LwmZ/z0a+WLWRb499S19w/vSwr/FnTfROUKvKfDMKjAWwYK7YOO7YCpW6Luo3sy5uZiSk6td8gswJHIILf1b8t7e94jLibPaPg7h4Rji4pCMFdNhQhAEoSKUp9vDTkmSVJIkRUqS1Pz643clg7OG2HQ5QQkta9nDwYXgWx9CohWMSqiK3Bx1fDqgBTP6N+N0cg73fLqdH/bH3/IynCRJTNk3BSedE6NbjS7dRmGdYfhuaP4E7JwB87pA0hGFvovqqzhGrlWtyj1+b0er1vJh5w/RqrWM2z7Oau3PHMLrgMmEIT7eKusLgiDYgt1NeLucno+boxZvF33pvzj5OFw5IJ/6ivZmwnUPtazFupc7EVnLk9dXnOC5rw+SmvP33qhrLq3hQMoBXmrxEj5OPqXfxNED+s6CJ36SB6t82QM2TwGTQaHvovr5M/mtuj1+/0sNlxpMip7E6YzTzDw80yp76MPlHxxE6YMgCNWJ/SW/GfmElbXN2aFFoHWEZgOUD0yo0mp5OfPd4La880AEu2LSufvT7aw+lgRAZlEm0w5MI9IvkkfqP1K+jerfDcP3QGR/2D4NvuwOKScU+A6qH0NMDConJ3RBVaL9eJn0qN2Dxxo8xtenv2Zn4k7F13eoI49tN4jkVxCEasTukt/Y9PyylTwU58HxZdD4IXDyUj4wocpTq1X8LzqM31/qRKiPCyOXHuHF7w8zZc+H5BnzeKf9O2jUCnQdcPKCfnNhwFLIuwrzu8K2aVBBk7+qiuILF3CoUweVunr/NfdK1CvU86rHmzvfJL0wXdG11c7O6IKCKL5wQdF1BUEQbKl6/6vwD8UmM0lZhWXr9HBmNRjyoOXTygcmVCvhfq78PKw9r/ZqwIbYbayP/50eNQZQz0vhl98b3gsj9kHjfrBlCnzVA66eUnaPKqz4QvXr9HArjlpHpneeToGxgDd2vIFFsii7fkQjik6dVnRNQRAEW7Kr5DchswCLVMZOD8e+B68wqN1O+cCEakerUTOoY02C6q5Faw7g580NeH35cXKKFD6ddfaGh7+C/t9CdqJ8GW7zFLvvCGHOysKUloZD/epZ7/tP4Z7hjGszjr3Je5l/fL6iazs2bowhLg5zbq6i6wqCINiKXSW/8ddbUdX2LuXJb1a8PNGt2ePioptQYrOPziatKJnZvaYyvEtDlh1M4O7s3bujAAAgAElEQVQZ21l/KkX5zSL6wIj90OQhuRZ4bkeI36v8PlXEzctudnDye8Mj9R7h/jr3M/vobHYn7lZsXcfGjQEoOn1GsTUFQRBsya6S3xvjaEvd6eHYj/JbcdFNKKFTGaf49sy3PFzvYdoHtWHcPQ1ZOTwaT2cdz397iOHfHSI1t+jOC5WGiw88NB+eXC73BV7YC34bC0X/mjpe7dlj8qtSqXir3VuEe4bz2o7XSM5LVmTdm8nvKVFSIwhC9WBXyW9esRkAF4dSXDqSJLnkIbQTeIVYKTKhOjFZTLy7+128Hb0ZEzXm5sebBXuyemRHXu3VgI1nUrnr420sO5Bwy77A5VLvLrkjRLvhcGABzG4H59Ypu0clV3TqNGp3d7Q1a9o6lArlrHPmk66fYLKYGLN1DAZz+Vvhab290QYGiuRXEIRqw76S3yITAG4OupJ/UcI+yLwkDxgQhBJYcnoJZzLP8EabN3DXu//tczqNmhHd6rLupU40DHRn3PLjPPHlPi6n5ysbhIMr3PM+DN4o9whe+hj8NAjyUpXdp5IqPHkSpyZNytbSsIoL9Qjlvej3OJlxkmkHpimypmNEhEh+BUGoNuwr+S02olGrcNSV4ts++j3oXKBRH+sFJlQbCbkJzDo6i67BXekZ0vO2z6vj58oPQ9rx/kNNOZmUTa9PtzNn60VMZmVv6lMrCp7fBt0mwNk18EVrOLJEfkWjmrIUFlJ8/jyOTZvaOhSb6RHSg0GNB/HjuR9ZfXF1uddzbByB4fJlzHl5CkQnCIJgW3aV/OYXm3HRa0p+GmQshFMr5ctEDq7WDU6o8swWMxN2TkCr1vJm2zfv+PtMrVbxeJvabBzTha4N/Phw3Vn6fLGLYwlZygam1UOXV2HYLvBrCL+OgEW9q21btKIzZ8FsxinSfpNfgFEtRxEVEMWkPZM4l3muXGs53bz0JlqeCYJQ9dlV8ptbZMLNsRQlDxc2QHGOPE1LEO5gyZklHE49zOttXqeGS40Sf12AuyPzBkYx96mWpOcV8+DsXYxfeYKsAoVHF/vVh0Froc/nkHYO5naCdeOhuHq1sCo6cRzArk9+AbRqLdO7TMdN78aYrWPIMZT94uOfl95E8isIQtVnV8lvfrEJVwdtyb/g5HJw8YPQztYLSqgWYq7F8Nnhz+gW3I0+4WUrkbmnSSCbxnZhUIcwfjyQQLePtvLjgXgsFgVLFNRqeVDLyEPQciDsnS2XQpz4udqUQhSeOIk2IACdv7+tQ7E5XydfPu76MUl5SYzbNg6TxVSmdbS+vmgDAsTJryAI1YJdJb95xaaSd3oozoXzf0DEg6ApRcIs2B2jxcj4neNx0bnwdvu3y3XJys1Rx8QHIlgzsiN1/V15bfkJHpm7m5OJ2QpGjDwc44GZ8oU4Fz9Y/hx80xfSq/4Y28ITx+2+5OGvWvi3YEK7CexK2sX0A9PLvI5j48bi0psgCNWCXSW/ucUmXEta9nBuHZgKocnD1g1KqPK+PP4lZzLPMLH9RHycfBRZs1GgO8uGtuejR5sRl1FAny928tYvJ8nMV7gUolYUPL8V7v0Iko7C7Paw8V0wKNx9ooKYs7IwxsXj2EQkv3/1cP2HeTriab4/+z0/nv2xTGs4No7AEBuLOa9q/t4QBEG4wa6SX7nsoYQnvyeXg3sQBLe1blBClXYq/RTzj8/n/jr3c1fIXYqurVKpeKRVLTa/0pWB7UL4fn88XadvYeHOWIxKdoVQa6DNEBh5UP5hb+cMmNVW/jNQxUohCk/KJ5Pi5PffxrQaQ6egTry//332JO0p9dc7Nm4MkkTxWTHpTRCEqs2ukt+8ohLW/BZeg5iN0LifXCMpCLdQZCpi/M7x+Dj58EbbN6y2j4eTjnf7NmHtS51oFuzJpDWnuefT7Ww9p3DPXld/eGge/O93cPSEn5+Vp8RdOaTsPlZ087Lb9Qtawp80ag3TOk8jzCOMsdvGEpsdW6qvdxKT3gRBqCbsKrPLKzbhWpIBF2fWgMUoSh6E//T5kc+5lH2JyR0m/2uYhTXUD3Djm2fb8OXTUZgtEv9bdIBBi/YTk6pw79XQaBi6DR74DDJj4avusOJ5yE5Udh8rKDxxEn1YGBp36///qIpc9a580eMLdGodIzePJLu45LXkWj8/tH5+N0/XBUEQqiq7SX4tFol8QwnLHk4uB68wqNnC+oEJVdKBlAN8e/pbHmvwGB2COlTYviqVip4RAfwxujPj723IwcvX6PXpdt5ceYLU3CLlNlJroNUzMOowdBwDp36Bz1vBlvcrbT2wJEnislsJBLkG8Wm3T0nKS2LM1jEYLcYSf61Ty5YUHDqo/EhuQRCECmQ3yW+B0YwkgavjHcoe8tIgdpt86muHo1GFO8sqyuL1Ha8T7BbMmFZjbBKDg1bD853D2fJqV55qW5sfDyTQdfpWPt14nvzisrWzuvVGbnDX2/DiAWhwD2z7AD6PgmM/gEXhaXTlZLp6FXNaurjsVgIt/FvwTod32J+yn3d2v4NFKtn/S+c2rTElJWNMrPyvAgiCINyO3SS/NxKCO5Y9nPsdJAtE9K2AqISqRpIk3tr9FplFmUzrMg1nnbNN4/F1deDdvk3YcH1K3KcbL9Bl+laW7I1T9lKcVwg8uhgGrQO3AFg5FL7qAbHbldujnAqPy/W+4uS3ZPqE92F4s+GsuriKGQdnlOg016VNGwAK9u23dniCIAhWYzfJb26RnPzesc/vmdXgGQI1xD+gwr8tPbuUrQlbGdNqDI19Ks+lqjBfF2Y/2YoVwzsQ5uvMhF9Ocvcn2/n1aKKyQzJC2sPgzdBvHuRdha8fgG8ehMTDyu1RRoXHjoFOh0PDhrYOpcoY1mwYjzd8nK9Pf83Ckwvv+Hx93bpovL0p2C+SX0EQqi67SX7zrp/8uv1X2UNRjlzy0OgBUfIg/MvZzLN8dPAjOtfqzFONnrJ1OLfUsrYXy4a2Z/7AVug1al764Si9Z+7gj1MpytVpqtXQbACMPAx3T4HkY/BlN1j2NKSdV2aPMijYuw+nZpGoHRxsFkNVo1KpeL3N6/QO7c2nhz9lxYUVd3y+c5s25O/fL+p+BUGosuwm+S1R2cOF9WA2QMP7KygqoarINeQydutYvBy8mBw9uVxT3KxNpVJxd+MarH2pEzMHNMdgtjD020P0nbWLbefTlEtadI7Q4UV46Rh0eQ1iNsHstvDrCMhKUGaPEjJdu0bR6dO4dKi4y4fVhVqlZkrHKUTXjObdPe+yKW7Tfz7fuU1rTMnJGK9cqaAIBUEQlGU3yW+Jyh7OrAYXfwhuU0FRCVWBJElM2DmBpLwkPur6Ed6O3rYOqUTUahV9mwexYXRnpj0SSUaegWcW7ufRuXvYrmQS7OgO3cbLSXDbYXB8GXzeEtaNh/x0Zfa4g4J9+0CScGnfvkL2q250Gh0zus6giW8Txm0fx4GUA7d97s26X1H6IAhCFWU3ye/Nsofbnfwai+TBFg3vlds8CcJ1i08tZnPCZsZEjaGFf9Vrf6fVqOkfFcyWV7oyuW9jErMKeXrhfh6cvZtNZ64qlwS7+MI978vlEJH9Yd8cmNkMNk2Ggkxl9riN/N17ULu44NRU1OqXlbPOmdk9ZhPsFszIzSM5lXHrfr768HBR9ysIQpVmN8nvjbKH2578XtoKhjy53lcQrjuQcoCZh2dyd8jdlbbOt6T0WjUD24ey9dWuTO3XlIy8Yp77+iD3f76TdSeTlbsY5xkMfWfB8L1Q9y7Y8TF82hQ2vmO1k+D8PXtwbtsWlbYEExyF2/Jw8GBez3l46D0YumEoZzPP/us5f9b9HhB1v4IgVEl2k/zeOPm9bZ/fs6vBwQNCO1dgVEJllpiXyNitYwl2C2ZS9KRKXedbGg5aDU+0rc2WV7oy/ZFICgxmhi05zD0zt7P80BUMJoVapPk1gP5fw/A9UL8X7PwUPo2E9W/J/bQVYkhIwJiQIEoeFBLgEsBXvb7CSevE4PWDOZd57l/PEXW/giBUZXaV/Oo1ahy0tzj5NZvg7O9Q/27Q6is+OKHSKTAWMGrzKEwWE593/xwXnYutQ1KcTqPm0ahgNozuzMwBzVGhYuxPx+gyfQtf7bh08wfGcvNvBI8shBH7oOF9sOcL+SR47euKXIzL37MHAJcOIvlVSrBbMAt7LcRR43jLBFjU/QqCUJXZT/JbZLp9yUPCPijMFF0eBAAskoUJuyYQkxXD9C7TCfUItXVIVqXVqOnbPIh1L3di0aDWhPg4895vZ2j//iamrTur3Nhkvwbw8JcwYj80fhAOfAmfNYeVL0Dqv19eL6n8PXvQ+vujr1NHmTgFQE6AF/VahIPGgWf/eJaT6Sdvfk4fHo7Gx4f8fftsGKEgCELZ2E/yW2y6fcnD+XWg1kHdHhUblFApzT46mw1xGxjbaizRQdG2DqfCqFQqujXw54fn2/PLiGg61fNlzraLdPxgC2OXHeNkYrYyG/nWg35zYdRRaD0ETv8it0hb+jgklO4kUbJYKNizF5cOHapNWUplEuwezOJ7FuOmd2Pw+sEcTDkIyL9XXKI7kL99B5JJwXHagiAIFcC+kt/bdXq4sB5COoCDW8UGJVQ6Ky+sZN7xefSr24+BEQNtHY7NNA/2ZPaTrdgytisD2gSz9mQy93++k0fn7ub3E8mYlBid7BkMvT+A0aeg6xsQvwcW9IQFveD0r3I50h0UnTmDOStLlDxYUS23Wnx9z9f4O/vzwsYX2Jm4EwC3nj0xZ2VRcOD2bdEEQRAqI/tJfotMuN6q7OHaZUg7C/XvqfCYhMpld9JuJu2ZRPvA9rzV/i1xkgiE+rowqW8T9rzRgwn3NSIlp4jh3x2my/StzN12kcx8Q/k3cfaGrq/LSfA9H0Jusjwt7rMWsPsLKLr9iXPB9Xpf53btyh+HcFsBLgEs6rWIUI9QRm4ayeqLq3Ht2BGVkxM569fbOjxBEIRSsZ/kt9iEq8Mtyh7OX/+Lu36vig1IqFTOZZ5jzNYx1PGsw4yuM9Cp/2MSoB3ycNIxuFMdtr7SjfkDW1Hb25kP1p6l3dRNjFx6hN0X08vf9krvAu2Gwagj8NgS+WR4/ZswIwJ+HwcZF//1JbkbNuLQsCE6f//y7S3ckY+TDwt7LaRVQCvG7xzP4otLce3cidyNG5EsCnUIEQRBqAB2k/zmF5twdbxFQnPhD/AOB5/wig9KqBQSchJ4YeMLuOhcmNVjFq56V1uHVGlp1PLo5KXPt2P96M480bY2286l8sSX++j+8Tbmb79IRl5x+TZRa+R+24N+h+e3yh0iDi6Up8Z9+xCc/Q3MJgxXrlB47Bju992rxLcmlICb3o3Zd82md2hvPjn0CZvqFGBOS6fwyBFbhyYIglBidpP85hbfouzBkA+xO0TJgx27mn+VIRuGYLAYmHfXPGq41LB1SFVG/QA33unTmP1v3sWM/s3wddUz9feztHt/Ey9+f1iZ0+CaLeCh+fDyCeg6HlJPww9PwMxm5Mx6HQCPe0XyW5H0Gj0fdP6A/zX+H1847cGkVZGxdo2twxIEQSgxuxmHlH+rsodL28BcLPf3FezOtaJrDN0wlGtF11jQawF1veraOqQqyVGn4aGWtXioZS3OX81l6f54VhxOZM3xZMJ8XXi4ZRAPtgiilpdz2TdxD4Sur0GnMXBuLRxcQM6Pe3HyldDtGg8tBkJ4NzGavIKoVWrGRo0lxD2E46veJmzNz0ijBlHbvbatQxMEQbgjuzj5NVskCgxmXP6Z/J5fB3o3qN3BNoEJNpNdnM3QDUNJyE3gix5f0MS3ia1DqhbqB7jx9gON2Te+B5881gw/Vwc+Wn+ejh9uof+8PfywP57sQmPZN9DoIKIPxe0/ojhbh3uXKIjdBt89LA/O2DTplrXBgnU8Uv8RGj38LF5ZJsZ/2Z/dibttHZIgCMId2UXye3O08V+TX0mSW5yFdxNT3exMVlEWg9cP5mLWRT7p9gmta7S2dUjVjqNOQ78WtVg2rD07xnXjlbvrk55XzOsrTtB6ykaGf3eIDaevlnmUcvZvv4FajfvoWTD2HDz6NfhHwM5P5Nrghb3hyBIozlP4OxP+qVm/50CjoeMFLcM2DmP+8flYJHEBThCEyssuyh7yrye/bn8dcpFyQm6pJLo82JXMokyGrB/C5ezLfNb9M7saYmErwd7OvNi9HiO61eX4lWxWHklk9bEkfj+Rgpezjvsja9KneU1a1fZCrb5zezlJksj57Xdc2rVD6+srf7Dxg/IjJwmO/SAnvr+OkLtENH4Qmj8JtduD2i5+3q9QGk9PXNq3p/vZC8Q+djefH/mcE2knmBw9GU9HT1uHJwiC8C92kfzeOPn9W9nDxU3y27p32SAiwRau5l9l2MZhJOQm8HmPz+lQU5S7VCSVSkWzYE+aBXvy5n2N2HEhjZVHklh2MIFv98bh5+ZAr8YB9G4SSNswb7SaWyeqRcePY0xIwHfYsH9/0r2mXBfccbQ8tvzIEji1Eo5+B241oXE/aPIwBLUE0cdZMZ6P9Sdx5CjeNNxNZJsWfHTwIx5Z/Qgfdv6QVgGtbB2eIAjC39hF8ptbdIuyh4ubwb8xuInb/fbgUvYlhm0YRo4hh9k9ZtMmsI2tQ7JrOo2a7g0D6N4wgLxiE5vPprLuZDLLDyWyZG88Xs467o6owT1NaxAd7ote+2cinL16DSqdDree//GDq0oFtdvJj94fyu3RTq6A/fNh7yzwDIEmD0Hjh6BGU5EIl5Nbt27oatbk2pIlPPntNzT3b864beN49o9nGRo5lCGRQ0TvbEEQKg27SH7/VfZgyIf4vdDmeRtGJVSU42nHGbFpBBqVhkW9FtHIp5GtQxL+wtVBS59mNenTrCaFBjPbzqey9mQKv51I5seDCbg5aunZKIC7G9egQ4CO7BUrcOvVC427e8k20LtAZH/5UXjteiK8HHZ9JtcI+9T7MxH2b2jdb7aaUmm1eD35BKnTP6Lo3DkaN2jMsgeW8d7e95hzbA7br2xnSscphHuKfuqCINieXRTA/avsIW43mA0Q3t2GUQkV4Y/Lf/DcH8/hqnPl297fisS3knPSa7inSSAzB7Tg4IS7WPBMFL0a12DT2VSGLTnEB8OmYCkoYFure7iUllf6PsJOXtDiKRi4El45D/fNkF/92TYNZreFz1vB+rcgbg9YzNb5Jqspz4cfRuXoyLUlSwBw0bnwfqf3+ajLRyTmJdJ/dX8Wn1yMyWKycaSCINg7Vbmb0JdCVFSUdPDgwQrb74ZlBxMY9/Nxdr7WTe41uu4NOLAAXo8DnVOFxyNYn0WyMPfYXOYcm0ML/xZ80vUTfJx8bB2WUEZGs4WD51JwerofMV7BvNJqEAAhPs50a+BPt4b+tA3zxlFXxj6/OclwZjWc+x0u7wSLEZx95AE4DXpDnW7gICb/3UnyxLfJ/vVX6m7dgtbL6+bH0wvTmbRnElsSttDIuxFvd3ibxj6NbRipIAj2QKVSHZIkKeqfH7eLsoe86zW/bg7Xa84uboaQDiLxrabyDHlM3D2RDXEb6Bvel4ntJ6LXiHZ2VZlOo6bB0a1czcvh3jmv0Ta8MVvOpbLlbCpL98ezePdlnHQaOoT70KGuL9F1fajv71ai7hGAPESj7fPyoygbYjbJifDZNfJlOY0D1OkiJ8J1e4JnsHW/4SrKe+BTZC1bRtbPP+M7ZMjNj/s6+TKz20zWx63ng/0f8MRvT/BEwycY3nw4bno3G0YsCII9souT3882XWDGhvPETOmNNi8ZPomAnpMhelSFxyJY17nMc4zdNpYruVcY3Wo0T0c8jUpcZqryJJOJi73uQevrS8gPS//2/7TQYGbvpQw2n01l+4U04jIKAPBx0dM+3IcO4XIyXNvbufS/F8xGiN8jT5U7+xtkxckf96kn9wgP7w6hHcFBJHA3xA0aRHFMDOFr16FxdfnX53MMOcw8NJOfzv+El6MXo1qM4sG6D6IR0/kEQVDY7U5+7SL5ff/3M3y95zJnJ/f+s//nsF1QQ0z1qi4kSeLnCz/zwb4P8HTwZFqXaaLFUjWSvXoNSa++Sq1ZX+DWo8d/PvfKtQJ2X8xgz8UMdsWkk5pbDECQp9P1k2EfokK8qeXlVLpkWJIg7az8ytHFLXJ5hKkQ1Fqo1ebPZLhmC7ses1x47BiXHxuAz5Ah+I8dc9vnnco4xYf7P+RI6hEaeTfi5VYvi/aDgiAoyq6T3/ErT7D+VAoHJ/SEn5+F2B3yZRdxIlgtpBWk8e6ed9l2ZRsdanZgasepor63GrEUFXGpb19UOh11Vq1CVYpBFZIkcTEtnz0X09kVk8GeSxk3xysHuDsQFeJNqxAvokK9aBToju42vYVvyVQs9xK+kQwnHwMkcPSAkI4Q0l4enR4YKY9ltiNJr79Bzm+/UWf1KvShobd9niRJrI1dy8zDM0nKT6JtYFtebvmyGDcuCIIi7Dr5HbX0CMevZLF1bBeYHg717oaH5lV4HIKyJEli3eV1TNk3hSJTES+3fJknGj2BWmUXTUzsRupHH5Hx1QJqL16MS7u25VrLYpE4k5LDobhrHLx8jUNx10jMKgTASaehebAnUaFetAzxonktT7xcSlErnp8BsVvlRDhuF2Rekj+uc4Hg1nIiHNIegqJA71yu76OyM6amcqn3vTi3bk3w3Dl3fL7BbOCn8z8x//h8Mosy6RTUiaHNhtLMr1kFRCsIQnVl18nvc4sPcDW3iDUPu8L8rtBvPjR7rMLjEJQTnxPP1H1T2ZW0i0jfSN7r+B5hHmG2DktQWOGpU1zu/xge/R6k5nvvWWWP5OzCm4nwwbhMTiflYLn+12KwtxORQZ40reVBZJAHjYM88HAq4SluborcVjF+j9w67epJQAK1Ti6NqN0WglpBzZbgWbvavRKVsWAhqdOnEzxvLq5dupToa/KN+Xx/5nu+Of0NWcVZtA1sy6DGg+hQs4Oo3RcEodTsOvntP28PKuDHxvtg49sw9pyY7FZFFRgLWHRqEQtPLESn0TGyxUgea/AYWrVdNC6xK5LJRGz//pjS0ghfswaNh0eF7JtfbOJYQhbHE7M5cSWb44lZJGQW3vx8mK8LTYM8iKzlQUSgOw0D3fEuyQlxYRYk7JdPheP3QNJRMMv1yDj7yolwUMs/E2KXql26IxkMXOrTF8lsJmz5zyUfSoL853zZuWV8ffpr0gvTqetZl6caPUXvsN4466r3qbkgCMqx6+T3vs92EOjhyFeaDyArHl48UOExCOVjsphYcWEFc47NIb0wnd5hvXk16lX8nP1sHZpgJenz5pP2yScEfTYT97vvtmks1/INnEjM5kRiNsevZHHiSjZJ2UU3P+/v5kDDQHca1nC7/nAn3N8FB+1/XHwzGSD1FCQevv44JF+o4/rfyZ4h8glxjSYQ0BQCGoNHrSp1Qlxw+DBxz/wPl7ZtCZ43F5WmdBcBDWYD6y6v45tT33Du2jlcdC7cE3oPD9V7iKa+TcVpsCAI/8muk98u07fQqpYrMy71hWYD4P4ZFR6DUDZGi5F1seuYf3w+l3Mu08K/BWNajaG5f3NbhyZYUc769SS+PBq3u++m1qef2DqcW0rPK+Zsci5nU3I4c/3that5GMwWALRqFXX8XKgX4Ea4nyvhfi6E+7lSx88FZ/1tXqkozpUvziUekh9JR/9srwbg6AkBTeREuMb1t/4Rlbpn+bUfl5Hy9tt4DxpEwGvjyrSGJEkcTj3Migsr2BC3gUJTIXU96/Jg3Qd5IPwBvB29FY5aEITqwK6T31aTNzAkLJ1hMcPg0cXQuF+FxyCUToGxgNUXV7Po1CIS8xKp51WPkc1H0jW4qzjtqeby9+0nYfBgHJs0ofbCBaidKm9i908ms4XLGfk3k+GzybnEpOWRkFlws44Y5LZrda4nw+F+LoT5uhLi40yghyPaf3acKMqB1NNyzXDKSbh6Sn4Y868/QQVeIeBb/9+PSlI6kTJpMte+/57AD97H88EHy7VWniGPdZfXsfLCSo6nH0er0tI2sC3da3enW3A38WqQIAg32XXy22DCWuaFbqXrlbnw6kVw8a3wGISSicuJ44ezP/BrzK/kGnOJ9Ivk+abP07lWZ5H02oGiM2eIG/g0usAahHz7LRpPT1uHpIgio5m4jAIupuVxMTWPi2l5XErP52JqHvkG883nadUqgrycqO3tTIiPMyHeLgRff7+2tzMuDtdPjC0WyLosJ8OppyH9PKSdh4wLYPqzHAMnb/BrAL71wCsMvMPAK1R+36nifm0lo5H4wUMoOHSIwHfexvORRxRZN+ZaDKsurmJj/EYSchMAiPSLpHtwd3rU7kGoR6gi+wiCUDXZbfJrMFmoP2Et22t8Sm19HgzfU6H7C3eWXZzN+rj1rLm4hsOph9GqtPQM6cmAhgNo4d9CJL12InfjRpLeGI/a1ZXQpd+jq1H9L6VKkkRKThGX0wuIz8wnLqOAuMwCEjILiMsouNmT+AZ3Ry01PZ2o6elEoIfjzbeBHk4EeToR4K7DIS9JToZvPNKuvy1I//vmTl7XE+FQORn2CpW7TnjUAvcgxduxmXNzSXx5NPm7duEzZDB+o0eXqmfzf5EkiZisGDbHb2ZT/CbOZJ4BIMg1iNY1WtO6Rmva1GhDDZfq/3tKEIQ/2W3yey3fQNvJv3PaeSjaqGfg3mkVur9wa+mF6WxL2MbmhM3sSdqD0WIk1D2UB8IfoF/dfuKlSzsiGQykfjyDzK+/xrFpU4I++QR9rSBbh1UpZBcYic8sIC4zn4TMQpKzC0nKKiQpq4jk7EKuFRj/9TW+rg7U9HQk0MMRfzdH/Nwc8HNzIMDBSE3pKn6mJDwKr6DNjoNrlyEzFrITwGL6+0JO3nIifCMZvvG+WyC4BoBbAOhdS3UBTzKZSJn8Hlk//ojrXT0IfPtttH7K/1lPzktmS8IW9qfs5+DVg2QXZwNQy7UWbQLb0NyvORE+EUzXrqMAAA0KSURBVNTxrINObV8DSATBntht8puQWcCY6XP4yWESPPYdNLq/QvcXZIWmQo6lHWNv0l72Ju/ldMZpJCSCXIPoFtyN++vcT4RPhDjltSOSJJG/Ywepn35K8ekzeD31FP7jXkWtL8VgCTtXaDBfT4iLSMouJPl6UpyYVUhKdhFpecVk3SJBBvBw0uHn5oCvq54AVy3h+mxqqdMJIAMfcxqehqu4FF/FsSAZbV4iqqLsfy+icwZXfzkZvvk24O8fc/GTE2m9C6hUSJJE5uKvSZ0xA7Vej+/wF/AeOBCVlf6/WyQLF65d4EDKgZvJcK4hFwC9Wk8D7wZE+EQQ4RNBfa/6hLqH4qp3tUosgiBULKskvyqV6h5gJqABvpIk6YP/er4tkt8zyTms/WI0o3XLUb0WK7/UJ1hVgbGA2OxYLmRd4EzGGY6lHeNc5jlMkgmtSkukXyTtaraje3B36nvVFwmvnZGMRvJ27SJjzlwKjx1DV7Mm/q+9hnsv27Yzq64MJgsZ+cWk5f75SM+7/n7ejf82kJpT9Lf643/y0Rlo4JhNiEMutfS5BKqz8VNl4WW5hof5Gq7GDJyNGTgYsm69gEYvJ8HO3uDkTXGBE6kbr5J3Jh2dnztePaPw6NkJbVAoOLiDgxs4uoPeDRQqjwA5GY7Pied0xmn5kXmaMxlnyDPm3XyOv5M/oR6hhHmEEeoeSm332tR0qUmgayAuOhfFYhEEwboUT35VKpUGOA/0BK4AB4DHJUk6fbuvsUXye+ByJqYF99HUF1xfEvW+Sik2F5OSn3LzEZcTR0xWDDFZMVzJvYJ0vVepk9aJpr5NaebXjOb+zYkKiBJN6u2Q6do1Co8eJXfTJvI2bMScnY02MBDfoUPxfKif1U79hNIpNpnJKjByrcDAtXwjWQUGrt38b/l9+WPy+zmFRnKLTDfbuwHoMeJDDn6qLPxUWfiqcvAkDz9NPr6afHw1eXip8vCQ8nCTclBdKSLzlDOF6Q6gknANLMYlsAgXfwN6dxMqFZh0rlh0bkgO8kPl6IHK0Q21owcaJw9Uehe5Rll3/aF3lsdK65z+fP+vn9c5/a1cwyJZSMhNICYrhtjsWGKzY7mcc5nYrFhyjbl/+zVy17sT6BJIoGsgAc4B+Dj54OPo87e33o7eOGudxQ/2gmBjt0t+yzMWqw0QI0nSpesb/AD0BW6b/NpCQUE+7dQXyK75DOKFrL8zW8wUmYsoNBX+v737jZGjruM4/v7O7Ozt3l1bLi000FILpShV/qgo/kXERP48gBj/4T8SYkKMYnyIPlETHqgPNMYoEkMIMTHyQIjWBCE+UNAAKRIrWIlaobYVml7/3dG72+3OztcHMz3K/Z3r7s3u3XxeydzM7Px25rv33Z397m9nZplqTTEZTzIVp+PxU+OMN8c50TzBieYJxppjjDXHGJ0a5dDEIY41jr1hXaGFbF27lR3rd3DLtlvYfs52Lhm5hM3DmwmDpV3YXlYWd8cbDdpjY7SPHyc+eozWK/+jdeAgpw4coLFnD60D6Zn4wdAQw9dfz9obPsrQtdfqEIc+M1AJ2bg2ZOPa2pLu12i1OdmMea0R81qjNWOcDocaLfZmbcYbLU42Y6ZOtZnaFmMXnuTCo//h/Xt3cdXBf7PulfQybs1qhYm1dU6trZIMG5XBhGrtJIP1owxFTdZEU6wJJxkMTi0p3gSjFdSIgxpxMEA7iBgKqlweVNkRDpAEVTyIaIfncTw8n1crzmjgHA7bjMYtRseOs2/sEM/5FK/53NsOMIbDGkNhnTVhneHKIEOVQdZEQ6ypDDNcHaIe1qhHderRELVKjXo0yGB1MJ2PhhiIBonCAaKwSiWsEoUDVMKISlChYhXtW0XOUifF7ybgwBnzB4FrOgun+0Yf+wF/eHo9yXNPEPzifQDTvZLT5u389gXa+Kwpm7P5Ij3rPmtinrv49N+F+hJO9+RbNp3gaXFCko19eswivf4hsB44z0JCq1AJQqIgohpWqQYbqYbVdD6oUg0jzALgUDb8GSd7guT5dqHANrPyP7tBYbF0o82ijydt1J1YkoSk2cAbTbzRIGmm4zlVKkSbLqB22WWMfPpT1N52OfW3X0UwMJAjGFlJalFILQrZMNxZbt09LaT37Wdi1y7sr7up7nsZ37+P4KXxN7RtYRxjkKM2RLs6QFwdII6quBluTA/pK8SzHbRjOFhC4AkBCeYJhmMkBEwSMIHhBDiBJdRwtuFsn26TLju9H06AlkHLLBvS6TZGbA3ajBEbtMluNyMG2mf0CsfAyWxY4n8MI93fB9kYf/09YuZ7hc0zDWBuudoVq5itq39+eTWHIz7+8O5eh/EGnRS/cz1fZr17mtmdwJ0AW7Zs6WBzZydqN7BWQIUEay3QOzDr0disObeZLRZ5ydisiYUb52xv2KxY5l5NQGCGYdiZYwswjADDgoDQQgILCOz16TAIp3sXgsV6F/J8tZenSa6vCHOtqPM2XXpMedaz6PMoz3q68ZhzbQeCgRpWG0jH9dr0OFy3jnBkhMrICNEFF1DZuHHJP2cr5WZm1KsV6pdezLmXXgyfvw1Ii+L2iRPEh0eJR0dpHz1Ce2ICn5wkmZwkmcjGk5N4uw3tGG8n0G7jyRzjeT7opR0D6WL3rKPA0ze2eHrZ6dsckgT3JBunhXXk6QAJZOux6c4GJ/21k9PrTtKOiaxzIu2sSLJOiuR02Q5zTE3PZ4/FLY0rfbOafkQz3pQXn5v5n3GYva/zeWe6bPlPyC/ulP/yiqP5zyXolU6O+X0v8G13vyGb/waAu39nvvv06kcuRERERKRc5jvmt5NTaJ8FtpvZRWZWBW4DdnawPhERERGRZXXWhz24e2xmdwGPkx4e+oC77+laZCIiIiIiXdbJMb+4+6PAo12KRURERERkWXXvyuEiIiIiIn1Oxa+IiIiIlIaKXxEREREpDRW/IiIiIlIaKn5FREREpDRU/IqIiIhIaaj4FREREZHSUPErIiIiIqWh4ldERERESkPFr4iIiIiUhopfERERESkNFb8iIiIiUhoqfkVERESkNFT8ioiIiEhpqPgVERERkdIwdy9uY2ajwH8L2+DrNgBHerBdKZbyXA7Kczkoz6ufclwOvczzm9z93Jk3Flr89oqZ/cXdr+51HLK8lOdyUJ7LQXle/ZTjcujHPOuwBxEREREpDRW/IiIiIlIaZSl+f9brAKQQynM5KM/loDyvfspxOfRdnktxzK+IiIiICJSn51dEREREZHUVv2Z2o5n908z2mtnX51huZvajbPnzZvaOXsQpncmR589l+X3ezJ4ysyt7Ead0ZrE8n9HuXWbWNrNPFBmfdC5Pjs3sOjPbbWZ7zOyJomOUzuXYZ68zs9+a2d+yPN/Rizjl7JnZA2Z22Mz+Ps/yvqq/Vk3xa2Yh8BPgJmAH8Bkz2zGj2U3A9my4E/hpoUFKx3Lm+WXgQ+5+BXAPfXi8kSwsZ55Pt/se8HixEUqn8uTYzM4B7gVucfe3Ap8sPFDpSM7X8leAf7j7lcB1wPfNrFpooNKpB4EbF1jeV/XXqil+gXcDe939JXc/BTwE3Dqjza3Azz31DHCOmZ1fdKDSkUXz7O5PufvxbPYZYHPBMUrn8ryeAb4KPAwcLjI46Yo8Of4s8Ii77wdwd+V55cmTZwfWmJkBw8AxIC42TOmEuz9Jmrf59FX9tZqK303AgTPmD2a3LbWN9Lel5vCLwO+WNSJZDovm2cw2AR8D7iswLumePK/lS4ERM/ujmT1nZrcXFp10S548/xi4DHgFeAH4mrsnxYQnBemr+qvSqw0vA5vjtpmXssjTRvpb7hya2YdJi98PLGtEshzy5PmHwN3u3k47jGSFyZPjCvBO4CNAHXjazJ5x938td3DSNXnyfAOwG7ge2Ab83sz+5O7jyx2cFKav6q/VVPweBC48Y34z6afIpbaR/pYrh2Z2BXA/cJO7Hy0oNumePHm+GngoK3w3ADebWezuvy4mROlQ3n32EXefACbM7EngSkDF78qRJ893AN/19Nqre83sZeAtwK5iQpQC9FX9tZoOe3gW2G5mF2UHyt8G7JzRZidwe3bW4XuAMXd/tehApSOL5tnMtgCPAF9QD9GKtWie3f0id9/q7luBXwFfVuG7ouTZZ/8G+KCZVcxsELgGeLHgOKUzefK8n7R3HzPbCLwZeKnQKGW59VX9tWp6ft09NrO7SM/6DoEH3H2PmX0pW34f8ChwM7AXmCT9tCkrSM48fxNYD9yb9QrG7n51r2KWpcuZZ1nB8uTY3V80s8eA54EEuN/d57yUkvSnnK/le4AHzewF0q/H73b3Iz0LWpbMzH5JeqWODWZ2EPgWEEF/1l/6hTcRERERKY3VdNiDiIiIiMiCVPyKiIiISGmo+BURERGR0lDxKyIiIiKloeJXREREREpDxa+IiIiIlIaKXxEREREpDRW/IiIiIlIa/wea9nJBvC4LOQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "from scipy.stats import beta\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "n = 10\n",
    "k = 2\n",
    "\n",
    "x = np.linspace(0, 1, 200)\n",
    "\n",
    "plt.figure(figsize=(12, 8))\n",
    "for a, b in [(1/2, 1/2), (1, 1), (10, 10), (100, 100)]:\n",
    "    plt.plot(x, beta.pdf(x, k + a - 1, n - k + b - 1), label='prior = Beta(' + str(a) + ', ' + str(b) +')')\n",
    "    plt.legend()\n",
    "\n",
    "plt.show();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Exercise 12.11.6**.  Let $X_1, \\dots, X_n \\sim \\text{Poisson}(\\lambda)$.\n",
    "\n",
    "**(a)** Let $\\lambda \\sim \\text{Gamma}(\\alpha, \\beta)$ be the prior.  Show that the posterior is also a Gamma.  Find the posterior mean.\n",
    "\n",
    "**(b)** Find the Jeffreys' prior.  Find the posterior."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Solution**.\n",
    "\n",
    "**(a)**  The posterior is proportional to the likelihood times the prior:\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "f(\\lambda | X^n) &\\propto \\mathcal{L}(\\lambda | X^n) f(\\lambda) \\\\\n",
    "&= \\left(\\prod_{i=1}^n \\frac{\\lambda^{X_i}e^{-\\lambda}}{X_i!} \\right) \\frac{\\beta^{\\alpha}}{\\Gamma(\\alpha)}\\lambda^{\\alpha - 1}e^{-\\beta \\lambda} \\\\\n",
    "& \\propto \\lambda^{\\sum_i X_i} e^{-n\\lambda} \\lambda^{\\alpha - 1}e^{-\\beta \\lambda} \\\\\n",
    "&= \\lambda^{\\left(\\alpha + \\sum_i X_i \\right) - 1} e^{-(\\beta + n) \\lambda}\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "Therefore, the posterior is proportional to a Gamma distribution with parameters $\\left(\\alpha + \\sum_{i=1}^n X_i, \\beta + n\\right)$ -- and thus it must be drawn from that distribution.\n",
    "\n",
    "$$\\lambda | X^n \\sim \\text{Gamma}\\left(\\alpha + \\sum_{i=1}^n X_i, \\beta + n \\right)$$\n",
    "\n",
    "The posterior mean is the mean of Gamma distribution it is drawn from,\n",
    "\n",
    "$$ \\overline{\\lambda} | X^n = \\frac{\\alpha + \\sum_{i=1}^n X_i}{\\beta + n} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(b)** For a Poisson distribution, the log likelihood is\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "\\ell_n(\\lambda) &= \\sum_{i=1}^n \\log \\left( \\frac{\\lambda^{X_i} e^{-\\lambda}}{X_i!}\\right)  \\\\\n",
    "&= \\sum_{i=1}^n \\left( X_i \\log \\lambda - \\lambda - \\log (X_i!)\\right) \\\\\n",
    "&= \\log \\lambda \\left( \\sum_{i=1}^n X_i \\right) - n\\lambda - \\sum_{i=1}^n \\log (X_i!)\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "Its second derivative is $\\ell_n''(\\lambda) = -n \\overline{X} / \\lambda^2$, so Fisher's information is $I(\\lambda) = -\\mathbb{E}[\\ell''(\\lambda)] = 1 / \\lambda$. \n",
    "\n",
    "Jeffreys' prior is:\n",
    "\n",
    "$$f(\\lambda) \\propto I(\\lambda)^{1/2} = \\lambda^{-1/2}$$\n",
    "\n",
    "The posterior is proportional to the likelihood times the prior:\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "f(\\lambda | X^n) &\\propto \\mathcal{L}(\\lambda | X^n) f(\\lambda) \\\\\n",
    "&\\propto \\left(\\prod_{i=1}^n \\frac{\\lambda^{X_i}e^{-\\lambda}}{X_i!} \\right) \\lambda^{-1/2} \\\\\n",
    "&\\propto \\lambda^{\\left(1/2 + \\sum_i X_i \\right) - 1} e^{-n\\lambda}\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "Therefore, the posterior is proportional to a Gamma distribution with parameters $\\left(\\left(1/2 + \\sum_{i=1}^n X_i \\right), n\\right)$ -- and thus it must be drawn from that distribution.\n",
    "\n",
    "$$\\lambda | X^n \\sim \\text{Gamma}\\left(\\frac{1}{2} + \\sum_{i=1}^n X_i, n \\right)$$"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}