example/bayesian-methods/sgld.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Stochastic Gradient Langevin Dynamics in MXNet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we will show how to replicate the toy example in the paper <a name=\"ref-1\"/>[(Welling and Teh, 2011)](#cite-welling2011bayesian). Here we have observed 20 instances from a mixture of Gaussians with tied means:\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\theta_1 &\\sim N(0, \\sigma_1^2)\\\\\n",
    "\\theta_2 &\\sim N(0, \\sigma_2^2)\\\\\n",
    "x_i &\\sim \\frac{1}{2}N(0, \\sigma_x^2) + \\frac{1}{2}N(\\theta_1 + \\theta_2, \\sigma_x^2)\n",
    "\\end{aligned}\n",
    "$$\n",
    "We are asked to draw samples from the posterior distribution $p(\\theta_1, \\theta_2 \\mid X)$. In the following, we will use stochastic gradient langevin dynamics (SGLD) to do the sampling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mxnet as mx\n",
    "import mxnet.ndarray as nd\n",
    "import numpy\n",
    "import logging\n",
    "import time\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "def load_synthetic(theta1, theta2, sigmax, num=20):\n",
    "    flag = numpy.random.randint(0, 2, (num,))\n",
    "    X = flag * numpy.random.normal(theta1, sigmax, (num, )) \\\n",
    "                    + (1.0 - flag) * numpy.random.normal(theta1 + theta2, sigmax, (num, ))\n",
    "    return X.astype('float32')\n",
    "\n",
    "\n",
    "class SGLDScheduler(mx.lr_scheduler.LRScheduler):\n",
    "    def __init__(self, begin_rate, end_rate, total_iter_num, factor):\n",
    "        super(SGLDScheduler, self).__init__()\n",
    "        if factor >= 1.0:\n",
    "            raise ValueError(\"Factor must be less than 1 to make lr reduce\")\n",
    "        self.begin_rate = begin_rate\n",
    "        self.end_rate = end_rate\n",
    "        self.total_iter_num = total_iter_num\n",
    "        self.factor = factor\n",
    "        self.b = (total_iter_num - 1.0) / ((begin_rate / end_rate) ** (1.0 / factor) - 1.0)\n",
    "        self.a = begin_rate / (self.b ** (-factor))\n",
    "        self.count = 0\n",
    "\n",
    "    def __call__(self, num_update):\n",
    "        self.base_lr = self.a * ((self.b + num_update) ** (-self.factor))\n",
    "        self.count += 1\n",
    "        return self.base_lr\n",
    "\n",
    "\n",
    "def synthetic_grad(X, theta, sigma1, sigma2, sigmax, rescale_grad=1.0, grad=None):\n",
    "    if grad is None:\n",
    "        grad = nd.empty(theta.shape, theta.context)\n",
    "    theta1 = theta.asnumpy()[0]\n",
    "    theta2 = theta.asnumpy()[1]\n",
    "    v1 = sigma1 **2\n",
    "    v2 = sigma2 **2\n",
    "    vx = sigmax **2\n",
    "    denominator = numpy.exp(-(X - theta1)**2/(2*vx)) + numpy.exp(-(X - theta1 - theta2)**2/(2*vx))\n",
    "    grad_npy = numpy.zeros(theta.shape)\n",
    "    grad_npy[0] = -rescale_grad*((numpy.exp(-(X - theta1)**2/(2*vx))*(X - theta1)/vx\n",
    "                + numpy.exp(-(X - theta1 - theta2)**2/(2*vx))*(X - theta1-theta2)/vx)/denominator).sum()\\\n",
    "                + theta1/v1\n",
    "    grad_npy[1] = -rescale_grad*((numpy.exp(-(X - theta1 - theta2)**2/(2*vx))*(X - theta1-theta2)/vx)/denominator).sum()\\\n",
    "                + theta2/v2\n",
    "    grad[:] = grad_npy\n",
    "    return grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We first write the generation process. In the paper, the data instances are generated with the following parameter, $\\theta_1^2=10, \\theta_2^2=1, \\theta_x^2=2$.\n",
    "Also, we need to write a new learning rate schedule as described in the paper $\\epsilon_t = a(b+t)^{-r}$\n",
    "and calculate the gradient. After these preparations, we can go on with the sampling process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter:100000, Time spent: 21.831000\n",
      "Iter:200000, Time spent: 21.752000\n",
      "Iter:300000, Time spent: 21.982000\n",
      "Iter:400000, Time spent: 21.866000\n",
      "Iter:500000, Time spent: 21.835000\n",
      "Iter:600000, Time spent: 21.917000\n",
      "Iter:700000, Time spent: 22.352000\n",
      "Iter:800000, Time spent: 21.952000\n",
      "Iter:900000, Time spent: 21.885000\n",
      "Iter:1000000, Time spent: 21.919000\n"
     ]
    }
   ],
   "source": [
    "numpy.random.seed(100)\n",
    "mx.random.seed(100)\n",
    "\n",
    "theta1 = 0\n",
    "theta2 = 1\n",
    "sigma1 = numpy.sqrt(10)\n",
    "sigma2 = 1\n",
    "sigmax = numpy.sqrt(2)\n",
    "X = load_synthetic(theta1=theta1, theta2=theta2, sigmax=sigmax, num=100)\n",
    "minibatch_size = 1\n",
    "total_iter_num = 1000000\n",
    "lr_scheduler = SGLDScheduler(begin_rate=0.01, end_rate=0.0001, total_iter_num=total_iter_num,\n",
    "                             factor=0.55)\n",
    "optimizer = mx.optimizer.create('sgld',\n",
    "                        learning_rate=None,\n",
    "                        rescale_grad=1.0,\n",
    "                        lr_scheduler=lr_scheduler,\n",
    "                        wd=0)\n",
    "updater = mx.optimizer.get_updater(optimizer)\n",
    "theta = mx.random.normal(0, 1, (2,), mx.cpu())\n",
    "grad = nd.empty((2,), mx.cpu())\n",
    "samples = numpy.zeros((2, total_iter_num))\n",
    "start = time.time()\n",
    "for i in xrange(total_iter_num):\n",
    "    if (i+1)%100000 == 0:\n",
    "        end = time.time()\n",
    "        print \"Iter:%d, Time spent: %f\" %(i + 1, end-start)\n",
    "        start = time.time()\n",
    "    ind = numpy.random.randint(0, X.shape[0])\n",
    "    synthetic_grad(X[ind], theta, sigma1, sigma2, sigmax, rescale_grad=\n",
    "                            X.shape[0] / float(minibatch_size), grad=grad)\n",
    "    updater('theta', grad, theta)\n",
    "    samples[:, i] = theta.asnumpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have collected 1000000 samples in the **samples** variable. Now we can draw the density plot. For more about SGLD, the original paper and <a name=\"ref-2\"/>[(Neal, 2011)](#cite-neal2011mcmc) are good references."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVwAAAD/CAYAAAC96W7YAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvX14XFd57v1bkWR9y6qkWIolxeNYTmzHJk5iYpMEbCAQ\natJ8lNBAT0qBw9X2UA6UAy0Jp6cl5zov0Ja3UPrdk0IgeUlIQ5OSEHAwoBATYnAcBxvbwUo8wbKR\nHEkdS4okWyOv949nPbPWHkuyZX1Yltd9XXPNXnuvvfeaPTP3fva9ng9jrSUiIiIiYvpx3pkeQERE\nRMS5gki4ERERETOESLgRERERM4RIuBEREREzhEi4ERERETOESLgRERERM4RJEa4xpsQYs9UYs8MY\ns9sY85mpGlhERETEXIOZrB+uMabMWjtgjCkEtgAft9ZumZLRRURERMwhTFpSsNYOuMV5QAHQM9lj\nRkRERMxFFE72AMaY84DtwBLgH621u5PbPxVD2SIiIk4Z1n7KnOkxTBcmLSnkDmTMfGATcIe1tjVY\nHwk3IiLilGGtPW3CPR2+mcz5JopJW7gKa+0RY8y3gDVAa3Lr+mA55V7TiVZgw2nsVwQMj7I8VWjl\n9MY13WgljmsiaCWOayJoZexxpd1L8eSkz/Z/JtD3Tyd9tolhUoRrjKkDstbajDGmFHgLcNeJPTdM\n5jQziOExliMiIqYHKZIG2OQJt2jSR5g+TNbCvQD4itNxzwPutdZ+b/LDioiIiDg9TNlj+zRgUmOz\n1u4ErpiisUwhUmd6AGMgdaYHMAZSZ3oAYyB1pgcwBlJnegBjIHWmBzAGUjN6ttIZPdvEMJtvBpNA\nagJ99QFkJiSE1Ayc43SQOtMDGAOpMz2AMZA60wMYA6kzPYAxkJrRs81lSWEOIGq1ERFzCbOZ1Gbz\n2CIiIiImjGjhRkRERMwQZjOpzeaxRUREREwY0cKNiIiImCFEwo2IiIiYIUS3sIiIiIgZwmwmtdk8\ntoiIiIgJI0oKERERETOE2Uxqs3lsERERERNGtHAjRsGppH+scu+90zyWiIi5g9lMarN5bHMMNcBg\n0M7i51OzyFeh21PAQeLXExExcUQLd84g/ConmoMhJNhBt6yWa2ne9h63rOXhZjLBTkTE2Y3Z7BY2\n6SKSZz+K3Ks0rz3afXI4eJ0O+txrMG/9oDvmYLCcnww9km1ExKmgaAKvsWCMKTDGPGeMedS1rzLG\n/MSt+6kx5rVB3zuNMfuMMXuNMW8db2znuIUb6qjDyGN/X952RfjIP97x9FhhuzB4AVTmnUf7KumH\n1m9ERMREMEWk9hFgN/JnBfhL4H9ZazcZY37dtd9ojFkB3AasABqBzcaYi621x0c76Dlu4ZbmLfeN\nsS1cl39vLA3W56PSvUqBemQSrAovH+S/1JJVS/dk9+GIiIh8FBWe+ms0GGOagI3A3YAWmPwVMN8t\nVyOTLAA3Afdba4ettWmgDbhqrLGd4xZuRETEXEPhRFgtO+razwN/jHcTArgD2GKM+RxiqL7OrV8I\nPBP0a0cs3dHHNoGhzUGEj+06kaWyQSnJy1OFSA6Kg4xfdLKIpFuXHh93XPVMAG/x6jEq3fLovwZ/\n/KjrRkTko6jg9Pc1xtwAHLbWPmeM2RBs+lfgw9bah40x7wS+hBTNHQ1jlmo/xwkXkrrrYF47JMVe\nt6yeA9mg31jQvpWcqP+GOm7W9VVC176FwXu4fyTbiIixMJ6F+2RWXuPgauBGY8xGoASoMsbcC1xl\nrb3O9XkIkRtALK/mYP8mvNxwAoy1Y5LxlMAYY+HPp/Uck0PoCxu2R7NYBzlxQmw04isieS/L98EN\nz6fnDC3aXryrmKIwOKe6jEXSjZhruAtrrTl5v9FhjLF2wQT6H2bM8xlj1gMft9b+hjFmO/BRa+2T\nxpg3A5+11r7WTZp9DdFtG4HNQIsdg1ijhZsjQrUa9ZKE8gKc6GmgBFzv2r15y+Gxs8h3kU+gagEr\nyYc3xtDazh9rRETEmJhaVlPi/D3g740xxcgf8fcArLW7jTEPIh4NWeCDY5EtRAs3D6FlGhKxopCk\njj6I18cL8/bpIanRqtsZwFIgnXfctqCdDdbjzplvXYfkGy3diLmCKbBwL5xA/1+ObeFOB85hC1ct\n2rHCaxX5lyicVGsM9u/Fa7KDwCI8cTYi4bpp1+5x6/RYnXCCT3BI9L2caO3mT/Dl689R5404RzGL\nWW0WD22qEBJVDf4xPpQHdFsoESgJhp4JIMQJMumlk116Hj3ecqAbeI1rq3/tNa7dhpCkWrVLScoN\nah0rodYgVm449p6gv/ZTH221qBWReCPOIUzCS2G6cQ4Qbkg2vXgC1fVqoeZn5krl7Tvo+uj2boRk\nW4L9twfHLgL2uHYLos8q+TdCYRlkdSwHgSuCY/e5Yyj5p0m6leVPwqnEEN5cQgkiIuIcwixmtVk8\ntOlASFI1iCWolqValaGnQBFJH+YU3rLMJ73GoO8ekhbvYqgAhlwza52BXSbtoaWI5h4SailQG4wt\nFZwvjZCx9q9HyD5MhhPJNuIcRfGZHsDYOMcINyIiYs5jFrPaLB7adCG0aKvwlqBavKEeGkaL9SAW\npEoInfJWuELeG5CgPoDUCjlUnWtngH68hVtt4Hb8HNpeIL0CsvvdioNuPN2uvQh4GVAHwxqSUXJ7\n8PoteOs8WrkR5yBmMavN4qFNNVTfVN1UH7tDX9lCZAILvIbqSK5wLWT3kSNaSqFwsb+CGYR0QeJT\nANa492qEVHWOrAF4LG942W6gLBhbLTQ5Dbd9P7CM5ARdLZ7hR0v5GMk24hzFXJ00M8Y0A19FWMkC\n/2Kt/eJUDGxqEOqoatmGrlRFeMtwmGRwwnIShJztRKxbddnbB9leWOks4Aa88bsaIdnWYCgptw6g\nA1gHfMe1+4GSWm8BK9m3D7h2vRurEvKVwDdIRseF+nPMnxtxDmMWm5GTHdowEu62wxhTATxrjPmu\ntXbPyXacGeQTTw3e0h0k6YkwTDJY4SByH2ly7W6gD+ocwRYuFYJUkl0X7LoMaBqGaneuRxBrVyWG\nR4AdeH5MIdavWsZDTY4/lWDb3TjUa8GNJXezGMRXidB2RMQ5irlKuNbaDsRew1rbb4zZg6QrmyWE\nG+qYRSR9XevxOQtw21L4x3bVS5Vgq+A6YJtbXYiQqB6yDs/Njh8r1r0CwLwNxwDo+Y7zYrjW9Q+H\nk0JkCfCWcL97r2iCDou3rtXa1lBgjWwLXd1ispuIcxRzlXBDGGNSwOXA1qk65uSQH5lViJBoKC0M\n4jXdKjyRkbcNsT534XXaJnfI2127g5yFetNt97PpyPU0lx8AoM9Zoj16tVeSJNgOYAPwgGtXk7R4\nlwF7DaR1oqwIuWGEfrzg9eX8axBa+icj3/ztJ2tHRMwyzHW3MCcnPAR8xFrbf7L+M4P8rF6lwGE8\nSfUiE2RhaqEmYJ9bTsk+am2uRCa+drl2B/ABvBVaTY5Ad3A5Q+ka9qekPZSukSutGu0QNK3fR/V6\n2WHXC6+loukV+pedrweQ425xn6G/yO0bVvVtx3sq6E1EI9v2kUwt2YifsSskaQGfrHRQJNeIswxz\n2cI1xhQhMzj3WWsfGb1Xa7CcwofHTjfCOmFKKmFwQBrvm5UiadU2Ad2eUHcgpKo6rMoJaqVqpkwg\nc7SalZf9lKt5GoCBy8poZQPtQ0tzfdoPNtPeJe3zGl7l6FCx/zaGkIm3rLtZVCD8mq732wGJTgNR\ncPbgD6CBE5rb4SCwyi0HXhaA137D6zQeyUYCjphKpEkmcpoCzGEvBYNkQt9trf3C2D03TOY0ERER\ncxYpkgbYk5M/5By2cK9BVMyfGWOec+vutNZ+Z5x9ZgijJQgvxQcy6KN02K8G7wkA0AlZ179/WEJx\nVcNNITKDttvJXc0j2xo4km6g789Fux2gjFeev5CilFjXw9+pgpISippc+5kqji+zbvoRsZxL8HLG\nNvK+qXbERSzt2rVuQDp2Deg4HHxuncccJpliUgM8FD1EnTbirMZcJVxr7RZmbeXf/BSHIEymj9RV\nCNmoa0F+CsT9+FwGwLoy2V1JMO2WtV2N13eHgBS8/DfLAFj0kb1cdNnPeelFF5WWAjbD8Dohvje9\n/TG+v+mG3HxdyQ09HBsq5vh95W4FQsZhqbpEcpsykh4XYZIebWueh/zyPX2c6M0RyTbiLMYslhTm\ncALyfOKocS+1+hYE/UDISsN7Aa6AkqpksMLb8EZlFuHqa127AW+hQqCzSp9rb/ouW17eIO1tRRRt\n6GVh7SEAKumjkj66HMEfPlrPquKdbDtypRyq69fg74wfWpd7KQEPkXQxYx9i0Ya6dRjQMQhoUIXe\nhEINN0tS645+vREzhSlIQP6BCfS/e/QE5MaYAuTZst2V2Pkr4AbgGPAi8D5r7RHX907g/cAIUmjy\nibHON4uN78kiP/0ieLIFP1kUZv9S0gUYhKEq75rVAGxBAhxAJs/qEOIDsW5Dt68W4Dp3M8saXuAS\naprk/GWLBqilm0o3qXWMeaxiJwuc9Z0uXszTXM3a+T8B4IX5F9PxgYs82e9CVIXQ8SANZJ91KzTf\nrlrAoYzQ6V75RTDzK1zkp6uMiDhLUHLyLqeAjyAp/DS66AngE9ba48aYzwJ3Ane4mma3AStwNc2M\nMRdba4+PdtA5TLhKHNngPSRYlRzGqlHmLOC0a6o1q14JaxDLUr0WlgWnbpH1NSlnwRb0sYQ2DhfI\nsZs5QDMHSLmDF3OUBRzOEfBPWMsatjHgQnkv5ACXXPoLDl8qY9rTdYV4LqgHRcZ9vKxYxLRfCTwL\nhVcGl0Dd3dJ4n2PwRJtfvieUJ3R9iCg7RMxSTFJSMMY0ARuB/wf4HwDW2u8GXbYC73DLNwH3W2uH\ngbQxpg0pKJkQABVzmHDzNVx1/lcLNgzjBZ/MpifYvtwfQ1PnVrjNa5D6nNouwU+2lkDT2n2sdTEg\npQySoZr/w58CYtEWMMJR5gGwmxV0UUcxRwGZZLuAQ4y4X847eIjHeTudI+5msBqxcFW22OHeK4L3\n/it9OzOMT8qTQn4vKin04mum6QcNyTQsBQQ+OXpExCzF5Fnt88Afk5xdDvF+4H63vJAkubaTTKI9\nxUOb1VAnfzixzHklQqpq0Q4izrRq0dUjVqDr39EkfrEqKXwHWVayy5C7mjXvPcjASBl9Bd5LoZI+\nDrjy9TtZxSp20oxEor2RVsoYoC0X5QZLeJFnEQv1MPXcykNcXiCOIF9d+R56Mo3eeG9AtGZ9lOpH\nrPEterQifPFRLRukN5alwM/wv638SbUsyUoXkCzhHsk3YpZhEqxmjLkBOGytfc4Ys2GU7f8TOGat\n/do4hxlzYmyOE274aDxIMqpKJQYlkkIkQEMtwb1IHloNHlgseq1KCstcl5Rrt5FzEVtd8ByHqSfr\nLNSfsYoPcHduVBt5nAzVOYv2RZbwI67OSQoA1WR4Bw8BsInruYQX2OmCF1YU7Ca9foD2ajfWCgtD\nJjmWQkTiByHgB9y8QDvQvwKGtPNOxJIPr0M4aZaPyuCaQJxUi5h1GEdSaH1ZXuPgauBGY8xGxISp\nMsZ81Vr7HmPMexGp4c1B/4PgLClBEz7W/gTMccKNiIg45zAOq21YIi/FXVuS2621nwQ+CWCMWQ98\n3JHt2xCZYb21NvRB+ibwNWPMXyNSwlLgJ6cxtLMV+bptGMKqVq5uC3VefUxOu3YnkgZRfXG/CyyH\nvc5vtw3J4PUBZzkGrlkjFHI9m9jmMpD/Ghme4HqWsxuAejopYyC3/RYe5iq28jRXA3CACylghG9w\nKwAXcChn3QKkSJOhmosvkyzmC/kVrX+7gfbnncW7DjHWNT95Ez4nzxBiibdpPbUroK4IuvSXp5Ut\nVOuuwddo0+sUoo+IiFmFqWM1g5cH/haYB3xXAmz5sbX2g9ba3caYBxGPhizwQTuOr+0c9sOFJDmo\nvKDEkV+Gpgr5plTv7iNZjFHL7ag7wmIhMa3qcAM5j4WiNb3U13bmJAL1SNDcCr995EEA7p3/bgAW\n0Ekd3az7t+flAOvgYHNNjmTftu1JmA/fX/o6QCSG69lExjkJ38FnqKObHx8Uwj6vcITjQ/OgzX3W\navwE2udcW2MBS3ApJzXJWy9JrbsHcafTa6k3qfBGBqeu5cbAiojxMAV+uH8xgf6fGN0Pd7owxyzc\nkFDhxLBenQACr1FqW93teoL+msIR5FLVQ8liaaYQslLiqiZHvsOZKtqvncf6xlZANNyLeYFO52q2\naf6bKGSEQywE4Eq2UciI/zaegwVVPVw/4uLKHwDKIXWX1Dy7ja9TzNEc4b6BpwA42iheD31Ucoxi\nMg2yvbo4w4FOkZmOUy4TfUrA7XqdlrsVaXcN0q6dRVzkeoK27hO+nwz6vRROYJ+IiNPALGa1WTy0\niSKs5ACju4WFfrihxQZixbWQnAAqJHmJemHIEfXeKtm00m26Bx+I0ASkS3jy9g3Sbi/hwNpmVjhJ\nYQW7aT7SwdsOCKH2Li+is6CefbeIXLH0nnYOz6+hcYcb60tAOVx0nwtlu6YDOx8uPfQSAAUrRxih\ngP1u1qySftKk6MvITeRIpiEocIlY4q2uPaTXLl8yUEu/B59fAbwLWRgocSoEmu/XGxExTZjFrDaL\nhzZRjFbHKz9x9uAo20LLK8ynsJxkGRuNUvuRe18A2dfADtd/DT6XAogHw3XOT6thmF+xkEoXqdBN\nHRc93sFL7xa3hov+voOq9e0+cfJGaKOFo6t/Kdv3dNC5F9rvk81XbgTzEXIuaW9c/gMOFSzk7TwO\nQCf19FHB0XqxeDseuMh/0x2IxZxy7Qqgqzv43EvdRv2cmqMh9F+eiIWqhNybt04RejlEyzdiCjCL\ncynMIcI9GfL/zEoE+ZFm+ufXicawIkQ7/pFaAwa0sm6V35RBJAZHkLytiM5F9RziAkAmzYY3wqCL\nJDv4hzU07uthy1LJb3vt3u00L/glx5SBb4T6Cljwkjveu2F4LRxbL3mDNhVcT5YC5iGlfI4xj0v4\nRU4z/rtbP0xJhQQ6DD1Wk0ykngGZGHSBECU4q1evy1KSuRY0BFoJ9GSTZqOR6GhPHxERU4RZzGqz\neGjTjfwcAvm+pEoIYY2zQZKP2gERdQDXOT24BIkGU5/dbfDyyhRPNb7BnamM2vndXLHVpUx0VeD6\nl4o1vWXZFVx7eLuvn7YaOACm27X3QNFCOPQm8aC4+chjvDh/Uc7rQQMtlNAva9zB8y+7MN8diOSh\nYcH9CPmmnNdCGvzEmS6HCcqLXKdQyx1NVhiPUMNto/nwjpZaMyLiFDGLWW0WD202ILw8muwmJIuD\n+JwETdC6VhbXIXykmcS6gLYSHtvyTgAe37CRA/XNVK4V6/C/80Vohh85t7AbeVROF6gXvBMGvinN\nsn3AUli0VYpU2qUiI6xxDJ2hmgM0sxUZTxkDXLZIEts8v2ydz7erYxvCa7xYEmkpc6kf88kvLFUE\nJ8o3jLJ+vHUT2R4RMQ6ipDBbMFGXpDBBuV6qlHvXkuvDfnvWsVZHk0RX73Wb1KJ07tLHV5aTqa/m\nh7weEIt0Ddsoc9ZeLV1y+D9w+x8B/gbKbnTtl4C/B5zDRPb/wtpXva/11eVPk6GaFOLV0Mobc4ly\nXrj1Ykn32OA8YbYh1nmOcNVDRj+nWvV6Y+l028YK9dVrdbLos2jFRkwTpiZb2LTgHCPciIiIOY9Z\nzGqzeGjTgYlYU/kTO5p7QUvVaORXyr2X+eIRGs2l0V0NSLxE2rW71JNANNtqMryVTblJsov+pkPk\nBLVolwHNYP9GmoNHoex2cr60RVuhaDkMOwn58p49mCF406EfA1C/5jALkVSRnfMX8Pj8t9NdIbLB\nccrFTUwTre8C0pZEXmB68OHh6lqn2/tcn8qgf/hkoAiT4YzmURIRMUWIksLZCPU3rQzWZfGPygcR\n0lEiKvSP5XWL5TFdCbgV0XVdYERRUy+dLODYiBDs0wVXk2I//+3AV6TDEwhpf8/t/+/AhWCukWbP\no1A2guf+ZUA5FLrYBfM48Crs+30ZQIZqnnLyxbv4OnV080L9JQBsu+lKOrZeJBNzAJ8CsgbaVTLQ\nZOZhYEh4TXQSMcyvO1q15JjgJmKGMItZbRYP7UwhP1eARqIdRMgldODPgkuhKG1HRFmEcNXLQNM4\nOk13eE0V+0ouy1mVx9ZLxMTPmy8CYMG3DnP+4/04oxT+GXEDfqs0m1YhbsIPuu2HgIfB3O7a+4A/\nJJdPdx7HKGAEgP+P/0KalOjEQMfBhaIva9BGBULg1U4gziyG9mEkhSMImXaSjNg7GFwX1XPzfZ6j\nZhsxQ5jFrDaLhzbT0Amw0CJTFyjwOWHD7YvIVVKoXurdwIYQcg0TxuzAJwx/DEm9667+7ldX8Lny\nP86F+l7IAVZv3MFrdzhn2W1IqoMDbv9lwP+G/e7Ui3chE2zq1XBUlps3yg5txUty8kUdXaTYzxNc\nD0BT4wHa+4Ox17lx6sTDM5Ak2Cp3XdSc1yg0nTTLDzDJJ9qYSyFimhElhdmK8M9/Kv6kIRYg7lPu\nuT4TkPVQH7DBJyevRqzI0BXrAeCz0ryt/OvsZgUXOkbNUsCanl3givZyEcJrb3LtC6H94zgvW2C9\ne3eSA0eB+6H8GSmrdMOa73PDuu8D8NfN/40+KlnCi+5THCbTVE3/rvNl31uRe4zmiMglolPC7Xaf\nPSwyqfo2eL/lkHRHy5kbLd6IaUL0UpgtyM8OBqdueelkUb42qRpuJ8mS689C+2ukeZ8jF928GrFS\n64TNnmM11/E9funyGB+jmM011/KWbkmZ2PuxIqqeGfbfVgE0fQyvZnwaeCc+c9khxCpWIgZxLQOW\nNL/I01yd89F96cUVkDG55Ol0IDcHPVYW6G8KqkcAbMdrt1lEbgjLsBeSDAWGZJY2iEQbMW2IFu5s\nQX7+Wxg/jj/sr2Qbzra3kSxNo8doRGbv9Rl/FZTU+jtvO7J8t6zYfu21bE9dw+1LpCrEdWymjSXs\nXCqeEP/j8D/Sc00JNY87c3MH8qPSvPPXSjpH1Wkbnjki1rFmXNwDQ1+Wxad4PQWM5Mr7HFsyj+qc\nngC7vvdakRe0OGYLcDc+u1h/p/t8YdKfepKW7TBJv9wwRwVEWSFiWjGLWW0WD206EP7RTzWPazgZ\n1BOsX4BMkinhhnXCOpEHflf5l0F5NNeS6m3IY7vWR+uH+U2dfL37NgDqaztZy1a2uciG2xbcw/Vs\n4rY3fx2A8tXHxYJVL4VaaCzu8fl1vg+8F/gn1/4DOFQuJuytPESWAj7b8ykAHqq5gS/zvhxZ71q2\nCgpLfOhvB8KdKdfe+xok70LarRjA138DH/QwlndCLMkTMc2Yxax23pkeQERERMSUonACrzwYY0qM\nMVuNMTuMMbuNMZ8Jtv13Y8weY8wuY8xfBOvvNMbsM8bsNca8dbyhzfGKD6PhdLNUqaWrFm0hsBZv\n6eVXlxgO+i6HiiZvJXYheq7qpmsQTXeNfBc1qUNUF2QocI/lb+dxUqRzFSRWsJt1e59n+AL3ifa5\n4Wjh5leRCTRn8W7/yHKueFjM4Z5bSuijMlfOp5ARChjh83wUkOoUjx/dyJEH3OAeQfRbtc5zUDex\nNKJj669XpZd8T4X8STZFlBYiQkxBxYedE+i/6sSKD8aYMmvtgDGmEPn1fxz5EX8S2GitHTbGnG+t\nfcUYswL4GvBaRGvbDFxsrT0+2vlmsfE9XchPsjJeO0QWnzMXREL4EUmi1YCAKk7w2a3GV+fJINqo\naroV7tCPyffec8NC6pd0sgr55VzAIRZyKJd+MUsBe5ctypXguXnpYxwrOY/yEvcdH0FkhY3SbOYA\n22+RqIgrHt8DQWrIeRzjaa6mzKVn7KaWgf4y5r9Lkp0faWsQ+UIj0erc+LvchGCuFL1KCvUkNdv8\nxOOD+KASOPGaR8khYpKYJKtZa12uUuYhsyX/CfwZ8Blr7bDr84rrcxNwv1ufNsa0AVfhHCqneGhn\nI8LJstEyYOVru0XBtjBhy2Hk8oUhrVpifRgRWJWUWqRrWNSxFbjZtbMIqenEVMaw58kreGGZRIM9\nXHgLa2q38XpXSmcjjzOPo5Q6knxx/iIyVLMu62qiLQd+k9xXfv6hfs4/7ATfNVDzoyHefM1m9ynq\nGaSU3awA4PX8kNW1O/gqvwPAkfdaSBs/tpw/sV6frLtGKdduw5OuXpcsubzBJ2Qey7/hhdavrotW\ncMQEUHzyLuPBGHMe4oqzBPhHa+3PjTEXA28wxnwa+Qd83Fq7DUmuGpJrO95l5wScg4Sr0ECGEJ2j\n9Asn2UKvBXV90gTcNch3pNsG8XJDL6SXQrsjnX6EV/RrakfkhZRrd8ghjt8gjrjH6+BQ7UJeROo7\n72QV3dTmvAs6qZfyPUfd/k8i1m3Wt63EOXCopobGx3uoXy036MLyEbqpzSUr38kqVrGT93AvAF9d\n8jt0tF/kvRYqkMm+h/Q6pNznfdm19YaWDd7DhOVVjH6d4fRSOUZE5GEcVmv9sbzGg5MDVhtj5gOb\njDEb3FF/zVq7zhjzWiTO86KxDnEaQzs1GGO+BLwdOGytXXWy/mce4yUcH+/PPda2MJG5Xs5ahEX1\nHH3AdshKRQc66oW4tOpCneuuOmmTe2l6x3Z4uWQZV18ipHiMeYxQwA/YAMBi0qziZ14y/t/wytIK\nzn/YuRq85JMuNh7tYd97m2h+Vczt6qMZdhev4DrE4q2kj4FAJqnnMB0fCH5X/+TGqhZvUxV0rYAu\nte734eOEFT3BtdGE5vkJzHU5ImKSGIfVNrxeXoq7vjB2X2vtEWPMt5BZlnYkqwnW2p8aY44bY+qQ\nCYzmYLcmvHP+CZj0pJkx5vWIzfbV0Qh39k2anQz5f/7SvG3ho3QN/tutD/qpXKGJvHvx5ivBcjAR\npyG1IF9ZEPpLC5Tc3sNQuwQT/NalX2Ejj+d013due0z0YFfklxrE2tVItVokh64u/x3wIdfWn8qF\n8jZUDp8r/xhfR1zUOqmnkj5e+tal0qEdl1HM7bcDKaSpeSO6OhHSzZ+5CP2d8wk31HMLObEiRCTi\ncwdTMGmqb8f5AAAgAElEQVR2+OT9cv0XJCfNHIlmrbUZY0wpsAm4C5l1WWit/XMnL2y21l4YTJpd\nhZ80a7FjEOukLVxr7VPGmNRkjzN7kP/nHsxbDiWFMMeATqppvypydcJy8oISbKc7jvrp9kFXLznp\nZ+9yyJYlCj0O7a2hZJnoy20sYRtX8hpHav+x5q2k2E/xSplU66KWa/9+u5eUwad6/CbwW8C/ubaW\nZZM0DpSsh99f9s+55DaHqWcz19F9ndw8jmxrEMlDUYdY4gnpVZOUg9xswoTkg/J5Exp5fsXlWOE3\n4vRhJxdpdgHwFafjngfca639njHmh8CXjDE7gWPAewCstbuNMQ8Cu5F/wQfHIls4pzXcyULJN5wc\nUuQHSRxE8ueq1bcA+W7C2uW6H0AndC32XgzbgEIY6hcLd3vDtRy+tJ4dXA7AKnZSSxeLndn5/sfv\nl32Uz+eDK+grVu8u4COu/TG3PQjdPf9b/dz4sUcB6KOSeRyjtlgI+Hurr6M/VQlbgoD1DF5FqK6H\nTFNwXepl8AlG/hnJHBbhtkF8WLBOYkYLN+LUMTIJVrPiVHbFKOuHwc0kn7jt00iA/UkRCTciImJO\nYTKEO92YoaG1Bsspknrm2YaTTZ6B1yHVylvklheM0V9z6zpLr9DlolUDuAuR7dWK7IdDdQvJ1suz\nU4ZqNvADLuYFAPZtbGLp4XY/Kfcg8Edu+Qjwh8Cjrv0XiLzgkpeTlXbjAWehv7OHjdd8i8Nu7NeX\nb+Ib6du93ryD5BxZBZBpDA5YBXwXL71oOkf17qgkKTdUkXw6iD65cxtp/ITA1OBo8bwJ9D42pec+\nGWaIcDfMzGmmHKfruQDJS6sJuQeDbaX45Ac1JBLjZHshW+WrMHQhOqlWAW6A4zvKGdwgwQvVxS/S\nTR1f478A8DdHPiZ8pwnM/2ewvAaJRNOS64eA78Gzjpzrgaal+NSSxVB3TTfvRbLfvJN/g+ohSDlJ\nIY1M8qmu2477LEqwINq0yik1yPyDyicH8QRbhVxTvXZ9RD/cuY4USQPsydG7TQAjBbM3XdhUuIXd\njyQCrDXGHAD+zFr75UmPbFbgdP/oIblqO8wZq+v08uf79BZC/z7Y4ma9SvBhwCAPDP0w73q5O2/5\n3lu4/c3/N7f50/P/hDev3UzLWsl52/D9I97n9xrE8tVJs0JgMVyp5H4UuQ8oXy6Axr09/GDZGwFJ\nHfm6xqfZWS0OKf0l5xMkGxMtuD8/SVApPht7CpnIVQu4FG8iDwb7KCLZRkwMI7M4P+NUeCm8eyoG\nMreghBMSbDhTr+vVlazHLSu5pJOH07wLmhR8C9ACr/yz8+VqgYeOvIOW+UKwK9jNVtZypfPVeuOb\nWnPJy5s5wNK97bkS68xHInO0mkQ3Qrp/Is3e64uoOuRJ7xYeZjcrqC0XE7m/zSUu1+xi64DN4aTh\ngPsAYUme+uBa7MO7LebflCIiJo7sXCbciNGg/rlKOqUICYc5YwvxBKuEpO0+RNt0NXTa6qG9yisz\nLhLNxSpACt4w/yl+5nIrbGUth7oXUlArKRePUcz7CB46fgmuwo5YvmtIJiu/FvY2LwLEWtjfnMrl\nbfjCN+6EdUPQ5SSFOpKpJluB1UWwQ28m3e6zq0/yIMkngKpgWztiXueHVUdEnDpGZjGtzd6RnXXI\nf4yuxLs7KcHo5a5K7koaT0zg8zYE3ilDeAu30LWV5LKwmxV0fM9FhLUD11pGauVOH4YBN/zlEVgL\nL/2+aBQXreyAV8G6Cg/mJSm3vmyfhOp+ZukfsYLdfL5bsomRAtpKfKSZVidWySJDnoFeK337w3Xr\n8TefrSQDH0rxVu6pTFBGRCQxmyWFczA942QQkup4kzm6LbTQwnwCNXnvGmGlGm7K9dUcGMuhsAyy\nLpCiqUwm0jS/QR0SmaZlcQpl3VvX/wcAv8O9EvoLPMUbuJWHckUld7KK3zz8bV5aIARczDEaD/ew\nd4FYuF3UMkgZ29zBH+FmfvLkei+77kDGooS6DTeBpukbG/GhziDk2ktStw0TCoXadyFi8YYEHL0W\n5jYmH2n2kr3glPtfZH41qfNNFNHCnRDGynI1Wr8UyZDqMGWhuj+FCB3+9yHWcUDu2UK/T3uZWJVh\nsbwhPKddJ7urVdvKBupc5NhCDrkKDz7X7ssLzucXSGayZg6wecF1OULuppYU6Vz/t7KJo+uLeb7Q\nmdcbLHwo+L32g68CoZ+7NvjsejPSipcH8TciLTWv++5EXOnUGtbMZIqxCn9GnMs4ykTcwmYWkXCn\nDQcZmwhC97CwHb6Hllw9ku4x5drDkFnsPRcq8IluIOeilXFJbDupzxWNrKSPF1nCC45g0yzmejZR\n7FKNbWUtKdIccz/aArL8gA25Gmjf4Fb2PHsF57W8CsDxZ8rhXXjyzwIdZTDkagq3dZNM5LNUPk+u\nNFz4WXXSTK3fYU6MNMu/UcXkNxFJRA03IiIiYoYwmzXcSLjTgtEec/PXDeIlB82elT+pppbfnrz2\nIPQPQFtgRb631ru6PgbcMcQTm24C4LzVr3KsXizWAcp48dUlLCkXF7KFHGIFu3M/0p+xisfZSKk7\n17NcSR+V/IYLTUuRZk//FRzPuFRkyxBvCc0W1oZIGo/o5yzFeyEAdEJhfZBsfWlwXfTz63XRicac\nOUyyuoQiWrYRHpFwzzmcjGx1nSKcONJ2fon2VHAMjdoKwmfv2U/OufZ24LoS0XmB43eUk6kXeWHP\nz6+A6iHShSkANhVfT5oUH+QfAKlx1kZLrn5aASMcpZgdubA3aFq/j85u0VmHW6s8eYJM3GXJ80oI\nUS/ygwZLVAOZ8IZyGB8CrSHPYarMSK4R4yP64c5ZnIqnAiQ9G7Sd78Uw2rH061FXqdCPd2fQvgJY\n7OuObUas3be5dgPs2SouZuelXuX43nJGquVHmS5OUcYA97pESCvYzSp2ct9BaV/VuJUyBuhyVmof\nlXT86CJISTKF5e/Yzp6XV8E291l2IV4LmmuBIhmXWt8qz65077uGyd0ZSJOcJCtCJhI1BDpf246I\nOBFRw52zmGiehbEKVmqim7DwYugCpWQbVpfowXs1HASWepLrR1zFNHlNNbkJtuOZchiC/h0SIXbV\nNVspYIQvdn4YgFvqH865gAG8cPQSBvrLGG4XmeOKy7aw4prdHHOFo+rpZM/eKzxnrkH8hfUQ/UVi\npF7n2uuQBOgZ/Sz7SLrLhZ4dw3hXOb0OkXAjxkeUFM5JjCcjDOdtGy3peZjsXAtWquyguqZqma5Y\n5dB+114sZKt+uv3Bcgli+bpH/qe630BZxQDHd4gmu/v6FZQxwEWNovEOUMqRXQ0sulLcIKR+8EKO\njQjhbmnfIPKAararSWY2U7JVL4o2XCCEyghhyLMGPoSlifJz5UZEjI9j0S3sXMOpll8PCy7mk7AG\nQ2i/wbxtWfzkWrfr6/xXCxGLVi3cIfwj/Q3IpJpzHRt+pooj1VU5C3jPs1dACTRcKjV5OjZJAckB\nV1b9lWcvFLLW7GCFCIlrCfg04gWmGm4LcA/ewtUCmrmKxvuCwYXReNoerepDdAOLGBuT0XCNMVqs\nqhgpk/4f1to7jTE1wNeRXKtp4LestRm3z53A+4ER4MPW2ifGOn4k3GnByYhACTifRMPZ+PA9H73I\n47ee57Db3xFXthvaD5IjtRY8AXYhVqZanCXutLq9Qfp3ZCRMuOa6g/RlKnnlRy5RzjbgBguFLtih\nEIk2awj211wPIGS/wQ8NXFst4PalkHY5I3JZ1fS61JAk4bAuWkTE6JiMhmutHTLGvNFaO2CMKQS2\nGGOuRYpUfdda+5fGmE8AdwB3uJpmtwErcDXNjDEXu8q/JyAS7oxgrMkzgnZo4VZxIhnna5lFJN3H\nfoYvYlaEkK1LettW6yfQNBpNc+uWuOVW174Bitb0UlYhYcQ9bRpe7NAE7DKeMFsQB4qwHNA6/ARe\nIULyadduQ/IuZLTeW1nwuZvwiXvA69RjVVqOiDgRk9VwrbX645wHFAD/iRCupnj6CvKPuQO4Cbjf\nleBJG2PakIKSzzAKzpvUyCIiIiJmGUYoOOXXaDDGnGeM2YFMkvzAWvtzoN5aq5Mmnfj484V4Mwa3\nnGeleEQL94wg/5FYc+eGllw4mdTj2mFO2UqS+mZt0G4nmcT7Cmh1QRLrkEf+MBT3n/CpH7sgVbs/\n54e7vaMBMrD8pu0AHDq6kCP3NMDNLulRvxE3NJ2UW+eOrRJFxm1Tv1v12a1z4xkC2tWfOE2y4KTK\nC7F6b8SpYzwNd1drNz9v7RlzO4CTA1YbY+YDm4wxb8zbbiUp19iHGGtDJNwpxViTOaeSYCV8XM6f\nQNNj6A22yr2UmLqRR+/87XrMMk9024D+Xuhy5F2NEKRGijXAvm9c5k87BKTcZBrIg1IWyIiGO39l\nB0eyDVQsewWA0vJBXtl0oZ9Euw8heNV49Xyayzfnr6vXIEziU+U+k16LMAgiImJ0qMviaLh4w0Iu\n3rAw137wrrYx+1prjxhjvgVcCXQaYxqstR3GmAvwzuEHgeZgtyaSvo0JRMKdUkxX/tYw0grE4tsT\nbC9CNNymoH8NiVpRXXrTTQOLccnD5D2FJ+QHELcutYC1lpp6JdQhum1IlHXD9LeKX+/Rdb3Ciw8E\nx2/HE/oGdwzVePuBFjcB16aBDmqB9CIEHGYeG986iYiYjIZrjKkDstbajDGmFHgLcBfwTeB3kbKr\nv4t3hPwm8DVjzF8jUsJS4CdjHT8S7pTidN2VTjXFoD5aa/WIfLey+qDvQfzM1ip8PoM+oBsqXFvd\ntJRAhxB3Mo3kTSFeCM6tq+Jdr9D/zPkULZOxHNnRAF3Q8HZxIzs2UkxPRZWXGJTY9XjbkPuCWsD9\nBApYFXJDUAOh3q3TipcwevDDdLmJxdSPZyMmGdp7AfAVY8x5yBzXvdba7xljngMeNMb8V5xbGIC1\ndrcx5kFgN/JP+qAdJ8l4TEA+KzDZP7YGDKhW34sQlT6ad+I9GDSi7SrXLkseqgKxbm9w7dUkCVnd\nyNSvthWRC/TW3YIQ9udc+2a8Fa1Ik/RqUOt5l0Us95fdinaSrnEaeZbvJhYxdzD5BOT/an/7lPv/\nV/O1mID83MNkrahChEjz3c5CTbfVLV/h+ivRWiD4va1ECFJ5rhUhWyXcDCI51PldaCcx6cYyhGhB\niLULKXwJMqnWhZ9E6wd26Ti13E6YNS28NmrVh5No0QqNSCKG9kZMA/IT34SWoBawVIlhmKSHQyfe\nr3cVCS+WZ4w88mv9tDWIBauHd4ERvpyPhayhqE5IcLikSixc7a9OByob6GSZBl60ACVunEOr3LjS\no3xGxQJOLLkTCTfCIxJuxDQgPxdDSLpZfCFK8F4LIJOrq0jWEfsJPiXiILTXk9N8tyGkqAUr04gE\noJNitwM7oOxm8RU/0lEl1mt/0F+tZvDkrb+8JgINt8+NS8ean7SnnqR120ck24h8RMKNiIiImCEc\nHcct7EwjEu6cQfhV6qN2mO6xJ+h3kKSF20iypHsbOQtXcyXoxNYGcr65ADxgoBCOfME52pYE+4C4\nlO3Fu4HVBccCsYT1WHtXkfS+0AKToRxShU/M8CP3rvKI6hfR6j2XES3ciBlAPsGWBu0wtWNNXt+D\nJGf7ryGhnZYgkkCuJA7Caepl0IDIBhrY0IWQ68qgHQaPfce1q4P9VW5o0RwNOtYrSJLnADLZp+5u\nmj9Xf8aNCPmGvrqRfM81RMKNmCHk10xT4sr3XS0kqYsuIJn4Bm91phErVAm2wq1TkizBlckJtu/F\nW7Et+JSNIBV+u/CTZ/0ky/FsqA0yifUiNwAN6lE9NxxceGMpxevAEEOCz03EEjsRM4z87GShialE\nrCS1AB9IoeiFtBLwgOQ92KvBB7VCsmGgRDWeJDOIjKAEW4f32dV2O0GJnWBZgyB035STEtJKoC0k\ny8+rn3GIBZyYxjGb146Yy4gldiLOMEJdtA3Jobw02LaHZJRaD946TMHesEpuKQyVebeuCvdS0ixE\nfG5Trl1CMl/uM65/mFtBw35Tbn+NUM66fXNjr3VjC2u7haSbRVzewppo0co91xAlhYhZBNVzlUR1\nAi2UHBrxE1Vtbp+Ua6eRXMsO6uLVFHT/HN6o3IUQqhLsFpISQipoa2KbHUE7gQWIjqtjV3/jcFJN\nM69pW0vQE2wLS/pEq3euYU4TrjHmbcAXkES9d1tr/2LSo4qYYhSRrKIQ6pxaG02DH7IIqaZdu8n1\n1WQ51yZ9ZzX3gv6S1iEZwja49jaEUDXVx63u0Kr57sIT6w6EcPXU1UD/AN5i1dwKao3r51AC1slA\nJWBNaxmWoA9LE8XQ4LmIo3O1ppkxpgCpwXodYir91BjzTWvtnvH3jJhejGa5Kcn0ueUwLV0lyYxy\nYaaxNMKwGhjRC+3DeJJrkuNp6G4HMimmgRH9wB/gJYcMvgwPwJ8Gy21I/TMNsmgHrivzuXyz3W6c\nRUGHsCRPJcm0lFo5QqH6ru4f5Ya5iLms4V4FtFlr0wDGmAeQkhORcM8oRktwHpbvOZjXJ7R4K0k+\ndmt0l5q0bW6dktkeqFjrvRbSyK8qJNjv4CWHDoRQ9XCt+PI/WXzpH+3bBWR1rOqloLptvRtr2rUH\nSeZf0MnB/BI9McXjXMZclhQagQNBux1YO8ljRkRERJw25jLhTm9ux4hpwGiP0fkVJsLCjT2INasT\nT52Ihdzi+6aDaLD+AaAP2pzOquV81I1MrVn1cliJN1A1Ik0nzUr0tCoBpNw4dQcdn45VI+rC5DZh\n+SmtKBEnyuYy5rIfbn55iWaSBdUcWoPlFMnkqBGzA0pqGtqbn6kr/KlUkYxca8eH11YC9V6XLUHc\nwMJ0jqHXQgaZcgWRHarxk2gpJEBCo9yqi6A9dImoRchXJQYlW9WrO0mG+o52swk/ZyTimUea5A10\n8pjLGu42YKkxJgUcQuqzv/vEbhsmeZqI0TEdlQ7U2g1dp8KoNXURC3XRcHtW9tFflhaUVMKtc6+U\na7fiyVctWrWGt7jTKeECQqqhX3Dod9tIsgaajjcfoZdCadAnarszjxRJA+zJSR9xkiV2moGvIjOs\nFvgXa+0Xg+0fA/4KqLPW9rh1dwLvB0aAD1trnxjr+JMiXGtt1hjzIWAT4hb2r9FDYSYx1UQ71vqQ\nlBrxOQ3Ah9OGuXd7vQKRAaiEzY7MWmulioROsoWSwkrEutVyPPcg5KuRas/AiVZqCk+UB/H5FcBH\n2ClBq1QyGGyPngpzDccm5xY2DHzUWrvDGFMBPGuM+a61do8j47fgS5JgjFmBGJorkD/HZmPMxa7y\n7wmYtO1trf028O3JHidiNiP0Vw3rjYHkOdCqwSAEXIa3eIucP63LPtYAbOuFDmdlNuHdwFYiOq56\nrKUQwtZItAqgYjH0q2rV6c4bRpuFBKo3A90eZhXTzzVW8EP++hgkcbZgMhqutbYDJ4hZa/uNMXuA\nhYjn1V8DfwL8R7DLTcD91tphIG2MaUO8t54Z7fizV+yImGUIczG04MmrE1jOiRm6lJD3Q6YKShzh\nZoCVVf6XlwvfRaSHCvwTZgsiOej83DM4uUE13CakWrGawHoz2B6MpRHvpajubvpZ6knKEyGpjuZa\nF3E2YKo0XCeVXg5sNcbcBLRba39mTKIE2kKS5NpOooRKEpFwI04DPSRr6KTxerKWOXfJbkoWS1fV\naTX3Qn56Rz1UmEt3GULCYXWIFP7nnR0mmcs35Trr7z2NPP1puxSxyNXi1bDm8LNEq/Zsx3ga7iut\nu+lq3X3SYzg54SHgI8Bx4JOInJDrMs7uY3pvRcKNOAnyJ+aGSUalgZCYRqJ1I67YpckuafcbrJaE\n5TnCzeAn1J5BCDYMkghDfTV3rm5vL4KsShghVJPQ3Aq1wbbK4LM04lM8KmJmsbMd4xFuzYZV1GxY\nlWu/cNc3TuhjjCkCvgHcZ619xBizCrmbP++s2yZE213LiZ5aTZz4B8khEm5ERMScwmQ0XCOM+q/A\nbmvtFwCstTsJ0ukZY/YDV1pre4wx3wS+Zoz5a+QOvhQpEjgqIuFGnARjTSjlbw813q1ItQZgKEvC\n7SdTCk1F3kuhHW/tqjuYZg9LIRNu+ivtctt0Xz2l7q/77G0MGluDjVmS1m73GJ8Fopxw9mKSGu41\nSGnUnxljnnPrPumcAxQ5ycBau9sY8yCwG/mBfdBaGyWFiKmCTjypL6vmZ1TvAM1Rq09VRchj/GJp\nXksyPWO7hWVODssgLmGq927Gky5IafYtwb4VSG5eLdn+DC6CTSWCMmTGTb14Gt24lKlr3efIzzam\ny2Hi9jAXBUR3stmLybiFWWu3AOedpM9Fee1PA58+leNHwo04DYReCuqjq20tzx6W8Dnsd21bnExu\nc7NJxib+AfCYW16N6LsqyWopH933GXdq3T/r+medppvRbGKqL2uEnN4MUjqoYAChG3k2WA7DhSNm\nM+ZyaG/EOYPArzYxJ6BJvkPLMEvyUT3Id9BxELjW//J24SWBFHA3flsKsVjVIm5HPBdUUthA0q0M\nC10HodrNqlU3QabXn5saYB9ejqshOWnWTtLnGMaOWpuOKL+IqcBcDu2NOGcQhvIW4X86+aG/mp82\nTH4TBiY439eMI70wx4IapCoR7EXIWAm2GpER1CC9Dkn9qEgZyDb5oaUBqqBCE+uATCLvcx3CcGUd\nu+q/PXl98rXqiNmKuZwtLOKcRH52sTB/bp47WC6cNo+oMsG7ku4NnDgp1gHc7NqaR1d/ta346DQQ\n67cJr/PW4aoMq0ar+XN1jDsRuUEZXHMxgL+JhFptEUmfXYgW7uxDJNyIOYz85N5VCHGpVasWsLbD\nKDSEEG9wy0MIaWouhQeC9SCTaEN4Qs6vAJwhKUF0tUNXDYlqFQkJYRk+cg5OrHrRQ1IaCcm3kBPl\nh4jZgEi4EXMYYUUF8NauPpor2Wqfw4h3QCApbHabGhACVU1WLV+1YAvxcgMI8Ya/4CHEIh4KPQqy\n+Am9Vnde3Z5y2/qCtpKu1kPLlxDCnBFhLt6I2YKjFJ/pIYyJSLgRERFzCtHCjZjDCMNkwecm0AQy\nyxGrUh/rdyLpHfdLM11L7hG/vShZ06wFsVrV0m1DJsnUDawJZ9G6tuZiWONkgIx7T7vt2RsR//Tl\neZ9BddtekvKDWrEglm3Y1s+qGCskOHozzDQi4UbMYSih6KO4pksMqwRvJanxgi9C+TL+MT0Fu6q8\nm9hekolv+hFPBeW2bRY4DE1OnlA9VxOWryZZXaIfqF4BaU1eopN5erN4OTi4Sg9KqlpNojBv35MR\naSTamUb0w42YwxhGSDeMPOsjWQMNfG3R7SQn0SDh4ZAJa441yXKHI/UhrU+mhLmURCn0OreLTpp1\n4D0VQCzkdDc+vDftxqHnWxSMWy3cMGotjNDI98ooylsPkWzPDKIfbsQcRhFJtzBNcajtXpI5DbIk\nSBIQiQGEAHtI5F4oCfxqqUfyH6xInl9Df5e51z2u3eSGo4UrdwAP1SCyBoh8EBadXI6PinuNG7ue\n/Cdu3c9cOz+3LpxYfDO8LpF8ZwpRUoiYYwgTuxSS1DI1B0FoAYbRXgrVUTuDbangmAD1os9WB7tR\n6zXdLEKm6iZ2LSIh3OraHfjc5OBqqRlIv0ba1UBmsT9+FsjWB8uFeB/dDUjAxDWuvQex6nXsB0kG\nfIDcWKIXw0wjEm7EHIQ+QpcyuvVWGGzP4km5imQChNpgfyVn9Yv9EVAJmaXBMbPQ5eSLNYjvbViE\nsho/iVaChA4rQa9BOL3VtSvwujCIBZx2yytdu9qRc0bHrlFqTSStcbW+1VdX80uMFSARJ9OmC0eP\nTaqm2bQiEm7EaSAkiT68jgved7Uwb3sY+luI13BDK7AUMVPDvqng2ANQXeU12XX4iTWg6NZehtur\nfEWICoTX1cpNIeS8wbU3I6SqxwvDjEuAtwWJzQuBx0Jvi+1uWT/nAsRaD/2PtcKEfrbwyWCiUkNM\nF3mqGMnOXlqbvSOLOEuQXwMs/z0kT30P3avyk9wU4TVdtYQ1MOE1Yr2qBathwY4ohzdXJav8tiFR\nbEqku4C3QcMlLwFwOFXP8WfKfShwA97abXFD1VwNGoqckxg0n4SOXyf0XGkhmhBSDaWTMFNZvvww\nFqHq9TuVvhEAI9koKUSccwiJIkx2E3oh5Pc9mNc3v0LwVhha6wm0FZEJ1I1M8yykXPtdcp75DUKC\npcUDFDLCwIhYrgvqO+ngouS/oC44Vhde3diFTMjt1RVZRF5wOxde6+4pKpXsQW4cKjH0cOJkIXjv\nDkg+KWh7NA04ku14mAzhGmO+BLwdOGytXeXWXQX8HT6ZxgettT912+4E3g+MAB+21j4x3vHHTbQb\nERERcbYhO1xwyq9R8GW8X4viL4H/Za29HPgz18YYswK4DXGbeRvwD8aYcTk1WrgR04RQlw2tWm2r\nJRfmOqgn6dFQ6tZp1FojMOATjKdIeJCRlvb8mzsAONJVTVHJMY4OySTKquKd9FHJoa6FABzvKhcL\nVv8F15HM2xDm6l3j2tWuOsWOIihZ4aWGnHdYUfBei/dDXoBPXanbw4rBvfjqGODlldADJFaZOBUc\nHzl9WrPWPuXKo4f4FTDfLVfjH71uAu631g4DaWNMG3AVybLpCUTCjZhmKLkO57XD7UoyvQjR6MST\nEvEqElCSK0HsCiXJdqADjuxqyPUb7i9hOCXNzssWsJBfcdw9cl506c/pPlrLJZe+AEA1GZ5Ydj0A\nFdV99N93vteLA60YEKLvQtzQcmMK01a24Muwg/dw0M+aLxXka7UqrUSSnTCmXsO9A9hijPkcogq8\nzq1fSJJc2/E/3lERCTdiBpCvOeZ7LYSZxMLqETVINJmzEiuWihtXWPMsLLn+ByR+0UWrexl+pEq0\nV2Dfs5fRvbqOX298HIBLeIEfFr+eJbwIwKOv/gYMSaap/gxiy+i5HnLLSroNbugVQbu/CNr1/5ZF\nrGOeOzQAACAASURBVPMfubYGSYSfOz+ncBGeYNXdLrqPTRhDU05r/4rosw8bY94JfAl4yxh9xywg\nCZFwI84YxgoIWIQnnUZk1t9NmoUhuwBY6DA+4Y3mUqgTchrOVIrrWFo211x3kPcUfJVOd7wV7OaH\nvD7nKP/m8s38R/+73cGKYd0wJdViZQ/118j5lWC3IEasjiWNFMTESQ70upd6LZQh1q0r/5OLWFPU\n44NGQKz7ej/4iFPHeEU5ftIKP22d6BGvstaq78tDSCEokEeY5qBfE8mkyifAjFPRd0pgjLHw59N6\njoi5glLk0TtMVq46LkhobTCr/0fuXf9gLUATLHrH3lyXJbTR7x7jj1LMCAW8jy8D8APeyG5WkHKZ\ny4o5Rq0jyGPMYx7HyDg/sWPM4zku55UnL5QDlyBcqDeBzcjDZVoJVkOatSilRqaFmm1otaq/big5\nhFbwuRKxdhfWWnPyfqPDGGN5fgKcdpk54XxOw3008FLYDnzUWvukMebNwGetta91k2ZfQ3TbRuRX\n0BLLpEfMME7FTzR8VA4n0HrxpJNy746AC6vk0V09s9oRDTcbdC+BA52B0VEPq3kOgDSLaeYAd/MB\nAG7j66xgN90umc0Bmlnrcj4cYiEZqnPW7/v4MnfwGUaulXZPWyOsG4bNbuwr5dzsdYlxumqhbRjv\nh6tRaep/POA+iE4Iqk9v6K8c9dvTwiTKzhlj7gfWA3XGmAOIV8LvAX9vjClG7ny/B2Ct3W2MeRDJ\npqTuYuOyfbRwIyaJfHIdS3PMXx9GpoXL4OcdliMpEzW66zXSVzXblUjUWHvQVs0VYDU0XbmPvqNC\n4CPZAm4rF5IFeJhbeB9fps8RfDMH2OqymrWygYv5BW/kBwBsZS3buDJ36DIG2dq9lmHnAVFUcozh\nHVV+LJ9FJtzSuodmKXvWtQ8iFq9axEXupQfQahLnmoY7BRbuMxPgtHUnWrjTidO2cJ14/ClkSuK1\n1trt4+8RMTcx2oRYCCXk/PX5eWXBp0MMy9xU4S09l5tgyIm2/W5XrYn2AOLa5WqcVSx7hWoyrCgW\ngu0sFmniP51M8Ak+yw17v8+Ty64CYP2Bn7C1WdNIShKUnc5D4ijzKGOQjciE22EWUFA7Qr2b0Hvw\nn39XXMeU8D+OKH1p126qFb0360lbotby/4Jh2sos3iIOr0MRSZwrZHyKGDnTAxgbkwl82AncAvxw\nisYSERERMXlkJ/CaYZy2hWut3QtgzIxZ4xFnJcayeEdL1K3rVaStIWkF6kSUQxfiutXq2i1unfOd\n7W8/n3QTZMrFol1AZ84FDOAe3seqZTtp5pcA3N98E80cAOACDrGENiqdL/AAZYxQwIssybV3s4In\nX36zHKwJybugPsK7EP9gHW4DYnlrbgYAfjMYvLqMaR6JUrwfMojMoi5launG/AqjYujkXc4U4qRZ\nxAzjZKQQ/iR7SAY9aDVgR0pdZcJX6mnVgfCWVgFeDf3vOp/q9cKCq9jJAKW08ka3+Tl2cHlO072X\n99Dp9OL3cC9lDLCfxYDIC7/PP+e8FtKkuJFvsnmReAs1LzrAltA1Myvnz41ll2vrWNtTiF6rHhh7\nXDufRPPzSkAyD6+eLJJtDmfAcj1VjEu4xpjvkpyGUHzSWvvo9AwpYm4iP5nNeJmxVLesRCxc1TV7\nEcLV7WUS6aWTaLsIfHQRS6cC2qsln+6Ky77ITl5Dygmrg5TxNFfT6vI1LuRQzithM9eRYj/FHAOg\nlm5e4BJ+wcUALGc3ndSzxGUPy1DNZW9/Jufx0L66mabGA7Tf4HL5tgH34cmgpBaGwgoRpSR9c/U6\nhZpuT952xWjJgM4VN7JRMIsJd9JeCsaYHwAfG2vSTLwU1gdrUiQD4CPOLUzk0VctOXUTq0cs3pRr\np4GlUOImkZYhgQlhisUbgBY5X9OiNNeziU1I+G4pA9zIo2x2+RxbaGO5s3ZXsJs6unMeDDtZxRJe\nzO27m+UMUsbH+RwAXdTyFG/I+fQeo5gByihz1vi/HPx9uKfED/07yM1BAykeAbHc025FD8IcYR21\nsG5aId53VzGWG1l+/oqxZJ4zgTTJ4I4nJ++l8I0JcNo7zhIvhTycZMAbpug0EWc/TuWPHSZsqSRp\n4R7GW3r1iNeCa3bhqz6APMLXAY8I0bSvXspz6zs5SrE7ywAPcwstzkotYIQXnX78GnaSJsVqdgCw\n8dVvs7N8JZc7n96Lj7zEi/MXseybLwPQceN8LmcHBW6K/CleTyf1Oc33dY1Ps/OPVlFdLvJGe8lS\nkUM0RqMB6CjD12vbR9JK7Qw+tyY317+vEuZYrncnS3au+54J0k2RNMCenPwhZ7G6Mhm3sFuALyI/\n6W8ZY56z1v76lI0s4hzGYPBeTzKhSzcSJguSZyFAF6KRKompXqqE/Axs77qW+TeI7lBcfIxDI3W8\nUHAJIJruJfwCgLv5ADfzMC8g2+aVHwWEaAGKXoVlv3o5VwC4bGSAiwteyOnDO1nFH/F5nuZqALqp\n40B5c25Srr1/KXwIHySaQUKUNRVKewti+al1rwUvQcg3nFDTTGNhOZ+wrevyERL0LGapiWIWu4VN\nxkvhYeDhKRxLxDmL8I8fWreNJIklhRCPTiBtBW70CWWaSGIH8kQeSAw1Nx+kuUBIbwW7qSzoy5Fg\nhmp2sBqAeRzjWdZwKw8BcJh6XuASXrtrlxyrHHhcTg/wQsElHGIhB1xo/W/wKL/gEt7nSgg/ws3c\nzMMcdpNkP25BSHadG1sDIjOsdu2sgaHFXo/sB18evg8h39DCDVNg1uRdt/Es2zmIWazhRi+FiFmA\n/D9+mOGuhuRkUYjlwG7ocuTS5YIWlLQ6SD6tpqCgYITnnxeWG7islF8jk/M8OEAzV7pIsMt5jqt5\nmqybRKukj4Uc4h+v+V0AWniRDYu3UOSSgb22fBevLtydC7JYur8d9sNjb3oTAFfyLC9wSS5XQ9M1\n+2j/+VI/vhRyc1C3spWcOAmYuxktcu9a0LIyb7vWkVOMJTnMUUS3sIiIiIgZQrRwIyJOBfpoPF6h\nxnaSLmbqKqbohR3O4ltt5FAqOdwNr6QuzLmRHaOYCzjEoNOEP8g/8IJz+3ozmzlMPRuOSIXJ1vnX\nsoqdtLjACXUt44g79v8L5dnjLP2Q8yr4EPBecrkZnuZqrmcTdc7CPUAzzZce4MfPiwVMGyJ9uMNS\niMgM6vKWhpxgTDti1YaJccI8wp2c6BY2x3Ta8RAJNyLiVDBaXoYwKq2SZEWI7cizuLpHDUq7xTnN\ntCHygj6m1yHuV7dKs4wBDlPPzW4qopMFvId7AVjIr/gVC9k0/025dsOOI+x0csVW1rL0QLufoFsI\nHAXn1AD3wvPNS3m7y73wHu7l83w0N4l2mAVczC+ovUwI+LGSd/qk5riPdR8B4Q7jXcQGEZe5UGoJ\nvQw069hYQRRz3Ec3Em5ExGRQGrznu4yFk0VuQq1N3chqoNV4nTQLtMBll4orQPpoimxxAY9wCwDX\ns4lLRqTczryhYZrLD+QS3dTTycHVNTmLt+hHCMlqJshC4Al8CbMsXPbqvpxHW8e753MXf85WrgpG\n38/X+G0Afv2Sf+fbHb8pCXBA9NsGfAl3ivAuYxZJZaIh0D3uxDqZWEhyAjI/lHqOhwHP4o8WCTdi\nlqOKpOVWjyeSGreuJehfincbQwhMgwuapOvzP5dJs5plBzlGMdXOBK4mw8MFQr4vlF/Ch/kil/6b\nuIH9/J0XUcoAReIdxn0b38Ht+7/Bq78j+Z/KP3EcihHSBdgPvAqOy2m4W7SHG9Z8X1YshO8veB2X\nIARfxgDfbgjGilvWj3Y3fjIoY5AAEPVZVULVv3OY5Fyv23jRZ3NsUm0uuoVFREwPwkdffSxWAuhB\nSFarPlQh8kKnb1fXetJS3lF3sTQiK3S5o1UsoGdLI4feJlV8C2pHOIbkt/0d7uUAzTSsFqJc0fMS\nXTUVfGeBRE220AZHoXzHcTlYMey+B5Y7i/YrT8B734tPf1uDWK0OHR+YTzMHcud7Bw/RfMmBnFvZ\nw1fewstfWeY13dV4uaIC6FfdNkRQDYNeks/WJ3vOHs2P9ywl3uilEBFxqggnd8JHYjixLI1adepz\nehgyTck/XCu+DE4WuAcf+NhRBHUw3CFEVVvbxSNHxSS9oPgQj7OR9y2VcjwjFHABh1jg9OI+KuEX\neA13G6xQOQAY7hECXvFJt6IQeCu5YtvzOMpOrs75AD/KjaRI50KJO48skPGqW1gXeQZsGGk2jEyo\n6bXRG1C4w2huYrnRciLOUrKFyVZ8+BLwduBwUGLnr5Ag8WPAi8D7rLVH3LY7gfcjdvWHrbVPjHpg\nh0i4EbMYGv8fluAJH42HSfrpXgfsh6GgvHrWeB20hFwFX0Ae0+vIFaF8+tKreX3xUwA8xRt4PT/k\nITfDtoBObuTRXC6Fjx79PK/cWMH5uxybHwCaods95R9E4hhW/Ls7Vw/YLBgXWVazY4ja1VrtQSSF\nx9nIbqfTDmUq5cahJF6BTyP1CNBeBpkwmfnX8X9nrZ0WhvaGmu5o0kIpnqk02flEJIZZJEdMbghf\nBv4W+Gqw7gngE9ba48aYzwJ3Ane4mma3IeJ6I7DZGHOxtfb4WAePhBsxC6GPxr34cuGKMGFLJeKp\noJNkm5Hfvf6sn4Tscuh3JXr6jTzWqwWsFX2d58GLnS1k6n8NgMFXS3mxvCWXD1dxi/NoKD9ynPJf\n9oNLh5srTuEI91OLkWg0zdu0zSUc+aY0h78A6w/9BGfgUr02wyEWsoqdAHQtquXFW1s4vqNcOuzF\nW9PLyE0A6rHhN/Al2VV6CQk2nHDsJVlpIz+4ZDjv/VTkhVlAtIpJaLjW2qdcEclw3XeD5lbgHW75\nJuB+a+0wkP7/2zv34Lqq64z/diRhW5ZsRTKW4weWQQ62g0E4DnaNE2iB8JiGwAwpQ5KBJE0GSiht\np03ThE5T/mim6eRVSjrNTEigDSFQOiGYmHdjgiGYOFhgRxYg1xcsgyTLiiwJS1hX7P6x9rpnnyPd\nK8nSfcjsb+aOzj5nn333Ofb57jprf2stY0wbUlDyObIgEG5ACaIvse37Kv3/srqg5lu8KaIaaEuR\nPLOp6JTeDRHPbEMsR4ne5Z3euRza7AiuF7Y3fJibltwGSBjw287fCvB/Cxdx6j0dGY5r+W+Z5Qon\nq235X1jzc6JgMMdn9lvyd7Cqgn3zF9MwX5LfrNzazrLLDmT0vWfTTHV9P69sllwOA4tOji79IcRa\nV8t9PbCzkii3brJg5Tzi9dNqibsZXOmicZPczBDkVxb2OeAet72YOLm2ExeFj0Ig3ICAgBMLeSJc\nY8wtwDFr7U9ydMuZGzIQbsAMQNLn6L8a+7IxbWukWg+SUUzPLwc6IeUsQX/lH+K5DPYAbbO5ffOX\npL3NUHFFH1fX3QvAzdxG/Q1dzL1c3HVrLkOizpyLYc23YPhaqPi8G28WsBjMo9Kcd8ow8556DU5x\nx1fCda33cXBVrZtaM0ep5PS5kr3s8Fl1PPbaZdJ3c4UY7w3uXC2m2ayulbWItetnF+sjilQbdvuU\nmdTHm80PO8Ms3lzTfWMbvLlt0kMaYz4DXEbkRAJx1S/z2kuJl+YYhUC4ATMYftpGGO3vTepPzwV6\noNe9WrfWCVml3OE7ify7VYhCQKPWmpAy6O5x+wmf5LJZW7moWd7rh6+Eii6il81PQMWrkKmsvlK+\nOuOC/gfgBiIOHIK+9RUs2SokefMFt7FgVjd38ylAQoHPWC6/Dictf5sXZm92vlsiudkZTv+WQrh0\nyK+GXE/kctBSPsoNGsGXVIXMULyd41jd+fJR7Lp13OGMMZcAXwLOs9b6GpgHgZ8YY76NuBJWAs/n\nGisQbsAMhhKKX5LHzynQR7xu2PPE5FQdFVA+L5KNLQD+w21/2v1VUhsCmuCBt64A4Oq595KigVev\nTAESNPHG/Pdx1jXOaTsCw7+Aiv3u/COIv3exa5+KkLOTifENmHfPMC45GSPlZaymhU08C8AK9mcy\njfVTzQ8+OMjORmHzoUdqZb1QVQzPIe0qd90DfmFK/eurFvqIW8QzWIMLU5WF3YMsdS4wxhwAvoao\nEk4CHndFc39trb3RWttijLkPyZuZBm6045TQCYQbMAPhuxh6iMvG0sRzBgwSEckcIo0q7jyPcIeI\nXtM1tPZO126Sz5q5kn6xjsPMYTBTQudRLqaG33PWiCPc3VDxBlGO9P2Igf2Ga8+X8Tpvkmb9SuAC\neKtJItfayho5c2Q39WVd7vu62eKS7y7jgESoObLe3niRXLJqdpuQa1LPygCINesvRlYQVyb0EaV5\njCszZhym8Fthrb1mjN0/zNH/68DXJzp+INyAEwC+jMnXj4IQjb9ar6/XyP50H8x2MrQhC22ub1Wd\nWLeaaWwISMHzL4vOa9Ppz/Iy72fQWYmraWEFKTpWCQsuGjoC/wp80Z2/G2iG/S5l/4pvArdDvQv9\nJS3tuReLT3gde+FJSP9IwuIe5eIMufdTTRPNmWoUyz/YymtVqyILt4F4KbRuX8Gg96mHSM3Rn/g7\ng61bKOnQ3ikXkRz3C4yxYpUHBOQbGp6a9O36lREaichnDqNDhZ0FvKBe0iXqqZ9xp3oevI9+6ueZ\nyLOruJ8aelnsTNhmmvjE/oei88uAb4MriSb700SLZoeR6hFbXbsOkdZqtrDV8MIySce4hY+xk/Vc\n7kS993MVL7GWjl+cKn17kR8LJd3tQMd2onSOfYh7RS3eOUTmsN6nYhWavHXqRSQ/NglO2zIzi0gG\nBJQAyokL/DUyTYllIfEkL76/F4Rs3bFud7pGem0jylkL0ACP3fHxTJ6GBRd300k9H+ZXAFzOFgl8\n+GvX/1SwPwKj470O21rhPCcseKQH1m1xrgWAJxGi1GCHWbAuvReArhULWcYBfsWHAXExpCmja738\nkLwzUAntJpMzQlwKK4mcm6q79VUKfu245FuChlv791X3lyBCesaAgHzCJwclE93vJzCvQMgk5bUH\nEVMSosAAYFGdWJeaCGe729b2c0RVgYG7+RQr2J+p2ruGvSzh+ciYXgnmHqKikXVw/h8hQiPg0i7Z\nx0fd8deB1dBzjZi4x5jFop9JIp3VK1p4hk1cyJMAfJO/oZp+rqqXCI77Wq+Tuat6YSnQ6geP1CMB\nIb7P1idPLU+fTGaD1y5hlPD0AuEGBAScWMglCysyAuEGnABIZr7y9aT+inuKyHoDsdqWEOlR68m4\nHwaQp0OtxHK3z4UB82nER3qF+AsP3XsKhxpO4YINYnVWcpSnFp5D/zfFJF7AYdaMtDBng8y14gBi\nUKtMDGAvHGqS/v1U0U81lZ4io/nKpsx2PV08gEjUauhlAd2Z0OOTz3udQztPySTl4RFE/qb+4IE5\nSHCE77cd9O6VJnZXqP/Wz60AJWtKBpdCQEAh4EecQeS/9V+H+4hn1er3jrsSPQADnY6YvByz2/FS\nOyKv6nd66y2rIOXOv42b2cCOTPrFNhrpL6tm02zR1X7/jM9y/ZE7JDgCJOfCW9H5zTTRRiMXI6Fp\nI5TR5MLivseNzOJYZuy17GYrl2XcGdX0c+gMIs9Jg9uO5Yn1I/TqEvdJk90kg0gUJUq0ihKeXlAp\nBLwLoY5VTe+oFvFKIr+mSsjU6quFqooomXkN4s/VdI+9wCVkMo+d94FHWMYB9nEaIMlv6jjMGkTH\ne+XIzyhLjzB3t8vktxX4PAy73DlPz/8DmtjFvVydmXW1EwxXcpRq+jMEKxrg3ky5nk7qmcXbtH/F\nrcA9hFh9qloYAAn51+xivlZZ4Wt2NRF8H/nHNKgUzp4Ep+0KKoWAgDzCt9zSxIlGizNqv0FwVXfh\nGRg4B1pd+Z7ZwBlIRBdEVqQj5C4W8tS+i/mz074DwCe5m6f5SCbB+D+V3cJflX0H1gqh966voZca\nPnCPlPRZe81uqo8McfZ8sWrfYDEnOefkmyxmN2vpdnHBa2jhGLMyFu+eFz8kFri+Ws9GyDbjUsBN\nViURh4m5UzLWr7pa+ojyVGg7m0ysBKLUgkshIKBY8F+FVaOr/klNW6gE0UU8Kk1X87XdGY03VAvN\nlVFO2kaEdF3U2t7X1sLstzMJxbfxh/RTzRMuGcOZ7OZr3MraWZL/toZeKjnKr64RqdcxZtE5v54N\n7AAkAfp/cS0g7oUGUpQ7C/d+rqKRffzyyPkAnHXWc7w4e2OkqOhFSEgT8zAM5Ssg3ZLlXqXcPVrn\n2kldsyvWWapVgAPhBgQUC7lKyahFp1m2/MchTfwVupNYXl0aIN0IrY6EauZJ5jGNTFtVATUVPP0Z\nIdB99acxh6O8+poQcMvyNZzLs+xwFvTZ7OI09vE0H5Hh6OVa/pNdzkdx4dtPMGvWMYCMdatBF/to\n5NkjmxjqkOt4sXkjLLXQ7d6UNyJuhQwGIf0qo5kpGbH3sGsvIfLruvMz9wjiVm0JOFBLYArZEHy4\nAQGjBP2KhcSj1dJE+aXriSkcas4UC/KP3eEaJPBACfhCJIhC8za44U7e8DoAC+hmDkeZhZBqGSOc\nzssZt0E1A5lqE4epQxwQYrJ+n+tpo5EXX5ZqxMwehp0VsXprPES8XHxKRhJsc9eT8u5HJRFzDSI/\nOBqppmXZc+VcOF7WmwYf7rJJcNqB4MMNCCgw1GLzgwM0Cs13MTQQLbhptWC3yNY7DDUVkdJqAULA\n+lrfhuQ6cAT8nqVv8c4P5nIoJbG9h2pOgVXDnLdcZGUpGqjkKB9jCwDPsonvcSMAH+FpdrAhQ7hV\n9HOYOlaf/gIAe3esE1JVsq8BriByKfQCqcNEFvxmhHz98uoa9iyzkba6V5Rok2XZSwQlNh0fgXAD\nAgJOLJSwS+G4CTdX6eCAgJkJrRIMYsX1E1l5g4gVqLIxP3zYHe8th17v7XQRUd2xnYiP9wZpZopD\n6jCrhlm0/ACHnQthhDJeYi3ns83NpppXXHawDTxPmjJGXPLcwyzgYh7ljn2Smqx2/UHKNoxw6C6X\nGWcj4sp4RL8LaK+DDs1+fph4ZYzz3LWrSLiWqPAk7h741n/S952rekQBcCJmCzPGXAQ86ZUOxlr7\nd2P0Cz7cgBkIX70AQjqa5AUi94MvIxsm7uMF8YUi/ttFRH7Uq5BXfsdVS09/lTJGMtKvEcrpemsh\nA92SLWfl8hb2dYok4qTZb1MzvzcThdb11kIG9pzM7FVCgEN7amVs9eFqwINGyQ24fepP3gMMeflw\ny41IyAZUxdCGEK4SrFb8TSY0h3h+Yph8HoZp8OFWT4LT+meIDzdH6eCAgBmEpG7Uj7aCyMJNu23V\nppYj5KrnznFttRorgf2wYEXUfQ9xLWwbcImwYftrDZAuhzb37Ddalp7Wxtq5Ihs7iWPscxq0Y0Oz\n6NhzapQNrBtYCiNp9zg3DFFV089A6mRp17jv0gW9VuCnRE9/DdAxjCvkDul2GOgjckh3EifSZOmi\nYcZWLOAdJ8fxacYUfbjGmBokzdAHkAiRzyLm/r3AcsSp/SfW2t5sY2QdezpUCsaYLUh99lHVLIOF\nG1Da8Akgqdn1n1x9jfbDgn2pVDlCsirMXSLtBtdMEYUFa/cGoqThjchiVrPXnk0mV0NFTT+VVZKA\n/Mid7iS1YHVhTlNHrgcah+ABx+5pouAHkICNHxMtqj3ivnvI1QOqWQG9R4lUC12I20ELsqmeWe9X\nNdEPkQaPZItKG49wp8HCLZ8Ep6VHW7jGmLuAp6y1PzTGlCOJNm8Buq21/2KM+TLw3rHe6MedXy7C\nNcY8TvRfwsdXrbVbXJ9bgHXW2jEt3EC4AaULffiT+lL/eJJgl7v2WMVZ64ks4nrgTHBVGlhaKYSn\npLgZsUx9VcNmxN8K4vttRMKFQdJBXui2b0CeyiiXjfRVre2FiMtAfy+6iXI/4PZvJyrJA0La25M/\nPOrDnYOEAWt5ogZGhwP7BOuTscIfO8+Em7tSefKM2PcZY+YDu6y1pybGbUWKSHYaYxYB26y1q5gk\npmThutLBXwAuSFSz9PtYccIrGoh+9gMCZgqUKJJWsLaXEMmpQFit0WtXIATsMBvnN/UWoi6sjAi5\nCSHBBtdW0gRxTTQQt4ZbiSzYKxCXhVrAuP76hA64c9UlUQO0e9dWg1i8Opfe7cR1t1pWXRm9lngh\nz5yVwhNIEQ8oearYhNsEfB8pDHkW8FvgL4F2a+17XR8D9Gh7UvObwqLZJcC3ENbvztEvWLgBMwRJ\n62sO2asbaGUJX8M7SLRodqY7R4MFDrrjagF3EtVQV3ik14AQqX79Gd72UiRWQQm2EbF41aOoXJ/y\n2n61CvUfq8ULYmHrU9xqEX+uH/jwKlGym+TCoV+Y00/zmLimMfcl72vRLdz1wK+BTdba3xhjvotc\n0E0+wRpjeqy1taPHy42p6HD/jTFKB09hvICAIiNJDmPlCPArBPcQqRR01V4tvRTxXAwHETJOufY8\nxIiq8NrlZBbdUkch5fmA24gs1FXEUy1uR9wRmru3kbj7Qq1V/eoziMgVxI3xCJEFXWVgwM9/q9fn\nKzA6ibsJ1KWglSKSP0RJt02xxLLb3Ccr2hFr9jeufT9SJr3DGLPIWtthjHkfkUN7UpiKSmHl+L0C\nAkod2XyKYy2kJTONeblyRz1KKeJ+zaVun5OJ0em2lcR0kc63qOdAyjP29Cs63CeTnKYdHlpCRmXQ\ngRCsDrURIWi1aIcQl4aO144QsE/QmfJDet0aaQeRWsNXbCTDfCsS24VMcJOLzM91H8WtsaOOUA8Y\nY95vrX0F8Yj/zn2uA77h/j5wPDMLkWYBAQEnGKYc2/vnwN3GmJNwQV1I3eX7jDF/ipOFHc/AgXAD\n3uXIpRlN7k/qSZNWm6940D7qYtjLaKlZH3FLupp4mYa9Xv9B0ekCtH0Q6ISBem+cpZHFO3AUyiuj\nNsginC6i9SIuigVeO41nMTsfbrmz4NO/TVznU4lrHyuJTTFTN07NXWGtfRH40BiHLhxj36QQzxys\ndgAACR1JREFUCDcgYEz4C2LZ4KsUVAql7T7iq/WDSPYxdTMoYfuhwz4BJ0uXLydizP3u/B3eXHHB\nCgCHoX0eVDl/8GwkUboGXfR2ytjqx60COArt+mNhgJcgrWHMg94HoppnvgbZT3xTaBdCEiWWn9dD\nINyAgDEx3uKOWrx+cu5k3+SD30/cAvbL1mgkW7/X9s9vJ6rQ0EPkV4VIbqbtSqAuIuC2eeK3HToa\nzX0B0O3aA5oZTUmzh0jmpuP6c/flbHrc9/cWu/5Z6WavCYQbEDAtSCbg1vBXf5FN8zFAZPH2eP07\niecn6CF6ROcQz1er+XhBJFsH3XggSocWMqtkQy0IYapErR66/YrGPW48PxVlO/Caax91x5Twk6G9\n/jyT96MYJXdKNz9jINyAgLygnNH+3GS4q2pzIbIMlQT73Rj6iPq5DJS4dTxVKOlxDcP1AxV8/7Ba\nxOoDbiFe06zejaVRdbsR8vYJFuJE6lu8aeI620KTbrBwAwLeZUgjFquf/CbpYhhMbCdf030r1H9V\nV9+wRj7UIeSoccILEdJd7bW7iGRdR90Y/tyeIfqBeNid4yca96PoFiLWr/9jka2UUbBwfQTCDQjI\nK/x8A35uhmRhRoW/EAWjSQ+ixTRd1GonylamY9QTr7LrB12sRkhQI8RUXVHrtfu976skvghY7eaj\nFnLyh8NHsHB9BMINCMgLsgVT+L5N//HTLFvZVtj9hDBKYkrmak0rga4liueFyN+qlt8z7vxa77hv\nFeoCms61D7GQleBXEqVsVPg5I1SpoHOF0T8s+fTxBpVCQEBAQIEQXAoBAQExDBPX8WpZdr9KsK9n\nTeZ4SFq8fUSv+AfdePp4lyNuhzpvjGGi9Isadqwuic7E+DpWReK4nx/Bl4z5UEter2Ne4ng+Xv+D\nSyEgIGDUqztEBKUlfbKt/EP0Sp8scZOsMTbWeYNEPlglvQb3N+WOP+/aGsiw0Gv7srFkUIcvd9Pg\nj0HvXJ+8/cU37T/dBBks3ICAdyF8YlGVgm+VJpOfJ4MGfPhBFkq2SlRdCOklI7+0fRBZKPMJeZho\nEU3byaAMXyPcR0RknUTZzXDbyRSNybkkM49l8+1OB4KFGxDwLsRYizfJxSIlsWSehXRin0+GKtny\n6675r+1LiC9oJdUE5cQ1vhpi7JN/cj7JuVd7+32NsJJtNskYiXY+yDFYuAEBATGUM9qFkFQx+MTh\n513oIh5YoWkd9XFOht765KfHNVxX9yc1wH3e8Wpvv36fhv/q+H6QhV6LXmehQ32DhRsQEACMrgqc\nrfLBWKTRk+W4vqInk3z71nFP4rhPgkrA6hbQhS7tnyReHVsj3HxLHUb/UBQaQRYWEBAATDwd5GTH\n9H28ELkRIMp14Gcq832uum+J13+sgIxk6Rz/3FJCsHADAgIKBrVoffdELqtPF8V8d8Bg4jhEhIwb\nP1tGtRDamw3vKfYEAgICAqYXw5P4jIYx5hJjTKsx5lVjzJenc2YnKOGmij2BLEgVewJZkCr2BLIg\nVewJZEFqgv2Sdb3y/eqdcn/HymcwkXZy0c4/rlZzDyIzy01aY8+rUEhP4hOHMaYMuB24BFgDXGOM\nWT2q43EiEG5BkSr2BLIgVewJZEGq2BPIgtQE+2VTIOQLqSzffTzQfA1+wIJGuCWVB+ONkxqv0zRj\nShbuOUCbtTZlrR0Gfgp8fLpmFny4AQEBWaD0oCoEX7ebzHlbSpiSD3cJcMBrtwMbpjQdD4FwAwIC\nxoCvA9aFr2TQRqmpExRTkoXZ6ZrFWDDW5nV8jDH5/YKAgIATCtZac7znHg/f+N9njNkI/KO19hLX\n/grwjrX2G8c7p9j88k24AQEBATMFxphy4GXgAuANJKPPNdbavdMxfnApBAQEBDhYa9PGmJuAR4Ey\n4I7pIlsIFm5AQEBAwXBCyMKMMZ8wxvzOGDNijFmXo1/eBM1Zvq/WGPO4MeYVY8xjxpiaLP1SxpiX\njDG7jDHPj9VnmuYz7vUbY25zx180xpydr7lMZl7GmPONMUfc/dlljPn7Aszph8aYTmPM7hx9inGv\ncs6rSPdqmTHml+4Z3GOMuTlLv4Lfr5KDtXbGf4BVwPuBXwLrsvQpQ4o8NSDLq83A6jzP61+Av3Xb\nXwb+OUu//UBtnucy7vUDlwFb3fYG4LkC/NtNZF7nAw8W+P/Uh4Gzgd1Zjhf8Xk1wXsW4V4uAJrdd\nhfhAi/5/qxQ/J4SFa61ttda+Mk63vAqas+By4C63fRdwRY6+x70yO0FM5Poz87XW7gBqjDH15BcT\n/XfJ9/2JwVr7NPD7HF2Kca8mMi8o/L3qsNY2u+0BYC+wONGtKPer1HBCEO4EMZageUmWvtOFemut\npmXqJJ7CyYcFnjDG7DTGfCFPc5nI9Y/VZ2me5jOZeVlgk3sV3WqMWZPnOU0ExbhXE0FR75UxpgGx\nwHckDpXq/SooZoxKwRjzOPLqksRXrbVbJjBEXlYHc8zrltiXW2tzaATPtda+aYw5GXjcGNPqLJnp\nxESvP2kd5XtVdSLjvwAss9YeNcZcCjyAuJCKjULfq4mgaPfKGFMF3A/8hbN0R3VJtEvhfhUUM4Zw\nrbUXTXGIg8Ayr70M+ZWdEnLNyy1uLLLWdhhj3keUsTk5xpvu7yFjzM+Q1+zpJtyJXH+yz1Li1QLz\ngXHnZa3t97YfNsb8uzGm1lrrZ+QuNIpxr8ZFse6VMaYC+B/gx9baB8boUpL3q9A4EV0K2fxXO4GV\nxpgGY8xJwNXAg3mey4PAdW77OsTaiMEYU2mMqXbbc4GPAllXxqeAiVz/g8C1bi4bgV7PJZIvjDsv\nY0y9Mca47XMQOWMxyRaKc6/GRTHulfu+O4AWa+13s3QryftVcBR71W46PsCViH9oEOgAHnb7FwO/\n8PpdiqygtgFfKcC8aoEngFeAx4Ca5LyAU5GV+WZgTz7nNdb1A9cD13t9bnfHXySL4qPQ8wK+6O5N\nM/AssLEAc7oHiTQ65v5vfa5E7lXOeRXpXm0G3nHfuct9Li2F+1VqnxD4EBAQEFAgnIguhYCAgICS\nRCDcgICAgAIhEG5AQEBAgRAINyAgIKBACIQbEBAQUCAEwg0ICAgoEALhBgQEBBQIgXADAgICCoT/\nB6Hl879jtGzMAAAAAElFTkSuQmCC\n"
     },
     "output_type": "display_data",
     "metadata": {}
    }
   ],
   "source": [
    "plt.hist2d(samples[0, :], samples[1, :], (200, 200), cmap=plt.cm.jet)\n",
    "plt.colorbar()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!--bibtex\n",
    "@inproceedings{welling2011bayesian,\n",
    "  title={Bayesian learning via stochastic gradient Langevin dynamics},\n",
    "  author={Welling, Max and Teh, Yee W},\n",
    "  booktitle={Proceedings of the 28th International Conference on Machine Learning (ICML-11)},\n",
    "  pages={681--688},\n",
    "  url=\"http://www.icml-2011.org/papers/398_icmlpaper.pdf\",\n",
    "  year={2011}\n",
    "}\n",
    "\n",
    "@article{neal2011mcmc,\n",
    "  title={MCMC using Hamiltonian dynamics},\n",
    "  author={Neal, Radford M and others},\n",
    "  journal={Handbook of Markov Chain Monte Carlo},\n",
    "  volume={2},\n",
    "  pages={113--162},\n",
    "  url=\"www.mcmchandbook.net/HandbookChapter5.pdf\",\n",
    "  year={2011}\n",
    "}\n",
    "\n",
    "-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# References\n",
    "\n",
    "<a name=\"cite-welling2011bayesian\"/><sup>[^](#ref-1) </sup>Welling, Max and Teh, Yee W. 2011. _Bayesian learning via stochastic gradient Langevin dynamics_. [URL](http://www.icml-2011.org/papers/398_icmlpaper.pdf)\n",
    "\n",
    "<a name=\"cite-neal2011mcmc\"/><sup>[^](#ref-2) </sup>Neal, Radford M and others. 2011. _MCMC using Hamiltonian dynamics_. [URL](www.mcmchandbook.net/HandbookChapter5.pdf)\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2.0
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}