api/python/docs/_sources/tutorials/packages/gluon/blocks/save_load_params.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8af723c3",
   "metadata": {},
   "source": [
    "<!--- Licensed to the Apache Software Foundation (ASF) under one -->\n",
    "<!--- or more contributor license agreements.  See the NOTICE file -->\n",
    "<!--- distributed with this work for additional information -->\n",
    "<!--- regarding copyright ownership.  The ASF licenses this file -->\n",
    "<!--- to you under the Apache License, Version 2.0 (the -->\n",
    "<!--- \"License\"); you may not use this file except in compliance -->\n",
    "<!--- with the License.  You may obtain a copy of the License at -->\n",
    "\n",
    "<!---   http://www.apache.org/licenses/LICENSE-2.0 -->\n",
    "\n",
    "<!--- Unless required by applicable law or agreed to in writing, -->\n",
    "<!--- software distributed under the License is distributed on an -->\n",
    "<!--- \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->\n",
    "<!--- KIND, either express or implied.  See the License for the -->\n",
    "<!--- specific language governing permissions and limitations -->\n",
    "<!--- under the License. -->\n",
    "\n",
    "# Saving and Loading Gluon Models\n",
    "\n",
    "Training large models take a lot of time and it is a good idea to save the trained models to files to avoid training them again and again. There are a number of reasons to do this. For example, you might want to do inference on a machine that is different from the one where the model was trained. Sometimes model's performance on validation set decreases towards the end of the training because of overfitting. If you saved your model parameters after every epoch, at the end you can decide to use the model that performs best on the validation set. Another reason would be to train your model using one language (like Python that has a lot of tools for training) and run inference using a different language (like Scala probably because your application is built on Scala).\n",
    "\n",
    "In this tutorial, we will learn ways to save and load Gluon models. There are two ways to save/load Gluon models:\n",
    "\n",
    "**1. Save/load model parameters only**\n",
    "\n",
    "Parameters of any Gluon model can be saved using the `save_parameters` and `load_parameters` method. This does not save model architecture. This method is used to save parameters of dynamic (non-hybrid) models. Model architecture cannot be saved for dynamic models because model architecture changes during execution.\n",
    "\n",
    "**2. Save/load model parameters AND architecture**\n",
    "\n",
    "The Model architecture of `Hybrid` models stays static and don't change during execution. Therefore both model parameters AND architecture can be saved and loaded using `export`, `imports` methods.\n",
    "\n",
    "Let's look at the above methods in more detail. Let's start by importing the modules we'll need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1d692be0",
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "\n",
    "import mxnet as mx\n",
    "from mxnet import np, npx, autograd, gluon\n",
    "from mxnet.gluon.data.vision import transforms\n",
    "\n",
    "import numpy as onp"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e62c71ee",
   "metadata": {},
   "source": [
    "## Setup: build and train a simple model\n",
    "\n",
    "We need a trained model before we can save it to a file. So let's go ahead and build a very simple convolutional network and train it on MNIST data.\n",
    "\n",
    "Let's define a helper function to build a LeNet model and another helper to train LeNet with MNIST."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "42addbd1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading /home/jenkins_slave/.mxnet/datasets/mnist/train-images-idx3-ubyte.gz from https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/mnist/train-images-idx3-ubyte.gz...\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading /home/jenkins_slave/.mxnet/datasets/mnist/train-labels-idx1-ubyte.gz from https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/mnist/train-labels-idx1-ubyte.gz...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[02:44:18] /work/mxnet/src/storage/storage.cc:202: Using Pooled (Naive) StorageManager for CPU\n"
     ]
    }
   ],
   "source": [
    "# Use GPU if one exists, else use CPU\n",
    "device = mx.gpu() if mx.device.num_gpus() else mx.cpu()\n",
    "\n",
    "# MNIST images are 28x28. Total pixels in input layer is 28x28 = 784\n",
    "num_inputs = 784\n",
    "# Clasify the images into one of the 10 digits\n",
    "num_outputs = 10\n",
    "# 64 images in a batch\n",
    "batch_size = 64\n",
    "\n",
    "# Load the training data\n",
    "train_data = gluon.data.DataLoader(gluon.data.vision.MNIST(train=True).transform_first(transforms.ToTensor()),\n",
    "                                   batch_size, shuffle=True)\n",
    "\n",
    "# Build a simple convolutional network\n",
    "def build_lenet(net):    \n",
    "    # First convolution\n",
    "    net.add(gluon.nn.Conv2D(channels=20, kernel_size=5, activation='relu'))\n",
    "    net.add(gluon.nn.MaxPool2D(pool_size=2, strides=2))\n",
    "    # Second convolution\n",
    "    net.add(gluon.nn.Conv2D(channels=50, kernel_size=5, activation='relu'))\n",
    "    net.add(gluon.nn.MaxPool2D(pool_size=2, strides=2))\n",
    "    # Flatten the output before the fully connected layers\n",
    "    net.add(gluon.nn.Flatten())\n",
    "    # First fully connected layers with 512 neurons\n",
    "    net.add(gluon.nn.Dense(512, activation=\"relu\"))\n",
    "    # Second fully connected layer with as many neurons as the number of classes\n",
    "    net.add(gluon.nn.Dense(num_outputs))\n",
    "\n",
    "    return net\n",
    "\n",
    "# Train a given model using MNIST data\n",
    "def train_model(model):\n",
    "    # Initialize the parameters with Xavier initializer\n",
    "    model.initialize(mx.init.Xavier(), device=device)\n",
    "    # Use cross entropy loss\n",
    "    softmax_cross_entropy = gluon.loss.SoftmaxCrossEntropyLoss()\n",
    "    # Use Adam optimizer\n",
    "    trainer = gluon.Trainer(model.collect_params(), 'adam', {'learning_rate': .001})\n",
    "\n",
    "    # Train for one epoch\n",
    "    for epoch in range(1):\n",
    "        # Iterate through the images and labels in the training data\n",
    "        for batch_num, (data, label) in enumerate(train_data):\n",
    "            # get the images and labels\n",
    "            data = data.to_device(device)\n",
    "            label = label.to_device(device)\n",
    "            # Ask autograd to record the forward pass\n",
    "            with autograd.record():\n",
    "                # Run the forward pass\n",
    "                output = model(data)\n",
    "                # Compute the loss\n",
    "                loss = softmax_cross_entropy(output, label)\n",
    "            # Compute gradients\n",
    "            loss.backward()\n",
    "            # Update parameters\n",
    "            trainer.step(data.shape[0])\n",
    "\n",
    "            # Print loss once in a while\n",
    "            if batch_num % 50 == 0:\n",
    "                curr_loss = np.mean(loss).item()\n",
    "                print(\"Epoch: %d; Batch %d; Loss %f\" % (epoch, batch_num, curr_loss))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86f2a41b",
   "metadata": {},
   "source": [
    "Let's build a model and train it. After training, we will save and restore this model from a file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "6176ca86",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[02:44:20] /work/mxnet/src/storage/storage.cc:202: Using Pooled (Naive) StorageManager for GPU\n",
      "[02:44:22] /work/mxnet/src/operator/cudnn_ops.cc:421: Auto-tuning cuDNN op, set MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[02:44:23] /work/mxnet/src/operator/cudnn_ops.cc:421: Auto-tuning cuDNN op, set MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 0; Loss 2.286125\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 50; Loss 0.311711\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 100; Loss 0.264592\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 150; Loss 0.159274\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 200; Loss 0.231601\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 250; Loss 0.054218\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 300; Loss 0.105487\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 350; Loss 0.013362\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 400; Loss 0.150041\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 450; Loss 0.024895\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 500; Loss 0.069750\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 550; Loss 0.056951\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 600; Loss 0.017378\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 650; Loss 0.062836\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 700; Loss 0.040945\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 750; Loss 0.091340\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 800; Loss 0.131396\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 850; Loss 0.034920\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 900; Loss 0.069949\n"
     ]
    }
   ],
   "source": [
    "net = build_lenet(gluon.nn.Sequential())\n",
    "train_model(net)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e01c8829",
   "metadata": {},
   "source": [
    "<pre>Epoch: 0; Batch 0; Loss 2.288904 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 50; Loss 0.269372 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 100; Loss 0.238990 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 150; Loss 0.320592 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 200; Loss 0.048619 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 250; Loss 0.121555 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 300; Loss 0.083645 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 350; Loss 0.040627 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 400; Loss 0.195946 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 450; Loss 0.155514 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 500; Loss 0.031762 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 550; Loss 0.056516 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 600; Loss 0.095174 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 650; Loss 0.054901 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 700; Loss 0.030067 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 750; Loss 0.102611 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 800; Loss 0.010036 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 850; Loss 0.051853 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 900; Loss 0.008402 <!--notebook-skip-line-->\n",
    "</pre> <!--notebook-skip-line-->\n",
    "\n",
    "## Saving model parameters to file\n",
    "\n",
    "Okay, we now have a model (`net`) that we can save to a file. Let's save the parameters of this model to a file using the `save_parameters` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e56ebc59",
   "metadata": {},
   "outputs": [],
   "source": [
    "file_name = \"net.params\"\n",
    "net.save_parameters(file_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76c6bc10",
   "metadata": {},
   "source": [
    "We have successfully saved the parameters of the model into a file.\n",
    "\n",
    "## Loading model parameters from file\n",
    "\n",
    "Let's now create a network with the parameters we saved into the file. We build the network again using the helper first and then load the weights from the file we saved using the `load_parameters` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "12875aa4",
   "metadata": {},
   "outputs": [],
   "source": [
    "new_net = build_lenet(gluon.nn.Sequential())\n",
    "new_net.load_parameters(file_name, device=device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a6e9612",
   "metadata": {},
   "source": [
    "Note that to do this, we need the definition of the network as Python code. If we want to recreate this network on a different machine using the saved weights, we need the same Python code (`build_lenet`) that created the network to create the `new_net` object shown above. This means Python code needs to be copied over to any machine where we want to run this network.\n",
    "\n",
    "If our network is [Hybrid](./hybridize.ipynb), we can even save the network architecture into files and we won't need the network definition in a Python file to load the network. We'll see how to do it in the next section.\n",
    "\n",
    "Let's test the model we just loaded from file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a32e5e07",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading /home/jenkins_slave/.mxnet/datasets/mnist/t10k-images-idx3-ubyte.gz from https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/mnist/t10k-images-idx3-ubyte.gz...\n",
      "Downloading /home/jenkins_slave/.mxnet/datasets/mnist/t10k-labels-idx1-ubyte.gz from https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/mnist/t10k-labels-idx1-ubyte.gz...\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAABhCAYAAAB26sNJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABB4klEQVR4nO2dd3Sc1Zn/v9N70RRpitpo1KxiuUqWMcZgBwMOIYQ9YYFkgZNDNolhQ8iyWbJLWLK7h2z2nD0py6b8EcgJIQmcQAwEMDbGFtjGVV1Wb6NRmdFomqZo2vv7w797o7HlGmtmZN/POT6gmVfSnav3vfe5T/k+PI7jODAYDAaDwWBkCH62B8BgMBgMBuPGghkfDAaDwWAwMgozPhgMBoPBYGQUZnwwGAwGg8HIKMz4YDAYDAaDkVGY8cFgMBgMBiOjMOODwWAwGAxGRmHGB4PBYDAYjIzCjA8Gg8FgMBgZhRkfDAaDwWAwMsqyGR8vvvgiSktLIZVK0dTUhOPHjy/Xr2IwGAwGg7GCWBbj4w9/+AOeeuopPPfcczh9+jQaGhqwc+dOuFyu5fh1DAaDwWAwVhC85Wgs19TUhI0bN+J///d/AQCpVApFRUV44okn8M///M8X/d5UKoXJyUmoVCrweLxrPTQGg8FgMBjLAMdxCAaDsFgs4PMv7tsQXutfHovFcOrUKTzzzDP0NT6fjx07duDo0aPnXb+wsICFhQX6tdPpRE1NzbUeFoPBYDAYjAzgcDhQWFh40WuuedhldnYWyWQSBQUFaa8XFBRgenr6vOtfeOEFaDQa+o8ZHgwGg8FgrFxUKtUlr7nmno8r5ZlnnsFTTz1Fvw4EAigqKsriiBgMxvWGUChEU1MTjEYjiouLMTw8jIMHD2JhYQHxeDzbw1uSsrIyWCwWrFq1CvF4HK+//jrC4TCWIVLOYFxTLidl4pobHwaDAQKBADMzM2mvz8zMwGQynXe9RCKBRCK51sNgMBgMikgkwq5du9DQ0IAdO3bgrbfeQnt7O7xeb04aHzweD3V1ddi8eTMeeughhMNh7N27F9FoFMlkMtvDYzD+aq658SEWi7F+/Xp8+OGH+PznPw/gbBLphx9+iMcff/xa/zoGI6cQi8UQCoUQi8Xg8XiIx+OIx+NpeU25jF6vR35+Ph588EFIpVIsLCzg8OHD+OSTTxCJRJBIJLI9xCvGYDDAaDSirq4OZWVl8Pv9CAaDiMfjSKVS2R7eeRiNRthsNmzfvh3btm2DVqtFKBTK9rAYjGvKsoRdnnrqKTz88MPYsGEDGhsb8aMf/QihUAiPPvrocvw6xnWMVCoFj8cDj8dDIpFALBbL9pDOg8/ng8/nQyAQQKlUQiKRQKFQgMfjIRqNIhQKIRaL5bS7nMfjgc/nQ6/Xw2az4d5774VSqUQoFEIoFEJ7ezvi8fiKND40Gg3MZjOsVisMBgMCgQBCoRCSyWRO/k1UKhXKyspQXV2Nuro6cByXk+O8Gsh9JhaLIRAIwHEcUqkUIpFIVscjEAjoc3xulQa5T8hYU6kU/fp6+btkg2UxPu6//3643W5873vfw/T0NNasWYP333//vCRUBuNC8Pl8KBQKfOtb34LFYoFUKkVLSwt+9atfZXtoaUilUlgsFlRWVmLjxo0oKSmB0WhEQUEB+Hw+5ubmcPjwYfzkJz9BJBLJSeMJAHQ6HfR6PZ588knU19ejuLgYIpEIqVQKW7ZsQSwWw549ezA4OJjtoV42IpEIEokEzc3N2LBhA0wmExKJBD766COcPn0aXq83J0MYCoUCRUVFUCqVAC4vfr5SMJvNKC4uxqOPPoqGhgZ4vV6cOXMGzzzzTMa9g0KhEBaLBfn5+dQrRvJslEoleDweQqEQBgcHMTs7i6mpKTgcDszOzsLhcGB+fh6zs7PMALlKli3h9PHHH2dhlmWGz+dDKBTCarVCKpVe8DpSex2NRuHxeDI4wr8OoVCI0tJSlJWVQavVYmRkJNtDSkMsFkOj0aC6uho1NTVoaGhASUkJDAYD8vPzIRAIMDc3B6fTCbFYnLOGB3DW+CgtLUVVVRUqKyshk8kgEAgAAHK5HAqFgn69UtDr9SgpKUFVVRXsdjtkMhmi0Si8Xi+CwWDOeXH4fD7kcjn0ej2sVisUCgU4jkMkEkE4HM7JENGVwOfzYTAYsGrVKqxZswbr1q2D2+1GIpG4pCbEtUYul0OpVKKyshIWiwWrV69GeXk5ysvLYbVaabVGKBSCSqXC7Ows8vPzodVq4Xa7oVar4fV66d8n18JiIpEIBoMB8Xgc4XAYsVjskve7SCSCSCSC0WgEx3Hw+XznSWFcS7Je7cK4elQqFfR6PX75y1+ioaHhgtdFIhHs27cPbW1t+L//+7+cPO2dCwm18Pl8SKVSFBcXQ6/XZ3tYFIFAgKKiIqxZswY/+MEPoFaroVAoIBQKwefz6WnVYDBAq9VCIpFkzbV8MYjL+eabb8Zdd92Fmpoa6PX6tNO2w+HA4cOHV5ThCgD33HMP/vM//5Pm4YhEIkSj0WwP64KoVCps2rQJt912Gx588EEoFAqkUin09PSgu7sbsVhsxRogAoEAMpkMt956K775zW/CaDQCQFZCF3w+H+vXr0dVVRWeeOIJGAwGqNVqCIVCGn4hyOVy1NXV0XBLMplEKpVCLBbD1NQUfv7zn6OnpweHDh3K6Ge4GEKhEIWFhXjyySfhdDrR0tKC0dHRJaUuFlNYWIjCwkL80z/9ExYWFvCb3/wGg4OD6O7uXp5xLstPZWQEsnFoNBrodDq6YZz7MAcCATidTkxNTWXFRSgUCun4OI7D/Pw8wuHwRb+HxFdJnsG5i0K24TgOCwsLCIfDCAaDkMlkS3qfeDwedDod1qxZg56eHszPz2dhtBdGq9VSj4fNZoNcLqfzHA6HMTMzA4fDgenp6ZxPmuXxeNBoNJDL5TCbzaioqEBeXh6As0nvgUAALpcLAwMDl1yIs4FUKoXdbqduf4FAgGQyicHBQfT19SEej69YF79UKoXVakVBQQH0ej3EYnHWxsLj8VBZWYn6+noUFBRApVJBLBZjcnISk5OT8Pv9iMfj0Gq1EIlEND9FJBJBqVRCKpVCqVSioKAAGzZsgFKpRDgcps9JtuHxeJDJZKiqqoJarcbs7CwCgcAlx1ZUVISqqioUFxcjFApBq9VCLpcv2ziZ8XEdcaGFKRwO449//COGh4ezcnKSSqVYs2YNBAIBEokE+vv7MTY2dtHvSaVSSCQSCIVCiEQiObfoplIpuFwujIyM4MiRI1i9evWSpeQAYLfb8dhjj+Gll17C8PBwhkd6YXg8Hux2Ox544AHccsstWLNmTdr7brcb+/btw/HjxzEwMJCTJamLEQqFKCsrg81mw2c/+1nU1tbS91KpFEZHR9HV1YV33nkHPp8vewNdAh6PB7VajS1btqCqqgpisRipVAoLCwvYv38/Tp06ldNem0uh0WjQ2NgIu91Oc1my5cURCATYsWMHNm/eTA9FsVgMhw8fxhtvvIHOzk74/X6sXbsWarUaOp0OSqUSGo0GFRUVsFgsqK2tRV5eHh566CE4HA40NDTg1VdfxbvvvpuVz7QYPp8PpVKJTZs2we/3Q6fTweVyoaen54Lfw+Px0NjYiC1btqC0tBSzs7MwGo30b7UcXBfGR1FREUwmEzZv3kxjcT6fDw6HA8DZTXl4eBh+vx/z8/M5t5FdLcTzsVRCGsnIbm9vx+DgIPx+f9ZyDpRKJW699Vao1WpIJBK8++67CAaDtNxxKchJQ6PR5Gyfn0QiAbfbjT//+c84ceIEPvjgAwBnx75q1SoUFBRg06ZNUKlUqKqqQnl5OUpKSjAzM5P1jUQoFEKv16O8vBzNzc0wm810jlOpFEKhEBwOBw4dOoTh4WEkEomcfm7kcjm0Wi1uvfVWlJWVoaGhAfn5+QDOqi7Pzc3hz3/+M3p7ey9632UDgUAAu92OmpoarFq1Cvn5+eDxePD7/fB4PHA6nZienl6xIRfg7P2Wl5cHmUxG77NkMon+/n4MDg5m7LNptVqaXK1SqcDn8+H1etHd3Y3W1lZ0d3fD7XZjYWEB/f39kEgkkMlkEIvFkEqlaG1thVarxa5du1BcXIy6ujrk5eWhtrYW9fX1GBsbw+joaM7kgCyu5LkUJNRNJAKmp6cRCASWbWzXhfFRXFyMhoYGPPHEEygsLMTw8DDGx8dx5MgRAH85RY+Pj6eJ9Kz0UimyQS+1MXMch0QigdOnT6O1tZW6ErOBUqnEtm3bYDKZkJeXh8nJSXR2diIajV7U+CAJnWq1OieNj1QqBY/Hg/feey/tdZFIhM9//vNoaGjAhg0boFKpaPlkSUkJ/H5/1o0PkUgEk8kEu92OxsbGtPlNpVIIBoOYmJjAJ598Ap/Pl9MbH4/Hg1KphNFopMZHRUUFXXBnZ2cxOjqKd999F4ODgzl3ABEIBKioqEBtbS31enAcR13lk5OTK74juEgkglarhUwmo68lk0kMDAxgaGgoY/eXRqOB1WqFVqulFS0+nw8nT55Ee3s7zpw5Q68dGhpa8meIxWIoFAqsWbMGq1atogek2tpajI6OYnZ2NuvGx+Ln+UIH1IsRi8UwPT0Nv99/rYdGWdHGh9VqRV1dHT73uc9h06ZNtMLAarUiLy8vTab9tttuQzgcRjQaRSwWQzAYRH9/P1pbW9HT05MTsbor5c4778RnPvMZWK1WAGcNDnKTDQ0Nob+/H++88w46OzsxPz+flQ1EIBDQRC5iUIjF4gsaTQShUAiJRAKj0ZiWz7ISSKVScDqd0Ol0acm9Op0ONpsNAwMDWXP783g8mEwmlJWV4R/+4R9QXl6e9j7J3v/4449x4sQJeDyenK7SMZvNsFgseOihh1BbW4u6ujqqseL1euF2u/H73/8ex44dw+DgIAKBQE4ZHsDZzez2229HXV0dBAIBTWw8evQoWlpa4Ha7sz3EvxqtVovNmzejpKSEvkYqKnw+X8b+JnNzc0gmk2htbQWPx0N9fT04jkMsFrvsRPxEIoG9e/dicHAQhYWFKCoqQkVFBerr6yEUCtHT03OewnemEAgEqK2tRU1NzWVXp5HEfqPRCIvFglQqhfn5eTgcDni93mUb64o2PuRyOUwmE8rLy1FbW0vdS0qlEkqlkrpdOY5DRUUF9XTEYjHMzc3BYDAgFovB5/MhEolgfn5+RVSCkBLbiooKNDU10bIwktDJcRzcbjcGBgYwMDCA0dHRrIyTxLE1Gg01NhYWFi7Lhb9YjCibyWlXAyltDgQCaQafVCqFRqOBUJi9x47P50Or1cJqtaK5uRlarZa+x3EckskkwuEwBgYGMDY2lpP5NovJy8tDSUkJmpubsXbtWgiFQmqoBgIBjI6Oor29HSdPnoTf78/J51soFMJms6GkpAR8Ph+xWAyhUAgjIyPo7OxcMjmbuMdz2SNFEAqFUCqVKCoqglarpfdTKpWiarOZgtzPo6Oj0Gq1sNvtdE263LlMpVIYGRlBPB7H0NAQJBIJKisrYTQakUgk0rw7mYaUMxuNRvD5fFqZc7H7ns/n02RatVpN7z+/37+sFXor2vgYHx+H1+vFunXrsGbNGmg0mgvGtha/LhAIYDKZcOutt2Ljxo247777MD4+jmefffaSiZC5AEl4stvt0Gg0aRZuPB5HMBjEmTNncOjQIczNzWVljCKRCAqFAk8//TRWr14Nu90Oh8OBPXv24OOPP8bQ0NBFw0DkARgfH4dcLk87MTGuHj6fj1WrVtGEucULJcmNGhgYwG9/+1tMT0/nrOHB5/MhkUhQX1+Pu+66CxaL5Tyj7uDBg3jhhRfgcrly9mAhl8upgU4OEQMDA9i3bx8tj1+qykgul0Mul2c1l+tyEIvFqK+vx+rVq1FYWEj7eJF1qqWlBb29vRn72yQSCYTDYfz617/G/v37afuDQCBwRdVc8Xgcbrcbr7/+OrZv344tW7ZAJBKl6eNkA3LgI/eSx+PB8ePHL+qJ0Wq1MBqNKC0tRUFBAT766CO0trZSnY/lYkUbH7FYDF6vF0NDQ2hra0NlZSV9iBcbG+e67MmpmvyReDwe5HI58vPz4fV6lzXJ5q+FxLfJKUIsFqd91mAwiL6+PoyMjGBycjIruQU8Hg9arRYGgwGVlZWw2+0Qi8WYn59HV1cXpqenL7lgkr+RSCTKqqfgekIsFkOpVNJTNtEkAf5S2ux0OjEyMgK3241gMEg1SxY/Q9kW5+Lz+VCpVLBYLLDZbCgtLYVcLgfHcTS0Sjx/Y2NjV3SqzTRarRYFBQVQKBQQiUSIx+PweDzo7e2Fy+WiJ0+SsEmk+7VaLVQqFf07OZ3OnOxVIxKJYLPZqOFBnuVwOIxAIEDX20waualUCm63G8lkEu3t7QAAp9N5RR4YjuMQj8fh9Xpp+fy5z0mm4fF4EAgEyM/PR35+Pvh8PqLRKFwu10WlDTQaDYqLi6lX1uVyYXZ2dtmTzFf0qk7cxL/+9a/x1ltv4YknnkBdXR1uvvnmy+6Uy+PxkJ+fD4VCgebmZiiVSnz88cdZX2CXgsfj0UTBxsbGNAlmQm9vL/7jP/4Dg4ODGB0dzfhiRB6AtWvXYs2aNVi/fj2sVitisRgGBwfx2muvXVYylkQigUqlQmVlJcrKynKqOmGlYjabUVhYiC9+8YuorKxMC2eRBOW9e/fixIkTCIVC4PF4kEqlVPmQXJdNWXKiAlpTU4Mvf/nL2LBhAxoaGiAQCBCPx9HX14e+vj688sorGBkZyWmvAAD6nJjNZshkMszNzaGvrw979+6l8Xai5XPXXXehtLQU69evR35+PgwGA4aHh+FwOPDv//7vmJmZySkhOx6PB4VCgS984QuorKxMC4k5HA6ag5MN/Zh4PA6Xy4Uf//jHAP6yl6xkhEIhFAoFbrvtNpp/4vf70dvbe9HcjZqaGtxzzz2wWCxIJpMYGxuDw+FYdoNwRRsfBBLHO3bsGD0tkBPFYgQCAYxGI8RiMSQSCc2RIDkU5IHO1eRGsViMwsJC2nQqLy8vTViM5FS43W6EQqGMGx58Ph8WiwXl5eW46aabUFdXB5lMBr/fj2PHjqGtrQ3hcPiyDLvFCqe5JC52pSzldcsWRUVFNNxyriDa1NQUpqamqNfDZDJBo9GgqqoKCoWCVgYkEgn09PTA4/FgYGAAiUQio4u2VCrFxo0bUV9fj9raWuTn59PTdCwWQ1tbG3p7ezEyMpK1kOPlQNYcm82GmpoaSKVSRKNR9Pf3Y3x8HIFAABKJBEqlEhs3boTVakVTUxPteKtWq6FUKpFMJiGXy3HbbbdhYGAAhw8fzhnvh1arhclkQmFhIS0fJmuu2+3GxMREVg8VJP/vaiEHQSJkl21Ikr5Op4NWqwWPx0MkEsHMzMxFPR8SiYQqvC5unrfs413235ABYrEYbXylUChw7NgxlJaWYvPmzWmLvUQiwaZNm6jcNfCXzYBI0no8npzd7ORyOVavXo0NGzZg8+bN9PXFVS7E+LiUgui1hsfjQSgUoqamBg888AA2b96M8vJyhEIhjI6O4pe//CWGh4ezXmKaSS5keGTLAKmrq8Ott966pMJkX18fjh8/jq6uLrhcLmzYsAF1dXW4//77kZ+fD6PRCIFAgIWFBbz55pvo6urC1NQU7Q6bKdRqNf7mb/4GNTU1uOmmm9Lei0aj2Lt3L/r6+jAwMJCz+SoAaH7A6tWrcfPNN0OhUMDtduPIkSPo6elBIBBASUkJioqK8PTTT6O2thYajea8e0etVqO4uBhCoRDHjh3D0aNHc8L44PF4sFqtsNvtNBnzXMn+3t7enFfNvRgk0ZRUG2Ybck8ZjUYYDAYAwPz8PIaHhy/6jEokEloUkEmuC+ODQBQxBwYGMDMzg/Hx8bQbXigU4tNPP6WNsqxWKz155OXlrbjKimxvZuR35+XlwWg0YtOmTVi3bh0aGxuh0+kQiUSwZ88enDlzBj09PVdUtmWz2VBeXn7Rhnm5CnE3k5JPQjQaRSAQyFpIj3gDF5c5J5NJJBIJqsNw3333geM4rFmzBgUFBSguLoZMJqPfI5FI0NjYCLPZjGQyic7OThw+fPiKShWvFJL7s3PnTlRVVdGy+sVev08++QR9fX3o7e29okRZk8kEvV5PvQhdXV0Z6aEiEAjoiZMkypNQQCKRgNlsxs6dO9Hc3ExzWng8HoLBIBwOB5LJJJLJJOx2OxQKBQoLC+FyuVBZWQm3250z5bnnrlEejwczMzM4duwYjhw5knU9jKuFSJjX19ejtLQUwNk8lrm5uZwIEcfjcczMzFy06y7xvmm1WpjNZnAcB7/fj46ODvT39y/7M3BdGR/kgXQ6nXA6nWmCMcDZyT516hSNYa9ZswZbtmyByWSCTqejr+dq2OVSZKNBE+meaLPZcPvtt6OqqgqrVq3CwsICAoEAWlpa0N7ejtHR0SvadEn4hhiDRLF1JUCMD5KPQ8qfs218kEaEi7PxSa4HUZOtqqqCVqtFbW3teTkhHMdBKBSiuroaJpOJ5ve0tbUta8yclFxv3boVa9euRU1NDR0bGX9raytOnDiBsbGxy+qfQ5QfCwoKYLPZYDQaEYvFaCgpE8aHWCymByHg7OGJaF6YzWY0Nzfjs5/9LFQqFc1p8fl8VOo+kUjQ3iRGoxFmsxlFRUW0EiPbkJDp4vXU7/djbGwM3d3d6OzsXDHP9Lnw+XzIZDKUl5fDbDYDOGt8eL3erD3fixWv4/E4pqamaOfdpeaZNO1Uq9UwGAy0/9HQ0BDGx8dZzse1hNSVk1yCaDQKlUqVcXfT1UCSTdVq9QXryDNtNO3atQvV1dVobGyE0WhEZWUlXUgPHz6M9vZ2HDlyhJ7UrgTSvInP5yMSiaCrqwsTExPL8TGuKSTpa8eOHWhoaIBYLEYkEoHH40F7eztaWloyLjBGJKKVSiVkMllaWJEsQKtWrUJpaSmtSFhcYUSk8EkGfE1NDZRKJdatW0e7lb799ttoa2tblvELBAJIpVLU1dWhoaEhbWzd3d3o6enBn//8Z3R3d18y4ZKc9FavXo27774bJSUlMJlMUKlUmJmZwcGDB6nuw3JCKsHUajUAUGOC9Ni46aabUF5eDqVSCZ/Ph5mZGbz00kuYmJjAwMAA/Xva7XaYzea0xmeXm2y/XJAOwk1NTWhoaEgbTyAQwNjYGEKh0Io2PMrKyrBq1SqUl5fTnI+uri689957WVOjXbNmDerr66FSqRAKhfDpp59iYGDggoa0Xq9HU1MTampqoNPpcOrUKfT392es/cANZXwASJvUc0uJpFIp1fGPx+M5lf1M3LRGo5HWcGcLoVAIsViMiooKrFu3DuvXr4daraYJsBzHIRQKwev10s0NODvfxKV9qZMlUUYlVvzs7GxGxYguhkgkol4fctogCy6RXi4rK0NRUVGa0A+5pxarWGYCsnkvpSxLDBGNRgONRpP2fclkEgsLC/B6vZidncXY2Bii0SgtZyedPaurq9HS0rJs4yceFq1WSxd68tz6fD6Mjo7C6XRiZmbmovcVCRtZrVZUVlZi06ZN1Ospk8kgl8sz5vkkuRrnlgjLZDJYLBasW7cOEokEHMdhamoKQ0NDOHbsGCYnJ+FwOJCXlweDwYBIJELHS9zo2c5Zk8lkUKvVKCkpQWlpaZqxGA6Hae+UlQrpnqzT6aBSqSCRSJBMJuF2uzE8PJy1iqP8/HzYbDaIxWIsLCxgbGzsooaQXC5HaWkpDAYDxGIx5ubmMDk5iVgslhHD8IYzPhbj8/kwNDSEjRs3QigUYuPGjVCr1dizZw8mJydz5qRNdEhsNhu++c1v0kzmbFFUVITy8nLceeedaGpqgkQiOW/Bu/XWW7Fp0yZs374d09PTOHz4MBwOBzo7O+HxeK6oZ0AsFoPD4ciZ6gXiprfb7VCr1TCZTLQdNZGRLykpoYJDYrEYBoMBn/vc52Cz2bBnzx4MDw9jcnIyIwaIWCyGSqWCXC6HVCq97HvH4/GgtbUVvb29OHPmDD799FO43W7YbDaUlZXhoYceglAohMFgyEpuDlGSdblciEajlzRoiafg+eefR0lJCSorK6kqcqafp7q6Ovzd3/0dSkpKEI/H8emnn8Lv92Pjxo0oLy+HWq2Gx+PB7Owsnn32WXR2dmJmZoZqeajValit1pzMiVq/fj2am5vxuc99DmVlZWljHB4exgcffLAi21kQSJ6bTqejirRerxcjIyPo7u6+rLDfclBRUYHGxkaIxWJMTk7iT3/6E2ZnZy94vV6vx7Zt21BaWgqO4zA4OHhBUbvl4IY2PhYWFtIUAhcnnOaSS5DP56OwsBDFxcXnqVJmg1gshvn5ebhcLkxMTCy5cEskEqokq1AoEAqFYDKZoNVq4XK5MDc3h/n5ecRiMSwsLCAajVL9iFQqleb5SCQSWfN8kHCXQqGgqpIbNmyAxWJBYWEhlEolDAYDTCYTSktLaZxboVDQEx8xQMxmM1KpFGZmZmCxWGj9/cTExLKWt5FQEPF+8Pn8tAopAsmfiEajGBgYwOTkJFpbWzE6OoqRkRFMTU1hbm6Ozsn8/DzkcjktW88kyWQSsViMNo27WHUXKUGsq6tDVVUVNR7FYnHGx01yawoKCqi2B/HsSKVSGAwGKJVKBINB9Pb2YmBgACMjI+d1QlYoFDAajdQ7kkgkEIlEst7UjM/nQ6fTobi4GFqtFnK5HMBZj8fMzAycTidcLteK9XwQD7Tdbofdbqe5OIFAAPPz8wiHw1nzmJPOu2TNDAaD53lhiKFNwqWk0zhwdj+MRCIZq5a6oY2PUCiEyclJ+gdKJBLUIMklsR6xWIydO3di9erVWZXuJUxOTmJmZga/+93vcPTo0SW7JhYVFSE/Px833XQTLBYLGhoaAIBuvm63G11dXXC73XA6nZiYmEBLSwtCoRDC4TAkEgnkcjnN+ejt7cXU1FTGP6tIJEJeXh6qq6tRUVGByspK3H333bDb7dRAXdyKerGWAXmfPOjV1dWorq7Gtm3bEAqFsH//frS2tuIXv/gFdbsvBwqFAgUFBdDpdDTH4EKbrt/vh8PhwL/8y7/Q9uAkkZssSpOTk1AqlQiFQnRjzzTE8Dh16hT27Nlz0QWTJNQ9+eSTqKmpoR6PbHgPtVotGhsbUVdXh5KSEvB4PCSTSVgsFvB4PNhsNng8HnR3d+OXv/wl3nvvPQSDwfM2NLPZjDVr1kClUoHjOMzPz2N6ehrHjh3L2tpF8k5Iw8/F4eHp6Wm8/vrrOHTo0BUnn+cSMpkMeXl5uP/++2lOl8/ng8PhgMfjybjEwVIQrY6lcpdI6FWhUKSFjYCzz1Q0Gs3YwfuGNj6i0Sg8Hg+CwSCi0SjEYjHUajWKioqQSqVyxs1PPB9Wq/WC8dxUKoVoNErdz8t5A5HKhsHBQczNzS05psHBQajVakxNTUGr1UKtVkOr1aKoqAiJRAJ8Ph+VlZUoLS1FZWUlZmdnUVhYiGAwiPn5eWzZsgVVVVW0tfjk5OSydlg8F1J+abfbYbVa0djYSPUuSMVIPB5HLBZDIBCAVCqlVQnE+Egmk7SZWSKRgFQqhVQqpQmgVVVViMfjWLduHYaHhzE8PLxsn+dS9wPxPh04cCBNpIvEfxcbWnK5HDqdDpWVlQCQ1QX3cjxGDQ0NqKurg91uh8FgSAuzBAIB2sRtaGho2XVLiB7OYuOHNAMDQD1KAwMDVK9nqc+n0+lQVlYGuVyOVCpFn5tsSsnrdDrU1NSguroaVqs1rVpqfn6edntNJpM55Vm+Emw2GyoqKmieBNHRePfdd5f1+b0SBAIB9Ho97rvvvjRDlFThicVi5OXlobS0lFatud1uOBwOjI+PZ6xU+IY2Pkjyk9/vRzgcpgltFRUVWFhYwMDAQLaHCODszVRaWori4uKLGh/BYJAuVsv9cHMch97e3otew+PxcOTIESiVSuh0OpSXl2P79u3Q6/XQarWoqamhJ/FoNIrbbrsNgUAAfr+fZpEHg0H4fD6Mj4/D4/Es62daPG6TyQSLxYI77rgDVVVVuP3229M0C8jJYn5+HmNjY9Dr9dSFTojH45ienqYbu06ng0ajoZoyRNlyaGiINnVbDojxQO4LYhyR/wJnVYK9Xi/++Mc/4tSpUxgbG1tyEyaNq0wmExoaGuDxeNDf378s475WbN68GXfeeWdaNRb53F6vF5OTk3j//ffR39+PQCCwrIsvcXkvNoB4PB4KCgromPx+Pzo7O+FyuS6owGk0GlFdXQ2lUolUKgWfz0cN3Wxt7CaTCTt27MDatWup9gUZSzAYRGtr6yWTgnOd2tpabNmyBQaDAUKhELOzs+ju7sYrr7ySM5olfD4fBQUF+NrXvnbeM6zRaCCVSpGfn08T5Ek4jHRBzxQ3tPEhk8mg1+vT4qaxWGzJWFmuEwwG8dZbb9HW4bkgdMNxHGZnZ+Hz+eB2uzE5OUnLBKVSKaxWKzQaDWpra2EwGFBWVga1Wk3DA8Q9aDabcccdd6CzsxOffPLJso5569attILHaDTCYrFQZcn+/n709vZifHycGqzRaBRzc3PYsmULvvjFL9JNxe12Y3x8HD/72c8QCAQQjUaRl5cHjUZDmwLW1tbSzpHLedom+TQkJr2482YsFkM4HMZ7772HI0eOoK2tDS6X64IbBOlSWl1dndWqCqFQCJVKhbKyMmzYsAEDAwM0iZloMNTX16O5uRlbtmxBYWFhWkk9qTY6ffo0Dh8+jCNHjmBmZgYLCwsZ37yJIRiLxTA9PY3Ozk589NFHcDqdadeR9gWbNm1CU1MTrFYrRCIRQqEQ9u3bh46Ojqxt7AKBAIWFhdi1axcsFgt9PRKJ4MCBAzh58iQmJydzIixxNRCBvk2bNmHbtm1Qq9WYn5/Hvn37cPz4cQSDwayHkpxOJ/r6+lBSUgKVSgWbzUbvZXIAEYvFNG+FPL89PT3Yv38/JicnMzreG9r4EIlEVOeDxF7j8Tjm5+dXRELU4tNrNBpFe3s7BgYGcspwWpz5PTMzg8HBQVoRkpeXB61WC5/PB7vdTkMaRP6etE0nugzLpY9BdF+EQiFqa2vxmc98Bhs3bkReXh4tDSYy8SdOnEBHRwftIUQ2MavVSl3eqVSKJkJ+8MEH8Hq9tERVo9GgrKwM+fn5AP6iyrucxmIymUQ0GsXCwgLi8TikUildlBYWFjA7O4vW1lZ88MEHcDqdF7x/SHlxaWkpLSMGspOcTRI0CwoKUFFRkdYBliR1VldXY+fOndSLJhAIaDgsHo8jEolgaGgIx48fx5kzZ7LWzZp4QIjImMvlomXNJDmYGFTE+LDb7dBoNFhYWEAoFEJnZ2dGVCkvNH6RSAS9Xo/6+no6ZrIunT59Gl1dXfD5fCvS60HKvEnIpbKyEnw+H3Nzczhz5gzGx8epbMNib2KmcbvdGBsbg9/vh1gspp5YPp+PRCJBw91k/SX33eTkJE6ePJlx/aEb2vggJYgajQZqtZq6YI8fP57R/IKrJRfk1a8Gkrw4NzcHv9+PN954AxKJBC+//DJ27tyJe++9F1VVVdDr9eDz+VAqlVi7di3Gxsau+VjIwlJZWYmtW7fi1ltvpRoLLpcLb7/9NsbHx9HW1obp6WnMzMwgFAohkUhArVajvLwc3/jGN2h5ZDKZRDAYxKuvvoqOjg54vV5qyM7Pz9PwhkgkwvHjx8FxHA3fLBd5eXmoqqqiYlrE65FKpXDixAn85Cc/QW9vLxwOx0UbbeXl5cFsNuPhhx9GWVkZhEIhFhYWqPcmkxDDtK6uDgKBAHa7neZoaTQa1NXVwWazYdWqVVQ4jSQv9/X1YXh4GMePH8fhw4fR2dmZEwa7VCpFRUUF7rrrLmg0Gnz88cfo7u6Gz+eDWq3GI488goqKCjQ1NdFkzpMnT6K3txeHDx/GzMxMVjY+mUyGhoYG2O32tLVobm4OExMT+Oijj7LSYfuvhcfjQaVSoba2Frfccgu++MUvUqObCO9xHAeNRoNVq1ZRA9/tdmelh9XBgwdx6tQp7N+/HyqVilZGqtVqWr1GWiY8//zzUKlUWFhYgMvlovlOmYQZH/8/21coFOZstctSLLayQ6EQAoEAAoHAinFrchyHeDyOeDye9qDW1NSkuTCj0ShCoRDm5uau+cNBpK1tNhuqq6vR0NBAy2edTiempqbQ0dGB0dFRnD59GqFQiI6BlK8aDAbU1tZSQ2lmZgYzMzN0g4vH43RDIFUjZIPPVEKzUqmE1WpNK/9dHIsfGhqCx+O5oAFBukAXFxfT3CODwUB7WTgcjmVfuFKpFObn5xEMBmmfE6K3QHQKSCk2CeXp9XpoNBp6CidiXr29vejv70dHRwecTmdGdRkSiQTm5+fh9/sxNzcHpVJJEzMFAgHkcjnNpyHS2HNzc1Cr1WhoaEBxcTFMJhMtdx8eHkZfXx88Hk9Wcg6IR8Zms6GgoCCt2isSiSAYDMLtdmf8VH01EKkFsVgMoVAIpVKJvLw81NfX0yqpxeXZRJ6feE0jkQii0ShGR0ep95yE8TNhFHq9Xvh8PhpaJeJ5Go0G/f39cDqdKC0tpQnwRK7f7/dnpe3DDW18kBLKc5UdVwKLG4OdPHkSPT09+Oijj1aEx+ZySSQS6OvrQ1dXF55//vlrulkLhUKsXbsWlZWVePTRR1FQUIDCwkKkUimEw2H86Ec/ou5iUrJGNjC5XA6VSoUdO3agrq4OZWVltH31K6+8gr1796Kjo2PJEslsUFpaijvuuOO87pscx9GGche7b0pKSlBdXY0HHngAdXV1MBgM1NW/b98+vPzyy8uaDEwUYo8fP45oNIrm5maqRGw2m1FQUIDVq1enVeQspfQZiUQwPDyMZ599FnNzc4hEIhlfcP1+P06cOAG5XA6BQIAdO3agrKws7Rqj0QidTof6+noaViFJ0MR4dDgcGB4exquvvorW1tasqf+qVCoUFxfjy1/+Mk0yBUCr1DweDzweT9ZCWpcDSQIuLi6mAnr5+fnYsmULjEYjVQ1dLJamUqmgUCiwe/duGmolG/qJEycwNTWF/v5+jIyM4IMPPsiIZD9wdt49Hg94PB6mpqaoMUh+t1KphFarBZ/Ph8fjQUtLC86cOYO5ubmMe81uaONDJBJBLpenyf+uJEjsempqChMTEwiFQhd1m680UqkUpqen4XQ6r/nJTiAQYNWqVaipqaGJrxKJBA6HAxMTExgZGYHT6UwTDSI9T8gJdN26dVQ+empqiiqBOhwORCKRrBse5CSt0+lgMpnOU8MkG9rWrVthtVoxMTFBVTRVKhVtfFZWVoby8nIqzhUIBOByuWj7d6/Xu6z3HTE+2tvbEQqFIJPJYDKZYLfbaR7K5fRnItdqtVpEIpErUtm9VqRSKUQiETgcDpw+fRqrVq2iZd0kHEZi8kQfhogKEg9JLBaD0+lEe3s7pqenEQwGsxLSIFUVhYWFsFgsVPoe+EvXV5VKBYPBQMNzucJi4T+VSoX8/HwqB282m6HValFaWgq1Wg21Wp2m3UO8tiQXh7RWIPlEZWVl0Gq1UKlUKCgoQCwWw+DgIPr6+jLy2RZXt537mSsqKlBTUwOhUIj5+Xn09/fD7XZnJVy3MnfdawTR9Vipxkc8Hkc4HEZPTw+6uroQjUazvuFdS5LJJG0cdq2TMkUiEXbt2oXVq1ejsLCQnpJbW1vR0tKCjo6O8+T1pVIpNBoNvvKVr+Dmm29GYWEh3RBaW1vx05/+FGfOnMmKGNpSSCQSFBYWoqSkBGVlZecp4/L5fKxduxb19fXo7+/H6Ogo9u7dC6/XS5tNlZSUwGg0Ij8/n5YXtre3o62tDT/+8Y/h9/uX3d1PkuX+8Ic/QKPRYGRkBBs3bsQjjzxClXQXs5R6K3D2edfpdLj55pvR0dGRFYlvosfT0dGB3t5e2kW4oqLiPJl3opa72LAi4ZZTp07htddew/j4eNYOHAKBAA0NDWhoaKCaI4vHbzQakUql0NDQAJlMhlOnTuWMvodUKkVeXh7uvPNOlJeX45ZbbkFBQQEKCgoAXDiPjoQsFocpSGdYIrJWU1NDrw8Gg9i5cyd+9atfZcz4uBACgQC7du3C5s2bIZfLMTs7iwMHDmB0dDQr41mZu+41Ynx8HPv370dBQQG9eUhpJykRzWWI8TEwMIC+vr6sl3pdaziOo/HIa71oEffk5OQkgsEgXdSPHDmC48ePL+kmNpvNqK6uBnBWsZF05vT7/Th9+vRlt3PPFIlEAl6vFw6HA11dXaioqIBOp8Pg4CAVbrNarVi7di0KCgroRh6NRmE0Gmm8mEjLj42NYXZ2Fm+//TaGhoYQCAQyvvGRjTsajUKpVEKpVFIJ73MhG0hhYSFUKhWGhobgdDrhdDqz4vVYDMn9OX36NOLxOAYHB6HX67Fq1Soq409O2+QZaGtrg9frhdPpxKefforJycmsVuXxeDxotVrqxj93w5ZIJFAqlSgpKclq2IXP50Ov19MOxnq9HgaDARqNBuvWrYPBYIDZbIZCoUir4Eomk9Q74PF4MDU1hWg0Skuiyb1PNFfsdjuKiooglUqplzSZTNJS+1yACNwtLCwgHA7D5/NlbWw3tPExMTEBv9+PxsZGlJaWwmg0UilqkoyTy5BywcHBQQwODmZ7ONecVCqFQCCwLAlbqVQKbrebblyBQADT09M4evQoTpw4saQhZzKZsGbNGiSTSarBEIlE4HQ60dbWhvHx8ZzyPMXjcXi9XoyPj6OjowN6vR5qtRr9/f3U7d/Y2Ii1a9dCr9dDr9efV7GwWCeANM764x//CJfLlZUEx4WFBSrLT4QBL5azxePx0NTUBIvFggMHDmBqagqTk5NZNz5InkBraysmJycxODhIE3l1Oh0NkZGEU6fTiXfeeQcOhwO9vb3w+XxZz6NYbHws5SkQiUSQyWQoLCyEy+XKShkqyf8xGo3YsGEDGhoaUFFRgaKiIqhUKhiNRuo5W6zkS3SfvF4vTpw4gf7+frS2tiIcDiMSiWBkZIRu2jabDdu2bcNtt90GhUIBrVYLqVQKPp9P95FcKWAgAnfE+AgEAlkzYG9o44NoSJjNZlitVoTDYfj9foyMjFxXiZsrAT6fT8NgBoOBViBVVlYiEong448/vqYbeywWw29+8xsaiiCVTi6XiyaXnktPTw9cLhc92QB/ieH7fL4Lfl+2ILHp9vZ2zM7OYmxsDDabDfv27aO5NB0dHWhpaUFRURFMJhPVxCgqKqLNzTweD+bm5vDxxx9jbGyMinFlE5/Ph9OnT19WvsexY8cglUoxOzuLaDSaUzo+DocDbrcbIyMjkMlkOHjwIK0uAkCrRqLRKM0lCgaDOSEiyOfzUVtbi9WrV58XuuY4Dm1tbRgeHsZrr70Gh8ORlbwUmUyGuro6bN68GQ8//DD15MlksrSkZJIjkUwmMTQ0hOnpaap709HRgfn5efh8vvMq1oCz+kUffvgh2tvbodPpYLFYoNPpsHXrVng8Huzfvx89PT0Z/+yLIeE7Ymj5/X54vV7Mzs4y4yMbEKU3uVwOpVJJF6VAIJAzlupSkAeAyHavtPr5pSAZ5yKRiCrOJpNJWg57rbVMksnkFS8IJHN/JUE8PB6PB3q9Hh6PBx0dHfB4PPD5fFR1tqqqCqWlpQiHw1Tqe25uDu3t7ZiamsLU1BROnz6dM63QiaF4OWQrpn05zM/PY35+nrY+v1TLglyC6GCoVKrz2grE43EMDw+jp6cHZ86cydphTigUIi8vD1arFbW1tTQ8RAQByfpJKlUWFhYwPDyMsbExtLS0YHJyEuPj4xc9VITDYYyPj2N8fBwAYLVakZ+fD41GA5/Ph87OTrjd7kx95CUhomOkUzgpW89kF9tzuaGND7LBkSQisgGSOu9czaFwuVzo7+/H8PAwHA5HzjTA+2tIJpM0BrlY5vro0aPo7+/PqXDGSoO4+I8cOYKTJ0/S5mmpVArxeByJRAKdnZ3o7e3FwYMHaafaZDKJSCRCr8mVuDUjN0gkEvjwww8xOzuLL33pS9SL2NfXh76+PvziF79AT08P5ubmsraWRqNR9Pb2orS0FE6nE3l5eVAqlWkG+OzsLHp6emh3ba/X+1d5M2dmZuDxeDAxMYFkMklFCbNJZWUlqqurUVpaCplMhr1796K7uzurntorMj5eeOEFvPHGG+jt7YVMJsPmzZvxX//1X6iqqqLXbNu2DYcOHUr7vr//+7/Hz3/+82sz4msIsXhjsRgWFhZofFAmkyEajWb9hjkXop45NTWFrq4ujIyMYHJyMqe9NFcCx3Fwu93o7u6GQqFAKpXC2NhY1krBrjcWi6QRSJw7EokgEolkPY+AsXJIpVIYHh6GQCDAiRMnaJ7KmTNnMDg4iLGxMbhcrqweHMiaOTExgZMnT0Kr1UKhUKCzs5MqLHs8HvT19VEP319bJk+qs3LJWCcHCqFQCI7jMDMzk/VD6xUZH4cOHcLu3buxceNGJBIJfPe738Xtt9+Onp4e2i0SAB577DF8//vfp19fKBs925AKB4/Hg9nZWepCLCwsBI/Hyzm10HA4jI6ODhw6dAgvvfQSfD5fTljV15KDBw/i8OHD9OvF/VIYDEbukEgksH//fhw4cAC/+tWv6OvkeY3H41l/bpPJJLxeL/bu3YsDBw5Q0S3S/ffcjs/ZHu9yEQwGaZJ4MBhEd3c3RkZGVo7n4/3330/7+uWXX0Z+fj5OnTqFrVu30teJRHCuQ242YqmShlT19fUQCATn6Txki3g8jgMHDkCn09HeFERGNxcSz64lJAzGYDByH3LwyWVxQ1K5cj0d0q4Uv9+PiYkJvPXWW1AqlRgcHMx6HspflfNBytV0Ol3a67/97W/xyiuvwGQy4e6778azzz57Qe/HwsJCWrZtpt2+JMM5mUxCKpUiPz8fO3bsgEAgwJEjRzI6lgsRiUTw4osvZnsYDAaDwViBuFwuuFwudHd3Z3solKs2PlKpFJ588kncdNNNqKuro68/+OCDKCkpgcViQUdHB77zne+gr68Pb7zxxpI/54UXXsDzzz9/tcNgMBgMBoOxwrhq42P37t3o6urCJ598kvb6V7/6Vfr/9fX1MJvN2L59O4aGhmC328/7Oc888wyeeuop+nUgEEBRUdHVDuuKIPG+xUmnpAHQ9Rr7YzAYDAYj2/C4q8g4efzxx7Fnzx60tLTAZrNd9NpQKASlUon3338fO3fuvOTPDgQCGesyy+fzIRAIUFJSgry8PMhkMpqgRDQQGAwGg8FgXD5+vx9qtfqi11yR54PjODzxxBN48803cfDgwUsaHgDQ1tYG4GxfjMv9HZmCZGVfj9LkDAaDwWBkg8vZx6/I+Ni9ezdeffVV7NmzByqViqodajQayGQyDA0N4dVXX8Vdd90FvV6Pjo4OfOtb38LWrVuxevXqy/odwWDwSobEYDAYDAYjhwgGg5eMYFxR2OVCEtcvvfQSHnnkETgcDnzpS19CV1cXQqEQioqKcO+99+Jf//VfL+mCIaRSKfT19aGmpgYOh+Oyv49xZZDcGjbHywOb3+WHzfHywuZ3+bne5pjjOASDQVgsFto350JccdjlYhQVFZ2nbnql8Pl8WK1WAIBarb4u/iC5DJvj5YXN7/LD5nh5YfO7/FxPc3y5OZsXN00YDAaDwWAwrjHM+GAwGAwGg5FRctL4kEgkeO655yCRSLI9lOsWNsfLC5vf5YfN8fLC5nf5uZHn+Kp0PhgMBoPBYDCulpz0fDAYDAaDwbh+YcYHg8FgMBiMjMKMDwaDwWAwGBmFGR8MBoPBYDAyCjM+GAwGg8FgZJScND5efPFFlJaWQiqVoqmpCcePH8/2kFYk//Zv/wYej5f2r7q6mr4fjUaxe/du6PV6KJVK3HfffZiZmcniiHOflpYW3H333bBYLODxePjTn/6U9j7Hcfje974Hs9kMmUyGHTt2YGBgIO2aubk5PPTQQ1Cr1dBqtfjKV76C+fn5DH6K3OVS8/vII4+cd0/fcccdadew+b0wL7zwAjZu3AiVSoX8/Hx8/vOfR19fX9o1l7MujI+PY9euXZDL5cjPz8fTTz+NRCKRyY+Ss1zOHG/btu28+/hrX/ta2jXX+xznnPHxhz/8AU899RSee+45nD59Gg0NDdi5cydcLle2h7Yiqa2txdTUFP33ySef0Pe+9a1v4e2338brr7+OQ4cOYXJyEl/4wheyONrcJxQKoaGhAS+++OKS7//whz/ET37yE/z85z/HsWPHoFAosHPnTkSjUXrNQw89hO7ubuzbtw/vvPMOWlpa8NWvfjVTHyGnudT8AsAdd9yRdk//7ne/S3ufze+FOXToEHbv3o1PP/0U+/btQzwex+23345QKESvudS6kEwmsWvXLsRiMRw5cgS//vWv8fLLL+N73/teNj5SznE5cwwAjz32WNp9/MMf/pC+d0PMMZdjNDY2crt376ZfJ5NJzmKxcC+88EIWR7Uyee6557iGhoYl3/P5fJxIJOJef/11+tqZM2c4ANzRo0czNMKVDQDuzTffpF+nUinOZDJx//3f/01f8/l8nEQi4X73u99xHMdxPT09HADuxIkT9Jr33nuP4/F4nNPpzNjYVwLnzi/HcdzDDz/M3XPPPRf8Hja/V4bL5eIAcIcOHeI47vLWhXfffZfj8/nc9PQ0veZnP/sZp1aruYWFhcx+gBXAuXPMcRx3yy23cN/85jcv+D03whznlOcjFovh1KlT2LFjB32Nz+djx44dOHr0aBZHtnIZGBiAxWJBWVkZHnroIYyPjwMATp06hXg8njbX1dXVKC4uZnN9lYyMjGB6ejptTjUaDZqamuicHj16FFqtFhs2bKDX7NixA3w+H8eOHcv4mFciBw8eRH5+PqqqqvD1r38dHo+Hvsfm98rw+/0AAJ1OB+Dy1oWjR4+ivr4eBQUF9JqdO3ciEAigu7s7g6NfGZw7x4Tf/va3MBgMqKurwzPPPINwOEzfuxHm+Iq62i43s7OzSCaTaRMOAAUFBejt7c3SqFYuTU1NePnll1FVVYWpqSk8//zzuPnmm9HV1YXp6WmIxWJotdq07ykoKMD09HR2BrzCIfO21P1L3puenkZ+fn7a+0KhEDqdjs37ZXDHHXfgC1/4Amw2G4aGhvDd734Xd955J44ePQqBQMDm9wpIpVJ48skncdNNN6Gurg4ALmtdmJ6eXvIeJ+8x/sJScwwADz74IEpKSmCxWNDR0YHvfOc76OvrwxtvvAHgxpjjnDI+GNeWO++8k/7/6tWr0dTUhJKSErz22muQyWRZHBmDcXX87d/+Lf3/+vp6rF69Gna7HQcPHsT27duzOLKVx+7du9HV1ZWWB8a4tlxojhfnINXX18NsNmP79u0YGhqC3W7P9DCzQk6FXQwGAwQCwXmZ1TMzMzCZTFka1fWDVqtFZWUlBgcHYTKZEIvF4PP50q5hc331kHm72P1rMpnOS55OJBKYm5tj834VlJWVwWAwYHBwEACb38vl8ccfxzvvvIOPPvoIhYWF9PXLWRdMJtOS9zh5j3GWC83xUjQ1NQFA2n18vc9xThkfYrEY69evx4cffkhfS6VS+PDDD9Hc3JzFkV0fzM/PY2hoCGazGevXr4dIJEqb676+PoyPj7O5vkpsNhtMJlPanAYCARw7dozOaXNzM3w+H06dOkWvOXDgAFKpFF2AGJfPxMQEPB4PzGYzADa/l4LjODz++ON48803ceDAAdhstrT3L2ddaG5uRmdnZ5qRt2/fPqjVatTU1GTmg+Qwl5rjpWhrawOAtPv4up/jbGe8nsvvf/97TiKRcC+//DLX09PDffWrX+W0Wm1a1i/j8vj2t7/NHTx4kBsZGeEOHz7M7dixgzMYDJzL5eI4juO+9rWvccXFxdyBAwe4kydPcs3NzVxzc3OWR53bBINBrrW1lWttbeUAcP/zP//Dtba2cmNjYxzHcdwPfvADTqvVcnv27OE6Ojq4e+65h7PZbFwkEqE/44477uDWrl3LHTt2jPvkk0+4iooK7oEHHsjWR8opLja/wWCQ+8d//Efu6NGj3MjICLd//35u3bp1XEVFBReNRunPYPN7Yb7+9a9zGo2GO3jwIDc1NUX/hcNhes2l1oVEIsHV1dVxt99+O9fW1sa9//77nNFo5J555plsfKSc41JzPDg4yH3/+9/nTp48yY2MjHB79uzhysrKuK1bt9KfcSPMcc4ZHxzHcT/96U+54uJiTiwWc42Njdynn36a7SGtSO6//37ObDZzYrGYs1qt3P33388NDg7S9yORCPeNb3yDy8vL4+RyOXfvvfdyU1NTWRxx7vPRRx9xAM779/DDD3Mcd7bc9tlnn+UKCgo4iUTCbd++nevr60v7GR6Ph3vggQc4pVLJqdVq7tFHH+WCwWAWPk3ucbH5DYfD3O23384ZjUZOJBJxJSUl3GOPPXbewYTN74VZam4BcC+99BK95nLWhdHRUe7OO+/kZDIZZzAYuG9/+9tcPB7P8KfJTS41x+Pj49zWrVs5nU7HSSQSrry8nHv66ac5v9+f9nOu9znmcRzHZc7PwmAwGAwG40Ynp3I+GAwGg8FgXP8w44PBYDAYDEZGYcYHg8FgMBiMjMKMDwaDwWAwGBmFGR8MBoPBYDAyCjM+GAwGg8FgZBRmfDAYDAaDwcgozPhgMBgMBoORUZjxwWAwGAwGI6Mw44PBYDAYDEZGYcYHg8FgMBiMjPL/AOiPOWE3vpnEAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model predictions:  [7 1 4 2 6 6 6 1 2 4]\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def verify_loaded_model(net):\n",
    "    \"\"\"Run inference using ten random images.\n",
    "    Print both input and output of the model\"\"\"\n",
    "\n",
    "    def transform(data, label):\n",
    "        return data.astype(np.float32)/255, label.astype(np.float32)\n",
    "\n",
    "    # Load ten random images from the test dataset\n",
    "    sample_data = mx.gluon.data.DataLoader(mx.gluon.data.vision.MNIST(train=False).transform(transform),\n",
    "                                  10, shuffle=True)\n",
    "\n",
    "    for data, label in sample_data:\n",
    "\n",
    "        # Display the images\n",
    "        img = np.transpose(data, (1,0,2,3))\n",
    "        img = npx.reshape(img, (28,10*28,1))\n",
    "        imtiles = np.tile(img, (1,1,3))\n",
    "        plt.imshow(imtiles.asnumpy())\n",
    "        plt.show()\n",
    "\n",
    "        # Display the predictions\n",
    "        data = np.transpose(data, (0, 3, 1, 2))\n",
    "        out = net(data.to_device(device))\n",
    "        predictions = np.argmax(out, axis=1)\n",
    "        print('Model predictions: ', predictions.asnumpy())\n",
    "\n",
    "        break\n",
    "\n",
    "verify_loaded_model(new_net)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "220605ec",
   "metadata": {},
   "source": [
    "![Model inputs mnist in 1](https://raw.githubusercontent.com/indhub/web-data/4a9c100aa996df3dff0e7f493029d411c2b526c3/mxnet/tutorials/gluon/save_load_params/mnist_in_1.png) <!--notebook-skip-line-->\n",
    "\n",
    "Model predictions:  [1. 1. 4. 5. 0. 5. 7. 0. 3. 6.] <!--notebook-skip-line-->\n",
    "\n",
    "## Saving model parameters AND architecture to file\n",
    "\n",
    "[Hybrid](./hybridize.ipynb) models can be serialized as JSON files using the `export` function. Once serialized, these models can be loaded from other language bindings like C++ or Scala for faster inference or inference in different environments.\n",
    "\n",
    "Note that the network we created above is not a Hybrid network and therefore cannot be serialized into a JSON file. So, let's create a Hybrid version of the same network and train it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4f4870fd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 0; Loss 2.309808\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 50; Loss 0.270773\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 100; Loss 0.234400\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 150; Loss 0.194822\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 200; Loss 0.094595\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 250; Loss 0.056388\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 300; Loss 0.113372\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 350; Loss 0.059691\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 400; Loss 0.206528\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 450; Loss 0.082874\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 500; Loss 0.020909\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 550; Loss 0.156303\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 600; Loss 0.015081\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 650; Loss 0.028083\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 700; Loss 0.035424\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 750; Loss 0.039477\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 800; Loss 0.073234\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 850; Loss 0.008373\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0; Batch 900; Loss 0.095614\n"
     ]
    }
   ],
   "source": [
    "net = build_lenet(gluon.nn.HybridSequential())\n",
    "net.hybridize()\n",
    "train_model(net)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f6bfe09",
   "metadata": {},
   "source": [
    "<pre>Epoch: 0; Batch 0; Loss 2.323284 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 50; Loss 0.444733 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 100; Loss 0.103407 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 150; Loss 0.166772 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 200; Loss 0.227569 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 250; Loss 0.069515 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 300; Loss 0.074086 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 350; Loss 0.074382 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 400; Loss 0.026569 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 450; Loss 0.097248 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 500; Loss 0.059895 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 550; Loss 0.053194 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 600; Loss 0.076294 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 650; Loss 0.047274 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 700; Loss 0.007898 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 750; Loss 0.039478 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 800; Loss 0.031342 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 850; Loss 0.059289 <!--notebook-skip-line-->\n",
    "Epoch: 0; Batch 900; Loss 0.037809 <!--notebook-skip-line-->\n",
    "</pre> <!--notebook-skip-line-->\n",
    "\n",
    "We now have a trained hybrid network. This can be exported into files using the `export` function. The `export` function will export the model architecture into a `.json` file and model parameters into a `.params` file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "4ca3aae2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('lenet-symbol.json', 'lenet-0001.params')"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "net.export(\"lenet\", epoch=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ac2e69f",
   "metadata": {},
   "source": [
    "`export` in this case creates `lenet-symbol.json` and `lenet-0001.params` in the current directory.\n",
    "\n",
    "## Loading model parameters AND architecture from file\n",
    "\n",
    "\n",
    "### From Python\n",
    "\n",
    "Serialized Hybrid networks (saved as .JSON and .params file) can be loaded and used inside Python frontend using `gluon.nn.SymbolBlock`. To demonstrate that, let's load the network we serialized above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "c0e17933",
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "with warnings.catch_warnings():\n",
    "    warnings.simplefilter(\"ignore\")\n",
    "    deserialized_net = gluon.nn.SymbolBlock.imports(\"lenet-symbol.json\", ['data'], \"lenet-0001.params\", device=device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33b8c3f3",
   "metadata": {},
   "source": [
    "`deserialized_net` now contains the network we deserialized from files. Let's test the deserialized network to make sure it works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "7737e0ef",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAABhCAYAAAB26sNJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABIVElEQVR4nO2deXSb1Zn/v9r33ZK1eN/j3bEdYyBkIWQrtGFvoW3KTMuUQktLp4eB82tpO2eGGTpnTmdaTpmeOQU6ZSsQkrKThKzE2byv8m7LsiXZlmXJWixZen9/5LwXmziJQ2zJSe7nHB+I9Eq6unrf+z73Wb4Ph2EYBhQKhUKhUChxgpvoAVAoFAqFQrm+oMYHhUKhUCiUuEKNDwqFQqFQKHGFGh8UCoVCoVDiCjU+KBQKhUKhxBVqfFAoFAqFQokr1PigUCgUCoUSV6jxQaFQKBQKJa5Q44NCoVAoFEpcocYHhUKhUCiUuLJixsfzzz+PjIwMiMVi1NTU4PTp0yv1URQKhUKhUK4iVsT4eOONN/DEE0/gmWeeQUNDA8rKyrBt2za4XK6V+DgKhUKhUChXEZyVaCxXU1OD6upq/P73vwcAxGIxpKam4oc//CH+6Z/+6aKvjcViGB0dhUKhAIfDWe6hUSgUCoVCWQEYhoHP54PZbAaXe3HfBn+5PzwcDqO+vh5PPfUUeYzL5WLLli2oq6s77/jZ2VnMzs6Sf9vtdhQWFi73sCgUCoVCocQBm82GlJSUix6z7GGXiYkJRKNRJCcnL3g8OTkZDofjvOOfffZZqFQq8kcNDwqFQqFQrl4UCsUlj1l2z8fl8tRTT+GJJ54g//Z6vUhNTU3giCgUCoVCuTbh8/mwWCywWCxYt24dGIZBNBrFhx9+iIGBAcRisSv+jKWkTCy78ZGUlAQejwen07ngcafTCaPReN7xIpEIIpFouYdBoVAoFArlCwiFQuTn5+Pmm2/G008/jVgshnA4DJvNhqGhoWUxPpbCshsfQqEQlZWVOHjwIHbt2gXgXBLpwYMH8dhjjy33x1FWMTweD2VlZbBYLNi0aRMAIBgM4sMPP8Tx48cTPDoKhUK5vjCZTEhOTsaGDRtQVFQEADh16hSOHTuG3t5eRKPRuI1lRcIuTzzxBHbv3o2qqiqsW7cOv/3tb+H3+/HQQw+txMddEDbblmEYCIVC8Hg84g5iXU2xWIz8d7XA4/HI3xdZ6skRi8XId0wUHA4HGRkZKCoqwr333guGYeB2u9Hd3U2Nj6sULpcLiUQC4FyyeCwWW1XXzsVgr6fFxsvn88HlcsHhcMDhcMDj8RCNRhGNRjE3N7fs35H9LPY6pVBWGi6Xi6SkJKSmpqKwsBBpaWlgGAZ9fX04cOAAnE5nXM/FFTE+7r//foyPj+MXv/gFHA4HysvL8dFHH52XhLqSKJVKpKenIxAIIBQK4Vvf+hYKCgqg1WoRjUYxNTWFvr4+tLW1oaOjAw6HA36/P+ELqVKpRE5ODsrLy7Fu3boFz/n9fvT09CxpjHa7HWNjY+jt7YXX612p4V4UHo+HNWvWoKSkBCqVCjabDceOHcPIyEhCxkO5MmQyGfR6PX77298iGo3iT3/6E/r7+9HZ2ZnooV0UNrSbl5cHDoeDwcFBcg3x+XwIhUKsX78eRqMROp0OBoMBpaWlaG1tRXNzM9577z0MDAxc8TiEQiG4XC64XC4MBgOMRiOGhoYwPT296PHhcBhzc3NX/LkUCp/Ph0gkwp133ony8nLU1tZCKBTC6/VidHQU3d3dmJmZie+YVuqNH3vssYSEWfh8PvR6PQwGA9asWUOMj7KyMhQWFiIpKQnRaBSTk5NQKBTgcrmYnZ0Fl8vF4ODggrLfRCCRSJCZmYk1a9agoqJiwXN+vx9SqXRJ3gydTgedTgePx4NIJIJQKBT3HRaHw4FUKoVMJgOfz8fc3BwmJycRDAbjOg7K8mA0GpGRkYGKigoEg0HodLrzcrtWCzweDyKRCGKxGDqdDiqVCkVFReByuZBKpcT4EAgEEAqFKC0thclkQlJSEvR6PYqLi+H1ejE4OAiBQHBZn816TlgvikKhgEgkgkqlIjeB5ORkmEwmyOVyeDyeRd9namoKPp+PXMOr3UPCfm+ZTAapVAq5XA4+/+K3mHA4jGg0Cr/fj9nZWbIBXO3f9WqCw+FApVJBp9MhOzsbWVlZEIlECIVCGB4extjYGLxeb9wN3YRXuyw3Op0OP/vZz5Cfn48bbrgBDMOAYRhIpVIIBAISdklOTkZ+fj62bduGU6dOobOzE88991zCd+Vmsxm7d+9GdnY2cnNzFzzHMAxqamqW9D6xWAyRSAS/+tWvcOLECbS0tMTdsGIYBpFIhCycc3NzmJmZQSQSies4KFcOh8PB7t27sWHDBmi1WoyMjJBra7UhEAigVCqRm5uL0tJSbNq0Cfn5+bBYLBAKhYhGo2TcrIEgEAiIVyIUCmFkZAStra347LPP4Ha7L+vzhUIh1Go1BAIBRCIRtm/fjqysLFRXV0Oj0SA5ORk8Hg9cLnfBWL7IkSNH0NzcjNdeew0jIyMIhUJXPDcrCWtgbdq0CbW1tdi0aRNMJtMFj49GoxgeHobL5UJdXR3a29tx7NgxBINBzM7OJjRkfK3A5XLB5/OxefNmbNiwAbfccgt0Oh26u7vR3NyM//3f/4XNZoPP54v7tXxNGB88Hg98Ph/JyclIS0tDdnY2FAoFBgcHEQgEEAwGEQqFEI1Gyc4jKSkJOp0ORqMRFosFsVgMRUVFEIlEy1Zu9GWIxWLw+/2IRqOL7riEQuFlvVdlZSV4PB4mJibgdrsv6OJdCRiGwdjYGPR6PWKxGKRSKdLS0pZUA361wOFwIBKJkJaWhqKiIoyPj8Pr9cLtdhMBvXA4vOpvHBdDLpdDqVQiJSUFFovlsj0B8YLH48FoNEKj0SAvL494EHNycmAymaBSqS449mg0ikgkAqfTCYfDgaamJrS0tGBsbOyyfzuz2YyNGzdCIBCAz+ejqqoKJpMJFosFEolkQXXffBVIDocDPp9P1rPs7GwAwNDQEIaHh9Hf3w+/34/p6elVlWvDXgMWiwVr165FVVUVCgsLYTKZoNVqL/g6Nt9OJpMhGAxCo9FApVLB4/HA5/NhaGgIPp8PExMTq+a7Xm2IRCKo1WpkZGSgoKAAEokEoVAI7e3t6OjowMjICLxeb0I2EdeE8SEWiyGTybBhwwYUFBSgoqICg4OD+NOf/oTh4WGMjIxgZGQEgUAAUqkUycnJWL9+PdavX4+dO3ciKysLGRkZGBkZQXNzM/7nf/4nYeGXmZkZdHZ2IikpCSUlJVf0XhwOBw8++CC2b98Ou92Orq4utLS0xO1Ei0ajOHv2LGZnZ/HAAw/AYDBg48aNaGhoiMvnxwOBQACNRoO77roL//Iv/4JDhw6hvb0dJ06cwMTEBBwOBzweD+x2e6KH+qUxmUwoKChAfn4+0tPTV23bA6FQiJtuuglr1qzBPffcQ0Io82EYZtHxh8NhTE9P4+jRo2hoaMDLL79MduCXS3V1Nf74xz+e93gkEkEwGFxUbBE4ZzzJ5XKIxWLI5XKUlpaiuLgYubm5GB4exuuvv46hoSE0NzcTo3Y1wOPxoFarUVNTg6eeegrJyckXNTpYOBwOkpOTiReaXZdGRkYwNjaGV199FVarFUePHk14OPxqRaVSoaCgAOvWrcOGDRvg9Xpht9vx1ltvoa+vj3gwE8FVbXyIxWIoFArccMMNKCgoQGpqKkQiEd566y0MDw/j1KlTmJ6ehtfrJe5+1oI+ffo0YrEYuFwuampqYDabsWbNGpL/kSjcbjcOHz6M/v5+HD16FMC5i5uNDxuNxkUXT6lUioqKigWekfku5dTUVExPT6O1tTXuJxv7ebOzs3C5XAgEAnH9/JWAw+FALpfDbDZj165dqK2tBYfDQVZWFnnc7/fD6/XC7/fD4/Ggs7MTfX19GB4eht/vh0gkQiwWW7ULK5/Ph0QiQUFBATZv3gy9Xg+GYeByuWCz2TAwMIDJyclEDxMAkJOTA7PZjG3btiE9PR0GgwFSqRQcDgeTk5Pwer3Eo5mUlESuIbfbDa/Xi5GREUxNTaGpqQl2ux3BYPBLx8A5HA64XC78fj/8fj+am5vhdrsRDAYxPT2N/v7+Ra9BgUCApKQkKJVK6PV65Obmwmw2Q6PRQCwW46677oLVaiVJszab7YrmbDkQiUTQ6/W49957UVpaumDeL8UXj2H/rVKpwOPxsHPnTuTk5KChoYFUHVGWhkAggF6vR3l5OXbt2kWMu9OnT6Orqwt9fX1wuVwJDZte9cYHu5vesmULfD4fRkdH8ec//xl2ux2Dg4PnvWZubg7BYBDj4+MIBoMIh8NITU1Famoq8vLyEAwGFy1xjRdTU1PE6GDh8/moqKhAcnIyysvLiXHEXqwcDgc6nQ7FxcULjA/2xOJyuTCbzRgfH0/IrpX9zFAoBJfLBb/fH/cxLDccDgdKpRJZWVl48MEHyY05PT0daWlpqKqqAsMwpGIhHA5j37592L9/P6anpxEOhyGTychzqzF3gs/nQ6VSIT8/Hxs3bkRSUhJisRjGxsZgs9kwODgY1zDeheBwOMjOzkZxcTFuu+026PV6El5hGAYTExOw2+04fPgwwuEw8vPzyTXO7v5aW1sxMTGBkZGRZcs18Pv9GB8fx8GDB9Hb24uZmRmMj4+jubl50d9bJBIhJSUFOp0Oqamp2L59O3g8HrKysmA2m5GdnY2WlhY4HA4Eg8FVYXyIxWIkJyfj/vvvh9lshk6nW7DGLPY9L+R9Aj5PjlSpVLBYLMjKyiJSDdT4WDpCoRApKSmorKzEfffdB5FIhGg0ilOnTqG+vh79/f0JDwVf1cYHj8eDUChEOBzG5OQkXnzxRfT29sJqtS5pYkdHR3H8+HFUVFRALpdDp9PFYdSXTzQaRU9PD4aHh9Hd3U0uXLPZDIPBgE2bNiEjI2PRzHI2Xrxnzx4MDQ3FNYmLy+UiKysL2dnZ4PF4cLvdOHPmzAXdzlcLKpUKarUa999/PwoKCkgsn4VhGHg8HszNzZEkRj6fjw0bNqCkpARf+9rXMDMzA4lEArfbjYaGBrS2tq66cFRmZiYeeughrF27Funp6RCLxQgEAnjvvffQ3t4Ot9u9Klz/HA4HhYWFqKqqgkKhINdBT08PGhsbcfjwYVitVjgcDkSjUcjlcnIN+f1+BINB+Hw+oluyXFitVpw5cwbHjh3D0NDQJQ3NSCSCsbExuN1u4lnS6/W4++67kZOTg9raWqSmpuIb3/gG+Hw+pqamMDY2lhBPIutRraqqQlFREdLS0qBUKs87jmEYknc3OTkJt9uNiYkJFBQUICkpCcDn4ab5mykWiUSCqqoq9PT0xDVkvFyweTwCgYAUPLCdX1cij4XL5SI1NRXZ2dl4/PHHkZ6eDolEgunpaXg8HtTX16O+vn5VXLdXtfHBMAxisRgCgQCmpqbQ2NgIq9W65GqKmZkZhMNhjI+PY2pqCmq1GsDSdOnjCXsz43K58Hq9ZHxyuRxarRZarRZ6vf68cBGb8NnX14fu7m64XK64jpvD4UCr1UKn04HL5SIQCGBoaChhuiPLAZfLhUajgdlsRnl5OUluZnfS4XAY4XAYDocD4XAYQqEQAoEAEokEKpWKlFjOzc1BLBbD5XKR37W9vX1BaDDRqNVqVFVVISMjA0qlkuQsWK1W9PT0IBQKJXSsHA4HYrEYUqkUFosFqampxPPH3sgbGxtJNVu8ylXD4TA8Hg9GRkZgtVphs9kwNjZ2ydexaxlrTDidTgiFQmRnZyMWi6G8vBxSqRSFhYVITU2FRqPBxMTESn+dRREIBJBKpcjOzkZOTg6USiUxwNk5ZpN4JyYmSGjL4XBgdHQUYrGY3ADZCiSRSETeg13jhEIhcnJyEAgE4h4yvpQHZ/5x7CZjfpk1n88nOjJisRhisZgIy7ESD8ttOHK5XJhMJmKsyuVysr7Y7XbY7XY4nc5VscZc1cbH9PQ0AoEADhw4gJ6eHjidzssq42RPGo1GA5PJBLFYDKFQCLlcTrQxVhNJSUm48847iUv5tttuQ0VFBZRKJYRC4Xmej1gshhdeeAEHDhxISGyew+HAbDbDYrGAy+XC4XDgwIEDq25elwqrWfLwww+jpqYGxcXFkMlkC4y+uro6tLa2Ys+ePRgfHyfVDXK5HNnZ2UhPT8eWLVuQkpIClUoFpVKJ5ORkKBQKEpMdHR1N4Lf8HDaZlr0hDA0NYXBwEGfPnoXNZkvoAsbqeHz961/HXXfdhZKSEuh0OohEIni9XrS0tOCDDz7Aiy++SPK94nXjqq+vx2OPPQa73Q6Hw4Gpqakv9T5sPtCbb76J06dPw2w2k6oFhUJBdEMSQVlZGQoKCvDd734XmZmZCyp42CqWwcFBjIyM4Le//S0cDgfGx8cxNzeHSCQCiURC1jGVSoXa2lpUV1fjjjvuWFARpNVq8aMf/QgffPABDhw4sCJqs4vBhn/YcujZ2dlF1y0ulwuZTAaDwUCqdSwWCzQaDYqLi8nvxK4dHA4Hc3NzqK+vR1dXF/7whz8sq/SASCTCD37wA5SWlkKpVIJhGASDQbz77rt49913MTAwsGpKmK9q44OVP2ZlYS9XvIr1nLDvw4Zx5HI5Kc9NFEKhkOwEhEIhpFIpzGYzysrKyIKTm5t7Xgdgn8+HkZERzM7OIhgMoqenJyE3My6XC4FAQDwzHA4HkUhkyV6P+TLXfD4fMpkM4XAYgUDgotoIKwG7i9FqtTCZTMjOzkZ2dvaC0s1QKIRAIIC+vj60traip6cHk5OTEIvFZJcYCoXg9XqRlpaGaDSKkpIS8jxbqmu1WhNufHC5XMjlcqhUKshkMggEAjAMA4fDgYGBAXi93oReG3w+H1KpFHq9npTT6nQ6iMVizM7OYmpqCp2dnQlLiPV6vejo6MD09PQV69qwLQmEQiF6e3shFotRWFhIPGnxLntmQ4gWiwX5+fmLltOy10JnZyd6e3vR1dWFiYmJC+YHKRQKYoQHg0EIhUKSF8JKKGg0mnh8PZI/J5PJkJ6eDj6fj2g0iunpaUxMTEAkEpEwCrt5VSqVMJlMUKvVUCqVMJvNUKlUKCwshEwmg0KhgEQigUQiAcMwmJ2dRX9/PzFGlgupVAqNRgOLxQKTyQQej4dAIICJiQnYbDb09/evKoHHq9r4YBkYGPhS+Qxzc3OYm5uD0+nEyMgIzGYz1Go1srOzweVyL6g8GA8MBgMyMzNRUFAAk8mE0tJSGI1GrF27lrj4F0uMbWlpwb/+679iYGAAw8PDCbtJiMViqFQqFBcXo6io6LKTeFmXpUajgUajwdq1a2G329Ha2gqfzxfX7yUUCqHVanHzzTdj48aNqKyshMViWbBwjI6Oorm5GXv27MGRI0dISGJ2dpZUXNjtdvB4PLS1tSE3Nxe///3vSRfosrIypKWloaGhIeFy5SKRCFVVVaioqEBaWhq4XC5isRgOHTqEo0ePwufzJWxsPB4PKpUK6enp2LBhA9atW4e0tDRwOBxEo1FSLv/888/HPczI4vF4FmgnXKmhzLaD2LNnD6amprBp0yYYDAYUFhaipaVlOYa8ZNgb3M0334xt27YtahSMjY1hYGAAzz33HBobGy8ZSvT5fDh27Bi0Wi1cLhcx1uMNj8eDQCDA1q1bUVpaittvvx1yuRxzc3Po7u7GmTNnkJqaCp1OB71eT+TyFQoFTCYT2TCxf/N7BQHnDBt288Tes5bTi5OTk4OcnBwYDAZi2DidThw/fhytra0YGhpaVTkz14TxcaUN1NxuN0ZHRxGJREjjrMsR81pOkpKSUFVVhfT0dJLlrlarSUKXSCRatBQ4FAqhp6cHIyMjUKlU4HA4Ca0qicVimJubg8/ng8/ng0wmW9Lr2PhoXl4e9Ho9srOzoVarkZmZiampKRQXF6OzsxM2mw2jo6MIBoMrppjK5hSkpKRgw4YNKCsrQ1FREXHHsjF6m82GlpYWHD58GIODg+flQsyPgcdiMWg0GhgMhgWNDn0+H2w2W8LLkDkcDiQSCcrKypCTkwMul4uJiQk4nU5SGZIohVrWrV1TU4O0tDRUV1cjJSVlQfWXQqEgXZTZ5EYAxDPK6mwEAgF4PJ4VU2ldidAAq4oKAIFAAJOTk3FNHORwODAYDCgrK0NGRgZ0Ot2CsM/s7Cx8Ph/a29tJYvlSy8ij0Sjcbjc6OjoglUpJMurFKmOWG5VKBb1ej9LSUlRUVECv10MikSAajSIzMxMMw0Cr1RLRPT6fT9aI+QnnPp8PkUgE0WgU4XAYMzMzRHDQ5XJhamoKDQ0NGB4eXpbzhPVG5efno6qqCkqlkqz/o6OjaGhoiHvTuKVwTRgfV8ro6CisViu2bNlCej8kyvjIyMjAT3/6U2RmZiIrK+u85y90Ic7MzOCjjz5CIBBAXl5ewmXi2eTE0dFRGAwGGAyGJb2ObV62c+dOrF27Frfccst5iqhvvfUWjh49iv3798PpdGJubm5FLiwulwuVSoWysjI8+eST0Gq1JCkZ+HxHevToURw6dAh//etfL/meHA4HeXl5KC0tXXCOjY6O4uTJk5ct5b3c8Hg8KJVK3HbbbcjKygKXy8XAwABOnDiBxsZG9PT0JGRcXC4XKSkpyM7Oxg9+8AOkpKQgIyPjvGMMBgNUKhVMJhM8Hg/Gx8cBgJQIj4+P48iRIxgZGSHlm6slBn4x2B026xFwu93o7++Pq7HK6tjs2rULJSUl513Tfr8fw8PDOHjwIN55553LDnmNjY3h8OHDMJvNyM3NjavhAZzrXVRaWorNmzejsrJywXNKpRJ5eXkXfO38bumsyjEbZh0cHMTU1BQmJibQ1dUFp9OJ9vb2ZfPesp6im2++GTt37iTCepOTk+jq6sIHH3yQsMTki0GND5zLTg8Gg4jFYqQ+ml204o1UKkVOTg40Gs1lXXgymQybN28mtfDFxcW466670NDQgJGREZw8eTKu8T6hUAiZTIa0tDSkpqYuOexSXV2NXbt2obi4mCQBs42nQqEQZmZmkJWVBa1Wi+HhYQAgrszlRiaT4fbbb0dFRQXJKQDOGR2hUAh1dXXo6+vD22+/TcZyMUpLS1FYWIitW7ciJycHQqEQfr+f/D4ffPDBkqoiVgpWcTItLQ1paWlkEYtEIis2x0uFx+Nh8+bNKCkpIdU3X4S9XthwnVQqhVarJd6N1NRUYpzX19fjrbfegtPp/NIJofFEKBQiLy8PKSkpCdPqEQqFMJlMqKqqWiDUxuJyufDpp5/CarVienr6snU5BAIBFAoFMcrj/T21Wi2ys7OJl/ZCnx+LxeB2u+H3++F0OjEzM0O8bFNTUxgeHiaVlOFwGF6vl+TgTU1NIRAILKv3UKlUwmKxkK7MfD4fPp8PjY2NJN9mKYZOfn4+CeexCb9jY2Po7+/H4ODgsudPUeMDWFB/P19lMBGIRCKYTKbL9rxIJJIF1npNTQ1isRjeeOMNNDc3o62tjbgC4+F+Y3vo6PV66PX6BXHQC7kaeTweCgsLcd9990EqlZJkr9nZWXg8HkxPT2N8fBwFBQUoLi7G3/72N0xMTKyYfolEIsGNN96IgoICqFQqkqDMupfr6+vR1taGw4cPX/LzWY/HrbfeiurqalgsFgAgO6O2tjbU1dUlfBeelJQEk8lEPAjAOeM80SJPXC4Xa9euRXV1NYxGI3H3s78J+/8sbHUBC8MwJAa/du1aSKVSnD59GsFgkIRfVits75SMjAwYjUZiTCUi6Vqn06GgoGDRY9hwAtvKYn7zvi9yIYVXhUKxIImW/X3j8V3ZkB1bEjsftpcOW6AwPj4Ot9tNbu5DQ0Po7++HzWaD3W5HIBCI2/Uik8lgMplI92bgXBi+q6uLJIhfCi6Xi4yMDOTn52PHjh3E29zR0YHjx49jenoabrd7WX8Hanzg85Jb4NxCa7fbEyYbPTMzg9bWVpjNZpjN5it6Lw6Hg82bN6O8vByZmZkYGBjA/v37Sa39ShIIBMDhcFBfX49oNIry8nKoVCqUlJRgdHT0PM+SyWTCpk2bsG7dOiiVSni9XkxPT+Mvf/kLhoeH0dXVhXA4jEgkgqeffhq7du3CTTfdBIVCQQyr5YbtysuGdSYnJ+FyubB37150dHSgoaEBHo/nknFblUqFlJQUrFmzBvn5+ZDL5QDOGb0ulwuHDh1CT09P3Kt4vgiXy0VxcTHKysogFApJF+LW1la8//77cDqdCRvbhRgZGYHdbofL5VrUs8eKOvH5fJSXl0OtVkOj0aCsrAzPPPMMXnvtNXzyySew2WyrqhKAhcvloqSkBPn5+di8eTPUajUJJ42NjSVEmv9C4RCJRAKLxYKpqSmEQiE4HA5EIhEYjUaIRKIF5bgul4t4B9hzPiUlBdu2bVtQwRcMBtHU1ISOjo4VN0JYITQ2j2b+92Q3GR0dHbDb7cSb4fV6iSwDWyG53GJ1l0KpVCIjI4MY2wzDwOv14sCBAxgaGrrk61UqFbRaLcrKylBSUoKioiKyRqWkpGDt2rXg8/lEO2e58oyue+OD3VWwsdRIJILJyUnMzMwkZDzBYJBkJfN4vMsWnWLbWrPVImyZWjAYhFqtRnd3N8Lh8IobH2yyldvtxtTUFBiGgVgsRlJS0nlublZPoqSkBGazGXw+n4g01dfXo6+vD52dncT16/F4iLw5m1y1EsRiMRKrdTqdGB0dhc1mQ0NDA1paWjA8PHxRo4dVgdRoNMjJyYHFYkFSUhIEAgFisRhmZmYwOTmJ/v7+Zd9VfBnYnAmj0Qgej0duIKOjoxgZGUnozZk1/sbGxiAUCkmybl9fHwYHBzE6OnpegvX884ItlY5EIgsMkLq6OiQlJRHJ8tUGm+uSmZlJyifZnIKZmZm4eqNYY5wth/2ixohYLIbRaITX6yXqvqFQCBkZGZBIJKTnC8MwEIlE5NpiNxVSqRQmk2lBpcvc3BzsdjvpbLuS10goFILH4yFtN+Z7YNjkzY6ODvT09BCjYzVIvovFYqJxA4DIGoyOjl40pMjn80m/MIvFgrS0NKJRwv4GAoEAQqEQKpUKQqFwWdfa69r4EAgEEIvFKCkpwY033khKk44ePZowz4fdbsdLL70EgUAAHo+H7u7uyxpLTU0NnnjiCaSnpyMlJQXAue9ZVFRELux9+/ahra1tpb7CojAMA5lMhqysLIyPj5McCT6fj8zMTFRVVeGBBx6ASqVCLBbD22+/jaNHj6Kuro5IEVssFhQXF8NoNGJubg5NTU1oaGhYseoLj8eDF154AWq1GhaLBWNjYxgbGyO7nkstPGKxGAUFBVi/fj0effRRUq3Bqr2+//77aGhowMcff5zw5nKslkxOTg7y8/PB5/PR1dWF//iP/0B7e3vC2m6zhMNh/PM///N5Nz22XP5SXiM+n4+zZ8+ipKQEv/jFLyCRSMDj8ZCeno7y8nIMDg6uOuVdHo8HiUSCHTt2oLKyEjKZDCMjIzh48CCamppgs9niFqZj9SmGh4dx5MgRlJSUkPWFJTs7G9/97nfh9/sRCATQ3d2NUCiE3NxcSKXSBYnjvb29GB0dxcsvv4yhoaELlpeHQiG0tbVdsBnfcsJWnxQXF4PP5yM3N5fc0NPS0rB161aS+FtXV7dqjFWTyYTa2toFHZxjsRi8Xu9Fx5iRkYH77rsP5eXlqKioIIKC80P+g4OD+Oyzz9DY2Aibzbas1VXXtfGh0+mQkZGBzMxMosIZiUTg9/sTdjMIBoMYHh4mO4ShoaHLat7V3d2Nuro6RCIRyGQyyOVyCAQCiEQiKBQKpKenE0GcQCCwoqV6sVgMExMTcLlciMVikEqlSE9PJwsNl8uFWCzGmjVrkJubC7VajUgkArvdjqGhIQwNDcHv9yMSiRCpdla4Z2ZmBna7HaOjoyvm4oxGo5icnCS9KdxuNyYnJ5f0eazyYWlpKfLz84nrmc/nkzlpa2tDb28v/H5/wr0ebK6H0WiEVqslu1aHw5Fww4PlShJDWW/a/O/CeqYuVL6eaJRKJZKSkkgPJ1bsqru7m6iFxgs292JychIdHR0wGo1ISkoiWhfAuU2OSqWCRCIh+WWzs7Mkj0IikZB1LRKJQCwWw2KxIBKJwOfzkd37/OT0ubk52Gw2jI+Pr/g5yDYCtFqtEIlE0Gq1RJ2U9coUFhaCz+fDZrORflWJujbY81cmk0GtVkMgEBANkZ6eHoTD4fOMU61WSwQrs7OzUVhYiKysLLI+sXPPfiefz4ehoSF4PJ5lVwm+ro2PiooKfPvb3yZCRdPT06SSIVFaBmyMneVyf+z29nY89dRTeOihh0gMn1UglMlkKC4uRldXF0pLS9HV1bWiQkxsF8VAIICvfvWrMBqN2LJlCzo7O3H8+HGS3Pt3f/d3yMnJgUQiQV9fH86ePYv6+npYrVbMzc2RZLfc3Fw88MADEIlEsNlsOHv2LJqamlbst2IbQPl8vste/NiF9ZFHHoHFYiGiP7FYDKdOnUJ7eztef/11TE5Oroobe1VVFTZs2LBAQI0NQa6WHd5KEO/EzcshNzcXxcXFKCwshMVigdfrRV9fH95///2EVOPFYjFYrVa88sor0Gq1MBgMRGwL+DxHgm2ilpubC2DxhFOz2QyNRoOKigoYjUZkZWWhqqoKarV6gWfL7/fj6NGjcWlGOT09Da/Xi9dffx3Hjh0jXavz8vKgVquhUqmQl5dH7g9NTU349NNPE3avYEO6rOgZq/D78ssvo7m5meiNsLBJ23l5edi9ezd0Oh20Wi1EItGiIRWGYeB0OnHmzBm4XK5l97Jd18aHUCgkpV1cLhcikQhKpRJFRUWw2+0J08q4ksWQYRjMzc2hubkZIpGInGDA55LlZrMZN954I0mgXEkikQi5AIRCIXQ6HflLSkpCSkoKLBYLVCoVAoEArFYrPvnkE4yNjSEajYLD4UAmkyE3Nxd5eXlITk4m6q1erzeubueLwUrz5+bmwmw2IzMzE2lpaTCZTFAoFOBwOJiamsLU1BTp/zI9PZ3w7pJyuRx6vR6FhYUoLy+HQqFANBqFw+GA3W6H2+2+JowPhmEwMzMDn8+HYDBIJLJXI6zSZmlpKbZs2QKpVAqv14u2tjZ0d3cn9DcJBoNwOp3o7OyEVqtFfn4+aZg4X+xt/n+/yPznRSIRkpKSsGbNGmRmZoLP55/nhYpXtQvweRNPhmHw/vvvIzMzExs2bIDBYIDFYiE9hW6++Wbo9Xo4nU64XK6EtESIRqMIBALw+/3w+XzQaDQLvEbzPXys9+yGG25ARkYGUQM+c+YMCgoKsGbNmgXNMdkOuCdPnsTQ0NCK5EBe18aHQCCAXC4nljZ7s66pqUFLS0vChbquhNOnT6O+vh47d+5EUVHRgufS09OxdetWnD17Fu3t7Ss6jmg0StzD7ELDJjVmZ2cjIyMDaWlpkEqlcLvdaGpqwltvvUVez7pya2trUVpaCpPJhJMnT+LMmTPwer2rojsj8Pm5dOONN6K2thYbNmyATqdbkDzncrnQ29uLAwcOoLm5OeGGBwDSg6KyshI33HADhEIhQqEQ+vr60N/fj4mJiVWRVHelsBUAU1NTmJmZId2GVyNs75ra2lrce++9JDHz9OnTaGlpSZgGEQDSdbehoYG49dlE6qUmI37R+JBKpdixYwfEYjFJJE6kN4ot63/llVeQnp4OhmFQXl6O5ORk0m9qx44dKCkpgdVqRUdHR8KMD5/PB4/HA4/HA6PRCLFYTIoNgM8rObOyslBWVobbbrsNRqMRNpsNvb29ePfdd3H33XcjNzeXGH2hUAg2mw2vvfYaurq6VkxY8Lo2Ptrb2/GnP/0J69evR35+PsrKyqDVarFr1y6IxWIcPnw40UNcEbRaLdGuWElisRjsdjsUCgVcLhdp2FRdXQ0Oh4O0tDQkJSVBIpFgfHwcH374IckHyczMhMFgwIYNG2CxWFBZWQmj0QgOh4OOjg7s378/ob13WPh8PtRqNdasWYPa2lpyLmm1WggEApJ13tDQgNOnT6OpqQkDAwMJv6GzlUMlJSX40Y9+hNzcXCJO5HA48Prrr5OwV6IMvIyMDFRWVpIOwPv27SNl8Jd7c+JwOFCr1dDpdFAoFEQwbjXBqiuXl5dj586dKC8vRzQaxfHjx9Hd3Y29e/fGJfywFDo7O+FwONDe3g6tVovy8nKYzWYUFRUtaoTweDwi+sZKpwsEAtTU1JDcr/ntBqLRKD777DO0t7evSFUP+3kX6grMho98Ph8++eQTeDwekpBtMBhIDtoDDzyAI0eOoLOzE4FAIOEdu3k8HrKzsxEMBlFXVweNRoP8/Hxs2bIF1dXVSE9Ph0QigUwmg8ViQVZWFtLT0yEQCDA+Pg6Px4NDhw6hr68Pp06dWlEBvuva+LDb7ZidnYVMJsPc3Bzy8vKg0WhQXl4Oq9UKLpe7qmPCF4O10BdDKBRCqVSu+M6PdWGOj4/D5XJBJBKRPi0CgQAmkwlyuRxCoRCBQICUsCkUCqSlpSErKwvbtm2DxWIhOxC2l0qim6+xsKGk3NxcrF+/fkEVQCwWI71ETp8+jWPHjqG+vj6u7d0vBI/Hg0KhQGZmJjZt2kTO9ZmZGbhcLpw+fRp2uz1hhgePx4PRaERNTQ2KioqQnJyMtrY2Ug55uTcjLpcLtVoNtVpNOgkD525yl1vOvlLweDzI5XLk5ORg+/btsFgsmJubg9VqRVNTE5qamhJ+c2NxOBxwOBzo6emBTCbD2NgY8vLySL+T+bChCo1GQ/K82McXayHBlum3tbWhpaUFs7OzyxZe5fF4pDmhSCQi3WkvxtDQEIxGI0ZHR2E2m8nxUqkU69atw/j4OORyOdH7iDdsAi8rpMe2FtDr9WTjVl1djXXr1pHXsJ148/LyiHd6fHwcdrsdJ06cwMDAAAYGBlb0urimjA+5XA4ul0vcgZdKBGKrWvbt24e6ujqSV8DmSWRnZxNr8GrDZDLBYrEsqtR69OhR/PWvf0V9ff2Kj2N2dhYjIyN49tlnsWnTJjzxxBNISUmBXq8n5cRsyeOPfvQj4tZl++skJSUhGo2it7cXbW1tOHjwIE6cOLHi474UrM5IdnY2HnnkEeTm5qK0tJQ0mIpGo/B4PDhw4AAaGxuxZ88euN3uVWF4iMVipKWl4cc//jEKCwuJ6mwkEsGxY8fQ2toKm82WsNJTuVyOkpISbN68Gffccw9RvXzooYfQ2dmJ3/zmNyT7fimw1QD33XcfioqKFpQStra24r333ktYaf18lEolNm7ciHXr1iE3Nxd+vx82mw319fVobm5OWGLjxWCF6Jqbm2G1WnHkyJEFzwuFQhQUFMBgMGDNmjUQCATIzMy86HtarVb09fXhzTffRHd39wK11Ctl7dq1KC4uxrZt22A2myGXy5cULpLL5dBoNAuEvFjvCJs3xSofx5uRkREcOXIEer0eycnJqKqqQn5+PvLz86FUKonn8EJ0dHSgpaUFb7/9NlFsjYdQ2jVhfLDJY1lZWRCLxSRr+VJ9MqLRKOmmyDDMArW9+aqnVwNsd0W2MVtKSgpSUlIW1NaHw2FMT0+jv78fjY2NcWlixnYTtVqtSE9Px+Tk5Hk1/8C5G2JqaipJimX1G9iYptVqRVtbGxobGxPeJIlVa2Qz4QsLC2E2m6FSqRAMBknH0fHxcbS3t6O7uxtjY2OrwvDgcDgkQbCkpITIvPv9fni9XvT29qK3txfBYDBhoSE2JKRQKJCcnEySELOzs0mJJqu4uBQ0Gg1xMaemppKS+tnZWbjdbjidzoTf2NmwREZGBkwmEyQSCYaHhzEwMICxsTEi1LcU2M7QK9nxeT7RaBQzMzOYmZnBxMQEWTvlcjlJ6BeLxaTU/EKEw2GSb2C1WjEyMoLx8fFlvQmaTCai4ms2myGTyS57nWe7qIfDYYyPj2NiYoJ0rE4EbIsGVhZALpdDIpGQDu1qtXqBgRWLxYgSq8/nQ09PD1pbW9HR0YHe3t64rVHXhPFhMBhgNpvx3HPPISMjAw0NDThx4gT+67/+a0mZ0jKZDFqtFikpKUhOTiY3j97e3lXhjl0KIpEIJSUlWL9+PZ588kmyYM9PeHQ4HHjzzTdx7NgxNDY2xq1SJBwOY3BwEGfOnMHrr7+O2tpaVFRULHos+1tNT0/D4/Hg3XffRV9fH8nxcLvdCe9/kpOTg4yMDDz++ONIS0tDeno6CXGxCoh79uzB0NAQrFYrwuFwwkXEWPh8Pm688UaUlpaipKSEnB+NjY04e/YsXn/9dfT39yd0vKFQCFarFeXl5QA+3wjk5OSAy+UiKSmJyFkvhdtuuw3r168nDQK5XC7pLGqz2RIeymDzBzIzM7Fjxw4StnvzzTfxxhtvYHh4+LJ2/6mpqSgtLUVjYyPsdvuKdX2+EGKxGFKpFFu2bEFeXh6++c1vEgGri4V6HQ4HWlpasHfvXpw8eRIOh2PZ19/q6mo8+OCDUKlUSwq5LAartmuz2fC73/0OPT09aGtrS9i6NDw8DLfbjZycHHA4HFRXV0Mul5ME2S/ChrS6u7vx/vvvkw3HcnqYlsI1YXywVigrLZ6VlYWJiQmsWbMGLpeLCEN9cWLZxJuSkhJkZ2eT3bjP54tLF0+ZTAaJREIuBOBcaer09DQJP8xHLBYjPT19wQnF5XKRlpZGcilKSkoumHnO5iCEQqG472rZMITVakV+fj75LVgRIa/XuyBj3Ov1kuZtbC+YRO7GARDRtuLiYuTl5SE1NZXIpQcCAXg8HrS0tKC5uRm9vb1wuVzw+/2rxoDl8/mQSCRIT09Heno6KTFnGAZTU1MYGRkh6q2JJBaLkQ3AwMAAjEYjKSNUKBRYt24d1Go16uvrEQqFLlg1xN4E2RAk2zAsEonA4XCgrq4uoV2EWbhcLvR6PYxGIwwGw4J1yOl0YnZ29oI3BVb2mtXWkEqlKCgoQGVlJWw22xVXYXA4HOK9YLVqLoRYLIZGo4FarYZSqURNTQ1JKpfJZOflg7Chi2g0imAwCJvNhlOnTpEqq5Xw2rBlwsnJyZBKpUsOuwQCAVItxXazdTgc6O7uhsPhSOi6xApjjo6OYnBwEGVlZSTnj22GNzw8DIfDQe4rXV1dGB4eRk9PD5xOZ0LWqWvC+PD5fODxeKSnQElJCUQiERwOB44dO4a6urrzkpY4HA50Oh2ys7Oxe/du3HDDDTAYDKTvSTzabLPty0tLS6HRaAAAbrcbra2tGBoawsDAwILjtVot7rrrrgUxaz6fj29/+9tkt5SIdttLxeVy4fDhwwu8HoFAAB9//DHa29uxd+9ecgGwCol+vz/hng4Wo9GI3Nxc3HPPPSgvL0dKSgoxGh0OBxobG/HGG2/g8OHDcd9tLgWpVAqdTod169YRpUa2S6fdbkdHR8d5Bm8iiEaj8Hq96Onpwccff4zNmzeT60Oj0eDxxx/HiRMn4HK54HQ6LxiG02g0pFNnTk4OhEIhOaeamprw+9//PmE9nObD4/GQn5+PoqIiYhRyOBzSqv1ipKenQ6/XQ6VSkQZjJSUlWLduHVpbW6+4jQKPx0NaWhrJgbtYiMJisaC6uppoSrACVheCXasikQicTidOnz6NP/7xj5iZmVkxb9Tx48cxPj6OtWvXwmg0Ii8vb0mJ90NDQ2htbcXZs2cxMDAAl8uFUCiEmZmZhF/nbIi6o6MDALBlyxZotVoiahiJRLB//3589NFHGBwcJN3BI5FIQsv9rwnjIxKJYGZmBp988gmcTifuvfde6PV6bNq0CWlpabjpppvQ2dkJt9sNj8dDWkPn5eWhvLwc5eXl0Ol0pCzy5ZdfXnH9C+CcC3Dnzp0wGAwkUTEYDKKqqorUbs9HoVCgrKxsQRULm8V/sUWB7SPS29uLTz/9lPRVSQTzL1TWKmez4dmmc1NTUwtaWCcKHo9H8mduuukmmM1mWCwWFBYWQq1Ww+fzwe1247PPPsPg4CCJmSa6O+2FqKqqQklJCXJzc5GcnAzgXLJaQ0MDzp49i76+vlUlKOZyuXDq1Cnk5eUhNzeXNJRLSkpCRUUFvv/972NkZASjo6NEdG5mZgYymQz5+flIT08nJfRqtZpk9H/88cc4efIk/H5/wkueWdhcJ/b/AaC0tBR33HEHpqamwOPxUFhYeN6NMjMzE0qlEjKZDGKxGGq1Gnq9HgqFAgUFBRgZGUFTU9N5TfeWCluRwoptXaiCDjiXMGs2m6FQKBb1dLAwDIORkRFMTU3B4XAQfZ+2tjbSTmGlYLuyTkxMQC6XkzDcpZieniZiYqz3Y7VsMFitlPLyctxwww2k8sblcsFut6OtrQ1Hjx5FZ2cnPB4PZmdnF5VejzfXjPERjUZx4MABjI6O4vbbb4dOp8P69euxbt06hEIhfPTRRxgYGMDg4CBEIhFycnJQXl6O9evXk3JaNo73l7/8JS47osrKSnzrW98CsHweC/ZimP/fmZkZksnc0tKS8AuGnW82SYsVVdLr9YuGmxIFn8+HyWRCVVUVfvjDHxKXMnBud866Lf/85z/DZrOhv78/sQO+BJWVldi6dStyc3Mhl8vBMAzsdjs+/PBDsqNbTUxMTODMmTPYtGkTAoEAEU/SarXQaDQoKirCwMAAhoaGcOLECdjtdjgcDhgMBtx+++1EnpztSzI9PQ2Hw4F33nkHAwMDq8rQAj6/LthwRGlpKQQCAfr7+yEUCnH33XeTTQqLTqeDWCxe4A1l15K8vDy43W50d3dfkfGRmZmJ0tJS7Ny580uX589fjxiGwfDwMAYHB9HU1ISRkREcOHAAwWBwxX8Tp9MJp9NJvATXAmwZc1lZGW666SZIJBKSuNvY2Ii9e/eiu7sbNpst0UNdwDVhfADndtFDQ0MIh8N44403kJeXh9raWpJFfvPNN6OiogKBQAA8Hg8ymQwqlQoMw5C272+//TY6Ozvj5opiRXoyMjIgl8uX5T29Xi9sNhtaWlrQ3t6OoaEhTE5OorW1FT6fL6GGB9uorKWlBYcPH0ZpaSnkcjkqKyuhVqths9niEu5aCiqVCgaDAffddx/WrFkDg8FAFviJiQm43W68+uqr6OnpQUdHx5de3OOJVCqFSqUiu1c23HLgwIG4VD5dLqyU99/+9jfYbDZ885vfRGpqKmlQxufzYTaboVarkZqailAohFAoBJFIBKPRuOCaikajRNG3tbV1VZXPx2IxjI2NQa1Wo7OzkyTQFxQUwGKxwO/3g8vlwmw2n+d5CAQC8Pl86OrqIrk6SqUSWq0W4XAYSqVyVVTteTwedHZ2orOzEy0tLejt7cXk5CSmpqbId0j0TvxqJRAIwOFw4A9/+APeeustiMVixGIxUinocDhWRXjxi1wzxgdw7sbL5XJJJUdOTg4UCsWCzF8ul7sg1sgmD42OjqK+vp64zeOBx+PB8PAwCZuIxeIFC8V878BiLr7FnmOrdBoaGlBXV4eOjo5Vs9CymgCsSJjBYCAJm2yr9EQvlOzuWq/Xk6qB9PR0IkQXDAaJO7OxsRGDg4OYmJhIuDdpKXC5XPB4PJKgPTMzg/HxcQwNDSV6aIvCxrLZ1uzr16+HWCwm6rGsx0wul8NgMCz6+kAgQH63vr4+kgi8GqTtWVjp9/HxcfT392Nubo6I7xkMhkXPLXZuJicn4fF40N7eTrwGOp0ORqMR09PTyx5WYr0ySzlubm6ONOp0Op2wWq04c+YMjh8/jrGxMWpwLBPsudDc3JzooVwWHGaVrZper/eKZL+5XC6USiU0Gg0yMzNx66234pZbbiE7IZ1OBw6Hg3A4jObmZnz22Wc4cOAAurq6MDU1RWrN40FJSQny8/NRWlqK1NRU3HPPPQt2awzDoLe3F+Pj4+js7DzPJTk4OAiHw4Guri5i2bJtrNnwRTzEYi4HDocDuVwOmUwGmUy24Gbo8/nI7jVRFBUVIScnB9/85jdJSS2r69HW1obm5mbs27cPVquVlGkmujpkqTz88MNYv3498vLy4Pf78c4776CtrQ2HDh1K9NAuCmucZmRkwGg0Yvv27UhJSUFhYSGSk5Oh1+sXfV17eztOnTqFrq4uInHvdrtXpbHIdhbV6XSwWCwkZ+VC362npwe9vb0YHR3FzMwMpqenyXXOzhcb2/8yirAsUqkU3/ve91BaWooHH3xwyWGXWCyG+vp6DA8PY8+ePRgbG0N/fz/8fj+RS19N6xJleZmenl5U4HI+l+X5ePbZZ7Fnzx50dXVBIpHgxhtvxL//+78jPz+fHLNx48bzVO7+4R/+AS+88MLlfNSXJhaLwePxkBI8g8FAPB+sngdwrta5vb0dLS0t6O7uTsjuz+12Y2BgADweD263G2lpaURBj2VgYADj4+Nk9zcfm80Gp9OJnp6eq8LtDyxsU79aYJUK2TLU4uJi5ObmkoZ3s7Oz5Ddobm5Gd3c3BgcHEQgErqoFdHR0FJ2dnfD7/fD7/Whra4Pdbk/0sC4J2xl5cHAQHo8HZrMZExMTCAaD0Ov1RK77i3R1daG5uRk9PT1wOBwYHR2Nu5bBUpmdnSWbBrak3Ov1XvC79fb2krXhi2JirId3ObrBxmIx4hkeGxsjVTVsHg1btj9/XlmhvebmZgwNDaGjowMTExNwOBxXbbsKyvJzWZ6P7du34+tf/zqqq6sxNzeHp59+Gm1tbejo6CA3zY0bNyIvLw+//vWvyeukUuklrSCWK/V8zIeNC7MNi+ZnlAOfK5wmKmuZHQ87vsWyw9kFZLGFhH38aroBrjbYeU9OTkZ+fj4eeOABbN26lbj2WXf93r17ceLECZw4cWJZe03Ek/nXAgCiuHo1nT/zr5P51/VisGXEq6Fy6nKYHx6+2HeL12/HGuXf+c53UFlZiU2bNhGvSk9PD8kpY70rrKrv4OAgfD4fUY6mRsf1w7J7Pj766KMF/37ppZdgMBhQX1+PW265hTwulUphNBov561XBLbhTqKlky8Ee0GyC8hqikNfL7BddtPS0lBSUoKCggLI5XIiuMWqYJ49exbDw8MIhUJXzU3si7Cx4asZ9pq+lmFzvVYL4XAYk5OTOHnyJJxOJwYHBzE3N0f0OXw+H2w2G7kuRkZG4HA44PP5LiqQRrm+uaKE0+npaQAgoQyWV155BX/5y19gNBpxxx134Oc///kCme/5sO5GlkQ1s6Jcn+h0OuzevRsFBQWoqqoCcM4jxuoO/N///R/cbjc9LynXLdFoFOPj49i7d2+ih0K5hvjSCaexWAxf/epX4fF4cPz4cfL4H//4R6Snp8NsNqOlpQVPPvkk1q1bhz179iz6Pr/85S/xq1/96suNnkK5QhQKBW644QZoNBoivMUwDPr6+jA5OYmurq6LSnhTKBQKZSFLCbt8aePjkUcewYcffojjx48Tae/F+PTTT3Hrrbeit7cX2dnZ5z2/mOcjNTX1ywyJQqFQKBRKgln2nA+Wxx57DO+99x6OHj16UcMDAGpqagDggsYHW8ZIoVAoFArl+uCyjA+GYfDDH/4Q77zzDg4fPozMzMxLvqapqQkAYDKZlvwZFAqFQqFQrk6Wch+/LOPj0Ucfxauvvop9+/ZBoVDA4XAAOCdFLZFI0NfXh1dffRU7d+6ETqdDS0sLfvKTn+CWW25BaWnpkj5jNek/UCgUCoVCuTx8Pt8lJTMuK+fjQjXnL774Ir7zne+Q/gtsd8LU1FTceeed+H//7/8tWecjFovBarWisLAQNpttya+jXB5sbg2d45WBzu/KQ+d4ZaHzu/Jca3PMikiazeZLtsq47LDLxUhNTT1P3fRy4XK5sFgsAM41SLoWfpDVDJ3jlYXO78pD53hlofO78lxLc7xUkdDEtzukUCgUCoVyXUGNDwqFQqFQKHFlVRofIpEIzzzzDC3BXUHoHK8sdH5XHjrHKwud35Xnep7jLy0yRqFQKBQKhfJlWJWeDwqFQqFQKNcu1PigUCgUCoUSV6jxQaFQKBQKJa5Q44NCoVAoFEpcocYHhUKhUCiUuLIqjY/nn38eGRkZEIvFqKmpwenTpxM9pKuSX/7yl+BwOAv+CgoKyPOhUAiPPvoodDod5HI57r77bjidzgSOePVz9OhR3HHHHTCbzeBwONi7d++C5xmGwS9+8QuYTCZIJBJs2bIFPT09C45xu9148MEHoVQqoVar8fd///eYmZmJ47dYvVxqfr/zne+cd05v3759wTF0fi/Ms88+i+rqaigUChgMBuzatQtWq3XBMUtZF4aHh/GVr3wFUqkUBoMBP/vZzzA3NxfPr7JqWcocb9y48bzz+Pvf//6CY671OV51xscbb7yBJ554As888wwaGhpQVlaGbdu2weVyJXpoVyVFRUUYGxsjf8ePHyfP/eQnP8G7776LN998E0eOHMHo6CjuuuuuBI529eP3+1FWVobnn39+0eefe+45/Pd//zdeeOEFnDp1CjKZDNu2bUMoFCLHPPjgg2hvb8f+/fvx3nvv4ejRo3j44Yfj9RVWNZeaXwDYvn37gnP6tddeW/A8nd8Lc+TIETz66KM4efIk9u/fj0gkgq1bt8Lv95NjLrUuRKNRfOUrX0E4HMaJEyfw8ssv46WXXsIvfvGLRHylVcdS5hgAvve97y04j5977jny3HUxx8wqY926dcyjjz5K/h2NRhmz2cw8++yzCRzV1ckzzzzDlJWVLfqcx+NhBAIB8+abb5LHOjs7GQBMXV1dnEZ4dQOAeeedd8i/Y7EYYzQamd/85jfkMY/Hw4hEIua1115jGIZhOjo6GADMmTNnyDEffvghw+FwGLvdHrexXw18cX4ZhmF2797NfO1rX7vga+j8Xh4ul4sBwBw5coRhmKWtCx988AHD5XIZh8NBjvnDH/7AKJVKZnZ2Nr5f4Crgi3PMMAyzYcMG5vHHH7/ga66HOV5Vno9wOIz6+nps2bKFPMblcrFlyxbU1dUlcGRXLz09PTCbzcjKysKDDz6I4eFhAEB9fT0ikciCuS4oKEBaWhqd6y/JwMAAHA7HgjlVqVSoqakhc1pXVwe1Wo2qqipyzJYtW8DlcnHq1Km4j/lq5PDhwzAYDMjPz8cjjzyCyclJ8hyd38tjenoaAKDVagEsbV2oq6tDSUkJkpOTyTHbtm2D1+tFe3t7HEd/dfDFOWZ55ZVXkJSUhOLiYjz11FMIBALkuethji+rq+1KMzExgWg0umDCASA5ORldXV0JGtXVS01NDV566SXk5+djbGwMv/rVr7B+/Xq0tbXB4XBAKBRCrVYveE1ycjIcDkdiBnyVw87bYucv+5zD4YDBYFjwPJ/Ph1arpfO+BLZv34677roLmZmZ6Ovrw9NPP40dO3agrq4OPB6Pzu9lEIvF8OMf/xg33XQTiouLAWBJ64LD4Vj0HGefo3zOYnMMAA888ADS09NhNpvR0tKCJ598ElarFXv27AFwfczxqjI+KMvLjh07yP+XlpaipqYG6enp+Otf/wqJRJLAkVEoX46vf/3r5P9LSkpQWlqK7OxsHD58GLfeemsCR3b18eijj6KtrW1BHhhlebnQHM/PQSopKYHJZMKtt96Kvr4+ZGdnx3uYCWFVhV2SkpLA4/HOy6x2Op0wGo0JGtW1g1qtRl5eHnp7e2E0GhEOh+HxeBYcQ+f6y8PO28XOX6PReF7y9NzcHNxuN533L0FWVhaSkpLQ29sLgM7vUnnsscfw3nvv4dChQ0hJSSGPL2VdMBqNi57j7HOUc1xojhejpqYGABacx9f6HK8q40MoFKKyshIHDx4kj8ViMRw8eBC1tbUJHNm1wczMDPr6+mAymVBZWQmBQLBgrq1WK4aHh+lcf0kyMzNhNBoXzKnX68WpU6fInNbW1sLj8aC+vp4c8+mnnyIWi5EFiLJ0RkZGMDk5CZPJBIDO76VgGAaPPfYY3nnnHXz66afIzMxc8PxS1oXa2lq0trYuMPL2798PpVKJwsLC+HyRVcyl5ngxmpqaAGDBeXzNz3GiM16/yOuvv86IRCLmpZdeYjo6OpiHH36YUavVC7J+KUvjpz/9KXP48GFmYGCA+eyzz5gtW7YwSUlJjMvlYhiGYb7//e8zaWlpzKeffsqcPXuWqa2tZWpraxM86tWNz+djGhsbmcbGRgYA85//+Z9MY2MjMzQ0xDAMw/zbv/0bo1armX379jEtLS3M1772NSYzM5MJBoPkPbZv385UVFQwp06dYo4fP87k5uYy3/jGNxL1lVYVF5tfn8/H/OM//iNTV1fHDAwMMAcOHGDWrl3L5ObmMqFQiLwHnd8L88gjjzAqlYo5fPgwMzY2Rv4CgQA55lLrwtzcHFNcXMxs3bqVaWpqYj766CNGr9czTz31VCK+0qrjUnPc29vL/PrXv2bOnj3LDAwMMPv27WOysrKYW265hbzH9TDHq874YBiG+d3vfsekpaUxQqGQWbduHXPy5MlED+mq5P7772dMJhMjFAoZi8XC3H///Uxvby95PhgMMj/4wQ8YjUbDSKVS5s4772TGxsYSOOLVz6FDhxgA5/3t3r2bYZhz5bY///nPmeTkZEYkEjG33norY7VaF7zH5OQk841vfIORy+WMUqlkHnroIcbn8yXg26w+Lja/gUCA2bp1K6PX6xmBQMCkp6cz3/ve987bmND5vTCLzS0A5sUXXyTHLGVdGBwcZHbs2MFIJBImKSmJ+elPf8pEIpE4f5vVyaXmeHh4mLnlllsYrVbLiEQiJicnh/nZz37GTE9PL3ifa32OOQzDMPHzs1AoFAqFQrneWVU5HxQKhUKhUK59qPFBoVAoFAolrlDjg0KhUCgUSlyhxgeFQqFQKJS4Qo0PCoVCoVAocYUaHxQKhUKhUOIKNT4oFAqFQqHEFWp8UCgUCoVCiSvU+KBQKBQKhRJXqPFBoVAoFAolrlDjg0KhUCgUSlz5/z2UE1blnXD8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model predictions:  [3 3 6 7 1 5 2 9 5 6]\n"
     ]
    }
   ],
   "source": [
    "verify_loaded_model(deserialized_net)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3249ced",
   "metadata": {},
   "source": [
    "![Model inputs mnist in 2](https://raw.githubusercontent.com/indhub/web-data/4a9c100aa996df3dff0e7f493029d411c2b526c3/mxnet/tutorials/gluon/save_load_params/mnist_in_2.png) <!--notebook-skip-line-->\n",
    "\n",
    "Model predictions:  [4. 8. 0. 1. 5. 5. 8. 8. 1. 9.] <!--notebook-skip-line-->\n",
    "\n",
    "That's all! We learned how to save and load Gluon networks from files. Parameters of any Gluon network can be persisted into files. For hybrid networks, both the architecture of the network and the parameters can be saved to and loaded from files."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}