versions/1.9.1/api/python/docs/tutorials/packages/ndarray/sparse/train.html - mxnet-site - Git at Google

 <!DOCTYPE html>

 <html xmlns="http://www.w3.org/1999/xhtml">
   <head>
     <meta charset="utf-8" />
     <meta charset="utf-8">
     <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
     <meta http-equiv="x-ua-compatible" content="ie=edge">
     <style>
     .dropdown {
         position: relative;
         display: inline-block;
     }

     .dropdown-content {
         display: none;
         position: absolute;
         background-color: #f9f9f9;
         min-width: 160px;
         box-shadow: 0px 8px 16px 0px rgba(0,0,0,0.2);
         padding: 12px 16px;
         z-index: 1;
         text-align: left;
     }

     .dropdown:hover .dropdown-content {
         display: block;
     }

     .dropdown-option:hover {
         color: #FF4500;
     }

     .dropdown-option-active {
         color: #FF4500;
         font-weight: lighter;
     }

     .dropdown-option {
         color: #000000;
         font-weight: lighter;
     }

     .dropdown-header {
         color: #FFFFFF;
         display: inline-flex;
     }

     .dropdown-caret {
         width: 18px;
     }

     .dropdown-caret-path {
         fill: #FFFFFF;
     }
     </style>

     <title>Train a Linear Regression Model with Sparse Symbols &#8212; Apache MXNet  documentation</title>

     <link rel="stylesheet" href="../../../../_static/basic.css" type="text/css" />
     <link rel="stylesheet" href="../../../../_static/pygments.css" type="text/css" />
     <link rel="stylesheet" type="text/css" href="../../../../_static/mxnet.css" />
     <link rel="stylesheet" href="../../../../_static/material-design-lite-1.3.0/material.blue-deep_orange.min.css" type="text/css" />
     <link rel="stylesheet" href="../../../../_static/sphinx_materialdesign_theme.css" type="text/css" />
     <link rel="stylesheet" href="../../../../_static/fontawesome/all.css" type="text/css" />
     <link rel="stylesheet" href="../../../../_static/fonts.css" type="text/css" />
     <link rel="stylesheet" href="../../../../_static/feedback.css" type="text/css" />
     <script id="documentation_options" data-url_root="../../../../" src="../../../../_static/documentation_options.js"></script>
     <script src="../../../../_static/jquery.js"></script>
     <script src="../../../../_static/underscore.js"></script>
     <script src="../../../../_static/doctools.js"></script>
     <script src="../../../../_static/language_data.js"></script>
     <script src="../../../../_static/matomo_analytics.js"></script>
     <script src="../../../../_static/autodoc.js"></script>
     <script crossorigin="anonymous" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"></script>
     <script async="async" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
     <script type="text/x-mathjax-config">MathJax.Hub.Config({"tex2jax": {"inlineMath": [["$", "$"], ["\\(", "\\)"]], "processEscapes": true, "ignoreClass": "document", "processClass": "math|output_area"}})</script>
     <link rel="shortcut icon" href="../../../../_static/mxnet-icon.png"/>
     <link rel="index" title="Index" href="../../../../genindex.html" />
     <link rel="search" title="Search" href="../../../../search.html" />
     <link rel="next" title="Sparse NDArrays with Gluon" href="train_gluon.html" />
     <link rel="prev" title="RowSparseNDArray - NDArray for Sparse Gradient Updates" href="row_sparse.html" />
   </head>
 <body><header class="site-header" role="banner">
   <div class="wrapper">
       <a class="site-title" rel="author" href="/versions/1.9.1/"><img
             src="../../../../_static/mxnet_logo.png" class="site-header-logo"></a>
     <nav class="site-nav">
       <input type="checkbox" id="nav-trigger" class="nav-trigger"/>
       <label for="nav-trigger">
           <span class="menu-icon">
             <svg viewBox="0 0 18 15" width="18px" height="15px">
               <path d="M18,1.484c0,0.82-0.665,1.484-1.484,1.484H1.484C0.665,2.969,0,2.304,0,1.484l0,0C0,0.665,0.665,0,1.484,0 h15.032C17.335,0,18,0.665,18,1.484L18,1.484z M18,7.516C18,8.335,17.335,9,16.516,9H1.484C0.665,9,0,8.335,0,7.516l0,0 c0-0.82,0.665-1.484,1.484-1.484h15.032C17.335,6.031,18,6.696,18,7.516L18,7.516z M18,13.516C18,14.335,17.335,15,16.516,15H1.484 C0.665,15,0,14.335,0,13.516l0,0c0-0.82,0.665-1.483,1.484-1.483h15.032C17.335,12.031,18,12.695,18,13.516L18,13.516z"/>
             </svg>
           </span>
       </label>

       <div class="trigger">
         <a class="page-link" href="/versions/1.9.1/get_started">Get Started</a>
         <a class="page-link" href="/versions/1.9.1/features">Features</a>
         <a class="page-link" href="/versions/1.9.1/ecosystem">Ecosystem</a>
         <a class="page-link page-current" href="/versions/1.9.1/api">Docs & Tutorials</a>
         <a class="page-link" href="/versions/1.9.1/trusted_by">Trusted By</a>
         <a class="page-link" href="https://github.com/apache/mxnet">GitHub</a>
         <div class="dropdown" style="min-width:100px">
           <span class="dropdown-header">Apache
             <svg class="dropdown-caret" viewBox="0 0 32 32" class="icon icon-caret-bottom" aria-hidden="true"><path class="dropdown-caret-path" d="M24 11.305l-7.997 11.39L8 11.305z"></path></svg>
           </span>
           <div class="dropdown-content" style="min-width:250px">
             <a href="https://www.apache.org/foundation/">Apache Software Foundation</a>
             <a href="https://incubator.apache.org/">Apache Incubator</a>
             <a href="https://www.apache.org/licenses/">License</a>
             <a href="/versions/1.9.1/api/faq/security.html">Security</a>
             <a href="https://privacy.apache.org/policies/privacy-policy-public.html">Privacy</a>
             <a href="https://www.apache.org/events/current-event">Events</a>
             <a href="https://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
             <a href="https://www.apache.org/foundation/thanks.html">Thanks</a>
           </div>
         </div>
         <div class="dropdown">
           <span class="dropdown-header">1.9.1
             <svg class="dropdown-caret" viewBox="0 0 32 32" class="icon icon-caret-bottom" aria-hidden="true"><path class="dropdown-caret-path" d="M24 11.305l-7.997 11.39L8 11.305z"></path></svg>
           </span>
           <div class="dropdown-content">
             <a class="dropdown-option" href="/">master</a><br>
             <a class="dropdown-option-active" href="/versions/1.9.1/">1.9.1</a><br>
             <a class="dropdown-option" href="/versions/1.8.0/">1.8.0</a><br>
             <a class="dropdown-option" href="/versions/1.7.0/">1.7.0</a><br>
             <a class="dropdown-option" href="/versions/1.6.0/">1.6.0</a><br>
             <a class="dropdown-option" href="/versions/1.5.0/">1.5.0</a><br>
             <a class="dropdown-option" href="/versions/1.4.1/">1.4.1</a><br>
             <a class="dropdown-option" href="/versions/1.3.1/">1.3.1</a><br>
             <a class="dropdown-option" href="/versions/1.2.1/">1.2.1</a><br>
             <a class="dropdown-option" href="/versions/1.1.0/">1.1.0</a><br>
             <a class="dropdown-option" href="/versions/1.0.0/">1.0.0</a><br>
             <a class="dropdown-option" href="/versions/0.12.1/">0.12.1</a><br>
             <a class="dropdown-option" href="/versions/0.11.0/">0.11.0</a>
           </div>
         </div>
       </div>
     </nav>
   </div>
 </header>
     <div class="mdl-layout mdl-js-layout mdl-layout--fixed-header mdl-layout--fixed-drawer"><header class="mdl-layout__header mdl-layout__header--waterfall ">
     <div class="mdl-layout__header-row">

         <nav class="mdl-navigation breadcrumb">
             <a class="mdl-navigation__link" href="../../../index.html">Python Tutorials</a><i class="material-icons">navigate_next</i>
             <a class="mdl-navigation__link" href="../../index.html">Packages</a><i class="material-icons">navigate_next</i>
             <a class="mdl-navigation__link" href="../index.html">NDArray</a><i class="material-icons">navigate_next</i>
             <a class="mdl-navigation__link" href="index.html">Tutorials</a><i class="material-icons">navigate_next</i>
             <a class="mdl-navigation__link is-active">Train a Linear Regression Model with Sparse Symbols</a>
         </nav>
         <div class="mdl-layout-spacer"></div>
         <nav class="mdl-navigation">

 <form class="form-inline pull-sm-right" action="../../../../search.html" method="get">
       <div class="mdl-textfield mdl-js-textfield mdl-textfield--expandable mdl-textfield--floating-label mdl-textfield--align-right">
         <label id="quick-search-icon" class="mdl-button mdl-js-button mdl-button--icon"  for="waterfall-exp">
           <i class="material-icons">search</i>
         </label>
         <div class="mdl-textfield__expandable-holder">
           <input class="mdl-textfield__input" type="text" name="q"  id="waterfall-exp" placeholder="Search" />
           <input type="hidden" name="check_keywords" value="yes" />
           <input type="hidden" name="area" value="default" />
         </div>
       </div>
       <div class="mdl-tooltip" data-mdl-for="quick-search-icon">
       Quick search
       </div>
 </form>

 <a id="button-show-source"
     class="mdl-button mdl-js-button mdl-button--icon"
     href="../../../../_sources/tutorials/packages/ndarray/sparse/train.ipynb" rel="nofollow">
   <i class="material-icons">code</i>
 </a>
 <div class="mdl-tooltip" data-mdl-for="button-show-source">
 Show Source
 </div>
         </nav>
     </div>
     <div class="mdl-layout__header-row header-links">
       <div class="mdl-layout-spacer"></div>
       <nav class="mdl-navigation">
       </nav>
     </div>
 </header><header class="mdl-layout__drawer">

       <div class="globaltoc">
         <span class="mdl-layout-title toc">Table Of Contents</span>


             <nav class="mdl-navigation">
                 <ul class="current">
 <li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python Tutorials</a><ul class="current">
 <li class="toctree-l2"><a class="reference internal" href="../../../getting-started/index.html">Getting Started</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../getting-started/crash-course/index.html">Crash Course</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../getting-started/crash-course/1-ndarray.html">Manipulate data with <code class="docutils literal notranslate"><span class="pre">ndarray</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../getting-started/crash-course/2-nn.html">Create a neural network</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../getting-started/crash-course/3-autograd.html">Automatic differentiation with <code class="docutils literal notranslate"><span class="pre">autograd</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../getting-started/crash-course/4-train.html">Train the neural network</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../getting-started/crash-course/5-predict.html">Predict with a pre-trained model</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../getting-started/crash-course/6-use_gpus.html">Use GPUs</a></li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../../getting-started/to-mxnet/index.html">Moving to MXNet from Other Frameworks</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../getting-started/to-mxnet/pytorch.html">PyTorch vs Apache MXNet</a></li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../../getting-started/gluon_from_experiment_to_deployment.html">Gluon: from experiment to deployment</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../getting-started/logistic_regression_explained.html">Logistic regression explained</a></li>
 <li class="toctree-l3"><a class="reference external" href="https://mxnet.apache.org/api/python/docs/tutorials/packages/gluon/image/mnist.html">MNIST</a></li>
 </ul>
 </li>
 <li class="toctree-l2 current"><a class="reference internal" href="../../index.html">Packages</a><ul class="current">
 <li class="toctree-l3"><a class="reference internal" href="../../autograd/index.html">Automatic Differentiation</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../gluon/index.html">Gluon</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../gluon/blocks/index.html">Blocks</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/custom-layer.html">Custom Layers</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/custom_layer_beginners.html">Customer Layers (Beginners)</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/hybridize.html">Hybridize</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/init.html">Initialization</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/naming.html">Parameter and Block Naming</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/nn.html">Layers and Blocks</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/parameters.html">Parameter Management</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/save_load_params.html">Saving and Loading Gluon Models</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/activations/activations.html">Activation Blocks</a></li>
 </ul>
 </li>
 <li class="toctree-l4"><a class="reference internal" href="../../gluon/data/index.html">Data Tutorials</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/data_augmentation.html">Image Augmentation</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/data_augmentation.html#Spatial-Augmentation">Spatial Augmentation</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/data_augmentation.html#Color-Augmentation">Color Augmentation</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/data_augmentation.html#Composed-Augmentations">Composed Augmentations</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/datasets.html">Gluon <code class="docutils literal notranslate"><span class="pre">Dataset</span></code>s and <code class="docutils literal notranslate"><span class="pre">DataLoader</span></code></a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/datasets.html#Using-own-data-with-included-Datasets">Using own data with included <code class="docutils literal notranslate"><span class="pre">Dataset</span></code>s</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/datasets.html#Using-own-data-with-custom-Datasets">Using own data with custom <code class="docutils literal notranslate"><span class="pre">Dataset</span></code>s</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/datasets.html#Appendix:-Upgrading-from-Module-DataIter-to-Gluon-DataLoader">Appendix: Upgrading from Module <code class="docutils literal notranslate"><span class="pre">DataIter</span></code> to Gluon <code class="docutils literal notranslate"><span class="pre">DataLoader</span></code></a></li>
 </ul>
 </li>
 <li class="toctree-l4"><a class="reference internal" href="../../gluon/image/index.html">Image Tutorials</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/image/image-augmentation.html">Image Augmentation</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/image/info_gan.html">Image similarity search with InfoGAN</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/image/mnist.html">Handwritten Digit Recognition</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/image/pretrained_models.html">Using pre-trained models in MXNet</a></li>
 </ul>
 </li>
 <li class="toctree-l4"><a class="reference internal" href="../../gluon/loss/index.html">Losses</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/loss/custom-loss.html">Custom Loss Blocks</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/loss/kl_divergence.html">Kullback-Leibler (KL) Divergence</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/loss/loss.html">Loss functions</a></li>
 </ul>
 </li>
 <li class="toctree-l4"><a class="reference internal" href="../../gluon/text/index.html">Text Tutorials</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/text/gnmt.html">Google Neural Machine Translation</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/text/transformer.html">Machine Translation with Transformer</a></li>
 </ul>
 </li>
 <li class="toctree-l4"><a class="reference internal" href="../../gluon/training/index.html">Training</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/training/fit_api_tutorial.html">MXNet Gluon Fit API</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/training/trainer.html">Trainer</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/training/learning_rates/index.html">Learning Rates</a><ul>
 <li class="toctree-l6"><a class="reference internal" href="../../gluon/training/learning_rates/learning_rate_finder.html">Learning Rate Finder</a></li>
 <li class="toctree-l6"><a class="reference internal" href="../../gluon/training/learning_rates/learning_rate_schedules.html">Learning Rate Schedules</a></li>
 <li class="toctree-l6"><a class="reference internal" href="../../gluon/training/learning_rates/learning_rate_schedules_advanced.html">Advanced Learning Rate Schedules</a></li>
 </ul>
 </li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/training/normalization/index.html">Normalization Blocks</a></li>
 </ul>
 </li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../kvstore/index.html">KVStore</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../kvstore/kvstore.html">Distributed Key-Value Store</a></li>
 </ul>
 </li>
 <li class="toctree-l3 current"><a class="reference internal" href="../index.html">NDArray</a><ul class="current">
 <li class="toctree-l4"><a class="reference internal" href="../01-ndarray-intro.html">An Intro: Manipulate Data the MXNet Way with NDArray</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../02-ndarray-operations.html">NDArray Operations</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../03-ndarray-contexts.html">NDArray Contexts</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../gotchas_numpy_in_mxnet.html">Gotchas using NumPy in Apache MXNet</a></li>
 <li class="toctree-l4 current"><a class="reference internal" href="index.html">Tutorials</a><ul class="current">
 <li class="toctree-l5"><a class="reference internal" href="csr.html">CSRNDArray - NDArray in Compressed Sparse Row Storage Format</a></li>
 <li class="toctree-l5"><a class="reference internal" href="row_sparse.html">RowSparseNDArray - NDArray for Sparse Gradient Updates</a></li>
 <li class="toctree-l5 current"><a class="current reference internal" href="#">Train a Linear Regression Model with Sparse Symbols</a></li>
 <li class="toctree-l5"><a class="reference internal" href="train_gluon.html">Sparse NDArrays with Gluon</a></li>
 </ul>
 </li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../onnx/index.html">ONNX</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../onnx/fine_tuning_gluon.html">Fine-tuning an ONNX model</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../onnx/inference_on_onnx_model.html">Running inference on MXNet/Gluon from an ONNX model</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../onnx/super_resolution.html">Importing an ONNX model into MXNet</a></li>
 <li class="toctree-l4"><a class="reference external" href="https://mxnet.apache.org/api/python/docs/tutorials/deploy/export/onnx.html">Export ONNX Models</a></li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../optimizer/index.html">Optimizers</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../viz/index.html">Visualization</a><ul>
 <li class="toctree-l4"><a class="reference external" href="https://mxnet.apache.org/api/faq/visualize_graph">Visualize networks</a></li>
 </ul>
 </li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="../../../performance/index.html">Performance</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../performance/compression/index.html">Compression</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../performance/compression/int8.html">Deploy with int-8</a></li>
 <li class="toctree-l4"><a class="reference external" href="https://mxnet.apache.org/api/faq/float16">Float16</a></li>
 <li class="toctree-l4"><a class="reference external" href="https://mxnet.apache.org/api/faq/gradient_compression">Gradient Compression</a></li>
 <li class="toctree-l4"><a class="reference external" href="https://gluon-cv.mxnet.io/build/examples_deployment/int8_inference.html">GluonCV with Quantized Models</a></li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../../performance/backend/index.html">Accelerated Backend Tools</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../performance/backend/mkldnn/index.html">Intel MKL-DNN</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../../performance/backend/mkldnn/mkldnn_quantization.html">Quantize with MKL-DNN backend</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../../performance/backend/mkldnn/mkldnn_quantization.html#Improving-accuracy-with-Intel®-Neural-Compressor">Improving accuracy with Intel® Neural Compressor</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../../performance/backend/mkldnn/mkldnn_readme.html">Install MXNet with MKL-DNN</a></li>
 </ul>
 </li>
 <li class="toctree-l4"><a class="reference internal" href="../../../performance/backend/tensorrt/index.html">TensorRT</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../../performance/backend/tensorrt/tensorrt.html">Optimizing Deep Learning Computation Graphs with TensorRT</a></li>
 </ul>
 </li>
 <li class="toctree-l4"><a class="reference internal" href="../../../performance/backend/tvm.html">Use TVM</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../performance/backend/profiler.html">Profiling MXNet Models</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../performance/backend/amp.html">Using AMP: Automatic Mixed Precision</a></li>
 </ul>
 </li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="../../../deploy/index.html">Deployment</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../deploy/export/index.html">Export</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/export/onnx.html">Exporting to ONNX format</a></li>
 <li class="toctree-l4"><a class="reference external" href="https://gluon-cv.mxnet.io/build/examples_deployment/export_network.html">Export Gluon CV Models</a></li>
 <li class="toctree-l4"><a class="reference external" href="https://mxnet.apache.org/api/python/docs/tutorials/packages/gluon/blocks/save_load_params.html">Save / Load Parameters</a></li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../../deploy/inference/index.html">Inference</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/inference/cpp.html">Deploy into C++</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/inference/image_classification_jetson.html">Image Classication using pretrained ResNet-50 model on Jetson module</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/inference/scala.html">Deploy into a Java or Scala Environment</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/inference/wine_detector.html">Real-time Object Detection with MXNet On The Raspberry Pi</a></li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../../deploy/run-on-aws/index.html">Run on AWS</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/run-on-aws/use_ec2.html">Run on an EC2 Instance</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/run-on-aws/use_sagemaker.html">Run on Amazon SageMaker</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/run-on-aws/cloud.html">MXNet on the Cloud</a></li>
 </ul>
 </li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="../../../extend/index.html">Extend</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../extend/custom_layer.html">Custom Layers</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../extend/customop.html">Custom Numpy Operators</a></li>
 <li class="toctree-l3"><a class="reference external" href="https://mxnet.apache.org/api/faq/new_op">New Operator Creation</a></li>
 <li class="toctree-l3"><a class="reference external" href="https://mxnet.apache.org/api/faq/add_op_in_backend">New Operator in MXNet Backend</a></li>
 </ul>
 </li>
 </ul>
 </li>
 <li class="toctree-l1"><a class="reference internal" href="../../../../api/index.html">Python API</a><ul>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/ndarray/index.html">mxnet.ndarray</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/ndarray.html">ndarray</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/contrib/index.html">ndarray.contrib</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/image/index.html">ndarray.image</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/linalg/index.html">ndarray.linalg</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/op/index.html">ndarray.op</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/random/index.html">ndarray.random</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/register/index.html">ndarray.register</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/sparse/index.html">ndarray.sparse</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/utils/index.html">ndarray.utils</a></li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/gluon/index.html">mxnet.gluon</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/block.html">gluon.Block</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/hybrid_block.html">gluon.HybridBlock</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/symbol_block.html">gluon.SymbolBlock</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/constant.html">gluon.Constant</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/parameter.html">gluon.Parameter</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/parameter_dict.html">gluon.ParameterDict</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/trainer.html">gluon.Trainer</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/contrib/index.html">gluon.contrib</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/data/index.html">gluon.data</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../../api/gluon/data/vision/index.html">data.vision</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../../../api/gluon/data/vision/datasets/index.html">vision.datasets</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../../../api/gluon/data/vision/transforms/index.html">vision.transforms</a></li>
 </ul>
 </li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/loss/index.html">gluon.loss</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/model_zoo/index.html">gluon.model_zoo.vision</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/nn/index.html">gluon.nn</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/rnn/index.html">gluon.rnn</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/utils/index.html">gluon.utils</a></li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/autograd/index.html">mxnet.autograd</a></li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/initializer/index.html">mxnet.initializer</a></li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/optimizer/index.html">mxnet.optimizer</a></li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/lr_scheduler/index.html">mxnet.lr_scheduler</a></li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/metric/index.html">mxnet.metric</a></li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/kvstore/index.html">mxnet.kvstore</a></li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/symbol/index.html">mxnet.symbol</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/symbol.html">symbol</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/contrib/index.html">symbol.contrib</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/image/index.html">symbol.image</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/linalg/index.html">symbol.linalg</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/op/index.html">symbol.op</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/random/index.html">symbol.random</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/register/index.html">symbol.register</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/sparse/index.html">symbol.sparse</a></li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/module/index.html">mxnet.module</a></li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/contrib/index.html">mxnet.contrib</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/autograd/index.html">contrib.autograd</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/io/index.html">contrib.io</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/ndarray/index.html">contrib.ndarray</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/onnx/index.html">contrib.onnx</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/quantization/index.html">contrib.quantization</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/symbol/index.html">contrib.symbol</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/tensorboard/index.html">contrib.tensorboard</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/tensorrt/index.html">contrib.tensorrt</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/text/index.html">contrib.text</a></li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/mxnet/index.html">mxnet</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/attribute/index.html">mxnet.attribute</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/base/index.html">mxnet.base</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/callback/index.html">mxnet.callback</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/context/index.html">mxnet.context</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/engine/index.html">mxnet.engine</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/executor/index.html">mxnet.executor</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/executor_manager/index.html">mxnet.executor_manager</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/image/index.html">mxnet.image</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/io/index.html">mxnet.io</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/kvstore_server/index.html">mxnet.kvstore_server</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/libinfo/index.html">mxnet.libinfo</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/log/index.html">mxnet.log</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/model/index.html">mxnet.model</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/monitor/index.html">mxnet.monitor</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/name/index.html">mxnet.name</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/notebook/index.html">mxnet.notebook</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/operator/index.html">mxnet.operator</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/profiler/index.html">mxnet.profiler</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/random/index.html">mxnet.random</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/recordio/index.html">mxnet.recordio</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/registry/index.html">mxnet.registry</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/rtc/index.html">mxnet.rtc</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/runtime/index.html">mxnet.runtime</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/test_utils/index.html">mxnet.test_utils</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/torch/index.html">mxnet.torch</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/util/index.html">mxnet.util</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/visualization/index.html">mxnet.visualization</a></li>
 </ul>
 </li>
 </ul>
 </li>
 </ul>

             </nav>

         </div>

 </header>
         <main class="mdl-layout__content" tabIndex="0">

         <script type="text/javascript" src="../../../../_static/sphinx_materialdesign_theme.js "></script>
         <script type="text/javascript" src="../../../../_static/feedback.js"></script>
     <header class="mdl-layout__drawer">

       <div class="globaltoc">
         <span class="mdl-layout-title toc">Table Of Contents</span>


             <nav class="mdl-navigation">
                 <ul class="current">
 <li class="toctree-l1 current"><a class="reference internal" href="../../../index.html">Python Tutorials</a><ul class="current">
 <li class="toctree-l2"><a class="reference internal" href="../../../getting-started/index.html">Getting Started</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../getting-started/crash-course/index.html">Crash Course</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../getting-started/crash-course/1-ndarray.html">Manipulate data with <code class="docutils literal notranslate"><span class="pre">ndarray</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../getting-started/crash-course/2-nn.html">Create a neural network</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../getting-started/crash-course/3-autograd.html">Automatic differentiation with <code class="docutils literal notranslate"><span class="pre">autograd</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../getting-started/crash-course/4-train.html">Train the neural network</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../getting-started/crash-course/5-predict.html">Predict with a pre-trained model</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../getting-started/crash-course/6-use_gpus.html">Use GPUs</a></li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../../getting-started/to-mxnet/index.html">Moving to MXNet from Other Frameworks</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../getting-started/to-mxnet/pytorch.html">PyTorch vs Apache MXNet</a></li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../../getting-started/gluon_from_experiment_to_deployment.html">Gluon: from experiment to deployment</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../getting-started/logistic_regression_explained.html">Logistic regression explained</a></li>
 <li class="toctree-l3"><a class="reference external" href="https://mxnet.apache.org/api/python/docs/tutorials/packages/gluon/image/mnist.html">MNIST</a></li>
 </ul>
 </li>
 <li class="toctree-l2 current"><a class="reference internal" href="../../index.html">Packages</a><ul class="current">
 <li class="toctree-l3"><a class="reference internal" href="../../autograd/index.html">Automatic Differentiation</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../gluon/index.html">Gluon</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../gluon/blocks/index.html">Blocks</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/custom-layer.html">Custom Layers</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/custom_layer_beginners.html">Customer Layers (Beginners)</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/hybridize.html">Hybridize</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/init.html">Initialization</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/naming.html">Parameter and Block Naming</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/nn.html">Layers and Blocks</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/parameters.html">Parameter Management</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/save_load_params.html">Saving and Loading Gluon Models</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/blocks/activations/activations.html">Activation Blocks</a></li>
 </ul>
 </li>
 <li class="toctree-l4"><a class="reference internal" href="../../gluon/data/index.html">Data Tutorials</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/data_augmentation.html">Image Augmentation</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/data_augmentation.html#Spatial-Augmentation">Spatial Augmentation</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/data_augmentation.html#Color-Augmentation">Color Augmentation</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/data_augmentation.html#Composed-Augmentations">Composed Augmentations</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/datasets.html">Gluon <code class="docutils literal notranslate"><span class="pre">Dataset</span></code>s and <code class="docutils literal notranslate"><span class="pre">DataLoader</span></code></a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/datasets.html#Using-own-data-with-included-Datasets">Using own data with included <code class="docutils literal notranslate"><span class="pre">Dataset</span></code>s</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/datasets.html#Using-own-data-with-custom-Datasets">Using own data with custom <code class="docutils literal notranslate"><span class="pre">Dataset</span></code>s</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/data/datasets.html#Appendix:-Upgrading-from-Module-DataIter-to-Gluon-DataLoader">Appendix: Upgrading from Module <code class="docutils literal notranslate"><span class="pre">DataIter</span></code> to Gluon <code class="docutils literal notranslate"><span class="pre">DataLoader</span></code></a></li>
 </ul>
 </li>
 <li class="toctree-l4"><a class="reference internal" href="../../gluon/image/index.html">Image Tutorials</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/image/image-augmentation.html">Image Augmentation</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/image/info_gan.html">Image similarity search with InfoGAN</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/image/mnist.html">Handwritten Digit Recognition</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/image/pretrained_models.html">Using pre-trained models in MXNet</a></li>
 </ul>
 </li>
 <li class="toctree-l4"><a class="reference internal" href="../../gluon/loss/index.html">Losses</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/loss/custom-loss.html">Custom Loss Blocks</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/loss/kl_divergence.html">Kullback-Leibler (KL) Divergence</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/loss/loss.html">Loss functions</a></li>
 </ul>
 </li>
 <li class="toctree-l4"><a class="reference internal" href="../../gluon/text/index.html">Text Tutorials</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/text/gnmt.html">Google Neural Machine Translation</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/text/transformer.html">Machine Translation with Transformer</a></li>
 </ul>
 </li>
 <li class="toctree-l4"><a class="reference internal" href="../../gluon/training/index.html">Training</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/training/fit_api_tutorial.html">MXNet Gluon Fit API</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/training/trainer.html">Trainer</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/training/learning_rates/index.html">Learning Rates</a><ul>
 <li class="toctree-l6"><a class="reference internal" href="../../gluon/training/learning_rates/learning_rate_finder.html">Learning Rate Finder</a></li>
 <li class="toctree-l6"><a class="reference internal" href="../../gluon/training/learning_rates/learning_rate_schedules.html">Learning Rate Schedules</a></li>
 <li class="toctree-l6"><a class="reference internal" href="../../gluon/training/learning_rates/learning_rate_schedules_advanced.html">Advanced Learning Rate Schedules</a></li>
 </ul>
 </li>
 <li class="toctree-l5"><a class="reference internal" href="../../gluon/training/normalization/index.html">Normalization Blocks</a></li>
 </ul>
 </li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../kvstore/index.html">KVStore</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../kvstore/kvstore.html">Distributed Key-Value Store</a></li>
 </ul>
 </li>
 <li class="toctree-l3 current"><a class="reference internal" href="../index.html">NDArray</a><ul class="current">
 <li class="toctree-l4"><a class="reference internal" href="../01-ndarray-intro.html">An Intro: Manipulate Data the MXNet Way with NDArray</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../02-ndarray-operations.html">NDArray Operations</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../03-ndarray-contexts.html">NDArray Contexts</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../gotchas_numpy_in_mxnet.html">Gotchas using NumPy in Apache MXNet</a></li>
 <li class="toctree-l4 current"><a class="reference internal" href="index.html">Tutorials</a><ul class="current">
 <li class="toctree-l5"><a class="reference internal" href="csr.html">CSRNDArray - NDArray in Compressed Sparse Row Storage Format</a></li>
 <li class="toctree-l5"><a class="reference internal" href="row_sparse.html">RowSparseNDArray - NDArray for Sparse Gradient Updates</a></li>
 <li class="toctree-l5 current"><a class="current reference internal" href="#">Train a Linear Regression Model with Sparse Symbols</a></li>
 <li class="toctree-l5"><a class="reference internal" href="train_gluon.html">Sparse NDArrays with Gluon</a></li>
 </ul>
 </li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../onnx/index.html">ONNX</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../onnx/fine_tuning_gluon.html">Fine-tuning an ONNX model</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../onnx/inference_on_onnx_model.html">Running inference on MXNet/Gluon from an ONNX model</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../onnx/super_resolution.html">Importing an ONNX model into MXNet</a></li>
 <li class="toctree-l4"><a class="reference external" href="https://mxnet.apache.org/api/python/docs/tutorials/deploy/export/onnx.html">Export ONNX Models</a></li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../optimizer/index.html">Optimizers</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../viz/index.html">Visualization</a><ul>
 <li class="toctree-l4"><a class="reference external" href="https://mxnet.apache.org/api/faq/visualize_graph">Visualize networks</a></li>
 </ul>
 </li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="../../../performance/index.html">Performance</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../performance/compression/index.html">Compression</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../performance/compression/int8.html">Deploy with int-8</a></li>
 <li class="toctree-l4"><a class="reference external" href="https://mxnet.apache.org/api/faq/float16">Float16</a></li>
 <li class="toctree-l4"><a class="reference external" href="https://mxnet.apache.org/api/faq/gradient_compression">Gradient Compression</a></li>
 <li class="toctree-l4"><a class="reference external" href="https://gluon-cv.mxnet.io/build/examples_deployment/int8_inference.html">GluonCV with Quantized Models</a></li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../../performance/backend/index.html">Accelerated Backend Tools</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../performance/backend/mkldnn/index.html">Intel MKL-DNN</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../../performance/backend/mkldnn/mkldnn_quantization.html">Quantize with MKL-DNN backend</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../../performance/backend/mkldnn/mkldnn_quantization.html#Improving-accuracy-with-Intel®-Neural-Compressor">Improving accuracy with Intel® Neural Compressor</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../../performance/backend/mkldnn/mkldnn_readme.html">Install MXNet with MKL-DNN</a></li>
 </ul>
 </li>
 <li class="toctree-l4"><a class="reference internal" href="../../../performance/backend/tensorrt/index.html">TensorRT</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../../performance/backend/tensorrt/tensorrt.html">Optimizing Deep Learning Computation Graphs with TensorRT</a></li>
 </ul>
 </li>
 <li class="toctree-l4"><a class="reference internal" href="../../../performance/backend/tvm.html">Use TVM</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../performance/backend/profiler.html">Profiling MXNet Models</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../performance/backend/amp.html">Using AMP: Automatic Mixed Precision</a></li>
 </ul>
 </li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="../../../deploy/index.html">Deployment</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../deploy/export/index.html">Export</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/export/onnx.html">Exporting to ONNX format</a></li>
 <li class="toctree-l4"><a class="reference external" href="https://gluon-cv.mxnet.io/build/examples_deployment/export_network.html">Export Gluon CV Models</a></li>
 <li class="toctree-l4"><a class="reference external" href="https://mxnet.apache.org/api/python/docs/tutorials/packages/gluon/blocks/save_load_params.html">Save / Load Parameters</a></li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../../deploy/inference/index.html">Inference</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/inference/cpp.html">Deploy into C++</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/inference/image_classification_jetson.html">Image Classication using pretrained ResNet-50 model on Jetson module</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/inference/scala.html">Deploy into a Java or Scala Environment</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/inference/wine_detector.html">Real-time Object Detection with MXNet On The Raspberry Pi</a></li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../../deploy/run-on-aws/index.html">Run on AWS</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/run-on-aws/use_ec2.html">Run on an EC2 Instance</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/run-on-aws/use_sagemaker.html">Run on Amazon SageMaker</a></li>
 <li class="toctree-l4"><a class="reference internal" href="../../../deploy/run-on-aws/cloud.html">MXNet on the Cloud</a></li>
 </ul>
 </li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="../../../extend/index.html">Extend</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../extend/custom_layer.html">Custom Layers</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../extend/customop.html">Custom Numpy Operators</a></li>
 <li class="toctree-l3"><a class="reference external" href="https://mxnet.apache.org/api/faq/new_op">New Operator Creation</a></li>
 <li class="toctree-l3"><a class="reference external" href="https://mxnet.apache.org/api/faq/add_op_in_backend">New Operator in MXNet Backend</a></li>
 </ul>
 </li>
 </ul>
 </li>
 <li class="toctree-l1"><a class="reference internal" href="../../../../api/index.html">Python API</a><ul>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/ndarray/index.html">mxnet.ndarray</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/ndarray.html">ndarray</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/contrib/index.html">ndarray.contrib</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/image/index.html">ndarray.image</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/linalg/index.html">ndarray.linalg</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/op/index.html">ndarray.op</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/random/index.html">ndarray.random</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/register/index.html">ndarray.register</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/sparse/index.html">ndarray.sparse</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/ndarray/utils/index.html">ndarray.utils</a></li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/gluon/index.html">mxnet.gluon</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/block.html">gluon.Block</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/hybrid_block.html">gluon.HybridBlock</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/symbol_block.html">gluon.SymbolBlock</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/constant.html">gluon.Constant</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/parameter.html">gluon.Parameter</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/parameter_dict.html">gluon.ParameterDict</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/trainer.html">gluon.Trainer</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/contrib/index.html">gluon.contrib</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/data/index.html">gluon.data</a><ul>
 <li class="toctree-l4"><a class="reference internal" href="../../../../api/gluon/data/vision/index.html">data.vision</a><ul>
 <li class="toctree-l5"><a class="reference internal" href="../../../../api/gluon/data/vision/datasets/index.html">vision.datasets</a></li>
 <li class="toctree-l5"><a class="reference internal" href="../../../../api/gluon/data/vision/transforms/index.html">vision.transforms</a></li>
 </ul>
 </li>
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/loss/index.html">gluon.loss</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/model_zoo/index.html">gluon.model_zoo.vision</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/nn/index.html">gluon.nn</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/rnn/index.html">gluon.rnn</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/gluon/utils/index.html">gluon.utils</a></li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/autograd/index.html">mxnet.autograd</a></li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/initializer/index.html">mxnet.initializer</a></li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/optimizer/index.html">mxnet.optimizer</a></li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/lr_scheduler/index.html">mxnet.lr_scheduler</a></li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/metric/index.html">mxnet.metric</a></li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/kvstore/index.html">mxnet.kvstore</a></li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/symbol/index.html">mxnet.symbol</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/symbol.html">symbol</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/contrib/index.html">symbol.contrib</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/image/index.html">symbol.image</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/linalg/index.html">symbol.linalg</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/op/index.html">symbol.op</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/random/index.html">symbol.random</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/register/index.html">symbol.register</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/symbol/sparse/index.html">symbol.sparse</a></li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/module/index.html">mxnet.module</a></li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/contrib/index.html">mxnet.contrib</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/autograd/index.html">contrib.autograd</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/io/index.html">contrib.io</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/ndarray/index.html">contrib.ndarray</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/onnx/index.html">contrib.onnx</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/quantization/index.html">contrib.quantization</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/symbol/index.html">contrib.symbol</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/tensorboard/index.html">contrib.tensorboard</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/tensorrt/index.html">contrib.tensorrt</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/contrib/text/index.html">contrib.text</a></li>
 </ul>
 </li>
 <li class="toctree-l2"><a class="reference internal" href="../../../../api/mxnet/index.html">mxnet</a><ul>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/attribute/index.html">mxnet.attribute</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/base/index.html">mxnet.base</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/callback/index.html">mxnet.callback</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/context/index.html">mxnet.context</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/engine/index.html">mxnet.engine</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/executor/index.html">mxnet.executor</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/executor_manager/index.html">mxnet.executor_manager</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/image/index.html">mxnet.image</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/io/index.html">mxnet.io</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/kvstore_server/index.html">mxnet.kvstore_server</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/libinfo/index.html">mxnet.libinfo</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/log/index.html">mxnet.log</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/model/index.html">mxnet.model</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/monitor/index.html">mxnet.monitor</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/name/index.html">mxnet.name</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/notebook/index.html">mxnet.notebook</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/operator/index.html">mxnet.operator</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/profiler/index.html">mxnet.profiler</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/random/index.html">mxnet.random</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/recordio/index.html">mxnet.recordio</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/registry/index.html">mxnet.registry</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/rtc/index.html">mxnet.rtc</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/runtime/index.html">mxnet.runtime</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/test_utils/index.html">mxnet.test_utils</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/torch/index.html">mxnet.torch</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/util/index.html">mxnet.util</a></li>
 <li class="toctree-l3"><a class="reference internal" href="../../../../api/mxnet/visualization/index.html">mxnet.visualization</a></li>
 </ul>
 </li>
 </ul>
 </li>
 </ul>

             </nav>

         </div>

 </header>

     <div class="document">
         <div class="page-content" role="main">

   <!--- Licensed to the Apache Software Foundation (ASF) under one --><!--- or more contributor license agreements.  See the NOTICE file --><!--- distributed with this work for additional information --><!--- regarding copyright ownership.  The ASF licenses this file --><!--- to you under the Apache License, Version 2.0 (the --><!--- "License"); you may not use this file except in compliance --><!--- with the License.  You may obtain a copy of the License at --><!---   http://www.apache.org/licenses/LICENSE-2.0 --><!--- Unless required by applicable law or agreed to in writing, --><!--- software distributed under the License is distributed on an --><!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY --><!--- KIND, either express or implied.  See the License for the --><!--- specific language governing permissions and limitations --><!--- under the License. --><div class="section" id="Train-a-Linear-Regression-Model-with-Sparse-Symbols">
 <h1>Train a Linear Regression Model with Sparse Symbols<a class="headerlink" href="#Train-a-Linear-Regression-Model-with-Sparse-Symbols" title="Permalink to this headline">¶</a></h1>
 <p>In previous tutorials, we introduced <code class="docutils literal notranslate"><span class="pre">CSRNDArray</span></code> and <code class="docutils literal notranslate"><span class="pre">RowSparseNDArray</span></code>, the basic data structures for manipulating sparse data. MXNet also provides <code class="docutils literal notranslate"><span class="pre">Sparse</span> <span class="pre">Symbol</span></code> API, which enables symbolic expressions that handle sparse arrays. In this tutorial, we first focus on how to compose a symbolic graph with sparse operators, then train a linear regression model using sparse symbols with the Module API.</p>
 <div class="section" id="Prerequisites">
 <h2>Prerequisites<a class="headerlink" href="#Prerequisites" title="Permalink to this headline">¶</a></h2>
 <p>To complete this tutorial, we need:</p>
 <ul class="simple">
 <li><p>MXNet. See the instructions for your operating system in <a class="reference external" href="/get_started">Setup and Installation</a>.</p></li>
 <li><p><a class="reference external" href="https://jupyter.org/index.html">Jupyter Notebook</a> and <a class="reference external" href="https://3.python-requests.org/">Python Requests</a> packages.</p></li>
 </ul>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">pip</span> <span class="n">install</span> <span class="n">jupyter</span> <span class="n">requests</span>
 </pre></div>
 </div>
 <ul class="simple">
 <li><p>Basic knowledge of Symbol in MXNet. See the detailed tutorial for Symbol in <a class="reference external" href="https://mxnet.apache.org/tutorials/basic/symbol.html">Symbol - Neural Network Graphs and Auto-differentiation</a>.</p></li>
 <li><p>Basic knowledge of CSRNDArray in MXNet. See the detailed tutorial for CSRNDArray in <a class="reference external" href="/api/python/docs/tutorials/packages/ndarray/sparse/csr.html">CSRNDArray - NDArray in Compressed Sparse Row Storage Format</a>.</p></li>
 <li><p>Basic knowledge of RowSparseNDArray in MXNet. See the detailed tutorial for RowSparseNDArray in <a class="reference external" href="/api/python/docs/tutorials/packages/ndarray/sparse/row_sparse.html">RowSparseNDArray - NDArray for Sparse Gradient Updates</a>.</p></li>
 </ul>
 </div>
 <div class="section" id="Variables">
 <h2>Variables<a class="headerlink" href="#Variables" title="Permalink to this headline">¶</a></h2>
 <p>Variables are placeholder for arrays. We can use them to hold sparse arrays too.</p>
 <div class="section" id="Variable-Storage-Types">
 <h3>Variable Storage Types<a class="headerlink" href="#Variable-Storage-Types" title="Permalink to this headline">¶</a></h3>
 <p>The <code class="docutils literal notranslate"><span class="pre">stype</span></code> attribute of a variable is used to indicate the storage type of the array. By default, the <code class="docutils literal notranslate"><span class="pre">stype</span></code> of a variable is “default” which indicates the default dense storage format. We can specify the <code class="docutils literal notranslate"><span class="pre">stype</span></code> of a variable as “csr” or “row_sparse” to hold sparse arrays.</p>
 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">mxnet</span> <span class="k">as</span> <span class="nn">mx</span>
 <span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
 <span class="kn">import</span> <span class="nn">random</span>

 <span class="c1"># set the seeds for repeatability</span>
 <span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span>
 <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span>
 <span class="n">mx</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span>

 <span class="c1"># Create a variable to hold an NDArray</span>
 <span class="n">a</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">Variable</span><span class="p">(</span><span class="s1">&#39;a&#39;</span><span class="p">)</span>
 <span class="c1"># Create a variable to hold a CSRNDArray</span>
 <span class="n">b</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">Variable</span><span class="p">(</span><span class="s1">&#39;b&#39;</span><span class="p">,</span> <span class="n">stype</span><span class="o">=</span><span class="s1">&#39;csr&#39;</span><span class="p">)</span>
 <span class="c1"># Create a variable to hold a RowSparseNDArray</span>
 <span class="n">c</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">Variable</span><span class="p">(</span><span class="s1">&#39;c&#39;</span><span class="p">,</span> <span class="n">stype</span><span class="o">=</span><span class="s1">&#39;row_sparse&#39;</span><span class="p">)</span>
 <span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">)</span>
 </pre></div>
 </div>
 <p><code class="docutils literal notranslate"><span class="pre">(&lt;Symbol</span> <span class="pre">a&gt;,</span> <span class="pre">&lt;Symbol</span> <span class="pre">b&gt;,</span> <span class="pre">&lt;Symbol</span> <span class="pre">c&gt;)</span></code></p>
 </div>
 <div class="section" id="Bind-with-Sparse-Arrays">
 <h3>Bind with Sparse Arrays<a class="headerlink" href="#Bind-with-Sparse-Arrays" title="Permalink to this headline">¶</a></h3>
 <p>The sparse symbols constructed above declare storage types of the arrays to hold. To evaluate them, we need to feed the free variables with sparse data.</p>
 <p>You can instantiate an executor from a sparse symbol by using the <code class="docutils literal notranslate"><span class="pre">simple_bind</span></code> method, which allocate zeros to all free variables according to their storage types. The executor provides <code class="docutils literal notranslate"><span class="pre">forward</span></code> method for evaluation and an attribute <code class="docutils literal notranslate"><span class="pre">outputs</span></code> to get all the results. Later, we will show the use of the <code class="docutils literal notranslate"><span class="pre">backward</span></code> method and other methods computing the gradients and updating parameters. A simple example first:</p>
 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">shape</span> <span class="o">=</span> <span class="p">(</span><span class="mi">2</span><span class="p">,</span><span class="mi">2</span><span class="p">)</span>
 <span class="c1"># Instantiate an executor from sparse symbols</span>
 <span class="n">b_exec</span> <span class="o">=</span> <span class="n">b</span><span class="o">.</span><span class="n">simple_bind</span><span class="p">(</span><span class="n">ctx</span><span class="o">=</span><span class="n">mx</span><span class="o">.</span><span class="n">cpu</span><span class="p">(),</span> <span class="n">b</span><span class="o">=</span><span class="n">shape</span><span class="p">)</span>
 <span class="n">c_exec</span> <span class="o">=</span> <span class="n">c</span><span class="o">.</span><span class="n">simple_bind</span><span class="p">(</span><span class="n">ctx</span><span class="o">=</span><span class="n">mx</span><span class="o">.</span><span class="n">cpu</span><span class="p">(),</span> <span class="n">c</span><span class="o">=</span><span class="n">shape</span><span class="p">)</span>
 <span class="n">b_exec</span><span class="o">.</span><span class="n">forward</span><span class="p">()</span>
 <span class="n">c_exec</span><span class="o">.</span><span class="n">forward</span><span class="p">()</span>
 <span class="c1"># Sparse arrays of zeros are bound to b and c</span>
 <span class="nb">print</span><span class="p">(</span><span class="n">b_exec</span><span class="o">.</span><span class="n">outputs</span><span class="p">,</span> <span class="n">c_exec</span><span class="o">.</span><span class="n">outputs</span><span class="p">)</span>
 </pre></div>
 </div>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">([</span>
 <span class="o">&lt;</span><span class="n">CSRNDArray</span> <span class="mi">2</span><span class="n">x2</span> <span class="nd">@cpu</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">&gt;</span><span class="p">],</span> <span class="p">[</span>
 <span class="o">&lt;</span><span class="n">RowSparseNDArray</span> <span class="mi">2</span><span class="n">x2</span> <span class="nd">@cpu</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">&gt;</span><span class="p">])</span>
 </pre></div>
 </div>
 <p>You can update the array held by the variable by accessing executor’s <code class="docutils literal notranslate"><span class="pre">arg_dict</span></code> and assigning new values.</p>
 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">b_exec</span><span class="o">.</span><span class="n">arg_dict</span><span class="p">[</span><span class="s1">&#39;b&#39;</span><span class="p">][:]</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">nd</span><span class="o">.</span><span class="n">ones</span><span class="p">(</span><span class="n">shape</span><span class="p">)</span><span class="o">.</span><span class="n">tostype</span><span class="p">(</span><span class="s1">&#39;csr&#39;</span><span class="p">)</span>
 <span class="n">b_exec</span><span class="o">.</span><span class="n">forward</span><span class="p">()</span>
 <span class="c1"># The array `b` holds are updated to be ones</span>
 <span class="n">eval_b</span> <span class="o">=</span> <span class="n">b_exec</span><span class="o">.</span><span class="n">outputs</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
 <span class="p">{</span><span class="s1">&#39;eval_b&#39;</span><span class="p">:</span> <span class="n">eval_b</span><span class="p">,</span> <span class="s1">&#39;eval_b.asnumpy()&#39;</span><span class="p">:</span> <span class="n">eval_b</span><span class="o">.</span><span class="n">asnumpy</span><span class="p">()}</span>
 </pre></div>
 </div>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">{</span><span class="s1">&#39;eval_b&#39;</span><span class="p">:</span>
  <span class="o">&lt;</span><span class="n">CSRNDArray</span> <span class="mi">2</span><span class="n">x2</span> <span class="nd">@cpu</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">&gt;</span><span class="p">,</span> <span class="s1">&#39;eval_b.asnumpy()&#39;</span><span class="p">:</span> <span class="n">array</span><span class="p">([[</span> <span class="mf">1.</span><span class="p">,</span>  <span class="mf">1.</span><span class="p">],</span>
         <span class="p">[</span> <span class="mf">1.</span><span class="p">,</span>  <span class="mf">1.</span><span class="p">]],</span> <span class="n">dtype</span><span class="o">=</span><span class="n">float32</span><span class="p">)}</span>
 </pre></div>
 </div>
 </div>
 </div>
 <div class="section" id="Symbol-Composition-and-Storage-Type-Inference">
 <h2>Symbol Composition and Storage Type Inference<a class="headerlink" href="#Symbol-Composition-and-Storage-Type-Inference" title="Permalink to this headline">¶</a></h2>
 <div class="section" id="Basic-Symbol-Composition">
 <h3>Basic Symbol Composition<a class="headerlink" href="#Basic-Symbol-Composition" title="Permalink to this headline">¶</a></h3>
 <p>The following example builds a simple element-wise addition expression with different storage types. The sparse symbols are available in the <code class="docutils literal notranslate"><span class="pre">mx.sym.sparse</span></code> package.</p>
 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># Element-wise addition of variables with &quot;default&quot; stype</span>
 <span class="n">d</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">elemwise_add</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">a</span><span class="p">)</span>
 <span class="c1"># Element-wise addition of variables with &quot;csr&quot; stype</span>
 <span class="n">e</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">sparse</span><span class="o">.</span><span class="n">negative</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>
 <span class="c1"># Element-wise addition of variables with &quot;row_sparse&quot; stype</span>
 <span class="n">f</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">sparse</span><span class="o">.</span><span class="n">elemwise_add</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="n">c</span><span class="p">)</span>
 <span class="p">{</span><span class="s1">&#39;d&#39;</span><span class="p">:</span><span class="n">d</span><span class="p">,</span> <span class="s1">&#39;e&#39;</span><span class="p">:</span><span class="n">e</span><span class="p">,</span> <span class="s1">&#39;f&#39;</span><span class="p">:</span><span class="n">f</span><span class="p">}</span>
 </pre></div>
 </div>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">{</span><span class="s1">&#39;d&#39;</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">Symbol</span> <span class="n">elemwise_add0</span><span class="o">&gt;</span><span class="p">,</span>
  <span class="s1">&#39;e&#39;</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">Symbol</span> <span class="n">negative0</span><span class="o">&gt;</span><span class="p">,</span>
  <span class="s1">&#39;f&#39;</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">Symbol</span> <span class="n">elemwise_add1</span><span class="o">&gt;</span><span class="p">}</span>
 </pre></div>
 </div>
 </div>
 <div class="section" id="Storage-Type-Inference">
 <h3>Storage Type Inference<a class="headerlink" href="#Storage-Type-Inference" title="Permalink to this headline">¶</a></h3>
 <p>What will be the output storage types of sparse symbols? In MXNet, for any sparse symbol, the result storage types are inferred based on storage types of inputs. You can read the <a class="reference external" href="/api/python/docs/api/symbol/sparse/index.html">Sparse Symbol API</a> documentation to find what output storage types are. In the example below we will try out the storage types introduced in the Row Sparse and Compressed Sparse Row tutorials: <code class="docutils literal notranslate"><span class="pre">default</span></code> (dense), <code class="docutils literal notranslate"><span class="pre">csr</span></code>, and <code class="docutils literal notranslate"><span class="pre">row_sparse</span></code>.</p>
 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">add_exec</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">Group</span><span class="p">([</span><span class="n">d</span><span class="p">,</span> <span class="n">e</span><span class="p">,</span> <span class="n">f</span><span class="p">])</span><span class="o">.</span><span class="n">simple_bind</span><span class="p">(</span><span class="n">ctx</span><span class="o">=</span><span class="n">mx</span><span class="o">.</span><span class="n">cpu</span><span class="p">(),</span> <span class="n">a</span><span class="o">=</span><span class="n">shape</span><span class="p">,</span> <span class="n">b</span><span class="o">=</span><span class="n">shape</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="n">shape</span><span class="p">)</span>
 <span class="n">add_exec</span><span class="o">.</span><span class="n">forward</span><span class="p">()</span>
 <span class="n">dense_add</span> <span class="o">=</span> <span class="n">add_exec</span><span class="o">.</span><span class="n">outputs</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
 <span class="c1"># The output storage type of elemwise_add(csr, csr) will be inferred as &quot;csr&quot;</span>
 <span class="n">csr_add</span> <span class="o">=</span> <span class="n">add_exec</span><span class="o">.</span><span class="n">outputs</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
 <span class="c1"># The output storage type of elemwise_add(row_sparse, row_sparse) will be inferred as &quot;row_sparse&quot;</span>
 <span class="n">rsp_add</span> <span class="o">=</span> <span class="n">add_exec</span><span class="o">.</span><span class="n">outputs</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
 <span class="p">{</span><span class="s1">&#39;dense_add.stype&#39;</span><span class="p">:</span> <span class="n">dense_add</span><span class="o">.</span><span class="n">stype</span><span class="p">,</span> <span class="s1">&#39;csr_add.stype&#39;</span><span class="p">:</span><span class="n">csr_add</span><span class="o">.</span><span class="n">stype</span><span class="p">,</span> <span class="s1">&#39;rsp_add.stype&#39;</span><span class="p">:</span> <span class="n">rsp_add</span><span class="o">.</span><span class="n">stype</span><span class="p">}</span>
 </pre></div>
 </div>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">{</span><span class="s1">&#39;csr_add.stype&#39;</span><span class="p">:</span> <span class="s1">&#39;csr&#39;</span><span class="p">,</span>
  <span class="s1">&#39;dense_add.stype&#39;</span><span class="p">:</span> <span class="s1">&#39;default&#39;</span><span class="p">,</span>
  <span class="s1">&#39;rsp_add.stype&#39;</span><span class="p">:</span> <span class="s1">&#39;row_sparse&#39;</span><span class="p">}</span>
 </pre></div>
 </div>
 </div>
 <div class="section" id="Storage-Type-Fallback">
 <h3>Storage Type Fallback<a class="headerlink" href="#Storage-Type-Fallback" title="Permalink to this headline">¶</a></h3>
 <p>For operators that don’t specialize in certain sparse arrays, you can still use them with sparse inputs with some performance penalty. In MXNet, dense operators require all inputs and outputs to be in the dense format. If sparse inputs are provided, MXNet will convert sparse inputs into dense ones temporarily so that the dense operator can be used. If sparse outputs are provided, MXNet will convert the dense outputs generated by the dense operator into the provided sparse format. Warning
 messages will be printed when such a storage fallback event happens.</p>
 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># `log` operator doesn&#39;t support sparse inputs at all, but we can fallback on the dense implementation</span>
 <span class="n">csr_log</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
 <span class="c1"># `elemwise_add` operator doesn&#39;t support adding csr with row_sparse, but we can fallback on the dense implementation</span>
 <span class="n">csr_rsp_add</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">elemwise_add</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">)</span>
 <span class="n">fallback_exec</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">Group</span><span class="p">([</span><span class="n">csr_rsp_add</span><span class="p">,</span> <span class="n">csr_log</span><span class="p">])</span><span class="o">.</span><span class="n">simple_bind</span><span class="p">(</span><span class="n">ctx</span><span class="o">=</span><span class="n">mx</span><span class="o">.</span><span class="n">cpu</span><span class="p">(),</span> <span class="n">a</span><span class="o">=</span><span class="n">shape</span><span class="p">,</span> <span class="n">b</span><span class="o">=</span><span class="n">shape</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="n">shape</span><span class="p">)</span>
 <span class="n">fallback_exec</span><span class="o">.</span><span class="n">forward</span><span class="p">()</span>
 <span class="n">fallback_add</span> <span class="o">=</span> <span class="n">fallback_exec</span><span class="o">.</span><span class="n">outputs</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
 <span class="n">fallback_log</span> <span class="o">=</span> <span class="n">fallback_exec</span><span class="o">.</span><span class="n">outputs</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
 <span class="p">{</span><span class="s1">&#39;fallback_add&#39;</span><span class="p">:</span> <span class="n">fallback_add</span><span class="p">,</span> <span class="s1">&#39;fallback_log&#39;</span><span class="p">:</span> <span class="n">fallback_log</span><span class="p">}</span>
 </pre></div>
 </div>
 <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">{</span><span class="s1">&#39;fallback_add&#39;</span><span class="p">:</span>
  <span class="p">[[</span> <span class="mf">0.</span>  <span class="mf">0.</span><span class="p">]</span>
   <span class="p">[</span> <span class="mf">0.</span>  <span class="mf">0.</span><span class="p">]]</span>
  <span class="o">&lt;</span><span class="n">NDArray</span> <span class="mi">2</span><span class="n">x2</span> <span class="nd">@cpu</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">&gt;</span><span class="p">,</span> <span class="s1">&#39;fallback_log&#39;</span><span class="p">:</span>
  <span class="p">[[</span><span class="o">-</span><span class="n">inf</span> <span class="o">-</span><span class="n">inf</span><span class="p">]</span>
   <span class="p">[</span><span class="o">-</span><span class="n">inf</span> <span class="o">-</span><span class="n">inf</span><span class="p">]]</span>
  <span class="o">&lt;</span><span class="n">NDArray</span> <span class="mi">2</span><span class="n">x2</span> <span class="nd">@cpu</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span><span class="o">&gt;</span><span class="p">}</span>
 </pre></div>
 </div>
 </div>
 <div class="section" id="Inspecting-Storage-Types-of-the-Symbol-Graph">
 <h3>Inspecting Storage Types of the Symbol Graph<a class="headerlink" href="#Inspecting-Storage-Types-of-the-Symbol-Graph" title="Permalink to this headline">¶</a></h3>
 <p>When the environment variable <code class="docutils literal notranslate"><span class="pre">MXNET_INFER_STORAGE_TYPE_VERBOSE_LOGGING</span></code> is set to <code class="docutils literal notranslate"><span class="pre">1</span></code>, MXNet will log the storage type information of operators’ inputs and outputs in the computation graph. For example, we can inspect the storage types of a linear classification network with sparse operators. Uncomment the line below and inspect your console.:</p>
 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># Set logging level for executor</span>
 <span class="kn">import</span> <span class="nn">mxnet</span> <span class="k">as</span> <span class="nn">mx</span>
 <span class="kn">import</span> <span class="nn">os</span>
 <span class="c1">#os.environ[&#39;MXNET_INFER_STORAGE_TYPE_VERBOSE_LOGGING&#39;] = &quot;1&quot;</span>
 <span class="c1"># Data in csr format</span>
 <span class="n">data</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">var</span><span class="p">(</span><span class="s1">&#39;data&#39;</span><span class="p">,</span> <span class="n">stype</span><span class="o">=</span><span class="s1">&#39;csr&#39;</span><span class="p">,</span> <span class="n">shape</span><span class="o">=</span><span class="p">(</span><span class="mi">32</span><span class="p">,</span> <span class="mi">10000</span><span class="p">))</span>
 <span class="c1"># Weight in row_sparse format</span>
 <span class="n">weight</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">var</span><span class="p">(</span><span class="s1">&#39;weight&#39;</span><span class="p">,</span> <span class="n">stype</span><span class="o">=</span><span class="s1">&#39;row_sparse&#39;</span><span class="p">,</span> <span class="n">shape</span><span class="o">=</span><span class="p">(</span><span class="mi">10000</span><span class="p">,</span> <span class="mi">2</span><span class="p">))</span>
 <span class="n">bias</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">symbol</span><span class="o">.</span><span class="n">Variable</span><span class="p">(</span><span class="s2">&quot;bias&quot;</span><span class="p">,</span> <span class="n">shape</span><span class="o">=</span><span class="p">(</span><span class="mi">2</span><span class="p">,))</span>
 <span class="n">dot</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">symbol</span><span class="o">.</span><span class="n">sparse</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">data</span><span class="p">,</span> <span class="n">weight</span><span class="p">)</span>
 <span class="n">pred</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">symbol</span><span class="o">.</span><span class="n">broadcast_add</span><span class="p">(</span><span class="n">dot</span><span class="p">,</span> <span class="n">bias</span><span class="p">)</span>
 <span class="n">y</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">symbol</span><span class="o">.</span><span class="n">Variable</span><span class="p">(</span><span class="s2">&quot;label&quot;</span><span class="p">)</span>
 <span class="n">output</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">symbol</span><span class="o">.</span><span class="n">SoftmaxOutput</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="n">pred</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="n">y</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s2">&quot;output&quot;</span><span class="p">)</span>
 <span class="n">executor</span> <span class="o">=</span> <span class="n">output</span><span class="o">.</span><span class="n">simple_bind</span><span class="p">(</span><span class="n">ctx</span><span class="o">=</span><span class="n">mx</span><span class="o">.</span><span class="n">cpu</span><span class="p">())</span>
 </pre></div>
 </div>
 </div>
 </div>
 <div class="section" id="Training-with-Module-APIs">
 <h2>Training with Module APIs<a class="headerlink" href="#Training-with-Module-APIs" title="Permalink to this headline">¶</a></h2>
 <p>In the following section we’ll walk through how one can implement <strong>linear regression</strong> using sparse symbols and sparse optimizers.</p>
 <p>The function you will explore is: <em>y = x1 + 2x2 + … 100x100</em>, where <em>(x1,x2, …, x100)</em> are input features and <em>y</em> is the corresponding label.</p>
 <div class="section" id="Preparing-the-Data">
 <h3>Preparing the Data<a class="headerlink" href="#Preparing-the-Data" title="Permalink to this headline">¶</a></h3>
 <p>In MXNet, both <a class="reference external" href="/api/python/docs/api/mxnet/io/index.html#mxnet.io.LibSVMIter">mx.io.LibSVMIter</a> and <a class="reference external" href="/api/python/docs/api/mxnet/io/index.html#mxnet.io.NDArrayIter">mx.io.NDArrayIter</a> support loading sparse data in CSR format. In this example, we’ll use the <code class="docutils literal notranslate"><span class="pre">NDArrayIter</span></code>.</p>
 <p>You may see some warnings from SciPy. You don’t need to worry about those for this example.</p>
 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># Random training data</span>
 <span class="n">feature_dimension</span> <span class="o">=</span> <span class="mi">100</span>
 <span class="n">train_data</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">test_utils</span><span class="o">.</span><span class="n">rand_ndarray</span><span class="p">((</span><span class="mi">1000</span><span class="p">,</span> <span class="n">feature_dimension</span><span class="p">),</span> <span class="s1">&#39;csr&#39;</span><span class="p">,</span> <span class="mf">0.01</span><span class="p">)</span>
 <span class="n">target_weight</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">nd</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">feature_dimension</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">reshape</span><span class="p">((</span><span class="n">feature_dimension</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span>
 <span class="n">train_label</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">nd</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">train_data</span><span class="p">,</span> <span class="n">target_weight</span><span class="p">)</span>
 <span class="n">batch_size</span> <span class="o">=</span> <span class="mi">1</span>
 <span class="n">train_iter</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">io</span><span class="o">.</span><span class="n">NDArrayIter</span><span class="p">(</span><span class="n">train_data</span><span class="p">,</span> <span class="n">train_label</span><span class="p">,</span> <span class="n">batch_size</span><span class="p">,</span> <span class="n">last_batch_handle</span><span class="o">=</span><span class="s1">&#39;discard&#39;</span><span class="p">,</span> <span class="n">label_name</span><span class="o">=</span><span class="s1">&#39;label&#39;</span><span class="p">)</span>
 </pre></div>
 </div>
 </div>
 <div class="section" id="Defining-the-Model">
 <h3>Defining the Model<a class="headerlink" href="#Defining-the-Model" title="Permalink to this headline">¶</a></h3>
 <p>Below is an example of a linear regression model specifying the storage type of the variables.</p>
 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">initializer</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">initializer</span><span class="o">.</span><span class="n">Normal</span><span class="p">(</span><span class="n">sigma</span><span class="o">=</span><span class="mf">0.01</span><span class="p">)</span>
 <span class="n">X</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">Variable</span><span class="p">(</span><span class="s1">&#39;data&#39;</span><span class="p">,</span> <span class="n">stype</span><span class="o">=</span><span class="s1">&#39;csr&#39;</span><span class="p">)</span>
 <span class="n">Y</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">symbol</span><span class="o">.</span><span class="n">Variable</span><span class="p">(</span><span class="s1">&#39;label&#39;</span><span class="p">)</span>
 <span class="n">weight</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">symbol</span><span class="o">.</span><span class="n">Variable</span><span class="p">(</span><span class="s1">&#39;weight&#39;</span><span class="p">,</span> <span class="n">stype</span><span class="o">=</span><span class="s1">&#39;row_sparse&#39;</span><span class="p">,</span> <span class="n">shape</span><span class="o">=</span><span class="p">(</span><span class="n">feature_dimension</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="n">init</span><span class="o">=</span><span class="n">initializer</span><span class="p">)</span>
 <span class="n">bias</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">symbol</span><span class="o">.</span><span class="n">Variable</span><span class="p">(</span><span class="s1">&#39;bias&#39;</span><span class="p">,</span> <span class="n">shape</span><span class="o">=</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="p">))</span>
 <span class="n">pred</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">broadcast_add</span><span class="p">(</span><span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">sparse</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">weight</span><span class="p">),</span> <span class="n">bias</span><span class="p">)</span>
 <span class="n">lro</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">sym</span><span class="o">.</span><span class="n">LinearRegressionOutput</span><span class="p">(</span><span class="n">data</span><span class="o">=</span><span class="n">pred</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="n">Y</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s2">&quot;lro&quot;</span><span class="p">)</span>
 </pre></div>
 </div>
 <p>The above network uses the following symbols:</p>
 <ol class="arabic simple">
 <li><p><code class="docutils literal notranslate"><span class="pre">Variable</span> <span class="pre">X</span></code>: The placeholder for sparse data inputs. The <code class="docutils literal notranslate"><span class="pre">csr</span></code> stype indicates that the array to hold is in CSR format.</p></li>
 <li><p><code class="docutils literal notranslate"><span class="pre">Variable</span> <span class="pre">Y</span></code>: The placeholder for dense labels.</p></li>
 <li><p><code class="docutils literal notranslate"><span class="pre">Variable</span> <span class="pre">weight</span></code>: The placeholder for the weight to learn. The <code class="docutils literal notranslate"><span class="pre">stype</span></code> of weight is specified as <code class="docutils literal notranslate"><span class="pre">row_sparse</span></code> so that it is initialized as RowSparseNDArray, and the optimizer will perform sparse update rules on it. The <code class="docutils literal notranslate"><span class="pre">init</span></code> attribute specifies what initializer to use for this variable.</p></li>
 <li><p><code class="docutils literal notranslate"><span class="pre">Variable</span> <span class="pre">bias</span></code>: The placeholder for the bias to learn.</p></li>
 <li><p><code class="docutils literal notranslate"><span class="pre">sparse.dot</span></code>: The dot product operation of <code class="docutils literal notranslate"><span class="pre">X</span></code> and <code class="docutils literal notranslate"><span class="pre">weight</span></code>. The sparse implementation will be invoked to handle <code class="docutils literal notranslate"><span class="pre">csr</span></code> and <code class="docutils literal notranslate"><span class="pre">row_sparse</span></code> inputs.</p></li>
 <li><p><code class="docutils literal notranslate"><span class="pre">broadcast_add</span></code>: The broadcasting add operation to apply <code class="docutils literal notranslate"><span class="pre">bias</span></code>.</p></li>
 <li><p><code class="docutils literal notranslate"><span class="pre">LinearRegressionOutput</span></code>: The output layer which computes <em>l2</em> loss against its input and the labels provided to it.</p></li>
 </ol>
 </div>
 <div class="section" id="Training-the-model">
 <h3>Training the model<a class="headerlink" href="#Training-the-model" title="Permalink to this headline">¶</a></h3>
 <p>Once we have defined the model structure, the next step is to create a module and initialize the parameters and optimizer.</p>
 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># Create module</span>
 <span class="n">mod</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">mod</span><span class="o">.</span><span class="n">Module</span><span class="p">(</span><span class="n">symbol</span><span class="o">=</span><span class="n">lro</span><span class="p">,</span> <span class="n">data_names</span><span class="o">=</span><span class="p">[</span><span class="s1">&#39;data&#39;</span><span class="p">],</span> <span class="n">label_names</span><span class="o">=</span><span class="p">[</span><span class="s1">&#39;label&#39;</span><span class="p">])</span>
 <span class="c1"># Allocate memory by giving the input data and label shapes</span>
 <span class="n">mod</span><span class="o">.</span><span class="n">bind</span><span class="p">(</span><span class="n">data_shapes</span><span class="o">=</span><span class="n">train_iter</span><span class="o">.</span><span class="n">provide_data</span><span class="p">,</span> <span class="n">label_shapes</span><span class="o">=</span><span class="n">train_iter</span><span class="o">.</span><span class="n">provide_label</span><span class="p">)</span>
 <span class="c1"># Initialize parameters by random numbers</span>
 <span class="n">mod</span><span class="o">.</span><span class="n">init_params</span><span class="p">(</span><span class="n">initializer</span><span class="o">=</span><span class="n">initializer</span><span class="p">)</span>
 <span class="c1"># Use SGD as the optimizer, which performs sparse update on &quot;row_sparse&quot; weight</span>
 <span class="n">sgd</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">optimizer</span><span class="o">.</span><span class="n">SGD</span><span class="p">(</span><span class="n">learning_rate</span><span class="o">=</span><span class="mf">0.05</span><span class="p">,</span> <span class="n">rescale_grad</span><span class="o">=</span><span class="mf">1.0</span><span class="o">/</span><span class="n">batch_size</span><span class="p">,</span> <span class="n">momentum</span><span class="o">=</span><span class="mf">0.9</span><span class="p">)</span>
 <span class="n">mod</span><span class="o">.</span><span class="n">init_optimizer</span><span class="p">(</span><span class="n">optimizer</span><span class="o">=</span><span class="n">sgd</span><span class="p">)</span>
 </pre></div>
 </div>
 <p>Finally, we train the parameters of the model to fit the training data by using the <code class="docutils literal notranslate"><span class="pre">forward</span></code>, <code class="docutils literal notranslate"><span class="pre">backward</span></code>, and <code class="docutils literal notranslate"><span class="pre">update</span></code> methods in Module.</p>
 <div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># Use mean square error as the metric</span>
 <span class="n">metric</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">metric</span><span class="o">.</span><span class="n">create</span><span class="p">(</span><span class="s1">&#39;MSE&#39;</span><span class="p">)</span>
 <span class="c1"># Train 10 epochs</span>
 <span class="k">for</span> <span class="n">epoch</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
     <span class="n">train_iter</span><span class="o">.</span><span class="n">reset</span><span class="p">()</span>
     <span class="n">metric</span><span class="o">.</span><span class="n">reset</span><span class="p">()</span>
     <span class="k">for</span> <span class="n">batch</span> <span class="ow">in</span> <span class="n">train_iter</span><span class="p">:</span>
         <span class="n">mod</span><span class="o">.</span><span class="n">forward</span><span class="p">(</span><span class="n">batch</span><span class="p">,</span> <span class="n">is_train</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>       <span class="c1"># compute predictions</span>
         <span class="n">mod</span><span class="o">.</span><span class="n">update_metric</span><span class="p">(</span><span class="n">metric</span><span class="p">,</span> <span class="n">batch</span><span class="o">.</span><span class="n">label</span><span class="p">)</span>  <span class="c1"># accumulate prediction accuracy</span>
         <span class="n">mod</span><span class="o">.</span><span class="n">backward</span><span class="p">()</span>                          <span class="c1"># compute gradients</span>
         <span class="n">mod</span><span class="o">.</span><span class="n">update</span><span class="p">()</span>                            <span class="c1"># update parameters</span>
     <span class="nb">print</span><span class="p">(</span><span class="s1">&#39;Epoch </span><span class="si">%d</span><span class="s1">, Metric = </span><span class="si">%s</span><span class="s1">&#39;</span> <span class="o">%</span> <span class="p">(</span><span class="n">epoch</span><span class="p">,</span> <span class="n">metric</span><span class="o">.</span><span class="n">get</span><span class="p">()))</span>
 <span class="k">assert</span> <span class="n">metric</span><span class="o">.</span><span class="n">get</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span> <span class="o">&lt;</span> <span class="mi">1</span><span class="p">,</span> <span class="s2">&quot;Achieved MSE (</span><span class="si">%f</span><span class="s2">) is larger than expected (1.0)&quot;</span> <span class="o">%</span> <span class="n">metric</span><span class="o">.</span><span class="n">get</span><span class="p">()[</span><span class="mi">1</span><span class="p">]</span>
 </pre></div>
 </div>
 <p><code class="docutils literal notranslate"><span class="pre">Epoch</span> <span class="pre">9,</span> <span class="pre">Metric</span> <span class="pre">=</span> <span class="pre">('mse',</span> <span class="pre">0.35979430613957991)</span></code></p>
 </div>
 <div class="section" id="Training-the-model-with-multiple-machines-or-multiple-devices">
 <h3>Training the model with multiple machines or multiple devices<a class="headerlink" href="#Training-the-model-with-multiple-machines-or-multiple-devices" title="Permalink to this headline">¶</a></h3>
 <p>Distributed training with <code class="docutils literal notranslate"><span class="pre">row_sparse</span></code> weights and gradients are supported in MXNet, which significantly reduces communication cost for large models. To train a sparse model with multiple machines, you need to call <code class="docutils literal notranslate"><span class="pre">prepare</span></code> before <code class="docutils literal notranslate"><span class="pre">forward</span></code>, or <code class="docutils literal notranslate"><span class="pre">save_checkpoint</span></code>. Please refer to the example in <a class="reference external" href="https://github.com/apache/mxnet/tree/master/example/sparse/linear_classification">mxnet/example/sparse/linear_classification</a> for more details.</p>
 <!-- INSERT SOURCE DOWNLOAD BUTTONS --></div>
 </div>
 </div>


         <hr class="feedback-hr-top" />
 <div class="feedback-container">
     <div class="feedback-question">Did this page help you?</div>
     <div class="feedback-answer-container">
         <div class="feedback-answer yes-link" data-response="yes">Yes</div>
         <div class="feedback-answer no-link" data-response="no">No</div>
     </div>
     <div class="feedback-thank-you">Thanks for your feedback!</div>
 </div>
 <hr class="feedback-hr-bottom" />
         </div>
         <div class="side-doc-outline">
             <div class="side-doc-outline--content">
 <div class="localtoc">
     <p class="caption">
       <span class="caption-text">Table Of Contents</span>
     </p>
     <ul>
 <li><a class="reference internal" href="#">Train a Linear Regression Model with Sparse Symbols</a><ul>
 <li><a class="reference internal" href="#Prerequisites">Prerequisites</a></li>
 <li><a class="reference internal" href="#Variables">Variables</a><ul>
 <li><a class="reference internal" href="#Variable-Storage-Types">Variable Storage Types</a></li>
 <li><a class="reference internal" href="#Bind-with-Sparse-Arrays">Bind with Sparse Arrays</a></li>
 </ul>
 </li>
 <li><a class="reference internal" href="#Symbol-Composition-and-Storage-Type-Inference">Symbol Composition and Storage Type Inference</a><ul>
 <li><a class="reference internal" href="#Basic-Symbol-Composition">Basic Symbol Composition</a></li>
 <li><a class="reference internal" href="#Storage-Type-Inference">Storage Type Inference</a></li>
 <li><a class="reference internal" href="#Storage-Type-Fallback">Storage Type Fallback</a></li>
 <li><a class="reference internal" href="#Inspecting-Storage-Types-of-the-Symbol-Graph">Inspecting Storage Types of the Symbol Graph</a></li>
 </ul>
 </li>
 <li><a class="reference internal" href="#Training-with-Module-APIs">Training with Module APIs</a><ul>
 <li><a class="reference internal" href="#Preparing-the-Data">Preparing the Data</a></li>
 <li><a class="reference internal" href="#Defining-the-Model">Defining the Model</a></li>
 <li><a class="reference internal" href="#Training-the-model">Training the model</a></li>
 <li><a class="reference internal" href="#Training-the-model-with-multiple-machines-or-multiple-devices">Training the model with multiple machines or multiple devices</a></li>
 </ul>
 </li>
 </ul>
 </li>
 </ul>

 </div>
             </div>
         </div>

       <div class="clearer"></div>
     </div><div class="pagenation">
      <a id="button-prev" href="row_sparse.html" class="mdl-button mdl-js-button mdl-js-ripple-effect mdl-button--colored" role="botton" accesskey="P">
          <i class="pagenation-arrow-L fas fa-arrow-left fa-lg"></i>
          <div class="pagenation-text">
             <span class="pagenation-direction">Previous</span>
             <div>RowSparseNDArray - NDArray for Sparse Gradient Updates</div>
          </div>
      </a>
      <a id="button-next" href="train_gluon.html" class="mdl-button mdl-js-button mdl-js-ripple-effect mdl-button--colored" role="botton" accesskey="N">
          <i class="pagenation-arrow-R fas fa-arrow-right fa-lg"></i>
         <div class="pagenation-text">
             <span class="pagenation-direction">Next</span>
             <div>Sparse NDArrays with Gluon</div>
         </div>
      </a>
   </div>
             <footer class="site-footer h-card">
     <div class="wrapper">
         <div class="row">
             <div class="col-4">
                 <h4 class="footer-category-title">Resources</h4>
                 <ul class="contact-list">
                     <li><a class="u-email" href="mailto:dev@mxnet.apache.org">Dev list</a></li>
                     <li><a class="u-email" href="mailto:user@mxnet.apache.org">User mailing list</a></li>
                     <li><a href="https://cwiki.apache.org/confluence/display/MXNET/Apache+MXNet+Home">Developer Wiki</a></li>
                     <li><a href="https://issues.apache.org/jira/projects/MXNET/issues">Jira Tracker</a></li>
                     <li><a href="https://github.com/apache/mxnet/labels/Roadmap">Github Roadmap</a></li>
                     <li><a href="https://medium.com/apache-mxnet">Blog</a></li>
                     <li><a href="https://discuss.mxnet.io">Forum</a></li>
                     <li><a href="/community/contribute">Contribute</a></li>

                 </ul>
             </div>

             <div class="col-4"><ul class="social-media-list"><li><a href="https://github.com/apache/mxnet"><svg class="svg-icon"><use xlink:href="../../../../_static/minima-social-icons.svg#github"></use></svg> <span class="username">apache/mxnet</span></a></li><li><a href="https://www.twitter.com/apachemxnet"><svg class="svg-icon"><use xlink:href="../../../../_static/minima-social-icons.svg#twitter"></use></svg> <span class="username">apachemxnet</span></a></li><li><a href="https://youtube.com/apachemxnet"><svg class="svg-icon"><use xlink:href="../../../../_static/minima-social-icons.svg#youtube"></use></svg> <span class="username">apachemxnet</span></a></li></ul>
 </div>

             <div class="col-4 footer-text">
                 <p>A flexible and efficient library for deep learning.</p>
             </div>
         </div>
     </div>
 </footer>

 <footer class="site-footer2">
     <div class="wrapper">
         <div class="row">
             <div class="col-3">
                 <img src="../../../../_static/apache_incubator_logo.png" class="footer-logo col-2">
             </div>
             <div class="footer-bottom-warning col-9">
                 <p>Apache MXNet is an effort undergoing incubation at <a href="http://www.apache.org/">The Apache Software Foundation</a> (ASF), <span style="font-weight:bold">sponsored by the <i>Apache Incubator</i></span>. Incubation is required
                     of all newly accepted projects until a further review indicates that the infrastructure,
                     communications, and decision making process have stabilized in a manner consistent with other
                     successful ASF projects. While incubation status is not necessarily a reflection of the completeness
                     or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
                 </p><p>"Copyright © 2017-2018, The Apache Software Foundation Apache MXNet, MXNet, Apache, the Apache
                     feather, and the Apache MXNet project logo are either registered trademarks or trademarks of the
                     Apache Software Foundation."</p>
             </div>
         </div>
     </div>
 </footer>

   </body>
 </html>