blob: b7b57b3045bff5a49e11fb9984645a5bacde81cf [file]
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<meta content="IE=edge" http-equiv="X-UA-Compatible"/>
<meta content="width=device-width, initial-scale=1" name="viewport"/>
<meta content="Autograd Package" property="og:title">
<meta content="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/og-logo.png" property="og:image">
<meta content="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/og-logo.png" property="og:image:secure_url">
<meta content="Autograd Package" property="og:description"/>
<title>Autograd Package — mxnet documentation</title>
<link crossorigin="anonymous" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css" integrity="sha384-1q8mTJOASx8j1Au+a5WDVnPi2lkFfwwEAa8hDDdjZlpLegxhjVME1fgjWPGmkzs7" rel="stylesheet"/>
<link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.5.0/css/font-awesome.min.css" rel="stylesheet"/>
<link href="../../../_static/basic.css" rel="stylesheet" type="text/css">
<link href="../../../_static/pygments.css" rel="stylesheet" type="text/css">
<link href="../../../_static/mxnet.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../../../',
VERSION: '',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script src="https://code.jquery.com/jquery-1.11.1.min.js" type="text/javascript"></script>
<script src="../../../_static/underscore.js" type="text/javascript"></script>
<script src="../../../_static/searchtools_custom.js" type="text/javascript"></script>
<script src="../../../_static/doctools.js" type="text/javascript"></script>
<script src="../../../_static/selectlang.js" type="text/javascript"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
<script type="text/javascript"> jQuery(function() { Search.loadIndex("/versions/1.5.0/searchindex.js"); Search.init();}); </script>
<!-- -->
<!-- <script type="text/javascript" src="../../../_static/jquery.js"></script> -->
<!-- -->
<!-- <script type="text/javascript" src="../../../_static/underscore.js"></script> -->
<!-- -->
<!-- <script type="text/javascript" src="../../../_static/doctools.js"></script> -->
<!-- -->
<!-- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script> -->
<!-- -->
<link href="../../../genindex.html" rel="index" title="Index">
<link href="../../../search.html" rel="search" title="Search"/>
<link href="../index.html" rel="up" title="MXNet - Python API"/>
<link href="../callback/callback.html" rel="next" title="Callback API"/>
<link href="../index.html" rel="prev" title="MXNet - Python API"/>
<link href="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/mxnet-icon.png" rel="icon" type="image/png"/>
</link></link></link></meta></meta></meta></head>
<body background="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/mxnet-background-compressed.jpeg" role="document">
<div class="content-block"><div class="navbar navbar-fixed-top">
<div class="container" id="navContainer">
<div class="innder" id="header-inner">
<h1 id="logo-wrap">
<a href="../../../" id="logo"><img src="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/mxnet_logo.png"/></a>
</h1>
<nav class="nav-bar" id="main-nav">
<a class="main-nav-link" href="/versions/1.5.0/install/index.html">Install</a>
<span id="dropdown-menu-position-anchor">
<a aria-expanded="true" aria-haspopup="true" class="main-nav-link dropdown-toggle" data-toggle="dropdown" href="#" role="button">Gluon <span class="caret"></span></a>
<ul class="dropdown-menu navbar-menu" id="package-dropdown-menu">
<li><a class="main-nav-link" href="/versions/1.5.0/tutorials/gluon/gluon.html">About</a></li>
<li><a class="main-nav-link" href="https://www.d2l.ai/">Dive into Deep Learning</a></li>
<li><a class="main-nav-link" href="https://gluon-cv.mxnet.io">GluonCV Toolkit</a></li>
<li><a class="main-nav-link" href="https://gluon-nlp.mxnet.io/">GluonNLP Toolkit</a></li>
</ul>
</span>
<span id="dropdown-menu-position-anchor">
<a aria-expanded="true" aria-haspopup="true" class="main-nav-link dropdown-toggle" data-toggle="dropdown" href="#" role="button">API <span class="caret"></span></a>
<ul class="dropdown-menu navbar-menu" id="package-dropdown-menu">
<li><a class="main-nav-link" href="/versions/1.5.0/api/python/index.html">Python</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/api/c++/index.html">C++</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/api/clojure/index.html">Clojure</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/api/java/index.html">Java</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/api/julia/index.html">Julia</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/api/perl/index.html">Perl</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/api/r/index.html">R</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/api/scala/index.html">Scala</a></li>
</ul>
</span>
<span id="dropdown-menu-position-anchor-docs">
<a aria-expanded="true" aria-haspopup="true" class="main-nav-link dropdown-toggle" data-toggle="dropdown" href="#" role="button">Docs <span class="caret"></span></a>
<ul class="dropdown-menu navbar-menu" id="package-dropdown-menu-docs">
<li><a class="main-nav-link" href="/versions/1.5.0/faq/index.html">FAQ</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/tutorials/index.html">Tutorials</a>
<li><a class="main-nav-link" href="https://github.com/apache/incubator-mxnet/tree/1.5.0/example">Examples</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/architecture/index.html">Architecture</a></li>
<li><a class="main-nav-link" href="https://cwiki.apache.org/confluence/display/MXNET/Apache+MXNet+Home">Developer Wiki</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/model_zoo/index.html">Model Zoo</a></li>
<li><a class="main-nav-link" href="https://github.com/onnx/onnx-mxnet">ONNX</a></li>
</li></ul>
</span>
<span id="dropdown-menu-position-anchor-community">
<a aria-expanded="true" aria-haspopup="true" class="main-nav-link dropdown-toggle" data-toggle="dropdown" href="#" role="button">Community <span class="caret"></span></a>
<ul class="dropdown-menu navbar-menu" id="package-dropdown-menu-community">
<li><a class="main-nav-link" href="http://discuss.mxnet.io">Forum</a></li>
<li><a class="main-nav-link" href="https://github.com/apache/incubator-mxnet/tree/1.5.0">Github</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/community/contribute.html">Contribute</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/community/ecosystem.html">Ecosystem</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/community/powered_by.html">Powered By</a></li>
</ul>
</span>
<span id="dropdown-menu-position-anchor-version" style="position: relative"><a href="#" class="main-nav-link dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="true">1.5.0<span class="caret"></span></a><ul id="package-dropdown-menu" class="dropdown-menu"><li><a href="/">master</a></li><li><a href="/versions/1.7.0/">1.7.0</a></li><li><a href=/versions/1.6.0/>1.6.0</a></li><li><a href=/versions/1.5.0/>1.5.0</a></li><li><a href=/versions/1.4.1/>1.4.1</a></li><li><a href=/versions/1.3.1/>1.3.1</a></li><li><a href=/versions/1.2.1/>1.2.1</a></li><li><a href=/versions/1.1.0/>1.1.0</a></li><li><a href=/versions/1.0.0/>1.0.0</a></li><li><a href=/versions/0.12.1/>0.12.1</a></li><li><a href=/versions/0.11.0/>0.11.0</a></li></ul></span></nav>
<script> function getRootPath(){ return "../../../" } </script>
<div class="burgerIcon dropdown">
<a class="dropdown-toggle" data-toggle="dropdown" href="#" role="button">☰</a>
<ul class="dropdown-menu" id="burgerMenu">
<li><a href="/versions/1.5.0/install/index.html">Install</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/tutorials/index.html">Tutorials</a></li>
<li class="dropdown-submenu dropdown">
<a aria-expanded="true" aria-haspopup="true" class="dropdown-toggle burger-link" data-toggle="dropdown" href="#" tabindex="-1">Gluon</a>
<ul class="dropdown-menu navbar-menu" id="package-dropdown-menu">
<li><a class="main-nav-link" href="/versions/1.5.0/tutorials/gluon/gluon.html">About</a></li>
<li><a class="main-nav-link" href="http://gluon.mxnet.io">The Straight Dope (Tutorials)</a></li>
<li><a class="main-nav-link" href="https://gluon-cv.mxnet.io">GluonCV Toolkit</a></li>
<li><a class="main-nav-link" href="https://gluon-nlp.mxnet.io/">GluonNLP Toolkit</a></li>
</ul>
</li>
<li class="dropdown-submenu">
<a aria-expanded="true" aria-haspopup="true" class="dropdown-toggle burger-link" data-toggle="dropdown" href="#" tabindex="-1">API</a>
<ul class="dropdown-menu">
<li><a class="main-nav-link" href="/versions/1.5.0/api/python/index.html">Python</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/api/c++/index.html">C++</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/api/clojure/index.html">Clojure</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/api/java/index.html">Java</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/api/julia/index.html">Julia</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/api/perl/index.html">Perl</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/api/r/index.html">R</a></li>
<li><a class="main-nav-link" href="/versions/1.5.0/api/scala/index.html">Scala</a></li>
</ul>
</li>
<li class="dropdown-submenu">
<a aria-expanded="true" aria-haspopup="true" class="dropdown-toggle burger-link" data-toggle="dropdown" href="#" tabindex="-1">Docs</a>
<ul class="dropdown-menu">
<li><a href="/versions/1.5.0/faq/index.html" tabindex="-1">FAQ</a></li>
<li><a href="/versions/1.5.0/tutorials/index.html" tabindex="-1">Tutorials</a></li>
<li><a href="https://github.com/apache/incubator-mxnet/tree/1.5.0/example" tabindex="-1">Examples</a></li>
<li><a href="/versions/1.5.0/architecture/index.html" tabindex="-1">Architecture</a></li>
<li><a href="https://cwiki.apache.org/confluence/display/MXNET/Apache+MXNet+Home" tabindex="-1">Developer Wiki</a></li>
<li><a href="/versions/1.5.0/model_zoo/index.html" tabindex="-1">Gluon Model Zoo</a></li>
<li><a href="https://github.com/onnx/onnx-mxnet" tabindex="-1">ONNX</a></li>
</ul>
</li>
<li class="dropdown-submenu dropdown">
<a aria-haspopup="true" class="dropdown-toggle burger-link" data-toggle="dropdown" href="#" role="button" tabindex="-1">Community</a>
<ul class="dropdown-menu">
<li><a href="http://discuss.mxnet.io" tabindex="-1">Forum</a></li>
<li><a href="https://github.com/apache/incubator-mxnet/tree/1.5.0" tabindex="-1">Github</a></li>
<li><a href="/versions/1.5.0/community/contribute.html" tabindex="-1">Contribute</a></li>
<li><a href="/versions/1.5.0/community/ecosystem.html" tabindex="-1">Ecosystem</a></li>
<li><a href="/versions/1.5.0/community/powered_by.html" tabindex="-1">Powered By</a></li>
</ul>
</li>
<li id="dropdown-menu-position-anchor-version-mobile" class="dropdown-submenu" style="position: relative"><a href="#" tabindex="-1">1.5.0</a><ul class="dropdown-menu"><li><a tabindex="-1" href=/>master</a></li><li><a tabindex="-1" href=/versions/1.6.0/>1.6.0</a></li><li><a tabindex="-1" href=/versions/1.5.0/>1.5.0</a></li><li><a tabindex="-1" href=/versions/1.4.1/>1.4.1</a></li><li><a tabindex="-1" href=/versions/1.3.1/>1.3.1</a></li><li><a tabindex="-1" href=/versions/1.2.1/>1.2.1</a></li><li><a tabindex="-1" href=/versions/1.1.0/>1.1.0</a></li><li><a tabindex="-1" href=/versions/1.0.0/>1.0.0</a></li><li><a tabindex="-1" href=/versions/0.12.1/>0.12.1</a></li><li><a tabindex="-1" href=/versions/0.11.0/>0.11.0</a></li></ul></li></ul>
</div>
<div class="plusIcon dropdown">
<a class="dropdown-toggle" data-toggle="dropdown" href="#" role="button"><span aria-hidden="true" class="glyphicon glyphicon-plus"></span></a>
<ul class="dropdown-menu dropdown-menu-right" id="plusMenu"></ul>
</div>
<div id="search-input-wrap">
<form action="../../../search.html" autocomplete="off" class="" method="get" role="search">
<div class="form-group inner-addon left-addon">
<i class="glyphicon glyphicon-search"></i>
<input class="form-control" name="q" placeholder="Search" type="text"/>
</div>
<input name="check_keywords" type="hidden" value="yes">
<input name="area" type="hidden" value="default"/>
</input></form>
<div id="search-preview"></div>
</div>
<div id="searchIcon">
<span aria-hidden="true" class="glyphicon glyphicon-search"></span>
</div>
<!-- <div id="lang-select-wrap"> -->
<!-- <label id="lang-select-label"> -->
<!-- <\!-- <i class="fa fa-globe"></i> -\-> -->
<!-- <span></span> -->
<!-- </label> -->
<!-- <select id="lang-select"> -->
<!-- <option value="en">Eng</option> -->
<!-- <option value="zh">中文</option> -->
<!-- </select> -->
<!-- </div> -->
<!-- <a id="mobile-nav-toggle">
<span class="mobile-nav-toggle-bar"></span>
<span class="mobile-nav-toggle-bar"></span>
<span class="mobile-nav-toggle-bar"></span>
</a> -->
</div>
</div>
</div>
<script type="text/javascript">
$('body').css('background', 'white');
</script>
<div class="container">
<div class="row">
<div aria-label="main navigation" class="sphinxsidebar leftsidebar" role="navigation">
<div class="sphinxsidebarwrapper">
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../index.html">MXNet APIs</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../architecture/index.html">MXNet Architecture</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../community/index.html">MXNet Community</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../faq/index.html">MXNet FAQ</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../gluon/index.html">About Gluon</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../install/index.html">Installing MXNet</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../install/index.html#nvidia-jetson-tx-family">Nvidia Jetson TX family</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../install/index.html#source-download">Source Download</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../model_zoo/index.html">MXNet Model Zoo</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../../tutorials/index.html">Tutorials</a></li>
</ul>
</div>
</div>
<div class="content">
<div class="page-tracker"></div>
<!--- Licensed to the Apache Software Foundation (ASF) under one -->
<!--- or more contributor license agreements. See the NOTICE file -->
<!--- distributed with this work for additional information -->
<!--- regarding copyright ownership. The ASF licenses this file -->
<!--- to you under the Apache License, Version 2.0 (the -->
<!--- "License"); you may not use this file except in compliance -->
<!--- with the License. You may obtain a copy of the License at --><!--- http://www.apache.org/licenses/LICENSE-2.0 --><!--- Unless required by applicable law or agreed to in writing, -->
<!--- software distributed under the License is distributed on an -->
<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
<!--- KIND, either express or implied. See the License for the -->
<!--- specific language governing permissions and limitations -->
<!--- under the License. --><div class="section" id="autograd-package">
<span id="autograd-package"></span><h1>Autograd Package<a class="headerlink" href="#autograd-package" title="Permalink to this headline"></a></h1>
<div class="section" id="overview">
<span id="overview"></span><h2>Overview<a class="headerlink" href="#overview" title="Permalink to this headline"></a></h2>
<p>The <code class="docutils literal"><span class="pre">autograd</span></code> package enables automatic
differentiation of NDArray operations.
In machine learning applications,
<code class="docutils literal"><span class="pre">autograd</span></code> is often used to calculate the gradients
of loss functions with respect to parameters.</p>
</div>
<div class="section" id="record-vs-pause">
<span id="record-vs-pause"></span><h2>Record vs Pause<a class="headerlink" href="#record-vs-pause" title="Permalink to this headline"></a></h2>
<p><code class="docutils literal"><span class="pre">autograd</span></code> records computation history on the fly to calculate gradients later.
This is only enabled inside a <code class="docutils literal"><span class="pre">with</span> <span class="pre">autograd.record():</span></code> block.
A <code class="docutils literal"><span class="pre">with</span> <span class="pre">auto_grad.pause()</span></code> block can be used inside a <code class="docutils literal"><span class="pre">record()</span></code> block
to temporarily disable recording.</p>
<p>To compute gradient with respect to an <code class="docutils literal"><span class="pre">NDArray</span></code> <code class="docutils literal"><span class="pre">x</span></code>, first call <code class="docutils literal"><span class="pre">x.attach_grad()</span></code>
to allocate space for the gradient. Then, start a <code class="docutils literal"><span class="pre">with</span> <span class="pre">autograd.record()</span></code> block,
and do some computation. Finally, call <code class="docutils literal"><span class="pre">backward()</span></code> on the result:</p>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">x</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">nd</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">])</span>
<span class="gp">>>> </span><span class="n">x</span><span class="o">.</span><span class="n">attach_grad</span><span class="p">()</span>
<span class="gp">>>> </span><span class="k">with</span> <span class="n">mx</span><span class="o">.</span><span class="n">autograd</span><span class="o">.</span><span class="n">record</span><span class="p">():</span>
<span class="gp">... </span> <span class="n">y</span> <span class="o">=</span> <span class="n">x</span> <span class="o">*</span> <span class="n">x</span> <span class="o">+</span> <span class="mi">1</span>
<span class="gp">>>> </span><span class="n">y</span><span class="o">.</span><span class="n">backward</span><span class="p">()</span>
<span class="gp">>>> </span><span class="k">print</span><span class="p">(</span><span class="n">x</span><span class="o">.</span><span class="n">grad</span><span class="p">)</span>
<span class="go">[ 2. 4. 6. 8.]</span>
<span class="go"><NDArray 4 @cpu(0)></span>
</pre></div>
</div>
</div>
<div class="section" id="train-mode-and-predict-mode">
<span id="train-mode-and-predict-mode"></span><h2>Train mode and Predict Mode<a class="headerlink" href="#train-mode-and-predict-mode" title="Permalink to this headline"></a></h2>
<p>Some operators (Dropout, BatchNorm, etc) behave differently in
training and making predictions.
This can be controlled with <code class="docutils literal"><span class="pre">train_mode</span></code> and <code class="docutils literal"><span class="pre">predict_mode</span></code> scope.</p>
<p>By default, MXNet is in <code class="docutils literal"><span class="pre">predict_mode</span></code>.
A <code class="docutils literal"><span class="pre">with</span> <span class="pre">autograd.record()</span></code> block by default turns on <code class="docutils literal"><span class="pre">train_mode</span></code>
(equivalent to <code class="docutils literal"><span class="pre">with</span> <span class="pre">autograd.record(train_mode=True)</span></code>).
To compute a gradient in prediction mode (as when generating adversarial examples),
call record with <code class="docutils literal"><span class="pre">train_mode=False</span></code> and then call <code class="docutils literal"><span class="pre">backward(train_mode=False)</span></code></p>
<p>Although training usually coincides with recording,
this isn’t always the case.
To control <em>training</em> vs <em>predict_mode</em> without changing
<em>recording</em> vs <em>not recording</em>,
use a <code class="docutils literal"><span class="pre">with</span> <span class="pre">autograd.train_mode():</span></code>
or <code class="docutils literal"><span class="pre">with</span> <span class="pre">autograd.predict_mode():</span></code> block.</p>
<p>Detailed tutorials are available in Part 1 of
<a class="reference external" href="http://gluon.mxnet.io/">the MXNet gluon book</a>.</p>
<script src="../../../_static/js/auto_module_index.js" type="text/javascript"></script></div>
<div class="section" id="autograd">
<span id="autograd"></span><h2>Autograd<a class="headerlink" href="#autograd" title="Permalink to this headline"></a></h2>
<table border="1" class="longtable docutils">
<colgroup>
<col width="10%"/>
<col width="90%"/>
</colgroup>
<tbody valign="top">
<tr class="row-odd"><td><a class="reference internal" href="#mxnet.autograd.record" title="mxnet.autograd.record"><code class="xref py py-obj docutils literal"><span class="pre">record</span></code></a></td>
<td>Returns an autograd recording scope context to be used in ‘with’ statement and captures code that needs gradients to be calculated.</td>
</tr>
<tr class="row-even"><td><a class="reference internal" href="#mxnet.autograd.pause" title="mxnet.autograd.pause"><code class="xref py py-obj docutils literal"><span class="pre">pause</span></code></a></td>
<td>Returns a scope context to be used in ‘with’ statement for codes that do not need gradients to be calculated.</td>
</tr>
<tr class="row-odd"><td><a class="reference internal" href="#mxnet.autograd.train_mode" title="mxnet.autograd.train_mode"><code class="xref py py-obj docutils literal"><span class="pre">train_mode</span></code></a></td>
<td>Returns a scope context to be used in ‘with’ statement in which forward pass behavior is set to training mode, without changing the recording states.</td>
</tr>
<tr class="row-even"><td><a class="reference internal" href="#mxnet.autograd.predict_mode" title="mxnet.autograd.predict_mode"><code class="xref py py-obj docutils literal"><span class="pre">predict_mode</span></code></a></td>
<td>Returns a scope context to be used in ‘with’ statement in which forward pass behavior is set to inference mode, without changing the recording states.</td>
</tr>
<tr class="row-odd"><td><a class="reference internal" href="#mxnet.autograd.backward" title="mxnet.autograd.backward"><code class="xref py py-obj docutils literal"><span class="pre">backward</span></code></a></td>
<td>Compute the gradients of heads w.r.t previously marked variables.</td>
</tr>
<tr class="row-even"><td><a class="reference internal" href="#mxnet.autograd.set_training" title="mxnet.autograd.set_training"><code class="xref py py-obj docutils literal"><span class="pre">set_training</span></code></a></td>
<td>Set status to training/predicting.</td>
</tr>
<tr class="row-odd"><td><a class="reference internal" href="#mxnet.autograd.is_training" title="mxnet.autograd.is_training"><code class="xref py py-obj docutils literal"><span class="pre">is_training</span></code></a></td>
<td>Get status on training/predicting.</td>
</tr>
<tr class="row-even"><td><a class="reference internal" href="#mxnet.autograd.set_recording" title="mxnet.autograd.set_recording"><code class="xref py py-obj docutils literal"><span class="pre">set_recording</span></code></a></td>
<td>Set status to recording/not recording.</td>
</tr>
<tr class="row-odd"><td><a class="reference internal" href="#mxnet.autograd.is_recording" title="mxnet.autograd.is_recording"><code class="xref py py-obj docutils literal"><span class="pre">is_recording</span></code></a></td>
<td>Get status on recording/not recording.</td>
</tr>
<tr class="row-even"><td><a class="reference internal" href="#mxnet.autograd.mark_variables" title="mxnet.autograd.mark_variables"><code class="xref py py-obj docutils literal"><span class="pre">mark_variables</span></code></a></td>
<td>Mark NDArrays as variables to compute gradient for autograd.</td>
</tr>
<tr class="row-odd"><td><a class="reference internal" href="#mxnet.autograd.Function" title="mxnet.autograd.Function"><code class="xref py py-obj docutils literal"><span class="pre">Function</span></code></a></td>
<td>Customize differentiation in autograd.</td>
</tr>
</tbody>
</table>
</div>
<div class="section" id="api-reference">
<span id="api-reference"></span><h2>API Reference<a class="headerlink" href="#api-reference" title="Permalink to this headline"></a></h2>
<script src="../../../_static/js/auto_module_index.js" type="text/javascript"></script><span class="target" id="module-mxnet.autograd"></span><p>Autograd for NDArray.</p>
<dl class="function">
<dt id="mxnet.autograd.set_recording">
<code class="descclassname">mxnet.autograd.</code><code class="descname">set_recording</code><span class="sig-paren">(</span><em>is_recording</em><span class="sig-paren">)</span><a class="reference internal" href="../../../_modules/mxnet/autograd.html#set_recording"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mxnet.autograd.set_recording" title="Permalink to this definition"></a></dt>
<dd><p>Set status to recording/not recording. When recording, graph will be constructed
for gradient computation.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name"/>
<col class="field-body"/>
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>is_recording</strong> (<em>bool</em>) – </td>
</tr>
<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"></td>
</tr>
<tr class="field-odd field"><th class="field-name">Return type:</th><td class="field-body">previous state before this set.</td>
</tr>
</tbody>
</table>
</dd></dl>
<dl class="function">
<dt id="mxnet.autograd.set_training">
<code class="descclassname">mxnet.autograd.</code><code class="descname">set_training</code><span class="sig-paren">(</span><em>train_mode</em><span class="sig-paren">)</span><a class="reference internal" href="../../../_modules/mxnet/autograd.html#set_training"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mxnet.autograd.set_training" title="Permalink to this definition"></a></dt>
<dd><p>Set status to training/predicting. This affects ctx.is_train in operator
running context. For example, Dropout will drop inputs randomly when
train_mode=True while simply passing through if train_mode=False.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name"/>
<col class="field-body"/>
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>train_mode</strong> (<em>bool</em>) – </td>
</tr>
<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"></td>
</tr>
<tr class="field-odd field"><th class="field-name">Return type:</th><td class="field-body">previous state before this set.</td>
</tr>
</tbody>
</table>
</dd></dl>
<dl class="function">
<dt id="mxnet.autograd.is_recording">
<code class="descclassname">mxnet.autograd.</code><code class="descname">is_recording</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../_modules/mxnet/autograd.html#is_recording"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mxnet.autograd.is_recording" title="Permalink to this definition"></a></dt>
<dd><p>Get status on recording/not recording.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name"/>
<col class="field-body"/>
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Returns:</th><td class="field-body"></td>
</tr>
<tr class="field-even field"><th class="field-name">Return type:</th><td class="field-body">Current state of recording.</td>
</tr>
</tbody>
</table>
</dd></dl>
<dl class="function">
<dt id="mxnet.autograd.is_training">
<code class="descclassname">mxnet.autograd.</code><code class="descname">is_training</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../_modules/mxnet/autograd.html#is_training"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mxnet.autograd.is_training" title="Permalink to this definition"></a></dt>
<dd><p>Get status on training/predicting.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name"/>
<col class="field-body"/>
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Returns:</th><td class="field-body"></td>
</tr>
<tr class="field-even field"><th class="field-name">Return type:</th><td class="field-body">Current state of training/predicting.</td>
</tr>
</tbody>
</table>
</dd></dl>
<dl class="function">
<dt id="mxnet.autograd.record">
<code class="descclassname">mxnet.autograd.</code><code class="descname">record</code><span class="sig-paren">(</span><em>train_mode=True</em><span class="sig-paren">)</span><a class="reference internal" href="../../../_modules/mxnet/autograd.html#record"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mxnet.autograd.record" title="Permalink to this definition"></a></dt>
<dd><p>Returns an autograd recording scope context to be used in ‘with’ statement
and captures code that needs gradients to be calculated.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">When forwarding with train_mode=False, the corresponding backward
should also use train_mode=False, otherwise gradient is undefined.</p>
</div>
<p>Example:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="k">with</span> <span class="n">autograd</span><span class="o">.</span><span class="n">record</span><span class="p">():</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">model</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="n">backward</span><span class="p">([</span><span class="n">y</span><span class="p">])</span>
<span class="n">metric</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
<span class="n">optim</span><span class="o">.</span><span class="n">step</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
</pre></div>
</div>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name"/>
<col class="field-body"/>
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>train_mode</strong> (<em>bool</em><em>, </em><em>default True</em>) – Whether the forward pass is in training or predicting mode. This controls the behavior
of some layers such as Dropout, BatchNorm.</td>
</tr>
</tbody>
</table>
</dd></dl>
<dl class="function">
<dt id="mxnet.autograd.pause">
<code class="descclassname">mxnet.autograd.</code><code class="descname">pause</code><span class="sig-paren">(</span><em>train_mode=False</em><span class="sig-paren">)</span><a class="reference internal" href="../../../_modules/mxnet/autograd.html#pause"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mxnet.autograd.pause" title="Permalink to this definition"></a></dt>
<dd><p>Returns a scope context to be used in ‘with’ statement for codes that do not need
gradients to be calculated.</p>
<p>Example:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="k">with</span> <span class="n">autograd</span><span class="o">.</span><span class="n">record</span><span class="p">():</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">model</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="n">backward</span><span class="p">([</span><span class="n">y</span><span class="p">])</span>
<span class="k">with</span> <span class="n">autograd</span><span class="o">.</span><span class="n">pause</span><span class="p">():</span>
<span class="c1"># testing, IO, gradient updates...</span>
</pre></div>
</div>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name"/>
<col class="field-body"/>
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>train_mode</strong> (<em>bool</em><em>, </em><em>default False</em>) – Whether to do forward for training or predicting.</td>
</tr>
</tbody>
</table>
</dd></dl>
<dl class="function">
<dt id="mxnet.autograd.train_mode">
<code class="descclassname">mxnet.autograd.</code><code class="descname">train_mode</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../_modules/mxnet/autograd.html#train_mode"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mxnet.autograd.train_mode" title="Permalink to this definition"></a></dt>
<dd><p>Returns a scope context to be used in ‘with’ statement
in which forward pass behavior is set to training mode,
without changing the recording states.</p>
<p>Example:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">y</span> <span class="o">=</span> <span class="n">model</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="k">with</span> <span class="n">autograd</span><span class="o">.</span><span class="n">train_mode</span><span class="p">():</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">dropout</span><span class="p">(</span><span class="n">y</span><span class="p">)</span>
</pre></div>
</div>
</dd></dl>
<dl class="function">
<dt id="mxnet.autograd.predict_mode">
<code class="descclassname">mxnet.autograd.</code><code class="descname">predict_mode</code><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="reference internal" href="../../../_modules/mxnet/autograd.html#predict_mode"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mxnet.autograd.predict_mode" title="Permalink to this definition"></a></dt>
<dd><p>Returns a scope context to be used in ‘with’ statement
in which forward pass behavior is set to inference mode,
without changing the recording states.</p>
<p>Example:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="k">with</span> <span class="n">autograd</span><span class="o">.</span><span class="n">record</span><span class="p">():</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">model</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="k">with</span> <span class="n">autograd</span><span class="o">.</span><span class="n">predict_mode</span><span class="p">():</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">sampling</span><span class="p">(</span><span class="n">y</span><span class="p">)</span>
<span class="n">backward</span><span class="p">([</span><span class="n">y</span><span class="p">])</span>
</pre></div>
</div>
</dd></dl>
<dl class="function">
<dt id="mxnet.autograd.mark_variables">
<code class="descclassname">mxnet.autograd.</code><code class="descname">mark_variables</code><span class="sig-paren">(</span><em>variables</em>, <em>gradients</em>, <em>grad_reqs='write'</em><span class="sig-paren">)</span><a class="reference internal" href="../../../_modules/mxnet/autograd.html#mark_variables"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mxnet.autograd.mark_variables" title="Permalink to this definition"></a></dt>
<dd><p>Mark NDArrays as variables to compute gradient for autograd.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name"/>
<col class="field-body"/>
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
<li><strong>variables</strong> (<a class="reference internal" href="../ndarray/ndarray.html#mxnet.ndarray.NDArray" title="mxnet.ndarray.NDArray"><em>NDArray</em></a><em> or </em><em>list of NDArray</em>) – </li>
<li><strong>gradients</strong> (<a class="reference internal" href="../ndarray/ndarray.html#mxnet.ndarray.NDArray" title="mxnet.ndarray.NDArray"><em>NDArray</em></a><em> or </em><em>list of NDArray</em>) – </li>
<li><strong>grad_reqs</strong> (<em>str</em><em> or </em><em>list of str</em>) – </li>
</ul>
</td>
</tr>
</tbody>
</table>
</dd></dl>
<dl class="function">
<dt id="mxnet.autograd.backward">
<code class="descclassname">mxnet.autograd.</code><code class="descname">backward</code><span class="sig-paren">(</span><em>heads</em>, <em>head_grads=None</em>, <em>retain_graph=False</em>, <em>train_mode=True</em><span class="sig-paren">)</span><a class="reference internal" href="../../../_modules/mxnet/autograd.html#backward"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mxnet.autograd.backward" title="Permalink to this definition"></a></dt>
<dd><p>Compute the gradients of heads w.r.t previously marked variables.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name"/>
<col class="field-body"/>
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
<li><strong>heads</strong> (<a class="reference internal" href="../ndarray/ndarray.html#mxnet.ndarray.NDArray" title="mxnet.ndarray.NDArray"><em>NDArray</em></a><em> or </em><em>list of NDArray</em>) – Output NDArray(s)</li>
<li><strong>head_grads</strong> (<a class="reference internal" href="../ndarray/ndarray.html#mxnet.ndarray.NDArray" title="mxnet.ndarray.NDArray"><em>NDArray</em></a><em> or </em><em>list of NDArray</em><em> or </em><em>None</em>) – Gradients with respect to heads.</li>
<li><strong>train_mode</strong> (<em>bool</em><em>, </em><em>optional</em>) – Whether to do backward for training or predicting.</li>
</ul>
</td>
</tr>
</tbody>
</table>
</dd></dl>
<dl class="function">
<dt id="mxnet.autograd.grad">
<code class="descclassname">mxnet.autograd.</code><code class="descname">grad</code><span class="sig-paren">(</span><em>heads</em>, <em>variables</em>, <em>head_grads=None</em>, <em>retain_graph=None</em>, <em>create_graph=False</em>, <em>train_mode=True</em><span class="sig-paren">)</span><a class="reference internal" href="../../../_modules/mxnet/autograd.html#grad"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mxnet.autograd.grad" title="Permalink to this definition"></a></dt>
<dd><p>Compute the gradients of heads w.r.t variables. Gradients will be
returned as new NDArrays instead of stored into <cite>variable.grad</cite>.
Supports recording gradient graph for computing higher order gradients.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Currently only a very limited set of operators support higher order gradients.</p>
</div>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name"/>
<col class="field-body"/>
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
<li><strong>heads</strong> (<a class="reference internal" href="../ndarray/ndarray.html#mxnet.ndarray.NDArray" title="mxnet.ndarray.NDArray"><em>NDArray</em></a><em> or </em><em>list of NDArray</em>) – Output NDArray(s)</li>
<li><strong>variables</strong> (<a class="reference internal" href="../ndarray/ndarray.html#mxnet.ndarray.NDArray" title="mxnet.ndarray.NDArray"><em>NDArray</em></a><em> or </em><em>list of NDArray</em>) – Input variables to compute gradients for.</li>
<li><strong>head_grads</strong> (<a class="reference internal" href="../ndarray/ndarray.html#mxnet.ndarray.NDArray" title="mxnet.ndarray.NDArray"><em>NDArray</em></a><em> or </em><em>list of NDArray</em><em> or </em><em>None</em>) – Gradients with respect to heads.</li>
<li><strong>retain_graph</strong> (<em>bool</em>) – Whether to keep computation graph to differentiate again, instead
of clearing history and release memory. Defaults to the same value
as create_graph.</li>
<li><strong>create_graph</strong> (<em>bool</em>) – Whether to record gradient graph for computing higher order</li>
<li><strong>train_mode</strong> (<em>bool</em><em>, </em><em>optional</em>) – Whether to do backward for training or prediction.</li>
</ul>
</td>
</tr>
<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first">Gradients with respect to variables.</p>
</td>
</tr>
<tr class="field-odd field"><th class="field-name">Return type:</th><td class="field-body"><p class="first last"><a class="reference internal" href="../ndarray/ndarray.html#mxnet.ndarray.NDArray" title="mxnet.ndarray.NDArray">NDArray</a> or list of NDArray</p>
</td>
</tr>
</tbody>
</table>
<p class="rubric">Examples</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">x</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">nd</span><span class="o">.</span><span class="n">ones</span><span class="p">((</span><span class="mi">1</span><span class="p">,))</span>
<span class="gp">>>> </span><span class="n">x</span><span class="o">.</span><span class="n">attach_grad</span><span class="p">()</span>
<span class="gp">>>> </span><span class="k">with</span> <span class="n">mx</span><span class="o">.</span><span class="n">autograd</span><span class="o">.</span><span class="n">record</span><span class="p">():</span>
<span class="gp">... </span> <span class="n">z</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">nd</span><span class="o">.</span><span class="n">elemwise_add</span><span class="p">(</span><span class="n">mx</span><span class="o">.</span><span class="n">nd</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">x</span><span class="p">),</span> <span class="n">x</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">dx</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">autograd</span><span class="o">.</span><span class="n">grad</span><span class="p">(</span><span class="n">z</span><span class="p">,</span> <span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">create_graph</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="gp">>>> </span><span class="nb">print</span><span class="p">(</span><span class="n">dx</span><span class="p">)</span>
<span class="go">[</span>
<span class="go">[ 3.71828175]</span>
<span class="go"><NDArray 1 @cpu(0)>]</span>
</pre></div>
</div>
</dd></dl>
<dl class="function">
<dt id="mxnet.autograd.get_symbol">
<code class="descclassname">mxnet.autograd.</code><code class="descname">get_symbol</code><span class="sig-paren">(</span><em>x</em><span class="sig-paren">)</span><a class="reference internal" href="../../../_modules/mxnet/autograd.html#get_symbol"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mxnet.autograd.get_symbol" title="Permalink to this definition"></a></dt>
<dd><p>Retrieve recorded computation history as <cite>Symbol</cite>.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name"/>
<col class="field-body"/>
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><strong>x</strong> (<a class="reference internal" href="../ndarray/ndarray.html#mxnet.ndarray.NDArray" title="mxnet.ndarray.NDArray"><em>NDArray</em></a>) – Array representing the head of computation graph.</td>
</tr>
<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body">The retrieved Symbol.</td>
</tr>
<tr class="field-odd field"><th class="field-name">Return type:</th><td class="field-body"><a class="reference internal" href="../symbol/symbol.html#mxnet.symbol.Symbol" title="mxnet.symbol.Symbol">Symbol</a></td>
</tr>
</tbody>
</table>
</dd></dl>
<dl class="class">
<dt id="mxnet.autograd.Function">
<em class="property">class </em><code class="descclassname">mxnet.autograd.</code><code class="descname">Function</code><a class="reference internal" href="../../../_modules/mxnet/autograd.html#Function"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mxnet.autograd.Function" title="Permalink to this definition"></a></dt>
<dd><p>Customize differentiation in autograd.</p>
<p>If you don’t want to use the gradients computed by the default
chain-rule, you can use Function to customize differentiation for
computation. You define your computation in
the forward method and provide the customized differentiation
in the backward method. During gradient computation, autograd will
use the user-defined backward function instead of the default chain-rule.
You can also cast to numpy array and back for some operations in
forward and backward.</p>
<p>For example, a stable sigmoid function can be defined as:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">sigmoid</span><span class="p">(</span><span class="n">mx</span><span class="o">.</span><span class="n">autograd</span><span class="o">.</span><span class="n">Function</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span>
<span class="n">y</span> <span class="o">=</span> <span class="mi">1</span> <span class="o">/</span> <span class="p">(</span><span class="mi">1</span> <span class="o">+</span> <span class="n">mx</span><span class="o">.</span><span class="n">nd</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="o">-</span><span class="n">x</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">save_for_backward</span><span class="p">(</span><span class="n">y</span><span class="p">)</span>
<span class="k">return</span> <span class="n">y</span>
<span class="k">def</span> <span class="nf">backward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">dy</span><span class="p">):</span>
<span class="c1"># backward takes as many inputs as forward's return value,</span>
<span class="c1"># and returns as many NDArrays as forward's arguments.</span>
<span class="n">y</span><span class="p">,</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">saved_tensors</span>
<span class="k">return</span> <span class="n">dy</span> <span class="o">*</span> <span class="n">y</span> <span class="o">*</span> <span class="p">(</span><span class="mi">1</span><span class="o">-</span><span class="n">y</span><span class="p">)</span>
</pre></div>
</div>
<p>Then, the function can be used in the following way:</p>
<div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">func</span> <span class="o">=</span> <span class="n">sigmoid</span><span class="p">()</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">mx</span><span class="o">.</span><span class="n">nd</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">uniform</span><span class="p">(</span><span class="n">shape</span><span class="o">=</span><span class="p">(</span><span class="mi">10</span><span class="p">,))</span>
<span class="n">x</span><span class="o">.</span><span class="n">attach_grad</span><span class="p">()</span>
<span class="k">with</span> <span class="n">mx</span><span class="o">.</span><span class="n">autograd</span><span class="o">.</span><span class="n">record</span><span class="p">():</span>
<span class="n">m</span> <span class="o">=</span> <span class="n">func</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="n">m</span><span class="o">.</span><span class="n">backward</span><span class="p">()</span>
<span class="n">dx</span> <span class="o">=</span> <span class="n">x</span><span class="o">.</span><span class="n">grad</span><span class="o">.</span><span class="n">asnumpy</span><span class="p">()</span>
</pre></div>
</div>
<dl class="method">
<dt id="mxnet.autograd.Function.forward">
<code class="descname">forward</code><span class="sig-paren">(</span><em>*inputs</em><span class="sig-paren">)</span><a class="reference internal" href="../../../_modules/mxnet/autograd.html#Function.forward"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mxnet.autograd.Function.forward" title="Permalink to this definition"></a></dt>
<dd><p>Forward computation.</p>
</dd></dl>
<dl class="method">
<dt id="mxnet.autograd.Function.backward">
<code class="descname">backward</code><span class="sig-paren">(</span><em>*output_grads</em><span class="sig-paren">)</span><a class="reference internal" href="../../../_modules/mxnet/autograd.html#Function.backward"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#mxnet.autograd.Function.backward" title="Permalink to this definition"></a></dt>
<dd><p>Backward computation.</p>
<p>Takes as many inputs as forward’s outputs,
and returns as many NDArrays as forward’s inputs.</p>
</dd></dl>
</dd></dl>
<script>auto_index("api-reference");</script></div>
</div>
</div>
</div>
<div aria-label="main navigation" class="sphinxsidebar rightsidebar" role="navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="../../../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Autograd Package</a><ul>
<li><a class="reference internal" href="#overview">Overview</a></li>
<li><a class="reference internal" href="#record-vs-pause">Record vs Pause</a></li>
<li><a class="reference internal" href="#train-mode-and-predict-mode">Train mode and Predict Mode</a></li>
<li><a class="reference internal" href="#autograd">Autograd</a></li>
<li><a class="reference internal" href="#api-reference">API Reference</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div><div class="footer">
<div class="section-disclaimer">
<div class="container">
<div>
<img height="60" src="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/apache_incubator_logo.png"/>
<p>
Apache MXNet is an effort undergoing incubation at The Apache Software Foundation (ASF), <strong>sponsored by the <i>Apache Incubator</i></strong>. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
</p>
<p>
"Copyright © 2017-2018, The Apache Software Foundation
Apache MXNet, MXNet, Apache, the Apache feather, and the Apache MXNet project logo are either registered trademarks or trademarks of the Apache Software Foundation."
</p>
</div>
</div>
</div>
</div> <!-- pagename != index -->
</div>
<script crossorigin="anonymous" integrity="sha384-0mSbJDEHialfmuBBQP6A4Qrprq5OVfW37PRR3j5ELqxss1yVqOtnepnHVP9aJ7xS" src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/js/bootstrap.min.js"></script>
<script src="../../../_static/js/sidebar.js" type="text/javascript"></script>
<script src="../../../_static/js/search.js" type="text/javascript"></script>
<script src="../../../_static/js/navbar.js" type="text/javascript"></script>
<script src="../../../_static/js/clipboard.min.js" type="text/javascript"></script>
<script src="../../../_static/js/copycode.js" type="text/javascript"></script>
<script src="../../../_static/js/page.js" type="text/javascript"></script>
<script src="../../../_static/js/docversion.js" type="text/javascript"></script>
<script type="text/javascript">
$('body').ready(function () {
$('body').css('visibility', 'visible');
});
</script>
</body>
</html>