| <!DOCTYPE html> |
| |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"/> |
| <meta content="IE=edge" http-equiv="X-UA-Compatible"/> |
| <meta content="width=device-width, initial-scale=1" name="viewport"/> |
| <meta content="mxnet.autograd" property="og:title"> |
| <meta content="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/og-logo.png" property="og:image"> |
| <meta content="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/og-logo.png" property="og:image:secure_url"> |
| <meta content="mxnet.autograd" property="og:description"/> |
| <title>mxnet.autograd — mxnet documentation</title> |
| <link crossorigin="anonymous" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css" integrity="sha384-1q8mTJOASx8j1Au+a5WDVnPi2lkFfwwEAa8hDDdjZlpLegxhjVME1fgjWPGmkzs7" rel="stylesheet"/> |
| <link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.5.0/css/font-awesome.min.css" rel="stylesheet"/> |
| <link href="../../_static/basic.css" rel="stylesheet" type="text/css"> |
| <link href="../../_static/pygments.css" rel="stylesheet" type="text/css"> |
| <link href="../../_static/mxnet.css" rel="stylesheet" type="text/css"/> |
| <script type="text/javascript"> |
| var DOCUMENTATION_OPTIONS = { |
| URL_ROOT: '../../', |
| VERSION: '', |
| COLLAPSE_INDEX: false, |
| FILE_SUFFIX: '.html', |
| HAS_SOURCE: true, |
| SOURCELINK_SUFFIX: '.txt' |
| }; |
| </script> |
| <script src="https://code.jquery.com/jquery-1.11.1.min.js" type="text/javascript"></script> |
| <script src="../../_static/underscore.js" type="text/javascript"></script> |
| <script src="../../_static/searchtools_custom.js" type="text/javascript"></script> |
| <script src="../../_static/doctools.js" type="text/javascript"></script> |
| <script src="../../_static/selectlang.js" type="text/javascript"></script> |
| <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script> |
| <script type="text/javascript"> jQuery(function() { Search.loadIndex("/versions/1.2.1/searchindex.js"); Search.init();}); </script> |
| <script> |
| (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ |
| (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new |
| Date();a=s.createElement(o), |
| m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) |
| })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); |
| |
| ga('create', 'UA-96378503-1', 'auto'); |
| ga('send', 'pageview'); |
| |
| </script> |
| <!-- --> |
| <!-- <script type="text/javascript" src="../../_static/jquery.js"></script> --> |
| <!-- --> |
| <!-- <script type="text/javascript" src="../../_static/underscore.js"></script> --> |
| <!-- --> |
| <!-- <script type="text/javascript" src="../../_static/doctools.js"></script> --> |
| <!-- --> |
| <!-- <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script> --> |
| <!-- --> |
| <link href="../../genindex.html" rel="index" title="Index"> |
| <link href="../../search.html" rel="search" title="Search"/> |
| <link href="../index.html" rel="up" title="Module code"/> |
| <link href="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/mxnet-icon.png" rel="icon" type="image/png"/> |
| </link></link></link></meta></meta></meta></head> |
| <body background="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/mxnet-background-compressed.jpeg" role="document"> |
| <div class="content-block"><div class="navbar navbar-fixed-top"> |
| <div class="container" id="navContainer"> |
| <div class="innder" id="header-inner"> |
| <h1 id="logo-wrap"> |
| <a href="../../" id="logo"><img src="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/mxnet_logo.png"/></a> |
| </h1> |
| <nav class="nav-bar" id="main-nav"> |
| <a class="main-nav-link" href="/versions/1.2.1/install/index.html">Install</a> |
| <span id="dropdown-menu-position-anchor"> |
| <a aria-expanded="true" aria-haspopup="true" class="main-nav-link dropdown-toggle" data-toggle="dropdown" href="#" role="button">Gluon <span class="caret"></span></a> |
| <ul class="dropdown-menu navbar-menu" id="package-dropdown-menu"> |
| <li><a class="main-nav-link" href="/versions/1.2.1/tutorials/gluon/gluon.html">About</a></li> |
| <li><a class="main-nav-link" href="https://www.d2l.ai/">Dive into Deep Learning</a></li> |
| <li><a class="main-nav-link" href="https://gluon-cv.mxnet.io">GluonCV Toolkit</a></li> |
| <li><a class="main-nav-link" href="https://gluon-nlp.mxnet.io/">GluonNLP Toolkit</a></li> |
| </ul> |
| </span> |
| <span id="dropdown-menu-position-anchor"> |
| <a aria-expanded="true" aria-haspopup="true" class="main-nav-link dropdown-toggle" data-toggle="dropdown" href="#" role="button">API <span class="caret"></span></a> |
| <ul class="dropdown-menu navbar-menu" id="package-dropdown-menu"> |
| <li><a class="main-nav-link" href="/versions/1.2.1/api/python/index.html">Python</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/api/c++/index.html">C++</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/api/julia/index.html">Julia</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/api/perl/index.html">Perl</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/api/r/index.html">R</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/api/scala/index.html">Scala</a></li> |
| </ul> |
| </span> |
| <span id="dropdown-menu-position-anchor-docs"> |
| <a aria-expanded="true" aria-haspopup="true" class="main-nav-link dropdown-toggle" data-toggle="dropdown" href="#" role="button">Docs <span class="caret"></span></a> |
| <ul class="dropdown-menu navbar-menu" id="package-dropdown-menu-docs"> |
| <li><a class="main-nav-link" href="/versions/1.2.1/faq/index.html">FAQ</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/tutorials/index.html">Tutorials</a> |
| <li><a class="main-nav-link" href="https://github.com/apache/incubator-mxnet/tree/1.2.1/example">Examples</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/architecture/index.html">Architecture</a></li> |
| <li><a class="main-nav-link" href="https://cwiki.apache.org/confluence/display/MXNET/Apache+MXNet+Home">Developer Wiki</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/model_zoo/index.html">Model Zoo</a></li> |
| <li><a class="main-nav-link" href="https://github.com/onnx/onnx-mxnet">ONNX</a></li> |
| </li></ul> |
| </span> |
| <span id="dropdown-menu-position-anchor-community"> |
| <a aria-expanded="true" aria-haspopup="true" class="main-nav-link dropdown-toggle" data-toggle="dropdown" href="#" role="button">Community <span class="caret"></span></a> |
| <ul class="dropdown-menu navbar-menu" id="package-dropdown-menu-community"> |
| <li><a class="main-nav-link" href="http://discuss.mxnet.io">Forum</a></li> |
| <li><a class="main-nav-link" href="https://github.com/apache/incubator-mxnet/tree/1.2.1">Github</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/community/contribute.html">Contribute</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/community/powered_by.html">Powered By</a></li> |
| </ul> |
| </span> |
| <span id="dropdown-menu-position-anchor-version" style="position: relative"><a href="#" class="main-nav-link dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="true">1.2.1<span class="caret"></span></a><ul id="package-dropdown-menu" class="dropdown-menu"><li><a href="/">master</a></li><li><a href=/versions/1.6/>1.6</a></li><li><a href=/versions/1.5.0/>1.5.0</a></li><li><a href=/versions/1.4.1/>1.4.1</a></li><li><a href=/versions/1.3.1/>1.3.1</a></li><li><a href=/versions/1.2.1/>1.2.1</a></li><li><a href=/versions/1.1.0/>1.1.0</a></li><li><a href=/versions/1.0.0/>1.0.0</a></li><li><a href=/versions/0.12.1/>0.12.1</a></li><li><a href=/versions/0.11.0/>0.11.0</a></li></ul></span></nav> |
| <script> function getRootPath(){ return "../../" } </script> |
| <div class="burgerIcon dropdown"> |
| <a class="dropdown-toggle" data-toggle="dropdown" href="#" role="button">☰</a> |
| <ul class="dropdown-menu" id="burgerMenu"> |
| <li><a href="/versions/1.2.1/install/index.html">Install</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/tutorials/index.html">Tutorials</a></li> |
| <li class="dropdown-submenu dropdown"> |
| <a aria-expanded="true" aria-haspopup="true" class="dropdown-toggle burger-link" data-toggle="dropdown" href="#" tabindex="-1">Gluon</a> |
| <ul class="dropdown-menu navbar-menu" id="package-dropdown-menu"> |
| <li><a class="main-nav-link" href="/versions/1.2.1/tutorials/gluon/gluon.html">About</a></li> |
| <li><a class="main-nav-link" href="http://gluon.mxnet.io">The Straight Dope (Tutorials)</a></li> |
| <li><a class="main-nav-link" href="https://gluon-cv.mxnet.io">GluonCV Toolkit</a></li> |
| <li><a class="main-nav-link" href="https://gluon-nlp.mxnet.io/">GluonNLP Toolkit</a></li> |
| </ul> |
| </li> |
| <li class="dropdown-submenu"> |
| <a aria-expanded="true" aria-haspopup="true" class="dropdown-toggle burger-link" data-toggle="dropdown" href="#" tabindex="-1">API</a> |
| <ul class="dropdown-menu"> |
| <li><a class="main-nav-link" href="/versions/1.2.1/api/python/index.html">Python</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/api/c++/index.html">C++</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/api/julia/index.html">Julia</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/api/perl/index.html">Perl</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/api/r/index.html">R</a></li> |
| <li><a class="main-nav-link" href="/versions/1.2.1/api/scala/index.html">Scala</a></li> |
| </ul> |
| </li> |
| <li class="dropdown-submenu"> |
| <a aria-expanded="true" aria-haspopup="true" class="dropdown-toggle burger-link" data-toggle="dropdown" href="#" tabindex="-1">Docs</a> |
| <ul class="dropdown-menu"> |
| <li><a href="/versions/1.2.1/faq/index.html" tabindex="-1">FAQ</a></li> |
| <li><a href="/versions/1.2.1/tutorials/index.html" tabindex="-1">Tutorials</a></li> |
| <li><a href="https://github.com/apache/incubator-mxnet/tree/1.2.1/example" tabindex="-1">Examples</a></li> |
| <li><a href="/versions/1.2.1/architecture/index.html" tabindex="-1">Architecture</a></li> |
| <li><a href="https://cwiki.apache.org/confluence/display/MXNET/Apache+MXNet+Home" tabindex="-1">Developer Wiki</a></li> |
| <li><a href="/versions/1.2.1/model_zoo/index.html" tabindex="-1">Gluon Model Zoo</a></li> |
| <li><a href="https://github.com/onnx/onnx-mxnet" tabindex="-1">ONNX</a></li> |
| </ul> |
| </li> |
| <li class="dropdown-submenu dropdown"> |
| <a aria-haspopup="true" class="dropdown-toggle burger-link" data-toggle="dropdown" href="#" role="button" tabindex="-1">Community</a> |
| <ul class="dropdown-menu"> |
| <li><a href="http://discuss.mxnet.io" tabindex="-1">Forum</a></li> |
| <li><a href="https://github.com/apache/incubator-mxnet/tree/1.2.1" tabindex="-1">Github</a></li> |
| <li><a href="/versions/1.2.1/community/contribute.html" tabindex="-1">Contribute</a></li> |
| <li><a href="/versions/1.2.1/community/powered_by.html" tabindex="-1">Powered By</a></li> |
| </ul> |
| </li> |
| <li id="dropdown-menu-position-anchor-version-mobile" class="dropdown-submenu" style="position: relative"><a href="#" tabindex="-1">1.2.1</a><ul class="dropdown-menu"><li><a tabindex="-1" href=/>master</a></li><li><a tabindex="-1" href=/versions/1.6/>1.6</a></li><li><a tabindex="-1" href=/versions/1.5.0/>1.5.0</a></li><li><a tabindex="-1" href=/versions/1.4.1/>1.4.1</a></li><li><a tabindex="-1" href=/versions/1.3.1/>1.3.1</a></li><li><a tabindex="-1" href=/versions/1.2.1/>1.2.1</a></li><li><a tabindex="-1" href=/versions/1.1.0/>1.1.0</a></li><li><a tabindex="-1" href=/versions/1.0.0/>1.0.0</a></li><li><a tabindex="-1" href=/versions/0.12.1/>0.12.1</a></li><li><a tabindex="-1" href=/versions/0.11.0/>0.11.0</a></li></ul></li></ul> |
| </div> |
| <div class="plusIcon dropdown"> |
| <a class="dropdown-toggle" data-toggle="dropdown" href="#" role="button"><span aria-hidden="true" class="glyphicon glyphicon-plus"></span></a> |
| <ul class="dropdown-menu dropdown-menu-right" id="plusMenu"></ul> |
| </div> |
| <div id="search-input-wrap"> |
| <form action="../../search.html" autocomplete="off" class="" method="get" role="search"> |
| <div class="form-group inner-addon left-addon"> |
| <i class="glyphicon glyphicon-search"></i> |
| <input class="form-control" name="q" placeholder="Search" type="text"/> |
| </div> |
| <input name="check_keywords" type="hidden" value="yes"> |
| <input name="area" type="hidden" value="default"/> |
| </input></form> |
| <div id="search-preview"></div> |
| </div> |
| <div id="searchIcon"> |
| <span aria-hidden="true" class="glyphicon glyphicon-search"></span> |
| </div> |
| <!-- <div id="lang-select-wrap"> --> |
| <!-- <label id="lang-select-label"> --> |
| <!-- <\!-- <i class="fa fa-globe"></i> -\-> --> |
| <!-- <span></span> --> |
| <!-- </label> --> |
| <!-- <select id="lang-select"> --> |
| <!-- <option value="en">Eng</option> --> |
| <!-- <option value="zh">中文</option> --> |
| <!-- </select> --> |
| <!-- </div> --> |
| <!-- <a id="mobile-nav-toggle"> |
| <span class="mobile-nav-toggle-bar"></span> |
| <span class="mobile-nav-toggle-bar"></span> |
| <span class="mobile-nav-toggle-bar"></span> |
| </a> --> |
| </div> |
| </div> |
| </div> |
| <script type="text/javascript"> |
| $('body').css('background', 'white'); |
| </script> |
| <div class="container"> |
| <div class="row"> |
| <div aria-label="main navigation" class="sphinxsidebar leftsidebar" role="navigation"> |
| <div class="sphinxsidebarwrapper"> |
| <ul> |
| <li class="toctree-l1"><a class="reference internal" href="../../api/python/index.html">Python Documents</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../api/r/index.html">R Documents</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../api/julia/index.html">Julia Documents</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../api/c++/index.html">C++ Documents</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../api/scala/index.html">Scala Documents</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../api/perl/index.html">Perl Documents</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../faq/index.html">HowTo Documents</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../architecture/index.html">System Documents</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../tutorials/index.html">Tutorials</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../community/index.html">Community</a></li> |
| </ul> |
| </div> |
| </div> |
| <div class="content"> |
| <div class="page-tracker"></div> |
| <h1>Source code for mxnet.autograd</h1><div class="highlight"><pre> |
| <span></span><span class="c1"># Licensed to the Apache Software Foundation (ASF) under one</span> |
| <span class="c1"># or more contributor license agreements. See the NOTICE file</span> |
| <span class="c1"># distributed with this work for additional information</span> |
| <span class="c1"># regarding copyright ownership. The ASF licenses this file</span> |
| <span class="c1"># to you under the Apache License, Version 2.0 (the</span> |
| <span class="c1"># "License"); you may not use this file except in compliance</span> |
| <span class="c1"># with the License. You may obtain a copy of the License at</span> |
| <span class="c1">#</span> |
| <span class="c1"># http://www.apache.org/licenses/LICENSE-2.0</span> |
| <span class="c1">#</span> |
| <span class="c1"># Unless required by applicable law or agreed to in writing,</span> |
| <span class="c1"># software distributed under the License is distributed on an</span> |
| <span class="c1"># "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY</span> |
| <span class="c1"># KIND, either express or implied. See the License for the</span> |
| <span class="c1"># specific language governing permissions and limitations</span> |
| <span class="c1"># under the License.</span> |
| |
| <span class="c1"># coding: utf-8</span> |
| <span class="sd">"""Autograd for NDArray."""</span> |
| <span class="kn">from</span> <span class="nn">__future__</span> <span class="k">import</span> <span class="n">absolute_import</span> |
| <span class="kn">from</span> <span class="nn">__future__</span> <span class="k">import</span> <span class="n">division</span> |
| |
| <span class="kn">from</span> <span class="nn">array</span> <span class="k">import</span> <span class="n">array</span> |
| <span class="kn">from</span> <span class="nn">threading</span> <span class="k">import</span> <span class="n">Lock</span> |
| <span class="kn">import</span> <span class="nn">traceback</span> |
| <span class="kn">import</span> <span class="nn">ctypes</span> |
| <span class="kn">from</span> <span class="nn">ctypes</span> <span class="k">import</span> <span class="n">c_int</span><span class="p">,</span> <span class="n">c_void_p</span><span class="p">,</span> <span class="n">CFUNCTYPE</span><span class="p">,</span> <span class="n">POINTER</span><span class="p">,</span> <span class="n">cast</span> |
| <span class="kn">from</span> <span class="nn">.base</span> <span class="k">import</span> <span class="n">_LIB</span><span class="p">,</span> <span class="n">check_call</span><span class="p">,</span> <span class="n">string_types</span><span class="p">,</span> <span class="n">mx_uint</span> |
| <span class="kn">from</span> <span class="nn">.base</span> <span class="k">import</span> <span class="n">NDArrayHandle</span><span class="p">,</span> <span class="n">c_array</span><span class="p">,</span> <span class="n">c_handle_array</span><span class="p">,</span> <span class="n">c_array_buf</span><span class="p">,</span> <span class="n">MXCallbackList</span><span class="p">,</span> <span class="n">SymbolHandle</span> |
| <span class="kn">from</span> <span class="nn">.ndarray</span> <span class="k">import</span> <span class="n">NDArray</span><span class="p">,</span> <span class="n">_ndarray_cls</span> |
| <span class="kn">from</span> <span class="nn">.ndarray</span> <span class="k">import</span> <span class="n">_GRAD_REQ_MAP</span> |
| <span class="kn">from</span> <span class="nn">.symbol</span> <span class="k">import</span> <span class="n">Symbol</span> |
| |
| |
| <div class="viewcode-block" id="set_recording"><a class="viewcode-back" href="../../api/python/autograd/autograd.html#mxnet.autograd.set_recording">[docs]</a><span class="k">def</span> <span class="nf">set_recording</span><span class="p">(</span><span class="n">is_recording</span><span class="p">):</span> <span class="c1">#pylint: disable=redefined-outer-name</span> |
| <span class="sd">"""Set status to recording/not recording. When recording, graph will be constructed</span> |
| <span class="sd"> for gradient computation.</span> |
| |
| <span class="sd"> Parameters</span> |
| <span class="sd"> ----------</span> |
| <span class="sd"> is_recording: bool</span> |
| |
| <span class="sd"> Returns</span> |
| <span class="sd"> -------</span> |
| <span class="sd"> previous state before this set.</span> |
| <span class="sd"> """</span> |
| <span class="n">prev</span> <span class="o">=</span> <span class="n">ctypes</span><span class="o">.</span><span class="n">c_int</span><span class="p">()</span> |
| <span class="n">check_call</span><span class="p">(</span><span class="n">_LIB</span><span class="o">.</span><span class="n">MXAutogradSetIsRecording</span><span class="p">(</span> |
| <span class="n">ctypes</span><span class="o">.</span><span class="n">c_int</span><span class="p">(</span><span class="n">is_recording</span><span class="p">),</span> <span class="n">ctypes</span><span class="o">.</span><span class="n">byref</span><span class="p">(</span><span class="n">prev</span><span class="p">)))</span> |
| <span class="k">return</span> <span class="nb">bool</span><span class="p">(</span><span class="n">prev</span><span class="o">.</span><span class="n">value</span><span class="p">)</span></div> |
| |
| <div class="viewcode-block" id="set_training"><a class="viewcode-back" href="../../api/python/autograd/autograd.html#mxnet.autograd.set_training">[docs]</a><span class="k">def</span> <span class="nf">set_training</span><span class="p">(</span><span class="n">train_mode</span><span class="p">):</span> <span class="c1">#pylint: disable=redefined-outer-name</span> |
| <span class="sd">"""Set status to training/predicting. This affects ctx.is_train in operator</span> |
| <span class="sd"> running context. For example, Dropout will drop inputs randomly when</span> |
| <span class="sd"> train_mode=True while simply passing through if train_mode=False.</span> |
| |
| <span class="sd"> Parameters</span> |
| <span class="sd"> ----------</span> |
| <span class="sd"> train_mode: bool</span> |
| |
| <span class="sd"> Returns</span> |
| <span class="sd"> -------</span> |
| <span class="sd"> previous state before this set.</span> |
| <span class="sd"> """</span> |
| <span class="n">prev</span> <span class="o">=</span> <span class="n">ctypes</span><span class="o">.</span><span class="n">c_int</span><span class="p">()</span> |
| <span class="n">check_call</span><span class="p">(</span><span class="n">_LIB</span><span class="o">.</span><span class="n">MXAutogradSetIsTraining</span><span class="p">(</span> |
| <span class="n">ctypes</span><span class="o">.</span><span class="n">c_int</span><span class="p">(</span><span class="n">train_mode</span><span class="p">),</span> <span class="n">ctypes</span><span class="o">.</span><span class="n">byref</span><span class="p">(</span><span class="n">prev</span><span class="p">)))</span> |
| <span class="k">return</span> <span class="nb">bool</span><span class="p">(</span><span class="n">prev</span><span class="o">.</span><span class="n">value</span><span class="p">)</span></div> |
| |
| <div class="viewcode-block" id="is_recording"><a class="viewcode-back" href="../../api/python/autograd/autograd.html#mxnet.autograd.is_recording">[docs]</a><span class="k">def</span> <span class="nf">is_recording</span><span class="p">():</span> |
| <span class="sd">"""Get status on recording/not recording.</span> |
| |
| <span class="sd"> Returns</span> |
| <span class="sd"> -------</span> |
| <span class="sd"> Current state of recording.</span> |
| <span class="sd"> """</span> |
| <span class="n">curr</span> <span class="o">=</span> <span class="n">ctypes</span><span class="o">.</span><span class="n">c_bool</span><span class="p">()</span> |
| <span class="n">check_call</span><span class="p">(</span><span class="n">_LIB</span><span class="o">.</span><span class="n">MXAutogradIsRecording</span><span class="p">(</span><span class="n">ctypes</span><span class="o">.</span><span class="n">byref</span><span class="p">(</span><span class="n">curr</span><span class="p">)))</span> |
| <span class="k">return</span> <span class="n">curr</span><span class="o">.</span><span class="n">value</span></div> |
| |
| <div class="viewcode-block" id="is_training"><a class="viewcode-back" href="../../api/python/autograd/autograd.html#mxnet.autograd.is_training">[docs]</a><span class="k">def</span> <span class="nf">is_training</span><span class="p">():</span> |
| <span class="sd">"""Get status on training/predicting.</span> |
| |
| <span class="sd"> Returns</span> |
| <span class="sd"> -------</span> |
| <span class="sd"> Current state of training/predicting.</span> |
| <span class="sd"> """</span> |
| <span class="n">curr</span> <span class="o">=</span> <span class="n">ctypes</span><span class="o">.</span><span class="n">c_bool</span><span class="p">()</span> |
| <span class="n">check_call</span><span class="p">(</span><span class="n">_LIB</span><span class="o">.</span><span class="n">MXAutogradIsTraining</span><span class="p">(</span><span class="n">ctypes</span><span class="o">.</span><span class="n">byref</span><span class="p">(</span><span class="n">curr</span><span class="p">)))</span> |
| <span class="k">return</span> <span class="n">curr</span><span class="o">.</span><span class="n">value</span></div> |
| |
| |
| <span class="k">class</span> <span class="nc">_RecordingStateScope</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span> |
| <span class="sd">"""Scope for managing training state.</span> |
| |
| <span class="sd"> Example::</span> |
| |
| <span class="sd"> with _RecordingStateScope(True, True):</span> |
| <span class="sd"> y = model(x)</span> |
| <span class="sd"> backward([y])</span> |
| |
| <span class="sd"> """</span> |
| <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">is_record</span><span class="p">,</span> <span class="n">train_mode</span><span class="p">):</span> <span class="c1">#pylint: disable=redefined-outer-name</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">_enter_is_record</span> <span class="o">=</span> <span class="n">is_record</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">_enter_train_mode</span> <span class="o">=</span> <span class="n">train_mode</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">_prev_is_record</span> <span class="o">=</span> <span class="kc">None</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">_prev_train_mode</span> <span class="o">=</span> <span class="kc">None</span> |
| |
| <span class="k">def</span> <span class="nf">__enter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> |
| <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_enter_is_record</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">_prev_is_record</span> <span class="o">=</span> <span class="n">set_recording</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_enter_is_record</span><span class="p">)</span> |
| <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_enter_train_mode</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">_prev_train_mode</span> <span class="o">=</span> <span class="n">set_training</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_enter_train_mode</span><span class="p">)</span> |
| |
| <span class="k">def</span> <span class="nf">__exit__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">ptype</span><span class="p">,</span> <span class="n">value</span><span class="p">,</span> <span class="n">trace</span><span class="p">):</span> |
| <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_enter_is_record</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">_prev_is_record</span> <span class="o">!=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_enter_is_record</span><span class="p">:</span> |
| <span class="n">set_recording</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_prev_is_record</span><span class="p">)</span> |
| <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">_enter_train_mode</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="ow">and</span> <span class="bp">self</span><span class="o">.</span><span class="n">_prev_train_mode</span> <span class="o">!=</span> <span class="bp">self</span><span class="o">.</span><span class="n">_enter_train_mode</span><span class="p">:</span> |
| <span class="n">set_training</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_prev_train_mode</span><span class="p">)</span> |
| |
| |
| <div class="viewcode-block" id="record"><a class="viewcode-back" href="../../api/python/autograd/autograd.html#mxnet.autograd.record">[docs]</a><span class="k">def</span> <span class="nf">record</span><span class="p">(</span><span class="n">train_mode</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span> <span class="c1">#pylint: disable=redefined-outer-name</span> |
| <span class="sd">"""Returns an autograd recording scope context to be used in 'with' statement</span> |
| <span class="sd"> and captures code that needs gradients to be calculated.</span> |
| |
| <span class="sd"> .. note:: When forwarding with train_mode=False, the corresponding backward</span> |
| <span class="sd"> should also use train_mode=False, otherwise gradient is undefined.</span> |
| |
| <span class="sd"> Example::</span> |
| |
| <span class="sd"> with autograd.record():</span> |
| <span class="sd"> y = model(x)</span> |
| <span class="sd"> backward([y])</span> |
| <span class="sd"> metric.update(...)</span> |
| <span class="sd"> optim.step(...)</span> |
| |
| <span class="sd"> Parameters</span> |
| <span class="sd"> ----------</span> |
| <span class="sd"> train_mode: bool, default True</span> |
| <span class="sd"> Whether the forward pass is in training or predicting mode. This controls the behavior</span> |
| <span class="sd"> of some layers such as Dropout, BatchNorm.</span> |
| <span class="sd"> """</span> |
| <span class="k">return</span> <span class="n">_RecordingStateScope</span><span class="p">(</span><span class="kc">True</span><span class="p">,</span> <span class="n">train_mode</span><span class="p">)</span></div> |
| |
| |
| <div class="viewcode-block" id="pause"><a class="viewcode-back" href="../../api/python/autograd/autograd.html#mxnet.autograd.pause">[docs]</a><span class="k">def</span> <span class="nf">pause</span><span class="p">(</span><span class="n">train_mode</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span> <span class="c1">#pylint: disable=redefined-outer-name</span> |
| <span class="sd">"""Returns a scope context to be used in 'with' statement for codes that do not need</span> |
| <span class="sd"> gradients to be calculated.</span> |
| |
| <span class="sd"> Example::</span> |
| |
| <span class="sd"> with autograd.record():</span> |
| <span class="sd"> y = model(x)</span> |
| <span class="sd"> backward([y])</span> |
| <span class="sd"> with autograd.pause():</span> |
| <span class="sd"> # testing, IO, gradient updates...</span> |
| |
| <span class="sd"> Parameters</span> |
| <span class="sd"> ----------</span> |
| <span class="sd"> train_mode: bool, default False</span> |
| <span class="sd"> Whether to do forward for training or predicting.</span> |
| <span class="sd"> """</span> |
| <span class="k">return</span> <span class="n">_RecordingStateScope</span><span class="p">(</span><span class="kc">False</span><span class="p">,</span> <span class="n">train_mode</span><span class="p">)</span></div> |
| |
| |
| <div class="viewcode-block" id="train_mode"><a class="viewcode-back" href="../../api/python/autograd/autograd.html#mxnet.autograd.train_mode">[docs]</a><span class="k">def</span> <span class="nf">train_mode</span><span class="p">():</span> |
| <span class="sd">"""Returns a scope context to be used in 'with' statement</span> |
| <span class="sd"> in which forward pass behavior is set to training mode,</span> |
| <span class="sd"> without changing the recording states.</span> |
| |
| <span class="sd"> Example::</span> |
| |
| <span class="sd"> y = model(x)</span> |
| <span class="sd"> with autograd.train_mode():</span> |
| <span class="sd"> y = dropout(y)</span> |
| |
| <span class="sd"> """</span> |
| <span class="k">return</span> <span class="n">_RecordingStateScope</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">True</span><span class="p">)</span></div> |
| |
| |
| <div class="viewcode-block" id="predict_mode"><a class="viewcode-back" href="../../api/python/autograd/autograd.html#mxnet.autograd.predict_mode">[docs]</a><span class="k">def</span> <span class="nf">predict_mode</span><span class="p">():</span> |
| <span class="sd">"""Returns a scope context to be used in 'with' statement</span> |
| <span class="sd"> in which forward pass behavior is set to inference mode,</span> |
| <span class="sd"> without changing the recording states.</span> |
| |
| <span class="sd"> Example::</span> |
| |
| <span class="sd"> with autograd.record():</span> |
| <span class="sd"> y = model(x)</span> |
| <span class="sd"> with autograd.predict_mode():</span> |
| <span class="sd"> y = sampling(y)</span> |
| <span class="sd"> backward([y])</span> |
| <span class="sd"> """</span> |
| <span class="k">return</span> <span class="n">_RecordingStateScope</span><span class="p">(</span><span class="kc">None</span><span class="p">,</span> <span class="kc">False</span><span class="p">)</span></div> |
| |
| |
| <div class="viewcode-block" id="mark_variables"><a class="viewcode-back" href="../../api/python/autograd/autograd.html#mxnet.autograd.mark_variables">[docs]</a><span class="k">def</span> <span class="nf">mark_variables</span><span class="p">(</span><span class="n">variables</span><span class="p">,</span> <span class="n">gradients</span><span class="p">,</span> <span class="n">grad_reqs</span><span class="o">=</span><span class="s1">'write'</span><span class="p">):</span> |
| <span class="sd">"""Mark NDArrays as variables to compute gradient for autograd.</span> |
| |
| <span class="sd"> Parameters</span> |
| <span class="sd"> ----------</span> |
| <span class="sd"> variables: NDArray or list of NDArray</span> |
| <span class="sd"> gradients: NDArray or list of NDArray</span> |
| <span class="sd"> grad_reqs: str or list of str</span> |
| <span class="sd"> """</span> |
| <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">variables</span><span class="p">,</span> <span class="n">NDArray</span><span class="p">):</span> |
| <span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">gradients</span><span class="p">,</span> <span class="n">NDArray</span><span class="p">)</span> |
| <span class="n">variables</span> <span class="o">=</span> <span class="p">[</span><span class="n">variables</span><span class="p">]</span> |
| <span class="n">gradients</span> <span class="o">=</span> <span class="p">[</span><span class="n">gradients</span><span class="p">]</span> |
| |
| <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">grad_reqs</span><span class="p">,</span> <span class="n">string_types</span><span class="p">):</span> |
| <span class="n">grad_reqs</span> <span class="o">=</span> <span class="p">[</span><span class="n">_GRAD_REQ_MAP</span><span class="p">[</span><span class="n">grad_reqs</span><span class="p">]]</span><span class="o">*</span><span class="nb">len</span><span class="p">(</span><span class="n">variables</span><span class="p">)</span> |
| <span class="k">else</span><span class="p">:</span> |
| <span class="n">grad_reqs</span> <span class="o">=</span> <span class="p">[</span><span class="n">_GRAD_REQ_MAP</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">grad_reqs</span><span class="p">]</span> |
| |
| <span class="n">check_call</span><span class="p">(</span><span class="n">_LIB</span><span class="o">.</span><span class="n">MXAutogradMarkVariables</span><span class="p">(</span> |
| <span class="nb">len</span><span class="p">(</span><span class="n">variables</span><span class="p">),</span> |
| <span class="n">c_handle_array</span><span class="p">(</span><span class="n">variables</span><span class="p">),</span> |
| <span class="n">c_array_buf</span><span class="p">(</span><span class="n">mx_uint</span><span class="p">,</span> <span class="n">array</span><span class="p">(</span><span class="s1">'I'</span><span class="p">,</span> <span class="n">grad_reqs</span><span class="p">)),</span> |
| <span class="n">c_handle_array</span><span class="p">(</span><span class="n">gradients</span><span class="p">)))</span></div> |
| |
| |
| <span class="k">def</span> <span class="nf">_parse_head</span><span class="p">(</span><span class="n">heads</span><span class="p">,</span> <span class="n">head_grads</span><span class="p">):</span> |
| <span class="sd">"""parse head gradient for backward and grad."""</span> |
| <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">heads</span><span class="p">,</span> <span class="n">NDArray</span><span class="p">):</span> |
| <span class="n">heads</span> <span class="o">=</span> <span class="p">[</span><span class="n">heads</span><span class="p">]</span> |
| <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">head_grads</span><span class="p">,</span> <span class="n">NDArray</span><span class="p">):</span> |
| <span class="n">head_grads</span> <span class="o">=</span> <span class="p">[</span><span class="n">head_grads</span><span class="p">]</span> |
| |
| <span class="n">head_handles</span> <span class="o">=</span> <span class="n">c_handle_array</span><span class="p">(</span><span class="n">heads</span><span class="p">)</span> |
| |
| <span class="k">if</span> <span class="n">head_grads</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span> |
| <span class="n">hgrad_handles</span> <span class="o">=</span> <span class="n">ctypes</span><span class="o">.</span><span class="n">c_void_p</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> |
| <span class="k">else</span><span class="p">:</span> |
| <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">heads</span><span class="p">)</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="n">head_grads</span><span class="p">),</span> \ |
| <span class="s2">"heads and head_grads must be lists of the same length"</span> |
| <span class="n">hgrad_handles</span> <span class="o">=</span> <span class="n">c_array</span><span class="p">(</span><span class="n">NDArrayHandle</span><span class="p">,</span> |
| <span class="p">[</span><span class="n">i</span><span class="o">.</span><span class="n">handle</span> <span class="k">if</span> <span class="n">i</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">NDArrayHandle</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> |
| <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">head_grads</span><span class="p">])</span> |
| <span class="k">return</span> <span class="n">head_handles</span><span class="p">,</span> <span class="n">hgrad_handles</span> |
| |
| |
| <div class="viewcode-block" id="backward"><a class="viewcode-back" href="../../api/python/autograd/autograd.html#mxnet.autograd.backward">[docs]</a><span class="k">def</span> <span class="nf">backward</span><span class="p">(</span><span class="n">heads</span><span class="p">,</span> <span class="n">head_grads</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">retain_graph</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> <span class="n">train_mode</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span> <span class="c1">#pylint: disable=redefined-outer-name</span> |
| <span class="sd">"""Compute the gradients of heads w.r.t previously marked variables.</span> |
| |
| <span class="sd"> Parameters</span> |
| <span class="sd"> ----------</span> |
| <span class="sd"> heads: NDArray or list of NDArray</span> |
| <span class="sd"> Output NDArray(s)</span> |
| <span class="sd"> head_grads: NDArray or list of NDArray or None</span> |
| <span class="sd"> Gradients with respect to heads.</span> |
| <span class="sd"> train_mode: bool, optional</span> |
| <span class="sd"> Whether to do backward for training or predicting.</span> |
| <span class="sd"> """</span> |
| <span class="n">head_handles</span><span class="p">,</span> <span class="n">hgrad_handles</span> <span class="o">=</span> <span class="n">_parse_head</span><span class="p">(</span><span class="n">heads</span><span class="p">,</span> <span class="n">head_grads</span><span class="p">)</span> |
| |
| <span class="n">check_call</span><span class="p">(</span><span class="n">_LIB</span><span class="o">.</span><span class="n">MXAutogradBackwardEx</span><span class="p">(</span> |
| <span class="nb">len</span><span class="p">(</span><span class="n">head_handles</span><span class="p">),</span> |
| <span class="n">head_handles</span><span class="p">,</span> |
| <span class="n">hgrad_handles</span><span class="p">,</span> |
| <span class="mi">0</span><span class="p">,</span> |
| <span class="n">ctypes</span><span class="o">.</span><span class="n">c_void_p</span><span class="p">(</span><span class="mi">0</span><span class="p">),</span> |
| <span class="n">ctypes</span><span class="o">.</span><span class="n">c_int</span><span class="p">(</span><span class="n">retain_graph</span><span class="p">),</span> |
| <span class="n">ctypes</span><span class="o">.</span><span class="n">c_int</span><span class="p">(</span><span class="mi">0</span><span class="p">),</span> |
| <span class="n">ctypes</span><span class="o">.</span><span class="n">c_int</span><span class="p">(</span><span class="n">train_mode</span><span class="p">),</span> |
| <span class="n">ctypes</span><span class="o">.</span><span class="n">c_void_p</span><span class="p">(</span><span class="mi">0</span><span class="p">),</span> |
| <span class="n">ctypes</span><span class="o">.</span><span class="n">c_void_p</span><span class="p">(</span><span class="mi">0</span><span class="p">)))</span></div> |
| |
| |
| <div class="viewcode-block" id="grad"><a class="viewcode-back" href="../../api/python/autograd/autograd.html#mxnet.autograd.grad">[docs]</a><span class="k">def</span> <span class="nf">grad</span><span class="p">(</span><span class="n">heads</span><span class="p">,</span> <span class="n">variables</span><span class="p">,</span> <span class="n">head_grads</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">retain_graph</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">create_graph</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span> |
| <span class="n">train_mode</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span> <span class="c1">#pylint: disable=redefined-outer-name</span> |
| <span class="sd">"""Compute the gradients of heads w.r.t variables. Gradients will be</span> |
| <span class="sd"> returned as new NDArrays instead of stored into `variable.grad`.</span> |
| <span class="sd"> Supports recording gradient graph for computing higher order gradients.</span> |
| |
| <span class="sd"> .. Note: Currently only a very limited set of operators support higher order</span> |
| <span class="sd"> gradients.</span> |
| |
| <span class="sd"> Parameters</span> |
| <span class="sd"> ----------</span> |
| <span class="sd"> heads: NDArray or list of NDArray</span> |
| <span class="sd"> Output NDArray(s)</span> |
| <span class="sd"> variables: NDArray or list of NDArray</span> |
| <span class="sd"> Input variables to compute gradients for.</span> |
| <span class="sd"> head_grads: NDArray or list of NDArray or None</span> |
| <span class="sd"> Gradients with respect to heads.</span> |
| <span class="sd"> retain_graph: bool</span> |
| <span class="sd"> Whether to keep computation graph to differentiate again, instead</span> |
| <span class="sd"> of clearing history and release memory. Defaults to the same value</span> |
| <span class="sd"> as create_graph.</span> |
| <span class="sd"> create_graph: bool</span> |
| <span class="sd"> Whether to record gradient graph for computing higher order</span> |
| <span class="sd"> train_mode: bool, optional</span> |
| <span class="sd"> Whether to do backward for training or prediction.</span> |
| |
| <span class="sd"> Returns</span> |
| <span class="sd"> -------</span> |
| <span class="sd"> NDArray or list of NDArray:</span> |
| <span class="sd"> Gradients with respect to variables.</span> |
| |
| <span class="sd"> Examples</span> |
| <span class="sd"> --------</span> |
| <span class="sd"> >>> x = mx.nd.ones((1,))</span> |
| <span class="sd"> >>> x.attach_grad()</span> |
| <span class="sd"> >>> with mx.autograd.record():</span> |
| <span class="sd"> ... z = mx.nd.elemwise_add(mx.nd.exp(x), x)</span> |
| <span class="sd"> >>> dx = mx.autograd.grad(z, [x], create_graph=True)</span> |
| <span class="sd"> >>> dx.backward()</span> |
| <span class="sd"> >>> print(dx.grad)</span> |
| <span class="sd"> [</span> |
| <span class="sd"> [ 3.71828175]</span> |
| <span class="sd"> <NDArray 1 @cpu(0)>]</span> |
| <span class="sd"> """</span> |
| <span class="n">head_handles</span><span class="p">,</span> <span class="n">hgrad_handles</span> <span class="o">=</span> <span class="n">_parse_head</span><span class="p">(</span><span class="n">heads</span><span class="p">,</span> <span class="n">head_grads</span><span class="p">)</span> |
| |
| <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">variables</span><span class="p">,</span> <span class="n">NDArray</span><span class="p">):</span> |
| <span class="n">variables</span> <span class="o">=</span> <span class="p">[</span><span class="n">variables</span><span class="p">]</span> |
| <span class="k">else</span><span class="p">:</span> |
| <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">variables</span><span class="p">),</span> <span class="s2">"variables cannot be an empty list."</span> |
| <span class="n">var_handles</span> <span class="o">=</span> <span class="n">c_handle_array</span><span class="p">(</span><span class="n">variables</span><span class="p">)</span> |
| |
| <span class="n">retain_graph</span> <span class="o">=</span> <span class="n">retain_graph</span> <span class="k">if</span> <span class="n">retain_graph</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">create_graph</span> |
| <span class="n">grad_vars</span> <span class="o">=</span> <span class="n">ctypes</span><span class="o">.</span><span class="n">POINTER</span><span class="p">(</span><span class="n">NDArrayHandle</span><span class="p">)()</span> |
| <span class="n">grad_stypes</span> <span class="o">=</span> <span class="n">ctypes</span><span class="o">.</span><span class="n">POINTER</span><span class="p">(</span><span class="n">ctypes</span><span class="o">.</span><span class="n">c_int</span><span class="p">)()</span> |
| |
| <span class="n">check_call</span><span class="p">(</span><span class="n">_LIB</span><span class="o">.</span><span class="n">MXAutogradBackwardEx</span><span class="p">(</span> |
| <span class="nb">len</span><span class="p">(</span><span class="n">head_handles</span><span class="p">),</span> |
| <span class="n">head_handles</span><span class="p">,</span> |
| <span class="n">hgrad_handles</span><span class="p">,</span> |
| <span class="nb">len</span><span class="p">(</span><span class="n">var_handles</span><span class="p">),</span> |
| <span class="n">var_handles</span><span class="p">,</span> |
| <span class="n">ctypes</span><span class="o">.</span><span class="n">c_int</span><span class="p">(</span><span class="n">retain_graph</span><span class="p">),</span> |
| <span class="n">ctypes</span><span class="o">.</span><span class="n">c_int</span><span class="p">(</span><span class="n">create_graph</span><span class="p">),</span> |
| <span class="n">ctypes</span><span class="o">.</span><span class="n">c_int</span><span class="p">(</span><span class="n">train_mode</span><span class="p">),</span> |
| <span class="n">ctypes</span><span class="o">.</span><span class="n">byref</span><span class="p">(</span><span class="n">grad_vars</span><span class="p">),</span> |
| <span class="n">ctypes</span><span class="o">.</span><span class="n">byref</span><span class="p">(</span><span class="n">grad_stypes</span><span class="p">)))</span> |
| |
| <span class="n">ret</span> <span class="o">=</span> <span class="p">[</span><span class="n">_ndarray_cls</span><span class="p">(</span><span class="n">ctypes</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">grad_vars</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="n">NDArrayHandle</span><span class="p">),</span> |
| <span class="n">stype</span><span class="o">=</span><span class="n">grad_stypes</span><span class="p">[</span><span class="n">i</span><span class="p">])</span> |
| <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">var_handles</span><span class="p">))]</span> |
| <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">variables</span><span class="p">,</span> <span class="n">NDArray</span><span class="p">):</span> |
| <span class="k">return</span> <span class="n">ret</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> |
| <span class="k">return</span> <span class="n">ret</span></div> |
| |
| |
| <div class="viewcode-block" id="get_symbol"><a class="viewcode-back" href="../../api/python/autograd/autograd.html#mxnet.autograd.get_symbol">[docs]</a><span class="k">def</span> <span class="nf">get_symbol</span><span class="p">(</span><span class="n">x</span><span class="p">):</span> |
| <span class="sd">"""Retrieve recorded computation history as `Symbol`.</span> |
| |
| <span class="sd"> Parameters</span> |
| <span class="sd"> ----------</span> |
| <span class="sd"> x : NDArray</span> |
| <span class="sd"> Array representing the head of computation graph.</span> |
| |
| <span class="sd"> Returns</span> |
| <span class="sd"> -------</span> |
| <span class="sd"> Symbol</span> |
| <span class="sd"> The retrieved Symbol.</span> |
| <span class="sd"> """</span> |
| <span class="n">hdl</span> <span class="o">=</span> <span class="n">SymbolHandle</span><span class="p">()</span> |
| <span class="n">check_call</span><span class="p">(</span><span class="n">_LIB</span><span class="o">.</span><span class="n">MXAutogradGetSymbol</span><span class="p">(</span><span class="n">x</span><span class="o">.</span><span class="n">handle</span><span class="p">,</span> <span class="n">ctypes</span><span class="o">.</span><span class="n">byref</span><span class="p">(</span><span class="n">hdl</span><span class="p">)))</span> |
| <span class="k">return</span> <span class="n">Symbol</span><span class="p">(</span><span class="n">hdl</span><span class="p">)</span></div> |
| |
| |
| <div class="viewcode-block" id="Function"><a class="viewcode-back" href="../../api/python/autograd/autograd.html#mxnet.autograd.Function">[docs]</a><span class="k">class</span> <span class="nc">Function</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span> |
| <span class="sd">"""Customize differentiation in autograd.</span> |
| |
| <span class="sd"> If you don't want to use the gradients computed by the default</span> |
| <span class="sd"> chain-rule, you can use Function to customize differentiation for</span> |
| <span class="sd"> computation. You define your computation in</span> |
| <span class="sd"> the forward method and provide the customized differentiation</span> |
| <span class="sd"> in the backward method. During gradient computation, autograd will</span> |
| <span class="sd"> use the user-defined backward function instead of the default chain-rule.</span> |
| <span class="sd"> You can also cast to numpy array and back for some operations in</span> |
| <span class="sd"> forward and backward.</span> |
| |
| <span class="sd"> For example, a stable sigmoid function can be defined as::</span> |
| |
| <span class="sd"> class sigmoid(mx.autograd.Function):</span> |
| <span class="sd"> def forward(self, x):</span> |
| <span class="sd"> y = 1 / (1 + mx.nd.exp(-x))</span> |
| <span class="sd"> self.save_for_backward(y)</span> |
| <span class="sd"> return y</span> |
| |
| <span class="sd"> def backward(self, dy):</span> |
| <span class="sd"> # backward takes as many inputs as forward's return value,</span> |
| <span class="sd"> # and returns as many NDArrays as forward's arguments.</span> |
| <span class="sd"> y, = self.saved_tensors</span> |
| <span class="sd"> return dy * y * (1-y)</span> |
| |
| <span class="sd"> Then, the function can be used in the following way::</span> |
| |
| <span class="sd"> func = sigmoid()</span> |
| <span class="sd"> x = mx.nd.random.uniform(shape=(10,))</span> |
| <span class="sd"> x.attach_grad()</span> |
| |
| <span class="sd"> with mx.autograd.record():</span> |
| <span class="sd"> m = func(x)</span> |
| <span class="sd"> m.backward()</span> |
| <span class="sd"> dx = x.grad.asnumpy()</span> |
| |
| <span class="sd"> """</span> |
| <span class="n">_bwd_functype</span> <span class="o">=</span> <span class="n">CFUNCTYPE</span><span class="p">(</span><span class="n">c_int</span><span class="p">,</span> <span class="n">c_int</span><span class="p">,</span> <span class="n">c_int</span><span class="p">,</span> <span class="n">POINTER</span><span class="p">(</span><span class="n">c_void_p</span><span class="p">),</span> |
| <span class="n">POINTER</span><span class="p">(</span><span class="n">c_int</span><span class="p">),</span> <span class="n">c_int</span><span class="p">,</span> <span class="n">c_void_p</span><span class="p">)</span> |
| <span class="n">_del_functype</span> <span class="o">=</span> <span class="n">CFUNCTYPE</span><span class="p">(</span><span class="n">c_int</span><span class="p">,</span> <span class="n">c_void_p</span><span class="p">)</span> |
| <span class="k">class</span> <span class="nc">_Registry</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span> |
| <span class="sd">"""CustomOp registry."""</span> |
| <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">ref_holder</span> <span class="o">=</span> <span class="p">{}</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">=</span> <span class="mi">0</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">lock</span> <span class="o">=</span> <span class="n">Lock</span><span class="p">()</span> |
| |
| <span class="k">def</span> <span class="nf">inc</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> |
| <span class="sd">"""Get index for new entry."""</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">lock</span><span class="o">.</span><span class="n">acquire</span><span class="p">()</span> |
| <span class="n">cur</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">+=</span> <span class="mi">1</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">lock</span><span class="o">.</span><span class="n">release</span><span class="p">()</span> |
| <span class="k">return</span> <span class="n">cur</span> |
| |
| <span class="n">_registry</span> <span class="o">=</span> <span class="n">_Registry</span><span class="p">()</span> |
| |
| <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">_used</span> <span class="o">=</span> <span class="kc">False</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">saved_tensors</span> <span class="o">=</span> <span class="p">()</span> |
| |
| <span class="k">def</span> <span class="nf">save_for_backward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">):</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">saved_tensors</span> <span class="o">=</span> <span class="n">args</span> |
| |
| <span class="k">def</span> <span class="nf">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">inputs</span><span class="p">):</span> |
| <span class="k">assert</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">_used</span><span class="p">,</span> \ |
| <span class="s2">"Each Function instance can only be called once. "</span>\ |
| <span class="s2">"Please create another instance."</span> |
| <span class="bp">self</span><span class="o">.</span><span class="n">_used</span> <span class="o">=</span> <span class="kc">True</span> |
| |
| <span class="n">prev_recording</span> <span class="o">=</span> <span class="n">set_recording</span><span class="p">(</span><span class="kc">False</span><span class="p">)</span> |
| <span class="n">outputs</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">forward</span><span class="p">(</span><span class="o">*</span><span class="n">inputs</span><span class="p">)</span> |
| <span class="n">set_recording</span><span class="p">(</span><span class="n">prev_recording</span><span class="p">)</span> |
| |
| <span class="k">if</span> <span class="ow">not</span> <span class="n">prev_recording</span><span class="p">:</span> |
| <span class="k">return</span> <span class="n">outputs</span> |
| |
| <span class="n">ret_outputs</span> <span class="o">=</span> <span class="n">outputs</span> |
| <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">outputs</span><span class="p">,</span> <span class="n">NDArray</span><span class="p">):</span> |
| <span class="n">outputs</span> <span class="o">=</span> <span class="p">(</span><span class="n">outputs</span><span class="p">,)</span> |
| |
| <span class="n">key</span> <span class="o">=</span> <span class="n">Function</span><span class="o">.</span><span class="n">_registry</span><span class="o">.</span><span class="n">inc</span><span class="p">()</span> |
| |
| <span class="k">def</span> <span class="nf">backward_entry</span><span class="p">(</span><span class="n">num_ograds</span><span class="p">,</span> <span class="n">num_igrads</span><span class="p">,</span> <span class="n">ptrs</span><span class="p">,</span> <span class="n">reqs</span><span class="p">,</span> <span class="n">is_train</span><span class="p">,</span> <span class="n">_</span><span class="p">):</span> |
| <span class="sd">"""entry point for backward."""</span> |
| <span class="c1"># pylint: disable=W0613</span> |
| <span class="k">try</span><span class="p">:</span> |
| <span class="n">output_grads</span> <span class="o">=</span> <span class="p">[</span><span class="n">NDArray</span><span class="p">(</span><span class="n">ctypes</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">NDArrayHandle</span><span class="p">),</span> <span class="n">writable</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span> \ |
| <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">ptrs</span><span class="p">[:</span><span class="n">num_ograds</span><span class="p">]]</span> |
| <span class="n">input_grads</span> <span class="o">=</span> <span class="p">[</span><span class="n">NDArray</span><span class="p">(</span><span class="n">ctypes</span><span class="o">.</span><span class="n">cast</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">NDArrayHandle</span><span class="p">),</span> <span class="n">writable</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> \ |
| <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">ptrs</span><span class="p">[</span><span class="n">num_ograds</span><span class="p">:</span><span class="n">num_ograds</span><span class="o">+</span><span class="n">num_igrads</span><span class="p">]]</span> |
| <span class="n">reqs</span> <span class="o">=</span> <span class="p">[</span><span class="n">reqs</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">num_igrads</span><span class="p">)]</span> |
| <span class="n">rets</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">backward</span><span class="p">(</span><span class="o">*</span><span class="n">output_grads</span><span class="p">)</span> |
| <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">rets</span><span class="p">,</span> <span class="n">NDArray</span><span class="p">):</span> |
| <span class="n">rets</span> <span class="o">=</span> <span class="p">(</span><span class="n">rets</span><span class="p">,)</span> |
| <span class="k">assert</span> <span class="nb">len</span><span class="p">(</span><span class="n">rets</span><span class="p">)</span> <span class="o">==</span> <span class="nb">len</span><span class="p">(</span><span class="n">input_grads</span><span class="p">),</span> \ |
| <span class="s2">"</span><span class="si">%s</span><span class="s2">.backward must return exactly the same number "</span> \ |
| <span class="s2">"of NDArrays as the number of NDArrays arguments to forward."</span> \ |
| <span class="s2">"Expecting </span><span class="si">%d</span><span class="s2"> got </span><span class="si">%d</span><span class="s2">"</span><span class="o">%</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="vm">__class__</span><span class="o">.</span><span class="n">name</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">input_grads</span><span class="p">),</span> <span class="nb">len</span><span class="p">(</span><span class="n">rets</span><span class="p">))</span> |
| <span class="k">for</span> <span class="n">igrad</span><span class="p">,</span> <span class="n">ret</span><span class="p">,</span> <span class="n">req</span> <span class="ow">in</span> <span class="nb">zip</span><span class="p">(</span><span class="n">input_grads</span><span class="p">,</span> <span class="n">rets</span><span class="p">,</span> <span class="n">reqs</span><span class="p">):</span> |
| <span class="k">assert</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ret</span><span class="p">,</span> <span class="n">NDArray</span><span class="p">),</span> \ |
| <span class="s2">"autograd.Function.backward must return NDArrays, not </span><span class="si">%s</span><span class="s2">"</span><span class="o">%</span><span class="nb">type</span><span class="p">(</span><span class="n">ret</span><span class="p">)</span> |
| <span class="k">if</span> <span class="n">req</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span> <span class="c1"># null</span> |
| <span class="k">return</span> <span class="kc">True</span> |
| <span class="k">elif</span> <span class="n">req</span> <span class="o">==</span> <span class="mi">1</span> <span class="ow">or</span> <span class="n">req</span> <span class="o">==</span> <span class="mi">2</span><span class="p">:</span> <span class="c1"># write or inplace</span> |
| <span class="n">igrad</span><span class="p">[:]</span> <span class="o">=</span> <span class="n">ret</span> |
| <span class="k">elif</span> <span class="n">req</span> <span class="o">==</span> <span class="s1">'add'</span><span class="p">:</span> |
| <span class="n">igrad</span><span class="p">[:]</span> <span class="o">+=</span> <span class="n">ret</span> |
| <span class="k">except</span> <span class="ne">Exception</span><span class="p">:</span> <span class="c1"># pylint: disable=broad-except</span> |
| <span class="nb">print</span><span class="p">(</span><span class="s1">'Error in Function.backward: </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="n">traceback</span><span class="o">.</span><span class="n">format_exc</span><span class="p">())</span> |
| <span class="k">return</span> <span class="kc">False</span> |
| <span class="k">return</span> <span class="kc">True</span> |
| |
| <span class="k">def</span> <span class="nf">delete_entry</span><span class="p">(</span><span class="n">_</span><span class="p">):</span> |
| <span class="sd">"""C Callback for CustomFunction::delete"""</span> |
| <span class="k">try</span><span class="p">:</span> |
| <span class="k">del</span> <span class="n">Function</span><span class="o">.</span><span class="n">_registry</span><span class="o">.</span><span class="n">ref_holder</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> |
| <span class="k">except</span> <span class="ne">Exception</span><span class="p">:</span> <span class="c1"># pylint: disable=broad-except</span> |
| <span class="nb">print</span><span class="p">(</span><span class="s1">'Error in autograd.Function.delete: </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="n">traceback</span><span class="o">.</span><span class="n">format_exc</span><span class="p">())</span> |
| <span class="k">return</span> <span class="kc">False</span> |
| <span class="k">return</span> <span class="kc">True</span> |
| |
| <span class="n">callbacks</span> <span class="o">=</span> <span class="p">[</span><span class="n">Function</span><span class="o">.</span><span class="n">_bwd_functype</span><span class="p">(</span><span class="n">backward_entry</span><span class="p">),</span> |
| <span class="n">Function</span><span class="o">.</span><span class="n">_del_functype</span><span class="p">(</span><span class="n">delete_entry</span><span class="p">)]</span> |
| <span class="n">callbacks</span> <span class="o">=</span> <span class="p">[</span><span class="n">cast</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">CFUNCTYPE</span><span class="p">(</span><span class="n">c_int</span><span class="p">))</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">callbacks</span><span class="p">]</span> |
| <span class="n">context</span> <span class="o">=</span> <span class="n">MXCallbackList</span><span class="p">(</span><span class="n">c_int</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">callbacks</span><span class="p">)),</span> |
| <span class="n">cast</span><span class="p">(</span><span class="n">c_array</span><span class="p">(</span><span class="n">CFUNCTYPE</span><span class="p">(</span><span class="n">c_int</span><span class="p">),</span> <span class="n">callbacks</span><span class="p">),</span> |
| <span class="n">POINTER</span><span class="p">(</span><span class="n">CFUNCTYPE</span><span class="p">(</span><span class="n">c_int</span><span class="p">))),</span> |
| <span class="n">cast</span><span class="p">(</span><span class="n">c_array</span><span class="p">(</span><span class="n">c_void_p</span><span class="p">,</span> <span class="p">[</span><span class="kc">None</span><span class="p">]</span><span class="o">*</span><span class="nb">len</span><span class="p">(</span><span class="n">callbacks</span><span class="p">)),</span> |
| <span class="n">POINTER</span><span class="p">(</span><span class="n">c_void_p</span><span class="p">)))</span> |
| <span class="n">check_call</span><span class="p">(</span><span class="n">_LIB</span><span class="o">.</span><span class="n">MXCustomFunctionRecord</span><span class="p">(</span> |
| <span class="n">c_int</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">inputs</span><span class="p">)),</span> |
| <span class="n">c_handle_array</span><span class="p">(</span><span class="n">inputs</span><span class="p">),</span> |
| <span class="n">c_int</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">outputs</span><span class="p">)),</span> |
| <span class="n">c_handle_array</span><span class="p">(</span><span class="n">outputs</span><span class="p">),</span> |
| <span class="n">ctypes</span><span class="o">.</span><span class="n">byref</span><span class="p">(</span><span class="n">context</span><span class="p">)))</span> |
| |
| <span class="n">Function</span><span class="o">.</span><span class="n">_registry</span><span class="o">.</span><span class="n">ref_holder</span><span class="p">[</span><span class="n">key</span><span class="p">]</span> <span class="o">=</span> <span class="n">context</span> |
| |
| <span class="k">return</span> <span class="n">ret_outputs</span> |
| |
| <div class="viewcode-block" id="Function.forward"><a class="viewcode-back" href="../../api/python/autograd/autograd.html#mxnet.autograd.Function.forward">[docs]</a> <span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">inputs</span><span class="p">):</span> |
| <span class="sd">"""Forward computation."""</span> |
| <span class="k">raise</span> <span class="ne">NotImplementedError</span></div> |
| |
| <div class="viewcode-block" id="Function.backward"><a class="viewcode-back" href="../../api/python/autograd/autograd.html#mxnet.autograd.Function.backward">[docs]</a> <span class="k">def</span> <span class="nf">backward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">output_grads</span><span class="p">):</span> |
| <span class="sd">"""Backward computation.</span> |
| |
| <span class="sd"> Takes as many inputs as forward's outputs,</span> |
| <span class="sd"> and returns as many NDArrays as forward's inputs.</span> |
| <span class="sd"> """</span> |
| <span class="k">raise</span> <span class="ne">NotImplementedError</span></div></div> |
| </pre></div> |
| </div> |
| </div> |
| <div aria-label="main navigation" class="sphinxsidebar rightsidebar" role="navigation"> |
| <div class="sphinxsidebarwrapper"> |
| </div> |
| </div> |
| </div><div class="footer"> |
| <div class="section-disclaimer"> |
| <div class="container"> |
| <div> |
| <img height="60" src="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/apache_incubator_logo.png"/> |
| <p> |
| Apache MXNet is an effort undergoing incubation at The Apache Software Foundation (ASF), <strong>sponsored by the <i>Apache Incubator</i></strong>. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. |
| </p> |
| <p> |
| "Copyright © 2017-2018, The Apache Software Foundation |
| Apache MXNet, MXNet, Apache, the Apache feather, and the Apache MXNet project logo are either registered trademarks or trademarks of the Apache Software Foundation." |
| </p> |
| </div> |
| </div> |
| </div> |
| </div> <!-- pagename != index --> |
| </div> |
| <script crossorigin="anonymous" integrity="sha384-0mSbJDEHialfmuBBQP6A4Qrprq5OVfW37PRR3j5ELqxss1yVqOtnepnHVP9aJ7xS" src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/js/bootstrap.min.js"></script> |
| <script src="../../_static/js/sidebar.js" type="text/javascript"></script> |
| <script src="../../_static/js/search.js" type="text/javascript"></script> |
| <script src="../../_static/js/navbar.js" type="text/javascript"></script> |
| <script src="../../_static/js/clipboard.min.js" type="text/javascript"></script> |
| <script src="../../_static/js/copycode.js" type="text/javascript"></script> |
| <script src="../../_static/js/page.js" type="text/javascript"></script> |
| <script src="../../_static/js/docversion.js" type="text/javascript"></script> |
| <script type="text/javascript"> |
| $('body').ready(function () { |
| $('body').css('visibility', 'visible'); |
| }); |
| </script> |
| </body> |
| </html> |
