Google Git
Sign in
apache / mxnet-test / refs/heads/asf-site / . / versions / master / tutorials / r / symbol.html
blob: 374566dc20017f2bfc5e34d16cf05d1f2e72a494 [file] [log] [blame]
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<meta content="IE=edge" http-equiv="X-UA-Compatible"/>
<meta content="width=device-width, initial-scale=1" name="viewport"/>
<title>Symbol and Automatic Differentiation — mxnet documentation</title>
<link crossorigin="anonymous" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css" integrity="sha384-1q8mTJOASx8j1Au+a5WDVnPi2lkFfwwEAa8hDDdjZlpLegxhjVME1fgjWPGmkzs7" rel="stylesheet"/>
<link href="https://maxcdn.bootstrapcdn.com/font-awesome/4.5.0/css/font-awesome.min.css" rel="stylesheet"/>
<link href="../../_static/basic.css" rel="stylesheet" type="text/css">
<link href="../../_static/pygments.css" rel="stylesheet" type="text/css">
<link href="../../_static/mxnet.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../../',
VERSION: '',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: ''
};
</script>
<script src="../../_static/jquery-1.11.1.js" type="text/javascript"></script>
<script src="../../_static/underscore.js" type="text/javascript"></script>
<script src="../../_static/searchtools_custom.js" type="text/javascript"></script>
<script src="../../_static/doctools.js" type="text/javascript"></script>
<script src="../../_static/selectlang.js" type="text/javascript"></script>
<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script>
<script type="text/javascript"> jQuery(function() { Search.loadIndex("/searchindex.js"); Search.init();}); </script>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new
Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-96378503-1', 'auto');
ga('send', 'pageview');
</script>
<!-- -->
<!-- <script type="text/javascript" src="../../_static/jquery.js"></script> -->
<!-- -->
<!-- <script type="text/javascript" src="../../_static/underscore.js"></script> -->
<!-- -->
<!-- <script type="text/javascript" src="../../_static/doctools.js"></script> -->
<!-- -->
<!-- <script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script> -->
<!-- -->
<link href="https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/image/mxnet-icon.png" rel="icon" type="image/png"/>
</link></link></head>
<body role="document"><!-- Previous Navbar Layout
<div class="navbar navbar-default navbar-fixed-top">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a href="../../" class="navbar-brand">
<img src="http://data.mxnet.io/theme/mxnet.png">
</a>
</div>
<div id="navbar" class="navbar-collapse collapse">
<ul id="navbar" class="navbar navbar-left">
<li> <a href="../../get_started/index.html">Get Started</a> </li>
<li> <a href="../../tutorials/index.html">Tutorials</a> </li>
<li> <a href="../../how_to/index.html">How To</a> </li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="true">Packages <span class="caret"></span></a>
<ul class="dropdown-menu">
<li><a href="../../packages/python/index.html">
Python
</a></li>
<li><a href="../../packages/r/index.html">
R
</a></li>
<li><a href="../../packages/julia/index.html">
Julia
</a></li>
<li><a href="../../packages/c++/index.html">
C++
</a></li>
<li><a href="../../packages/scala/index.html">
Scala
</a></li>
<li><a href="../../packages/perl/index.html">
Perl
</a></li>
</ul>
</li>
<li> <a href="../../system/index.html">System</a> </li>
<li>
<form class="" role="search" action="../../search.html" method="get" autocomplete="off">
<div class="form-group inner-addon left-addon">
<i class="glyphicon glyphicon-search"></i>
<input type="text" name="q" class="form-control" placeholder="Search">
</div>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form> </li>
</ul>
<ul id="navbar" class="navbar navbar-right">
<li> <a href="../../index.html"><span class="flag-icon flag-icon-us"></span></a> </li>
<li> <a href="../..//zh/index.html"><span class="flag-icon flag-icon-cn"></span></a> </li>
</ul>
</div>
</div>
</div>
Previous Navbar Layout End -->
<div class="navbar navbar-fixed-top">
<div class="container" id="navContainer">
<div class="innder" id="header-inner">
<h1 id="logo-wrap">
<a href="../../" id="logo"><img src="http://data.mxnet.io/theme/mxnet.png"/></a>
</h1>
<nav class="nav-bar" id="main-nav">
<a class="main-nav-link" href="../../get_started/install.html">Install</a>
<a class="main-nav-link" href="../../tutorials/index.html">Tutorials</a>
<a class="main-nav-link" href="../../how_to/index.html">How To</a>
<span id="dropdown-menu-position-anchor">
<a aria-expanded="true" aria-haspopup="true" class="main-nav-link dropdown-toggle" data-toggle="dropdown" href="#" role="button">API <span class="caret"></span></a>
<ul class="dropdown-menu" id="package-dropdown-menu">
<li><a class="main-nav-link" href="../../api/python/index.html">Python</a></li>
<li><a class="main-nav-link" href="../../api/scala/index.html">Scala</a></li>
<li><a class="main-nav-link" href="../../api/r/index.html">R</a></li>
<li><a class="main-nav-link" href="../../api/julia/index.html">Julia</a></li>
<li><a class="main-nav-link" href="../../api/c++/index.html">C++</a></li>
<li><a class="main-nav-link" href="../../api/perl/index.html">Perl</a></li>
</ul>
</span>
<a class="main-nav-link" href="../../architecture/index.html">Architecture</a>
<!-- <a class="main-nav-link" href="../../community/index.html">Community</a> -->
<a class="main-nav-link" href="https://github.com/dmlc/mxnet">Github</a>
<span id="dropdown-menu-position-anchor-version" style="position: relative"><a href="#" class="main-nav-link dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="true">Versions(master)<span class="caret"></span></a><ul id="package-dropdown-menu" class="dropdown-menu"><li><a class="main-nav-link" href=http://mxnet.incubator.apache.org/test/>v0.10.14</a></li><li><a class="main-nav-link" href=http://mxnet.incubator.apache.org/test/versions/0.10/index.html>0.10</a></li><li><a class="main-nav-link" href=http://mxnet.incubator.apache.org/test/versions/master/index.html>master</a></li></ul></span></nav>
<script> function getRootPath(){ return "../../" } </script>
<div class="burgerIcon dropdown">
<a class="dropdown-toggle" data-toggle="dropdown" href="#" role="button">☰</a>
<ul class="dropdown-menu dropdown-menu-right" id="burgerMenu">
<li><a href="../../get_started/install.html">Install</a></li>
<li><a href="../../tutorials/index.html">Tutorials</a></li>
<li><a href="../../how_to/index.html">How To</a></li>
<li class="dropdown-submenu">
<a href="#" tabindex="-1">API</a>
<ul class="dropdown-menu">
<li><a href="../../api/python/index.html" tabindex="-1">Python</a>
</li>
<li><a href="../../api/scala/index.html" tabindex="-1">Scala</a>
</li>
<li><a href="../../api/r/index.html" tabindex="-1">R</a>
</li>
<li><a href="../../api/julia/index.html" tabindex="-1">Julia</a>
</li>
<li><a href="../../api/c++/index.html" tabindex="-1">C++</a>
</li>
<li><a href="../../api/perl/index.html" tabindex="-1">Perl</a>
</li>
</ul>
</li>
<li><a href="../../architecture/index.html">Architecture</a></li>
<li><a class="main-nav-link" href="https://github.com/dmlc/mxnet">Github</a></li>
<li id="dropdown-menu-position-anchor-version-mobile" class="dropdown-submenu" style="position: relative"><a href="#" tabindex="-1">Versions(master)</a><ul class="dropdown-menu"><li><a tabindex="-1" href=http://mxnet.incubator.apache.org/test/>v0.10.14</a></li><li><a tabindex="-1" href=http://mxnet.incubator.apache.org/test/versions/0.10/index.html>0.10</a></li><li><a tabindex="-1" href=http://mxnet.incubator.apache.org/test/versions/master/index.html>master</a></li></ul></li></ul>
</div>
<div class="plusIcon dropdown">
<a class="dropdown-toggle" data-toggle="dropdown" href="#" role="button"><span aria-hidden="true" class="glyphicon glyphicon-plus"></span></a>
<ul class="dropdown-menu dropdown-menu-right" id="plusMenu"></ul>
</div>
<div id="search-input-wrap">
<form action="../../search.html" autocomplete="off" class="" method="get" role="search">
<div class="form-group inner-addon left-addon">
<i class="glyphicon glyphicon-search"></i>
<input class="form-control" name="q" placeholder="Search" type="text"/>
</div>
<input name="check_keywords" type="hidden" value="yes">
<input name="area" type="hidden" value="default"/>
</input></form>
<div id="search-preview"></div>
</div>
<div id="searchIcon">
<span aria-hidden="true" class="glyphicon glyphicon-search"></span>
</div>
<!-- <div id="lang-select-wrap"> -->
<!-- <label id="lang-select-label"> -->
<!-- <\!-- <i class="fa fa-globe"></i> -\-> -->
<!-- <span></span> -->
<!-- </label> -->
<!-- <select id="lang-select"> -->
<!-- <option value="en">Eng</option> -->
<!-- <option value="zh">中文</option> -->
<!-- </select> -->
<!-- </div> -->
<!-- <a id="mobile-nav-toggle">
<span class="mobile-nav-toggle-bar"></span>
<span class="mobile-nav-toggle-bar"></span>
<span class="mobile-nav-toggle-bar"></span>
</a> -->
</div>
</div>
</div>
<div class="container">
<div class="row">
<div aria-label="main navigation" class="sphinxsidebar leftsidebar" role="navigation">
<div class="sphinxsidebarwrapper">
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../api/python/index.html">Python Documents</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/r/index.html">R Documents</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/julia/index.html">Julia Documents</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/c++/index.html">C++ Documents</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/scala/index.html">Scala Documents</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/perl/index.html">Perl Documents</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../how_to/index.html">HowTo Documents</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../architecture/index.html">System Documents</a></li>
<li class="toctree-l1"><a class="reference internal" href="../index.html">Tutorials</a></li>
</ul>
</div>
</div>
<div class="content">
<div class="section" id="symbol-and-automatic-differentiation">
<span id="symbol-and-automatic-differentiation"></span><h1>Symbol and Automatic Differentiation<a class="headerlink" href="#symbol-and-automatic-differentiation" title="Permalink to this headline"></a></h1>
<p>The computational unit <code class="docutils literal"><span class="pre">NDArray</span></code> requires a way to construct neural networks. MXNet provides a symbolic interface, named Symbol, to do this. Symbol combines both flexibility and efficiency.</p>
<div class="section" id="basic-composition-of-symbols">
<span id="basic-composition-of-symbols"></span><h2>Basic Composition of Symbols<a class="headerlink" href="#basic-composition-of-symbols" title="Permalink to this headline"></a></h2>
<p>The following code creates a two-layer perceptron network:</p>
<div class="highlight-r"><div class="highlight"><pre><span></span><span class="kn">require</span><span class="p">(</span>mxnet<span class="p">)</span>
net <span class="o"><-</span> mx.symbol.Variable<span class="p">(</span><span class="s">"data"</span><span class="p">)</span>
net <span class="o"><-</span> mx.symbol.FullyConnected<span class="p">(</span>data<span class="o">=</span>net<span class="p">,</span> name<span class="o">=</span><span class="s">"fc1"</span><span class="p">,</span> num_hidden<span class="o">=</span><span class="m">128</span><span class="p">)</span>
net <span class="o"><-</span> mx.symbol.Activation<span class="p">(</span>data<span class="o">=</span>net<span class="p">,</span> name<span class="o">=</span><span class="s">"relu1"</span><span class="p">,</span> act_type<span class="o">=</span><span class="s">"relu"</span><span class="p">)</span>
net <span class="o"><-</span> mx.symbol.FullyConnected<span class="p">(</span>data<span class="o">=</span>net<span class="p">,</span> name<span class="o">=</span><span class="s">"fc2"</span><span class="p">,</span> num_hidden<span class="o">=</span><span class="m">64</span><span class="p">)</span>
net <span class="o"><-</span> mx.symbol.Softmax<span class="p">(</span>data<span class="o">=</span>net<span class="p">,</span> name<span class="o">=</span><span class="s">"out"</span><span class="p">)</span>
<span class="kp">class</span><span class="p">(</span>net<span class="p">)</span>
</pre></div>
</div>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="c1">## [1] "Rcpp_MXSymbol"</span>
<span class="c1">## attr(,"package")</span>
<span class="c1">## [1] "mxnet"</span>
</pre></div>
</div>
<p>Each symbol takes a (unique) string name. <em>Variable</em> often defines the inputs,
or free variables. Other symbols take a symbol as the input (<em>data</em>),
and may accept other hyper parameters, such as the number of hidden neurons (<em>num_hidden</em>)
or the activation type (<em>act_type</em>).</p>
<p>A symbol can be viewed as a function that takes several arguments, whose
names are automatically generated and can be retrieved with the following command:</p>
<div class="highlight-r"><div class="highlight"><pre><span></span>arguments<span class="p">(</span>net<span class="p">)</span>
</pre></div>
</div>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="c1">## [1] "data" "fc1_weight" "fc1_bias" "fc2_weight" "fc2_bias"</span>
<span class="c1">## [6] "out_label"</span>
</pre></div>
</div>
<p>The arguments are the parameters need by each symbol:</p>
<ul class="simple">
<li><em>data</em>: Input data needed by the variable <em>data</em></li>
<li><em>fc1_weight</em> and <em>fc1_bias</em>: The weight and bias for the first fully connected layer, <em>fc1</em></li>
<li><em>fc2_weight</em> and <em>fc2_bias</em>: The weight and bias for the second fully connected layer, <em>fc2</em></li>
<li><em>out_label</em>: The label needed by the loss</li>
</ul>
<p>We can also specify the automatically generated names explicitly:</p>
<div class="highlight-r"><div class="highlight"><pre><span></span>data <span class="o"><-</span> mx.symbol.Variable<span class="p">(</span><span class="s">"data"</span><span class="p">)</span>
w <span class="o"><-</span> mx.symbol.Variable<span class="p">(</span><span class="s">"myweight"</span><span class="p">)</span>
net <span class="o"><-</span> mx.symbol.FullyConnected<span class="p">(</span>data<span class="o">=</span>data<span class="p">,</span> weight<span class="o">=</span>w<span class="p">,</span> name<span class="o">=</span><span class="s">"fc1"</span><span class="p">,</span> num_hidden<span class="o">=</span><span class="m">128</span><span class="p">)</span>
arguments<span class="p">(</span>net<span class="p">)</span>
</pre></div>
</div>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="c1">## [1] "data" "myweight" "fc1_bias"</span>
</pre></div>
</div>
</div>
<div class="section" id="more-complicated-composition-of-symbols">
<span id="more-complicated-composition-of-symbols"></span><h2>More Complicated Composition of Symbols<a class="headerlink" href="#more-complicated-composition-of-symbols" title="Permalink to this headline"></a></h2>
<p>MXNet provides well-optimized symbols for
commonly used layers in deep learning. You can also define new operators
in Python. The following example first performs an element-wise add between two
symbols, then feeds them to the fully connected operator:</p>
<div class="highlight-r"><div class="highlight"><pre><span></span>lhs <span class="o"><-</span> mx.symbol.Variable<span class="p">(</span><span class="s">"data1"</span><span class="p">)</span>
rhs <span class="o"><-</span> mx.symbol.Variable<span class="p">(</span><span class="s">"data2"</span><span class="p">)</span>
net <span class="o"><-</span> mx.symbol.FullyConnected<span class="p">(</span>data<span class="o">=</span>lhs <span class="o">+</span> rhs<span class="p">,</span> name<span class="o">=</span><span class="s">"fc1"</span><span class="p">,</span> num_hidden<span class="o">=</span><span class="m">128</span><span class="p">)</span>
arguments<span class="p">(</span>net<span class="p">)</span>
</pre></div>
</div>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="c1">## [1] "data1" "data2" "fc1_weight" "fc1_bias"</span>
</pre></div>
</div>
<p>We can construct a symbol more flexibly than by using the single
forward composition, for example:</p>
<div class="highlight-r"><div class="highlight"><pre><span></span>net <span class="o"><-</span> mx.symbol.Variable<span class="p">(</span><span class="s">"data"</span><span class="p">)</span>
net <span class="o"><-</span> mx.symbol.FullyConnected<span class="p">(</span>data<span class="o">=</span>net<span class="p">,</span> name<span class="o">=</span><span class="s">"fc1"</span><span class="p">,</span> num_hidden<span class="o">=</span><span class="m">128</span><span class="p">)</span>
net2 <span class="o"><-</span> mx.symbol.Variable<span class="p">(</span><span class="s">"data2"</span><span class="p">)</span>
net2 <span class="o"><-</span> mx.symbol.FullyConnected<span class="p">(</span>data<span class="o">=</span>net2<span class="p">,</span> name<span class="o">=</span><span class="s">"net2"</span><span class="p">,</span> num_hidden<span class="o">=</span><span class="m">128</span><span class="p">)</span>
composed.net <span class="o"><-</span> mx.apply<span class="p">(</span>net<span class="p">,</span> data<span class="o">=</span>net2<span class="p">,</span> name<span class="o">=</span><span class="s">"compose"</span><span class="p">)</span>
arguments<span class="p">(</span>composed.net<span class="p">)</span>
</pre></div>
</div>
<div class="highlight-python"><div class="highlight"><pre><span></span><span class="c1">## [1] "data2" "net2_weight" "net2_bias" "fc1_weight" "fc1_bias"</span>
</pre></div>
</div>
<p>In the example, <em>net</em> is used as a function to apply to an existing symbol
<em>net</em>. The resulting <em>composed.net</em> will replace the original argument <em>data</em> with
<em>net2</em> instead.</p>
</div>
<div class="section" id="training-a-neural-net">
<span id="training-a-neural-net"></span><h2>Training a Neural Net<a class="headerlink" href="#training-a-neural-net" title="Permalink to this headline"></a></h2>
<p>The <a class="reference external" href="http://mxnet.io/../R-package/R/model.R">model API</a> is a thin wrapper around the symbolic executors to support neural net training.</p>
<p>We encourage you to read <a class="reference external" href="http://mxnet.io/tutorials/python/symbol_in_pictures.md">Symbolic Configuration and Execution in Pictures for python package</a>for a detailed explanation of concepts in pictures.</p>
</div>
<div class="section" id="how-efficient-is-the-symbolic-api">
<span id="how-efficient-is-the-symbolic-api"></span><h2>How Efficient Is the Symbolic API?<a class="headerlink" href="#how-efficient-is-the-symbolic-api" title="Permalink to this headline"></a></h2>
<p>The Symbolic API brings the efficient C++
operations in powerful toolkits, such as CXXNet and Caffe, together with the
flexible dynamic NDArray operations. All of the memory and computation resources are
allocated statically during bind operations, to maximize runtime performance and memory
utilization.</p>
<p>The coarse-grained operators are equivalent to CXXNet layers, which are
extremely efficient. We also provide fine-grained operators for more flexible
composition. Because MXNet does more in-place memory allocation, it can
be more memory efficient than CXXNet and gets to the same runtime with
greater flexibility.</p>
</div>
<div class="section" id="next-steps">
<span id="next-steps"></span><h2>Next Steps<a class="headerlink" href="#next-steps" title="Permalink to this headline"></a></h2>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference external" href="http://mxnet.io/tutorials/r/CallbackFunctionTutorial.html">Write and use callback functions</a></li>
<li class="toctree-l1"><a class="reference external" href="http://mxnet.io/tutorials/r/fiveMinutesNeuralNetwork.html">Neural Networks with MXNet in Five Minutes</a></li>
<li class="toctree-l1"><a class="reference external" href="http://mxnet.io/tutorials/r/classifyRealImageWithPretrainedModel.html">Classify Real-World Images with Pre-trained Model</a></li>
<li class="toctree-l1"><a class="reference external" href="http://mxnet.io/tutorials/r/mnistCompetition.html">Handwritten Digits Classification Competition</a></li>
<li class="toctree-l1"><a class="reference external" href="http://mxnet.io/tutorials/r/charRnnModel.html">Character Language Model using RNN</a></li>
</ul>
</div>
</div>
</div>
<div class="container">
<div class="footer">
<p> © 2015-2017 DMLC. All rights reserved. </p>
</div>
</div>
</div>
<div aria-label="main navigation" class="sphinxsidebar rightsidebar" role="navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="../../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Symbol and Automatic Differentiation</a><ul>
<li><a class="reference internal" href="#basic-composition-of-symbols">Basic Composition of Symbols</a></li>
<li><a class="reference internal" href="#more-complicated-composition-of-symbols">More Complicated Composition of Symbols</a></li>
<li><a class="reference internal" href="#training-a-neural-net">Training a Neural Net</a></li>
<li><a class="reference internal" href="#how-efficient-is-the-symbolic-api">How Efficient Is the Symbolic API?</a></li>
<li><a class="reference internal" href="#next-steps">Next Steps</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- pagename != index -->
<script crossorigin="anonymous" integrity="sha384-0mSbJDEHialfmuBBQP6A4Qrprq5OVfW37PRR3j5ELqxss1yVqOtnepnHVP9aJ7xS" src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/js/bootstrap.min.js"></script>
<script src="../../_static/js/sidebar.js" type="text/javascript"></script>
<script src="../../_static/js/search.js" type="text/javascript"></script>
<script src="../../_static/js/navbar.js" type="text/javascript"></script>
<script src="../../_static/js/clipboard.min.js" type="text/javascript"></script>
<script src="../../_static/js/copycode.js" type="text/javascript"></script>
<script type="text/javascript">
$('body').ready(function () {
$('body').css('visibility', 'visible');
});
</script>
</div></body>
</html>