| |
| |
| |
| <!DOCTYPE html> |
| <!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]--> |
| <!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]--> |
| <head> |
| <meta charset="utf-8"> |
| |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| |
| <title>Python Binding — incubator-singa 0.3.0 documentation</title> |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" /> |
| |
| |
| |
| |
| |
| <link rel="top" title="incubator-singa 0.3.0 documentation" href="../index.html"/> |
| |
| |
| <script src="../_static/js/modernizr.min.js"></script> |
| |
| </head> |
| |
| <body class="wy-body-for-nav" role="document"> |
| |
| <div class="wy-grid-for-nav"> |
| |
| |
| <nav data-toggle="wy-nav-shift" class="wy-nav-side"> |
| <div class="wy-side-scroll"> |
| <div class="wy-side-nav-search"> |
| |
| |
| |
| <a href="../index.html" class="icon icon-home"> incubator-singa |
| |
| |
| |
| |
| <img src="../_static/singa.png" class="logo" /> |
| |
| </a> |
| |
| |
| |
| |
| <div class="version"> |
| 0.3.0 |
| </div> |
| |
| |
| |
| |
| <div role="search"> |
| <form id="rtd-search-form" class="wy-form" action="../search.html" method="get"> |
| <input type="text" name="q" placeholder="Search docs" /> |
| <input type="hidden" name="check_keywords" value="yes" /> |
| <input type="hidden" name="area" value="default" /> |
| </form> |
| </div> |
| |
| |
| </div> |
| |
| <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation"> |
| |
| |
| |
| <ul> |
| <li class="toctree-l1"><a class="reference internal" href="../downloads.html">Download SINGA</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="index.html">Documentation</a></li> |
| </ul> |
| <p class="caption"><span class="caption-text">Development</span></p> |
| <ul> |
| <li class="toctree-l1"><a class="reference internal" href="../develop/schedule.html">Development Schedule</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../develop/how-contribute.html">How to Contribute to SINGA</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../develop/contribute-code.html">How to Contribute Code</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../develop/contribute-docs.html">How to Contribute Documentation</a></li> |
| </ul> |
| <p class="caption"><span class="caption-text">Community</span></p> |
| <ul> |
| <li class="toctree-l1"><a class="reference internal" href="../community/source-repository.html">Source Repository</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../community/mail-lists.html">Project Mailing Lists</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../community/issue-tracking.html">Issue Tracking</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../community/team-list.html">The SINGA Team</a></li> |
| </ul> |
| |
| |
| |
| </div> |
| </div> |
| </nav> |
| |
| <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"> |
| |
| |
| <nav class="wy-nav-top" role="navigation" aria-label="top navigation"> |
| <i data-toggle="wy-nav-top" class="fa fa-bars"></i> |
| <a href="../index.html">incubator-singa</a> |
| </nav> |
| |
| |
| |
| <div class="wy-nav-content"> |
| <div class="rst-content"> |
| |
| |
| |
| |
| |
| |
| <div role="navigation" aria-label="breadcrumbs navigation"> |
| <ul class="wy-breadcrumbs"> |
| <li><a href="../index.html">Docs</a> »</li> |
| |
| <li>Python Binding</li> |
| <li class="wy-breadcrumbs-aside"> |
| |
| |
| |
| </li> |
| </ul> |
| <hr/> |
| </div> |
| <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article"> |
| <div itemprop="articleBody"> |
| |
| <div class="section" id="python-binding"> |
| <span id="python-binding"></span><h1>Python Binding<a class="headerlink" href="#python-binding" title="Permalink to this headline">¶</a></h1> |
| <hr class="docutils" /> |
| <p>Python binding provides APIs for configuring a training job following |
| <a class="reference external" href="http://keras.io/">keras</a>, including the configuration of neural net, training |
| algorithm, etc. It replaces the configuration file (e.g., <em>job.conf</em>) in |
| protobuf format, which is typically long and error-prone to prepare. We will add |
| python functions to interact with the layer and neural net |
| objects (see <a class="reference external" href="python_interactive_training.html">here</a>), which would enable users to train and debug their models |
| interactively.</p> |
| <p>Here is the layout of python related code,</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">SINGAROOT</span><span class="o">/</span><span class="n">tool</span><span class="o">/</span><span class="n">python</span> |
| <span class="o">|--</span> <span class="n">pb2</span> <span class="p">(</span><span class="n">has</span> <span class="n">job_pb2</span><span class="o">.</span><span class="n">py</span><span class="p">)</span> |
| <span class="o">|--</span> <span class="n">singa</span> |
| <span class="o">|--</span> <span class="n">model</span><span class="o">.</span><span class="n">py</span> |
| <span class="o">|--</span> <span class="n">layer</span><span class="o">.</span><span class="n">py</span> |
| <span class="o">|--</span> <span class="n">parameter</span><span class="o">.</span><span class="n">py</span> |
| <span class="o">|--</span> <span class="n">initialization</span><span class="o">.</span><span class="n">py</span> |
| <span class="o">|--</span> <span class="n">utils</span> |
| <span class="o">|--</span> <span class="n">utility</span><span class="o">.</span><span class="n">py</span> |
| <span class="o">|--</span> <span class="n">message</span><span class="o">.</span><span class="n">py</span> |
| <span class="o">|--</span> <span class="n">examples</span> |
| <span class="o">|--</span> <span class="n">cifar10_cnn</span><span class="o">.</span><span class="n">py</span><span class="p">,</span> <span class="n">mnist_mlp</span><span class="o">.</span><span class="n">py</span><span class="p">,</span> <span class="p">,</span> <span class="n">mnist_rbm1</span><span class="o">.</span><span class="n">py</span><span class="p">,</span> <span class="n">mnist_ae</span><span class="o">.</span><span class="n">py</span><span class="p">,</span> <span class="n">etc</span><span class="o">.</span> |
| <span class="o">|--</span> <span class="n">datasets</span> |
| <span class="o">|--</span> <span class="n">cifar10</span><span class="o">.</span><span class="n">py</span> |
| <span class="o">|--</span> <span class="n">mnist</span><span class="o">.</span><span class="n">py</span> |
| </pre></div> |
| </div> |
| <div class="section" id="compiling-and-running-instructions"> |
| <span id="compiling-and-running-instructions"></span><h2>Compiling and running instructions<a class="headerlink" href="#compiling-and-running-instructions" title="Permalink to this headline">¶</a></h2> |
| <p>In order to use the Python APIs, users need to add the following arguments when compiling |
| SINGA,</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="o">./</span><span class="n">configure</span> <span class="o">--</span><span class="n">enable</span><span class="o">-</span><span class="n">python</span> <span class="o">--</span><span class="k">with</span><span class="o">-</span><span class="n">python</span><span class="o">=</span><span class="n">PYTHON_DIR</span> |
| <span class="n">make</span> |
| </pre></div> |
| </div> |
| <p>where PYTHON_DIR has Python.h</p> |
| <p>The training program is launched by</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="nb">bin</span><span class="o">/</span><span class="n">singa</span><span class="o">-</span><span class="n">run</span><span class="o">.</span><span class="n">sh</span> <span class="o">-</span><span class="n">exec</span> <span class="o"><</span><span class="n">user_main</span><span class="o">.</span><span class="n">py</span><span class="o">></span> |
| </pre></div> |
| </div> |
| <p>where user_main.py creates the JobProto object and passes it to Driver::Train to |
| start the training.</p> |
| <p>For example,</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">cd</span> <span class="n">SINGAROOT</span> |
| <span class="nb">bin</span><span class="o">/</span><span class="n">singa</span><span class="o">-</span><span class="n">run</span><span class="o">.</span><span class="n">sh</span> <span class="o">-</span><span class="n">exec</span> <span class="n">tool</span><span class="o">/</span><span class="n">python</span><span class="o">/</span><span class="n">examples</span><span class="o">/</span><span class="n">cifar10_cnn</span><span class="o">.</span><span class="n">py</span> |
| </pre></div> |
| </div> |
| </div> |
| <div class="section" id="examples"> |
| <span id="examples"></span><h2>Examples<a class="headerlink" href="#examples" title="Permalink to this headline">¶</a></h2> |
| <div class="section" id="mlp-example"> |
| <span id="mlp-example"></span><h3>MLP Example<a class="headerlink" href="#mlp-example" title="Permalink to this headline">¶</a></h3> |
| <p>This example uses python APIs to configure and train a MLP model over the MNIST |
| dataset. The configuration content is the same as that written in <em>SINGAROOT/examples/mnist/job.conf</em>.</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">X_train</span><span class="p">,</span> <span class="n">X_test</span><span class="p">,</span> <span class="n">workspace</span> <span class="o">=</span> <span class="n">mnist</span><span class="o">.</span><span class="n">load_data</span><span class="p">()</span> |
| |
| <span class="n">m</span> <span class="o">=</span> <span class="n">Sequential</span><span class="p">(</span><span class="s1">'mlp'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">)</span> |
| |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">2500</span><span class="p">,</span> <span class="n">init</span><span class="o">=</span><span class="s1">'uniform'</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'stanh'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">2000</span><span class="p">,</span> <span class="n">init</span><span class="o">=</span><span class="s1">'uniform'</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'stanh'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">1500</span><span class="p">,</span> <span class="n">init</span><span class="o">=</span><span class="s1">'uniform'</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'stanh'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">1000</span><span class="p">,</span> <span class="n">init</span><span class="o">=</span><span class="s1">'uniform'</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'stanh'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">500</span><span class="p">,</span> <span class="n">init</span><span class="o">=</span><span class="s1">'uniform'</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'stanh'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="n">init</span><span class="o">=</span><span class="s1">'uniform'</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'softmax'</span><span class="p">))</span> |
| |
| <span class="n">sgd</span> <span class="o">=</span> <span class="n">SGD</span><span class="p">(</span><span class="n">lr</span><span class="o">=</span><span class="mf">0.001</span><span class="p">,</span> <span class="n">lr_type</span><span class="o">=</span><span class="s1">'step'</span><span class="p">)</span> |
| <span class="n">topo</span> <span class="o">=</span> <span class="n">Cluster</span><span class="p">(</span><span class="n">workspace</span><span class="p">)</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="n">loss</span><span class="o">=</span><span class="s1">'categorical_crossentropy'</span><span class="p">,</span> <span class="n">optimizer</span><span class="o">=</span><span class="n">sgd</span><span class="p">,</span> <span class="n">cluster</span><span class="o">=</span><span class="n">topo</span><span class="p">)</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">nb_epoch</span><span class="o">=</span><span class="mi">1000</span><span class="p">,</span> <span class="n">with_test</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> |
| <span class="n">result</span> <span class="o">=</span> <span class="n">m</span><span class="o">.</span><span class="n">evaluate</span><span class="p">(</span><span class="n">X_test</span><span class="p">,</span> <span class="n">batch_size</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span> <span class="n">test_steps</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">test_freq</span><span class="o">=</span><span class="mi">60</span><span class="p">)</span> |
| </pre></div> |
| </div> |
| </div> |
| <div class="section" id="cnn-example"> |
| <span id="cnn-example"></span><h3>CNN Example<a class="headerlink" href="#cnn-example" title="Permalink to this headline">¶</a></h3> |
| <p>This example uses python APIs to configure and train a CNN model over the Cifar10 |
| dataset. The configuration content is the same as that written in <em>SINGAROOT/examples/cifar10/job.conf</em>.</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">X_train</span><span class="p">,</span> <span class="n">X_test</span><span class="p">,</span> <span class="n">workspace</span> <span class="o">=</span> <span class="n">cifar10</span><span class="o">.</span><span class="n">load_data</span><span class="p">()</span> |
| |
| <span class="n">m</span> <span class="o">=</span> <span class="n">Sequential</span><span class="p">(</span><span class="s1">'cnn'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">)</span> |
| |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Convolution2D</span><span class="p">(</span><span class="mi">32</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="n">w_std</span><span class="o">=</span><span class="mf">0.0001</span><span class="p">,</span> <span class="n">b_lr</span><span class="o">=</span><span class="mi">2</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">MaxPooling2D</span><span class="p">(</span><span class="n">pool_size</span><span class="o">=</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">),</span> <span class="n">stride</span><span class="o">=</span><span class="mi">2</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Activation</span><span class="p">(</span><span class="s1">'relu'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">LRN2D</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="n">alpha</span><span class="o">=</span><span class="mf">0.00005</span><span class="p">,</span> <span class="n">beta</span><span class="o">=</span><span class="mf">0.75</span><span class="p">))</span> |
| |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Convolution2D</span><span class="p">(</span><span class="mi">32</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="n">b_lr</span><span class="o">=</span><span class="mi">2</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Activation</span><span class="p">(</span><span class="s1">'relu'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">AvgPooling2D</span><span class="p">(</span><span class="n">pool_size</span><span class="o">=</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">),</span> <span class="n">stride</span><span class="o">=</span><span class="mi">2</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">LRN2D</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="n">alpha</span><span class="o">=</span><span class="mf">0.00005</span><span class="p">,</span> <span class="n">beta</span><span class="o">=</span><span class="mf">0.75</span><span class="p">))</span> |
| |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Convolution2D</span><span class="p">(</span><span class="mi">64</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Activation</span><span class="p">(</span><span class="s1">'relu'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">AvgPooling2D</span><span class="p">(</span><span class="n">pool_size</span><span class="o">=</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">),</span> <span class="n">stride</span><span class="o">=</span><span class="mi">2</span><span class="p">))</span> |
| |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="n">w_wd</span><span class="o">=</span><span class="mi">250</span><span class="p">,</span> <span class="n">b_lr</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">b_wd</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'softmax'</span><span class="p">))</span> |
| |
| <span class="n">sgd</span> <span class="o">=</span> <span class="n">SGD</span><span class="p">(</span><span class="n">decay</span><span class="o">=</span><span class="mf">0.004</span><span class="p">,</span> <span class="n">lr_type</span><span class="o">=</span><span class="s1">'manual'</span><span class="p">,</span> <span class="n">step</span><span class="o">=</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span><span class="mi">60000</span><span class="p">,</span><span class="mi">65000</span><span class="p">),</span> <span class="n">step_lr</span><span class="o">=</span><span class="p">(</span><span class="mf">0.001</span><span class="p">,</span><span class="mf">0.0001</span><span class="p">,</span><span class="mf">0.00001</span><span class="p">))</span> |
| <span class="n">topo</span> <span class="o">=</span> <span class="n">Cluster</span><span class="p">(</span><span class="n">workspace</span><span class="p">)</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="n">updater</span><span class="o">=</span><span class="n">sgd</span><span class="p">,</span> <span class="n">cluster</span><span class="o">=</span><span class="n">topo</span><span class="p">)</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">nb_epoch</span><span class="o">=</span><span class="mi">1000</span><span class="p">,</span> <span class="n">with_test</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> |
| <span class="n">result</span> <span class="o">=</span> <span class="n">m</span><span class="o">.</span><span class="n">evaluate</span><span class="p">(</span><span class="n">X_test</span><span class="p">,</span> <span class="mi">1000</span><span class="p">,</span> <span class="n">test_steps</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">test_freq</span><span class="o">=</span><span class="mi">300</span><span class="p">)</span> |
| </pre></div> |
| </div> |
| </div> |
| <div class="section" id="rbm-example"> |
| <span id="rbm-example"></span><h3>RBM Example<a class="headerlink" href="#rbm-example" title="Permalink to this headline">¶</a></h3> |
| <p>This example uses python APIs to configure and train a RBM model over the MNIST |
| dataset. The configuration content is the same as that written in <em>SINGAROOT/examples/rbm</em>.conf*.</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">rbmid</span> <span class="o">=</span> <span class="mi">3</span> |
| <span class="n">X_train</span><span class="p">,</span> <span class="n">X_test</span><span class="p">,</span> <span class="n">workspace</span> <span class="o">=</span> <span class="n">mnist</span><span class="o">.</span><span class="n">load_data</span><span class="p">(</span><span class="n">nb_rbm</span><span class="o">=</span><span class="n">rbmid</span><span class="p">)</span> |
| <span class="n">m</span> <span class="o">=</span> <span class="n">Energy</span><span class="p">(</span><span class="s1">'rbm'</span><span class="o">+</span><span class="nb">str</span><span class="p">(</span><span class="n">rbmid</span><span class="p">),</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">)</span> |
| |
| <span class="n">out_dim</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1000</span><span class="p">,</span> <span class="mi">500</span><span class="p">,</span> <span class="mi">250</span><span class="p">]</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">RBM</span><span class="p">(</span><span class="n">out_dim</span><span class="p">,</span> <span class="n">w_std</span><span class="o">=</span><span class="mf">0.1</span><span class="p">,</span> <span class="n">b_wd</span><span class="o">=</span><span class="mi">0</span><span class="p">))</span> |
| |
| <span class="n">sgd</span> <span class="o">=</span> <span class="n">SGD</span><span class="p">(</span><span class="n">lr</span><span class="o">=</span><span class="mf">0.1</span><span class="p">,</span> <span class="n">decay</span><span class="o">=</span><span class="mf">0.0002</span><span class="p">,</span> <span class="n">momentum</span><span class="o">=</span><span class="mf">0.8</span><span class="p">)</span> |
| <span class="n">topo</span> <span class="o">=</span> <span class="n">Cluster</span><span class="p">(</span><span class="n">workspace</span><span class="p">)</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="n">optimizer</span><span class="o">=</span><span class="n">sgd</span><span class="p">,</span> <span class="n">cluster</span><span class="o">=</span><span class="n">topo</span><span class="p">)</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">alg</span><span class="o">=</span><span class="s1">'cd'</span><span class="p">,</span> <span class="n">nb_epoch</span><span class="o">=</span><span class="mi">6000</span><span class="p">)</span> |
| </pre></div> |
| </div> |
| </div> |
| <div class="section" id="autoencoder-example"> |
| <span id="autoencoder-example"></span><h3>AutoEncoder Example<a class="headerlink" href="#autoencoder-example" title="Permalink to this headline">¶</a></h3> |
| <p>This example uses python APIs to configure and train an autoencoder model over |
| the MNIST dataset. The configuration content is the same as that written in |
| <em>SINGAROOT/examples/autoencoder.conf</em>.</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">rbmid</span> <span class="o">=</span> <span class="mi">4</span> |
| <span class="n">X_train</span><span class="p">,</span> <span class="n">X_test</span><span class="p">,</span> <span class="n">workspace</span> <span class="o">=</span> <span class="n">mnist</span><span class="o">.</span><span class="n">load_data</span><span class="p">(</span><span class="n">nb_rbm</span><span class="o">=</span><span class="n">rbmid</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span> |
| <span class="n">m</span> <span class="o">=</span> <span class="n">Sequential</span><span class="p">(</span><span class="s1">'autoencoder'</span><span class="p">,</span> <span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">)</span> |
| |
| <span class="n">hid_dim</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1000</span><span class="p">,</span> <span class="mi">500</span><span class="p">,</span> <span class="mi">250</span><span class="p">,</span> <span class="mi">30</span><span class="p">]</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Autoencoder</span><span class="p">(</span><span class="n">hid_dim</span><span class="p">,</span> <span class="n">out_dim</span><span class="o">=</span><span class="mi">784</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'sigmoid'</span><span class="p">,</span> <span class="n">param_share</span><span class="o">=</span><span class="kc">True</span><span class="p">))</span> |
| |
| <span class="n">agd</span> <span class="o">=</span> <span class="n">AdaGrad</span><span class="p">(</span><span class="n">lr</span><span class="o">=</span><span class="mf">0.01</span><span class="p">)</span> |
| <span class="n">topo</span> <span class="o">=</span> <span class="n">Cluster</span><span class="p">(</span><span class="n">workspace</span><span class="p">)</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="n">loss</span><span class="o">=</span><span class="s1">'mean_squared_error'</span><span class="p">,</span> <span class="n">optimizer</span><span class="o">=</span><span class="n">agd</span><span class="p">,</span> <span class="n">cluster</span><span class="o">=</span><span class="n">topo</span><span class="p">)</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">alg</span><span class="o">=</span><span class="s1">'bp'</span><span class="p">,</span> <span class="n">nb_epoch</span><span class="o">=</span><span class="mi">12200</span><span class="p">)</span> |
| </pre></div> |
| </div> |
| </div> |
| <div class="section" id="to-run-singa-on-gpu"> |
| <span id="to-run-singa-on-gpu"></span><h3>To run SINGA on GPU<a class="headerlink" href="#to-run-singa-on-gpu" title="Permalink to this headline">¶</a></h3> |
| <p>Users need to set a list of gpu ids to <code class="docutils literal"><span class="pre">device</span></code> field in fit() or evaluate(). |
| The number of GPUs must be the same to the number of workers configured for |
| cluster topology.</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">gpu_id</span> <span class="o">=</span> <span class="p">[</span><span class="mi">0</span><span class="p">]</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">nb_epoch</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span> <span class="n">with_test</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> <span class="n">device</span><span class="o">=</span><span class="n">gpu_id</span><span class="p">)</span> |
| </pre></div> |
| </div> |
| </div> |
| <div class="section" id="tips"> |
| <span id="tips"></span><h3>TIPS<a class="headerlink" href="#tips" title="Permalink to this headline">¶</a></h3> |
| <p>Hidden layers for MLP can be configured as</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="k">for</span> <span class="n">n</span> <span class="ow">in</span> <span class="p">[</span><span class="mi">2500</span><span class="p">,</span> <span class="mi">2000</span><span class="p">,</span> <span class="mi">1500</span><span class="p">,</span> <span class="mi">1000</span><span class="p">,</span> <span class="mi">500</span><span class="p">]:</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">init</span><span class="o">=</span><span class="s1">'uniform'</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'tanh'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="n">init</span><span class="o">=</span><span class="s1">'uniform'</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'softmax'</span><span class="p">))</span> |
| </pre></div> |
| </div> |
| <p>Activation layer can be specified separately</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">2500</span><span class="p">,</span> <span class="n">init</span><span class="o">=</span><span class="s1">'uniform'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Activation</span><span class="p">(</span><span class="s1">'tanh'</span><span class="p">))</span> |
| </pre></div> |
| </div> |
| <p>Users can explicitly specify hyper-parameters of weight and bias</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">par</span> <span class="o">=</span> <span class="n">Parameter</span><span class="p">(</span><span class="n">init</span><span class="o">=</span><span class="s1">'uniform'</span><span class="p">,</span> <span class="n">scale</span><span class="o">=</span><span class="mf">0.05</span><span class="p">)</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">2500</span><span class="p">,</span> <span class="n">w_param</span><span class="o">=</span><span class="n">par</span><span class="p">,</span> <span class="n">b_param</span><span class="o">=</span><span class="n">par</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'tanh'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">2000</span><span class="p">,</span> <span class="n">w_param</span><span class="o">=</span><span class="n">par</span><span class="p">,</span> <span class="n">b_param</span><span class="o">=</span><span class="n">par</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'tanh'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">1500</span><span class="p">,</span> <span class="n">w_param</span><span class="o">=</span><span class="n">par</span><span class="p">,</span> <span class="n">b_param</span><span class="o">=</span><span class="n">par</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'tanh'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">1000</span><span class="p">,</span> <span class="n">w_param</span><span class="o">=</span><span class="n">par</span><span class="p">,</span> <span class="n">b_param</span><span class="o">=</span><span class="n">par</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'tanh'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">500</span><span class="p">,</span> <span class="n">w_param</span><span class="o">=</span><span class="n">par</span><span class="p">,</span> <span class="n">b_param</span><span class="o">=</span><span class="n">par</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'tanh'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="n">w_param</span><span class="o">=</span><span class="n">par</span><span class="p">,</span> <span class="n">b_param</span><span class="o">=</span><span class="n">par</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'softmax'</span><span class="p">))</span> |
| </pre></div> |
| </div> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">parw</span> <span class="o">=</span> <span class="n">Parameter</span><span class="p">(</span><span class="n">init</span><span class="o">=</span><span class="s1">'gauss'</span><span class="p">,</span> <span class="n">std</span><span class="o">=</span><span class="mf">0.0001</span><span class="p">)</span> |
| <span class="n">parb</span> <span class="o">=</span> <span class="n">Parameter</span><span class="p">(</span><span class="n">init</span><span class="o">=</span><span class="s1">'const'</span><span class="p">,</span> <span class="n">value</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Convolution</span><span class="p">(</span><span class="mi">32</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="n">w_param</span><span class="o">=</span><span class="n">parw</span><span class="p">,</span> <span class="n">b_param</span><span class="o">=</span><span class="n">parb</span><span class="p">,</span> <span class="n">b_lr</span><span class="o">=</span><span class="mi">2</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">MaxPooling2D</span><span class="p">(</span><span class="n">pool_size</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">),</span> <span class="n">stride</span><span class="o">=</span><span class="mi">2</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Activation</span><span class="p">(</span><span class="s1">'relu'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">LRN2D</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="n">alpha</span><span class="o">=</span><span class="mf">0.00005</span><span class="p">,</span> <span class="n">beta</span><span class="o">=</span><span class="mf">0.75</span><span class="p">))</span> |
| |
| <span class="n">parw</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">std</span><span class="o">=</span><span class="mf">0.01</span><span class="p">)</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Convolution</span><span class="p">(</span><span class="mi">32</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="n">w_param</span><span class="o">=</span><span class="n">parw</span><span class="p">,</span> <span class="n">b_param</span><span class="o">=</span><span class="n">parb</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Activation</span><span class="p">(</span><span class="s1">'relu'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">AvgPooling2D</span><span class="p">(</span><span class="n">pool_size</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">),</span> <span class="n">stride</span><span class="o">=</span><span class="mi">2</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">LRN2D</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="n">alpha</span><span class="o">=</span><span class="mf">0.00005</span><span class="p">,</span> <span class="n">beta</span><span class="o">=</span><span class="mf">0.75</span><span class="p">))</span> |
| |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Convolution</span><span class="p">(</span><span class="mi">64</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="n">w_param</span><span class="o">=</span><span class="n">parw</span><span class="p">,</span> <span class="n">b_param</span><span class="o">=</span><span class="n">parb</span><span class="p">,</span> <span class="n">b_lr</span><span class="o">=</span><span class="mi">1</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Activation</span><span class="p">(</span><span class="s1">'relu'</span><span class="p">))</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">AvgPooling2D</span><span class="p">(</span><span class="n">pool_size</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span><span class="mi">3</span><span class="p">),</span> <span class="n">stride</span><span class="o">=</span><span class="mi">2</span><span class="p">))</span> |
| |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="n">w_param</span><span class="o">=</span><span class="n">parw</span><span class="p">,</span> <span class="n">w_wd</span><span class="o">=</span><span class="mi">250</span><span class="p">,</span> <span class="n">b_param</span><span class="o">=</span><span class="n">parb</span><span class="p">,</span> <span class="n">b_lr</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">b_wd</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">activation</span><span class="o">=</span><span class="s1">'softmax'</span><span class="p">))</span> |
| </pre></div> |
| </div> |
| <p>Data can be added in this way,</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">X_train</span><span class="p">,</span> <span class="n">X_test</span> <span class="o">=</span> <span class="n">mnist</span><span class="o">.</span><span class="n">load_data</span><span class="p">()</span> <span class="o">//</span> <span class="n">parameter</span> <span class="n">values</span> <span class="n">are</span> <span class="nb">set</span> <span class="ow">in</span> <span class="n">load_data</span><span class="p">()</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span> <span class="o">//</span> <span class="n">Data</span> <span class="n">layer</span> <span class="k">for</span> <span class="n">training</span> <span class="ow">is</span> <span class="n">added</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">evaluate</span><span class="p">(</span><span class="n">X_test</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span> <span class="o">//</span> <span class="n">Data</span> <span class="n">layer</span> <span class="k">for</span> <span class="n">testing</span> <span class="ow">is</span> <span class="n">added</span> |
| </pre></div> |
| </div> |
| <p>or this way,</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">X_train</span><span class="p">,</span> <span class="n">X_test</span> <span class="o">=</span> <span class="n">mnist</span><span class="o">.</span><span class="n">load_data</span><span class="p">()</span> <span class="o">//</span> <span class="n">parameter</span> <span class="n">values</span> <span class="n">are</span> <span class="nb">set</span> <span class="ow">in</span> <span class="n">load_data</span><span class="p">()</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">X_train</span><span class="p">)</span> <span class="o">//</span> <span class="n">explicitly</span> <span class="n">add</span> <span class="n">Data</span> <span class="n">layer</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">X_test</span><span class="p">)</span> <span class="o">//</span> <span class="n">explicitly</span> <span class="n">add</span> <span class="n">Data</span> <span class="n">layer</span> |
| </pre></div> |
| </div> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">store</span> <span class="o">=</span> <span class="n">Store</span><span class="p">(</span><span class="n">path</span><span class="o">=</span><span class="s1">'train.bin'</span><span class="p">,</span> <span class="n">batch_size</span><span class="o">=</span><span class="mi">64</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span> <span class="o">//</span> <span class="n">parameter</span> <span class="n">values</span> <span class="n">are</span> <span class="nb">set</span> <span class="n">explicitly</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Data</span><span class="p">(</span><span class="n">load</span><span class="o">=</span><span class="s1">'recordinput'</span><span class="p">,</span> <span class="n">phase</span><span class="o">=</span><span class="s1">'train'</span><span class="p">,</span> <span class="n">conf</span><span class="o">=</span><span class="n">store</span><span class="p">))</span> <span class="o">//</span> <span class="n">Data</span> <span class="n">layer</span> <span class="ow">is</span> <span class="n">added</span> |
| <span class="n">store</span> <span class="o">=</span> <span class="n">Store</span><span class="p">(</span><span class="n">path</span><span class="o">=</span><span class="s1">'test.bin'</span><span class="p">,</span> <span class="n">batch_size</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span> <span class="o">//</span> <span class="n">parameter</span> <span class="n">values</span> <span class="n">are</span> <span class="nb">set</span> <span class="n">explicitly</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Data</span><span class="p">(</span><span class="n">load</span><span class="o">=</span><span class="s1">'recordinput'</span><span class="p">,</span> <span class="n">phase</span><span class="o">=</span><span class="s1">'test'</span><span class="p">,</span> <span class="n">conf</span><span class="o">=</span><span class="n">store</span><span class="p">))</span> <span class="o">//</span> <span class="n">Data</span> <span class="n">layer</span> <span class="ow">is</span> <span class="n">added</span> |
| </pre></div> |
| </div> |
| </div> |
| <div class="section" id="cases-to-run-singa"> |
| <span id="cases-to-run-singa"></span><h3>Cases to run SINGA<a class="headerlink" href="#cases-to-run-singa" title="Permalink to this headline">¶</a></h3> |
| <p>(1) Run SINGA for training</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">m</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">nb_epoch</span><span class="o">=</span><span class="mi">1000</span><span class="p">)</span> |
| </pre></div> |
| </div> |
| <p>(2) Run SINGA for training and validation</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">m</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">validate_data</span><span class="o">=</span><span class="n">X_valid</span><span class="p">,</span> <span class="n">nb_epoch</span><span class="o">=</span><span class="mi">1000</span><span class="p">)</span> |
| </pre></div> |
| </div> |
| <p>(3) Run SINGA for test while training</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">m</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">nb_epoch</span><span class="o">=</span><span class="mi">1000</span><span class="p">,</span> <span class="n">with_test</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> |
| <span class="n">result</span> <span class="o">=</span> <span class="n">m</span><span class="o">.</span><span class="n">evaluate</span><span class="p">(</span><span class="n">X_test</span><span class="p">,</span> <span class="n">batch_size</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span> <span class="n">test_steps</span><span class="o">=</span><span class="mi">100</span><span class="p">)</span> |
| </pre></div> |
| </div> |
| <p>(4) Run SINGA for test only |
| Assume a checkpoint exists after training</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">result</span> <span class="o">=</span> <span class="n">m</span><span class="o">.</span><span class="n">evaluate</span><span class="p">(</span><span class="n">X_test</span><span class="p">,</span> <span class="n">batch_size</span><span class="o">=</span><span class="mi">100</span><span class="p">,</span> <span class="n">checkpoint_path</span><span class="o">=</span><span class="n">workspace</span><span class="o">+</span><span class="s1">'/checkpoint/step100-worker0'</span><span class="p">)</span> |
| </pre></div> |
| </div> |
| </div> |
| </div> |
| <div class="section" id="implementation-details"> |
| <span id="implementation-details"></span><h2>Implementation Details<a class="headerlink" href="#implementation-details" title="Permalink to this headline">¶</a></h2> |
| <div class="section" id="layer-class-inherited"> |
| <span id="layer-class-inherited"></span><h3>Layer class (inherited)<a class="headerlink" href="#layer-class-inherited" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li>Data</li> |
| <li>Dense</li> |
| <li>Activation</li> |
| <li>Convolution2D</li> |
| <li>MaxPooling2D</li> |
| <li>AvgPooling2D</li> |
| <li>LRN2D</li> |
| <li>Dropout</li> |
| <li>RBM</li> |
| <li>Autoencoder</li> |
| </ul> |
| </div> |
| <div class="section" id="model-class"> |
| <span id="model-class"></span><h3>Model class<a class="headerlink" href="#model-class" title="Permalink to this headline">¶</a></h3> |
| <p>Model class has <code class="docutils literal"><span class="pre">jobconf</span></code> (JobProto) and <code class="docutils literal"><span class="pre">layers</span></code> (layer list)</p> |
| <p>Methods in Model class</p> |
| <ul class="simple"> |
| <li>add<ul> |
| <li>add Layer into Model</li> |
| <li>2 subclasses: Sequential model and Energy model</li> |
| </ul> |
| </li> |
| <li>compile<ul> |
| <li>set Updater (i.e., optimizer) and Cluster (i.e., topology) components</li> |
| </ul> |
| </li> |
| <li>fit<ul> |
| <li>set Training data and parameter values for the training<ul> |
| <li>(optional) set Validatiaon data and parameter values</li> |
| </ul> |
| </li> |
| <li>set Train_one_batch component</li> |
| <li>specify <code class="docutils literal"><span class="pre">with_test</span></code> field if a user wants to run SINGA with test data simultaneously.</li> |
| <li>[TODO] recieve train/validation results, e.g., accuracy, loss, ppl, etc.</li> |
| </ul> |
| </li> |
| <li>evaluate<ul> |
| <li>set Testing data and parameter values for the testing</li> |
| <li>specify <code class="docutils literal"><span class="pre">checkpoint_path</span></code> field if a user want to run SINGA only for testing.</li> |
| <li>[TODO] recieve test results, e.g., accuracy, loss, ppl, etc.</li> |
| </ul> |
| </li> |
| </ul> |
| </div> |
| <div class="section" id="results"> |
| <span id="results"></span><h3>Results<a class="headerlink" href="#results" title="Permalink to this headline">¶</a></h3> |
| <p>fit() and evaluate() return train/test results, a dictionary containing</p> |
| <ul class="simple"> |
| <li>[key]: step number</li> |
| <li>[value]: a list of dictionay<ul> |
| <li>‘acc’ for accuracy</li> |
| <li>‘loss’ for loss</li> |
| <li>‘ppl’ for ppl</li> |
| <li>‘se’ for squred error</li> |
| </ul> |
| </li> |
| </ul> |
| </div> |
| <div class="section" id="parameter-class"> |
| <span id="parameter-class"></span><h3>Parameter class<a class="headerlink" href="#parameter-class" title="Permalink to this headline">¶</a></h3> |
| <p>Users need to set parameter and initial values. For example,</p> |
| <ul class="simple"> |
| <li>Parameter (fields in Param proto)<ul> |
| <li>lr = (float) // learning rate multiplier, used to scale the learning rate when updating parameters.</li> |
| <li>wd = (float) // weight decay multiplier, used to scale the weight decay when updating parameters.</li> |
| </ul> |
| </li> |
| <li>Parameter initialization (fields in ParamGen proto)<ul> |
| <li>init = (string) // one of the types, ‘uniform’, ‘constant’, ‘gaussian’</li> |
| <li>high = (float) // for ‘uniform’</li> |
| <li>low = (float) // for ‘uniform’</li> |
| <li>value = (float) // for ‘constant’</li> |
| <li>mean = (float) // for ‘gaussian’</li> |
| <li>std = (float) // for ‘gaussian’</li> |
| </ul> |
| </li> |
| <li>Weight (<code class="docutils literal"><span class="pre">w_param</span></code>) is ‘gaussian’ with mean=0, std=0.01 at default</li> |
| <li>Bias (<code class="docutils literal"><span class="pre">b_param</span></code>) is ‘constant’ with value=0 at default</li> |
| <li>How to update the parameter fields<ul> |
| <li>for updating Weight, put <code class="docutils literal"><span class="pre">w_</span></code> in front of field name</li> |
| <li>for updating Bias, put <code class="docutils literal"><span class="pre">b_</span></code> in front of field name</li> |
| </ul> |
| </li> |
| </ul> |
| <p>Several ways to set Parameter values</p> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">parw</span> <span class="o">=</span> <span class="n">Parameter</span><span class="p">(</span><span class="n">lr</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">wd</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">init</span><span class="o">=</span><span class="s1">'gaussian'</span><span class="p">,</span> <span class="n">std</span><span class="o">=</span><span class="mf">0.1</span><span class="p">)</span> |
| <span class="n">parb</span> <span class="o">=</span> <span class="n">Parameter</span><span class="p">(</span><span class="n">lr</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">wd</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span> <span class="n">init</span><span class="o">=</span><span class="s1">'constant'</span><span class="p">,</span> <span class="n">value</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Convolution2D</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="n">w_param</span><span class="o">=</span><span class="n">parw</span><span class="p">,</span> <span class="n">b_param</span><span class="o">=</span><span class="n">parb</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span> |
| </pre></div> |
| </div> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="n">w_mean</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">w_std</span><span class="o">=</span><span class="mf">0.1</span><span class="p">,</span> <span class="n">w_lr</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">w_wd</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span> |
| </pre></div> |
| </div> |
| <div class="highlight-default"><div class="highlight"><pre><span></span><span class="n">parw</span> <span class="o">=</span> <span class="n">Parameter</span><span class="p">(</span><span class="n">init</span><span class="o">=</span><span class="s1">'constant'</span><span class="p">,</span> <span class="n">mean</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span> |
| <span class="n">m</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">Dense</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="n">w_param</span><span class="o">=</span><span class="n">parw</span><span class="p">,</span> <span class="n">w_lr</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">w_wd</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">b_value</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span> |
| </pre></div> |
| </div> |
| </div> |
| <div class="section" id="other-classes"> |
| <span id="other-classes"></span><h3>Other classes<a class="headerlink" href="#other-classes" title="Permalink to this headline">¶</a></h3> |
| <ul class="simple"> |
| <li>Store</li> |
| <li>Algorithm</li> |
| <li>Updater</li> |
| <li>SGD</li> |
| <li>AdaGrad</li> |
| <li>Cluster</li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| |
| |
| </div> |
| </div> |
| <footer> |
| |
| |
| <hr/> |
| |
| <div role="contentinfo"> |
| <p> |
| © Copyright 2016 The Apache Software Foundation. All rights reserved. Apache Singa, Apache, the Apache feather logo, and the Apache Singa project logos are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.. |
| |
| </p> |
| </div> |
| Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/snide/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>. |
| |
| </footer> |
| |
| </div> |
| </div> |
| |
| </section> |
| |
| </div> |
| |
| |
| |
| |
| |
| <script type="text/javascript"> |
| var DOCUMENTATION_OPTIONS = { |
| URL_ROOT:'../', |
| VERSION:'0.3.0', |
| COLLAPSE_INDEX:false, |
| FILE_SUFFIX:'.html', |
| HAS_SOURCE: true |
| }; |
| </script> |
| <script type="text/javascript" src="../_static/jquery.js"></script> |
| <script type="text/javascript" src="../_static/underscore.js"></script> |
| <script type="text/javascript" src="../_static/doctools.js"></script> |
| |
| |
| |
| |
| |
| <script type="text/javascript" src="../_static/js/theme.js"></script> |
| |
| |
| |
| |
| <script type="text/javascript"> |
| jQuery(function () { |
| SphinxRtdTheme.StickyNav.enable(); |
| }); |
| </script> |
| |
| |
| |
| <div class="rst-versions shift-up" data-toggle="rst-versions" role="note" aria-label="versions"> |
| <img src="../_static/apache.jpg"> |
| |
| <span class="rst-current-version" data-toggle="rst-current-version"> |
| <span class="fa fa-book"> incubator-singa </span> |
| v: 0.3.0 |
| <span class="fa fa-caret-down"></span> |
| </span> |
| <div class="rst-other-versions"> |
| <dl> |
| <dt>Languages</dt> |
| <dd><a href="../../en/index.html">English</a></dd> |
| <dd><a href="../../zh/index.html">中文</a></dd> |
| <dd><a href="../../jp/index.html">日本語</a></dd> |
| <dd><a href="../../kr/index.html">한국어</a></dd> |
| </dl> |
| </div> |
| </div> |
| |
| <a href="https://github.com/apache/incubator-singa"> |
| <img style="position: absolute; top: 0; right: 0; border: 0; z-index: 10000;" |
| src="https://s3.amazonaws.com/github/ribbons/forkme_right_orange_ff7600.png" |
| alt="Fork me on GitHub"> |
| </a> |
| |
| |
| |
| |
| </body> |
| </html> |
