{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements; and to You under the Apache License, Version 2.0. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Apache Singa](http://singa.apache.org/en/_static/singa.png)\n",
    "\n",
    "# A Tutorial of SINGA V1\n",
    "\n",
    "Welcome to this tutorial for Apache Incubator-singa.\n",
    "Please install [conda and SINGA](./installation.ipynb) before running the notebooks.\n",
    "\n",
    "## Outline\n",
    "\n",
    "* Introduction and installation\n",
    "* SINGA classes\n",
    "  * [Core classes](./core.ipynb) - Tensor and Device\n",
    "  * [Model classes](./model.ipynb) - Layer, Loss, Metric, FeedForwardNet, Optimzier, Initializer\n",
    "* Simple models \n",
    "  * [Linear Regression](./regression.ipynb)\n",
    "  * [Multi-layer Perceptron](./mlp.ipynb)\n",
    "* Deep learning models\n",
    "  * [Convolutional Neural Network (CNN)](./cnn.ipynb)\n",
    "  * [Recurrent Neural Networks (RNN)](./rnn.ipynb)\n",
    "  * [Restricted Boltzmann Machine (RBM)](./rbm.ipynb)\n",
    "* [Distributed training](./distributed.ipynb) (WIP)\n",
    "* [Rafiki](./rafiki.ipynb) (WIP)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction \n",
    "### Training and Inference of deep learning models\n",
    "\n",
    "#### Training objective\n",
    "\n",
    "$\\min_{\\theta} \\sum_{<x, y>\\in D}L(f(x|\\theta), y)$\n",
    "\n",
    "Training procedure\n",
    "1. randomly initialize $\\theta$\n",
    "2. get a mini-batch of data $\\{<x, y>\\}$ and compute $\\frac{\\partial L}{\\partial \\theta}$\n",
    "3. $\\theta \\leftarrow \\theta - \\alpha \\frac{\\partial L}{\\partial \\theta}$\n",
    "4. repeat 2-3 until converge, i.e., $||\\frac{\\partial L}{\\partial \\theta}|| < \\epsilon$ \n",
    "\n",
    "<img src=\"./static/sgd.png\" width=\"400px\"/>\n",
    "\n",
    "#### Back-propagation for gradient computation\n",
    "\n",
    "<img src=\"./static/bp.PNG\" width=\"300px\"/> \n",
    "\n",
    "#### Popular deep learning models\n",
    "\n",
    "<img src=\"./static/models.PNG\" width=\"200px\"/>\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SINGA V1\n",
    "#### Software Stack\n",
    "\n",
    "<img src=\"http://singa.apache.org/en/_static/images/singav1-sw.png\" width=\"500px\"/>\n",
    "\n",
    "#### Progress\n",
    "\n",
    "|Components|Feature|Status|\n",
    "|:---------|:------|:-----|\n",
    "|Core |Tensor implementation|V1.0|\n",
    "|     |Device implementation (CppCPU, CudaGPU, OpenclGPU) |V1.1|\n",
    "|Model|Basic layers for CNN and RNN| V1.0|\n",
    "|     |Basic loss and metric functions|V1.0|\n",
    "|     |Basic optimizer and initializers |V1.0|\n",
    "|IO   |Message paasing via socket| V1.0|\n",
    "|Installation| From source on Linux and Mac|V1.0|\n",
    "|            | From source on Windows|V1.1|\n",
    "|            | From wheel on Linux and Mac|V1.0|\n",
    "|            | From debian package on Ubuntu |V1.1|\n",
    "|            | Docker images | V1.1|\n",
    "|            | AWS AMI       | V1.1|\n",
    "\n",
    "#### Schedule\n",
    "|Components|Feature|Status|\n",
    "|:---------|:------|:-----|\n",
    "|Core     |Tensor API = Numpy API| V1.2|\n",
    "| |Operation scheduling and memory optimization| V1.3|\n",
    "|Model | To support more vison, NLP examples| V1.2|\n",
    "|      | General neural net API| V1.2|\n",
    "|      | To support reinforcement learning examples| V1.3|\n",
    "|      | Auto-grad + Functional API | TBD|\n",
    "|IO | HDFS integration  |V1.3|\n",
    "|Server| Consistency and communication optimization |V1.2|   \n",
    "|      |Message passing via Infiniband |V1.2|  \n",
    "|Installation | From wheel on Windows| V1.2|\n",
    "|             | From debian on Linux | V1.2|\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next: [SINGA core classes](./core.ipynb)"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "py3",
   "language": "python",
   "name": "py3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}