Preparing for V1.0 RC0.
Updated licenses and release notes.
Added the link of v0.3 in layout.html
diff --git a/LICENSE b/LICENSE
index 75001c1..4f9f923 100644
--- a/LICENSE
+++ b/LICENSE
@@ -205,25 +205,35 @@
subcomponents is subject to the terms and conditions of the following
licenses.
-============================================================================
-SINGA bundles the following under BSD 2-clause license: include/singa/utils/blob.h,
-src/utils/blob.cc, include/singa/utils/common.h, src/utils/common.cc, include/singa/utils/cuda_utils.h
-Copyright (c) 2014, 2015, The Regents of the University of California (Regents)
-Copyright (c) 2014, the respective contributors
-https://github.com/BVLC/caffe/blob/master/LICENSE
+============================================================================
+SINGA bundles the following under BSD 3-clause license:
+cmake/Profobuf.cmake
+
+Copyright 2009 Kitware, Inc.
+Copyright 2009-2011 Philip Lowman <philip@yhbt.com>
+Copyright 2008 Esben Mose Hansen, Ange Optimization ApS
=====================================================================
-SINGA bundles the following under BSD 2-clause license: include/singa/utils/tinydir.h
+SINGA bundles the following under BSD 2-clause license:
+include/singa/utils/tinydir.h
Copyright (c) 2013, Cong Xu, Baudouin Feildel
https://github.com/cxong/tinydir/blob/master/COPYING
-=====================================================================
-SINGA bundles the following under Apache v2.0 license: include/mshadow/*
-Copyright (c) 2014 by Contributors
-https://github.com/dmlc/mshadow/blob/master/LICENSE
+===========================================================================
+SINGA bundles the following under BSD 2-clause license:
+include/singa/utils/cuda_utils.h, src/core/tensor/distribution.cl
+
+All contributions by the University of California:
+Copyright (c) 2014, 2015, The Regents of the University of California (Regents)
+All rights reserved.
+
+All other contributions:
+Copyright (c) 2014, 2015, the respective contributors
+All rights reserved.
+https://github.com/BVLC/caffe/blob/master/LICENSE
=====================================================================
SINGA bundles the following under New BSD license: include/gtest/*
@@ -238,9 +248,13 @@
https://github.com/google/styleguide/tree/gh-pages/cpplint
=====================================================================
-SINGA bundles the following under New BSD license: examples/rnnlm/create_data.cc
+SINGA bundles the following under New BSD license: lib/cnmem/*
-Copyright (c) 2010-2012 Tomas Mikolov
-Copyright (c) 2013 Cantab Research Ltd
+Copyright (c) 2015, NVIDIA CORPORATION. All rights reserved.
+https://github.com/NVIDIA/cnmem
+
+=====================================================================
+SINGA bundles the following under New BSD license: src/python/swig/numpy.i
+
+Copyright (c) 2005-2015, NumPy Developers.
All rights reserved.
-http://www.fit.vutbr.cz/~imikolov/rnnlm/
diff --git a/NOTICE b/NOTICE
index c74e53a..092ec36 100644
--- a/NOTICE
+++ b/NOTICE
@@ -4,4 +4,4 @@
This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).
-Portions of this software were developed at the National University of Singapore.
+Codebase originally donated by National University of Singapore.
diff --git a/README.md b/README.md
index 07acd68..fc80db4 100644
--- a/README.md
+++ b/README.md
@@ -4,10 +4,11 @@
Distributed deep learning system
-This is the dev branch for V1.0, please refer to the following docs for compilation and examples
-* [Compilation](doc/docs/installation.md)
-* [CNN example](doc/docs/cnn.md)
+## Quick Start
+
+* [Installation](doc/en/docs/installation.md)
+* [Examples](examples)
##Mailing Lists
diff --git a/RELEASE_NOTES b/RELEASE_NOTES
index 5786ad2..e36dce8 100644
--- a/RELEASE_NOTES
+++ b/RELEASE_NOTES
@@ -1,3 +1,91 @@
+Release Notes - SINGA - Version singa-incubating-1.0.0
+
+SINGA is a general distributed deep learning platform for training big deep learning models over large datasets.
+
+This release includes following features:
+
+ * Core abstractions including Tensor and Device
+ * [SINGA-207] Update Tensor functions for matrices
+ * [SINGA-205] Enable slice and concatenate operations for Tensor objects
+ * [SINGA-197] Add CNMem as a submodule in lib/
+ * [SINGA-196] Rename class Blob to Block
+ * [SINGA-194] Add a Platform singleton
+ * [SINGA-175] Add memory management APIs and implement a subclass using CNMeM
+ * [SINGA-173] * [SINGA-162 OpenCL Implementation
+ * [SINGA-171] Create CppDevice and CudaDevice
+ * [SINGA-168] Implement Cpp Math functions APIs
+ * [SINGA-162] Overview of features for V1.x
+ * [SINGA-165] Add cross-platform timer API to singa
+ * [SINGA-167] Add Tensor Math function APIs
+ * [SINGA-166] light built-in logging for making glog optional
+ * [SINGA-164] Add the base Tensor class
+
+
+ * IO components for file read/write, network and data pre-processing
+ * [SINGA-233] New communication interface
+ * [SINGA-215] Implement Image Transformation for Image Pre-processing
+ * [SINGA-214] Add LMDBReader and LMDBWriter for LMDB
+ * [SINGA-213] Implement Encoder and Decoder for CSV
+ * [SINGA-211] Add TextFileReader and TextFileWriter for CSV files
+ * [SINGA-210] Enable checkpoint and resume for v1.0
+ * [SINGA-208] Add DataIter base class and a simple implementation
+ * [SINGA-203] Add OpenCV detection for cmake compilation
+ * [SINGA-202] Add reader and writer for binary file
+ * [SINGA-200] Implement Encoder and Decoder for data pre-processing
+
+
+
+ * Module components including layer classes, training algorithms and Python binding
+ * [SINGA-235] Unify the engines for cudnn and singa layers
+ * [SINGA-230] OpenCL Convolution layer and Pooling layer
+ * [SINGA-222] Fixed bugs in IO
+ * [SINGA-218] Implementation for RNN CUDNN version
+ * [SINGA-204] Support the training of feed-forward neural nets
+ * [SINGA-199] Implement Python classes for SGD optimizers
+ * [SINGA-198] Change Layer::Setup API to include input Tensor shapes
+ * [SINGA-193] Add Python layers
+ * [SINGA-192] Implement optimization algorithms for Singa v1 (nesterove, adagrad, rmsprop)
+ * [SINGA-191] Add "autotune" for CudnnConvolution Layer
+ * [SINGA-190] Add prelu layer and flatten layer
+ * [SINGA-189] Generate python outputs of proto files
+ * [SINGA-188] Add Dense layer
+ * [SINGA-187] Add popular parameter initialization methods
+ * [SINGA-186] Create Python Tensor class
+ * [SINGA-184] Add Cross Entropy loss computation
+ * [SINGA-183] Add the base classes for optimizer, constraint and regularizer
+ * [SINGA-180] Add Activation layer and Softmax layer
+ * [SINGA-178] Add Convolution layer and Pooling layer
+ * [SINGA-176] Add loss and metric base classes
+ * [SINGA-174] Add Batch Normalization layer and Local Response Nomalization layer.
+ * [SINGA-170] Add Dropout layer and CudnnDropout layer.
+ * [SINGA-169] Add base Layer class for V1.0
+
+
+ * Examples
+ * [SINGA-232] Alexnet on Imagenet
+ * [SINGA-231] Batchnormlized VGG model for cifar-10
+ * [SINGA-228] Add Cpp Version of Convolution and Pooling layer
+ * [SINGA-227] Add Split and Merge Layer and add ResNet Implementation
+
+ * Documentation
+ * [SINGA-239] Transfer documentation files of v0.3.0 to github
+ * [SINGA-238] RBM on mnist
+ * [SINGA-225] Documentation for installation and Cifar10 example
+ * [SINGA-223] Use Sphinx to create the website
+
+ * Tools for compilation and some utility code
+ * [SINGA-229] Complete install targets
+ * [SINGA-221] Support for Travis-CI
+ * [SINGA-217] build python package with setup.py
+ * [SINGA-216] add jenkins for CI support
+ * [SINGA-212] Disable the compilation of libcnmem if USE_CUDA is OFF
+ * [SINGA-195] Channel for sending training statistics
+ * [SINGA-185] Add CBLAS and GLOG detection for singav1
+ * [SINGA-181] Add NVCC supporting for .cu files
+ * [SINGA-177] Add fully cmake supporting for the compilation of singa_v1
+ * [SINGA-172] Add CMake supporting for Cuda and Cudnn libs
+
+----------------------------------------------------------
Release Notes - SINGA - Version singa-incubating-0.3.0
SINGA is a general distributed deep learning platform for training big deep learning models over large datasets.
diff --git a/cmake/Thirdparty/FindOpenCL.cmake b/cmake/Thirdparty/FindOpenCL.cmake
deleted file mode 100644
index 3c7daeb..0000000
--- a/cmake/Thirdparty/FindOpenCL.cmake
+++ /dev/null
@@ -1,170 +0,0 @@
-# This script was taken from https://github.com/elhigu/cmake-findopencl
-# and modified to support finding OpenCL 2.x C++ bindings.
-
-# Find OpenCL
-#
-# To set manually the paths, define these environment variables:
-# OpenCL_INCPATH - Include path (e.g. OpenCL_INCPATH=/opt/cuda/4.0/cuda/include)
-# OpenCL_LIBPATH - Library path (e.h. OpenCL_LIBPATH=/usr/lib64/nvidia)
-#
-# Once done this will define
-# OPENCL_FOUND - system has OpenCL
-# OPENCL_INCLUDE_DIRS - the OpenCL include directory
-# OPENCL_LIBRARIES - link these to use OpenCL
-# OPENCL_HAS_CPP_BINDINGS - system has also cl2.hpp
-
-FIND_PACKAGE(PackageHandleStandardArgs)
-
-SET (OPENCL_VERSION_STRING "0.1.0")
-SET (OPENCL_VERSION_MAJOR 0)
-SET (OPENCL_VERSION_MINOR 1)
-SET (OPENCL_VERSION_PATCH 0)
-
-IF (APPLE)
-
- # IF OpenCL_LIBPATH is given use it and don't use default path
- IF (DEFINED ENV{OpenCL_LIBPATH})
- FIND_LIBRARY(OPENCL_LIBRARIES OpenCL PATHS ENV OpenCL_LIBPATH NO_DEFAULT_PATH)
- ELSE ()
- FIND_LIBRARY(OPENCL_LIBRARIES OpenCL DOC "OpenCL lib for OSX")
- ENDIF ()
-
- # IF OpenCL_INCPATH is given use it and find for CL/cl.h and OpenCL/cl.h do not try to find default paths
- IF (DEFINED ENV{OpenCL_INCPATH})
- FIND_PATH(OPENCL_INCLUDE_DIRS CL/cl.h OpenCL/cl.h PATHS ENV OpenCL_INCPATH NO_DEFAULT_PATH)
- FIND_PATH(_OPENCL_CPP_INCLUDE_DIRS CL/cl2.hpp OpenCL/cl2.hpp PATHS ${OPENCL_INCLUDE_DIRS} NO_DEFAULT_PATH)
- ELSE ()
- FIND_PATH(OPENCL_INCLUDE_DIRS OpenCL/cl.h DOC "Include for OpenCL on OSX")
- FIND_PATH(_OPENCL_CPP_INCLUDE_DIRS OpenCL/cl2.hpp DOC "Include for OpenCL CPP bindings on OSX")
- ENDIF ()
-
-ELSE (APPLE)
-
- IF (WIN32)
-
- # Find OpenCL includes and libraries from environment variables provided by vendor
- SET(OPENCL_INCLUDE_SEARCH_PATHS)
- SET(OPENCL_LIBRARY_SEARCH_PATHS)
- SET(OPENCL_LIBRARY_64_SEARCH_PATHS)
-
- # Nvidia
- IF (DEFINED ENV{CUDA_INC_PATH})
- SET(OPENCL_INCLUDE_SEARCH_PATHS ${OPENCL_INCLUDE_SEARCH_PATHS} $ENV{CUDA_INC_PATH})
- SET(OPENCL_LIBRARY_64_SEARCH_PATHS ${OPENCL_LIBRARY_64_SEARCH_PATHS} $ENV{CUDA_LIB_PATH}/../lib64)
- SET(OPENCL_LIBRARY_SEARCH_PATHS ${OPENCL_LIBRARY_SEARCH_PATHS} $ENV{CUDA_LIB_PATH}/../lib)
- ENDIF()
-
- # Intel SDK
- IF (DEFINED ENV{INTELOCSDKROOT})
- SET(OPENCL_INCLUDE_SEARCH_PATHS ${OPENCL_INCLUDE_SEARCH_PATHS} $ENV{INTELOCSDKROOT}/include)
- SET(OPENCL_LIBRARY_64_SEARCH_PATHS ${OPENCL_LIBRARY_64_SEARCH_PATHS} $ENV{INTELOCSDKROOT}/lib/x64)
- SET(OPENCL_LIBRARY_SEARCH_PATHS ${OPENCL_LIBRARY_SEARCH_PATHS} $ENV{INTELOCSDKROOT}/lib/x86)
- ENDIF()
-
- # AMD SDK
- IF (DEFINED ENV{AMDAPPSDKROOT})
- SET(OPENCL_INCLUDE_SEARCH_PATHS ${OPENCL_INCLUDE_SEARCH_PATHS} $ENV{AMDAPPSDKROOT}/include)
- SET(OPENCL_LIBRARY_64_SEARCH_PATHS ${OPENCL_LIBRARY_64_SEARCH_PATHS} $ENV{AMDAPPSDKROOT}/lib/x86_64)
- SET(OPENCL_LIBRARY_SEARCH_PATHS ${OPENCL_LIBRARY_SEARCH_PATHS} $ENV{AMDAPPSDKROOT}/lib/x86)
- ENDIF()
-
- # Override search paths with OpenCL_INCPATH env variable
- IF (DEFINED ENV{OpenCL_INCPATH})
- SET(OPENCL_INCLUDE_SEARCH_PATHS $ENV{OpenCL_INCPATH})
- ENDIF ()
-
- # Override search paths with OpenCL_LIBPATH env variable
- IF (DEFINED ENV{OpenCL_INCPATH})
- SET(OPENCL_LIBRARY_SEARCH_PATHS $ENV{OpenCL_LIBPATH})
- SET(OPENCL_LIBRARY_64_SEARCH_PATHS $ENV{OpenCL_LIBPATH})
- ENDIF ()
-
- FIND_PATH(OPENCL_INCLUDE_DIRS CL/cl.h PATHS ${OPENCL_INCLUDE_SEARCH_PATHS})
- FIND_PATH(_OPENCL_CPP_INCLUDE_DIRS CL/cl2.hpp PATHS ${OPENCL_INCLUDE_SEARCH_PATHS})
-
- FIND_LIBRARY(_OPENCL_32_LIBRARIES OpenCL.lib HINTS ${OPENCL_LIBRARY_SEARCH_PATHS} PATHS ${OPENCL_LIB_DIR} ENV PATH)
- FIND_LIBRARY(_OPENCL_64_LIBRARIES OpenCL.lib HINTS ${OPENCL_LIBRARY_64_SEARCH_PATHS} PATHS ${OPENCL_LIB_DIR} ENV PATH)
-
- # Check if 64bit or 32bit versions links fine
- SET (_OPENCL_VERSION_SOURCE "${CMAKE_BINARY_DIR}${CMAKE_FILES_DIRECTORY}/openclversion.c")
- #SET (_OPENCL_VERSION_SOURCE "${CMAKE_BINARY_DIR}/test.c")
- FILE (WRITE "${_OPENCL_VERSION_SOURCE}"
- "
- #if __APPLE__
- #include <OpenCL/cl.h>
- #else /* !__APPLE__ */
- #include <CL/cl.h>
- #endif /* __APPLE__ */
- int main()
- {
- cl_int result;
- cl_platform_id id;
- result = clGetPlatformIDs(1, &id, NULL);
- return result != CL_SUCCESS;
- }
- ")
-
- TRY_COMPILE(_OPENCL_64_COMPILE_SUCCESS ${CMAKE_BINARY_DIR} "${_OPENCL_VERSION_SOURCE}"
- CMAKE_FLAGS
- "-DINCLUDE_DIRECTORIES:STRING=${OPENCL_INCLUDE_DIRS}"
- CMAKE_FLAGS
- "-DLINK_LIBRARIES:STRING=${_OPENCL_64_LIBRARIES}"
- )
-
- IF(_OPENCL_64_COMPILE_SUCCESS)
- message(STATUS "OpenCL 64bit lib found.")
- SET(OPENCL_LIBRARIES ${_OPENCL_64_LIBRARIES})
- ELSE()
- TRY_COMPILE(_OPENCL_32_COMPILE_SUCCESS ${CMAKE_BINARY_DIR} "${_OPENCL_VERSION_SOURCE}"
- CMAKE_FLAGS
- "-DINCLUDE_DIRECTORIES:STRING=${OPENCL_INCLUDE_DIRS}"
- CMAKE_FLAGS
- "-DLINK_LIBRARIES:STRING=${_OPENCL_32_LIBRARIES}"
- )
- IF(_OPENCL_32_COMPILE_SUCCESS)
- message(STATUS "OpenCL 32bit lib found.")
- SET(OPENCL_LIBRARIES ${_OPENCL_32_LIBRARIES})
- ELSE()
- message(STATUS "Couldn't link opencl..")
- ENDIF()
- ENDIF()
-
-
- ELSE (WIN32)
-
- IF (CYGWIN)
- SET (CMAKE_FIND_LIBRARY_SUFFIXES .lib)
- SET (OCL_LIB_SUFFIX .lib)
- ENDIF (CYGWIN)
-
- # Unix style platforms
- FIND_LIBRARY(OPENCL_LIBRARIES OpenCL${OCL_LIB_SUFFIX}
- PATHS ENV LD_LIBRARY_PATH ENV OpenCL_LIBPATH
- )
-
- GET_FILENAME_COMPONENT(OPENCL_LIB_DIR ${OPENCL_LIBRARIES} PATH)
- GET_FILENAME_COMPONENT(_OPENCL_INC_CAND ${OPENCL_LIB_DIR}/../../include ABSOLUTE)
-
- # The AMD SDK currently does not place its headers
- # in /usr/include, therefore also search relative
- # to the library
- FIND_PATH(OPENCL_INCLUDE_DIRS CL/cl.h PATHS ${_OPENCL_INC_CAND} "/usr/local/cuda/include" "/opt/AMDAPP/include" ENV OpenCL_INCPATH)
- FIND_PATH(_OPENCL_CPP_INCLUDE_DIRS CL/cl2.hpp PATHS ${_OPENCL_INC_CAND} "/usr/local/cuda/include" "/opt/AMDAPP/include" ENV OpenCL_INCPATH)
-
- ENDIF (WIN32)
-
-ENDIF (APPLE)
-
-FIND_PACKAGE_HANDLE_STANDARD_ARGS(OpenCL DEFAULT_MSG OPENCL_LIBRARIES OPENCL_INCLUDE_DIRS)
-
-IF(_OPENCL_CPP_INCLUDE_DIRS)
- SET( OPENCL_HAS_CPP_BINDINGS TRUE )
- LIST( APPEND OPENCL_INCLUDE_DIRS ${_OPENCL_CPP_INCLUDE_DIRS} )
- # This is often the same, so clean up
- LIST( REMOVE_DUPLICATES OPENCL_INCLUDE_DIRS )
-ENDIF(_OPENCL_CPP_INCLUDE_DIRS)
-
-MARK_AS_ADVANCED(
- OPENCL_INCLUDE_DIRS
-)
-
diff --git a/conf/hostfile b/conf/hostfile
deleted file mode 100644
index 2fbb50c..0000000
--- a/conf/hostfile
+++ /dev/null
@@ -1 +0,0 @@
-localhost
diff --git a/conf/profile b/conf/profile
deleted file mode 100644
index 72a8600..0000000
--- a/conf/profile
+++ /dev/null
@@ -1,3 +0,0 @@
-# Please add here the environment variables that cannot be recognized after ssh.
-# This file will be `source`ed upon ssh
-
diff --git a/conf/singa.conf b/conf/singa.conf
deleted file mode 100644
index 20cff98..0000000
--- a/conf/singa.conf
+++ /dev/null
@@ -1,7 +0,0 @@
-# point to your active zookeeper service
-# this is comma separated host:port pairs, each corresponding to a zk server
-# e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
-zookeeper_host: "localhost:2181"
-
-# set if you want to change log directory
-log_dir: "/tmp/singa-log/"
diff --git a/doc/_templates/layout.html b/doc/_templates/layout.html
index 1c10c5b..b149652 100755
--- a/doc/_templates/layout.html
+++ b/doc/_templates/layout.html
@@ -20,14 +20,14 @@
{% block extrahead %}
<link href="{{ pathto("_static/style.css", True) }}" rel="stylesheet" type="text/css">
{% endblock %}
-
+
{% block footer %}
<div class="rst-versions shift-up" data-toggle="rst-versions" role="note" aria-label="versions">
<a href="http://incubator.apache.org/">
-<img src= "{{pathto('_static/'+ 'apache.jpg' , 1) }}">
+<img src= "{{pathto('_static/'+ 'apache.jpg' , 1) }}">
</a>
-
+
<span class="rst-current-version" data-toggle="rst-current-version">
<span class="fa fa-book"> incubator-singa </span>
v: {{ version }}
@@ -39,6 +39,11 @@
<dd><a href="{{ pathto('../en/index.html', 1) }}">English</a></dd>
<dd><a href="{{ pathto('../zh/index.html', 1) }}">中文</a></dd>
</dl>
+ <dl>
+ <dt>Versions</dt>
+ <dd><a href="http://singa.apache.org/v0.3.0/">0.3</a></dd>
+ </dl>
+
</div>
</div>
diff --git a/doc/build.sh b/doc/build.sh
index db987d6..eb5b90c 100755
--- a/doc/build.sh
+++ b/doc/build.sh
@@ -1,5 +1,22 @@
#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
SPHINXBUILD="sphinx-build"
BUILDDIR="_build"
LANG_ARR=(en zh)
diff --git a/doc/conf.py b/doc/conf.py
index 86dc031..08e391e 100755
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -3,6 +3,22 @@
# incubator-singa documentation build configuration file, created by
# sphinx-quickstart on Sat Jul 9 20:36:57 2016.
#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
# This file is execfile()d with the current directory set to its
# containing dir.
#
@@ -16,9 +32,10 @@
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
-# import os
-# import sys
-# sys.path.insert(0, os.path.abspath('.'))
+import os
+import sys
+sys.path.insert(0, os.path.abspath('.'))
+sys.path.insert(1, os.path.abspath('../build/python/'))
# -- General configuration ------------------------------------------------
from recommonmark.parser import CommonMarkParser
@@ -34,10 +51,8 @@
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
-extensions = [
-
-]
-
+extensions = ['sphinx.ext.autodoc', 'sphinx.ext.napoleon']
+napoleon_google_docstring = True
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
diff --git a/doc/en/_templates/layout.html b/doc/en/_templates/layout.html
index 590e578..2f9ca0d 100755
--- a/doc/en/_templates/layout.html
+++ b/doc/en/_templates/layout.html
@@ -16,7 +16,7 @@
limitations under the License.
#}
{% extends "!layout.html" %}
-
+
{% block extrahead %}
<link href="{{ pathto("_static/style.css", True) }}" rel="stylesheet" type="text/css">
{% endblock %}
@@ -25,7 +25,7 @@
<div class="rst-versions shift-up" data-toggle="rst-versions" role="note" aria-label="versions">
<a href="http://incubator.apache.org/">
-<img src= "{{pathto('_static/'+ 'apache.jpg' , 1) }}">
+<img src= "{{pathto('_static/'+ 'apache.jpg' , 1) }}">
</a>
<span class="rst-current-version" data-toggle="rst-current-version">
@@ -36,18 +36,13 @@
<div class="rst-other-versions">
<dl>
<dd><a href="">English</a></dd>
- <dd><a href="{{pathto('zh/'+ 'index.html' , 1) }}">中文</a></dd>
+ <dd><a href="{{pathto('zh/'+ 'index.html' , 1) }}">中文</a></dd>
<!--dd><a href="/jp/latest/">日本語</a></dd>
<dd><a href="/kr/latest/">한국어</a></dd>
<dd><a href="/it/latest/">Italiano</a></dd>
<dd><a href="/ar/latest/">العربية</a></dd-->
</dl>
</dl>
- <dl>
- <dt>Versions</dt>
- <dd><a href="/{{ language }}/latest/">latest</a></dd>
- <dd><a href="/{{ language }}/0.3.0/">v0.3.0</a></dd>
- </dl>
</div>
</div>
diff --git a/doc/en/community/mail-lists.rst b/doc/en/community/mail-lists.rst
index 02b39de..a170042 100644
--- a/doc/en/community/mail-lists.rst
+++ b/doc/en/community/mail-lists.rst
@@ -1,3 +1,21 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
Project Mailing Lists
=====================
diff --git a/doc/en/community/team-list.rst b/doc/en/community/team-list.rst
index a677aff..abff0a8 100644
--- a/doc/en/community/team-list.rst
+++ b/doc/en/community/team-list.rst
@@ -1,3 +1,21 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
The SINGA Team
==============
@@ -25,7 +43,7 @@
+-------------------+--------------------------------+----------------------------------------------+
| Haibo Chen | hzchenhaibo@corp.netease.com | NetEase |
+-------------------+--------------------------------+----------------------------------------------+
-| Anh Dinh | dinhtta@apache.org | National University of Singapore |
+| Anh Dinh | dinhtta@apache.org | National University of Singapore |
+-------------------+--------------------------------+----------------------------------------------+
| Jinyang Gao | jinyang@apache.org | National University of Singapore |
+-------------------+--------------------------------+----------------------------------------------+
diff --git a/doc/en/develop/schedule.rst b/doc/en/develop/schedule.rst
index 2cf81f1..73d713c 100644
--- a/doc/en/develop/schedule.rst
+++ b/doc/en/develop/schedule.rst
@@ -1,3 +1,21 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
Development Schedule
====================
diff --git a/doc/en/docs.rst b/doc/en/docs.rst
index c1b143b..1b94d02 100644
--- a/doc/en/docs.rst
+++ b/doc/en/docs.rst
@@ -1,3 +1,21 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
Documentation
=============
diff --git a/doc/en/docs/device.rst b/doc/en/docs/device.rst
index 53faf48..57993f9 100644
--- a/doc/en/docs/device.rst
+++ b/doc/en/docs/device.rst
@@ -1,3 +1,21 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
Device
=======
diff --git a/doc/en/docs/index.rst b/doc/en/docs/index.rst
index a2ea540..d6d7516 100644
--- a/doc/en/docs/index.rst
+++ b/doc/en/docs/index.rst
@@ -1,3 +1,21 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
Documentation
=============
diff --git a/doc/en/docs/initializer.rst b/doc/en/docs/initializer.rst
index f334497..6790a8e 100644
--- a/doc/en/docs/initializer.rst
+++ b/doc/en/docs/initializer.rst
@@ -1,3 +1,21 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
Initializer
===========
diff --git a/doc/en/docs/layer.rst b/doc/en/docs/layer.rst
index 62ef3c3..1a576f1 100644
--- a/doc/en/docs/layer.rst
+++ b/doc/en/docs/layer.rst
@@ -1,3 +1,21 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
Layer
======
diff --git a/doc/en/docs/loss.rst b/doc/en/docs/loss.rst
index 27872dd..18c587a 100644
--- a/doc/en/docs/loss.rst
+++ b/doc/en/docs/loss.rst
@@ -1,3 +1,21 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
Loss
=========
diff --git a/doc/en/docs/metric.rst b/doc/en/docs/metric.rst
index 35fa24e..20a7144 100644
--- a/doc/en/docs/metric.rst
+++ b/doc/en/docs/metric.rst
@@ -1,3 +1,21 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
Metric
=========
diff --git a/doc/en/docs/optimizer.rst b/doc/en/docs/optimizer.rst
index 486c01e..e6f1da9 100644
--- a/doc/en/docs/optimizer.rst
+++ b/doc/en/docs/optimizer.rst
@@ -1,3 +1,21 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
Optimizer
=========
diff --git a/doc/en/docs/tensor.rst b/doc/en/docs/tensor.rst
index ff6142e..d9e7f18 100644
--- a/doc/en/docs/tensor.rst
+++ b/doc/en/docs/tensor.rst
@@ -1,3 +1,21 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
Tensor
========
diff --git a/doc/en/docs/utils.rst b/doc/en/docs/utils.rst
index 5306719..4736ce1 100644
--- a/doc/en/docs/utils.rst
+++ b/doc/en/docs/utils.rst
@@ -1,3 +1,21 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
Misc.
=========
diff --git a/doc/en/index.rst b/doc/en/index.rst
index 1bbbe9a..bdf5b1d 100755
--- a/doc/en/index.rst
+++ b/doc/en/index.rst
@@ -1,3 +1,21 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
.. Singa documentation master file, created by
sphinx-quickstart on Sat Jul 9 20:36:57 2016.
You can adapt this file completely to your liking, but it should at least
diff --git a/doc/zh/index.rst b/doc/zh/index.rst
index 4b49d5f..3d59dd3 100644
--- a/doc/zh/index.rst
+++ b/doc/zh/index.rst
@@ -1,3 +1,21 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
SINGA 中文文档
==============
diff --git a/examples/cifar10/README.md b/examples/cifar10/README.md
index 8076347..bd5ed7d 100644
--- a/examples/cifar10/README.md
+++ b/examples/cifar10/README.md
@@ -48,13 +48,13 @@
2. alexnet.cc. It trains the AlexNet model using the CPP APIs on a CudaGPU,
- run.sh
+ ./run.sh
3. alexnet-parallel.cc. It trains the AlexNet model using the CPP APIs on two CudaGPU devices.
The two devices run synchronously to compute the gradients of the mode parameters, which are
averaged on the host CPU device and then be applied to update the parameters.
- run-parallel.sh
+ ./run-parallel.sh
4. vgg-parallel.cc. It train the VGG model using the CPP APIs on two CudaGPU devices similar to alexnet-parallel.cc.
diff --git a/examples/cifar10/resnet.py b/examples/cifar10/resnet.py
index 477c5c7..6b573e9 100644
--- a/examples/cifar10/resnet.py
+++ b/examples/cifar10/resnet.py
@@ -49,7 +49,10 @@
net.add(layer.Merge(name + "-merge"), [br1bn2, split])
-def create_net():
+def create_net(use_cpu=False):
+ if use_cpu:
+ layer.engine = 'singacpp'
+
net = ffnet.FeedForwardNet(loss.SoftmaxCrossEntropy(), metric.Accuracy())
net.add(layer.Conv2D("conv1", 16, 3, 1, pad=1, input_sample_shape=(3, 32, 32)))
net.add(layer.BatchNormalization("bn1"))
diff --git a/examples/cifar10/train.py b/examples/cifar10/train.py
index b08ae3c..d2d70df 100644
--- a/examples/cifar10/train.py
+++ b/examples/cifar10/train.py
@@ -181,5 +181,6 @@
use_cpu=args.use_cpu)
else:
train_x, test_x = normalize_for_alexnet(train_x, test_x)
- net = resnet.create_net()
- train((train_x, train_y, test_x, test_y), net, 200, resnet_lr, 1e-4)
+ net = resnet.create_net(args.use_cpu)
+ train((train_x, train_y, test_x, test_y), net, 200, resnet_lr, 1e-4,
+ use_cpu=args.use_cpu)
diff --git a/rat-excludes b/rat-excludes
index 63cb327..f7f79d2 100644
--- a/rat-excludes
+++ b/rat-excludes
@@ -3,7 +3,6 @@
Makefile.*
configure
.gitignore
-conf/*
doc/*
config/*
\.dirstamp
diff --git a/rat_check b/rat_check
deleted file mode 100644
index 77f0a63..0000000
--- a/rat_check
+++ /dev/null
@@ -1,1108 +0,0 @@
-
-*****************************************************
-Summary
--------
-Generated at: 2016-08-17T13:11:07+08:00
-Notes: 5
-Binaries: 0
-Archives: 0
-Standards: 288
-
-Apache Licensed: 269
-Generated Documents: 0
-
-JavaDocs are generated and so license header is optional
-Generated files do not required license headers
-
-18 Unknown Licenses
-
-*******************************
-
-Unapproved licenses:
-
- ./.gitmodules
- ./.travis.yml
- ./rat_check
- ./cmake/Protobuf.cmake
- ./cmake/Thirdparty/FindOpenCL.cmake
- ./include/singa/utils/cuda_utils.h
- ./include/singa/utils/tinydir.h
- ./lib/cnmem/.git
- ./lib/cnmem/CMakeLists.txt
- ./lib/cnmem/include/cnmem.h
- ./lib/cnmem/src/cnmem.cpp
- ./lib/cnmem/tests/cnmem_tests.cpp
- ./src/core/tensor/distribution.cl
- ./src/python/swig/numpy.i
- ./test/gtest/gtest-all.cc
- ./test/gtest/gtest.h
- ./test/gtest/gtest_main.cc
- ./tool/cpplint.py
-
-*******************************
-
-Archives:
-
-*****************************************************
- Files with Apache License headers will be marked AL
- Binary files (which do not require AL headers) will be marked B
- Compressed archives will be marked A
- Notices, licenses etc will be marked N
- !????? ./.gitmodules
- !????? ./.travis.yml
- AL ./CMakeLists.txt
- N ./DISCLAIMER
- N ./LICENSE
- N ./NOTICE
- N ./RELEASE_NOTES
- AL ./jenkins.sh
- !????? ./rat_check
- AL ./bin/singa-cleanup.sh
- AL ./bin/singa-console.sh
- AL ./bin/singa-env.sh
- AL ./bin/singa-run.sh
- AL ./bin/singa-stop.sh
- AL ./bin/zk-service.sh
- AL ./cmake/Cuda.cmake
- AL ./cmake/Dependencies.cmake
- !????? ./cmake/Protobuf.cmake
- AL ./cmake/Utils.cmake
- AL ./cmake/Templates/singa_config.h.in
- AL ./cmake/Thirdparty/FindCBLAS.cmake
- AL ./cmake/Thirdparty/FindCUDNN.cmake
- AL ./cmake/Thirdparty/FindGlog.cmake
- AL ./cmake/Thirdparty/FindLMDB.cmake
- !????? ./cmake/Thirdparty/FindOpenCL.cmake
- AL ./examples/CMakeLists.txt
- AL ./examples/index.rst
- AL ./examples/char-rnn/sample.py
- AL ./examples/char-rnn/train.py
- AL ./examples/cifar10/CMakeLists.txt
- AL ./examples/cifar10/alexnet-parallel.cc
- AL ./examples/cifar10/alexnet.cc
- AL ./examples/cifar10/alexnet.py
- AL ./examples/cifar10/cifar10.h
- AL ./examples/cifar10/download_data.py
- AL ./examples/cifar10/predict.py
- AL ./examples/cifar10/run-parallel.sh
- AL ./examples/cifar10/run.sh
- AL ./examples/cifar10/train.py
- AL ./examples/cifar10/vgg-parallel.cc
- AL ./examples/cifar10/vgg.py
- AL ./examples/imagenet/CMakeLists.txt
- AL ./examples/imagenet/alexnet.cc
- AL ./examples/imagenet/create_data.sh
- AL ./examples/imagenet/ilsvrc12.cc
- AL ./examples/imagenet/ilsvrc12.h
- AL ./examples/imagenet/run.sh
- AL ./examples/mnist/train.py
- AL ./include/singa/core/common.h
- AL ./include/singa/core/device.h
- AL ./include/singa/core/memory.h
- AL ./include/singa/core/scheduler.h
- AL ./include/singa/core/tensor.h
- AL ./include/singa/io/decoder.h
- AL ./include/singa/io/encoder.h
- AL ./include/singa/io/integer.h
- AL ./include/singa/io/network.h
- AL ./include/singa/io/reader.h
- AL ./include/singa/io/snapshot.h
- AL ./include/singa/io/transformer.h
- AL ./include/singa/io/writer.h
- AL ./include/singa/model/feed_forward_net.h
- AL ./include/singa/model/initializer.h
- AL ./include/singa/model/layer.h
- AL ./include/singa/model/loss.h
- AL ./include/singa/model/metric.h
- AL ./include/singa/model/optimizer.h
- AL ./include/singa/model/updater.h
- AL ./include/singa/utils/channel.h
- !????? ./include/singa/utils/cuda_utils.h
- AL ./include/singa/utils/factory.h
- AL ./include/singa/utils/integer.h
- AL ./include/singa/utils/logging.h
- AL ./include/singa/utils/opencl_utils.h
- AL ./include/singa/utils/safe_queue.h
- AL ./include/singa/utils/singleton.h
- AL ./include/singa/utils/string.h
- AL ./include/singa/utils/timer.h
- !????? ./include/singa/utils/tinydir.h
- !????? ./lib/cnmem/.git
- !????? ./lib/cnmem/CMakeLists.txt
- N ./lib/cnmem/LICENSE
- !????? ./lib/cnmem/include/cnmem.h
- !????? ./lib/cnmem/src/cnmem.cpp
- !????? ./lib/cnmem/tests/cnmem_tests.cpp
- AL ./src/CMakeLists.txt
- AL ./src/core/device/cpp_cpu.cc
- AL ./src/core/device/cuda_gpu.cc
- AL ./src/core/device/device.cc
- AL ./src/core/device/opencl_device.cc
- AL ./src/core/device/platform.cc
- AL ./src/core/memory/memory.cc
- AL ./src/core/scheduler/scheduler.cc
- !????? ./src/core/tensor/distribution.cl
- AL ./src/core/tensor/math_kernel.cu
- AL ./src/core/tensor/math_kernel.h
- AL ./src/core/tensor/sparse_tensor.cc
- AL ./src/core/tensor/tensor.cc
- AL ./src/core/tensor/tensor_math.h
- AL ./src/core/tensor/tensor_math_cpp.h
- AL ./src/core/tensor/tensor_math_cuda.h
- AL ./src/core/tensor/tensor_math_opencl.cl
- AL ./src/core/tensor/tensor_math_opencl.h
- AL ./src/io/binfile_reader.cc
- AL ./src/io/binfile_writer.cc
- AL ./src/io/csv_decoder.cc
- AL ./src/io/csv_encoder.cc
- AL ./src/io/image_transformer.cc
- AL ./src/io/jpg_decoder.cc
- AL ./src/io/jpg_encoder.cc
- AL ./src/io/lmdb_reader.cc
- AL ./src/io/lmdb_writer.cc
- AL ./src/io/snapshot.cc
- AL ./src/io/textfile_reader.cc
- AL ./src/io/textfile_writer.cc
- AL ./src/io/network/endpoint.cc
- AL ./src/io/network/message.cc
- AL ./src/model/feed_forward_net.cc
- AL ./src/model/rnn.cc
- AL ./src/model/layer/activation.cc
- AL ./src/model/layer/activation.h
- AL ./src/model/layer/batchnorm.cc
- AL ./src/model/layer/batchnorm.h
- AL ./src/model/layer/convolution.cc
- AL ./src/model/layer/convolution.h
- AL ./src/model/layer/cudnn_activation.cc
- AL ./src/model/layer/cudnn_activation.h
- AL ./src/model/layer/cudnn_batchnorm.cc
- AL ./src/model/layer/cudnn_batchnorm.h
- AL ./src/model/layer/cudnn_convolution.cc
- AL ./src/model/layer/cudnn_convolution.h
- AL ./src/model/layer/cudnn_dropout.cc
- AL ./src/model/layer/cudnn_dropout.h
- AL ./src/model/layer/cudnn_lrn.cc
- AL ./src/model/layer/cudnn_lrn.h
- AL ./src/model/layer/cudnn_pooling.cc
- AL ./src/model/layer/cudnn_pooling.h
- AL ./src/model/layer/cudnn_rnn.cc
- AL ./src/model/layer/cudnn_rnn.h
- AL ./src/model/layer/cudnn_softmax.cc
- AL ./src/model/layer/cudnn_softmax.h
- AL ./src/model/layer/cudnn_utils.h
- AL ./src/model/layer/dense.cc
- AL ./src/model/layer/dense.h
- AL ./src/model/layer/dropout.cc
- AL ./src/model/layer/dropout.h
- AL ./src/model/layer/flatten.cc
- AL ./src/model/layer/flatten.h
- AL ./src/model/layer/lrn.cc
- AL ./src/model/layer/lrn.h
- AL ./src/model/layer/pooling.cc
- AL ./src/model/layer/pooling.h
- AL ./src/model/layer/prelu.cc
- AL ./src/model/layer/prelu.h
- AL ./src/model/layer/rnn.cc
- AL ./src/model/layer/rnn.h
- AL ./src/model/layer/softmax.cc
- AL ./src/model/layer/softmax.h
- AL ./src/model/loss/mse.cc
- AL ./src/model/loss/softmax_cross_entropy.cc
- AL ./src/model/metric/accuracy.cc
- AL ./src/model/optimizer/adagrad.cc
- AL ./src/model/optimizer/local_all_reduce.cc
- AL ./src/model/optimizer/nesterov.cc
- AL ./src/model/optimizer/optimizer.cc
- AL ./src/model/optimizer/rmsprop.cc
- AL ./src/model/optimizer/sgd.cc
- AL ./src/model/updater/local_updater.cc
- AL ./src/model/updater/updater.cc
- AL ./src/proto/core.proto
- AL ./src/proto/io.proto
- AL ./src/proto/model.proto
- AL ./src/python/setup.py.in
- AL ./src/python/singa/__init__.py
- AL ./src/python/singa/command.py
- AL ./src/python/singa/device.py
- AL ./src/python/singa/initializer.py
- AL ./src/python/singa/layer.py
- AL ./src/python/singa/loss.py
- AL ./src/python/singa/metric.py
- AL ./src/python/singa/model.py
- AL ./src/python/singa/net.py
- AL ./src/python/singa/optimizer.py
- AL ./src/python/singa/tensor.py
- AL ./src/python/singa/utils.py
- AL ./src/python/swig/core_device.i
- AL ./src/python/swig/core_tensor.i
- AL ./src/python/swig/model_layer.i
- AL ./src/python/swig/model_loss.i
- AL ./src/python/swig/model_metric.i
- AL ./src/python/swig/model_optimizer.i
- !????? ./src/python/swig/numpy.i
- AL ./src/python/swig/singa.i
- ./src/python/swig/singa_wrap.cxx
- AL ./src/utils/channel.cc
- AL ./src/utils/logging.cc
- AL ./src/utils/opencl_utils.cc
- AL ./test/CMakeLists.txt
- AL ./test/gtest/CMakeLists.txt
- !????? ./test/gtest/gtest-all.cc
- !????? ./test/gtest/gtest.h
- !????? ./test/gtest/gtest_main.cc
- AL ./test/python/test_layer.py
- AL ./test/python/test_optimizer.py
- AL ./test/python/test_tensor.py
- AL ./test/singa/test_accuracy.cc
- AL ./test/singa/test_activation.cc
- AL ./test/singa/test_adagrad.cc
- AL ./test/singa/test_batchnorm.cc
- AL ./test/singa/test_binfile_rw.cc
- AL ./test/singa/test_channel.cc
- AL ./test/singa/test_convolution.cc
- AL ./test/singa/test_cpp_cpu.cc
- AL ./test/singa/test_cross_entropy.cc
- AL ./test/singa/test_csv.cc
- AL ./test/singa/test_cudnn_activation.cc
- AL ./test/singa/test_cudnn_batchnorm.cc
- AL ./test/singa/test_cudnn_convolution.cc
- AL ./test/singa/test_cudnn_dropout.cc
- AL ./test/singa/test_cudnn_lrn.cc
- AL ./test/singa/test_cudnn_pooling.cc
- AL ./test/singa/test_cudnn_rnn.cc
- AL ./test/singa/test_cudnn_softmax.cc
- AL ./test/singa/test_dense.cc
- AL ./test/singa/test_dropout.cc
- AL ./test/singa/test_ep.cc
- AL ./test/singa/test_flatten.cc
- AL ./test/singa/test_image_transformer.cc
- AL ./test/singa/test_initializer.cc
- AL ./test/singa/test_jpg.cc
- AL ./test/singa/test_layer.cc
- AL ./test/singa/test_lmdb_rw.cc
- AL ./test/singa/test_logging.cc
- AL ./test/singa/test_lrn.cc
- AL ./test/singa/test_memory.cc
- AL ./test/singa/test_mse.cc
- AL ./test/singa/test_nesterov.cc
- AL ./test/singa/test_opencl.cc
- AL ./test/singa/test_platform.cc
- AL ./test/singa/test_pooling.cc
- AL ./test/singa/test_prelu.cc
- AL ./test/singa/test_rmsprop.cc
- AL ./test/singa/test_sgd.cc
- AL ./test/singa/test_snapshot.cc
- AL ./test/singa/test_softmax.cc
- AL ./test/singa/test_tensor.cc
- AL ./test/singa/test_tensor_math.cc
- AL ./test/singa/test_textfile_rw.cc
- AL ./test/singa/test_timer.cc
- AL ./thirdparty/install.sh
- !????? ./tool/cpplint.py
- AL ./tool/graph.py
- AL ./tool/node.sh
- AL ./tool/docker/mesos/.bashrc
- AL ./tool/docker/mesos/Dockerfile
- AL ./tool/docker/mesos/core-site.xml
- AL ./tool/docker/mesos/hdfs-site.xml
- AL ./tool/docker/mesos/install.sh
- AL ./tool/docker/mesos/mapred-site.xml
- AL ./tool/docker/mesos/yarn-site.xml
- AL ./tool/docker/singa/.bashrc
- AL ./tool/docker/singa/Dockerfile
- AL ./tool/docker/singa/Dockerfile_gpu
- AL ./tool/mesos/scheduler.proto
- AL ./tool/mesos/singa_scheduler.cc
- AL ./tool/python/singa.py
- AL ./tool/python/examples/__init__.py
- AL ./tool/python/examples/cifar10_cnn.py
- AL ./tool/python/examples/cifar10_cnn_cudnn.py
- AL ./tool/python/examples/cifar10_cnn_parameter.py
- AL ./tool/python/examples/mnist_ae.py
- AL ./tool/python/examples/mnist_mlp.py
- AL ./tool/python/examples/mnist_mlp_parameter.py
- AL ./tool/python/examples/mnist_mlp_test.py
- AL ./tool/python/examples/mnist_rbm1.py
- AL ./tool/python/examples/mnist_rbm2.py
- AL ./tool/python/examples/mnist_rbm3.py
- AL ./tool/python/examples/mnist_rbm4.py
- AL ./tool/python/examples/train_cifar10.py
- AL ./tool/python/examples/train_mnist.py
- AL ./tool/python/examples/datasets/__init__.py
- AL ./tool/python/examples/datasets/cifar10.py
- AL ./tool/python/examples/datasets/mnist.py
- AL ./tool/python/singa/__init__.py
- AL ./tool/python/singa/driver.i
- AL ./tool/python/singa/generatepy.sh
- AL ./tool/python/singa/initializations.py
- AL ./tool/python/singa/layer.py
- AL ./tool/python/singa/model.py
- AL ./tool/python/singa/parameter.py
- AL ./tool/python/singa/utils/__init__.py
- AL ./tool/python/singa/utils/message.py
- AL ./tool/python/singa/utils/utility.py
-
-*****************************************************
- Printing headers for files without AL header...
-
-
-=======================================================================
-==./.gitmodules
-=======================================================================
-[submodule "lib/cnmem"]
- path = lib/cnmem
- url = https://github.com/NVIDIA/cnmem.git
-
-=======================================================================
-==./.travis.yml
-=======================================================================
-sudo: required
-language: cpp
-compiler: gcc
-dist: trusty
-
-before_install:
- - sudo apt-get -qq update
- - sudo apt-get install -qq -y libopenblas-dev libgoogle-glog-dev libprotobuf-dev protobuf-compiler
- - sudo apt-get install -qq -y opencl-headers ocl-icd-*
- - wget https://github.com/KhronosGroup/OpenCL-CLHPP/releases/download/v2.0.9/cl2.hpp
- - sudo mv cl2.hpp /usr/include/CL/
-#- sudo apt-get install -qq libgtest-dev
-
-before_script:
- - mkdir build && cd build
- - cmake .. -DUSE_CUDA=OFF -DUSE_CUDNN=OFF -DUSE_PYTHON=OFF -DBUILD_OPENCL_TESTS=OFF
-
-script:
- - make
- - ./bin/test_singa --gtest_output=xml:./../gtest.xml
-
-
-=======================================================================
-==./rat_check
-=======================================================================
-
-=======================================================================
-==./cmake/Protobuf.cmake
-=======================================================================
-# This script is taken from
-# https://github.com/Kitware/CMake/blob/master/Modules/FindProtobuf.cmake
-# and modified to our compilation.
-
-function(PROTOBUF_GENERATE_PYTHON OUTPUT)
- if(NOT ARGN)
- message(SEND_ERROR "Error: PROTOBUF_GENERATE_PYTHON() called
- without any proto files")
- return()
- endif(NOT ARGN)
-
- set(${OUTPUT})
- foreach(FIL ${ARGN})
- get_filename_component(ABS_FIL ${FIL} ABSOLUTE)
- get_filename_component(FIL_WE ${FIL} NAME_WE)
- get_filename_component(PATH ${FIL} PATH)
-
- list(APPEND ${OUTPUT} "${CMAKE_BINARY_DIR}/python/singa/proto/${FIL_WE}_pb2.py")
-
- add_custom_command(
- OUTPUT "${CMAKE_BINARY_DIR}/python/singa/proto/${FIL_WE}_pb2.py"
- COMMAND ${PROTOBUF_PROTOC_EXECUTABLE}
- ARGS --python_out ${CMAKE_BINARY_DIR}/python/singa/proto
- --proto_path ${PATH} ${ABS_FIL}
- DEPENDS ${ABS_FIL}
- COMMENT "Running Python protocol buffer compiler on ${FIL}" VERBATIM)
- endforeach()
-
- set_source_files_properties(${${SRCS}} ${${HDRS}} PROPERTIES GENERATED TRUE)
- set(${OUTPUT} ${${OUTPUT}} PARENT_SCOPE)
-endfunction()
-
-=======================================================================
-==./cmake/Thirdparty/FindOpenCL.cmake
-=======================================================================
-# This script was taken from https://github.com/elhigu/cmake-findopencl
-# and modified to support finding OpenCL 2.x C++ bindings.
-
-# Find OpenCL
-#
-# To set manually the paths, define these environment variables:
-# OpenCL_INCPATH - Include path (e.g. OpenCL_INCPATH=/opt/cuda/4.0/cuda/include)
-# OpenCL_LIBPATH - Library path (e.h. OpenCL_LIBPATH=/usr/lib64/nvidia)
-#
-# Once done this will define
-# OPENCL_FOUND - system has OpenCL
-# OPENCL_INCLUDE_DIRS - the OpenCL include directory
-# OPENCL_LIBRARIES - link these to use OpenCL
-# OPENCL_HAS_CPP_BINDINGS - system has also cl2.hpp
-
-FIND_PACKAGE(PackageHandleStandardArgs)
-
-SET (OPENCL_VERSION_STRING "0.1.0")
-SET (OPENCL_VERSION_MAJOR 0)
-SET (OPENCL_VERSION_MINOR 1)
-SET (OPENCL_VERSION_PATCH 0)
-
-IF (APPLE)
-
- # IF OpenCL_LIBPATH is given use it and don't use default path
- IF (DEFINED ENV{OpenCL_LIBPATH})
- FIND_LIBRARY(OPENCL_LIBRARIES OpenCL PATHS ENV OpenCL_LIBPATH NO_DEFAULT_PATH)
- ELSE ()
- FIND_LIBRARY(OPENCL_LIBRARIES OpenCL DOC "OpenCL lib for OSX")
- ENDIF ()
-
- # IF OpenCL_INCPATH is given use it and find for CL/cl.h and OpenCL/cl.h do not try to find default paths
- IF (DEFINED ENV{OpenCL_INCPATH})
- FIND_PATH(OPENCL_INCLUDE_DIRS CL/cl.h OpenCL/cl.h PATHS ENV OpenCL_INCPATH NO_DEFAULT_PATH)
- FIND_PATH(_OPENCL_CPP_INCLUDE_DIRS CL/cl2.hpp OpenCL/cl2.hpp PATHS ${OPENCL_INCLUDE_DIRS} NO_DEFAULT_PATH)
- ELSE ()
- FIND_PATH(OPENCL_INCLUDE_DIRS OpenCL/cl.h DOC "Include for OpenCL on OSX")
- FIND_PATH(_OPENCL_CPP_INCLUDE_DIRS OpenCL/cl2.hpp DOC "Include for OpenCL CPP bindings on OSX")
- ENDIF ()
-
-ELSE (APPLE)
-
- IF (WIN32)
-
- # Find OpenCL includes and libraries from environment variables provided by vendor
- SET(OPENCL_INCLUDE_SEARCH_PATHS)
- SET(OPENCL_LIBRARY_SEARCH_PATHS)
- SET(OPENCL_LIBRARY_64_SEARCH_PATHS)
-
- # Nvidia
-
-=======================================================================
-==./include/singa/utils/cuda_utils.h
-=======================================================================
-// from caffe include/caffe/util/device_alternative.hpp
-#ifndef SINGA_UTILS_CUDA_UTILS_H_
-#define SINGA_UTILS_CUDA_UTILS_H_
-
-#include "singa/singa_config.h"
-#ifdef USE_CUDA
-#include <cublas_v2.h>
-#include <cuda.h>
-#include <cuda_runtime.h>
-#include <curand.h>
-
-inline const char* cublasGetErrorString(cublasStatus_t error) {
- switch (error) {
- case CUBLAS_STATUS_SUCCESS:
- return "CUBLAS_STATUS_SUCCESS";
- case CUBLAS_STATUS_NOT_INITIALIZED:
- return "CUBLAS_STATUS_NOT_INITIALIZED";
- case CUBLAS_STATUS_ALLOC_FAILED:
- return "CUBLAS_STATUS_ALLOC_FAILED";
- case CUBLAS_STATUS_INVALID_VALUE:
- return "CUBLAS_STATUS_INVALID_VALUE";
- case CUBLAS_STATUS_ARCH_MISMATCH:
- return "CUBLAS_STATUS_ARCH_MISMATCH";
- case CUBLAS_STATUS_MAPPING_ERROR:
- return "CUBLAS_STATUS_MAPPING_ERROR";
- case CUBLAS_STATUS_EXECUTION_FAILED:
- return "CUBLAS_STATUS_EXECUTION_FAILED";
- case CUBLAS_STATUS_INTERNAL_ERROR:
- return "CUBLAS_STATUS_INTERNAL_ERROR";
-#if CUDA_VERSION >= 6000
- case CUBLAS_STATUS_NOT_SUPPORTED:
- return "CUBLAS_STATUS_NOT_SUPPORTED";
-#endif
-#if CUDA_VERSION >= 6050
- case CUBLAS_STATUS_LICENSE_ERROR:
- return "CUBLAS_STATUS_LICENSE_ERROR";
-#endif
- }
- return "Unknown cublas status";
-}
-
-inline const char* curandGetErrorString(curandStatus_t error) {
- switch (error) {
- case CURAND_STATUS_SUCCESS:
- return "CURAND_STATUS_SUCCESS";
- case CURAND_STATUS_VERSION_MISMATCH:
- return "CURAND_STATUS_VERSION_MISMATCH";
- case CURAND_STATUS_NOT_INITIALIZED:
- return "CURAND_STATUS_NOT_INITIALIZED";
- case CURAND_STATUS_ALLOCATION_FAILED:
-
-=======================================================================
-==./include/singa/utils/tinydir.h
-=======================================================================
-/*
-Copyright (c) 2013-2014, Cong Xu, Baudouin Feildel
-All rights reserved.
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are met:
-
-1. Redistributions of source code must retain the above copyright notice, this
- list of conditions and the following disclaimer.
-2. Redistributions in binary form must reproduce the above copyright notice,
- this list of conditions and the following disclaimer in the documentation
- and/or other materials provided with the distribution.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
-ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
-DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
-ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
-(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
-LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
-ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-*/
-#ifndef TINYDIR_H
-#define TINYDIR_H
-
-#include <errno.h>
-#include <stdlib.h>
-#include <string.h>
-#ifdef _WIN32
-#define WIN32_LEAN_AND_MEAN
-#include <windows.h>
-#ifdef _MSC_VER
-#pragma warning (disable : 4996)
-#endif
-#else
-#include <dirent.h>
-#include <libgen.h>
-#include <sys/stat.h>
-#endif
-
-
-/* types */
-
-#define _TINYDIR_PATH_MAX 4096
-#ifdef _WIN32
-/* extra chars for the "\\*" mask */
-#define _TINYDIR_PATH_EXTRA 2
-#else
-
-=======================================================================
-==./lib/cnmem/.git
-=======================================================================
-gitdir: ../../.git/modules/lib/cnmem
-
-=======================================================================
-==./lib/cnmem/CMakeLists.txt
-=======================================================================
-# CMakeLists to build the cnmem library.
-cmake_minimum_required(VERSION 2.8.8)
-project(cnmem)
-
-# We need CUDA to build that library.
-find_package(CUDA QUIET REQUIRED)
-include_directories(${CUDA_INCLUDE_DIRS})
-
-# Rules to build the cnmem library.
-include_directories(include)
-add_definitions(-DCNMEM_DLLEXPORT)
-add_library(cnmem SHARED src/cnmem.cpp)
-set_target_properties(cnmem PROPERTIES VERSION 1.0.0 SOVERSION 1)
-target_link_libraries(cnmem LINK_PUBLIC ${CUDA_LIBRARIES})
-install(TARGETS cnmem RUNTIME DESTINATION bin ARCHIVE DESTINATION lib LIBRARY DESTINATION lib)
-install(FILES include/cnmem.h DESTINATION include)
-
-# Add the tests.
-if(WITH_TESTS)
-
- # Get Google tests.
- find_package(GTest QUIET REQUIRED)
- include_directories(${GTEST_INCLUDE_DIRS})
-
- # Build the executable.
- add_executable(cnmem_tests tests/cnmem_tests.cpp)
- if(MSVC)
- if(MSVC_VERSION GREATER 1700) # Visual Studio 11 or more.
- add_definitions(-DUSE_CPP_11)
- endif(MSVC_VERSION GREATER 1700)
- endif(MSVC)
- if(CMAKE_COMPILER_IS_GNUCC)
- add_definitions(-std=c++11 -DUSE_CPP_11)
- endif(CMAKE_COMPILER_IS_GNUCC)
- target_link_libraries(cnmem_tests LINK_PUBLIC cnmem ${CUDA_LIBRARIES} ${GTEST_LIBRARIES} -lpthread)
- install(TARGETS cnmem_tests RUNTIME DESTINATION bin)
-
- # On Windows, we copy the Google test DLL to the bin folder.
- if(MSVC)
- get_filename_component(gtest_dll_path ${GTEST_LIBRARIES} DIRECTORY)
- install(FILES ${gtest_dll_path}/gtest.dll DESTINATION bin)
- endif(MSVC)
-
-endif(WITH_TESTS)
-
-
-=======================================================================
-==./lib/cnmem/include/cnmem.h
-=======================================================================
-/* **********************************************************************
- * Copyright (c) 2015, NVIDIA CORPORATION. All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions
- * are met:
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in the
- * documentation and/or other materials provided with the distribution.
- * * Neither the name of NVIDIA CORPORATION nor the names of its
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
- * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
- * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
- * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
- * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
- * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
- * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- * ********************************************************************** */
-#pragma once
-
-#ifdef __cplusplus
-#include "cstdio"
-#else
-#include "stdio.h"
-#endif
-#include "cuda_runtime_api.h"
-
-#if defined(_MSC_VER) || defined(WIN32)
-#ifdef CNMEM_DLLEXPORT
-#define CNMEM_API __declspec(dllexport)
-#else
-#define CNMEM_API __declspec(dllimport)
-#endif
-#else
-#ifdef CNMEM_DLLEXPORT
-#define CNMEM_API __attribute__((visibility ("default")))
-#else
-#define CNMEM_API
-#endif
-#endif
-
-
-=======================================================================
-==./lib/cnmem/src/cnmem.cpp
-=======================================================================
-///////////////////////////////////////////////////////////////////////////////////////////////////
-// Copyright (c) 2015, NVIDIA CORPORATION. All rights reserved.
-//
-// Redistribution and use in source and binary forms, with or without
-// modification, are permitted provided that the following conditions
-// are met:
-// * Redistributions of source code must retain the above copyright
-// notice, this list of conditions and the following disclaimer.
-// * Redistributions in binary form must reproduce the above copyright
-// notice, this list of conditions and the following disclaimer in the
-// documentation and/or other materials provided with the distribution.
-// * Neither the name of NVIDIA CORPORATION nor the names of its
-// contributors may be used to endorse or promote products derived
-// from this software without specific prior written permission.
-//
-// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
-// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
-// PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
-// CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-// EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
-// PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
-// OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-///////////////////////////////////////////////////////////////////////////////////////////////////
-
-#include "cnmem.h"
-#include <cstddef>
-#include <vector>
-#include <cuda_runtime_api.h>
-
-#if !defined(WIN32) && defined(_MSC_VER)
-#define WIN32
-#endif
-
-#ifdef WIN32
-#include <Windows.h>
-#else
-#include <pthread.h>
-#endif
-
-#if defined(__SIZEOF_POINTER__) && __SIZEOF_POINTER__ == 4 // ARMv7 is the only 32-bit target that we support.
-#define CNMEM_BUILD_WITH_32_BIT_POINTERS
-#endif
-
-#define CNMEM_GRANULARITY 512
-
-///////////////////////////////////////////////////////////////////////////////////////////////////
-
-=======================================================================
-==./lib/cnmem/tests/cnmem_tests.cpp
-=======================================================================
-///////////////////////////////////////////////////////////////////////////////////////////////////
-// Copyright (c) 2015, NVIDIA CORPORATION. All rights reserved.
-//
-// Redistribution and use in source and binary forms, with or without
-// modification, are permitted provided that the following conditions
-// are met:
-// * Redistributions of source code must retain the above copyright
-// notice, this list of conditions and the following disclaimer.
-// * Redistributions in binary form must reproduce the above copyright
-// notice, this list of conditions and the following disclaimer in the
-// documentation and/or other materials provided with the distribution.
-// * Neither the name of NVIDIA CORPORATION nor the names of its
-// contributors may be used to endorse or promote products derived
-// from this software without specific prior written permission.
-//
-// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
-// EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
-// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
-// PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
-// CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-// EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-// PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
-// PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
-// OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-///////////////////////////////////////////////////////////////////////////////////////////////////
-
-#include <gtest/gtest.h>
-#include <cnmem.h>
-#include <fstream>
-#ifdef USE_CPP_11
-#include <thread>
-#endif
-
-///////////////////////////////////////////////////////////////////////////////////////////////////
-
-static std::size_t getFreeMemory() {
- cudaFree(0);
- std::size_t freeMem, totalMem;
- cudaMemGetInfo(&freeMem, &totalMem);
- return freeMem;
-}
-
-class CnmemTest : public ::testing::Test {
- /// We determine the amount of free memory.
- std::size_t mFreeMem;
-
-protected:
- /// Do we test memory leaks.
-
-=======================================================================
-==./src/core/tensor/distribution.cl
-=======================================================================
-// This code is adapted from https://github.com/amd/OpenCL-caffe/blob/stable/src/caffe/ocl/random.cl
-
-//Note: random generator has two parts
-//first part: the open sourced threefy random generator kernel from DE Shaw Research
-//second part. we wrap the kernel up to generate uniform, bernoulli and gaussion distribution generators.
-
-//begin: the open sourced random generator from DE Shaw Research
-//https://www.deshawresearch.com/resources_random123.html
-typedef uint uint32_t;
-
-struct r123array4x32 {
- uint32_t v[4];
-};
-
-enum r123_enum_threefry32x4 {
- R_32x4_0_0 = 10,
- R_32x4_0_1 = 26,
- R_32x4_1_0 = 11,
- R_32x4_1_1 = 21,
- R_32x4_2_0 = 13,
- R_32x4_2_1 = 27,
- R_32x4_3_0 = 23,
- R_32x4_3_1 = 5,
- R_32x4_4_0 = 6,
- R_32x4_4_1 = 20,
- R_32x4_5_0 = 17,
- R_32x4_5_1 = 11,
- R_32x4_6_0 = 25,
- R_32x4_6_1 = 10,
- R_32x4_7_0 = 18,
- R_32x4_7_1 = 20
-};
-
-inline uint32_t RotL_32(uint32_t x, unsigned int N) {
- return (x << (N & 31)) | (x >> ((32 - N) & 31));
-}
-
-typedef struct r123array4x32 threefry4x32_ctr_t;
-typedef struct r123array4x32 threefry4x32_key_t;
-typedef struct r123array4x32 threefry4x32_ukey_t;
-
-inline threefry4x32_ctr_t threefry4x32_R(unsigned int Nrounds, threefry4x32_ctr_t in, threefry4x32_key_t k) {
- threefry4x32_ctr_t X;
- uint32_t ks[4 + 1];
- int i;
- ks[4] = 0x1BD11BDA;
-
- {
- ks[0] = k.v[0];
- X.v[0] = in.v[0];
-
-=======================================================================
-==./src/python/swig/numpy.i
-=======================================================================
-/* -*- C -*- (not really, but good for syntax highlighting) */
-
-/*
- * Copyright (c) 2005-2015, NumPy Developers.
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are
- * met:
- *
- * * Redistributions of source code must retain the above copyright
- * notice, this list of conditions and the following disclaimer.
- *
- * * Redistributions in binary form must reproduce the above
- * copyright notice, this list of conditions and the following
- * disclaimer in the documentation and/or other materials provided
- * with the distribution.
- *
- * * Neither the name of the NumPy Developers nor the names of any
- * contributors may be used to endorse or promote products derived
- * from this software without specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- */
-
-#ifdef SWIGPYTHON
-
-%{
-#ifndef SWIG_FILE_WITH_INIT
-#define NO_IMPORT_ARRAY
-#endif
-#include "stdio.h"
-#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
-#include <numpy/arrayobject.h>
-%}
-
-/**********************************************************************/
-
-%fragment("NumPy_Backward_Compatibility", "header")
-{
-
-=======================================================================
-==./test/gtest/gtest-all.cc
-=======================================================================
-// Copyright 2008, Google Inc.
-// All rights reserved.
-//
-// Redistribution and use in source and binary forms, with or without
-// modification, are permitted provided that the following conditions are
-// met:
-//
-// * Redistributions of source code must retain the above copyright
-// notice, this list of conditions and the following disclaimer.
-// * Redistributions in binary form must reproduce the above
-// copyright notice, this list of conditions and the following disclaimer
-// in the documentation and/or other materials provided with the
-// distribution.
-// * Neither the name of Google Inc. nor the names of its
-// contributors may be used to endorse or promote products derived from
-// this software without specific prior written permission.
-//
-// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-//
-// Author: mheule@google.com (Markus Heule)
-//
-// Google C++ Testing Framework (Google Test)
-//
-// Sometimes it's desirable to build Google Test by compiling a single file.
-// This file serves this purpose.
-
-// This line ensures that gtest.h can be compiled on its own, even
-// when it's fused.
-#include "gtest/gtest.h"
-
-// The following lines pull in the real gtest *.cc files.
-// Copyright 2005, Google Inc.
-// All rights reserved.
-//
-// Redistribution and use in source and binary forms, with or without
-// modification, are permitted provided that the following conditions are
-// met:
-//
-// * Redistributions of source code must retain the above copyright
-// notice, this list of conditions and the following disclaimer.
-
-=======================================================================
-==./test/gtest/gtest.h
-=======================================================================
-// Copyright 2005, Google Inc.
-// All rights reserved.
-//
-// Redistribution and use in source and binary forms, with or without
-// modification, are permitted provided that the following conditions are
-// met:
-//
-// * Redistributions of source code must retain the above copyright
-// notice, this list of conditions and the following disclaimer.
-// * Redistributions in binary form must reproduce the above
-// copyright notice, this list of conditions and the following disclaimer
-// in the documentation and/or other materials provided with the
-// distribution.
-// * Neither the name of Google Inc. nor the names of its
-// contributors may be used to endorse or promote products derived from
-// this software without specific prior written permission.
-//
-// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-//
-// Author: wan@google.com (Zhanyong Wan)
-//
-// The Google C++ Testing Framework (Google Test)
-//
-// This header file defines the public API for Google Test. It should be
-// included by any test program that uses Google Test.
-//
-// IMPORTANT NOTE: Due to limitation of the C++ language, we have to
-// leave some internal implementation details in this header file.
-// They are clearly marked by comments like this:
-//
-// // INTERNAL IMPLEMENTATION - DO NOT USE IN A USER PROGRAM.
-//
-// Such code is NOT meant to be used by a user directly, and is subject
-// to CHANGE WITHOUT NOTICE. Therefore DO NOT DEPEND ON IT in a user
-// program!
-//
-// Acknowledgment: Google Test borrowed the idea of automatic test
-// registration from Barthelemy Dagenais' (barthelemy@prologique.com)
-// easyUnit framework.
-
-
-=======================================================================
-==./test/gtest/gtest_main.cc
-=======================================================================
-// Copyright 2006, Google Inc.
-// All rights reserved.
-//
-// Redistribution and use in source and binary forms, with or without
-// modification, are permitted provided that the following conditions are
-// met:
-//
-// * Redistributions of source code must retain the above copyright
-// notice, this list of conditions and the following disclaimer.
-// * Redistributions in binary form must reproduce the above
-// copyright notice, this list of conditions and the following disclaimer
-// in the documentation and/or other materials provided with the
-// distribution.
-// * Neither the name of Google Inc. nor the names of its
-// contributors may be used to endorse or promote products derived from
-// this software without specific prior written permission.
-//
-// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-#include <stdio.h>
-
-#include "gtest/gtest.h"
-
-GTEST_API_ int main(int argc, char **argv) {
- printf("Running main() from gtest_main.cc\n");
- testing::InitGoogleTest(&argc, argv);
- return RUN_ALL_TESTS();
-}
-
-=======================================================================
-==./tool/cpplint.py
-=======================================================================
-#!/usr/bin/env python
-#
-# Copyright (c) 2009 Google Inc. All rights reserved.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are
-# met:
-#
-# * Redistributions of source code must retain the above copyright
-# notice, this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above
-# copyright notice, this list of conditions and the following disclaimer
-# in the documentation and/or other materials provided with the
-# distribution.
-# * Neither the name of Google Inc. nor the names of its
-# contributors may be used to endorse or promote products derived from
-# this software without specific prior written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-"""Does google-lint on c++ files.
-
-The goal of this script is to identify places in the code that *may*
-be in non-compliance with google style. It does not attempt to fix
-up these problems -- the point is to educate. It does also not
-attempt to find all problems, or to ensure that everything it does
-find is legitimately a problem.
-
-In particular, we can get very confused by /* and // inside strings!
-We do a small hack, which is to ignore //'s with "'s after them on the
-same line, but it is far from perfect (in either direction).
-"""
-
-import codecs
-import copy
-import getopt
-import math # for log
-import os
-import re
-import sre_compile
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 174f05e..cc1ee0c 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -6,19 +6,19 @@
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
-#
+#
# http://www.apache.org/licenses/LICENSE-2.0
-#
+#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
-#
+#
-# generate protobuf sources
+# generate protobuf sources
-FILE(GLOB proto_files proto/*.proto)
+FILE(GLOB proto_files proto/*.proto)
protobuf_generate_cpp(proto_srcs proto_hdrs ${proto_files})
IF (USE_PYTHON)
protobuf_generate_python(proto_pys ${proto_files})
@@ -111,9 +111,9 @@
file(GLOB_RECURSE python_source_files RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} *.py)
create_symlinks(${python_source_files})
-
+
SET(python_cxxs "${core_source};${io_source};${model_source};${utils_source}")
- ADD_LIBRARY(_singa_wrap SHARED "${python_srcs} ${python_cxxs} ${cuda_objs}")
+ ADD_LIBRARY(_singa_wrap SHARED ${python_srcs} ${python_cxxs} ${cuda_objs})
SET(WRAPPER_LINKER_LIBS "${PREVIOUS_LINKER_LIBS}")
TARGET_LINK_LIBRARIES(_singa_wrap ${WRAPPER_LINKER_LIBS})
TARGET_INCLUDE_DIRECTORIES(_singa_wrap PRIVATE ${PYTHON_INCLUDE_DIRS})
diff --git a/tool/python/README.md b/tool/python/README.md
deleted file mode 100644
index 8c90cfe..0000000
--- a/tool/python/README.md
+++ /dev/null
@@ -1,375 +0,0 @@
-# Python Binding
-
----
-
-Python binding provides APIs for configuring a training job following
-[keras](http://keras.io/), including the configuration of neural net, training
-algorithm, etc. It replaces the configuration file (e.g., *job.conf*) in
-protobuf format, which is typically long and error-prone to prepare. In later
-version, we will add python functions to interact with the layer and neural net
-objects, which would enable users to train and debug their models
-interactively.
-
-Here is the layout of python related code,
-
- SINGAROOT/tool/python
- |-- pb2 (has job_pb2.py)
- |-- singa
- |-- model.py
- |-- layer.py
- |-- parameter.py
- |-- initialization.py
- |-- utils
- |-- utility.py
- |-- message.py
- |-- examples
- |-- cifar10_cnn.py, mnist_mlp.py, , mnist_rbm1.py, mnist_ae.py, etc.
- |-- datasets
- |-- cifar10.py
- |-- mnist.py
-
-## Compiling and running instructions
-
-In order to use the Python APIs, users need to add the following arguments when compiling
-SINGA,
-
- ./configure --enable-python --with-python=PYTHON_DIR
- make
-
-where PYTHON_DIR has Python.h
-
-
-The training program is launched by
-
- bin/singa-run.sh -exec <user_main.py>
-
-where user_main.py creates the JobProto object and passes it to Driver::Train to
-start the training.
-
-For example,
-
- cd SINGAROOT
- bin/singa-run.sh -exec tool/python/examples/cifar10_cnn.py
-
-
-
-## Examples
-
-
-### MLP Example
-
-This example uses python APIs to configure and train a MLP model over the MNIST
-dataset. The configuration content is the same as that written in *SINGAROOT/examples/mnist/job.conf*.
-
-```
-X_train, X_test, workspace = mnist.load_data()
-
-m = Sequential('mlp', sys.argv)
-
-m.add(Dense(2500, init='uniform', activation='tanh'))
-m.add(Dense(2000, init='uniform', activation='tanh'))
-m.add(Dense(1500, init='uniform', activation='tanh'))
-m.add(Dense(1000, init='uniform', activation='tanh'))
-m.add(Dense(500, init='uniform', activation='tanh'))
-m.add(Dense(10, init='uniform', activation='softmax'))
-
-sgd = SGD(lr=0.001, lr_type='step')
-topo = Cluster(workspace)
-m.compile(loss='categorical_crossentropy', optimizer=sgd, cluster=topo)
-m.fit(X_train, nb_epoch=1000, with_test=True)
-result = m.evaluate(X_test, batch_size=100, test_steps=10, test_freq=60)
-```
-
-### CNN Example
-
-This example uses python APIs to configure and train a CNN model over the Cifar10
-dataset. The configuration content is the same as that written in *SINGAROOT/examples/cifar10/job.conf*.
-
-
-```
-X_train, X_test, workspace = cifar10.load_data()
-
-m = Sequential('cnn', sys.argv)
-
-m.add(Convolution2D(32, 5, 1, 2, w_std=0.0001, b_lr=2))
-m.add(MaxPooling2D(pool_size=(3,3), stride=2))
-m.add(Activation('relu'))
-m.add(LRN2D(3, alpha=0.00005, beta=0.75))
-
-m.add(Convolution2D(32, 5, 1, 2, b_lr=2))
-m.add(Activation('relu'))
-m.add(AvgPooling2D(pool_size=(3,3), stride=2))
-m.add(LRN2D(3, alpha=0.00005, beta=0.75))
-
-m.add(Convolution2D(64, 5, 1, 2))
-m.add(Activation('relu'))
-m.add(AvgPooling2D(pool_size=(3,3), stride=2))
-
-m.add(Dense(10, w_wd=250, b_lr=2, b_wd=0, activation='softmax'))
-
-sgd = SGD(decay=0.004, lr_type='manual', step=(0,60000,65000), step_lr=(0.001,0.0001,0.00001))
-topo = Cluster(workspace)
-m.compile(updater=sgd, cluster=topo)
-m.fit(X_train, nb_epoch=1000, with_test=True)
-result = m.evaluate(X_test, 1000, test_steps=30, test_freq=300)
-```
-
-
-### RBM Example
-
-This example uses python APIs to configure and train a RBM model over the MNIST
-dataset. The configuration content is the same as that written in *SINGAROOT/examples/rbm*.conf*.
-
-```
-rbmid = 3
-X_train, X_test, workspace = mnist.load_data(nb_rbm=rbmid)
-m = Energy('rbm'+str(rbmid), sys.argv)
-
-out_dim = [1000, 500, 250]
-m.add(RBM(out_dim, w_std=0.1, b_wd=0))
-
-sgd = SGD(lr=0.1, decay=0.0002, momentum=0.8)
-topo = Cluster(workspace)
-m.compile(optimizer=sgd, cluster=topo)
-m.fit(X_train, alg='cd', nb_epoch=6000)
-```
-
-### AutoEncoder Example
-This example uses python APIs to configure and train an autoencoder model over
-the MNIST dataset. The configuration content is the same as that written in
-*SINGAROOT/examples/autoencoder.conf*.
-
-
-```
-rbmid = 4
-X_train, X_test, workspace = mnist.load_data(nb_rbm=rbmid+1)
-m = Sequential('autoencoder', sys.argv)
-
-hid_dim = [1000, 500, 250, 30]
-m.add(Autoencoder(hid_dim, out_dim=784, activation='sigmoid', param_share=True))
-
-agd = AdaGrad(lr=0.01)
-topo = Cluster(workspace)
-m.compile(loss='mean_squared_error', optimizer=agd, cluster=topo)
-m.fit(X_train, alg='bp', nb_epoch=12200)
-```
-
-### To run SINGA on GPU
-
-Users need to set a list of gpu ids to `device` field in fit() or evaluate().
-The number of GPUs must be the same to the number of workers configured for
-cluster topology.
-
-
-```
-gpu_id = [0]
-m.fit(X_train, nb_epoch=100, with_test=True, device=gpu_id)
-```
-
-### TIPS
-
-Hidden layers for MLP can be configured as
-
-```
-for n in [2500, 2000, 1500, 1000, 500]:
- m.add(Dense(n, init='uniform', activation='tanh'))
-m.add(Dense(10, init='uniform', activation='softmax'))
-```
-
-Activation layer can be specified separately
-
-```
-m.add(Dense(2500, init='uniform'))
-m.add(Activation('tanh'))
-```
-
-Users can explicitly specify hyper-parameters of weight and bias
-
-```
-par = Parameter(init='uniform', scale=0.05)
-m.add(Dense(2500, w_param=par, b_param=par, activation='tanh'))
-m.add(Dense(2000, w_param=par, b_param=par, activation='tanh'))
-m.add(Dense(1500, w_param=par, b_param=par, activation='tanh'))
-m.add(Dense(1000, w_param=par, b_param=par, activation='tanh'))
-m.add(Dense(500, w_param=par, b_param=par, activation='tanh'))
-m.add(Dense(10, w_param=par, b_param=par, activation='softmax'))
-```
-
-
-```
-parw = Parameter(init='gauss', std=0.0001)
-parb = Parameter(init='const', value=0)
-m.add(Convolution(32, 5, 1, 2, w_param=parw, b_param=parb, b_lr=2))
-m.add(MaxPooling2D(pool_size(3,3), stride=2))
-m.add(Activation('relu'))
-m.add(LRN2D(3, alpha=0.00005, beta=0.75))
-
-parw.update(std=0.01)
-m.add(Convolution(32, 5, 1, 2, w_param=parw, b_param=parb))
-m.add(Activation('relu'))
-m.add(AvgPooling2D(pool_size(3,3), stride=2))
-m.add(LRN2D(3, alpha=0.00005, beta=0.75))
-
-m.add(Convolution(64, 5, 1, 2, w_param=parw, b_param=parb, b_lr=1))
-m.add(Activation('relu'))
-m.add(AvgPooling2D(pool_size(3,3), stride=2))
-
-m.add(Dense(10, w_param=parw, w_wd=250, b_param=parb, b_lr=2, b_wd=0, activation='softmax'))
-```
-
-
-Data can be added in this way,
-
-```
-X_train, X_test = mnist.load_data() // parameter values are set in load_data()
-m.fit(X_train, ...) // Data layer for training is added
-m.evaluate(X_test, ...) // Data layer for testing is added
-```
-or this way,
-
-```
-X_train, X_test = mnist.load_data() // parameter values are set in load_data()
-m.add(X_train) // explicitly add Data layer
-m.add(X_test) // explicitly add Data layer
-```
-
-
-```
-store = Store(path='train.bin', batch_size=64, ...) // parameter values are set explicitly
-m.add(Data(load='recordinput', phase='train', conf=store)) // Data layer is added
-store = Store(path='test.bin', batch_size=100, ...) // parameter values are set explicitly
-m.add(Data(load='recordinput', phase='test', conf=store)) // Data layer is added
-```
-
-
-### Cases to run SINGA
-
-(1) Run SINGA for training
-
-```
-m.fit(X_train, nb_epoch=1000)
-```
-
-(2) Run SINGA for training and validation
-
-```
-m.fit(X_train, validate_data=X_valid, nb_epoch=1000)
-```
-
-(3) Run SINGA for test while training
-
-```
-m.fit(X_train, nb_epoch=1000, with_test=True)
-result = m.evaluate(X_test, batch_size=100, test_steps=100)
-```
-
-(4) Run SINGA for test only
-Assume a checkpoint exists after training
-
-```
-result = m.evaluate(X_test, batch_size=100, checkpoint_path=workspace+'/checkpoint/step100-worker0')
-```
-
-
-## Implementation Details
-
-### Layer class (inherited)
-
-* Data
-* Dense
-* Activation
-* Convolution2D
-* MaxPooling2D
-* AvgPooling2D
-* LRN2D
-* Dropout
-* RBM
-* Autoencoder
-
-### Model class
-
-Model class has `jobconf` (JobProto) and `layers` (layer list)
-
-Methods in Model class
-
-* add
- * add Layer into Model
- * 2 subclasses: Sequential model and Energy model
-
-* compile
- * set Updater (i.e., optimizer) and Cluster (i.e., topology) components
-
-* fit
- * set Training data and parameter values for the training
- * (optional) set Validatiaon data and parameter values
- * set Train_one_batch component
- * specify `with_test` field if a user wants to run SINGA with test data simultaneously.
- * [TODO] recieve train/validation results, e.g., accuracy, loss, ppl, etc.
-
-* evaluate
- * set Testing data and parameter values for the testing
- * specify `checkpoint_path` field if a user want to run SINGA only for testing.
- * [TODO] recieve test results, e.g., accuracy, loss, ppl, etc.
-
-### Results
-
-fit() and evaluate() return train/test results, a dictionary containing
-
-* [key]: step number
-* [value]: a list of dictionay
- * 'acc' for accuracy
- * 'loss' for loss
- * 'ppl' for ppl
- * 'se' for squred error
-
-
-### Parameter class
-
-Users need to set parameter and initial values. For example,
-
-* Parameter (fields in Param proto)
- * lr = (float) // learning rate multiplier, used to scale the learning rate when updating parameters.
- * wd = (float) // weight decay multiplier, used to scale the weight decay when updating parameters.
-
-* Parameter initialization (fields in ParamGen proto)
- * init = (string) // one of the types, 'uniform', 'constant', 'gaussian'
- * high = (float) // for 'uniform'
- * low = (float) // for 'uniform'
- * value = (float) // for 'constant'
- * mean = (float) // for 'gaussian'
- * std = (float) // for 'gaussian'
-
-* Weight (`w_param`) is 'gaussian' with mean=0, std=0.01 at default
-
-* Bias (`b_param`) is 'constant' with value=0 at default
-
-* How to update the parameter fields
- * for updating Weight, put `w_` in front of field name
- * for updating Bias, put `b_` in front of field name
-
-Several ways to set Parameter values
-
-```
-parw = Parameter(lr=2, wd=10, init='gaussian', std=0.1)
-parb = Parameter(lr=1, wd=0, init='constant', value=0)
-m.add(Convolution2D(10, w_param=parw, b_param=parb, ...)
-```
-
-```
-m.add(Dense(10, w_mean=1, w_std=0.1, w_lr=2, w_wd=10, ...)
-```
-
-```
-parw = Parameter(init='constant', mean=0)
-m.add(Dense(10, w_param=parw, w_lr=1, w_wd=1, b_value=1, ...)
-```
-
-### Other classes
-
-* Store
-* Algorithm
-* Updater
-* SGD
-* AdaGrad
-* Cluster
-
diff --git a/tool/python/examples/__init__.py b/tool/python/examples/__init__.py
deleted file mode 100644
index a796a7a..0000000
--- a/tool/python/examples/__init__.py
+++ /dev/null
@@ -1,22 +0,0 @@
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
diff --git a/tool/python/examples/cifar10_cnn.py b/tool/python/examples/cifar10_cnn.py
deleted file mode 100755
index 8d4e778..0000000
--- a/tool/python/examples/cifar10_cnn.py
+++ /dev/null
@@ -1,55 +0,0 @@
-#!/usr/bin/env python
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
-
-import sys, os
-sys.path.append(os.path.join(os.path.dirname(__file__),'..'))
-from singa.model import *
-from examples.datasets import cifar10
-
-X_train, X_test, workspace = cifar10.load_data()
-
-m = Sequential('cifar10-cnn', sys.argv)
-
-m.add(Convolution2D(32, 5, 1, 2, w_std=0.0001, b_lr=2))
-m.add(MaxPooling2D(pool_size=(3,3), stride=2))
-m.add(Activation('relu'))
-m.add(LRN2D(3, alpha=0.00005, beta=0.75))
-
-m.add(Convolution2D(32, 5, 1, 2, b_lr=2))
-m.add(Activation('relu'))
-m.add(AvgPooling2D(pool_size=(3,3), stride=2))
-m.add(LRN2D(3, alpha=0.00005, beta=0.75))
-
-m.add(Convolution2D(64, 5, 1, 2))
-m.add(Activation('relu'))
-m.add(AvgPooling2D(pool_size=(3,3), stride=2))
-
-m.add(Dense(10, w_wd=250, b_lr=2, b_wd=0, activation='softmax'))
-
-sgd = SGD(decay=0.004, momentum=0.9, lr_type='manual', step=(0,60000,65000), step_lr=(0.001,0.0001,0.00001))
-topo = Cluster(workspace)
-m.compile(loss='categorical_crossentropy', optimizer=sgd, cluster=topo)
-m.fit(X_train, nb_epoch=1000, with_test=True)
-result = m.evaluate(X_test, test_steps=100, test_freq=300)
-
diff --git a/tool/python/examples/cifar10_cnn_cudnn.py b/tool/python/examples/cifar10_cnn_cudnn.py
deleted file mode 100755
index e243834..0000000
--- a/tool/python/examples/cifar10_cnn_cudnn.py
+++ /dev/null
@@ -1,57 +0,0 @@
-#!/usr/bin/env python
-
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
-import sys, os
-sys.path.append(os.path.join(os.path.dirname(__file__),'..'))
-from singa.model import *
-from examples.datasets import cifar10
-
-X_train, X_test, workspace = cifar10.load_data()
-
-m = Sequential('cifar10-cnn', sys.argv)
-
-m.add(Convolution2D(32, 5, 1, 2, w_std=0.0001, b_lr=2))
-m.add(MaxPooling2D(pool_size=(3,3), stride=2))
-m.add(Activation('relu'))
-m.add(LRN2D(3, alpha=0.00005, beta=0.75))
-
-m.add(Convolution2D(32, 5, 1, 2, b_lr=2))
-m.add(Activation('relu'))
-m.add(AvgPooling2D(pool_size=(3,3), stride=2))
-m.add(LRN2D(3, alpha=0.00005, beta=0.75))
-
-m.add(Convolution2D(64, 5, 1, 2))
-m.add(Activation('relu'))
-m.add(AvgPooling2D(pool_size=(3,3), stride=2))
-
-m.add(Dense(10, w_wd=250, b_lr=2, b_wd=0, activation='softmax'))
-
-sgd = SGD(decay=0.004, momentum=0.9, lr_type='manual', step=(0,60000,65000), step_lr=(0.001,0.0001,0.00001))
-topo = Cluster(workspace)
-m.compile(loss='categorical_crossentropy', optimizer=sgd, cluster=topo)
-
-gpu_id = [0]
-m.fit(X_train, nb_epoch=7000, with_test=True, device=gpu_id)
-result = m.evaluate(X_test, test_steps=100, test_freq=1000)
-
diff --git a/tool/python/examples/cifar10_cnn_parameter.py b/tool/python/examples/cifar10_cnn_parameter.py
deleted file mode 100755
index c5470b6..0000000
--- a/tool/python/examples/cifar10_cnn_parameter.py
+++ /dev/null
@@ -1,57 +0,0 @@
-#!/usr/bin/env python
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
-import sys, os
-sys.path.append(os.path.join(os.path.dirname(__file__),'..'))
-from singa.model import *
-from examples.datasets import cifar10
-
-X_train, X_test, workspace = cifar10.load_data()
-
-m = Sequential('cifar10-cnn', sys.argv)
-
-parw = Parameter(init='gaussian', std=0.0001)
-parb = Parameter(init='constant')
-m.add(Convolution2D(32, 5, 1, 2, w_param=parw, b_param=parb, b_lr=2))
-m.add(MaxPooling2D(pool_size=(3,3), stride=2))
-m.add(Activation('relu'))
-m.add(LRN2D(3, alpha=0.00005, beta=0.75))
-
-parw.update(std=0.01)
-m.add(Convolution2D(32, 5, 1, 2, w_param=parw, b_param=parb))
-m.add(Activation('relu'))
-m.add(AvgPooling2D(pool_size=(3,3), stride=2))
-m.add(LRN2D(3, alpha=0.00005, beta=0.75))
-
-m.add(Convolution2D(64, 5, 1, 2, w_param=parw, b_param=parb, b_lr=1))
-m.add(Activation('relu'))
-m.add(AvgPooling2D(pool_size=(3,3), stride=2))
-
-m.add(Dense(10, w_param=parw, w_wd=250, b_param=parb, b_lr=2, b_wd=0, activation='softmax'))
-
-sgd = SGD(decay=0.004, lr_type='manual', step=(0,60000,65000), step_lr=(0.001,0.0001,0.00001))
-topo = Cluster(workspace)
-m.compile(loss='categorical_crossentropy', optimizer=sgd, cluster=topo)
-m.fit(X_train, nb_epoch=100, with_test=True)
-result = m.evaluate(X_test, test_steps=10, test_freq=300)
-
diff --git a/tool/python/examples/datasets/__init__.py b/tool/python/examples/datasets/__init__.py
deleted file mode 100644
index a796a7a..0000000
--- a/tool/python/examples/datasets/__init__.py
+++ /dev/null
@@ -1,22 +0,0 @@
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
diff --git a/tool/python/examples/datasets/cifar10.py b/tool/python/examples/datasets/cifar10.py
deleted file mode 100644
index ef5136f..0000000
--- a/tool/python/examples/datasets/cifar10.py
+++ /dev/null
@@ -1,57 +0,0 @@
-#!/usr/bin/env python
-
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
-from singa.model import *
-
-def load_data(
- workspace = None,
- backend = 'kvfile',
- batchsize = 64,
- random = 5000,
- shape = (3, 32, 32),
- std = 127.5,
- mean = 127.5
- ):
-
- # using cifar10 dataset
- data_dir = 'examples/cifar10'
- path_train = data_dir + '/train_data.bin'
- path_test = data_dir + '/test_data.bin'
- path_mean = data_dir + '/image_mean.bin'
- if workspace == None: workspace = data_dir
-
- store = Store(path=path_train, mean_file=path_mean, backend=backend,
- random_skip=random, batchsize=batchsize,
- shape=shape)
-
- data_train = Data(load='recordinput', phase='train', conf=store)
-
- store = Store(path=path_test, mean_file=path_mean, backend=backend,
- batchsize=batchsize,
- shape=shape)
-
- data_test = Data(load='recordinput', phase='test', conf=store)
-
- return data_train, data_test, workspace
-
diff --git a/tool/python/examples/datasets/mnist.py b/tool/python/examples/datasets/mnist.py
deleted file mode 100644
index 0f75393..0000000
--- a/tool/python/examples/datasets/mnist.py
+++ /dev/null
@@ -1,55 +0,0 @@
-#!/usr/bin/env python
-
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
-from singa.model import *
-
-def load_data(
- workspace = None,
- backend = 'kvfile',
- nb_rbm = 0, # the number of layers for RBM and Autoencoder
- checkpoint_steps = 0,
- **pvalues
- ):
-
- # using mnist dataset
- data_dir = 'examples/mnist'
- path_train = data_dir + '/train_data.bin'
- path_test = data_dir + '/test_data.bin'
- if workspace == None: workspace = data_dir
-
- # checkpoint path to load
- checkpoint_list = None
- if checkpoint_steps > 0:
- workerid = 0
- checkpoint_list = []
- for i in range(nb_rbm-1, 0, -1):
- checkpoint_list.append('examples/rbm/rbm{0}/checkpoint/step{1}-worker{2}'.format(str(i),checkpoint_steps,workerid))
-
- store = Store(path=path_train, backend=backend, **pvalues)
- data_train = Data(load='recordinput', phase='train', conf=store, checkpoint=checkpoint_list)
-
- store = Store(path=path_test, backend=backend, **pvalues)
- data_test = Data(load='recordinput', phase='test', conf=store)
-
- return data_train, data_test, workspace
diff --git a/tool/python/examples/mnist_ae.py b/tool/python/examples/mnist_ae.py
deleted file mode 100755
index 888f288..0000000
--- a/tool/python/examples/mnist_ae.py
+++ /dev/null
@@ -1,48 +0,0 @@
-#!/usr/bin/env python
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
-import sys, os
-sys.path.append(os.path.join(os.path.dirname(__file__),'..'))
-from singa.model import *
-from examples.datasets import mnist
-
-# Sample parameter values for Autoencoder example
-rbmid = 4
-pvalues = {'batchsize' : 100, 'shape' : 784, 'std_value' : 255}
-X_train, X_test, workspace = mnist.load_data(
- workspace = 'examples/rbm/autoencoder',
- nb_rbm = rbmid+1,
- checkpoint_steps = 6000,
- **pvalues)
-
-m = Sequential('autoencoder', sys.argv)
-
-hid_dim = [1000, 500, 250, 30]
-m.add(Autoencoder(hid_dim, out_dim=784, activation='sigmoid', param_share=True))
-
-agd = AdaGrad(lr=0.01)
-topo = Cluster(workspace)
-m.compile(loss='mean_squared_error', optimizer=agd, cluster=topo)
-m.fit(X_train, alg='bp', nb_epoch=12200, with_test=True)
-result = m.evaluate(X_test, test_steps=100, test_freq=1000)
-
diff --git a/tool/python/examples/mnist_mlp.py b/tool/python/examples/mnist_mlp.py
deleted file mode 100755
index 10cd15e..0000000
--- a/tool/python/examples/mnist_mlp.py
+++ /dev/null
@@ -1,55 +0,0 @@
-#!/usr/bin/env python
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
-import sys, os
-sys.path.append(os.path.join(os.path.dirname(__file__),'..'))
-from singa.model import *
-from examples.datasets import mnist
-
-# Sample parameter values for Mnist MLP example
-pvalues = {'batchsize' : 64, 'shape' : 784, 'random_skip' : 5000,
- 'std_value' : 127.5, 'mean_value' : 127.5}
-X_train, X_test, workspace = mnist.load_data(**pvalues)
-
-m = Sequential('mlp', argv=sys.argv)
-
-''' Weight and Bias are initialized by
- uniform distribution with scale=0.05 at default
-'''
-m.add(Dense(2500, init='uniform', activation='tanh'))
-m.add(Dense(2000, init='uniform', activation='tanh'))
-m.add(Dense(1500, init='uniform', activation='tanh'))
-m.add(Dense(1000, init='uniform', activation='tanh'))
-m.add(Dense(500, init='uniform', activation='tanh'))
-m.add(Dense(10, init='uniform', activation='softmax'))
-
-sgd = SGD(lr=0.001, lr_type='step')
-topo = Cluster(workspace)
-m.compile(loss='categorical_crossentropy', optimizer=sgd, cluster=topo)
-
-m.fit(X_train, nb_epoch=100, with_test=True)
-result = m.evaluate(X_test, batch_size=100, test_steps=10)
-
-#e.g., display result
-#for k, v in sorted(result.items(), key=lambda x: x[0]):
-# print k, v
diff --git a/tool/python/examples/mnist_mlp_parameter.py b/tool/python/examples/mnist_mlp_parameter.py
deleted file mode 100755
index 9080451..0000000
--- a/tool/python/examples/mnist_mlp_parameter.py
+++ /dev/null
@@ -1,50 +0,0 @@
-#!/usr/bin/env python
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
-import sys, os
-sys.path.append(os.path.join(os.path.dirname(__file__),'..'))
-from singa.model import *
-from singa.datasets import mnist
-
-# Sample parameter values for Mnist MLP example
-pvalues = {'batchsize' : 64, 'shape' : 784,
- 'random_skip' : 5000,
- 'std_value' : 127.5, 'mean_value' : 127.5}
-X_train, X_test, workspace = mnist.load_data(**pvalues)
-
-m = Sequential('mlp', argv=sys.argv)
-
-par = Parameter(init='uniform', scale=0.05)
-m.add(Dense(2500, w_param=par, b_param=par, activation='tanh'))
-m.add(Dense(2000, w_param=par, b_param=par, activation='tanh'))
-m.add(Dense(1500, w_param=par, b_param=par, activation='tanh'))
-m.add(Dense(1000, w_param=par, b_param=par, activation='tanh'))
-m.add(Dense(500, w_param=par, b_param=par, activation='tanh'))
-m.add(Dense(10, w_param=par, b_param=par, activation='softmax'))
-
-sgd = SGD(lr=0.001, lr_type='step')
-topo = Cluster(workspace)
-m.compile(loss='categorical_crossentropy', optimizer=sgd, cluster=topo)
-
-m.fit(X_train, nb_epoch=100, with_test=True)
-result = m.evaluate(X_test, batch_size=100, test_steps=10)
diff --git a/tool/python/examples/mnist_mlp_test.py b/tool/python/examples/mnist_mlp_test.py
deleted file mode 100755
index ee4e4aa..0000000
--- a/tool/python/examples/mnist_mlp_test.py
+++ /dev/null
@@ -1,52 +0,0 @@
-#!/usr/bin/env python
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
-import sys, os
-sys.path.append(os.path.join(os.path.dirname(__file__),'..'))
-from singa.model import *
-from examples.datasets import mnist
-
-# Sample parameter values for Mnist MLP example
-pvalues = {'batchsize' : 64, 'shape' : 784,
- 'std_value' : 127.5, 'mean_value' : 127.5}
-X_train, X_test, workspace = mnist.load_data(**pvalues)
-
-m = Sequential('mlp', argv=sys.argv)
-
-m.add(Dense(2500, init='uniform', activation='tanh'))
-m.add(Dense(2000, init='uniform', activation='tanh'))
-m.add(Dense(1500, init='uniform', activation='tanh'))
-m.add(Dense(1000, init='uniform', activation='tanh'))
-m.add(Dense(500, init='uniform', activation='tanh'))
-m.add(Dense(10, init='uniform', activation='softmax'))
-
-sgd = SGD(lr=0.001, lr_type='step')
-topo = Cluster(workspace)
-m.compile(loss='categorical_crossentropy', optimizer=sgd, cluster=topo)
-
-''' For doing test only, normally users sets checkpoint path
- e.g., assume that checkpoint exists by
- m.fit(X_train, nb_epoch=100, checkpoint_freq=100)
-'''
-path = workspace+'/checkpoint/step100-worker0'
-result = m.evaluate(X_test, batch_size=100, test_steps=100, checkpoint_path=path)
diff --git a/tool/python/examples/mnist_rbm1.py b/tool/python/examples/mnist_rbm1.py
deleted file mode 100755
index 5f22d52..0000000
--- a/tool/python/examples/mnist_rbm1.py
+++ /dev/null
@@ -1,46 +0,0 @@
-#!/usr/bin/env python
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
-import sys, os
-sys.path.append(os.path.join(os.path.dirname(__file__),'..'))
-from singa.model import *
-from examples.datasets import mnist
-
-rbmid = 1
-pvalues = {'batchsize' : 100, 'shape' : 784, 'std_value' : 255}
-X_train, X_test, workspace = mnist.load_data(
- workspace = 'examples/rbm/rbm1',
- nb_rbm = rbmid,
- checkpoint_steps = 6000,
- **pvalues)
-
-m = Energy('rbm'+str(rbmid), sys.argv)
-
-m.add(RBM(1000, w_std=0.1, b_wd=0))
-
-sgd = SGD(lr=0.1, decay=0.0002, momentum=0.8)
-topo = Cluster(workspace)
-m.compile(optimizer=sgd, cluster=topo)
-m.fit(X_train, alg='cd', nb_epoch=6000)
-#result = m.evaluate(X_test, test_steps=100, test_freq=500)
-
diff --git a/tool/python/examples/mnist_rbm2.py b/tool/python/examples/mnist_rbm2.py
deleted file mode 100755
index 1544f14..0000000
--- a/tool/python/examples/mnist_rbm2.py
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/usr/bin/env python
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
-import sys, os
-sys.path.append(os.path.join(os.path.dirname(__file__),'..'))
-from singa.model import *
-from examples.datasets import mnist
-
-rbmid = 2
-pvalues = {'batchsize' : 100, 'shape' : 784, 'std_value' : 255}
-X_train, X_test, workspace = mnist.load_data(
- workspace = 'examples/rbm/rbm2',
- nb_rbm = rbmid,
- checkpoint_steps = 6000,
- **pvalues)
-
-m = Energy('rbm'+str(rbmid), sys.argv)
-
-out_dim = [1000, 500]
-m.add(RBM(out_dim, w_std=0.1, b_wd=0))
-
-sgd = SGD(lr=0.1, decay=0.0002, momentum=0.8)
-topo = Cluster(workspace)
-m.compile(optimizer=sgd, cluster=topo)
-m.fit(X_train, alg='cd', nb_epoch=6000)
-#result = m.evaluate(X_test, test_steps=100, test_freq=500)
-
diff --git a/tool/python/examples/mnist_rbm3.py b/tool/python/examples/mnist_rbm3.py
deleted file mode 100755
index 3a6348d..0000000
--- a/tool/python/examples/mnist_rbm3.py
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/usr/bin/env python
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
-import sys, os
-sys.path.append(os.path.join(os.path.dirname(__file__),'..'))
-from singa.model import *
-from examples.datasets import mnist
-
-rbmid = 3
-pvalues = {'batchsize' : 100, 'shape' : 784, 'std_value' : 255}
-X_train, X_test, workspace = mnist.load_data(
- workspace = 'examples/rbm/rbm3',
- nb_rbm = rbmid,
- checkpoint_steps = 6000,
- **pvalues)
-
-m = Energy('rbm'+str(rbmid), sys.argv)
-
-out_dim = [1000, 500, 250]
-m.add(RBM(out_dim, w_std=0.1, b_wd=0))
-
-sgd = SGD(lr=0.1, decay=0.0002, momentum=0.8)
-topo = Cluster(workspace)
-m.compile(optimizer=sgd, cluster=topo)
-m.fit(X_train, alg='cd', nb_epoch=6000)
-#result = m.evaluate(X_test, test_steps=100, test_freq=500)
-
diff --git a/tool/python/examples/mnist_rbm4.py b/tool/python/examples/mnist_rbm4.py
deleted file mode 100755
index 8343b4f..0000000
--- a/tool/python/examples/mnist_rbm4.py
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/usr/bin/env python
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
-import sys, os
-sys.path.append(os.path.join(os.path.dirname(__file__),'..'))
-from singa.model import *
-from examples.datasets import mnist
-
-rbmid = 4
-pvalues = {'batchsize' : 100, 'shape' : 784, 'std_value' : 255}
-X_train, X_test, workspace = mnist.load_data(
- workspace = 'examples/rbm/rbm'+str(rbmid),
- nb_rbm = rbmid,
- checkpoint_steps = 6000,
- **pvalues)
-
-m = Energy('rbm'+str(rbmid), sys.argv)
-
-out_dim = [1000, 500, 250, 30]
-m.add(RBM(out_dim, sampling='gaussian', w_std=0.1, b_wd=0))
-
-sgd = SGD(lr=0.001, decay=0.0002, momentum=0.8)
-topo = Cluster(workspace)
-m.compile(optimizer=sgd, cluster=topo)
-m.fit(X_train, alg='cd', nb_epoch=6000)
-#result = m.evaluate(X_test, test_steps=100, test_freq=500)
-
diff --git a/tool/python/examples/train_cifar10.py b/tool/python/examples/train_cifar10.py
deleted file mode 100755
index e8ac973..0000000
--- a/tool/python/examples/train_cifar10.py
+++ /dev/null
@@ -1,142 +0,0 @@
-#!/usr/bin/env python
-
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-'''
-Example script of CNN model for CIFAR10 dataset
-'''
-import os, sys
-import numpy as np
-
-current_path_ = os.path.dirname(__file__)
-singa_root_ = os.path.abspath(os.path.join(current_path_,'../../..'))
-sys.path.append(os.path.join(singa_root_,'tool','python'))
-
-from singa.driver import Driver
-from singa.layer import *
-from singa.model import *
-
-
-'''
-CIFAR10 dataset can be downloaded at [https://www.cs.toronto.edu/~kriz/cifar.html]
-- please specify dataset_dir
-'''
-dataset_dir_ = singa_root_ + "/tool/python/examples/datasets/cifar-10-batches-py"
-mean_image = None
-
-def unpickle(file):
- ''' This method loads dataset provided at CIFAR10 website
- See [https://www.cs.toronto.edu/~kriz/cifar.html] for more details
- '''
- import cPickle
- fo = open(file, 'rb')
- dict = cPickle.load(fo)
- fo.close()
- return dict
-
-def compute_mean_image():
- ''' This is a sample script to cmopute the average image
- of all samples in 5 dataset of cifar10
- '''
- mean = None
- nb_samples_total = 0
- for did in range(1,6):
- fname_train_data = dataset_dir_ + "/data_batch_{}".format(did)
- cifar10 = unpickle(fname_train_data)
- image = cifar10['data'].astype(dtype=np.uint8)
- if did > 1:
- image = np.vstack((image, image))
- return np.average(image, axis=0)
-
-def load_dataset(did=1):
- ''' CIFAR10 dataset includes
- 5 binary dataset, each contains 10000 images
- 1 row (1 image) includes 1 label & 3072 pixels
- 3072 pixels are 3 channels of a 32x32 image
- '''
- assert mean_image is not None, 'mean_image is required'
- print '[Load CIFAR10 dataset {}]'.format(did)
- fname_train_data = dataset_dir_ + "/data_batch_{}".format(did)
- cifar10 = unpickle(fname_train_data)
- image = cifar10['data'].astype(dtype=np.uint8)
- image = image - mean_image
- print ' image x:', image.shape
- label = np.asarray(cifar10['labels'], dtype=np.uint8)
- label = label.reshape(label.size, 1)
- print ' label y:', label.shape
- return image, label
-
-#-------------------------------------------------------------------
-mean_image = compute_mean_image()
-# mean_image = np.fromfile('tool/python/examples/datasets/cifar10_mean_image')
-
-print '[Layer registration/declaration]'
-d = Driver()
-d.Init(sys.argv)
-
-input = ImageInput(32, 32, 3) # image width, height, channel
-label = LabelInput()
-
-nn = []
-nn.append(input)
-nn.append(Convolution2D(32, 5, 1, 2, w_std=0.0001, b_lr=2))
-nn.append(MaxPooling2D(pool_size=(3,3), stride=2))
-nn.append(Activation('relu'))
-nn.append(LRN2D(3, alpha=0.00005, beta=0.75))
-nn.append(Convolution2D(32, 5, 1, 2, b_lr=2))
-nn.append(Activation('relu'))
-nn.append(AvgPooling2D(pool_size=(3,3), stride=2))
-nn.append(LRN2D(3, alpha=0.00005, beta=0.75))
-nn.append(Convolution2D(64, 5, 1, 2))
-nn.append(Activation('relu'))
-nn.append(AvgPooling2D(pool_size=(3,3), stride=2))
-nn.append(Dense(10, w_wd=250, b_lr=2, b_wd=0))
-loss = Loss('softmaxloss')
-
-# updater
-sgd = SGD(decay=0.004, momentum=0.9, lr_type='manual', step=(0,60000,65000), step_lr=(0.001,0.0001,0.00001))
-
-#-------------------------------------------------------------------
-batchsize = 100
-disp_freq = 50
-train_step = 1000
-
-print '[Start training]'
-for dataset_id in range(train_step / batchsize):
-
- x, y = load_dataset(dataset_id%5+1)
-
- for i in range(x.shape[0] / batchsize):
- xb, yb = x[i*batchsize:(i+1)*batchsize,:], y[i*batchsize:(i+1)*batchsize,:]
- nn[0].Feed(xb)
- label.Feed(yb)
- for h in range(1, len(nn)):
- nn[h].ComputeFeature(nn[h-1])
- loss.ComputeFeature(nn[-1], label)
- if (i+1)%disp_freq == 0:
- print ' Step {:>3}: '.format(i+1 + dataset_id*(x.shape[0]/batchsize)),
- loss.display()
-
- loss.ComputeGradient()
- for h in range(len(nn)-1, 0, -1):
- nn[h].ComputeGradient()
- sgd.Update(i+1, nn[h])
diff --git a/tool/python/examples/train_mnist.py b/tool/python/examples/train_mnist.py
deleted file mode 100755
index b8e6217..0000000
--- a/tool/python/examples/train_mnist.py
+++ /dev/null
@@ -1,117 +0,0 @@
-#!/usr/bin/env python
-
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-'''
-Example script of MLP model for MNIST dataset
-'''
-import os, sys
-import numpy as np
-
-current_path_ = os.path.dirname(__file__)
-singa_root_=os.path.abspath(os.path.join(current_path_,'../../..'))
-sys.path.append(os.path.join(singa_root_,'tool','python'))
-
-from singa.driver import Driver
-from singa.layer import *
-from singa.model import *
-
-def swap32(x):
- return (((x << 24) & 0xFF000000) |
- ((x << 8) & 0x00FF0000) |
- ((x >> 8) & 0x0000FF00) |
- ((x >> 24) & 0x000000FF))
-
-def load_dataset():
- ''' MNIST dataset
- train-images: 4 int32 headers & int8 pixels
- train-labels: 2 int32 headers & int8 labels
- '''
- print '[Load MNIST dataset]'
- fname_train_image = "examples/mnist/train-images-idx3-ubyte"
- fname_train_label = "examples/mnist/train-labels-idx1-ubyte"
- nb_header = [4, 2]
-
- info = swap32(np.fromfile(fname_train_image, dtype=np.uint32, count=nb_header[0]))
- nb_samples = info[1]
- shape = (info[2],info[3])
-
- x = np.fromfile(fname_train_image, dtype=np.uint8)
- x = x[np.dtype(np.int32).itemsize*nb_header[0]:] # skip header
- x = x.reshape(nb_samples, shape[0]*shape[1])
- print ' data x:', x.shape
- y = np.fromfile(fname_train_label, dtype=np.uint8)
- y = y[np.dtype(np.int32).itemsize*nb_header[1]:] # skip header
- y = y.reshape(nb_samples, 1)
- print ' label y:', y.shape
-
- return x, y
-
-#-------------------------------------------------------------------
-print '[Layer registration/declaration]'
-d = Driver()
-d.Init(sys.argv)
-
-input = ImageInput(28, 28)
-label = LabelInput()
-
-nn = []
-nn.append(input)
-nn.append(Dense(2500, init='uniform'))
-nn.append(Activation('stanh'))
-nn.append(Dense(2000, init='uniform'))
-nn.append(Activation('stanh'))
-nn.append(Dense(1500, init='uniform'))
-nn.append(Activation('stanh'))
-nn.append(Dense(1000, init='uniform'))
-nn.append(Activation('stanh'))
-nn.append(Dense(500, init='uniform'))
-nn.append(Activation('stanh'))
-nn.append(Dense(10, init='uniform'))
-loss = Loss('softmaxloss')
-
-# updater
-sgd = SGD(lr=0.001, lr_type='step')
-
-#-------------------------------------------------------------------
-batchsize = 64
-disp_freq = 10
-
-x, y = load_dataset()
-
-print '[Start training]'
-for i in range(x.shape[0] / batchsize):
- xb, yb = x[i*batchsize:(i+1)*batchsize,:], y[i*batchsize:(i+1)*batchsize,:]
- nn[0].Feed(xb)
- label.Feed(yb)
- for h in range(1, len(nn)):
- nn[h].ComputeFeature(nn[h-1])
- loss.ComputeFeature(nn[-1], label)
- if (i+1)%disp_freq == 0:
- print ' Step {:>3}: '.format(i+1),
- loss.display()
-
- loss.ComputeGradient()
- for h in range(len(nn)-1, 0, -1):
- nn[h].ComputeGradient()
- sgd.Update(i+1, nn[h])
-
diff --git a/tool/python/singa.py b/tool/python/singa.py
deleted file mode 100755
index e44e94d..0000000
--- a/tool/python/singa.py
+++ /dev/null
@@ -1,46 +0,0 @@
-#!/usr/bin/env python
-
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-import os
-import sys
-import string
-import pb2.job_pb2 as job_pb2
-import singa.driver as driver
-from google.protobuf.text_format import Merge
-
-if __name__ == '__main__':
- """Invoke the training program using this python script.
- ./bin/singa-run.sh -exec tool/python/singa.py -conf examples/cifar10/job.conf
- """
-
- i = sys.argv.index('-conf')
- s = open(sys.argv[i+1], 'r').read()
- s = str(s)
- j = job_pb2.JobProto()
- Merge(s, j)
- b = j.SerializeToString()
- d = driver.Driver()
- d.InitLog(sys.argv[0])
- d.Init(sys.argv)
- d.Train(False, b)
- #d.Test(b)
diff --git a/tool/python/singa/__init__.py b/tool/python/singa/__init__.py
deleted file mode 100644
index a796a7a..0000000
--- a/tool/python/singa/__init__.py
+++ /dev/null
@@ -1,22 +0,0 @@
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
diff --git a/tool/python/singa/driver.i b/tool/python/singa/driver.i
deleted file mode 100644
index 63f2287..0000000
--- a/tool/python/singa/driver.i
+++ /dev/null
@@ -1,117 +0,0 @@
-/************************************************************
-*
-* Licensed to the Apache Software Foundation (ASF) under one
-* or more contributor license agreements. See the NOTICE file
-* distributed with this work for additional information
-* regarding copyright ownership. The ASF licenses this file
-* to you under the Apache License, Version 2.0 (the
-* "License"); you may not use this file except in compliance
-* with the License. You may obtain a copy of the License at
-*
-* http://www.apache.org/licenses/LICENSE-2.0
-*
-* Unless required by applicable law or agreed to in writing,
-* software distributed under the License is distributed on an
-* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-* KIND, either express or implied. See the License for the
-* specific language governing permissions and limitations
-* under the License.
-*
-*************************************************************/
-
-/*interface file for swig */
-
-%module driver
-%include "std_vector.i"
-%include "std_string.i"
-%include "argcargv.i"
-%include "carrays.i"
-%array_class(float, floatArray);
-
-%apply (int ARGC, char **ARGV) { (int argc, char **argv) }
-%{
-#include "singa/driver.h"
-#include "singa/worker.h"
-#include "singa/neuralnet/layer.h"
-#include "singa/neuralnet/neuron_layer.h"
-#include "singa/neuralnet/loss_layer.h"
-#include "singa/utils/blob.h"
-#include "singa/utils/param.h"
-#include "singa/utils/updater.h"
-#include "singa/proto/job.pb.h"
-#include "singa/proto/common.pb.h"
-%}
-
-namespace std {
- %template(strVector) vector<string>;
- %template(intVector) vector<int>;
- %template(floatVector) vector<float>;
- %template(layerVector) vector<singa::Layer*>;
- %template(paramVector) vector<singa::Param*>;
-}
-
-namespace singa{
- class Driver{
- public:
- void Train(bool resume, const std::string job_conf);
- void Init(int argc, char **argv);
- void InitLog(char* arg);
- void Test(const std::string job_conf);
- };
-
- %nodefault Worker;
- class Worker{
- public:
- static singa::Worker* CreateWorker(const std::string str);
- void InitNetParams(const std::string& folder, std::vector<singa::Layer*> net);
- void Checkpoint(int step, const std::string& folder, std::vector<singa::Layer*> net);
- };
-
- class DummyLayer{
- public:
- void Setup(const std::string str, const std::vector<singa::Layer*>& srclayers);
- void Feed(int batchsize, std::vector<float>& data, std::vector<int>& aux_data);
- singa::Layer* ToLayer();
- };
-
- %nodefault Layer;
- class Layer{
- public:
- static singa::Layer* CreateLayer(const std::string str);
- static void SetupLayer(singa::Layer* layer, const std::string str, const std::vector<singa::Layer*>& srclayers);
- virtual void ComputeFeature(int flag, const std::vector<singa::Layer*>& srclayers);
- virtual void ComputeGradient(int flag, const std::vector<singa::Layer*>& srclayers);
- virtual const singa::Blob<float>& data(const singa::Layer* from);
- virtual const std::vector<singa::Param*> GetParams();
- virtual const std::string ToString(bool debug, int flag);
- void SetParams(std::vector<singa::Param*> params);
- };
-
- %nodefault Updater;
- class Updater{
- public:
- static singa::Updater* CreateUpdater(const std::string str);
- virtual void Update(int step, singa::Param* param, float grad_scale);
- };
-
- template <typename Dtype>
- class Blob{
- public:
- inline int count();
- inline const std::vector<int>& shape();
- inline Dtype* mutable_cpu_data();
- inline const Dtype* cpu_data();
- };
-
- class Param{
- public:
- inline int size();
- inline const std::vector<int>& shape();
- inline float* mutable_cpu_data();
- void FromProto(const std::string str);
- /*void ToProto(singa::BlobProto* blob);
- */
- };
-
- %template(floatBlob) Blob<float>;
-}
diff --git a/tool/python/singa/generatepy.sh b/tool/python/singa/generatepy.sh
deleted file mode 100755
index 488d96a..0000000
--- a/tool/python/singa/generatepy.sh
+++ /dev/null
@@ -1,26 +0,0 @@
-#!/usr/bin/env bash
-#/**
-# * Licensed to the Apache Software Foundation (ASF) under one
-# * or more contributor license agreements. See the NOTICE file
-# * distributed with this work for additional information
-# * regarding copyright ownership. The ASF licenses this file
-# * to you under the Apache License, Version 2.0 (the
-# * "License"); you may not use this file except in compliance
-# * with the License. You may obtain a copy of the License at
-# *
-# * http://www.apache.org/licenses/LICENSE-2.0
-# *
-# * Unless required by applicable law or agreed to in writing, software
-# * distributed under the License is distributed on an "AS IS" BASIS,
-# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# * See the License for the specific language governing permissions and
-# * limitations under the License.
-# */
-
-#The following commands are only for developers adding new py apis.
-swig -c++ -python driver.i
-#g++ -fPIC ../../../src/driver.cc driver_wrap.cxx -shared -o _driver.so \
-# -L../../../.libs/ -lsinga -DMSHADOW_USE_CUDA=0 \
-# -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0 -std=c++11 \
-# -I../../../include \
-# -I/usr/include/python2.7/
diff --git a/tool/python/singa/initializations.py b/tool/python/singa/initializations.py
deleted file mode 100644
index f016f1f..0000000
--- a/tool/python/singa/initializations.py
+++ /dev/null
@@ -1,67 +0,0 @@
-#!/usr/bin/env python
-
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-'''
-This module pre-defines initial value for fields
-'''
-
-def get_init_values(identifier, **kwargs):
- '''
- This method returns field, a set of key-value pairs, that
- key is specified by identifier and values are initialized.
- '''
-
- field = {}
-
- if identifier == 'none':
- return
-
- if identifier == 'uniform':
- scale = kwargs['scale'] if 'scale' in kwargs else 0.05
- names = ['low', 'high']
- values = [-scale, scale]
-
- elif identifier == 'constant':
- names = ['value']
- values = [0]
-
- elif identifier == 'gaussian':
- names = ['mean', 'std']
- values = [0, 0.01]
-
- elif identifier == 'conv2d':
- names = ['stride', 'pad']
- values = [1, 0]
-
- elif identifier == 'lrn2d':
- names = ['alpha', 'beta', 'knorm']
- values = [1, 0.75, 1]
-
- elif identifier == 'dropout':
- names = ['ratio']
- values = [0.5]
-
- for i in range(len(names)):
- field[names[i]] = kwargs[names[i]] if names[i] in kwargs else values[i]
-
- return field
diff --git a/tool/python/singa/layer.py b/tool/python/singa/layer.py
deleted file mode 100644
index c9a992d..0000000
--- a/tool/python/singa/layer.py
+++ /dev/null
@@ -1,693 +0,0 @@
-#!/usr/bin/env python
-
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-'''
-This script includes Layer class and its subclasses that
-users can configure different types of layers for their model.
-'''
-import numpy as np
-from singa.parameter import Parameter, set_param_field
-from singa.initializations import get_init_values
-from singa.utils.utility import setval, generate_name
-from singa.utils.message import *
-from google.protobuf import text_format
-
-from singa.driver import Layer as SingaLayer, Updater as SingaUpdater,\
- intVector, floatVector, layerVector,\
- paramVector, floatArray_frompointer, DummyLayer
-
-class Layer(object):
-
- singaupdater = None
-
- def __init__(self, **kwargs):
- '''
- **kwargs (KEY=VALUE)
- partition_dim = (int) // partition dimension for net
- '''
-
- self.layer = Message('Layer', **kwargs).proto
- # required field
- if not 'name' in kwargs:
- setval(self.layer, name=generate_name('layer', 1))
-
- # layer connectivity is set in Model.build()
- self.is_datalayer = False
- self.singalayer = None
- self.srclayers = []
-
- # set src for Rafiki
- if 'src' in kwargs:
- self.src = kwargs['src']
- else:
- self.src = None
-
- def setup(self, srclys):
- ''' Create singa::Layer and store srclayers
- '''
- if self.singalayer == None:
- self.singalayer = SingaLayer.CreateLayer(
- self.layer.SerializeToString())
- self.singaSrclayerVector = layerVector(len(srclys))
- for i in range(len(srclys)):
- self.srclayers.append(srclys[i])
- self.singaSrclayerVector[i] = srclys[i].get_singalayer()
- # set up the layer
- SingaLayer.SetupLayer(self.singalayer,
- self.layer.SerializeToString(),
- self.singaSrclayerVector)
-
- def ComputeFeature(self, *srclys):
- ''' The method creates and sets up singa::Layer
- and maintains its source layers
- then call ComputeFeature for data transformation.
-
- *srclys = (list) // a list of source layers
- '''
- # create singa::Layer and store srclayers
- if self.singalayer == None:
- if self.src != None:
- srclys = self.src
- self.singalayer = SingaLayer.CreateLayer(
- self.layer.SerializeToString())
- self.singaSrclayerVector = layerVector(len(srclys))
- for i in range(len(srclys)):
- self.srclayers.append(srclys[i])
- self.singaSrclayerVector[i] = srclys[i].get_singalayer()
- # set up the layer
- SingaLayer.SetupLayer(self.singalayer,
- self.layer.SerializeToString(),
- self.singaSrclayerVector)
-
- self.singalayer.ComputeFeature(1, self.singaSrclayerVector)
-
- def ComputeGradient(self):
- ''' The method creates singa::Updater
- and calls ComputeGradient for gradient computation
- then updates the parameters.
- '''
- # call ComputeGradient of Singa
- self.singalayer.ComputeGradient(1, self.singaSrclayerVector)
-
- def UpdateParams(self, step, upd):
- ''' The method updates parameter values
- '''
- # update parameters
- singaParams = self.singalayer.GetParams()
- for par in singaParams:
- upd.singaupdater.Update(step, par, 1.0)
-
- def GetParams(self):
- ''' The method gets parameter values
- singaParams[0] for weight
- singaParams[1] for bias
- '''
- singaParams = self.singalayer.GetParams()
- assert len(singaParams) == 2, 'weight and bias'
- # for weight
- weight_array = floatArray_frompointer(singaParams[0].mutable_cpu_data())
- weight = [weight_array[i] for i in range(singaParams[0].size())]
- weight = np.array(weight).reshape(singaParams[0].shape())
- # for bias
- bias_array = floatArray_frompointer(singaParams[1].mutable_cpu_data())
- bias = [bias_array[i] for i in range(singaParams[1].size())]
- bias = np.array(bias).reshape(singaParams[1].shape()[0], 1)
-
- return weight, bias
-
- def SetParams(self, *params):
- ''' The method sets parameter values
- params[0] for weight
- params[1] for bias
- '''
- singaParams = self.singalayer.GetParams()
- import pb2.common_pb2 as cm
- for k in range(len(params)):
- bp = cm.BlobProto()
- bp.shape.append(int(params[k].shape[0]))
- bp.shape.append(int(params[k].shape[1]))
- for i in range(params[k].shape[0]):
- for j in range(params[k].shape[1]):
- bp.data.append(params[k][i, j])
- singaParams[k].FromProto(bp.SerializeToString())
-
- def GetData(self):
- ''' The method gets layer data values
- '''
- blobptr = self.singalayer.data(self.singalayer)
- data_array = floatArray_frompointer(blobptr.mutable_cpu_data())
- data = [data_array[i] for i in range(blobptr.count())]
- return data
-
- def display(self):
- debug, flag = False, 0
- print self.singalayer.ToString(debug, flag)
-
- def get_singalayer(self):
- return self.singalayer
-
-
-class Dummy(object):
-
- def __init__(self, **kwargs):
- ''' Dummy layer is used for data layer to feed/fetch input data
- or label information
- '''
- self.is_datalayer = True
- self.srclayers = None
- self.singalayer = None
-
- # create layer proto for Dummy layer
- kwargs = {'name':'dummy', 'type':kDummy}
- self.layer = Message('Layer', **kwargs).proto
-
- def setup(self, data_shape):
- ''' Create and Setup singa Dummy layer
- called by load_model_parameter
- '''
- if self.singalayer == None:
- setval(self.layer.dummy_conf, input=True)
- setval(self.layer.dummy_conf, shape=data_shape)
- self.singalayer = DummyLayer()
- self.singalayer.Setup(self.layer.SerializeToString(),
- layerVector(0))
-
- def Feed(self, shape, data, aux_data):
- ''' Create and Setup singa::DummyLayer for input data
- Insert data using Feed()
- '''
- batchsize = shape[0]
- hdim = reduce(lambda x, y: x*y, shape[1:])
- datasize = batchsize * hdim
-
- # create and setup the dummy layer
- if self.singalayer == None:
- self.setup(shape)
-
- if data is not None:
- data = data.astype(np.float)
- dataVector = floatVector(datasize)
- for i in range(batchsize):
- for j in range(hdim):
- dataVector[i*hdim+j] = data[i, j]
- labelVector = intVector(0)
-
- if aux_data is not None:
- aux_data = aux_data.astype(np.int)
- labelVector = intVector(datasize)
- for i in range(batchsize):
- labelVector[i] = aux_data[i, 0]
- dataVector = floatVector(0)
-
- self.singalayer.Feed(batchsize, dataVector, labelVector)
-
- def get_singalayer(self):
- return self.singalayer.ToLayer()
-
-class ImageInput(Dummy):
- ''' This class is used to feed image data
- '''
- def __init__(self, width=None, height=None, nb_channel=1):
- super(ImageInput, self).__init__()
- self.width = width
- self.height = height
- self.nb_channel = nb_channel
-
- def Feed(self, image_data):
- batchsize = image_data.shape[0]
- if self.width == None or self.height == None:
- hdim = image_data.shape[1]
- imgsize = int(np.sqrt(hdim/self.nb_channel))
- shape = [batchsize, self.nb_channel, self.width, self.height]
- Dummy.Feed(self, shape, image_data, None)
-
-class LabelInput(Dummy):
- ''' This class is used to feed label data
- '''
- def __init__(self):
- super(LabelInput, self).__init__()
-
- def Feed(self, label_data):
- Dummy.Feed(self, label_data.shape, None, label_data)
-
-
-class Data(Layer):
-
- def __init__(self, load, phase='train', checkpoint=None,
- conf=None, **kwargs):
- '''
- required
- load = (string) // type of data
- optional
- phase = (string) // phase of data layer
- checkpoint = (string) // checkpoint path
- conf = (Store) // Store object
- **kwargs (KEY=VALUE)
- partition_dim = (int) // partition dimension for net
- '''
-
- assert load != None, 'data type should be specified'
- if load == 'kData':
- super(Data, self).__init__(name=generate_name('data'),
- user_type=load, **kwargs)
- else:
- self.layer_type = enumLayerType(load)
- super(Data, self).__init__(name=generate_name('data'),
- type=self.layer_type, **kwargs)
- self.is_datalayer = True
-
- # include/exclude
- setval(self.layer, include=enumPhase(phase))
- #setval(self.layer, exclude=kTest if phase=='train' else kTrain)
-
- if conf == None:
- if load == 'kData':
- setval(self.layer.Extensions[data_conf], **kwargs)
- else:
- setval(self.layer.store_conf, **kwargs)
- else:
- setval(self.layer, store_conf=conf.proto)
-
- self.checkpoint = checkpoint # checkpoint for training data
-
-
-class Convolution2D(Layer):
-
- def __init__(self, nb_filter=0, kernel=0, stride=1, pad=0,
- init=None, w_param=None, b_param=None,
- activation=None, **kwargs):
- '''
- required
- nb_filter = (int) // the number of filters
- kernel = (int/tuple) // the size of filter
- optional
- stride = (int/tuple) // the size of stride
- pad = (int/tuple) // the size of padding
- init = (string) // 'uniform', 'gaussian', 'constant'
- w_param = (Parameter) // Parameter object for weight
- b_param = (Parameter) // Parameter object for bias
- **kwargs (KEY=VALUE)
- w_lr = (float) // learning rate multiplier for weight, used to
- // scale the learning rate when updating parameters.
- w_wd = (float) // weight decay multiplier for weight, used to
- // scale the weight decay when updating parameters.
- b_lr = (float) // learning rate multiplier for bias
- b_wd = (float) // weight decay multiplier for bias
- '''
-
- assert nb_filter > 0, 'nb_filter should be set as positive int'
- super(Convolution2D, self).__init__(name=generate_name('conv', 1),
- type=kCConvolution, **kwargs)
- fields = {"num_filters":nb_filter}
- # for kernel
- if type(kernel) == int:
- fields['kernel'] = kernel
- else:
- fields['kernel_x'] = kernel[0]
- fields['kernel_y'] = kernel[1]
- # for stride
- if type(stride) == int:
- fields['stride'] = stride
- else:
- fields['stride_x'] = stride[0]
- fields['stride_y'] = stride[1]
- # for pad
- if type(pad) == int:
- fields['pad'] = pad
- else:
- fields['pad_x'] = pad[0]
- fields['pad_y'] = pad[1]
-
- setval(self.layer.convolution_conf, **fields)
-
- # parameter w
- if w_param == None:
- self.init = 'gaussian' if init == None else init
- w_param = Parameter(init=self.init)
- set_param_field(w_param.param, 'w', True, **kwargs)
- setval(self.layer, param=w_param.param)
-
- # parameter b
- if b_param == None:
- self.init = 'constant' if init == None else init
- b_param = Parameter(init=self.init) # default: constant
- set_param_field(b_param.param, 'b', True, **kwargs)
- setval(self.layer, param=b_param.param)
-
- # following layers: e.g., activation, dropout, etc.
- if activation:
- self.mask = Activation(activation=activation).layer
-
-
-class MaxPooling2D(Layer):
-
- def __init__(self, pool_size=None,
- stride=1, ignore_border=True, **kwargs):
- '''
- Max Pooling layer
-
- required
- pool_size = (int|tuple) // the size for pooling
- optional
- stride = (int) // the size of striding
- ignore_border = (bool) // flag for padding
- **kwargs // fields for Layer class
- '''
-
- assert pool_size != None, 'pool_size is required'
- if type(pool_size) == int:
- pool_size = (pool_size, pool_size)
- assert type(pool_size) == tuple and pool_size[0] == pool_size[1], \
- 'currently pool size should be square in Singa'
- super(MaxPooling2D, self).__init__(name=generate_name('pool'),
- type=kCPooling, **kwargs)
- fields = {'pool' : PoolingProto().MAX,
- 'kernel' : pool_size[0],
- 'stride' : stride,
- 'pad' : 0 if ignore_border else 1}
- setval(self.layer.pooling_conf, **fields)
-
-class AvgPooling2D(Layer):
-
- def __init__(self, pool_size=None,
- stride=1, ignore_border=True, **kwargs):
- '''
- required
- pool_size = (int|tuple) // size for pooling
- optional
- stride = (int) // size of striding
- ignore_border = (bool) // flag for padding
- **kwargs // fields for Layer class
- '''
-
- assert pool_size != None, 'pool_size is required'
- if type(pool_size) == int:
- pool_size = (pool_size, pool_size)
- assert type(pool_size) == tuple and pool_size[0] == pool_size[1], \
- 'currently pool size should be square in Singa'
- super(AvgPooling2D, self).__init__(name=generate_name('pool'),
- type=kCPooling, **kwargs)
- self.layer.pooling_conf.pool = PoolingProto().AVG
- fields = {'pool' : PoolingProto().AVG,
- 'kernel' : pool_size[0],
- 'stride' : stride,
- 'pad' : 0 if ignore_border else 1}
- setval(self.layer.pooling_conf, **fields)
-
-class LRN2D(Layer):
-
- def __init__(self, size=0, **kwargs):
- '''
- required
- size = (int) // local size
- '''
-
- super(LRN2D, self).__init__(name=generate_name('norm'), type=kLRN, **kwargs)
- # required
- assert size != 0, 'local size should be set'
- self.layer.lrn_conf.local_size = size
- init_values = get_init_values('lrn2d', **kwargs)
- setval(self.layer.lrn_conf, **init_values)
-
-class Loss(Layer):
-
- def __init__(self, lossname, topk=1, **kwargs):
- '''
- required
- lossname = (string) // softmaxloss, euclideanloss
- '''
- self.layer_type = enumLayerType(lossname)
- super(Loss, self).__init__(name=generate_name(lossname),
- type=self.layer_type, **kwargs)
- if lossname == 'softmaxloss':
- self.layer.softmaxloss_conf.topk = topk
-
-class Activation(Layer):
-
- def __init__(self, activation='stanh', **kwargs):
- '''
- required
- activation = (string) // relu, sigmoid, tanh, stanh, softmax.
- '''
- if activation == 'tanh':
- print 'Warning: Tanh layer is not supported for CPU'
-
- self.name = activation
- self.layer_type = kActivation
- if activation == 'stanh':
- self.layer_type = kSTanh
- elif activation == 'softmax':
- self.layer_type = kSoftmax
- super(Activation, self).__init__(name=generate_name(self.name),
- type=self.layer_type, **kwargs)
- if activation == 'relu':
- self.layer.activation_conf.type = RELU
- elif activation == 'sigmoid':
- self.layer.activation_conf.type = SIGMOID
- elif activation == 'tanh':
- self.layer.activation_conf.type = TANH # for GPU
- #elif activation == 'stanh':
- # self.layer.activation_conf.type = STANH
-
-
-class Dropout(Layer):
-
- def __init__(self, ratio=0.5):
- '''
- required
- ratio = (float) // ratio of drop out nodes
- '''
-
- self.name = 'dropout'
- self.layer_type = enumLayerType(self.name)
- super(Dropout, self).__init__(name=generate_name(self.name),
- type=self.layer_type, **kwargs)
- self.layer.dropout_conf.dropout_ratio = ratio
-
-class Accuracy(Layer):
-
- def __init__(self, **kwargs):
- '''
- '''
-
- self.name = 'accuracy'
- self.layer_type = enumLayerType(self.name)
- super(Accuracy, self).__init__(name=generate_name(self.name),
- type=self.layer_type, **kwargs)
-
-class RGB(Layer):
-
- def __init__(self, meanfile=None, **kwargs):
- '''
- required
- meanfile = (string) // path to meanfile (depreciated)
- '''
-
- assert meanfile != None, 'meanfile should be specified'
- self.name = 'rgb'
- self.layer_type = kRGBImage
- super(RGB, self).__init__(name=generate_name(self.name),
- type=self.layer_type)
- self.layer.rgbimage_conf.meanfile = meanfile
-
-class Dense(Layer):
-
- def __init__(self, output_dim=0, activation=None,
- init=None, w_param=None, b_param=None, input_dim=None,
- **kwargs):
- '''
- required
- output_dim = (int)
- optional
- activation = (string)
- init = (string) // 'uniform', 'gaussian', 'constant'
- w_param = (Parameter) // Parameter object for weight
- b_param = (Parameter) // Parameter object for bias
- **kwargs
- w_lr = (float) // learning rate multiplier for weight, used to
- // scale the learning rate when updating parameters.
- w_wd = (float) // weight decay multiplier for weight, used to
- // scale the weight decay when updating parameters.
- b_lr = (float) // learning rate multiplier for bias
- b_wd = (float) // weight decay multiplier for bias
- '''
- # required
- assert output_dim > 0, 'output_dim should be set'
- super(Dense, self).__init__(type=kInnerProduct, **kwargs)
- self.layer.innerproduct_conf.num_output = output_dim
- if 'transpose' in kwargs:
- self.layer.innerproduct_conf.transpose = kwargs['transpose']
-
- # parameter w (default: gaussian)
- if w_param == None:
- self.init = 'gaussian' if init == None else init
- w_param = Parameter(init=self.init)
- set_param_field(w_param.param, 'w', False, **kwargs)
- setval(self.layer, param=w_param.param)
-
- # parameter b (default: constant)
- if b_param == None:
- self.init = 'constant' if init == None else init
- b_param = Parameter(init=self.init)
- set_param_field(b_param.param, 'b', False, **kwargs)
- setval(self.layer, param=b_param.param)
-
- # following layers: e.g., activation, dropout, etc.
- if activation:
- self.mask = Activation(activation=activation).layer
-
-
-''' Classes to deal with multiple layers
-'''
-class Autoencoder(object):
-
- def __init__(self, hid_dim=None, out_dim=0,
- activation=None, param_share=True):
- '''
- Generate a set of layers (like MLP) for encoder and decoder
- The layers are expanded and added in Sequential.add()
-
- required
- hid_dim = (int/list) // the number of nodes in hidden layers
- out_dim = (int) // the number of nodes in the top layer
- optional
- activation = (string)
- param_share = (bool) // to share params in encoder and decoder
- '''
-
- # required
- assert out_dim > 0, 'out_dim should be set'
- self.out_dim = out_dim
- assert hid_dim != None, 'hid_dim should be set'
- self.hid_dim = [hid_dim] if type(hid_dim) == int else hid_dim
-
- self.layer_type = 'AutoEncoder'
- self.activation = activation
- self.param_share = param_share
-
-class RBM(Layer):
-
- def __init__(self, out_dim=None, w_param=None, b_param=None,
- sampling=None, **kwargs):
- '''
- Generate a set of layers (like MLP) according to the number of elements
- in out_dim, and on top of it, two layers RBMVis and RBMHid with
- bidirectional connection
- The layers are expanded and added in Energy.add()
-
- required
- out_dim = (int) or (int list) // the number of hidden nodes
- optional
- w_param = (Parameter) // Parameter object for weight
- b_param = (Parameter) // Parameter object for bias
- sampling = (string)
- '''
-
- assert out_dim > 0, 'out_dim should be set'
- self.out_dim = [out_dim] if type(out_dim) == int else out_dim
-
- self.name = kwargs['name'] if 'name' in kwargs else 'RBMVis'
- self.layer_type = kwargs['type'] if 'type' in kwargs else kRBMVis
- super(RBM, self).__init__(name=generate_name(self.name,
- withnumber=False),
- type=self.layer_type, **kwargs)
- setval(self.layer.rbm_conf, hdim=self.out_dim[-1])
- if self.layer_type == kRBMHid and sampling != None:
- if sampling == 'gaussian':
- setval(self.layer.rbm_conf, gaussian=True)
-
- # parameter w
- if w_param == None:
- w_param = Parameter(init='gaussian', **kwargs)
- set_param_field(w_param.param, 'w', withnumber=False,
- level=len(self.out_dim), **kwargs)
- else:
- if self.layer_type == kRBMHid:
- del kwargs['name']
- else:
- set_param_field(w_param.param, 'w', withnumber=False,
- level=len(self.out_dim), **kwargs)
- setval(self.layer, param=w_param.param)
-
- # parameter b
- if b_param == None:
- b_param = Parameter(init='constant', **kwargs)
- set_param_field(b_param.param, 'b', withnumber=False,
- level=len(self.out_dim), **kwargs)
- else:
- if self.layer_type == kRBMHid:
- pass
- else:
- set_param_field(b_param.param, 'b', withnumber=False,
- level=len(self.out_dim), **kwargs)
- setval(self.layer, param=b_param.param)
-
- if self.layer_type == kRBMVis:
- wname = w_param.param.name
- parw = Parameter(name=wname+"_", init='none', share_from=wname)
- bname = b_param.param.name
- parb = Parameter(name=bname+"2", wd=0, init='constant')
- self.bidirect = RBM(self.out_dim, name='RBMHid', type=kRBMHid,
- w_param=parw, b_param=parb, sampling=sampling).layer
-
-class Embedding(Layer):
-
- def __init__(self, in_dim, out_dim, w_param=None, **kwargs):
-
- super(Embedding, self).__init__(name=generate_name('embedding', 1),
- user_type='kEmbedding')
- fields = {'vocab_size': in_dim,
- 'word_dim': out_dim}
- setval(self.layer.Extensions[embedding_conf], **fields)
- if w_param == None:
- # default: uniform
- w_param = Parameter(name=generate_name('w'), init=init)
- else:
- set_param_field(w_param.param, 'w', True, **kwargs)
- setval(self.layer, param=w_param.param)
-
-class RNNLM(Layer):
-
- def __init__(self, dim, w_param=None, **kwargs):
-
- super(RNNLM, self).__init__(name=generate_name('hidden', 1),
- user_type='kHidden')
- if w_param == None:
- # default: uniform
- w_param = Parameter(name=generate_name('w'), init=init)
- else:
- set_param_field(w_param.param, 'w', True, **kwargs)
- setval(self.layer, param=w_param.param)
-
-class UserLossRNNLM(Layer):
-
- def __init__(self, **kwargs):
-
- super(UserLossRNNLM, self).__init__(name=generate_name('loss', 1),
- user_type='kLoss')
- self.layer.Extensions[loss_conf].nclass = kwargs['nclass']
- self.layer.Extensions[loss_conf].vocab_size = kwargs['vocab_size']
- setval(self.layer, param=Parameter(name=generate_name('w'),
- init='uniform', scale=0.3).param)
- setval(self.layer, param=Parameter(name=generate_name('w', 1),
- init='uniform', scale=0.3).param)
diff --git a/tool/python/singa/model.py b/tool/python/singa/model.py
deleted file mode 100644
index 4a6a688..0000000
--- a/tool/python/singa/model.py
+++ /dev/null
@@ -1,716 +0,0 @@
-#!/usr/bin/env python
-
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-'''
-This script includes Model class and its subclasses that
-users can configure model parameter.
-'''
-
-import sys, re, subprocess
-from singa.layer import *
-from singa.utils.utility import *
-from singa.utils.message import *
-from google.protobuf import text_format
-
-from singa.driver import Updater as SingaUpdater
-
-class Model(object):
- ''' Configure model parameter
- - add(): add layer
- - compile(): specify Updater and Cluster protos
- - build(): construct a model (i.e., NetProto)
- - fit(): run singa for training
- - evaluate(): run singa for testing
- '''
-
- def __init__(self, name='my model', argv=None, label=False):
- '''
- optional
- name = (string) // name of model/job
- argv // pass sys.argv to source
- label = (bool) // exist label layer (depreciated)
- '''
- self.jobconf = Message('Job', name=name).proto
- self.layers = []
- self.label = label
- self.argv = argv
- self.result = None
- self.last_checkpoint_path = None
- self.cudnn = False
- self.accuracy = False
-
- def add(self, layer):
- '''
- add layer
- '''
- pass
-
- def exist_datalayer(self, phase):
- '''
- check if data layer exists
- '''
- for ly in self.layers:
- if enumPhase(phase) in ly.layer.include:
- return True
- return False
-
- def compile(self, optimizer=None, cluster=None,
- loss=None, topk=1, **kwargs):
- '''
- required
- optimizer = (Updater) // updater settings, e.g., SGD
- cluster = (Cluster) // cluster settings
- optional
- loss = (string) // name of loss function type
- topk = (int) // nb of results considered to compute accuracy
- '''
- assert optimizer != None, 'optimizer (Updater component) should be set'
- assert cluster != None, 'cluster (Cluster component) should be set'
- setval(self.jobconf, updater=optimizer.proto)
- setval(self.jobconf, cluster=cluster.proto)
-
- # take care of loss function layer
- if loss == None:
- print 'loss layer is not set'
- else:
- if hasattr(self.layers[-1], 'mask'):
- ly = self.layers[-1].mask
- else:
- ly = self.layers[-1].layer
-
- # take care of the last layer
- if ly.type == enumLayerType('softmax'):
- # revise the last layer
- if loss == 'categorical_crossentropy':
- setval(ly, type=enumLayerType('softmaxloss'))
- setval(ly.softmaxloss_conf, topk=topk)
- elif loss == 'mean_squared_error':
- setval(ly, type=enumLayerType('euclideanloss'))
- else:
- # add new layer
- if loss == 'categorical_crossentropy':
- self.add(Loss('softmaxloss', topk=topk))
- elif loss == 'mean_squared_error':
- self.add(Loss('euclideanloss'))
- elif loss == 'user_loss_rnnlm': # user-defined loss layer
- self.add(UserLossRNNLM(nclass=kwargs['nclass'],
- vocab_size=kwargs['in_dim']))
-
- def build(self):
- '''
- construct neuralnet proto
- '''
- net = NetProto()
- slyname = self.layers[0].layer.name
- for i in range(len(self.layers)):
- ly = net.layer.add()
- ly.CopyFrom(self.layers[i].layer)
- lastly = ly
- if self.layers[i].is_datalayer == True:
- continue
- getattr(ly, 'srclayers').append(slyname)
- slyname = ly.name
- if hasattr(self.layers[i], 'mask'):
- mly = net.layer.add()
- mly.CopyFrom(self.layers[i].mask)
- getattr(mly, 'srclayers').append(slyname)
- slyname = mly.name
- lastly = mly
- if hasattr(self.layers[i], 'bidirect'):
- bly = net.layer.add()
- bly.CopyFrom(self.layers[i].bidirect)
- getattr(bly, 'srclayers').append(slyname)
-
- # deal with label layer (depreciated)
- if self.label == True:
- label_layer = Layer(name='label', type=kLabel)
- ly = net.layer.add()
- ly.CopyFrom(label_layer.layer)
- getattr(ly, 'srclayers').append(self.layers[0].layer.name)
- getattr(lastly, 'srclayers').append(label_layer.layer.name)
- else:
- if lastly.name == 'RBMVis':
- getattr(lastly, 'srclayers').append(bly.name)
- else:
- getattr(lastly, 'srclayers').append(self.layers[0].layer.name)
-
- if self.accuracy == True:
- smly = net.layer.add()
- smly.CopyFrom(Layer(name='softmax', type=kSoftmax).layer)
- setval(smly, include=kTest)
- getattr(smly, 'srclayers').append(self.layers[-1].layer.name)
- aly = net.layer.add()
- aly.CopyFrom(Accuracy().layer)
- setval(aly, include=kTest)
- getattr(aly, 'srclayers').append('softmax')
- getattr(aly, 'srclayers').append(self.layers[0].layer.name)
-
- # use of cudnn
- if self.cudnn == True:
- self.set_cudnn_layer_type(net)
-
- setval(self.jobconf, neuralnet=net)
-
- def fit(self, data=None, alg='bp', nb_epoch=0,
- with_test=False, execpath='', device=None, **fields):
- '''
- required
- data = (Data) // Data class object for training data
- alg = (string) // algorithm, e.g., 'bp', 'cd'
- nb_epoch = (int) // the number of training steps
- optional
- with_test = (bool) // flag if singa runs for test data
- execpath = (string) // path to user own singa (executable file)
- device = (int/list) // a list of gpu ids
- **fields (KEY=VALUE)
- batch_size = (int) // batch size for training data
- train_steps = (int) // nb of steps for training, i.e., epoch
- disp_freq = (int) // frequency to display training info
- disp_after = (int) // display after this number
- validate_data = (Data) // valid data, specified in load_data()
- validate_freq = (int) // frequency of validation
- validate_steps = (int) // total number of steps for validation
- validate_after = (int) // start validation after this number
- checkpoint_path = (string) // path to checkpoint file
- checkpoint_freq = (int) // frequency for checkpoint
- checkpoint_after = (int) // start checkpointing after this number
- '''
- assert data != None, 'Training data shold be set'
- assert nb_epoch > 0, 'Training steps shold be set'
-
- if 'batch_size' in fields: # if new value is set, replace it
- setval(data.layer.store_conf, batchsize=fields['batch_size'])
-
- # insert layer for training
- if self.exist_datalayer('train') == False:
- self.layers.insert(0, data)
- setval(self.jobconf, train_steps=nb_epoch)
- setval(self.jobconf, disp_freq=nb_epoch/10)
- if 'disp_freq' in fields:
- setval(self.jobconf, disp_freq=fields['disp_freq'])
-
- if 'validate_data' in fields:
- self.layers.insert(1, fields['validate_data'])
- setval(self.jobconf, validate_freq=nb_epoch/10)
-
- setval(self.jobconf, **fields)
-
- # loading checkpoint if it is set
- if data.checkpoint != None:
- setval(self.jobconf, checkpoint_path=data.checkpoint)
-
- # save model parameter (i.e., checkpoint_path)
- setval(self.jobconf, checkpoint_freq=nb_epoch)
- self.last_checkpoint_path = '{0}/step{1}-worker0'.format(
- self.jobconf.cluster.workspace, nb_epoch)
-
- # set Train_one_batch component, using backprogapation at default
- setval(self.jobconf,
- train_one_batch=Algorithm(type=enumAlgType(alg)).proto)
-
- # use of cudnn
- if device != None:
- setval(self.jobconf, gpu=device)
- self.cudnn = True
-
- # start to run singa for training
- if with_test == False:
- self.build() # construct Nneuralnet Component
- #self.display()
- return SingaRun(jobproto=self.jobconf,
- argv=self.argv, execpath=execpath)
- else:
- # run singa in evaluate() with test data
- pass
-
- def evaluate(self, data=None, alg='bp',
- checkpoint_path=None, execpath='',
- device=None, show_acc=False, **fields):
- '''
- required
- data = (Data) // Data class object for testing data
- optional
- alg = (string) // algorithm type, (bp at default)
- checkpoint_path = (list) // checkpoint path
- execpaths = (string) // path to user's own executable
- device = (int/list) // a list of gpu ids
- show_acc = (bool) // compute and the accuacy
- **fields (KEY=VALUE)
- batch_size = (int) // batch size for testing data
- test_freq = (int) // frequency of testing
- test_steps = (int) // total number of steps for testing
- test_after = (int) // start testing after this number of steps
- '''
- assert data != None, 'Testing data should be set'
- is_testonly = False
-
- if 'batch_size' in fields: # if new value is set, replace it
- setval(data.layer.store_conf, batchsize=fields['batch_size'])
-
- # insert layer for testing
- if self.exist_datalayer('test') == False:
- self.layers.insert(0, data)
-
- # loading checkpoint if singa runs only for testing
- if self.exist_datalayer('train') == False:
- is_testonly = True
- if checkpoint_path == None:
- print 'checkpoint_path has not been specified'
- else:
- setval(self.jobconf, checkpoint_path=checkpoint_path)
-
- steps = fields['test_steps'] if 'test_steps' in fields else 10
- setval(self.jobconf, test_steps=steps)
- setval(self.jobconf, **fields)
-
- # set Train_one_batch component, using backprogapation at default
- setval(self.jobconf,
- train_one_batch=Algorithm(type=enumAlgType(alg)).proto)
-
- # use of cudnn
- if device != None:
- setval(self.jobconf, gpu=device)
- self.cudnn = True
-
- # set True if showing the accuracy
- self.accuracy = show_acc
-
- self.build() # construct Nneuralnet Component
-
- #--- generate job.conf file for debug purpose
- #filename = 'job.conf'
- #with open(filename, 'w') as f:
- # f.write(text_format.MessageToString(self.jobconf.cluster))
- #self.display()
-
- #--- run singa ---
- return SingaRun(jobproto=self.jobconf,
- argv=self.argv, execpath=execpath, testmode=is_testonly)
- #return SingaRun_script(filename=filename, execpath=execpath)
-
-
- def display(self):
- ''' print out job proto
- '''
- print text_format.MessageToString(self.jobconf)
-
- def set_cudnn_layer_type(self, net):
- ''' convert LayerType to CdunnLayerType
- '''
- for i in range(len(net.layer)):
- ly_type = net.layer[i].type
- cudnn_ly_type = ly_type
- if ly_type == kCConvolution: cudnn_ly_type = kCudnnConv
- elif ly_type == kCPooling: cudnn_ly_type = kCudnnPool
- elif ly_type == kLRN: cudnn_ly_type = kCudnnLRN
- elif ly_type == kSoftmax: cudnn_ly_type = kCudnnSoftmax
- elif ly_type == kSoftmaxLoss: cudnn_ly_type = kCudnnSoftmaxLoss
- elif ly_type == kActivation:
- cudnn_ly_type = kCudnnActivation
- elif ly_type == kSTanh:
- print 'Error report: STanh layer is not supported for GPU'
- '''
- elif ly_type == kReLU:
- cudnn_ly_type = kCudnnActivation
- net.layer[i].activation_conf.type = RELU
- elif ly_type == kSigmoid:
- cudnn_ly_type = kCudnnActivation
- net.layer[i].activation_conf.type = SIGMOID
- elif ly_type == kTanh:
- cudnn_ly_type = kCudnnActivation
- net.layer[i].activation_conf.type = TANH
- '''
- #elif ly_type == kSTanh:
- # print 'Error report: STanh layer is not supported for GPU'
- #cudnn_ly_type = kCudnnActivation
- #net.layer[i].activation_conf.type = STANH
- net.layer[i].type = cudnn_ly_type
-
- def show(self):
- for ly in self.jobconf.neuralnet.layer:
- print layer(ly.name)
-
- def layer_by_id(self, k):
- return self.jobconf.neuralnet.layer[k]
-
- def layer_by_name(self, name):
- return self.layers[k]
-
- def size(self):
- return len(self.jobconf.neuralnet.layer)
-
-class Energy(Model):
- ''' energy model
- '''
-
- def __init__(self, name='my model', argv=[], label=False):
- super(Energy, self).__init__(name=name, argv=argv, label=label)
-
- def add(self, layer):
- if hasattr(layer, 'layer_type'):
- if layer.layer_type == kRBMVis:
- dim = 0
- for i in range(1, len(layer.out_dim)):
- parw = Parameter(name='w', init='none', level=i)
- parb = Parameter(name='b', init='none', level=i)
- dim = layer.out_dim[i-1]
- self.layers.append(Dense(dim, w_param=parw, b_param=parb,
- activation='sigmoid'))
- self.layers.append(layer)
-
-class Sequential(Model):
- ''' sequential model
- '''
-
- def __init__(self, name='my model', argv=[], label=False):
- super(Sequential, self).__init__(name=name, argv=argv, label=label)
-
- def add(self, layer):
- if hasattr(layer, 'layer_type'):
- if layer.layer_type == 'AutoEncoder':
- dim = 0
- if layer.param_share == True:
- # Encoding
- for i in range(1, len(layer.hid_dim)+1):
- parw = Parameter(name='w',
- init='none', level=i)
- parb = Parameter(name='b',
- init='none', level=i)
- dim = layer.hid_dim[i-1]
- if i == len(layer.hid_dim): activation = None
- else: activation = layer.activation
- self.layers.append(Dense(dim,
- w_param=parw, b_param=parb,
- activation=activation))
- # Decoding
- for i in range(len(layer.hid_dim), 0, -1):
- parw = Parameter(name=generate_name('w', 2),
- init='none')
- parb = Parameter(name=generate_name('b', 2),
- init='none')
- setval(parw.param, share_from='w'+str(i))
- setval(parb.param, name='b'+str(i))
- if i == 1: dim = layer.out_dim
- else: dim = layer.hid_dim[i-2]
- self.layers.append(Dense(dim,
- w_param=parw, b_param=parb,
- activation=layer.activation,
- transpose=True))
- else:
- # MLP
- for i in range(1, len(layer.hid_dim)+2):
- parw = Parameter(name='w',
- init='none', level=i)
- parb = Parameter(name='b',
- init='none', level=i)
- if i == len(layer.hid_dim)+1: dim = layer.out_dim
- else: dim = layer.hid_dim[i-1]
- self.layers.append(Dense(dim,
- w_param=parw, b_param=parb,
- activation=layer.activation))
- else:
- self.layers.append(layer)
- else:
- self.layers.append(layer)
-
-
-class Store(object):
-
- def __init__(self, **kwargs):
- '''
- **kwargs
- path = (string) // path to dataset
- backend = (string) //
- batch_size = (int) // batch size of dataset
- shape = (int) //
- '''
- self.proto = Message('Store', **kwargs).proto
-
-class Algorithm(object):
-
- def __init__(self, type=enumAlgType('bp'), **kwargs):
- '''
- type = (string) // type of algorithm, bp at default
- '''
- alg = Message('Alg', alg=type, **kwargs).proto
- if type == enumAlgType('cd'):
- setval(alg.cd_conf, **kwargs)
- self.proto = alg
-
-class Updater(object):
-
- def __init__(self, upd_type, lr, lr_type,
- decay, momentum,
- step, step_lr, **fields):
- '''
- required
- upd_type = (enum) // enum type of updater
- lr = (float) // base learning rate
- optional
- lr_type = (string) // type of the learning rate (Fixed at default)
- '''
- upd = Message('Updater', type=upd_type, **fields).proto
- setval(upd.learning_rate, base_lr=lr)
- if decay > 0:
- setval(upd, weight_decay=decay)
- if momentum > 0:
- setval(upd, momentum=momentum)
-
- if lr_type == None or lr_type == "fixed":
- setval(upd.learning_rate, type=kFixed)
- elif lr_type == 'step':
- cp = Message('Step', change_freq=60, gamma=0.997)
- setval(upd.learning_rate, type=kStep, step_conf=cp.proto)
- elif lr_type == 'manual':
- cp = Message('FixedStep', step=step, step_lr=step_lr)
- setval(upd.learning_rate, type=kFixedStep, fixedstep_conf=cp.proto)
- elif lr_type == 'linear':
- cp = Message('Linear', change_freq=10, final_lr=0.1)
- setval(upd.learning_rate, type=kLinear, linear_conf=cp.proto)
-
- self.proto = upd
- self.singaupdater = None
-
- def Update(self, step, layer):
- ''' This method updates parameters of layer
- step = (int) // training step, i.e., param version
- '''
- if self.singaupdater == None:
- self.singaupdater = SingaUpdater.CreateUpdater(
- self.proto.SerializeToString())
-
- # update parameters
- singaParams = layer.singalayer.GetParams()
- for par in singaParams:
- self.singaupdater.Update(step, par, 1.0)
-
-
-class SGD(Updater):
-
- def __init__(self, lr=0.01, lr_type=None,
- decay=0, momentum=0,
- step=(0), step_lr=(0.01), **fields):
- '''
- required
- lr = (float) // base learning rate
- optional
- lr_type = (string) // type of learning rate, 'Fixed' at default
- decay = (float) // weight decay
- momentum = (float) // momentum
- step = (int/list) // steps
- step_lr = (float/list) // learning rate after the steps
- **fields (KEY=VALUE)
- '''
- assert lr
- super(SGD, self).__init__(upd_type=kSGD,
- lr=lr, lr_type=lr_type,
- decay=decay, momentum=momentum,
- step=step, step_lr=step_lr, **fields)
-
-class AdaGrad(Updater):
-
- def __init__(self, lr=0.01, lr_type=None,
- decay=0, momentum=0,
- step=(0), step_lr=(0.01), **fields):
- '''
- required
- lr = (float) // base learning rate
- optional
- lr_type = (string) // type of learning rate, 'Fixed' at default
- decay = (float) // weight decay
- momentum = (float) // momentum
- step = (int/list) // steps
- step_lr = (float/list) // learning rate after the steps
- **fields (KEY=VALUE)
- '''
- assert lr
- super(AdaGrad, self).__init__(upd_type=kAdaGrad,
- lr=lr, lr_type=lr_type,
- decay=decay, momentum=momentum,
- step=step, step_lr=step_lr, **fields)
-
-class Cluster(object):
- """ Specify the cluster topology, e.g., number of workers/servers.
-
- Currently we need to create this object in the .py file and also provide a
- cluster configuration file to the command line. TODO(wangwei) update SINGA
- code to eliminate the requirement of the cluster configuration file for
- training on a single node or the cluster object in the pyfile for training
- in a cluster.
- """
-
- def __init__(self, workspace=None,
- nworker_groups=1, nserver_groups=1,
- nworkers_per_group=1, nservers_per_group=1,
- nworkers_per_procs=1, nservers_per_procs=1,
- **fields):
- '''
- required
- workspace = (string) // workspace path
- optional
- nworker_groups = (int)
- nserver_groups = (int)
- nworkers_per_group = (int)
- nservers_per_group = (int)
- nworkers_per_procs = (int)
- nservers_per_procs = (int)
- **fields
- server_worker_separate = (bool)
- '''
- assert workspace != None, 'need to set workspace'
- self.proto = Message('Cluster', workspace=workspace).proto
- # optional
- self.proto.nworker_groups = nworker_groups
- self.proto.nserver_groups = nserver_groups
- self.proto.nworkers_per_group = nworkers_per_group
- self.proto.nservers_per_group = nservers_per_group
- self.proto.nworkers_per_procs = nworkers_per_procs
- self.proto.nservers_per_procs = nservers_per_procs
- # other fields
- setval(self.proto, **fields)
-
-
-def StoreResults(lines):
- """ Parsing metrics from each line in the log file.
-
- TODO(wangwei) format the log string to make them uniform for easy parsing
- Another approach is creating a protobuf message for metrics, which can be
- used for dumping metrics to string and loading perf string back to messages.
- """
-
- resultDic = {}
- for line in lines:
- line = re.findall(r'[\w|*.*]+', line)
- if 'Train' in line:
- step = line[line.index('step')+1]
- if 'accuracy' in line:
- resultDic.setdefault(step, {})['acc'] \
- = line[line.index('accuracy')+1]
- if 'loss' in line:
- resultDic.setdefault(step, {})['loss'] \
- = line[line.index('loss')+1]
- if 'ppl' in line:
- resultDic.setdefault(step, {})['ppl'] \
- = line[line.index('ppl')+1]
- if 'Squared' in line:
- resultDic.setdefault(step, {})['se'] \
- = line[line.index('Squared')+2]
- return resultDic
-
-def SingaRun(jobproto='', argv=None, execpath='', testmode=False):
- """
- Run Singa and receive the training/test results.
- """
-
- import singa.driver as driver
- d = driver.Driver()
- d.InitLog(argv[0])
- d.Init(argv)
- if testmode == True:
- d.Test(jobproto.SerializeToString())
- else:
- d.Train(False, jobproto.SerializeToString())
-
- # Get the performance from the latest log file.
- # TODO(wangwei) the log file would be overwritten by other running instance
- # of the same program, e.g., lt-singa
- logfile = '/tmp/singa-log/{0}.ERROR'.format(argv[0].split('/')[-1])
- fin = open(logfile, 'r')
- result = StoreResults(fin.readlines())
-
- return result
-
-def SingaRun_script(filename='', execpath=''):
- """
- Deprecated.
- Generate the job conf file and run the shell command.
- """
- SINGAROOT = '../../../'
- conf = 'examples/' + filename
- if execpath == '':
- cmd = SINGAROOT+'bin/singa-run.sh ' \
- + '-conf %s ' % conf
- else:
- cmd = SINGAROOT+'bin/singa-run.sh ' \
- + '-conf %s ' % conf \
- + '-exec %s ' % execpath
-
- procs = subprocess.Popen(cmd.strip().split(' '),
- stdout=subprocess.PIPE,
- stderr=subprocess.STDOUT)
-
- resultDic = {}
- outputlines = iter(procs.stdout.readline, '')
- resultDic = StoreResults(outputlines)
-
- #TODO better format to store the result??
- return resultDic
-
-def load_model_parameter(fin, neuralnet, batchsize=1, data_shape=None):
- """
- this method loads model parameter
- """
- hly_idx = 0
- for i in range(len(neuralnet)):
- if neuralnet[i].is_datalayer:
- if data_shape == None:
- shape = neuralnet[i].shape
- shape[0] = batchsize
- neuralnet[i].setup(shape)
- else:
- neuralnet[i].setup(data_shape)
- else:
- hly_idx = i
- break
-
- net = layerVector(len(neuralnet)-hly_idx)
- for i in range(hly_idx, len(neuralnet)):
- if neuralnet[i].src==None:
- neuralnet[i].setup(neuralnet[i-1])
- else:
- neuralnet[i].setup(neuralnet[i].src)
- net[i-hly_idx] = neuralnet[i].singalayer
-
- from singa.driver import Worker
- alg = Algorithm(type=enumAlgType('bp')).proto
- w = Worker.CreateWorker(alg.SerializeToString())
- w.InitNetParams(fin, net)
-
-def save_model_parameter(step, fout, neuralnet):
- """
- this method saves model parameter
- """
- hly_idx = 0
- for i in range(len(neuralnet)):
- if not neuralnet[i].is_datalayer:
- hly_idx = i
- break
-
- from singa.driver import Worker
- net = layerVector(len(neuralnet)-hly_idx)
- for i in range(hly_idx, len(neuralnet)):
- net[i-hly_idx] = neuralnet[i].singalayer
- alg = Algorithm(type=enumAlgType('bp')).proto
- w = Worker.CreateWorker(alg.SerializeToString())
- w.Checkpoint(step, fout, net)
-
diff --git a/tool/python/singa/parameter.py b/tool/python/singa/parameter.py
deleted file mode 100644
index 14ad852..0000000
--- a/tool/python/singa/parameter.py
+++ /dev/null
@@ -1,140 +0,0 @@
-#!/usr/bin/env python
-
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-'''
-This script includes Parameter class and a method, named set_param_field
-that users can configure Param and ParamGen protos.
-'''
-
-from singa.initializations import get_init_values
-from singa.utils.utility import setval, generate_name
-from singa.utils.message import *
-from google.protobuf import text_format
-
-
-class Parameter(object):
-
- def __init__(self, **kwargs):
- '''
- optional
- **kwargs
- name = (string) // parameter name
- lr = (float) // learning rate multiplier
- wd = (float) // weight decay multiplier
- init = (string) // init type {'constant','uniform','gaussian'}
- value = (int) // value for 'constant'
- scale = (float) // [low=-scale, high=scale] for 'uniform'
- low = (float) // low value for 'uniform'
- high = (float) // high value for 'uniform'
- mean = (float) // mean for 'gaussian'
- std = (float) // std for 'gaussian'
- '''
- fields = {'lr_scale' : kwargs['lr'] if 'lr' in kwargs else 1,
- 'wd_scale' : kwargs['wd'] if 'wd' in kwargs else 1
- }
- self.param = Message('Param', **fields).proto
-
- if not 'name' in kwargs:
- setval(self.param, name=generate_name('param', 1))
- else:
- pname = kwargs['name']
- # parameter name for RBM
- if 'level' in kwargs:
- pname += str(kwargs['level'])
- if pname[0] == 'b':
- pname += '2'
- setval(self.param, name=pname)
-
- if 'share_from' in kwargs:
- setval(self.param, share_from=kwargs['share_from'])
-
- if 'init' in kwargs:
- init_values = get_init_values(kwargs['init'], **kwargs)
- if not kwargs['init'] == 'none':
- pgen = Message('ParamGen', type=enumInitMethod(kwargs['init']),
- **init_values)
- del kwargs['init']
- setval(self.param, init=pgen.proto)
- else: # default: uniform
- pgen = Message('ParamGen', type=enumInitMethod('uniform'))
- setval(self.param, init=pgen.proto)
-
- def update(self, **fields):
- setval(self.param, **fields)
- setval(self.param.init, **fields)
-
-
-def set_param_field(param, pname, changename=False, withnumber=True, **kwargs):
- '''
- param = (ParamProto)
- pname = (string) // 'w' for wiehgt, or 'b' for bias
- changename = (bool) // update parameter name if True
- withnumber = (bool) // add layer number if True
- **kwargs
- w_lr = (float) // learning rate multiplier for weight, used to
- // scale the learning rate when updating parameters.
- w_wd = (float) // weight decay multiplier for weight, used to
- // scale the weight decay when updating parameters.
- b_lr = (float) // learning rate multiplier for bias
- b_wd = (float) // weight decay multiplier for bias
- '''
- assert pname == 'w' or pname == 'b', 'pname should be w or b'
-
- lr_ = param.lr_scale
- wd_ = param.wd_scale
- initkv = {}
-
- if pname == 'w':
- if 'w_lr' in kwargs:
- lr_ = kwargs['w_lr']
- del kwargs['w_lr']
- if 'w_wd' in kwargs:
- wd_ = kwargs['w_wd']
- del kwargs['w_wd']
- for key, val in kwargs.items():
- if key.startswith('w_'):
- initkv[key[2:]] = val
-
- elif pname == 'b':
- if 'b_lr' in kwargs:
- lr_ = kwargs['b_lr']
- del kwargs['b_lr']
- if 'b_wd' in kwargs:
- wd_ = kwargs['b_wd']
- del kwargs['b_wd']
- for key, val in kwargs.items():
- if key.startswith('b_'):
- initkv[key[2:]] = val
-
- field = {'lr_scale' : lr_, 'wd_scale' : wd_}
-
- # Set/update parameter fields
- if param.name.startswith('param') or changename == True:
- if 'level' in kwargs: # parameter name for RBM
- pname += str(kwargs['level'])
- setval(param, name=generate_name(pname, withnumber=withnumber), **field)
- else:
- setval(param, **field)
-
- # Set/update parameter init fields
- setval(param.init, **initkv)
diff --git a/tool/python/singa/utils/__init__.py b/tool/python/singa/utils/__init__.py
deleted file mode 100644
index a796a7a..0000000
--- a/tool/python/singa/utils/__init__.py
+++ /dev/null
@@ -1,22 +0,0 @@
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-
diff --git a/tool/python/singa/utils/message.py b/tool/python/singa/utils/message.py
deleted file mode 100644
index bfa9ef2..0000000
--- a/tool/python/singa/utils/message.py
+++ /dev/null
@@ -1,80 +0,0 @@
-#!/usr/bin/env python
-
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-import sys, os
-from utility import *
-sys.path.append(os.path.join(os.path.dirname(__file__), '../../pb2'))
-
-'''
-This script reads proto files in ../../pb2, generated by proto buffer compiler.
- - Message class creates an object for proto and sets initial vlaues for
- the fields, specified by kwargs
- - make_function method generates a method named enumInitMethod that returns
- enum values of given enum type, defined in the proto files
-'''
-
-MODULE_LIST = []
-
-# import all modules in dir singa_root/tool/python/pb2
-# except common, singa, and __init__
-for f in os.listdir(os.path.join(os.path.dirname(__file__), '../../pb2')):
- if (f.endswith(".pyc")):
- continue
- if(f == "__init__.py" or f == "common_pb2.py" or f == "singa_pb2.py"):
- continue
- module_name = f.split('.')[0]
- module_obj = __import__(module_name)
- MODULE_LIST.append(module_obj)
- for func_name in dir(module_obj):
- if not func_name.startswith("__"):
- globals()[func_name] = getattr(module_obj, func_name)
-
-class Message(object):
- def __init__(self, protoname, **kwargs):
- for module_obj in MODULE_LIST:
- if hasattr(module_obj, protoname+"Proto"):
- class_ = getattr(module_obj, protoname+"Proto")
- self.proto = class_()
- return setval(self.proto, **kwargs)
- raise Exception('invalid protoname')
-
-enumDict_ = dict()
-
-#get all enum type list in the modules
-for module_obj in MODULE_LIST:
- for enumtype in module_obj.DESCRIPTOR.enum_types_by_name:
- tempDict = enumDict_[enumtype] = dict()
- for name in getattr(module_obj, enumtype).DESCRIPTOR.values_by_name:
- tempDict[name[1:].lower()] = getattr(module_obj, name)
-
-def make_function(enumtype):
- def _function(key):
- return enumDict_[enumtype][key]
- return _function
-
-current_module = sys.modules[__name__]
-
-#def all the enumtypes
-for module_obj in MODULE_LIST:
- for enumtype in module_obj.DESCRIPTOR.enum_types_by_name:
- setattr(current_module, "enum"+enumtype, make_function(enumtype))
diff --git a/tool/python/singa/utils/utility.py b/tool/python/singa/utils/utility.py
deleted file mode 100644
index b88720c..0000000
--- a/tool/python/singa/utils/utility.py
+++ /dev/null
@@ -1,86 +0,0 @@
-#!/usr/bin/env python
-
-#/************************************************************
-#*
-#* Licensed to the Apache Software Foundation (ASF) under one
-#* or more contributor license agreements. See the NOTICE file
-#* distributed with this work for additional information
-#* regarding copyright ownership. The ASF licenses this file
-#* to you under the Apache License, Version 2.0 (the
-#* "License"); you may not use this file except in compliance
-#* with the License. You may obtain a copy of the License at
-#*
-#* http://www.apache.org/licenses/LICENSE-2.0
-#*
-#* Unless required by applicable law or agreed to in writing,
-#* software distributed under the License is distributed on an
-#* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-#* KIND, either express or implied. See the License for the
-#* specific language governing permissions and limitations
-#* under the License.
-#*
-#*************************************************************/
-
-'''
-This script includes methods to
-(1) generate name of layer, parameter, etc.
-(2) set field values for proto.
-(3) swap bits
-'''
-
-LAYERID = 0
-PARAMID = 0
-
-def generate_name(label, option=0, withnumber=True):
- ''' This method returns name of layer or parameter with unique id.
- option: 1 to increase id number
- withnumber: True to concatenate number to name
- '''
-
- global LAYERID, PARAMID
- num = LAYERID
- if label == 'layer':
- if option == 1: LAYERID += 1
- num = LAYERID
- elif label == 'param':
- if option == 1: PARAMID += 1
- num = PARAMID
- else:
- if option == 1: LAYERID += 1
- num = LAYERID
- if option == 2:
- num = LAYERID+1
-
- if withnumber == False:
- return '{0}'.format(label)
-
- return '{0}{1}'.format(label, num)
-
-def setval(proto, **kwargs):
- ''' This method sets field values for give proto.
- '''
-
- for key, val in kwargs.items():
- #print 'kv: ', k, ', ', v
- if hasattr(proto, key):
- flabel = proto.DESCRIPTOR.fields_by_name[key].label
- ftype = proto.DESCRIPTOR.fields_by_name[key].type
-
- fattr = getattr(proto, key)
- if flabel == 3: # repeated field
- if ftype == 11: # message type
- fattr = fattr.add()
- fattr.MergeFrom(val)
- else:
- if type(val) == list or type(val) == tuple:
- for i in range(len(val)):
- fattr.append(val[i])
- else:
- fattr.append(val)
- else:
- if ftype == 11: # message type
- fattr = getattr(proto, key)
- fattr.MergeFrom(val)
- else:
- setattr(proto, key, val)
-
