blob: a75ceea2b1c29aaf568b6118a9d5516e57f96ac3 [file] [log] [blame]
Release Notes - SINGA - Version singa-incubating-2.0.0
SINGA is a general distributed deep learning platform for training big deep
learning models over large datasets.
This release includes following features:
* Core components
* [SINGA-434] Support tensor broadcasting
* [SINGA-370] Improvement to tensor reshape and various misc. changes related to SINGA-341 and 351
* Model components
* [SINGA-333] Add support for Open Neural Network Exchange (ONNX) format
* [SINGA-385] Add new python module for optimizers
* [SINGA-394] Improve the CPP operations via Intel MKL DNN lib
* [SINGA-425] Add 3 operators , Abs(), Exp() and leakyrelu(), for Autograd
* [SINGA-410] Add two function, set_params() and get_params(), for Autograd Layer class
* [SINGA-383] Add Separable Convolution for autograd
* [SINGA-388] Develop some RNN layers by calling tiny operations like matmul, addbias.
* [SINGA-382] Implement concat operation for autograd
* [SINGA-378] Implement maxpooling operation and its related functions for autograd
* [SINGA-379] Implement batchnorm operation and its related functions for autograd
* Utility functions and CI
* [SINGA-432] Update depdent lib versions in conda-build config
* [SINGA-429] Update docker images for latest cuda and cudnn
* [SINGA-428] Move Docker images under Apache user name
* Documentation and usability
* [SINGA-395] Add documentation for autograd APIs
* [SINGA-344] Add a GAN example
* [SINGA-390] Update installation.md
* [SINGA-384] Implement ResNet using autograd API
* [SINGA-352] Complete SINGA documentation in Chinese version
* Bugs fixed
* [SINGA-431] Unit Test failed - Tensor Transpose
* [SINGA-422] ModuleNotFoundError: No module named "_singa_wrap"
* [SINGA-418] Unsupportive type 'long' in python3.
* [SINGA-409] Basic `singa-cpu` import throws error
* [SINGA-408] Unsupportive function definition in python3
* [SINGA-380] Fix bugs from Reshape
---------------------------------------------------------------
Release Notes - SINGA - Version singa-incubating-1.2.0
SINGA is a general distributed deep learning platform for training big deep
learning models over large datasets.
This release includes following features:
* Core components
* [SINGA-290] Upgrade to Python 3
* [SINGA-341] Added stride functionality to tensors for CPP
* [SINGA-347] Create a function that supports einsum
* [SINGA-351] Added stride support and cudnn codes to cuda
* Model components
* [SINGA-300] Add residual networks for imagenet classification
* [SINGA-312] Rename layer parameters
* [SINGA-313] Add L2 norm layer
* [SINGA-315] Reduce memory footprint by Python generator for parameter
* [SINGA-316] Add SigmoidCrossEntropy
* [SINGA-324] Extend RNN layer to accept variant seq length across batches
* [SINGA-326] Add Inception V4 for ImageNet classification
* [SINGA-328] Add VGG models for ImageNet classification
* [SINGA-329] Support layer freezing during training (fine-tuning)
* [SINGA-346] Update cudnn from V5 to V7
* [SINGA-349] Create layer operations for autograd
* [SINGA-363] Add DenseNet for Imagenet classification
* Utility functions and CI
* [SINGA-274] Improve Debian packaging with CPack
* [SINGA-303] Create conda packages
* [SINGA-337] Add test cases for code
* [SINGA-348] Support autograd MLP Example
* [SINGA-345] Update Jenkins and fix bugs in compliation
* [SINGA-354] Update travis scripts to use conda-build for all platforms
* [SINGA-358] Consolidated RUN steps and cleaned caches in Docker containers
* [SINGA-359] Create alias for conda packages
* Documentation and usability
* [SINGA-223] Fix side navigation menu in the website
* [SINGA-294] Add instructions to run CUDA unit tests on Windows
* [SINGA-305] Add jupyter notebooks for SINGA V1 tutorial
* [SINGA-319] Fix link errors on the index page
* [SINGA-352] Complete SINGA documentation in Chinese version
* [SINGA-361] Add git instructions for contributors and committers
* Bugs fixed
* [SINGA-330] fix openblas building on i7 7700k
* [SINGA-331] Fix the bug of tensor division operation
* [SINGA-350] Error from python3 test
* [SINGA-356] Error using travis tool to build SINGA on mac os
* [SINGA-363] Fix some bugs in imagenet examples
* [SINGA-368] Fix the bug in Cifar10 examples
* [SINGA-369] the errors of examples in testing
---------------------------------------------------------------
Release Notes - SINGA - Version singa-incubating-1.1.0
SINGA is a general distributed deep learning platform for training big deep learning models over large datasets.
This release includes following features:
* Core components
* [SINGA-296] Add sign and to_host function for pysinga tensor module
* Model components
* [SINGA-254] Implement Adam for V1
* [SINGA-264] Extend the FeedForwardNet to accept multiple inputs
* [SINGA-267] Add spatial mode in batch normalization layer
* [SINGA-271] Add Concat and Slice layers
* [SINGA-275] Add Cross Entropy Loss for multiple labels
* [SINGA-278] Convert trained caffe parameters to singa
* [SINGA-287] Add memory size check for cudnn convolution
* Utility functions and CI
* [SINGA-242] Compile all source files into a single library.
* [SINGA-244] Separating swig interface and python binding files
* [SINGA-246] Imgtool for image augmentation
* [SINGA-247] Add windows support for singa
* [SINGA-251] Implement image loader for pysinga
* [SINGA-252] Use the snapshot methods to dump and load models for pysinga
* [SINGA-255] Compile mandatory depedent libaries together with SINGA code
* [SINGA-259] Add maven pom file for building java classes
* [SINGA-261] Add version ID into the checkpoint files
* [SINGA-266] Add Rafiki python toolkits
* [SINGA-273] Improve license and contributions
* [SINGA-284] Add python unittest into Jenkins and link static libs into whl file
* [SINGA-280] Jenkins CI support
* [SINGA-288] Publish wheel of PySINGA generated by Jenkins to public servers
* Documentation and usability
* [SINGA-263] Create Amazon Machine Image
* [SINGA-268] Add IPython notebooks to the documentation
* [SINGA-276] Create docker images
* [SINGA-289] Update SINGA website automatically using Jenkins
* [SINGA-295] Add an example of image classification using GoogleNet
* Bugs fixed
* [SINGA-245] float as the first operand can not multiply with a tensor object
* [SINGA-293] Bug from compiling PySINGA on Mac OS X with multiple version of Python
---------------------------------------------------------------
Release Notes - SINGA - Version singa-incubating-1.0.0
SINGA is a general distributed deep learning platform for training big deep learning models over large datasets.
This release includes following features:
* Core abstractions including Tensor and Device
* [SINGA-207] Update Tensor functions for matrices
* [SINGA-205] Enable slice and concatenate operations for Tensor objects
* [SINGA-197] Add CNMem as a submodule in lib/
* [SINGA-196] Rename class Blob to Block
* [SINGA-194] Add a Platform singleton
* [SINGA-175] Add memory management APIs and implement a subclass using CNMeM
* [SINGA-173] OpenCL Implementation
* [SINGA-171] Create CppDevice and CudaDevice
* [SINGA-168] Implement Cpp Math functions APIs
* [SINGA-162] Overview of features for V1.x
* [SINGA-165] Add cross-platform timer API to singa
* [SINGA-167] Add Tensor Math function APIs
* [SINGA-166] light built-in logging for making glog optional
* [SINGA-164] Add the base Tensor class
* IO components for file read/write, network and data pre-processing
* [SINGA-233] New communication interface
* [SINGA-215] Implement Image Transformation for Image Pre-processing
* [SINGA-214] Add LMDBReader and LMDBWriter for LMDB
* [SINGA-213] Implement Encoder and Decoder for CSV
* [SINGA-211] Add TextFileReader and TextFileWriter for CSV files
* [SINGA-210] Enable checkpoint and resume for v1.0
* [SINGA-208] Add DataIter base class and a simple implementation
* [SINGA-203] Add OpenCV detection for cmake compilation
* [SINGA-202] Add reader and writer for binary file
* [SINGA-200] Implement Encoder and Decoder for data pre-processing
* Module components including layer classes, training algorithms and Python binding
* [SINGA-235] Unify the engines for cudnn and singa layers
* [SINGA-230] OpenCL Convolution layer and Pooling layer
* [SINGA-222] Fixed bugs in IO
* [SINGA-218] Implementation for RNN CUDNN version
* [SINGA-204] Support the training of feed-forward neural nets
* [SINGA-199] Implement Python classes for SGD optimizers
* [SINGA-198] Change Layer::Setup API to include input Tensor shapes
* [SINGA-193] Add Python layers
* [SINGA-192] Implement optimization algorithms for SINGA v1 (nesterove, adagrad, rmsprop)
* [SINGA-191] Add "autotune" for CudnnConvolution Layer
* [SINGA-190] Add prelu layer and flatten layer
* [SINGA-189] Generate python outputs of proto files
* [SINGA-188] Add Dense layer
* [SINGA-187] Add popular parameter initialization methods
* [SINGA-186] Create Python Tensor class
* [SINGA-184] Add Cross Entropy loss computation
* [SINGA-183] Add the base classes for optimizer, constraint and regularizer
* [SINGA-180] Add Activation layer and Softmax layer
* [SINGA-178] Add Convolution layer and Pooling layer
* [SINGA-176] Add loss and metric base classes
* [SINGA-174] Add Batch Normalization layer and Local Response Nomalization layer.
* [SINGA-170] Add Dropout layer and CudnnDropout layer.
* [SINGA-169] Add base Layer class for V1.0
* Examples
* [SINGA-232] Alexnet on Imagenet
* [SINGA-231] Batchnormlized VGG model for cifar-10
* [SINGA-228] Add Cpp Version of Convolution and Pooling layer
* [SINGA-227] Add Split and Merge Layer and add ResNet Implementation
* Documentation
* [SINGA-239] Transfer documentation files of v0.3.0 to github
* [SINGA-238] RBM on mnist
* [SINGA-225] Documentation for installation and Cifar10 example
* [SINGA-223] Use Sphinx to create the website
* Tools for compilation and some utility code
* [SINGA-229] Complete install targets
* [SINGA-221] Support for Travis-CI
* [SINGA-217] build python package with setup.py
* [SINGA-216] add jenkins for CI support
* [SINGA-212] Disable the compilation of libcnmem if USE_CUDA is OFF
* [SINGA-195] Channel for sending training statistics
* [SINGA-185] Add CBLAS and GLOG detection for singav1
* [SINGA-181] Add NVCC supporting for .cu files
* [SINGA-177] Add fully cmake supporting for the compilation of singa_v1
* [SINGA-172] Add CMake supporting for Cuda and Cudnn libs
----------------------------------------------------------
Release Notes - SINGA - Version singa-incubating-0.3.0
SINGA is a general distributed deep learning platform for training big deep learning models over large datasets.
This release includes following features:
* GPU Support
* [SINGA-131] Implement and optimize hybrid training using both CPU and GPU
* [SINGA-136] Support cuDNN v4
* [SINGA-134] Extend SINGA to run over a GPU cluster
* [SINGA-157] Change the priority of cudnn library and install libsingagpu.so
* Remove Dependencies
* [SINGA-156] Remove the dependency on ZMQ for single process training
* [SINGA-155] Remove zookeeper for single-process training
* Python Binding
* [SINGA-126] Python Binding for Interactive Training
* Other Improvements
* [SINGA-80] New Blob Level and Address Level Math Operation Interface
* [SINGA-130] Data Prefetching
* [SINGA-145] New SGD based optimization Updaters: AdaDelta, Adam, AdamMax
* Bugs Fixed
* [SINGA-148] Race condition between Worker threads and Driver
* [SINGA-150] Mesos Docker container failed
* [SIGNA-141] Undesired Hash collision when locating process id to worker
* [SINGA-149] Docker build fail
* [SINGA-143] The compilation cannot detect libsingagpu.so file
-----------------------------------------
Release Notes - SINGA - Version singa-incubating-0.2.0
SINGA is a general distributed deep learning platform for training big deep learning models over large datasets. It is
designed with an intuitive programming model based on the layer abstraction. SINGA supports a wide variety of popular
deep learning models.
This release includes following features:
* Programming model
* [SINGA-80] New Blob Level and Address Level Math Operation Interface
* [SINGA-82] Refactor input layers using data store abstraction
* [SINGA-87] Replace exclude field to include field for layer configuration
* [SINGA-110] Add Layer member datavec_ and gradvec_
* [SINGA-120] Implemented GRU and BPTT (BPTTWorker)
* Neuralnet layers
* [SINGA-91] Add SoftmaxLayer and ArgSortLayer
* [SINGA-106] Add dummy layer for test purpose
* [SINGA-120] Implemented GRU and BPTT (GRULayer and OneHotLayer)
* GPU training support
* [SINGA-100] Implement layers using CUDNN for GPU training
* [SINGA-104] Add Context Class
* [SINGA-105] Update GUN make files for compiling cuda related code
* [SINGA-98] Add Support for AlexNet ImageNet Classification Model
* Model/Hybrid partition
* [SINGA-109] Refine bridge layers
* [SINGA-111] Add slice, concate and split layers
* [SINGA-113] Model/Hybrid Partition Support
* Python binding
* [SINGA-108] Add Python wrapper to singa
* Predict-only mode
* [SINGA-85] Add functions for extracting features and test new data
* Integrate with third-party tools
* [SINGA-11] Start SINGA on Apache Mesos
* [SINGA-78] Use Doxygen to generate documentation
* [SINGA-89] Add Docker support
* Unit test
* [SINGA-95] Add make test after building
* Other improvment
* [SINGA-84] Header Files Rearrange
* [SINGA-93] Remove the asterisk in the log tcp://169.254.12.152:*:49152
* [SINGA-94] Move call to google::InitGoogleLogging() from Driver::Init() to main()
* [SINGA-96] Add Momentum to Cifar10 Example
* [SINGA-101] Add ll (ls -l) command in .bashrc file when using docker
* [SINGA-114] Remove short logs in tmp directory
* [SINGA-115] Print layer debug information in the neural net graph file
* [SINGA-118] Make protobuf LayerType field id easy to assign
* [SIGNA-97] Add HDFS Store
* Bugs fixed
* [SINGA-85] Fix compilation errors in examples
* [SINGA-90] Miscellaneous trivial bug fixes
* [SINGA-107] Error from loading pre-trained params for training stacked RBMs
* [SINGA-116] Fix a bug in InnerProductLayer caused by weight matrix sharing
-----------------------------------------
Features included in singa-incubating-0.1.0:
* Job management
* [SINGA-3] Use Zookeeper to check stopping (finish) time of the system
* [SINGA-16] Runtime Process id Management
* [SINGA-25] Setup glog output path
* [SINGA-26] Run distributed training in a single command
* [SINGA-30] Enhance easy-to-use feature and support concurrent jobs
* [SINGA-33] Automatically launch a number of processes in the cluster
* [SINGA-34] Support external zookeeper service
* [SINGA-38] Support concurrent jobs
* [SINGA-39] Avoid ssh in scripts for single node environment
* [SINGA-43] Remove Job-related output from workspace
* [SINGA-56] No automatic launching of zookeeper service
* [SINGA-73] Refine the selection of available hosts from host list
* Installation with GNU Auto tool
* [SINGA-4] Refine thirdparty-dependency installation
* [SINGA-13] Separate intermediate files of compilation from source files
* [SINGA-17] Add root permission within thirdparty/install.
* [SINGA-27] Generate python modules for proto objects
* [SINGA-53] Add lmdb compiling options
* [SINGA-62] Remove building scrips and auxiliary files
* [SINGA-67] Add singatest into build targets
* Distributed training
* [SINGA-7] Implement shared memory Hogwild algorithm
* [SINGA-8] Implement distributed Hogwild
* [SINGA-19] Slice large Param objects for load-balance
* [SINGA-29] Update NeuralNet class to enable layer partition type customization
* [SINGA-24] Implement Downpour training framework
* [SINGA-32] Implement AllReduce training framework
* [SINGA-57] Improve Distributed Hogwild
* Training algorithms for different model categories
* [SINGA-9] Add Support for Restricted Boltzman Machine (RBM) model
* [SINGA-10] Add Support for Recurrent Neural Networks (RNN)
* Checkpoint and restore
* [SINGA-12] Support Checkpoint and Restore
* Unit test
* [SINGA-64] Add the test module for utils/common
* Programming model
* [SINGA-36] Refactor job configuration, driver program and scripts
* [SINGA-37] Enable users to set parameter sharing in model configuration
* [SINGA-54] Refactor job configuration to move fields in ModelProto out
* [SINGA-55] Refactor main.cc and singa.h
* [SINGA-61] Support user defined classes
* [SINGA-65] Add an example of writing user-defined layers
* Other features
* [SINGA-6] Implement thread-safe singleton
* [SINGA-18] Update API for displaying performance metric
* [SINGA-77] Integrate with Apache RAT
Some bugs are fixed during the development of this release
* [SINGA-2] Check failed: zsock_connect
* [SINGA-5] Server early terminate when zookeeper singa folder is not initially empty
* [SINGA-15] Fixg a bug from ConnectStub function which gets stuck for connecting layer_dealer_
* [SINGA-22] Cannot find openblas library when it is installed in default path
* [SINGA-23] Libtool version mismatch error.
* [SINGA-28] Fix a bug from topology sort of Graph
* [SINGA-42] Issue when loading checkpoints
* [SINGA-44] A bug when reseting metric values
* [SINGA-46] Fix a bug in updater.cc to scale the gradients
* [SINGA-47] Fix a bug in data layers that leads to out-of-memory when group size is too large
* [SINGA-48] Fix a bug in trainer.cc that assigns the same NeuralNet instance to workers from diff groups
* [SINGA-49] Fix a bug in HandlePutMsg func that sets param fields to invalid values
* [SINGA-66] Fix bugs in Worker::RunOneBatch function and ClusterProto
* [SINGA-79] Fix bug in singatool that can not parse -conf flag
Features planned for the next release
* [SINGA-11] Start SINGA using Mesos
* [SINGA-31] Extend Blob to support xpu (cpu or gpu)
* [SINGA-35] Add random number generators
* [SINGA-40] Support sparse Param update
* [SINGA-41] Support single node single GPU training