commit | cde53611167c54ad7ce726a6b775faffcbdb0258 | [log] [tgz] |
---|---|---|
author | Matteo Salvarezza <matteo.salvarezza@gmail.com> | Mon May 15 18:43:59 2017 +0200 |
committer | Mu Li <muli@cs.cmu.edu> | Mon May 15 09:43:59 2017 -0700 |
tree | 97073c387a753a6a4c6488147ca6980fe53c8a22 | |
parent | 87927eebb80be498810df7753a51fb705a9152f1 [diff] |
Correctly import Caffe BatchNorm (#6176) * Correctly import Caffe BatchNorm * Compensate for cudnn epsilon shift by changing the variance Cudnn requires BatchNorm variance eps to be bigger than 1e-05 (CUDNN_BN_MIN_EPSILON). Before this commit eps values were clipped to 1.1e-05, thus introducing a small numerical discrepancy in evaluation. This discrepancy is avoided here by compensating from this shift in the actual variance value. * Improved epsilon shift compensation and comments
MXNet is a deep learning framework designed for both efficiency and flexibility. It allows you to mix symbolic and imperative programming to maximize efficiency and productivity. At its core, MXNet contains a dynamic dependency scheduler that automatically parallelizes both symbolic and imperative operations on the fly. A graph optimization layer on top of that makes symbolic execution fast and memory efficient. MXNet is portable and lightweight, scaling effectively to multiple GPUs and multiple machines.
MXNet is also more than a deep learning project. It is also a collection of blue prints and guidelines for building deep learning systems, and interesting insights of DL systems for hackers.
© Contributors, 2015-2017. Licensed under an Apache-2.0 license.
Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems. In Neural Information Processing Systems, Workshop on Machine Learning Systems, 2015
MXNet emerged from a collaboration by the authors of cxxnet, minerva, and purine2. The project reflects what we have learned from the past projects. MXNet combines aspects of each of these projects to achieve flexibility, speed, and memory efficiency.