| commit | e8ff13ce6bb84a0f22fe7a4a6edca1312bbe8ef0 | [log] [tgz] |
|---|---|---|
| author | Paweł Głomski <pawel.glomski@intel.com> | Thu Mar 31 09:42:36 2022 +0200 |
| committer | GitHub <noreply@github.com> | Thu Mar 31 09:42:36 2022 +0200 |
| tree | 8d81968264041bf0739b31b745c3f6e6b4032e84 | |
| parent | 4672ca186f911abdcee5bf178c73f4d13f6fb0ce [diff] |
Improve AMP, bf16 support. Support oneDNN ops in AMP (#20753) * Add amp_cast fuse * Do not fuse with quantized nodes * Port to 2.0, add convolution fuse support * Fix offline casting of parameters * Fix symbol_bf16 * for review * Fix amp fuse with branched output, add bf16 support in necessary places * Map dtype to name in a switch * Handle mutable inputs for widest_dtype nodes * Correct handling amp_cast nodes from the original graph * Add license * Fix sanity * Fix InferType functions * Fix importing of bf16 parameters, minor fixes * Refactor * Fix tests * Fix amp_out_dtype description * Simplify AMP python code, unify dtype operations * WIP compiled * Fix issues * Add convolution to amp_cast fuse op list * Fix tests * Undo infertype changes in unrelated ops * Fix sanity * Fix dtype comparison, standardize dtype operations * Mark AMP tests to use np semantics * Remove redundant const * Disable new bf16 tests on macos * Enable bf16 concat test * Remove unused pass * Use actual node entries for mapping * Remove extra ops from bf16 amp lists * Review fixes + comments and cleanup * Review fixes * Review fixes + additional checks during conversion * Hybridize only when necessary * Fix warning message * Review fixes
Apache MXNet is a deep learning framework designed for both efficiency and flexibility. It allows you to mix symbolic and imperative programming to maximize efficiency and productivity. At its core, MXNet contains a dynamic dependency scheduler that automatically parallelizes both symbolic and imperative operations on the fly. A graph optimization layer on top of that makes symbolic execution fast and memory efficient. MXNet is portable and lightweight, scalable to many GPUs and machines.
MXNet is more than a deep learning project. It is a community on a mission of democratizing AI. It is a collection of blue prints and guidelines for building deep learning systems, and interesting insights of DL systems for hackers.
Licensed under an Apache-2.0 license.
| Branch | Build Status |
|---|---|
| master |             |
| v1.x |            |
| Channel | Purpose |
|---|---|
| Follow MXNet Development on Github | See what's going on in the MXNet project. |
| MXNet Confluence Wiki for Developers | MXNet developer wiki for information related to project development, maintained by contributors and developers. To request write access, send an email to send request to the dev list . |
| dev@mxnet.apache.org mailing list | The “dev list”. Discussions about the development of MXNet. To subscribe, send an email to dev-subscribe@mxnet.apache.org . |
| discuss.mxnet.io | Asking & answering MXNet usage questions. |
| Apache Slack #mxnet Channel | Connect with MXNet and other Apache developers. To join the MXNet slack channel send request to the dev list . |
| Follow MXNet on Social Media | Get updates about new features and events. |
Keep connected with the latest MXNet news and updates.
MXNet emerged from a collaboration by the authors of cxxnet, minerva, and purine2. The project reflects what we have learned from the past projects. MXNet combines aspects of each of these projects to achieve flexibility, speed, and memory efficiency.
Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems. In Neural Information Processing Systems, Workshop on Machine Learning Systems, 2015
