| { |
| "cells": [ |
| { |
| "cell_type": "markdown", |
| "id": "793fe801", |
| "metadata": {}, |
| "source": [ |
| "<!--- Licensed to the Apache Software Foundation (ASF) under one -->\n", |
| "<!--- or more contributor license agreements. See the NOTICE file -->\n", |
| "<!--- distributed with this work for additional information -->\n", |
| "<!--- regarding copyright ownership. The ASF licenses this file -->\n", |
| "<!--- to you under the Apache License, Version 2.0 (the -->\n", |
| "<!--- \"License\"); you may not use this file except in compliance -->\n", |
| "<!--- with the License. You may obtain a copy of the License at -->\n", |
| "\n", |
| "<!--- http://www.apache.org/licenses/LICENSE-2.0 -->\n", |
| "\n", |
| "<!--- Unless required by applicable law or agreed to in writing, -->\n", |
| "<!--- software distributed under the License is distributed on an -->\n", |
| "<!--- \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->\n", |
| "<!--- KIND, either express or implied. See the License for the -->\n", |
| "<!--- specific language governing permissions and limitations -->\n", |
| "<!--- under the License. -->\n", |
| "\n", |
| "# Install MXNet with ONEDNN\n", |
| "\n", |
| "A better training and inference performance is expected to be achieved on Intel-Architecture CPUs with MXNet built with [Intel ONEDNN](https://github.com/oneapi-src/oneDNN) on multiple operating system, including Linux, Windows and MacOS.\n", |
| "In the following sections, you will find build instructions for MXNet with Intel ONEDNN on Linux, MacOS and Windows.\n", |
| "\n", |
| "Please find ONEDNN optimized operators and other features in the [ONEDNN operator list](https://github.com/apache/incubator-mxnet/blob/v1.5.x/docs/tutorials/mkldnn/operator_list.md).\n", |
| "\n", |
| "The detailed performance data collected on Intel Xeon CPU with MXNet built with Intel ONEDNN can be found [here](https://mxnet.apache.org/api/faq/perf#intel-cpu).\n", |
| "\n", |
| "\n", |
| "<h2 id=\"0\">Contents</h2>\n", |
| "\n", |
| "* [1. Linux](#1)\n", |
| "* [2. MacOS](#2)\n", |
| "* [3. Windows](#3)\n", |
| "* [4. Verify MXNet with python](#4)\n", |
| "* [5. Enable MKL BLAS](#5)\n", |
| "* [6. Enable graph optimization](#6)\n", |
| "* [7. Quantization](#7)\n", |
| "* [8. Support](#8)\n", |
| "\n", |
| "<h2 id=\"1\">Linux</h2>\n", |
| "\n", |
| "### Prerequisites" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "bb0f46a0", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "sudo apt-get update\n", |
| "sudo apt-get install -y build-essential git\n", |
| "sudo apt-get install -y libopenblas-dev liblapack-dev\n", |
| "sudo apt-get install -y libopencv-dev\n", |
| "sudo apt-get install -y graphviz\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "e31ebc32", |
| "metadata": {}, |
| "source": [ |
| "### Clone MXNet sources" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "2df7850f", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "git clone --recursive https://github.com/apache/incubator-mxnet.git\n", |
| "cd incubator-mxnet\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "bbe7282b", |
| "metadata": {}, |
| "source": [ |
| "### Build MXNet with ONEDNN\n", |
| "\n", |
| "To achieve better performance, the Intel OpenMP and llvm OpenMP are recommended as below instruction. Otherwise, default GNU OpenMP will be used and you may get the sub-optimal performance. If you don't have the full [MKL](https://software.intel.com/en-us/intel-mkl) library installation, you might use OpenBLAS as the blas library, by setting USE_BLAS=openblas." |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "92f0a4dc", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "# build with llvm OpenMP and Intel MKL/openblas\n", |
| "mkdir build && cd build\n", |
| "cmake -DUSE_CUDA=OFF -DUSE_ONEDNN=ON -DUSE_OPENMP=ON -DUSE_OPENCV=ON ..\n", |
| "make -j $(nproc)\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "e7f33710", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "# build with Intel MKL and Intel OpenMP\n", |
| "make -j $(nproc) USE_OPENCV=1 USE_ONEDNN=1 USE_BLAS=mkl USE_INTEL_PATH=/opt/intel\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "0e232e79", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "# build with openblas and GNU OpenMP(sub-optimal performance)\n", |
| "make -j $(nproc) USE_OPENCV=1 USE_ONEDNN=1 USE_BLAS=openblas\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "039f8463", |
| "metadata": {}, |
| "source": [ |
| "<h2 id=\"2\">MacOS</h2>\n", |
| "\n", |
| "### Prerequisites\n", |
| "\n", |
| "Install the dependencies, required for MXNet, with the following commands:\n", |
| "\n", |
| "- [Homebrew](https://brew.sh/)\n", |
| "- llvm (clang in macOS does not support OpenMP)\n", |
| "- OpenCV (for computer vision operations)" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "7238a117", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "# Paste this command in Mac terminal to install Homebrew\n", |
| "/usr/bin/ruby -e \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)\"\n", |
| "\n", |
| "# install dependency\n", |
| "brew update\n", |
| "brew install pkg-config\n", |
| "brew install graphviz\n", |
| "brew tap homebrew/core\n", |
| "brew install opencv\n", |
| "brew tap homebrew/versions\n", |
| "brew install llvm\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "12761bff", |
| "metadata": {}, |
| "source": [ |
| "### Clone MXNet sources" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "97693091", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "git clone --recursive https://github.com/apache/incubator-mxnet.git\n", |
| "cd incubator-mxnet\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "32be6dd9", |
| "metadata": {}, |
| "source": [ |
| "### Build MXNet with ONEDNN" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "c30dfbc8", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "LIBRARY_PATH=$(brew --prefix llvm)/lib/ make -j $(sysctl -n hw.ncpu) CC=$(brew --prefix llvm)/bin/clang CXX=$(brew --prefix llvm)/bin/clang++ USE_OPENCV=1 USE_OPENMP=1 USE_ONEDNN=1 USE_BLAS=apple\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "f4166427", |
| "metadata": {}, |
| "source": [ |
| "<h2 id=\"3\">Windows</h2>\n", |
| "\n", |
| "On Windows, you can use [Micrsoft Visual Studio 2015](https://www.visualstudio.com/vs/older-downloads/) and [Microsoft Visual Studio 2017](https://www.visualstudio.com/downloads/) to compile MXNet with Intel ONEDNN.\n", |
| "[Micrsoft Visual Studio 2015](https://www.visualstudio.com/vs/older-downloads/) is recommended.\n", |
| "\n", |
| "**Visual Studio 2015**\n", |
| "\n", |
| "To build and install MXNet yourself, you need the following dependencies. Install the required dependencies:\n", |
| "\n", |
| "1. If [Microsoft Visual Studio 2015](https://www.visualstudio.com/vs/older-downloads/) is not already installed, download and install it. You can download and install the free community edition.\n", |
| "2. Download and Install [CMake 3](https://cmake.org/files/v3.14/cmake-3.14.0-win64-x64.msi) if it is not already installed.\n", |
| "3. Download [OpenCV 3](https://sourceforge.net/projects/opencvlibrary/files/3.4.5/opencv-3.4.5-vc14_vc15.exe/download), and unzip the OpenCV package, set the environment variable ```OpenCV_DIR``` to point to the ```OpenCV build directory``` (e.g.,```OpenCV_DIR = C:\\opencv\\build ```). Also, add the OpenCV bin directory (```C:\\opencv\\build\\x64\\vc14\\bin``` for example) to the ``PATH`` variable.\n", |
| "4. If you have Intel Math Kernel Library (Intel MKL) installed, set ```MKLROOT``` environment variable to point to ```MKL``` directory that contains the ```include``` and ```lib```. If you want to use MKL blas, you should set ```-DUSE_BLAS=mkl``` when cmake. Typically, you can find the directory in ```C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl```.\n", |
| "5. If you don't have the Intel Math Kernel Library (MKL) installed, download and install [OpenBLAS](http://sourceforge.net/projects/openblas/files/v0.2.14/), or build the latest version of OpenBLAS from source. Note that you should also download ```mingw64.dll.zip``` along with openBLAS and add them to PATH.\n", |
| "6. Set the environment variable ```OpenBLAS_HOME``` to point to the ```OpenBLAS``` directory that contains the ```include``` and ```lib``` directories. Typically, you can find the directory in ```C:\\Downloads\\OpenBLAS\\```.\n", |
| "\n", |
| "After you have installed all of the required dependencies, build the MXNet source code:\n", |
| "\n", |
| "1. Start a Visual Studio command prompt by click windows Start menu>>Visual Studio 2015>>VS2015 X64 Native Tools Command Prompt, and download the MXNet source code from [GitHub](https://github.com/apache/incubator-mxnet) by the command:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "569fe65b", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "git clone --recursive https://github.com/apache/incubator-mxnet.git\n", |
| "cd C:\\incubator-mxent\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "6a677729", |
| "metadata": {}, |
| "source": [ |
| "2. Enable Intel ONEDNN by -DUSE_ONEDNN=1. Use [CMake 3](https://cmake.org/) to create a Visual Studio solution in ```./build```. Make sure to specify the architecture in the\n", |
| "command:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "6014a3f2", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| ">mkdir build\n", |
| ">cd build\n", |
| ">cmake -G \"Visual Studio 14 Win64\" .. -DUSE_CUDA=0 -DUSE_CUDNN=0 -DUSE_NVRTC=0 -DUSE_OPENCV=1 -DUSE_OPENMP=1 -DUSE_PROFILER=1 -DUSE_BLAS=open -DUSE_LAPACK=1 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_NAME=All -DUSE_ONEDNN=1 -DCMAKE_BUILD_TYPE=Release\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "e300350c", |
| "metadata": {}, |
| "source": [ |
| "3. Enable Intel ONEDNN and Intel MKL as BLAS library by the command:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "57429f22", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| ">\"C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\bin\\mklvars.bat\" intel64\n", |
| ">cmake -G \"Visual Studio 14 Win64\" .. -DUSE_CUDA=0 -DUSE_CUDNN=0 -DUSE_NVRTC=0 -DUSE_OPENCV=1 -DUSE_OPENMP=1 -DUSE_PROFILER=1 -DUSE_BLAS=mkl -DUSE_LAPACK=1 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_NAME=All -DUSE_ONEDNN=1 -DCMAKE_BUILD_TYPE=Release\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "b3b2046c", |
| "metadata": {}, |
| "source": [ |
| "4. After the CMake successfully completed, in Visual Studio, open the solution file ```.sln``` and compile it, or compile the MXNet source code by using following command:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "21ef2b00", |
| "metadata": {}, |
| "source": [ |
| "```r\n", |
| "msbuild mxnet.sln /p:Configuration=Release;Platform=x64 /maxcpucount\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "e1afcd29", |
| "metadata": {}, |
| "source": [ |
| "These commands produce mxnet library called ```libmxnet.dll``` in the ```./build/Release/``` or ```./build/Debug``` folder. Also ```libmkldnn.dll``` with be in the ```./build/3rdparty/onednn/src/Release/```\n", |
| "\n", |
| "5. Make sure that all the dll files used above(such as `libmkldnn.dll`, `libmklml*.dll`, `libiomp5.dll`, `libopenblas*.dll`, etc) are added to the system PATH. For convinence, you can put all of them to ```\\windows\\system32```. Or you will come across `Not Found Dependencies` when loading MXNet.\n", |
| "\n", |
| "**Visual Studio 2017**\n", |
| "\n", |
| "User can follow the same steps of Visual Studio 2015 to build MXNET with ONEDNN, but change the version related command, for example,```C:\\opencv\\build\\x64\\vc15\\bin``` and build command is as below:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "189df09e", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| ">cmake -G \"Visual Studio 15 Win64\" .. -DUSE_CUDA=0 -DUSE_CUDNN=0 -DUSE_NVRTC=0 -DUSE_OPENCV=1 -DUSE_OPENMP=1 -DUSE_PROFILER=1 -DUSE_BLAS=mkl -DUSE_LAPACK=1 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_NAME=All -DUSE_ONEDNN=1 -DCMAKE_BUILD_TYPE=Release\n", |
| "\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "bf22ec2f", |
| "metadata": {}, |
| "source": [ |
| "<h2 id=\"4\">Verify MXNet with python</h2>\n", |
| "\n", |
| "Preinstall python and some dependent modules:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "cff05a28", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "pip install numpy graphviz\n", |
| "set PYTHONPATH=[workdir]\\incubator-mxnet\\python\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "c60cb19a", |
| "metadata": {}, |
| "source": [ |
| "or install mxnet" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "3c4fdc49", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "cd python\n", |
| "sudo python setup.py install\n", |
| "python -c \"import mxnet as mx;print((mx.nd.ones((2, 3))*2).asnumpy());\"\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "2e612700", |
| "metadata": {}, |
| "source": [ |
| "Expected Output:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "5caed3bb", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "[[ 2. 2. 2.]\n", |
| " [ 2. 2. 2.]]\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "250cb8b1", |
| "metadata": {}, |
| "source": [ |
| "### Verify whether ONEDNN works\n", |
| "\n", |
| "After MXNet is installed, you can verify if ONEDNN backend works well with a single Convolution layer." |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "c1e62946", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "import mxnet as mx\n", |
| "import numpy as np\n", |
| "\n", |
| "num_filter = 32\n", |
| "kernel = (3, 3)\n", |
| "pad = (1, 1)\n", |
| "shape = (32, 32, 256, 256)\n", |
| "\n", |
| "x = mx.sym.Variable('x')\n", |
| "w = mx.sym.Variable('w')\n", |
| "y = mx.sym.Convolution(data=x, weight=w, num_filter=num_filter, kernel=kernel, no_bias=True, pad=pad)\n", |
| "exe = y.simple_bind(mx.cpu(), x=shape)\n", |
| "\n", |
| "exe.arg_arrays[0][:] = np.random.normal(size=exe.arg_arrays[0].shape)\n", |
| "exe.arg_arrays[1][:] = np.random.normal(size=exe.arg_arrays[1].shape)\n", |
| "\n", |
| "exe.forward(is_train=False)\n", |
| "o = exe.outputs[0]\n", |
| "t = o.asnumpy()\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "929045ab", |
| "metadata": {}, |
| "source": [ |
| "More detailed debugging and profiling information can be logged by setting the environment variable 'MKLDNN_VERBOSE':" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "625695c3", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "export MKLDNN_VERBOSE=1\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "be2114f3", |
| "metadata": {}, |
| "source": [ |
| "For example, by running above code snippet, the following debugging logs providing more insights on ONEDNN primitives `convolution` and `reorder`. That includes: Memory layout, infer shape and the time cost of primitive execution." |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "c9cd76d9", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "dnnl_verbose,info,DNNL v1.1.2 (commit cb2cc7ac17ff4e2ef50805c7048d33256d82be4d)\n", |
| "dnnl_verbose,info,Detected ISA is Intel AVX-512 with Intel DL Boost\n", |
| "dnnl_verbose,exec,cpu,reorder,jit:uni,undef,src_f32::blocked:abcd:f0 dst_f32::blocked:aBcd16b:f0,,,32x32x256x256,7.43701\n", |
| "dnnl_verbose,exec,cpu,reorder,jit:uni,undef,src_f32::blocked:abcd:f0 dst_f32::blocked:ABcd16b16a:f0,,,32x32x3x3,0.202148\n", |
| "dnnl_verbose,exec,cpu,convolution,jit:avx512_common,forward_inference,src_f32::blocked:aBcd16b:f0 wei_f32::blocked:ABcd16b16a:f0 bia_undef::undef::f0 dst_f32::blocked:aBcd16b:f0,,alg:convolution_direct,mb32_ic32oc32_ih256oh256kh3sh1dh0ph1_iw256ow256kw3sw1dw0pw1,20.7539\n", |
| "dnnl_verbose,exec,cpu,reorder,jit:uni,undef,src_f32::blocked:abcd:f0 dst_f32::blocked:ABcd16b16a:f0,,,32x32x3x3,1.86694\n", |
| "dnnl_verbose,exec,cpu,reorder,jit:uni,undef,src_f32::blocked:aBcd16b:f0 dst_f32::blocked:abcd:f0,,,32x32x256x256,35.9771\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "f78f8e6a", |
| "metadata": {}, |
| "source": [ |
| "You can find step-by-step guidance to do profiling for ONEDNN primitives in [Profiling ONEDNN Operators](https://mxnet.apache.org/api/python/docs/tutorials/performance/backend/profiler.html#Profiling-MKLDNN-Operators).\n", |
| "\n", |
| "<h2 id=\"5\">Enable MKL BLAS</h2>\n", |
| "\n", |
| "With MKL BLAS, the performace is expected to furtherly improved with variable range depending on the computation load of the models.\n", |
| "You can redistribute not only dynamic libraries but also headers, examples and static libraries on accepting the license [Intel Simplified license](https://software.intel.com/en-us/license/intel-simplified-software-license).\n", |
| "Installing the full MKL installation enables MKL support for all operators under the linalg namespace.\n", |
| "\n", |
| " 1. Download and install the latest full MKL version following instructions on the [intel website.](https://software.intel.com/en-us/mkl) You can also install MKL through [YUM](https://software.intel.com/en-us/articles/installing-intel-free-libs-and-python-yum-repo) or [APT](https://software.intel.com/en-us/articles/installing-intel-free-libs-and-python-apt-repo) Repository.\n", |
| "\n", |
| " 2. Run `make -j ${nproc} USE_BLAS=mkl`\n", |
| "\n", |
| " 3. Navigate into the python directory\n", |
| "\n", |
| " 4. Run `sudo python setup.py install`\n", |
| "\n", |
| "### Verify whether MKL works\n", |
| "\n", |
| "After MXNet is installed, you can verify if MKL BLAS works well with a single dot layer." |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "4207a9b6", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "import mxnet as mx\n", |
| "import numpy as np\n", |
| "\n", |
| "shape_x = (1, 10, 8)\n", |
| "shape_w = (1, 12, 8)\n", |
| "\n", |
| "x_npy = np.random.normal(0, 1, shape_x)\n", |
| "w_npy = np.random.normal(0, 1, shape_w)\n", |
| "\n", |
| "x = mx.sym.Variable('x')\n", |
| "w = mx.sym.Variable('w')\n", |
| "y = mx.sym.batch_dot(x, w, transpose_b=True)\n", |
| "exe = y.simple_bind(mx.cpu(), x=x_npy.shape, w=w_npy.shape)\n", |
| "\n", |
| "exe.forward(is_train=False)\n", |
| "o = exe.outputs[0]\n", |
| "t = o.asnumpy()\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "125740c5", |
| "metadata": {}, |
| "source": [ |
| "You can open the `MKL_VERBOSE` flag by setting environment variable:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "9a94d66e", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "export MKL_VERBOSE=1\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "0208945a", |
| "metadata": {}, |
| "source": [ |
| "Then by running above code snippet, you probably will get the following output message which means `SGEMM` primitive from MKL are called. Layout information and primitive execution performance are also demonstrated in the log message." |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "6b8723dd", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "Numpy + Intel(R) MKL: THREADING LAYER: (null)\n", |
| "Numpy + Intel(R) MKL: setting Intel(R) MKL to use INTEL OpenMP runtime\n", |
| "Numpy + Intel(R) MKL: preloading libiomp5.so runtime\n", |
| "MKL_VERBOSE Intel(R) MKL 2019.0 Update 3 Product build 20190125 for Intel(R) 64 architecture Intel(R) Advanced Vector Extensions 512 (Intel(R) AVX-512) enabled processors, Lnx 2.40GHz lp64 intel_thread NMICDev:0\n", |
| "MKL_VERBOSE SGEMM(T,N,12,10,8,0x7f7f927b1378,0x1bc2140,8,0x1ba8040,8,0x7f7f927b1380,0x7f7f7400a280,12) 8.93ms CNR:OFF Dyn:1 FastMM:1 TID:0 NThr:40 WDiv:HOST:+0.000\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "f11ed985", |
| "metadata": {}, |
| "source": [ |
| "<h2 id=\"6\">Enable graph optimization</h2>\n", |
| "\n", |
| "Graph optimization with subgraph is available and enabled by default in master branch. For MXNet release v1.5, you can manually enable it by:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "b77afc98", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "export MXNET_SUBGRAPH_BACKEND=MKLDNN\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "88edc89e", |
| "metadata": {}, |
| "source": [ |
| "This limitations of this experimental feature are:\n", |
| "\n", |
| "- Use this feature only for inference. When training, be sure to turn the feature off by unsetting the `MXNET_SUBGRAPH_BACKEND` environment variable.\n", |
| "\n", |
| "- This feature will only run on the CPU, even if you're using a GPU-enabled build of MXNet.\n", |
| "\n", |
| "\n", |
| "<h2 id=\"7\">Quantization and Inference with INT8</h2>\n", |
| "\n", |
| "Benefiting from Intel ONEDNN, MXNet built with Intel ONEDNN brings outstanding performance improvement on quantization and inference with INT8 Intel CPU Platform on Intel Xeon Scalable Platform.\n", |
| "\n", |
| "- [CNN Quantization Examples](https://github.com/apache/incubator-mxnet/tree/master/example/quantization).\n", |
| "\n", |
| "- [Model Quantization for Production-Level Neural Network Inference](https://cwiki.apache.org/confluence/display/MXNET/MXNet+Graph+Optimization+and+Quantization+based+on+subgraph+and+MKL-DNN).\n", |
| "\n", |
| "<h2 id=\"8\">Next Steps and Support</h2>\n", |
| "\n", |
| "- For questions or support specific to MKL, visit the [Intel MKL](https://software.intel.com/en-us/mkl) website.\n", |
| "\n", |
| "- For questions or support specific to ONEDNN, visit the [Intel ONEDNN](https://github.com/oneapi-src/oneDNN) website.\n", |
| "\n", |
| "- If you find bugs, please open an issue on GitHub for [MXNet with MKL](https://github.com/apache/incubator-mxnet/labels/MKL) or [MXNet with ONEDNN](https://github.com/apache/incubator-mxnet/labels/MKLDNN)." |
| ] |
| } |
| ], |
| "metadata": { |
| "language_info": { |
| "name": "python" |
| } |
| }, |
| "nbformat": 4, |
| "nbformat_minor": 5 |
| } |