| { |
| "cells": [ |
| { |
| "cell_type": "markdown", |
| "id": "7ee0485e", |
| "metadata": {}, |
| "source": [ |
| "<!--- Licensed to the Apache Software Foundation (ASF) under one -->\n", |
| "<!--- or more contributor license agreements. See the NOTICE file -->\n", |
| "<!--- distributed with this work for additional information -->\n", |
| "<!--- regarding copyright ownership. The ASF licenses this file -->\n", |
| "<!--- to you under the Apache License, Version 2.0 (the -->\n", |
| "<!--- \"License\"); you may not use this file except in compliance -->\n", |
| "<!--- with the License. You may obtain a copy of the License at -->\n", |
| "\n", |
| "<!--- http://www.apache.org/licenses/LICENSE-2.0 -->\n", |
| "\n", |
| "<!--- Unless required by applicable law or agreed to in writing, -->\n", |
| "<!--- software distributed under the License is distributed on an -->\n", |
| "<!--- \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->\n", |
| "<!--- KIND, either express or implied. See the License for the -->\n", |
| "<!--- specific language governing permissions and limitations -->\n", |
| "<!--- under the License. -->\n", |
| "\n", |
| "# Install MXNet with MKL-DNN\n", |
| "\n", |
| "A better training and inference performance is expected to be achieved on Intel-Architecture CPUs with MXNet built with [Intel MKL-DNN](https://github.com/intel/mkl-dnn) on multiple operating system, including Linux, Windows and MacOS.\n", |
| "In the following sections, you will find build instructions for MXNet with Intel MKL-DNN on Linux, MacOS and Windows.\n", |
| "\n", |
| "Please find MKL-DNN optimized operators and other features in the [MKL-DNN operator list](https://github.com/apache/mxnet/blob/v1.5.x/docs/tutorials/mkldnn/operator_list.md).\n", |
| "\n", |
| "The detailed performance data collected on Intel Xeon CPU with MXNet built with Intel MKL-DNN can be found [here](https://mxnet.apache.org/api/faq/perf#intel-cpu).\n", |
| "\n", |
| "\n", |
| "<h2 id=\"0\">Contents</h2>\n", |
| "\n", |
| "* [1. Linux](#1)\n", |
| "* [2. MacOS](#2)\n", |
| "* [3. Windows](#3)\n", |
| "* [4. Verify MXNet with python](#4)\n", |
| "* [5. Enable MKL BLAS](#5)\n", |
| "* [6. Enable graph optimization](#6)\n", |
| "* [7. Quantization](#7)\n", |
| "* [8. Support](#8)\n", |
| "\n", |
| "<h2 id=\"1\">Linux</h2>\n", |
| "\n", |
| "### Prerequisites" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "29616140", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "sudo apt-get update\n", |
| "sudo apt-get install -y build-essential git\n", |
| "sudo apt-get install -y libopenblas-dev liblapack-dev\n", |
| "sudo apt-get install -y libopencv-dev\n", |
| "sudo apt-get install -y graphviz\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "94a6b8c8", |
| "metadata": {}, |
| "source": [ |
| "### Clone MXNet sources" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "26b00a95", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "git clone --recursive https://github.com/apache/mxnet.git\n", |
| "cd mxnet\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "583543f6", |
| "metadata": {}, |
| "source": [ |
| "### Build MXNet with MKL-DNN\n", |
| "\n", |
| "To achieve better performance, the Intel OpenMP and llvm OpenMP are recommended as below instruction. Otherwise, default GNU OpenMP will be used and you may get the sub-optimal performance. If you don't have the full [MKL](https://software.intel.com/en-us/intel-mkl) library installation, you might use OpenBLAS as the blas library, by setting USE_BLAS=openblas." |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "8610d982", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "# build with llvm OpenMP and Intel MKL/openblas\n", |
| "mkdir build && cd build\n", |
| "cmake -DUSE_CUDA=OFF -DUSE_MKL_IF_AVAILABLE=ON -DUSE_MKLDNN=ON -DUSE_OPENMP=ON -DUSE_OPENCV=ON ..\n", |
| "make -j $(nproc)\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "7fd3737d", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "# build with Intel MKL and Intel OpenMP\n", |
| "make -j $(nproc) USE_OPENCV=1 USE_MKLDNN=1 USE_BLAS=mkl USE_INTEL_PATH=/opt/intel\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "d421c2f8", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "# build with openblas and GNU OpenMP(sub-optimal performance)\n", |
| "make -j $(nproc) USE_OPENCV=1 USE_MKLDNN=1 USE_BLAS=openblas\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "06b01719", |
| "metadata": {}, |
| "source": [ |
| "<h2 id=\"2\">MacOS</h2>\n", |
| "\n", |
| "### Prerequisites\n", |
| "\n", |
| "Install the dependencies, required for MXNet, with the following commands:\n", |
| "\n", |
| "- [Homebrew](https://brew.sh/)\n", |
| "- llvm (clang in macOS does not support OpenMP)\n", |
| "- OpenCV (for computer vision operations)" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "8c4fa4e4", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "# Paste this command in Mac terminal to install Homebrew\n", |
| "/usr/bin/ruby -e \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)\"\n", |
| "\n", |
| "# install dependency\n", |
| "brew update\n", |
| "brew install pkg-config\n", |
| "brew install graphviz\n", |
| "brew tap homebrew/core\n", |
| "brew install opencv\n", |
| "brew tap homebrew/versions\n", |
| "brew install llvm\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "d27cbccd", |
| "metadata": {}, |
| "source": [ |
| "### Clone MXNet sources" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "f84505c1", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "git clone --recursive https://github.com/apache/mxnet.git\n", |
| "cd mxnet\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "710d5e2a", |
| "metadata": {}, |
| "source": [ |
| "### Build MXNet with MKL-DNN" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "4246655e", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "LIBRARY_PATH=$(brew --prefix llvm)/lib/ make -j $(sysctl -n hw.ncpu) CC=$(brew --prefix llvm)/bin/clang CXX=$(brew --prefix llvm)/bin/clang++ USE_OPENCV=1 USE_OPENMP=1 USE_MKLDNN=1 USE_BLAS=apple\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "41e2ac9f", |
| "metadata": {}, |
| "source": [ |
| "<h2 id=\"3\">Windows</h2>\n", |
| "\n", |
| "On Windows, you can use [Micrsoft Visual Studio 2015](https://www.visualstudio.com/vs/older-downloads/) and [Microsoft Visual Studio 2017](https://www.visualstudio.com/downloads/) to compile MXNet with Intel MKL-DNN.\n", |
| "[Micrsoft Visual Studio 2015](https://www.visualstudio.com/vs/older-downloads/) is recommended.\n", |
| "\n", |
| "**Visual Studio 2015**\n", |
| "\n", |
| "To build and install MXNet yourself, you need the following dependencies. Install the required dependencies:\n", |
| "\n", |
| "1. If [Microsoft Visual Studio 2015](https://www.visualstudio.com/vs/older-downloads/) is not already installed, download and install it. You can download and install the free community edition.\n", |
| "2. Download and Install [CMake 3](https://cmake.org/files/v3.14/cmake-3.14.0-win64-x64.msi) if it is not already installed.\n", |
| "3. Download [OpenCV 3](https://sourceforge.net/projects/opencvlibrary/files/3.4.5/opencv-3.4.5-vc14_vc15.exe/download), and unzip the OpenCV package, set the environment variable ```OpenCV_DIR``` to point to the ```OpenCV build directory``` (e.g.,```OpenCV_DIR = C:\\opencv\\build ```). Also, add the OpenCV bin directory (```C:\\opencv\\build\\x64\\vc14\\bin``` for example) to the ``PATH`` variable.\n", |
| "4. If you have Intel Math Kernel Library (Intel MKL) installed, set ```MKL_ROOT``` to point to ```MKL``` directory that contains the ```include``` and ```lib```. If you want to use MKL blas, you should set ```-DUSE_BLAS=mkl``` when cmake. Typically, you can find the directory in ```C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl```.\n", |
| "5. If you don't have the Intel Math Kernel Library (MKL) installed, download and install [OpenBLAS](http://sourceforge.net/projects/openblas/files/v0.2.14/), or build the latest version of OpenBLAS from source. Note that you should also download ```mingw64.dll.zip``` along with openBLAS and add them to PATH.\n", |
| "6. Set the environment variable ```OpenBLAS_HOME``` to point to the ```OpenBLAS``` directory that contains the ```include``` and ```lib``` directories. Typically, you can find the directory in ```C:\\Downloads\\OpenBLAS\\```.\n", |
| "\n", |
| "After you have installed all of the required dependencies, build the MXNet source code:\n", |
| "\n", |
| "1. Start a Visual Studio command prompt by click windows Start menu>>Visual Studio 2015>>VS2015 X64 Native Tools Command Prompt, and download the MXNet source code from [GitHub](https://github.com/apache/mxnet) by the command:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "e61d7c63", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "git clone --recursive https://github.com/apache/mxnet.git\n", |
| "cd C:\\incubator-mxent\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "61df71df", |
| "metadata": {}, |
| "source": [ |
| "2. Enable Intel MKL-DNN by -DUSE_MKLDNN=1. Use [CMake 3](https://cmake.org/) to create a Visual Studio solution in ```./build```. Make sure to specify the architecture in the\n", |
| "command:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "99e674b5", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| ">mkdir build\n", |
| ">cd build\n", |
| ">cmake -G \"Visual Studio 14 Win64\" .. -DUSE_CUDA=0 -DUSE_CUDNN=0 -DUSE_NVRTC=0 -DUSE_OPENCV=1 -DUSE_OPENMP=1 -DUSE_PROFILER=1 -DUSE_BLAS=open -DUSE_LAPACK=1 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_NAME=All -DUSE_MKLDNN=1 -DCMAKE_BUILD_TYPE=Release\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "83274c68", |
| "metadata": {}, |
| "source": [ |
| "3. Enable Intel MKL-DNN and Intel MKL as BLAS library by the command:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "c000fe07", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| ">\"C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\bin\\mklvars.bat\" intel64\n", |
| ">cmake -G \"Visual Studio 14 Win64\" .. -DUSE_CUDA=0 -DUSE_CUDNN=0 -DUSE_NVRTC=0 -DUSE_OPENCV=1 -DUSE_OPENMP=1 -DUSE_PROFILER=1 -DUSE_BLAS=mkl -DUSE_LAPACK=1 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_NAME=All -DUSE_MKLDNN=1 -DCMAKE_BUILD_TYPE=Release -DMKL_ROOT=\"C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\"\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "bfdf7fae", |
| "metadata": {}, |
| "source": [ |
| "4. After the CMake successfully completed, in Visual Studio, open the solution file ```.sln``` and compile it, or compile the MXNet source code by using following command:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "b991d5b9", |
| "metadata": {}, |
| "source": [ |
| "```r\n", |
| "msbuild mxnet.sln /p:Configuration=Release;Platform=x64 /maxcpucount\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "3505ddbe", |
| "metadata": {}, |
| "source": [ |
| "These commands produce mxnet library called ```libmxnet.dll``` in the ```./build/Release/``` or ```./build/Debug``` folder. Also ```libmkldnn.dll``` with be in the ```./build/3rdparty/mkldnn/src/Release/```\n", |
| "\n", |
| "5. Make sure that all the dll files used above(such as `libmkldnn.dll`, `libmklml*.dll`, `libiomp5.dll`, `libopenblas*.dll`, etc) are added to the system PATH. For convinence, you can put all of them to ```\\windows\\system32```. Or you will come across `Not Found Dependencies` when loading MXNet.\n", |
| "\n", |
| "**Visual Studio 2017**\n", |
| "\n", |
| "User can follow the same steps of Visual Studio 2015 to build MXNET with MKL-DNN, but change the version related command, for example,```C:\\opencv\\build\\x64\\vc15\\bin``` and build command is as below:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "e6b79c2d", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| ">cmake -G \"Visual Studio 15 Win64\" .. -DUSE_CUDA=0 -DUSE_CUDNN=0 -DUSE_NVRTC=0 -DUSE_OPENCV=1 -DUSE_OPENMP=1 -DUSE_PROFILER=1 -DUSE_BLAS=mkl -DUSE_LAPACK=1 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_NAME=All -DUSE_MKLDNN=1 -DCMAKE_BUILD_TYPE=Release -DMKL_ROOT=\"C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\"\n", |
| "\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "5765890e", |
| "metadata": {}, |
| "source": [ |
| "<h2 id=\"4\">Verify MXNet with python</h2>\n", |
| "\n", |
| "Preinstall python and some dependent modules:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "69169c5c", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "pip install numpy graphviz\n", |
| "set PYTHONPATH=[workdir]\\mxnet\\python\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "848f4972", |
| "metadata": {}, |
| "source": [ |
| "or install mxnet" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "86b8ff08", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "cd python\n", |
| "sudo python setup.py install\n", |
| "python -c \"import mxnet as mx;print((mx.nd.ones((2, 3))*2).asnumpy());\"\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "e61cd81d", |
| "metadata": {}, |
| "source": [ |
| "Expected Output:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "3519d6aa", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "[[ 2. 2. 2.]\n", |
| " [ 2. 2. 2.]]\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "c18a5543", |
| "metadata": {}, |
| "source": [ |
| "### Verify whether MKL-DNN works\n", |
| "\n", |
| "After MXNet is installed, you can verify if MKL-DNN backend works well with a single Convolution layer." |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "3ea0347c", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "import mxnet as mx\n", |
| "import numpy as np\n", |
| "\n", |
| "num_filter = 32\n", |
| "kernel = (3, 3)\n", |
| "pad = (1, 1)\n", |
| "shape = (32, 32, 256, 256)\n", |
| "\n", |
| "x = mx.sym.Variable('x')\n", |
| "w = mx.sym.Variable('w')\n", |
| "y = mx.sym.Convolution(data=x, weight=w, num_filter=num_filter, kernel=kernel, no_bias=True, pad=pad)\n", |
| "exe = y.simple_bind(mx.cpu(), x=shape)\n", |
| "\n", |
| "exe.arg_arrays[0][:] = np.random.normal(size=exe.arg_arrays[0].shape)\n", |
| "exe.arg_arrays[1][:] = np.random.normal(size=exe.arg_arrays[1].shape)\n", |
| "\n", |
| "exe.forward(is_train=False)\n", |
| "o = exe.outputs[0]\n", |
| "t = o.asnumpy()\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "e97e0c3a", |
| "metadata": {}, |
| "source": [ |
| "More detailed debugging and profiling information can be logged by setting the environment variable 'MKLDNN_VERBOSE':" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "dd196742", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "export MKLDNN_VERBOSE=1\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "7af496d5", |
| "metadata": {}, |
| "source": [ |
| "For example, by running above code snippet, the following debugging logs providing more insights on MKL-DNN primitives `convolution` and `reorder`. That includes: Memory layout, infer shape and the time cost of primitive execution." |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "8aabb396", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "dnnl_verbose,info,DNNL v1.1.2 (commit cb2cc7ac17ff4e2ef50805c7048d33256d82be4d)\n", |
| "dnnl_verbose,info,Detected ISA is Intel AVX-512 with Intel DL Boost\n", |
| "dnnl_verbose,exec,cpu,reorder,jit:uni,undef,src_f32::blocked:abcd:f0 dst_f32::blocked:aBcd16b:f0,,,32x32x256x256,7.43701\n", |
| "dnnl_verbose,exec,cpu,reorder,jit:uni,undef,src_f32::blocked:abcd:f0 dst_f32::blocked:ABcd16b16a:f0,,,32x32x3x3,0.202148\n", |
| "dnnl_verbose,exec,cpu,convolution,jit:avx512_common,forward_inference,src_f32::blocked:aBcd16b:f0 wei_f32::blocked:ABcd16b16a:f0 bia_undef::undef::f0 dst_f32::blocked:aBcd16b:f0,,alg:convolution_direct,mb32_ic32oc32_ih256oh256kh3sh1dh0ph1_iw256ow256kw3sw1dw0pw1,20.7539\n", |
| "dnnl_verbose,exec,cpu,reorder,jit:uni,undef,src_f32::blocked:abcd:f0 dst_f32::blocked:ABcd16b16a:f0,,,32x32x3x3,1.86694\n", |
| "dnnl_verbose,exec,cpu,reorder,jit:uni,undef,src_f32::blocked:aBcd16b:f0 dst_f32::blocked:abcd:f0,,,32x32x256x256,35.9771\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "4339d82a", |
| "metadata": {}, |
| "source": [ |
| "You can find step-by-step guidance to do profiling for MKLDNN primitives in [Profiling MKLDNN Operators](https://mxnet.apache.org/api/python/docs/tutorials/performance/backend/profiler.html#Profiling-MKLDNN-Operators).\n", |
| "\n", |
| "<h2 id=\"5\">Enable MKL BLAS</h2>\n", |
| "\n", |
| "With MKL BLAS, the performace is expected to furtherly improved with variable range depending on the computation load of the models.\n", |
| "You can redistribute not only dynamic libraries but also headers, examples and static libraries on accepting the license [Intel Simplified license](https://software.intel.com/en-us/license/intel-simplified-software-license).\n", |
| "Installing the full MKL installation enables MKL support for all operators under the linalg namespace.\n", |
| "\n", |
| " 1. Download and install the latest full MKL version following instructions on the [intel website.](https://software.intel.com/en-us/mkl) You can also install MKL through [YUM](https://software.intel.com/en-us/articles/installing-intel-free-libs-and-python-yum-repo) or [APT](https://software.intel.com/en-us/articles/installing-intel-free-libs-and-python-apt-repo) Repository.\n", |
| "\n", |
| " 2. Run `make -j ${nproc} USE_BLAS=mkl`\n", |
| "\n", |
| " 3. Navigate into the python directory\n", |
| "\n", |
| " 4. Run `sudo python setup.py install`\n", |
| "\n", |
| "### Verify whether MKL works\n", |
| "\n", |
| "After MXNet is installed, you can verify if MKL BLAS works well with a single dot layer." |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "1215e9a3", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "import mxnet as mx\n", |
| "import numpy as np\n", |
| "\n", |
| "shape_x = (1, 10, 8)\n", |
| "shape_w = (1, 12, 8)\n", |
| "\n", |
| "x_npy = np.random.normal(0, 1, shape_x)\n", |
| "w_npy = np.random.normal(0, 1, shape_w)\n", |
| "\n", |
| "x = mx.sym.Variable('x')\n", |
| "w = mx.sym.Variable('w')\n", |
| "y = mx.sym.batch_dot(x, w, transpose_b=True)\n", |
| "exe = y.simple_bind(mx.cpu(), x=x_npy.shape, w=w_npy.shape)\n", |
| "\n", |
| "exe.forward(is_train=False)\n", |
| "o = exe.outputs[0]\n", |
| "t = o.asnumpy()\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "b09d0d2c", |
| "metadata": {}, |
| "source": [ |
| "You can open the `MKL_VERBOSE` flag by setting environment variable:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "9f036983", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "export MKL_VERBOSE=1\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "bf0fb5f2", |
| "metadata": {}, |
| "source": [ |
| "Then by running above code snippet, you probably will get the following output message which means `SGEMM` primitive from MKL are called. Layout information and primitive execution performance are also demonstrated in the log message." |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "73e977a3", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "Numpy + Intel(R) MKL: THREADING LAYER: (null)\n", |
| "Numpy + Intel(R) MKL: setting Intel(R) MKL to use INTEL OpenMP runtime\n", |
| "Numpy + Intel(R) MKL: preloading libiomp5.so runtime\n", |
| "MKL_VERBOSE Intel(R) MKL 2019.0 Update 3 Product build 20190125 for Intel(R) 64 architecture Intel(R) Advanced Vector Extensions 512 (Intel(R) AVX-512) enabled processors, Lnx 2.40GHz lp64 intel_thread NMICDev:0\n", |
| "MKL_VERBOSE SGEMM(T,N,12,10,8,0x7f7f927b1378,0x1bc2140,8,0x1ba8040,8,0x7f7f927b1380,0x7f7f7400a280,12) 8.93ms CNR:OFF Dyn:1 FastMM:1 TID:0 NThr:40 WDiv:HOST:+0.000\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "d12ac0d1", |
| "metadata": {}, |
| "source": [ |
| "<h2 id=\"6\">Enable graph optimization</h2>\n", |
| "\n", |
| "Graph optimization with subgraph is available and enabled by default in master branch. For MXNet release v1.5, you can manually enable it by:" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "ee7c6e84", |
| "metadata": {}, |
| "source": [ |
| "```\n", |
| "export MXNET_SUBGRAPH_BACKEND=MKLDNN\n", |
| "```\n" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "id": "c084a0f0", |
| "metadata": {}, |
| "source": [ |
| "This limitations of this experimental feature are:\n", |
| "\n", |
| "- Use this feature only for inference. When training, be sure to turn the feature off by unsetting the `MXNET_SUBGRAPH_BACKEND` environment variable.\n", |
| "\n", |
| "- This feature will only run on the CPU, even if you're using a GPU-enabled build of MXNet.\n", |
| "\n", |
| "\n", |
| "<h2 id=\"7\">Quantization and Inference with INT8</h2>\n", |
| "\n", |
| "Benefiting from Intel MKL-DNN, MXNet built with Intel MKL-DNN brings outstanding performance improvement on quantization and inference with INT8 Intel CPU Platform on Intel Xeon Scalable Platform.\n", |
| "\n", |
| "- [CNN Quantization Examples](https://github.com/apache/mxnet/tree/master/example/quantization).\n", |
| "\n", |
| "- [Model Quantization for Production-Level Neural Network Inference](https://cwiki.apache.org/confluence/display/MXNET/MXNet+Graph+Optimization+and+Quantization+based+on+subgraph+and+MKL-DNN).\n", |
| "\n", |
| "<h2 id=\"8\">Next Steps and Support</h2>\n", |
| "\n", |
| "- For questions or support specific to MKL, visit the [Intel MKL](https://software.intel.com/en-us/mkl) website.\n", |
| "\n", |
| "- For questions or support specific to MKL, visit the [Intel MKLDNN](https://github.com/intel/mkl-dnn) website.\n", |
| "\n", |
| "- If you find bugs, please open an issue on GitHub for [MXNet with MKL](https://github.com/apache/mxnet/labels/MKL) or [MXNet with MKLDNN](https://github.com/apache/mxnet/labels/MKLDNN)." |
| ] |
| } |
| ], |
| "metadata": { |
| "language_info": { |
| "name": "python" |
| } |
| }, |
| "nbformat": 4, |
| "nbformat_minor": 5 |
| } |