This guide covers the Native BLAS optimizations and software setup for using SystemDS native mode.
SystemDS implements all the matrix operations in Java. This simplifies deployment especially in a distributed environment.
In Some cases (such as Deep Neural Networks), to take advantage of native BLAS instead of SystemDS internal Java library for performing single node operations such as matrix multiplication, convolution etc.
By default, SystemDS will first attempt to use Intel MKL (if installed), and then OpenBLAS (if installed). If none of the libraries are available, SystemDS falls back to its internal java library.
Note: Current SystemDS version supported on Linux, Windows x86-64 platform.
To know Intel MKL system requirements, see Intel® oneAPI Math Kernel Library System Requirements
Either of the following software is required to be installed in your system:
Download Intel MKL. Supported MKL version is 2017 to 2019.5.
Install OpenBLAS. Installation instructions on GitHub.
Note: In Ubuntu 20.04, remove libopenblas0-pthread package and install libopenblas0-openmp instead. So that OpenBLAS will be installed with OpenMP support.
Scripts to set environmental variables
For example, in Linux to set Intel MKL libraries in the Path:
# path adjustments to that script according to your installation source /opt/intel/bin/compilervars.sh intel64
Via commandline:
Java:
-Djava.library.path=/path/to/blas-n-other-dependencies
Note: This property can also be set with sysds.native.blas.directory.
Set sysds.native.blas property to mkl, openblas as shown.
<!-- enables native blas for matrix multiplication and convolution, experimental feature (options: auto, mkl, openblas, none) --> <sysds.native.blas>mkl</sysds.native.blas>
If there are issues loading libs because the gcc ABI changed from gcc 4.x to 5 and above. In this case,
PATH, LD_LIBRARY_PATH)