These instructions will help you build Apache SystemDS from source code, which is the basis for the engine and algorithms development. The following conventions will be used to refer to directories on your machine:
<USER_HOME>
is your home directory.<JDK_18_HOME>
is the root directory for the 1.8 JDK.<MAVEN_HOME>
is the root directory for the Apache Maven source code.<SYSTEMDS_HOME>
is the root directory for the SystemDS source code.<SPARK_HOME>
is the root directory for the Apache Spark source code.<HADOOP_HOME>
is the root directory for<CUDA_HOME>
,<CUDA_PATH>
is the top directory for NVIDIA GPU Computing Toolkit.9.0
, it would like C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0
libnvvp
, bin
folders are accessible to command line.<CUDNN_PATH>
is the path Ex. Corresponding to the CUDA version, C:\Program Files\cudnn-9.0-windows10-x64-v7.2.1.38\cuda
. ├── MAVEN_HOME = './maven' | ├── bin | ├── mvn.exe # and other executables ├── SPARK_HOME = './spark' | ├── bin | ├── spark-shell # and other invocation commands ├── HADOOP_HOME = './hadoop' │ ├── bin | ├── wintuils.exe # The Default content layout and html file. ├── SYSTEMDS_HOME
SystemDS source code is available from github.com/apache/systemds by either cloning or downloading a zip file (based on a branch) into <SYSTEMDS_HOME>
. The default is the master branch.
git clone https://github.com/apache/systemds systemds
The master branch contains the source code which will be used to create the next major version of Apache SystemDS.
Speed Tip: If the complete repository history isn't needed then using a shallow clone (git clone --depth 1
) will save significant time.
IntelliJ IDEA
or Eclipse
is preferred for best developer experience.
Using IntelliJ IDEA File | Open, select the <SYSTEMDS_HOME>
directory.
JDK version 1.8 (u151 or newer) is required for building and developing for SystemDS developement.
<JDK_18_HOME>
.<JDK_18_HOME>/lib/tools.jar
to the Classpath tab for the 1.8 JDK.<USER_HOME>/.m2/repository
directory.-Xmx2G
. These changes will greatly reduce the compile time.To build SystemDS from source, choose Build | Build Project from the main menu. OR To maven build, run the mvn clean package
command in <SYSTEMDS_HOME>
directory. See the pom.xml
file for details.
To run the SystemDS built from source, choose Run | Run from the main menu.
To run tests on the build, apply these setting to the Run | Edit Configurations... | Defaults | JUnit configuration tab:
<SYSTEMDS_HOME>
-ea