blob: ef6fa671e1c9b11efeee896cf489e421a71b5c47 [file] [view]
---
title: Getting-Started
---
# Getting Started with Auron for Apache Spark
## Build from source
To build Auron from source, follow the steps below:
1. Install Rust
Auron's native execution lib is written in Rust. You need to install Rust (nightly) before compiling.
We recommend using [rustup](https://rustup.rs/) for installation.
2. Install JDK
Auron has been well tested with JDK 8, 11, and 17.
Make sure `JAVA_HOME` is properly set and points to your desired version.
3. Check out the source code.
4. Build the project.
You can build Auron either *locally* or *inside Docker with CentOS7* using a unified script: `auron-build.sh`.
Run `./auron-build.sh --help` to see all available options.
After the build completes, a fat JAR with all dependencies will be generated in either the `target/` directory (for local builds)
or `target-docker/` directory (for Docker builds), depending on the selected build mode.
## Run Spark Job with Auron Accelerator
This section describes how to submit and configure a Spark Job with Auron support.
1. Move the Auron JAR to the Spark client classpath (normally spark-xx.xx.xx/jars/).
2. Add the following configs to spark configuration in `spark-xx.xx.xx/conf/spark-default.conf`:
```properties
spark.auron.enable true
spark.sql.extensions org.apache.spark.sql.auron.AuronSparkSessionExtension
spark.shuffle.manager org.apache.spark.sql.execution.auron.shuffle.AuronShuffleManager
spark.memory.offHeap.enabled false
# suggested executor memory configuration
spark.executor.memory 4g
spark.executor.memoryOverhead 4096
```
3. submit a query with spark-sql, or other tools like spark-thriftserver:
```shell
spark-sql -f tpcds/q01.sql
```