blob: 720f06d987d729a36c4c5991d0315b77092ad6ad [file] [view]
# Set Up Develop Environment
## Start Here
If you are new to contributing, read [Contribution Path](./contribution-path.md) first. It will help you decide whether you should begin with docs, connector work, or engine debugging before setting up the full local environment.
In this section, we are going to show you how to set up your development environment for SeaTunnel, and then run a simple
example in your JetBrains IntelliJ IDEA.
> You can develop or test SeaTunnel code in any development environment that you like, but here we use
> [JetBrains IDEA](https://www.jetbrains.com/idea/) as an example to teach you to step by step.
## Prepare
Before we start talking about how to set up the environment, we need to do some preparation work. Make sure you already
have installed the following software:
* [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) installed.
* [Java](https://www.java.com/en/download/) ( JDK8/JDK11 are supported by now) installed and `JAVA_HOME` set.
* [Scala](https://www.scala-lang.org/download/2.11.12.html) (only scala 2.11.12 supported by now) installed.
* [JetBrains IDEA](https://www.jetbrains.com/idea/) installed.
## Set Up
### Clone the Source Code
First of all, you need to clone the SeaTunnel source code from [GitHub](https://github.com/apache/seatunnel).
```shell
git clone git@github.com:apache/seatunnel.git
```
### Install Subproject Locally
After cloning the source code, you should run the `./mvnw` command to install the subproject to the maven local repository.
Otherwise, your code could not start in JetBrains IntelliJ IDEA correctly.
```shell
./mvnw clean install -DskipTests
```
### Building SeaTunnel From Source
After you install the maven, you can use the following command to compile and package.
```
mvn clean package -pl seatunnel-dist -am -Dmaven.test.skip=true
```
### Building Sub Module
If you want to build submodules separately, you can use the following command to compile and package.
```ssh
# This is an example of building the redis connector separately
mvn clean package -pl seatunnel-connectors-v2/connector-redis -am -DskipTests -T 1C
```
### Install JetBrains IDEA Scala Plugin
Now, you can open your JetBrains IntelliJ IDEA and explore the source code. But before building Scala code in IDEA,
you should also install JetBrains IntelliJ IDEA's [Scala Plugin](https://plugins.jetbrains.com/plugin/1347-scala).
See [Install Plugins For IDEA](https://www.jetbrains.com/help/idea/managing-plugins.html#install-plugins) if you want to.
### Install JetBrains IDEA Lombok Plugin
Before running the following example, you should also install JetBrains IntelliJ IDEA's [Lombok plugin](https://plugins.jetbrains.com/plugin/6317-lombok).
See [install plugins for IDEA](https://www.jetbrains.com/help/idea/managing-plugins.html#install-plugins) if you want to.
### Code Style
Apache SeaTunnel uses `Spotless` for code style and format checks. You can run the following command and `Spotless` will automatically fix the code style and formatting errors for you:
```shell
./mvnw spotless:apply
```
You could copy the `pre-commit hook` file `/tools/spotless_check/pre-commit.sh` to your `.git/hooks/` directory so that every time you commit your code with `git commit`, `Spotless` will automatically fix things for you.
## Run Simple Example
After all the above things are done, you just finish the environment setup and can run an example we provide to you out
of box. All examples are in module `seatunnel-examples`, you could pick one you are interested in, [Running Or Debugging
It In IDEA](https://www.jetbrains.com/help/idea/run-debug-configuration.html) as you wish.
Here we use `seatunnel-examples/seatunnel-engine-examples/src/main/java/org/apache/seatunnel/example/engine/SeaTunnelEngineLocalExample.java`
as an example, when you run it successfully you can see the output as below:
```log
2024-08-10 11:45:32,839 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand -
***********************************************
Job Statistic Information
***********************************************
Start Time : 2024-08-10 11:45:30
End Time : 2024-08-10 11:45:32
Total Time(s) : 2
Total Read Count : 5
Total Write Count : 5
Total Failed Count : 0
***********************************************
```
## What's More
All our examples use simple source and sink to make it less dependent and easy to run. You can change the example configuration
in `resources/examples`. You can change your configuration as below, if you want to use PostgreSQL as the source and
sink to console.
Please note that when using connectors other than FakeSource and Console, you need to modify the dependencies in the `pom.xml` file of the corresponding submodule of seatunnel-example.
```conf
env {
parallelism = 1
job.mode = "BATCH"
}
source {
Jdbc {
driver = org.postgresql.Driver
url = "jdbc:postgresql://host:port/database"
username = postgres
password = "123456"
query = "select * from test"
table_path = "database.test"
}
}
sink {
Console {}
}
```