SeaTunnel was formerly named Waterdrop , and renamed SeaTunnel since October 12, 2021.
So, What we are?
An open-source web console to manage your seatunnel-script, and would push them to any scheduling-system easily. Click it if your want to know more about our design. 👉🏻Design
| SeaTunnel Web Version | SeaTunnel Version | Doc |
|---|---|---|
| 1.0.3-SNAPSHOT | 2.3.8 / 2.3.9-SNAPSHOT | Docs |
| 1.0.2 | 2.3.8 | Docs |
| 1.0.1 | 2.3.3 | Docs |
| 1.0.0 | 2.3.3 | Docs |
Notice: Some details please refer to the docs/QuickStart.md
Because SeaTunnel Web uses the SeaTunnel Java Client to submit jobs, running SeaTunnel Web requires preparing a SeaTunnel Zeta Engine service first.
Based on the usage requirements of SeaTunnel Zeta Engine, the SeaTunnel Client node that submits the job must have the same operating system and installation directory structure as the SeaTunnel Server node that runs the job. Therefore, if you want to run SeaTunnel Web in IDEA, you must install and run a SeaTunnel Zeta Engine Server on the same machine as the IDEA.
Don't worry, the next steps will tell you how to correctly install SeaTunnel Zeta Engine Server in different situations.
If you want to deploy and run SeaTunnel Web, Please turn to [3 Run SeaTunnel Web In Server](#3 Run SeaTunnel Web In Server)
You have two ways to get the SeaTunnel installer package. Build from source code or download from the SeaTunnel website.
The SeaTunnel version used here is only for writing this document to show you the process used, and does not necessarily represent the correct version. SeaTunnel Web and SeaTunnel Engine have strict version dependencies, and you can confirm the specific version mapping through xxx. Now only support build SeaTunnel Web and Seatunnel Zeta Engine in local, Because it is necessary to ensure that the seatunnel-api in SeaTunnel Web and the version in SeaTunnel Zeta Engine are the same.
ST_WEB_BASEDIR_PATH to represent the location of the data source shade package. A custom class loader will be used to load the data source shade package based on this. For example: ST_WEB_BASEDIR_PATH=/seatunnel-web-dist/target/apache-seatunnel-web-1.0.3-SNAPSHOT/${Your_code_dir}/seatunnel-dist/target, For example:apache-seatunnel-2.3.8-bin.tar.gztar -zxvf apache-seatunnel-2.3.8-bin.tar.gz to unzip the installer package.cd apache-seatunnel-2.3.8 & sh bin/seatunnel-cluster.sh -d to run the SeaTunnel Zeta Engine Server.The other way to install SeaTunnel Zeta Engine Server is download the installer package from https://seatunnel.apache.org/download and deploy.
cd apache-seatunnel-2.3.8 & sh bin/seatunnel-cluster.sh -d to run the SeaTunnel Zeta Engine Server.Edit seatunnel-server/seatunnel-app/src/main/resources/script/seatunnel_server_env.sh file, Complete the installed database address, port, username, and password. Here is an example:
export HOSTNAME="localhost" export PORT="3306" export USERNAME="root" export PASSWORD="123456"
Run init shell sh seatunnel-server/seatunnel-app/src/main/resources/script/init_sql.sh If there are no errors during operation, it indicates successful initialization.
sh build.sh code
seatunnel-server/seatunnel-app/src/main/resources/application.yml Fill in the database connection information 
seatunnel-server/seatunnel-app/src/main/resources/application.yml, add jwt.secretKey value. Eg: https://github.com/apache/seatunnel(Notice that cannot be too short).apache-seatunnel-2.3.8/connectors/plugin-mapping.properties file to seatunnel-web/seatunnel-server/seatunnel-app/src/main/resources dir.seatunnel-server/seatunnel-app/src/main/java/org/apache/seatunnel/app/SeatunnelApplication.java If there are no errors reported, the seatunnel web backend service is successfully started. Notice that, you must set -DSEATUNNEL_HOME=${your_seatunnel_install_path} like this:
Because the data source plugin is dynamically loaded, it is necessary to set relevant environment variables:
cd seatunnel-ui npm install npm run dev
If there are no issues with the operation, the following information will be displayed:
➜ Local: http://127.0.0.1:5173/ ➜ Network: use --host to expose ➜ press h to show help
Accessing in a browser http://127.0.0.1:5173/login Okay, the default username and password are admin/admin.
To run SeaTunnel Web on the server, you need to first have a SeaTunnel Zeta Engine Server environment. If you do not already have one, you can refer to the following steps for deployment.
You have two ways to get the SeaTunnel installer package. Build from source code or download from the SeaTunnel website.
The SeaTunnel version used here is only for writing this document to show you the process used, and does not necessarily represent the correct version. SeaTunnel Web and SeaTunnel Engine have strict version dependencies, and you can confirm the specific version mapping through xxx
./mvnw -U -T 1C clean install -DskipTests -D"maven.test.skip"=true -D"maven.javadoc.skip"=true -D"checkstyle.skip"=true -D"license.skipAddThirdParty" ${Your_code_dir}/seatunnel-dist/target, For example:apache-seatunnel-2.3.8-bin.tar.gzThe other way to get SeaTunnel Zeta Engine Server installer package is download the installer package from https://seatunnel.apache.org/download and install plugins online.
tar -zcvf apache-seatunnel-2.3.8-bin.tar.gz apache-seatunnel-2.3.8After 3.1.1 or 3.1.2 you can get an installer package apache-seatunnel-2.3.8-bin.tar.gz, Then you can copy it to you server node and deploy reference https://seatunnel.apache.org/docs/seatunnel-engine/deployment.
If you use SeaTunnel Web, you need deploy a SeaTunnel Zeta Client in the SeaTunnel Web run Node. If you run SeaTunnel Zeta Server and SeaTunnel Web in same node, you can skip this step.
apache-seatunnel-2.3.8-bin.tar.gz to the SeaTunnel Web node and unzip it in the same path of SeaTunnel Zeta Server node.SEATUNNEL_HOME to environment variable like SeaTunnel Zeta Server node.hazelcast-client.yaml reference https://seatunnel.apache.org/docs/seatunnel-engine/deployment#6-config-seatunnel-engine-client$SEATUNNEL_HOME/bin/seatunnel.sh --config $SEATUNNEL_HOME/config/v2.batch.config.template, If this job run finished, it indicates successful client deployment.cd seatunnel-web sh build.sh code
Then you can find the installer package in dir seatunnel-web/seatunnel-web-dist/target/apache-seatunnel-web-${project.version}.tar.gz.
Copy the apache-seatunnel-web-${project.version}.tar.gz to your server node and unzip it.
tar -zxvf apache-seatunnel-web-${project.version}.tar.gz
Edit apache-seatunnel-web-${project.version}/script/seatunnel_server_env.sh file, Complete the installed database address, port, username, and password. Here is an example:
export HOSTNAME="localhost" export PORT="3306" export USERNAME="root" export PASSWORD="123456"
Run init shell sh apache-seatunnel-web-${project.version}/script/init_sql.sh If there are no errors during operation, it indicates successful initialization.
apache-seatunnel-web-${project.version}/conf/application.yml Fill in the database connection information and DS interface related information in the file.seatunnel-server/seatunnel-app/src/main/resources/application.yml, add jwt.secretKey value. Eg: https://github.com/apache/seatunnel(Notice that cannot be too short).
$SEATUNNEL_HOME/config/hazelcast-client.yaml to apache-seatunnel-web-${project.version}/conf/apache-seatunnel-2.3.8/connectors/plugin-mapping.properties file to apache-seatunnel-web-${project.version}/conf/ dir.cd apache-seatunnel-web-${project.version} sh bin/seatunnel-backend-daemon.sh start
Accessing in a browser http://127.0.0.1:8801/ui/ Okay, the default username and password are admin/admin.
After all the pre-work is done, we can open the following URL: 127.0.0.1:7890(please replace it according to your configuration) to use it.
Now ,let me show you how to use it.
Placeholders are used in configurations to represent values that will be dynamically substituted at runtime or during processing. They enhance configuration flexibility and reusability by allowing specific values to be injected as needed.
${p1:v1}: p1 is a placeholder for a configuration value. If the p1 value is not provided at execution time, the default value v1 is used. ${p1}: If the p1 value is not provided at execution time, the execution will fail with an error.
To escape placeholders, add a backslash before the placeholder. For example, \${p1:v1} or \${p1}. This is necessary when you want to pass the placeholder as-is to the engine without replacing the value in Seatunnel-web.
During job execution, before sending the job to the engine, Seatunnel-web replaces the actual values in placeholders. This ensures that the actual executed job is recorded in Seatunnel-web history.
NOTE: This feature is currently useful when execution is done through the API. The UI does not provide an option to pass placeholder values.
Execute the following SQL to upgrade the database:
ALTER TABLE `t_st_job_instance` ADD COLUMN `error_message` varchar(4096) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL;
ALTER TABLE `user` ADD COLUMN `auth_provider` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL DEFAULT 'DB';
# sample application.yaml
spring:
ldap:
url: ldap://localhost:389
search:
base: ou=people,dc=example,dc=com
filter: (uid={0})
domain: example.com
seatunnel:
authentication:
providers:
- DB
- LDAP