Apache Ambari plugin of Apache DolphinScheduler
⚠️⚠️Important⚠️⚠️
⚠️⚠️ NOTE ⚠️⚠️: Apache Ambari plugin only support installation DolphinScheduler for version 1.3.0 - 1.3.9. The reason we split to this repo as below:
- Ambari plug-in is a deployment plugin and not update for two years, and only support DolphinScheduler 1.3.x. DolphinScheduler 2.0 have quit different between 1.3.x and it seem not work in 2.0 anymore.
You could find more detail in mail thread: https://lists.apache.org/thread/5l2zy28q1l5tl87q84h8qsw59z5vg2ly
Note
- This document is intended for users with a basic understanding of Ambari
- This document is a description of adding the Dolphin Scheduler service to the installed Ambari service
- This document is based on version 2.5.2 of Ambari
Installation preparation
Prepare the RPM packages
# Only support version 1.3.0 - 1.3.9, we here use 1.3.3 as example
DOLPHINSCHEDULER_VERSION=1.3.3
git clone --depth 1 --branch ${DOLPHINSCHEDULER_VERSION} git@github.com:apache/dolphinscheduler.git dolphinscheduler_${DOLPHINSCHEDULER_VERSION}_ambari
cd dolphinscheduler_${DOLPHINSCHEDULER_VERSION}_ambari
# It is generated by executing the command, the project root directory (In the directory: dolphinscheduler-dist/target/rpm/apache-dolphinscheduler/RPMS/noarch )
mvn -U clean install -Prpmbuild -Dmaven.test.skip=true -X
Create an installation for DS,who have read and write access to the installation directory (/opt/soft)
Install with rpm package
Copy plug-in directory
- copy directory ambari_plugin/common-services/DOLPHIN to ambari-server/resources/common-services/
- copy directory ambari_plugin/statcks/DOLPHIN to ambari-server/resources/stacks/HDP/2.6/services/--stack version is selected based on the actual situation
Initializes the database information
-- Create the database for the Dolphin Scheduler:dolphinscheduler
CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE
utf8_general_ci;
-- Initialize the user and password for the dolphinscheduler database and assign permissions
-- Replace the {user} in the SQL statement below with the user of the dolphinscheduler database
GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'%' IDENTIFIED BY '{password}';
GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'localhost' IDENTIFIED BY
'{password}';
flush privileges;
Ambari Install Dolphin Scheduler
NOTE: You have to install zookeeper first
Install Dolphin Scheduler on ambari web interface
Select the nodes for the Dolphin Scheduler's Master installation
Configure the Dolphin Scheduler's nodes for Worker, Api, Logger, Alert installation
Set the installation users of the Dolphin Scheduler service (created in step 1) and the user groups they belong to
System Env Optimization will export some system environment config. Modify according to actual situation
Configure the database information (same as in the initialization database in step 1)
Configure additional information if needed
Perform the next steps as normal
The interface after successful installation
Add components to the node through Ambari -- for example, add a DS Worker
NOTE: DS Logger is the installation dependent component of DS Worker in Dolphin's Ambari installation (need to add installation first; Prevent the Job log on the corresponding Worker from being checked)
Locate the component node to add -- for example, node ark3
Add components -- the drop-down list is all addable
Confirm component addition
After adding DS Worker and DS Logger components
Start the component
Remove the component from the node with Ambari
Stop the component in the corresponding node
Remove components