For Griffin users, please follow the instructions below to deploy Griffin in your environment. Note that there are some dependencies that should be installed firstly.
You need to install following items
$SPARK_HOME/lib/
, and put them into HDFS.datanucleus-api-jdo-3.2.6.jar datanucleus-core-3.2.10.jar datanucleus-rdbms-3.2.9.jar
Create database ‘quartz’ in PostgreSQL
createdb -O <username> quartz
Init quartz tables in PostgreSQL using Init_quartz_postgres.sql
psql -p <password> -h <host address> -U <username> -f Init_quartz_postgres.sql quartz
Create database ‘quartz’ in MySQL
mysql -u <username> -e "create database quartz" -p
Init quartz tables in MySQL using Init_quartz_mysql_innodb.sql.sql
mysql -u <username> -p quartz < Init_quartz_mysql_innodb.sql.sql
You should also modify some configurations of Griffin for your environment.
service/src/main/resources/application.properties
# griffin server port (default 8080) server.port = 8080 # jpa spring.datasource.url = jdbc:postgresql://<your IP>:5432/quartz?autoReconnect=true&useSSL=false spring.datasource.username = <user name> spring.datasource.password = <password> spring.jpa.generate-ddl=true spring.datasource.driverClassName = org.postgresql.Driver spring.jpa.show-sql = true # hive metastore hive.metastore.uris = thrift://<your IP>:9083 hive.metastore.dbname = <hive database name> # default is "default" # external properties directory location, ignore it if not required external.config.location = # login strategy, default is "default" login.strategy = <default or ldap> # ldap properties, ignore them if ldap is not enabled ldap.url = ldap://hostname:port ldap.email = @example.com ldap.searchBase = DC=org,DC=example ldap.searchPattern = (sAMAccountName={0}) # hdfs, ignore it if you do not need predicate job fs.defaultFS = hdfs://<hdfs-default-name> # elasticsearch elasticsearch.host = <your IP> elasticsearch.port = <your elasticsearch rest port> # authentication properties, uncomment if basic authentication is enabled # elasticsearch.user = user # elasticsearch.password = password # livy # Port Livy: 8998 Livy2:8999 livy.uri=http://localhost:8999/batches # yarn url yarn.uri=http://localhost:8088
service/src/main/resources/sparkProperties.json
{ "file": "hdfs:///<griffin measure path>/griffin-measure.jar", "className": "org.apache.griffin.measure.Application", "name": "griffin", "queue": "default", "numExecutors": 3, "executorCores": 1, "driverMemory": "1g", "executorMemory": "1g", "conf": { "spark.yarn.dist.files": "hdfs:///<path to>/hive-site.xml" }, "files": [ ] }
Build the whole project and deploy. (NPM should be installed)
mvn clean install
Put jar file of measure module into <griffin measure path> in HDFS
cp measure/target/measure-<version>-incubating-SNAPSHOT.jar measure/target/griffin-measure.jar hdfs dfs -put measure/target/griffin-measure.jar <griffin measure path>/
After all environment services startup, we can start our server.
java -jar service/target/service.jar
After a few seconds, we can visit our default UI of Griffin (by default the port of spring boot is 8080).
http://<your IP>:8080
You can use UI following the steps here.
Note: The UI does not support all the backend features, to experience the advanced features you can use services directly.