Hive source connector
Get data from hive
:::tip
Engine Supported and plugin name
:::
name | type | required | default value |
---|---|---|---|
pre_sql | string | yes | - |
common-options | string | yes | - |
For preprocessed sql
, if preprocessing is not required, you can use select * from hive_db.hive_table
.
Source plugin common parameters, please refer to Source Plugin for details
Note: The following configuration must be done to use hive source:
# In the spark section in the seatunnel configuration file: env { ... spark.sql.catalogImplementation = "hive" ... }
env { ... spark.sql.catalogImplementation = "hive" ... } source { hive { pre_sql = "select * from mydb.mytb" result_table_name = "myTable" } } ...
It must be ensured that the metastore
of hive
is in service. Start the command hive --service metastore
service default port 9083
cluster
, client
, local
mode, hive-site.xml
must be placed in the $HADOOP_CONF
directory of the task submission node (or placed under $SPARK_HOME/conf
), IDE local Debug put it in the resources
directory