| :py:mod:`airflow.providers.apache.druid.hooks.druid` |
| ==================================================== |
| |
| .. py:module:: airflow.providers.apache.druid.hooks.druid |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.apache.druid.hooks.druid.DruidHook |
| airflow.providers.apache.druid.hooks.druid.DruidDbApiHook |
| |
| |
| |
| |
| .. py:class:: DruidHook(druid_ingest_conn_id = 'druid_ingest_default', timeout = 1, max_ingestion_time = None) |
| |
| Bases: :py:obj:`airflow.hooks.base.BaseHook` |
| |
| Connection to Druid overlord for ingestion |
| |
| To connect to a Druid cluster that is secured with the druid-basic-security |
| extension, add the username and password to the druid ingestion connection. |
| |
| :param druid_ingest_conn_id: The connection id to the Druid overlord machine |
| which accepts index jobs |
| :param timeout: The interval between polling |
| the Druid job for the status of the ingestion job. |
| Must be greater than or equal to 1 |
| :param max_ingestion_time: The maximum ingestion time before assuming the job failed |
| |
| .. py:method:: get_conn_url(self) |
| |
| Get Druid connection url |
| |
| |
| .. py:method:: get_auth(self) |
| |
| Return username and password from connections tab as requests.auth.HTTPBasicAuth object. |
| |
| If these details have not been set then returns None. |
| |
| |
| .. py:method:: submit_indexing_job(self, json_index_spec) |
| |
| Submit Druid ingestion job |
| |
| |
| |
| .. py:class:: DruidDbApiHook(*args, schema = None, **kwargs) |
| |
| Bases: :py:obj:`airflow.hooks.dbapi.DbApiHook` |
| |
| Interact with Druid broker |
| |
| This hook is purely for users to query druid broker. |
| For ingestion, please use druidHook. |
| |
| .. py:attribute:: conn_name_attr |
| :annotation: = druid_broker_conn_id |
| |
| |
| |
| .. py:attribute:: default_conn_name |
| :annotation: = druid_broker_default |
| |
| |
| |
| .. py:attribute:: conn_type |
| :annotation: = druid |
| |
| |
| |
| .. py:attribute:: hook_name |
| :annotation: = Druid |
| |
| |
| |
| .. py:attribute:: supports_autocommit |
| :annotation: = False |
| |
| |
| |
| .. py:method:: get_conn(self) |
| |
| Establish a connection to druid broker. |
| |
| |
| .. py:method:: get_uri(self) |
| |
| Get the connection uri for druid broker. |
| |
| e.g: druid://localhost:8082/druid/v2/sql/ |
| |
| |
| .. py:method:: set_autocommit(self, conn, autocommit) |
| :abstractmethod: |
| |
| Sets the autocommit flag on the connection |
| |
| |
| .. py:method:: insert_rows(self, table, rows, target_fields = None, commit_every = 1000, replace = False, **kwargs) |
| :abstractmethod: |
| |
| A generic way to insert a set of tuples into a table, |
| a new transaction is created every commit_every rows |
| |
| :param table: Name of the target table |
| :param rows: The rows to insert into the table |
| :param target_fields: The names of the columns to fill in the table |
| :param commit_every: The maximum number of rows to insert in one |
| transaction. Set to 0 to insert all rows in one transaction. |
| :param replace: Whether to replace instead of insert |
| |
| |
| |
