| :py:mod:`airflow.providers.apache.hive.sensors.hive_partition` |
| ============================================================== |
| |
| .. py:module:: airflow.providers.apache.hive.sensors.hive_partition |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.apache.hive.sensors.hive_partition.HivePartitionSensor |
| |
| |
| |
| |
| .. py:class:: HivePartitionSensor(*, table, partition = "ds='{{ ds }}'", metastore_conn_id = 'metastore_default', schema = 'default', poke_interval = 60 * 3, **kwargs) |
| |
| Bases: :py:obj:`airflow.sensors.base.BaseSensorOperator` |
| |
| Waits for a partition to show up in Hive. |
| |
| Note: Because ``partition`` supports general logical operators, it |
| can be inefficient. Consider using NamedHivePartitionSensor instead if |
| you don't need the full flexibility of HivePartitionSensor. |
| |
| :param table: The name of the table to wait for, supports the dot |
| notation (my_database.my_table) |
| :param partition: The partition clause to wait for. This is passed as |
| is to the metastore Thrift client ``get_partitions_by_filter`` method, |
| and apparently supports SQL like notation as in ``ds='2015-01-01' |
| AND type='value'`` and comparison operators as in ``"ds>=2015-01-01"`` |
| :param metastore_conn_id: reference to the |
| :ref: `metastore thrift service connection id <howto/connection:hive_metastore>` |
| |
| .. py:attribute:: template_fields |
| :annotation: :Sequence[str] = ['schema', 'table', 'partition'] |
| |
| |
| |
| .. py:attribute:: ui_color |
| :annotation: = #C5CAE9 |
| |
| |
| |
| .. py:method:: poke(self, context) |
| |
| Function that the sensors defined while deriving this class should |
| override. |
| |
| |
| |
