| :mod:`airflow.providers.apache.hive.sensors.metastore_partition` |
| ================================================================ |
| |
| .. py:module:: airflow.providers.apache.hive.sensors.metastore_partition |
| |
| |
| Module Contents |
| --------------- |
| |
| .. py:class:: MetastorePartitionSensor(*, table: str, partition_name: str, schema: str = 'default', mysql_conn_id: str = 'metastore_mysql', **kwargs) |
| |
| Bases: :class:`airflow.sensors.sql.SqlSensor` |
| |
| An alternative to the HivePartitionSensor that talk directly to the |
| MySQL db. This was created as a result of observing sub optimal |
| queries generated by the Metastore thrift service when hitting |
| subpartitioned tables. The Thrift service's queries were written in a |
| way that wouldn't leverage the indexes. |
| |
| :param schema: the schema |
| :type schema: str |
| :param table: the table |
| :type table: str |
| :param partition_name: the partition name, as defined in the PARTITIONS |
| table of the Metastore. Order of the fields does matter. |
| Examples: ``ds=2016-01-01`` or |
| ``ds=2016-01-01/sub=foo`` for a sub partitioned table |
| :type partition_name: str |
| :param mysql_conn_id: a reference to the MySQL conn_id for the metastore |
| :type mysql_conn_id: str |
| |
| .. attribute:: template_fields |
| :annotation: = ['partition_name', 'table', 'schema'] |
| |
| |
| |
| .. attribute:: ui_color |
| :annotation: = #8da7be |
| |
| |
| |
| .. attribute:: poke_context_fields |
| :annotation: = ['partition_name', 'table', 'schema', 'mysql_conn_id'] |
| |
| |
| |
| |
| .. method:: poke(self, context: Dict[str, Any]) |
| |
| |
| |
| |
