| :mod:`airflow.sensors.metastore_partition_sensor` |
| ================================================= |
| |
| .. py:module:: airflow.sensors.metastore_partition_sensor |
| |
| |
| Module Contents |
| --------------- |
| |
| .. py:class:: MetastorePartitionSensor(table, partition_name, schema='default', mysql_conn_id='metastore_mysql', *args, **kwargs) |
| |
| Bases: :class:`airflow.sensors.sql_sensor.SqlSensor` |
| |
| An alternative to the HivePartitionSensor that talk directly to the |
| MySQL db. This was created as a result of observing sub optimal |
| queries generated by the Metastore thrift service when hitting |
| subpartitioned tables. The Thrift service's queries were written in a |
| way that wouldn't leverage the indexes. |
| |
| :param schema: the schema |
| :type schema: str |
| :param table: the table |
| :type table: str |
| :param partition_name: the partition name, as defined in the PARTITIONS |
| table of the Metastore. Order of the fields does matter. |
| Examples: ``ds=2016-01-01`` or |
| ``ds=2016-01-01/sub=foo`` for a sub partitioned table |
| :type partition_name: str |
| :param mysql_conn_id: a reference to the MySQL conn_id for the metastore |
| :type mysql_conn_id: str |
| |
| .. attribute:: template_fields |
| :annotation: = ['partition_name', 'table', 'schema'] |
| |
| |
| |
| .. attribute:: ui_color |
| :annotation: = #8da7be |
| |
| |
| |
| |
| .. method:: poke(self, context) |
| |
| |
| |
| |
