blob: 0d2d3bcaf46418e6cb6626c13b72ea03841be843 [file] [log] [blame]
:py:mod:`airflow.providers.apache.hive.sensors.metastore_partition`
===================================================================
.. py:module:: airflow.providers.apache.hive.sensors.metastore_partition
Module Contents
---------------
Classes
~~~~~~~
.. autoapisummary::
airflow.providers.apache.hive.sensors.metastore_partition.MetastorePartitionSensor
.. py:class:: MetastorePartitionSensor(*, table, partition_name, schema = 'default', mysql_conn_id = 'metastore_mysql', **kwargs)
Bases: :py:obj:`airflow.providers.common.sql.sensors.sql.SqlSensor`
An alternative to the HivePartitionSensor that talk directly to the MySQL db.
This was created as a result of observing sub optimal queries generated by the
Metastore thrift service when hitting subpartitioned tables. The Thrift service's
queries were written in a way that would not leverage the indexes.
:param schema: the schema
:param table: the table
:param partition_name: the partition name, as defined in the PARTITIONS
table of the Metastore. Order of the fields does matter.
Examples: ``ds=2016-01-01`` or
``ds=2016-01-01/sub=foo`` for a sub partitioned table
:param mysql_conn_id: a reference to the MySQL conn_id for the metastore
.. py:attribute:: template_fields
:type: Sequence[str]
:value: ('partition_name', 'table', 'schema')
.. py:attribute:: ui_color
:value: '#8da7be'
.. py:method:: poke(context)
Override when deriving this class.