docs-archive/1.10.13/_sources/_api/airflow/contrib/hooks/aws_glue_catalog_hook/index.rst.txt

:mod:`airflow.contrib.hooks.aws_glue_catalog_hook`
==================================================

.. py:module:: airflow.contrib.hooks.aws_glue_catalog_hook

.. autoapi-nested-parse::

   This module contains AWS Glue Catalog Hook



Module Contents
---------------

.. py:class:: AwsGlueCatalogHook(aws_conn_id='aws_default', region_name=None, *args, **kwargs)

   Bases: :class:`airflow.contrib.hooks.aws_hook.AwsHook`

   Interact with AWS Glue Catalog

   :param aws_conn_id: ID of the Airflow connection where
       credentials and extra configuration are stored
   :type aws_conn_id: str
   :param region_name: aws region name (example: us-east-1)
   :type region_name: str

   
   .. method:: get_conn(self)

      Returns glue connection object.



   
   .. method:: get_partitions(self, database_name, table_name, expression='', page_size=None, max_items=None)

      Retrieves the partition values for a table.

      :param database_name: The name of the catalog database where the partitions reside.
      :type database_name: str
      :param table_name: The name of the partitions' table.
      :type table_name: str
      :param expression: An expression filtering the partitions to be returned.
          Please see official AWS documentation for further information.
          https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-catalog-partitions.html#aws-glue-api-catalog-partitions-GetPartitions
      :type expression: str
      :param page_size: pagination size
      :type page_size: int
      :param max_items: maximum items to return
      :type max_items: int
      :return: set of partition values where each value is a tuple since
          a partition may be composed of multiple columns. For example:
          ``{('2018-01-01','1'), ('2018-01-01','2')}``



   
   .. method:: check_for_partition(self, database_name, table_name, expression)

      Checks whether a partition exists

      :param database_name: Name of hive database (schema) @table belongs to
      :type database_name: str
      :param table_name: Name of hive table @partition belongs to
      :type table_name: str
      :expression: Expression that matches the partitions to check for
          (eg `a = 'b' AND c = 'd'`)
      :type expression: str
      :rtype: bool

      >>> hook = AwsGlueCatalogHook()
      >>> t = 'static_babynames_partitioned'
      >>> hook.check_for_partition('airflow', t, "ds='2015-01-01'")
      True



   
   .. method:: get_table(self, database_name, table_name)

      Get the information of the table

      :param database_name: Name of hive database (schema) @table belongs to
      :type database_name: str
      :param table_name: Name of hive table
      :type table_name: str
      :rtype: dict

      >>> hook = AwsGlueCatalogHook()
      >>> r = hook.get_table('db', 'table_foo')
      >>> r['Name'] = 'table_foo'



   
   .. method:: get_table_location(self, database_name, table_name)

      Get the physical location of the table

      :param database_name: Name of hive database (schema) @table belongs to
      :type database_name: str
      :param table_name: Name of hive table
      :type table_name: str
      :return: str