| :py:mod:`airflow.providers.amazon.aws.operators.glue_crawler` |
| ============================================================= |
| |
| .. py:module:: airflow.providers.amazon.aws.operators.glue_crawler |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.amazon.aws.operators.glue_crawler.GlueCrawlerOperator |
| airflow.providers.amazon.aws.operators.glue_crawler.AwsGlueCrawlerOperator |
| |
| |
| |
| |
| .. py:class:: GlueCrawlerOperator(config, aws_conn_id='aws_default', poll_interval = 5, wait_for_completion = True, **kwargs) |
| |
| Bases: :py:obj:`airflow.models.BaseOperator` |
| |
| Creates, updates and triggers an AWS Glue Crawler. AWS Glue Crawler is a serverless |
| service that manages a catalog of metadata tables that contain the inferred |
| schema, format and data types of data stores within the AWS cloud. |
| |
| .. seealso:: |
| For more information on how to use this operator, take a look at the guide: |
| :ref:`howto/operator:GlueCrawlerOperator` |
| |
| :param config: Configurations for the AWS Glue crawler |
| :param aws_conn_id: aws connection to use |
| :param poll_interval: Time (in seconds) to wait between two consecutive calls to check crawler status |
| :param wait_for_completion: Whether or not wait for crawl execution completion. (default: True) |
| |
| .. py:attribute:: ui_color |
| :annotation: = #ededed |
| |
| |
| |
| .. py:method:: hook(self) |
| |
| Create and return an GlueCrawlerHook. |
| |
| |
| .. py:method:: execute(self, context) |
| |
| Executes AWS Glue Crawler from Airflow |
| |
| :return: the name of the current glue crawler. |
| |
| |
| |
| .. py:class:: AwsGlueCrawlerOperator(*args, **kwargs) |
| |
| Bases: :py:obj:`GlueCrawlerOperator` |
| |
| This operator is deprecated. |
| Please use :class:`airflow.providers.amazon.aws.operators.glue_crawler.GlueCrawlerOperator`. |
| |
| |
