| :mod:`airflow.providers.amazon.aws.operators.glue_crawler` |
| ========================================================== |
| |
| .. py:module:: airflow.providers.amazon.aws.operators.glue_crawler |
| |
| |
| Module Contents |
| --------------- |
| |
| .. py:class:: AwsGlueCrawlerOperator(config, aws_conn_id='aws_default', poll_interval: int = 5, **kwargs) |
| |
| Bases: :class:`airflow.models.BaseOperator` |
| |
| Creates, updates and triggers an AWS Glue Crawler. AWS Glue Crawler is a serverless |
| service that manages a catalog of metadata tables that contain the inferred |
| schema, format and data types of data stores within the AWS cloud. |
| |
| :param config: Configurations for the AWS Glue crawler |
| :type config: dict |
| :param aws_conn_id: aws connection to use |
| :type aws_conn_id: Optional[str] |
| :param poll_interval: Time (in seconds) to wait between two consecutive calls to check crawler status |
| :type poll_interval: Optional[int] |
| |
| .. attribute:: ui_color |
| :annotation: = #ededed |
| |
| |
| |
| |
| .. method:: hook(self) |
| |
| Create and return an AwsGlueCrawlerHook. |
| |
| |
| |
| |
| .. method:: execute(self, context) |
| |
| Executes AWS Glue Crawler from Airflow |
| |
| :return: the name of the current glue crawler. |
| |
| |
| |
| |
