| :py:mod:`airflow.providers.amazon.aws.hooks.base_aws` |
| ===================================================== |
| |
| .. py:module:: airflow.providers.amazon.aws.hooks.base_aws |
| |
| .. autoapi-nested-parse:: |
| |
| This module contains Base AWS Hook. |
| |
| .. seealso:: |
| For more information on how to use this hook, take a look at the guide: |
| :ref:`howto/connection:AWSHook` |
| |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.amazon.aws.hooks.base_aws.BaseSessionFactory |
| airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook |
| airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook |
| |
| |
| |
| Functions |
| ~~~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.amazon.aws.hooks.base_aws.resolve_session_factory |
| |
| |
| |
| Attributes |
| ~~~~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.amazon.aws.hooks.base_aws.BaseAwsConnection |
| airflow.providers.amazon.aws.hooks.base_aws.SessionFactory |
| |
| |
| .. py:data:: BaseAwsConnection |
| |
| |
| |
| |
| .. py:class:: BaseSessionFactory(conn, region_name = None, config = None) |
| |
| Bases: :py:obj:`airflow.utils.log.logging_mixin.LoggingMixin` |
| |
| Base AWS Session Factory class to handle boto3 session creation. |
| It can handle most of the AWS supported authentication methods. |
| |
| User can also derive from this class to have full control of boto3 session |
| creation or to support custom federation. |
| |
| .. seealso:: |
| :ref:`howto/connection:aws:session-factory` |
| |
| .. py:method:: conn() |
| |
| Cached AWS Connection Wrapper. |
| |
| |
| .. py:method:: basic_session() |
| |
| Cached property with basic boto3.session.Session. |
| |
| |
| .. py:method:: extra_config() |
| :property: |
| |
| AWS Connection extra_config. |
| |
| |
| .. py:method:: region_name() |
| :property: |
| |
| AWS Region Name read-only property. |
| |
| |
| .. py:method:: config() |
| :property: |
| |
| Configuration for botocore client read-only property. |
| |
| |
| .. py:method:: role_arn() |
| :property: |
| |
| Assume Role ARN from AWS Connection |
| |
| |
| .. py:method:: create_session() |
| |
| Create boto3 Session from connection config. |
| |
| |
| |
| .. py:class:: AwsGenericHook(aws_conn_id = default_conn_name, verify = None, region_name = None, client_type = None, resource_type = None, config = None) |
| |
| Bases: :py:obj:`airflow.hooks.base.BaseHook`, :py:obj:`Generic`\ [\ :py:obj:`BaseAwsConnection`\ ] |
| |
| Interact with AWS. |
| This class is a thin wrapper around the boto3 python library. |
| |
| :param aws_conn_id: The Airflow connection used for AWS credentials. |
| If this is None or empty then the default boto3 behaviour is used. If |
| running Airflow in a distributed manner and aws_conn_id is None or |
| empty, then default boto3 configuration would be used (and must be |
| maintained on each worker node). |
| :param verify: Whether or not to verify SSL certificates. See: |
| https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html |
| :param region_name: AWS region_name. If not specified then the default boto3 behaviour is used. |
| :param client_type: boto3.client client_type. Eg 's3', 'emr' etc |
| :param resource_type: boto3.resource resource_type. Eg 'dynamodb' etc |
| :param config: Configuration for botocore client. See: |
| https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html |
| |
| .. py:attribute:: conn_name_attr |
| :annotation: = aws_conn_id |
| |
| |
| |
| .. py:attribute:: default_conn_name |
| :annotation: = aws_default |
| |
| |
| |
| .. py:attribute:: conn_type |
| :annotation: = aws |
| |
| |
| |
| .. py:attribute:: hook_name |
| :annotation: = Amazon Web Services |
| |
| |
| |
| .. py:method:: conn_config() |
| |
| Get the Airflow Connection object and wrap it in helper (cached). |
| |
| |
| .. py:method:: region_name() |
| :property: |
| |
| AWS Region Name read-only property. |
| |
| |
| .. py:method:: config() |
| :property: |
| |
| Configuration for botocore client read-only property. |
| |
| |
| .. py:method:: get_session(region_name = None) |
| |
| Get the underlying boto3.session.Session(region_name=region_name). |
| |
| |
| .. py:method:: get_client_type(region_name = None, config = None) |
| |
| Get the underlying boto3 client using boto3 session |
| |
| |
| .. py:method:: get_resource_type(region_name = None, config = None) |
| |
| Get the underlying boto3 resource using boto3 session |
| |
| |
| .. py:method:: conn() |
| |
| Get the underlying boto3 client/resource (cached) |
| |
| :return: boto3.client or boto3.resource |
| :rtype: Union[boto3.client, boto3.resource] |
| |
| |
| .. py:method:: conn_client_meta() |
| |
| Get botocore client metadata from Hook connection (cached). |
| |
| |
| .. py:method:: conn_region_name() |
| :property: |
| |
| Get actual AWS Region Name from Hook connection (cached). |
| |
| |
| .. py:method:: conn_partition() |
| :property: |
| |
| Get associated AWS Region Partition from Hook connection (cached). |
| |
| |
| .. py:method:: get_conn() |
| |
| Get the underlying boto3 client/resource (cached) |
| |
| Implemented so that caching works as intended. It exists for compatibility |
| with subclasses that rely on a super().get_conn() method. |
| |
| :return: boto3.client or boto3.resource |
| :rtype: Union[boto3.client, boto3.resource] |
| |
| |
| .. py:method:: get_credentials(region_name = None) |
| |
| Get the underlying `botocore.Credentials` object. |
| |
| This contains the following authentication attributes: access_key, secret_key and token. |
| |
| |
| .. py:method:: expand_role(role, region_name = None) |
| |
| If the IAM role is a role name, get the Amazon Resource Name (ARN) for the role. |
| If IAM role is already an IAM role ARN, no change is made. |
| |
| :param role: IAM role name or ARN |
| :param region_name: Optional region name to get credentials for |
| :return: IAM role ARN |
| |
| |
| .. py:method:: retry(should_retry) |
| :staticmethod: |
| |
| A decorator that provides a mechanism to repeat requests in response to exceeding a temporary quote |
| limit. |
| |
| |
| .. py:method:: get_ui_field_behaviour() |
| :staticmethod: |
| |
| Returns custom UI field behaviour for AWS Connection. |
| |
| |
| .. py:method:: test_connection() |
| |
| Tests the AWS connection by call AWS STS (Security Token Service) GetCallerIdentity API. |
| |
| .. seealso:: |
| https://docs.aws.amazon.com/STS/latest/APIReference/API_GetCallerIdentity.html |
| |
| |
| |
| .. py:class:: AwsBaseHook(aws_conn_id = default_conn_name, verify = None, region_name = None, client_type = None, resource_type = None, config = None) |
| |
| Bases: :py:obj:`AwsGenericHook`\ [\ :py:obj:`Union`\ [\ :py:obj:`boto3.client`\ , :py:obj:`boto3.resource`\ ]\ ] |
| |
| Interact with AWS. |
| This class is a thin wrapper around the boto3 python library |
| with basic conn annotation. |
| |
| .. seealso:: |
| :class:`~airflow.providers.amazon.aws.hooks.base_aws.AwsGenericHook` |
| |
| |
| .. py:function:: resolve_session_factory() |
| |
| Resolves custom SessionFactory class |
| |
| |
| .. py:data:: SessionFactory |
| |
| |
| |
| |