| :py:mod:`airflow.providers.google.cloud.log.gcs_task_handler` |
| ============================================================= |
| |
| .. py:module:: airflow.providers.google.cloud.log.gcs_task_handler |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.google.cloud.log.gcs_task_handler.GCSTaskHandler |
| |
| |
| |
| |
| .. py:class:: GCSTaskHandler(*, base_log_folder, gcs_log_folder, filename_template, gcp_key_path = None, gcp_keyfile_dict = None, gcp_scopes = _DEFAULT_SCOPESS, project_id = None) |
| |
| Bases: :py:obj:`airflow.utils.log.file_task_handler.FileTaskHandler`, :py:obj:`airflow.utils.log.logging_mixin.LoggingMixin` |
| |
| GCSTaskHandler is a python log handler that handles and reads |
| task instance logs. It extends airflow FileTaskHandler and |
| uploads to and reads from GCS remote storage. Upon log reading |
| failure, it reads from host machine's local disk. |
| |
| :param base_log_folder: Base log folder to place logs. |
| :param gcs_log_folder: Path to a remote location where logs will be saved. It must have the prefix |
| ``gs://``. For example: ``gs://bucket/remote/log/location`` |
| :param filename_template: template filename string |
| :param gcp_key_path: Path to Google Cloud Service Account file (JSON). Mutually exclusive with |
| gcp_keyfile_dict. |
| If omitted, authorization based on `the Application Default Credentials |
| <https://cloud.google.com/docs/authentication/production#finding_credentials_automatically>`__ will |
| be used. |
| :param gcp_keyfile_dict: Dictionary of keyfile parameters. Mutually exclusive with gcp_key_path. |
| :param gcp_scopes: Comma-separated string containing OAuth2 scopes |
| :param project_id: Project ID to read the secrets from. If not passed, the project ID from credentials |
| will be used. |
| |
| .. py:method:: client(self) |
| |
| Returns GCS Client. |
| |
| |
| .. py:method:: set_context(self, ti) |
| |
| Provide task_instance context to airflow task handler. |
| |
| :param ti: task instance object |
| |
| |
| .. py:method:: close(self) |
| |
| Close and upload local log file to remote storage GCS. |
| |
| |
| .. py:method:: gcs_write(self, log, remote_log_location) |
| |
| Writes the log to the remote_log_location. Fails silently if no log |
| was created. |
| |
| :param log: the log to write to the remote_log_location |
| :param remote_log_location: the log's location in remote storage |
| |
| |
| |