blob: 04b324fe11ba69d4a69f90c83f6da5b0c434252a [file] [log] [blame]
:py:mod:`airflow.providers.google.cloud.log.gcs_task_handler`
=============================================================
.. py:module:: airflow.providers.google.cloud.log.gcs_task_handler
Module Contents
---------------
Classes
~~~~~~~
.. autoapisummary::
airflow.providers.google.cloud.log.gcs_task_handler.GCSTaskHandler
.. py:class:: GCSTaskHandler(*, base_log_folder, gcs_log_folder, filename_template, gcp_key_path = None, gcp_keyfile_dict = None, gcp_scopes = _DEFAULT_SCOPESS, project_id = None)
Bases: :py:obj:`airflow.utils.log.file_task_handler.FileTaskHandler`, :py:obj:`airflow.utils.log.logging_mixin.LoggingMixin`
GCSTaskHandler is a python log handler that handles and reads
task instance logs. It extends airflow FileTaskHandler and
uploads to and reads from GCS remote storage. Upon log reading
failure, it reads from host machine's local disk.
:param base_log_folder: Base log folder to place logs.
:param gcs_log_folder: Path to a remote location where logs will be saved. It must have the prefix
``gs://``. For example: ``gs://bucket/remote/log/location``
:param filename_template: template filename string
:param gcp_key_path: Path to Google Cloud Service Account file (JSON). Mutually exclusive with
gcp_keyfile_dict.
If omitted, authorization based on `the Application Default Credentials
<https://cloud.google.com/docs/authentication/production#finding_credentials_automatically>`__ will
be used.
:param gcp_keyfile_dict: Dictionary of keyfile parameters. Mutually exclusive with gcp_key_path.
:param gcp_scopes: Comma-separated string containing OAuth2 scopes
:param project_id: Project ID to read the secrets from. If not passed, the project ID from credentials
will be used.
.. py:method:: client(self)
Returns GCS Client.
.. py:method:: set_context(self, ti)
Provide task_instance context to airflow task handler.
:param ti: task instance object
.. py:method:: close(self)
Close and upload local log file to remote storage GCS.
.. py:method:: gcs_write(self, log, remote_log_location)
Writes the log to the remote_log_location. Fails silently if no log
was created.
:param log: the log to write to the remote_log_location
:param remote_log_location: the log's location in remote storage