blob: 9a0e02dcbda01558ff1295a0247a5b8454a16374 [file] [log] [blame]
:mod:`airflow.providers.microsoft.azure.log.wasb_task_handler`
==============================================================
.. py:module:: airflow.providers.microsoft.azure.log.wasb_task_handler
Module Contents
---------------
.. py:class:: WasbTaskHandler(base_log_folder: str, wasb_log_folder: str, wasb_container: str, filename_template: str, delete_local_copy: str)
Bases: :class:`airflow.utils.log.file_task_handler.FileTaskHandler`, :class:`airflow.utils.log.logging_mixin.LoggingMixin`
WasbTaskHandler is a python log handler that handles and reads
task instance logs. It extends airflow FileTaskHandler and
uploads to and reads from Wasb remote storage.
.. method:: hook(self)
Returns WasbHook.
.. method:: set_context(self, ti)
.. method:: close(self)
Close and upload local log file to remote storage Wasb.
.. method:: _read(self, ti, try_number: str, metadata: Optional[str] = None)
Read logs of given task instance and try_number from Wasb remote storage.
If failed, read the log from task instance host machine.
:param ti: task instance object
:param try_number: task instance try_number to read logs from
:param metadata: log metadata,
can be used for steaming log reading and auto-tailing.
.. method:: wasb_log_exists(self, remote_log_location: str)
Check if remote_log_location exists in remote storage
:param remote_log_location: log's location in remote storage
:return: True if location exists else False
.. method:: wasb_read(self, remote_log_location: str, return_error: bool = False)
Returns the log found at the remote_log_location. Returns '' if no
logs are found or there is an error.
:param remote_log_location: the log's location in remote storage
:type remote_log_location: str (path)
:param return_error: if True, returns a string error message if an
error occurs. Otherwise returns '' when an error occurs.
:type return_error: bool
.. method:: wasb_write(self, log: str, remote_log_location: str, append: bool = True)
Writes the log to the remote_log_location. Fails silently if no hook
was created.
:param log: the log to write to the remote_log_location
:type log: str
:param remote_log_location: the log's location in remote storage
:type remote_log_location: str (path)
:param append: if False, any existing log file is overwritten. If True,
the new log is appended to any existing logs.
:type append: bool