| :py:mod:`airflow.providers.ftp.hooks.ftp` |
| ========================================= |
| |
| .. py:module:: airflow.providers.ftp.hooks.ftp |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.ftp.hooks.ftp.FTPHook |
| airflow.providers.ftp.hooks.ftp.FTPSHook |
| |
| |
| |
| |
| .. py:class:: FTPHook(ftp_conn_id = default_conn_name) |
| |
| Bases: :py:obj:`airflow.hooks.base.BaseHook` |
| |
| Interact with FTP. |
| |
| Errors that may occur throughout but should be handled downstream. |
| You can specify mode for data transfers in the extra field of your |
| connection as ``{"passive": "true"}``. |
| |
| :param ftp_conn_id: The :ref:`ftp connection id <howto/connection:ftp>` |
| reference. |
| |
| .. py:attribute:: conn_name_attr |
| :annotation: = ftp_conn_id |
| |
| |
| |
| .. py:attribute:: default_conn_name |
| :annotation: = ftp_default |
| |
| |
| |
| .. py:attribute:: conn_type |
| :annotation: = ftp |
| |
| |
| |
| .. py:attribute:: hook_name |
| :annotation: = FTP |
| |
| |
| |
| .. py:method:: __enter__(self) |
| |
| |
| .. py:method:: __exit__(self, exc_type, exc_val, exc_tb) |
| |
| |
| .. py:method:: get_conn(self) |
| |
| Returns a FTP connection object |
| |
| |
| .. py:method:: close_conn(self) |
| |
| Closes the connection. An error will occur if the |
| connection wasn't ever opened. |
| |
| |
| .. py:method:: describe_directory(self, path) |
| |
| Returns a dictionary of {filename: {attributes}} for all files |
| on the remote system (where the MLSD command is supported). |
| |
| :param path: full path to the remote directory |
| |
| |
| .. py:method:: list_directory(self, path) |
| |
| Returns a list of files on the remote system. |
| |
| :param path: full path to the remote directory to list |
| |
| |
| .. py:method:: create_directory(self, path) |
| |
| Creates a directory on the remote system. |
| |
| :param path: full path to the remote directory to create |
| |
| |
| .. py:method:: delete_directory(self, path) |
| |
| Deletes a directory on the remote system. |
| |
| :param path: full path to the remote directory to delete |
| |
| |
| .. py:method:: retrieve_file(self, remote_full_path, local_full_path_or_buffer, callback=None) |
| |
| Transfers the remote file to a local location. |
| |
| If local_full_path_or_buffer is a string path, the file will be put |
| at that location; if it is a file-like buffer, the file will |
| be written to the buffer but not closed. |
| |
| :param remote_full_path: full path to the remote file |
| :param local_full_path_or_buffer: full path to the local file or a |
| file-like buffer |
| :param callback: callback which is called each time a block of data |
| is read. if you do not use a callback, these blocks will be written |
| to the file or buffer passed in. if you do pass in a callback, note |
| that writing to a file or buffer will need to be handled inside the |
| callback. |
| [default: output_handle.write()] |
| |
| .. code-block:: python |
| |
| hook = FTPHook(ftp_conn_id="my_conn") |
| |
| remote_path = "/path/to/remote/file" |
| local_path = "/path/to/local/file" |
| |
| # with a custom callback (in this case displaying progress on each read) |
| def print_progress(percent_progress): |
| self.log.info("Percent Downloaded: %s%%" % percent_progress) |
| |
| |
| total_downloaded = 0 |
| total_file_size = hook.get_size(remote_path) |
| output_handle = open(local_path, "wb") |
| |
| |
| def write_to_file_with_progress(data): |
| total_downloaded += len(data) |
| output_handle.write(data) |
| percent_progress = (total_downloaded / total_file_size) * 100 |
| print_progress(percent_progress) |
| |
| |
| hook.retrieve_file(remote_path, None, callback=write_to_file_with_progress) |
| |
| # without a custom callback data is written to the local_path |
| hook.retrieve_file(remote_path, local_path) |
| |
| |
| |
| .. py:method:: store_file(self, remote_full_path, local_full_path_or_buffer) |
| |
| Transfers a local file to the remote location. |
| |
| If local_full_path_or_buffer is a string path, the file will be read |
| from that location; if it is a file-like buffer, the file will |
| be read from the buffer but not closed. |
| |
| :param remote_full_path: full path to the remote file |
| :param local_full_path_or_buffer: full path to the local file or a |
| file-like buffer |
| |
| |
| .. py:method:: delete_file(self, path) |
| |
| Removes a file on the FTP Server. |
| |
| :param path: full path to the remote file |
| |
| |
| .. py:method:: rename(self, from_name, to_name) |
| |
| Rename a file. |
| |
| :param from_name: rename file from name |
| :param to_name: rename file to name |
| |
| |
| .. py:method:: get_mod_time(self, path) |
| |
| Returns a datetime object representing the last time the file was modified |
| |
| :param path: remote file path |
| |
| |
| .. py:method:: get_size(self, path) |
| |
| Returns the size of a file (in bytes) |
| |
| :param path: remote file path |
| |
| |
| .. py:method:: test_connection(self) |
| |
| Test the FTP connection by calling path with directory |
| |
| |
| |
| .. py:class:: FTPSHook(ftp_conn_id = default_conn_name) |
| |
| Bases: :py:obj:`FTPHook` |
| |
| Interact with FTPS. |
| |
| .. py:method:: get_conn(self) |
| |
| Returns a FTPS connection object. |
| |
| |
| |