| :py:mod:`airflow.providers.google.cloud.utils.field_sanitizer` |
| ============================================================== |
| |
| .. py:module:: airflow.providers.google.cloud.utils.field_sanitizer |
| |
| .. autoapi-nested-parse:: |
| |
| Sanitizer for body fields sent via Google Cloud API. |
| |
| The sanitizer removes fields specified from the body. |
| |
| Context |
| ------- |
| In some cases where Google Cloud operation requires modification of existing resources (such |
| as instances or instance templates) we need to sanitize body of the resources returned |
| via Google Cloud APIs. This is in the case when we retrieve information from Google Cloud first, |
| modify the body and either update the existing resource or create a new one with the |
| modified body. Usually when you retrieve resource from Google Cloud you get some extra fields which |
| are Output-only, and we need to delete those fields if we want to use |
| the body as input for subsequent create/insert type operation. |
| |
| |
| Field specification |
| ------------------- |
| |
| Specification of fields is an array of strings which denote names of fields to be removed. |
| The field can be either direct field name to remove from the body or the full |
| specification of the path you should delete - separated with '.' |
| |
| |
| >>> FIELDS_TO_SANITIZE = [ |
| >>> "kind", |
| >>> "properties.disks.kind", |
| >>> "properties.metadata.kind", |
| >>>] |
| >>> body = { |
| >>> "kind": "compute#instanceTemplate", |
| >>> "name": "instance", |
| >>> "properties": { |
| >>> "disks": [ |
| >>> { |
| >>> "name": "a", |
| >>> "kind": "compute#attachedDisk", |
| >>> "type": "PERSISTENT", |
| >>> "mode": "READ_WRITE", |
| >>> }, |
| >>> { |
| >>> "name": "b", |
| >>> "kind": "compute#attachedDisk", |
| >>> "type": "PERSISTENT", |
| >>> "mode": "READ_WRITE", |
| >>> } |
| >>> ], |
| >>> "metadata": { |
| >>> "kind": "compute#metadata", |
| >>> "fingerprint": "GDPUYxlwHe4=" |
| >>> }, |
| >>> } |
| >>> } |
| >>> sanitizer=GcpBodyFieldSanitizer(FIELDS_TO_SANITIZE) |
| >>> sanitizer.sanitize(body) |
| >>> json.dumps(body, indent=2) |
| { |
| "name": "instance", |
| "properties": { |
| "disks": [ |
| { |
| "name": "a", |
| "type": "PERSISTENT", |
| "mode": "READ_WRITE", |
| }, |
| { |
| "name": "b", |
| "type": "PERSISTENT", |
| "mode": "READ_WRITE", |
| } |
| ], |
| "metadata": { |
| "fingerprint": "GDPUYxlwHe4=" |
| }, |
| } |
| } |
| |
| Note that the components of the path can be either dictionaries or arrays of dictionaries. |
| In case they are dictionaries, subsequent component names key of the field, in case of |
| arrays - the sanitizer iterates through all dictionaries in the array and searches |
| components in all elements of the array. |
| |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.google.cloud.utils.field_sanitizer.GcpBodyFieldSanitizer |
| |
| |
| |
| |
| .. py:exception:: GcpFieldSanitizerException |
| |
| Bases: :py:obj:`airflow.exceptions.AirflowException` |
| |
| Thrown when sanitizer finds unexpected field type in the path |
| (other than dict or array). |
| |
| |
| .. py:class:: GcpBodyFieldSanitizer(sanitize_specs) |
| |
| Bases: :py:obj:`airflow.utils.log.logging_mixin.LoggingMixin` |
| |
| Sanitizes the body according to specification. |
| |
| :param sanitize_specs: array of strings that specifies which fields to remove |
| |
| |
| .. py:method:: sanitize(self, body) |
| |
| Sanitizes the body according to specification. |
| |
| |
| |