| # Licensed to the Apache Software Foundation (ASF) under one |
| # or more contributor license agreements. See the NOTICE file |
| # distributed with this work for additional information |
| # regarding copyright ownership. The ASF licenses this file |
| # to you under the Apache License, Version 2.0 (the |
| # "License"); you may not use this file except in compliance |
| # with the License. You may obtain a copy of the License at |
| # |
| # http://www.apache.org/licenses/LICENSE-2.0 |
| # |
| # Unless required by applicable law or agreed to in writing, |
| # software distributed under the License is distributed on an |
| # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| # KIND, either express or implied. See the License for the |
| # specific language governing permissions and limitations |
| # under the License. |
| |
| """ |
| Airflow API (Stable) |
| |
| # Overview To facilitate management, Apache Airflow supports a range of REST API endpoints across its objects. This section provides an overview of the API design, methods, and supported use cases. Most of the endpoints accept `JSON` as input and return `JSON` responses. This means that you must usually add the following headers to your request: ``` Content-type: application/json Accept: application/json ``` ## Resources The term `resource` refers to a single type of object in the Airflow metadata. An API is broken up by its endpoint's corresponding resource. The name of a resource is typically plural and expressed in camelCase. Example: `dagRuns`. Resource names are used as part of endpoint URLs, as well as in API parameters and responses. ## CRUD Operations The platform supports **C**reate, **R**ead, **U**pdate, and **D**elete operations on most resources. You can review the standards for these operations and their standard parameters below. Some endpoints have special behavior as exceptions. ### Create To create a resource, you typically submit an HTTP `POST` request with the resource's required metadata in the request body. The response returns a `201 Created` response code upon success with the resource's metadata, including its internal `id`, in the response body. ### Read The HTTP `GET` request can be used to read a resource or to list a number of resources. A resource's `id` can be submitted in the request parameters to read a specific resource. The response usually returns a `200 OK` response code upon success, with the resource's metadata in the response body. If a `GET` request does not include a specific resource `id`, it is treated as a list request. The response usually returns a `200 OK` response code upon success, with an object containing a list of resources' metadata in the response body. When reading resources, some common query parameters are usually available. e.g.: ``` v1/connections?limit=25&offset=25 ``` |Query Parameter|Type|Description| |---------------|----|-----------| |limit|integer|Maximum number of objects to fetch. Usually 25 by default| |offset|integer|Offset after which to start returning objects. For use with limit query parameter.| ### Update Updating a resource requires the resource `id`, and is typically done using an HTTP `PATCH` request, with the fields to modify in the request body. The response usually returns a `200 OK` response code upon success, with information about the modified resource in the response body. ### Delete Deleting a resource requires the resource `id` and is typically executing via an HTTP `DELETE` request. The response usually returns a `204 No Content` response code upon success. ## Conventions - Resource names are plural and expressed in camelCase. - Names are consistent between URL parameter name and field name. - Field names are in snake_case. ```json { \"name\": \"string\", \"slots\": 0, \"occupied_slots\": 0, \"used_slots\": 0, \"queued_slots\": 0, \"open_slots\": 0 } ``` ### Update Mask Update mask is available as a query parameter in patch endpoints. It is used to notify the API which fields you want to update. Using `update_mask` makes it easier to update objects by helping the server know which fields to update in an object instead of updating all fields. The update request ignores any fields that aren't specified in the field mask, leaving them with their current values. Example: ``` resource = request.get('/resource/my-id').json() resource['my_field'] = 'new-value' request.patch('/resource/my-id?update_mask=my_field', data=json.dumps(resource)) ``` ## Versioning and Endpoint Lifecycle - API versioning is not synchronized to specific releases of the Apache Airflow. - APIs are designed to be backward compatible. - Any changes to the API will first go through a deprecation phase. # Summary of Changes | Airflow version | Description | |-|-| | v2.0 | Initial release | # Trying the API You can use a third party client, such as [curl](https://curl.haxx.se/), [HTTPie](https://httpie.org/), [Postman](https://www.postman.com/) or [the Insomnia rest client](https://insomnia.rest/) to test the Apache Airflow API. Note that you will need to pass credentials data. For e.g., here is how to pause a DAG with [curl](https://curl.haxx.se/), when basic authorization is used: ```bash curl -X POST 'https://example.com/api/v1/dags/{dag_id}?update_mask=is_paused' \\ -H 'Content-Type: application/json' \\ --user \"username:password\" \\ -d '{ \"is_paused\": true }' ``` Using a graphical tool such as [Postman](https://www.postman.com/) or [Insomnia](https://insomnia.rest/), it is possible to import the API specifications directly: 1. Download the API specification by clicking the **Download** button at top of this document 2. Import the JSON specification in the graphical tool of your choice. - In *Postman*, you can click the **import** button at the top - With *Insomnia*, you can just drag-and-drop the file on the UI Note that with *Postman*, you can also generate code snippets by selecting a request and clicking on the **Code** button. # Authentication To be able to meet the requirements of many organizations, Airflow supports many authentication methods, and it is even possible to add your own method. If you want to check which auth backend is currently set, you can use `airflow config get-value api auth_backend` command as in the example below. ```bash $ airflow config get-value api auth_backend airflow.api.auth.backend.basic_auth ``` The default is to deny all requests. For details on configuring the authentication, see [API Authorization](https://airflow.apache.org/docs/apache-airflow/stable/security/api.html). # Errors We follow the error response format proposed in [RFC 7807](https://tools.ietf.org/html/rfc7807) also known as Problem Details for HTTP APIs. As with our normal API responses, your client must be prepared to gracefully handle additional members of the response. ## Unauthenticated This indicates that the request has not been applied because it lacks valid authentication credentials for the target resource. Please check that you have valid credentials. ## PermissionDenied This response means that the server understood the request but refuses to authorize it because it lacks sufficient rights to the resource. It happens when you do not have the necessary permission to execute the action you performed. You need to get the appropriate permissions in other to resolve this error. ## BadRequest This response means that the server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing). To resolve this, please ensure that your syntax is correct. ## NotFound This client error response indicates that the server cannot find the requested resource. ## MethodNotAllowed Indicates that the request method is known by the server but is not supported by the target resource. ## NotAcceptable The target resource does not have a current representation that would be acceptable to the user agent, according to the proactive negotiation header fields received in the request, and the server is unwilling to supply a default representation. ## AlreadyExists The request could not be completed due to a conflict with the current state of the target resource, meaning that the resource already exists ## Unknown This means that the server encountered an unexpected condition that prevented it from fulfilling the request. # noqa: E501 |
| |
| The version of the OpenAPI document: 1.0.0 |
| Contact: dev@airflow.apache.org |
| Generated by: https://openapi-generator.tech |
| """ |
| |
| |
| import copy |
| import logging |
| import multiprocessing |
| import sys |
| import urllib3 |
| |
| from http import client as http_client |
| from airflow_client.client.exceptions import ApiValueError |
| |
| |
| JSON_SCHEMA_VALIDATION_KEYWORDS = { |
| 'multipleOf', 'maximum', 'exclusiveMaximum', |
| 'minimum', 'exclusiveMinimum', 'maxLength', |
| 'minLength', 'pattern', 'maxItems', 'minItems' |
| } |
| |
| class Configuration(object): |
| """NOTE: This class is auto generated by OpenAPI Generator |
| |
| Ref: https://openapi-generator.tech |
| Do not edit the class manually. |
| |
| :param host: Base url |
| :param api_key: Dict to store API key(s). |
| Each entry in the dict specifies an API key. |
| The dict key is the name of the security scheme in the OAS specification. |
| The dict value is the API key secret. |
| :param api_key_prefix: Dict to store API prefix (e.g. Bearer) |
| The dict key is the name of the security scheme in the OAS specification. |
| The dict value is an API key prefix when generating the auth data. |
| :param username: Username for HTTP basic authentication |
| :param password: Password for HTTP basic authentication |
| :param discard_unknown_keys: Boolean value indicating whether to discard |
| unknown properties. A server may send a response that includes additional |
| properties that are not known by the client in the following scenarios: |
| 1. The OpenAPI document is incomplete, i.e. it does not match the server |
| implementation. |
| 2. The client was generated using an older version of the OpenAPI document |
| and the server has been upgraded since then. |
| If a schema in the OpenAPI document defines the additionalProperties attribute, |
| then all undeclared properties received by the server are injected into the |
| additional properties map. In that case, there are undeclared properties, and |
| nothing to discard. |
| :param disabled_client_side_validations (string): Comma-separated list of |
| JSON schema validation keywords to disable JSON schema structural validation |
| rules. The following keywords may be specified: multipleOf, maximum, |
| exclusiveMaximum, minimum, exclusiveMinimum, maxLength, minLength, pattern, |
| maxItems, minItems. |
| By default, the validation is performed for data generated locally by the client |
| and data received from the server, independent of any validation performed by |
| the server side. If the input data does not satisfy the JSON schema validation |
| rules specified in the OpenAPI document, an exception is raised. |
| If disabled_client_side_validations is set, structural validation is |
| disabled. This can be useful to troubleshoot data validation problem, such as |
| when the OpenAPI document validation rules do not match the actual API data |
| received by the server. |
| :param server_index: Index to servers configuration. |
| :param server_variables: Mapping with string values to replace variables in |
| templated server configuration. The validation of enums is performed for |
| variables with defined enum values before. |
| :param server_operation_index: Mapping from operation ID to an index to server |
| configuration. |
| :param server_operation_variables: Mapping from operation ID to a mapping with |
| string values to replace variables in templated server configuration. |
| The validation of enums is performed for variables with defined enum values before. |
| :param ssl_ca_cert: str - the path to a file of concatenated CA certificates |
| in PEM format |
| |
| :Example: |
| |
| HTTP Basic Authentication Example. |
| Given the following security scheme in the OpenAPI specification: |
| components: |
| securitySchemes: |
| http_basic_auth: |
| type: http |
| scheme: basic |
| |
| Configure API client with HTTP basic authentication: |
| |
| conf = client.Configuration( |
| username='the-user', |
| password='the-password', |
| ) |
| |
| """ |
| |
| _default = None |
| |
| def __init__(self, host=None, |
| api_key=None, api_key_prefix=None, |
| access_token=None, |
| username=None, password=None, |
| discard_unknown_keys=False, |
| disabled_client_side_validations="", |
| server_index=None, server_variables=None, |
| server_operation_index=None, server_operation_variables=None, |
| ssl_ca_cert=None, |
| ): |
| """Constructor |
| """ |
| self._base_path = "http://localhost/api/v1" if host is None else host |
| """Default Base url |
| """ |
| self.server_index = 0 if server_index is None and host is None else server_index |
| self.server_operation_index = server_operation_index or {} |
| """Default server index |
| """ |
| self.server_variables = server_variables or {} |
| self.server_operation_variables = server_operation_variables or {} |
| """Default server variables |
| """ |
| self.temp_folder_path = None |
| """Temp file folder for downloading files |
| """ |
| # Authentication Settings |
| self.access_token = access_token |
| self.api_key = {} |
| if api_key: |
| self.api_key = api_key |
| """dict to store API key(s) |
| """ |
| self.api_key_prefix = {} |
| if api_key_prefix: |
| self.api_key_prefix = api_key_prefix |
| """dict to store API prefix (e.g. Bearer) |
| """ |
| self.refresh_api_key_hook = None |
| """function hook to refresh API key if expired |
| """ |
| self.username = username |
| """Username for HTTP basic authentication |
| """ |
| self.password = password |
| """Password for HTTP basic authentication |
| """ |
| self.discard_unknown_keys = discard_unknown_keys |
| self.disabled_client_side_validations = disabled_client_side_validations |
| self.logger = {} |
| """Logging Settings |
| """ |
| self.logger["package_logger"] = logging.getLogger("client") |
| self.logger["urllib3_logger"] = logging.getLogger("urllib3") |
| self.logger_format = '%(asctime)s %(levelname)s %(message)s' |
| """Log format |
| """ |
| self.logger_stream_handler = None |
| """Log stream handler |
| """ |
| self.logger_file_handler = None |
| """Log file handler |
| """ |
| self.logger_file = None |
| """Debug file location |
| """ |
| self.debug = False |
| """Debug switch |
| """ |
| |
| self.verify_ssl = True |
| """SSL/TLS verification |
| Set this to false to skip verifying SSL certificate when calling API |
| from https server. |
| """ |
| self.ssl_ca_cert = ssl_ca_cert |
| """Set this to customize the certificate file to verify the peer. |
| """ |
| self.cert_file = None |
| """client certificate file |
| """ |
| self.key_file = None |
| """client key file |
| """ |
| self.assert_hostname = None |
| """Set this to True/False to enable/disable SSL hostname verification. |
| """ |
| |
| self.connection_pool_maxsize = multiprocessing.cpu_count() * 5 |
| """urllib3 connection pool's maximum number of connections saved |
| per pool. urllib3 uses 1 connection as default value, but this is |
| not the best value when you are making a lot of possibly parallel |
| requests to the same host, which is often the case here. |
| cpu_count * 5 is used as default value to increase performance. |
| """ |
| |
| self.proxy = None |
| """Proxy URL |
| """ |
| self.proxy_headers = None |
| """Proxy headers |
| """ |
| self.safe_chars_for_path_param = '' |
| """Safe chars for path_param |
| """ |
| self.retries = None |
| """Adding retries to override urllib3 default value 3 |
| """ |
| # Enable client side validation |
| self.client_side_validation = True |
| |
| # Options to pass down to the underlying urllib3 socket |
| self.socket_options = None |
| |
| def __deepcopy__(self, memo): |
| cls = self.__class__ |
| result = cls.__new__(cls) |
| memo[id(self)] = result |
| for k, v in self.__dict__.items(): |
| if k not in ('logger', 'logger_file_handler'): |
| setattr(result, k, copy.deepcopy(v, memo)) |
| # shallow copy of loggers |
| result.logger = copy.copy(self.logger) |
| # use setters to configure loggers |
| result.logger_file = self.logger_file |
| result.debug = self.debug |
| return result |
| |
| def __setattr__(self, name, value): |
| object.__setattr__(self, name, value) |
| if name == 'disabled_client_side_validations': |
| s = set(filter(None, value.split(','))) |
| for v in s: |
| if v not in JSON_SCHEMA_VALIDATION_KEYWORDS: |
| raise ApiValueError( |
| "Invalid keyword: '{0}''".format(v)) |
| self._disabled_client_side_validations = s |
| |
| @classmethod |
| def set_default(cls, default): |
| """Set default instance of configuration. |
| |
| It stores default configuration, which can be |
| returned by get_default_copy method. |
| |
| :param default: object of Configuration |
| """ |
| cls._default = copy.deepcopy(default) |
| |
| @classmethod |
| def get_default_copy(cls): |
| """Return new instance of configuration. |
| |
| This method returns newly created, based on default constructor, |
| object of Configuration class or returns a copy of default |
| configuration passed by the set_default method. |
| |
| :return: The configuration object. |
| """ |
| if cls._default is not None: |
| return copy.deepcopy(cls._default) |
| return Configuration() |
| |
| @property |
| def logger_file(self): |
| """The logger file. |
| |
| If the logger_file is None, then add stream handler and remove file |
| handler. Otherwise, add file handler and remove stream handler. |
| |
| :param value: The logger_file path. |
| :type: str |
| """ |
| return self.__logger_file |
| |
| @logger_file.setter |
| def logger_file(self, value): |
| """The logger file. |
| |
| If the logger_file is None, then add stream handler and remove file |
| handler. Otherwise, add file handler and remove stream handler. |
| |
| :param value: The logger_file path. |
| :type: str |
| """ |
| self.__logger_file = value |
| if self.__logger_file: |
| # If set logging file, |
| # then add file handler and remove stream handler. |
| self.logger_file_handler = logging.FileHandler(self.__logger_file) |
| self.logger_file_handler.setFormatter(self.logger_formatter) |
| for _, logger in self.logger.items(): |
| logger.addHandler(self.logger_file_handler) |
| |
| @property |
| def debug(self): |
| """Debug status |
| |
| :param value: The debug status, True or False. |
| :type: bool |
| """ |
| return self.__debug |
| |
| @debug.setter |
| def debug(self, value): |
| """Debug status |
| |
| :param value: The debug status, True or False. |
| :type: bool |
| """ |
| self.__debug = value |
| if self.__debug: |
| # if debug status is True, turn on debug logging |
| for _, logger in self.logger.items(): |
| logger.setLevel(logging.DEBUG) |
| # turn on http_client debug |
| http_client.HTTPConnection.debuglevel = 1 |
| else: |
| # if debug status is False, turn off debug logging, |
| # setting log level to default `logging.WARNING` |
| for _, logger in self.logger.items(): |
| logger.setLevel(logging.WARNING) |
| # turn off http_client debug |
| http_client.HTTPConnection.debuglevel = 0 |
| |
| @property |
| def logger_format(self): |
| """The logger format. |
| |
| The logger_formatter will be updated when sets logger_format. |
| |
| :param value: The format string. |
| :type: str |
| """ |
| return self.__logger_format |
| |
| @logger_format.setter |
| def logger_format(self, value): |
| """The logger format. |
| |
| The logger_formatter will be updated when sets logger_format. |
| |
| :param value: The format string. |
| :type: str |
| """ |
| self.__logger_format = value |
| self.logger_formatter = logging.Formatter(self.__logger_format) |
| |
| def get_api_key_with_prefix(self, identifier, alias=None): |
| """Gets API key (with prefix if set). |
| |
| :param identifier: The identifier of apiKey. |
| :param alias: The alternative identifier of apiKey. |
| :return: The token for api key authentication. |
| """ |
| if self.refresh_api_key_hook is not None: |
| self.refresh_api_key_hook(self) |
| key = self.api_key.get(identifier, self.api_key.get(alias) if alias is not None else None) |
| if key: |
| prefix = self.api_key_prefix.get(identifier) |
| if prefix: |
| return "%s %s" % (prefix, key) |
| else: |
| return key |
| |
| def get_basic_auth_token(self): |
| """Gets HTTP basic authentication header (string). |
| |
| :return: The token for basic HTTP authentication. |
| """ |
| username = "" |
| if self.username is not None: |
| username = self.username |
| password = "" |
| if self.password is not None: |
| password = self.password |
| return urllib3.util.make_headers( |
| basic_auth=username + ':' + password |
| ).get('authorization') |
| |
| def auth_settings(self): |
| """Gets Auth Settings dict for api client. |
| |
| :return: The Auth Settings information dict. |
| """ |
| auth = {} |
| if self.username is not None and self.password is not None: |
| auth['Basic'] = { |
| 'type': 'basic', |
| 'in': 'header', |
| 'key': 'Authorization', |
| 'value': self.get_basic_auth_token() |
| } |
| return auth |
| |
| def to_debug_report(self): |
| """Gets the essential information for debugging. |
| |
| :return: The report for debugging. |
| """ |
| return "Python SDK Debug Report:\n"\ |
| "OS: {env}\n"\ |
| "Python Version: {pyversion}\n"\ |
| "Version of the API: 1.0.0\n"\ |
| "SDK Package Version: 2.0.0".\ |
| format(env=sys.platform, pyversion=sys.version) |
| |
| def get_host_settings(self): |
| """Gets an array of host settings |
| |
| :return: An array of host settings |
| """ |
| return [ |
| { |
| 'url': "/api/v1", |
| 'description': "Apache Airflow Stable API.", |
| } |
| ] |
| |
| def get_host_from_settings(self, index, variables=None, servers=None): |
| """Gets host URL based on the index and variables |
| :param index: array index of the host settings |
| :param variables: hash of variable and the corresponding value |
| :param servers: an array of host settings or None |
| :return: URL based on host settings |
| """ |
| if index is None: |
| return self._base_path |
| |
| variables = {} if variables is None else variables |
| servers = self.get_host_settings() if servers is None else servers |
| |
| try: |
| server = servers[index] |
| except IndexError: |
| raise ValueError( |
| "Invalid index {0} when selecting the host settings. " |
| "Must be less than {1}".format(index, len(servers))) |
| |
| url = server['url'] |
| |
| # go through variables and replace placeholders |
| for variable_name, variable in server.get('variables', {}).items(): |
| used_value = variables.get( |
| variable_name, variable['default_value']) |
| |
| if 'enum_values' in variable \ |
| and used_value not in variable['enum_values']: |
| raise ValueError( |
| "The variable `{0}` in the host URL has invalid value " |
| "{1}. Must be {2}.".format( |
| variable_name, variables[variable_name], |
| variable['enum_values'])) |
| |
| url = url.replace("{" + variable_name + "}", used_value) |
| |
| return url |
| |
| @property |
| def host(self): |
| """Return generated host.""" |
| return self.get_host_from_settings(self.server_index, variables=self.server_variables) |
| |
| @host.setter |
| def host(self, value): |
| """Fix base path.""" |
| self._base_path = value |
| self.server_index = None |
