| :py:mod:`airflow.providers.google.cloud.hooks.mlengine` |
| ======================================================= |
| |
| .. py:module:: airflow.providers.google.cloud.hooks.mlengine |
| |
| .. autoapi-nested-parse:: |
| |
| This module contains a Google ML Engine Hook. |
| |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.google.cloud.hooks.mlengine.MLEngineHook |
| |
| |
| |
| |
| Attributes |
| ~~~~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.google.cloud.hooks.mlengine.log |
| |
| |
| .. py:data:: log |
| |
| |
| |
| |
| .. py:class:: MLEngineHook(gcp_conn_id = 'google_cloud_default', delegate_to = None, impersonation_chain = None) |
| |
| Bases: :py:obj:`airflow.providers.google.common.hooks.base_google.GoogleBaseHook` |
| |
| Hook for Google ML Engine APIs. |
| |
| All the methods in the hook where project_id is used must be called with |
| keyword arguments rather than positional. |
| |
| .. py:method:: get_conn(self) |
| |
| Retrieves the connection to MLEngine. |
| |
| :return: Google MLEngine services object. |
| |
| |
| .. py:method:: create_job(self, job, project_id, use_existing_job_fn = None) |
| |
| Launches a MLEngine job and wait for it to reach a terminal state. |
| |
| :param project_id: The Google Cloud project id within which MLEngine |
| job will be launched. If set to None or missing, the default project_id from the Google Cloud |
| connection is used. |
| :param job: MLEngine Job object that should be provided to the MLEngine |
| API, such as: :: |
| |
| { |
| 'jobId': 'my_job_id', |
| 'trainingInput': { |
| 'scaleTier': 'STANDARD_1', |
| ... |
| } |
| } |
| |
| :param use_existing_job_fn: In case that a MLEngine job with the same |
| job_id already exist, this method (if provided) will decide whether |
| we should use this existing job, continue waiting for it to finish |
| and returning the job object. It should accepts a MLEngine job |
| object, and returns a boolean value indicating whether it is OK to |
| reuse the existing job. If 'use_existing_job_fn' is not provided, |
| we by default reuse the existing MLEngine job. |
| :return: The MLEngine job object if the job successfully reach a |
| terminal state (which might be FAILED or CANCELLED state). |
| :rtype: dict |
| |
| |
| .. py:method:: cancel_job(self, job_id, project_id) |
| |
| Cancels a MLEngine job. |
| |
| :param project_id: The Google Cloud project id within which MLEngine |
| job will be cancelled. If set to None or missing, the default project_id from the Google Cloud |
| connection is used. |
| :param job_id: A unique id for the want-to-be cancelled Google MLEngine training job. |
| |
| :return: Empty dict if cancelled successfully |
| :rtype: dict |
| :raises: googleapiclient.errors.HttpError |
| |
| |
| .. py:method:: create_version(self, model_name, version_spec, project_id) |
| |
| Creates the Version on Google Cloud ML Engine. |
| |
| :param version_spec: A dictionary containing the information about the version. (templated) |
| :param model_name: The name of the Google Cloud ML Engine model that the version belongs to. |
| (templated) |
| :param project_id: The Google Cloud project name to which MLEngine model belongs. |
| If set to None or missing, the default project_id from the Google Cloud connection is used. |
| (templated) |
| :return: If the version was created successfully, returns the operation. |
| Otherwise raises an error . |
| :rtype: dict |
| |
| |
| .. py:method:: set_default_version(self, model_name, version_name, project_id) |
| |
| Sets a version to be the default. Blocks until finished. |
| |
| :param model_name: The name of the Google Cloud ML Engine model that the version belongs to. |
| (templated) |
| :param version_name: A name to use for the version being operated upon. (templated) |
| :param project_id: The Google Cloud project name to which MLEngine model belongs. If set to None |
| or missing, the default project_id from the Google Cloud connection is used. (templated) |
| :return: If successful, return an instance of Version. |
| Otherwise raises an error. |
| :rtype: dict |
| :raises: googleapiclient.errors.HttpError |
| |
| |
| .. py:method:: list_versions(self, model_name, project_id) |
| |
| Lists all available versions of a model. Blocks until finished. |
| |
| :param model_name: The name of the Google Cloud ML Engine model that the version |
| belongs to. (templated) |
| :param project_id: The Google Cloud project name to which MLEngine model belongs. If set to None or |
| missing, the default project_id from the Google Cloud connection is used. (templated) |
| :return: return an list of instance of Version. |
| :rtype: List[Dict] |
| :raises: googleapiclient.errors.HttpError |
| |
| |
| .. py:method:: delete_version(self, model_name, version_name, project_id) |
| |
| Deletes the given version of a model. Blocks until finished. |
| |
| :param model_name: The name of the Google Cloud ML Engine model that the version |
| belongs to. (templated) |
| :param project_id: The Google Cloud project name to which MLEngine |
| model belongs. |
| :return: If the version was deleted successfully, returns the operation. |
| Otherwise raises an error. |
| :rtype: Dict |
| |
| |
| .. py:method:: create_model(self, model, project_id) |
| |
| Create a Model. Blocks until finished. |
| |
| :param model: A dictionary containing the information about the model. |
| :param project_id: The Google Cloud project name to which MLEngine model belongs. If set to None or |
| missing, the default project_id from the Google Cloud connection is used. (templated) |
| :return: If the version was created successfully, returns the instance of Model. |
| Otherwise raises an error. |
| :rtype: Dict |
| :raises: googleapiclient.errors.HttpError |
| |
| |
| .. py:method:: get_model(self, model_name, project_id) |
| |
| Gets a Model. Blocks until finished. |
| |
| :param model_name: The name of the model. |
| :param project_id: The Google Cloud project name to which MLEngine model belongs. If set to None |
| or missing, the default project_id from the Google Cloud connection is used. (templated) |
| :return: If the model exists, returns the instance of Model. |
| Otherwise return None. |
| :rtype: Dict |
| :raises: googleapiclient.errors.HttpError |
| |
| |
| .. py:method:: delete_model(self, model_name, project_id, delete_contents = False) |
| |
| Delete a Model. Blocks until finished. |
| |
| :param model_name: The name of the model. |
| :param delete_contents: Whether to force the deletion even if the models is not empty. |
| Will delete all version (if any) in the dataset if set to True. |
| The default value is False. |
| :param project_id: The Google Cloud project name to which MLEngine model belongs. If set to None |
| or missing, the default project_id from the Google Cloud connection is used. (templated) |
| :raises: googleapiclient.errors.HttpError |
| |
| |
| |