blob: 8c14ca0c002bec3f327f1c468ae80afcd71369d3 [file] [log] [blame]
:mod:`airflow.contrib.hooks.gcp_mlengine_hook`
==============================================
.. py:module:: airflow.contrib.hooks.gcp_mlengine_hook
Module Contents
---------------
.. data:: log
.. function:: _poll_with_exponential_delay(request, max_n, is_done_func, is_error_func)
.. py:class:: MLEngineHook(gcp_conn_id='google_cloud_default', delegate_to=None)
Bases: :class:`airflow.contrib.hooks.gcp_api_base_hook.GoogleCloudBaseHook`
.. method:: get_conn(self)
Returns a Google MLEngine service object.
.. method:: create_job(self, project_id, job, use_existing_job_fn=None)
Launches a MLEngine job and wait for it to reach a terminal state.
:param project_id: The Google Cloud project id within which MLEngine
job will be launched.
:type project_id: str
:param job: MLEngine Job object that should be provided to the MLEngine
API, such as: ::
{
'jobId': 'my_job_id',
'trainingInput': {
'scaleTier': 'STANDARD_1',
...
}
}
:type job: dict
:param use_existing_job_fn: In case that a MLEngine job with the same
job_id already exist, this method (if provided) will decide whether
we should use this existing job, continue waiting for it to finish
and returning the job object. It should accepts a MLEngine job
object, and returns a boolean value indicating whether it is OK to
reuse the existing job. If 'use_existing_job_fn' is not provided,
we by default reuse the existing MLEngine job.
:type use_existing_job_fn: function
:return: The MLEngine job object if the job successfully reach a
terminal state (which might be FAILED or CANCELLED state).
:rtype: dict
.. method:: _get_job(self, project_id, job_id)
Gets a MLEngine job based on the job name.
:return: MLEngine job object if succeed.
:rtype: dict
Raises:
googleapiclient.errors.HttpError: if HTTP error is returned from server
.. method:: _wait_for_job_done(self, project_id, job_id, interval=30)
Waits for the Job to reach a terminal state.
This method will periodically check the job state until the job reach
a terminal state.
Raises:
googleapiclient.errors.HttpError: if HTTP error is returned when getting
the job
.. method:: create_version(self, project_id, model_name, version_spec)
Creates the Version on Google Cloud ML Engine.
Returns the operation if the version was created successfully and
raises an error otherwise.
.. method:: set_default_version(self, project_id, model_name, version_name)
Sets a version to be the default. Blocks until finished.
.. method:: list_versions(self, project_id, model_name)
Lists all available versions of a model. Blocks until finished.
.. method:: delete_version(self, project_id, model_name, version_name)
Deletes the given version of a model. Blocks until finished.
.. method:: create_model(self, project_id, model)
Create a Model. Blocks until finished.
.. method:: get_model(self, project_id, model_name)
Gets a Model. Blocks until finished.