| :mod:`airflow.contrib.hooks.gcp_mlengine_hook` |
| ============================================== |
| |
| .. py:module:: airflow.contrib.hooks.gcp_mlengine_hook |
| |
| |
| Module Contents |
| --------------- |
| |
| .. data:: log |
| |
| |
| |
| |
| .. function:: _poll_with_exponential_delay(request, max_n, is_done_func, is_error_func) |
| |
| .. py:class:: MLEngineHook(gcp_conn_id='google_cloud_default', delegate_to=None) |
| |
| Bases: :class:`airflow.contrib.hooks.gcp_api_base_hook.GoogleCloudBaseHook` |
| |
| |
| .. method:: get_conn(self) |
| |
| Returns a Google MLEngine service object. |
| |
| |
| |
| |
| .. method:: create_job(self, project_id, job, use_existing_job_fn=None) |
| |
| Launches a MLEngine job and wait for it to reach a terminal state. |
| |
| :param project_id: The Google Cloud project id within which MLEngine |
| job will be launched. |
| :type project_id: str |
| |
| :param job: MLEngine Job object that should be provided to the MLEngine |
| API, such as: :: |
| |
| { |
| 'jobId': 'my_job_id', |
| 'trainingInput': { |
| 'scaleTier': 'STANDARD_1', |
| ... |
| } |
| } |
| |
| :type job: dict |
| |
| :param use_existing_job_fn: In case that a MLEngine job with the same |
| job_id already exist, this method (if provided) will decide whether |
| we should use this existing job, continue waiting for it to finish |
| and returning the job object. It should accepts a MLEngine job |
| object, and returns a boolean value indicating whether it is OK to |
| reuse the existing job. If 'use_existing_job_fn' is not provided, |
| we by default reuse the existing MLEngine job. |
| :type use_existing_job_fn: function |
| |
| :return: The MLEngine job object if the job successfully reach a |
| terminal state (which might be FAILED or CANCELLED state). |
| :rtype: dict |
| |
| |
| |
| |
| .. method:: _get_job(self, project_id, job_id) |
| |
| Gets a MLEngine job based on the job name. |
| |
| :return: MLEngine job object if succeed. |
| :rtype: dict |
| |
| Raises: |
| googleapiclient.errors.HttpError: if HTTP error is returned from server |
| |
| |
| |
| |
| .. method:: _wait_for_job_done(self, project_id, job_id, interval=30) |
| |
| Waits for the Job to reach a terminal state. |
| |
| This method will periodically check the job state until the job reach |
| a terminal state. |
| |
| Raises: |
| googleapiclient.errors.HttpError: if HTTP error is returned when getting |
| the job |
| |
| |
| |
| |
| .. method:: create_version(self, project_id, model_name, version_spec) |
| |
| Creates the Version on Google Cloud ML Engine. |
| |
| Returns the operation if the version was created successfully and |
| raises an error otherwise. |
| |
| |
| |
| |
| .. method:: set_default_version(self, project_id, model_name, version_name) |
| |
| Sets a version to be the default. Blocks until finished. |
| |
| |
| |
| |
| .. method:: list_versions(self, project_id, model_name) |
| |
| Lists all available versions of a model. Blocks until finished. |
| |
| |
| |
| |
| .. method:: delete_version(self, project_id, model_name, version_name) |
| |
| Deletes the given version of a model. Blocks until finished. |
| |
| |
| |
| |
| .. method:: create_model(self, project_id, model) |
| |
| Create a Model. Blocks until finished. |
| |
| |
| |
| |
| .. method:: get_model(self, project_id, model_name) |
| |
| Gets a Model. Blocks until finished. |
| |
| |
| |
| |