blob: aa5d6a45d33ff1da5f72a9f2e0909da57181b283 [file] [log] [blame]
:py:mod:`airflow.providers.google.cloud.example_dags.example_vertex_ai`
=======================================================================
.. py:module:: airflow.providers.google.cloud.example_dags.example_vertex_ai
.. autoapi-nested-parse::
Example Airflow DAG that demonstrates operators for the Google Vertex AI service in the Google
Cloud Platform.
This DAG relies on the following OS environment variables:
* GCP_VERTEX_AI_BUCKET - Google Cloud Storage bucket where the model will be saved
after training process was finished.
* CUSTOM_CONTAINER_URI - path to container with model.
* PYTHON_PACKAGE_GSC_URI - path to test model in archive.
* LOCAL_TRAINING_SCRIPT_PATH - path to local training script.
* DATASET_ID - ID of dataset which will be used in training process.
* MODEL_ID - ID of model which will be used in predict process.
* MODEL_ARTIFACT_URI - The artifact_uri should be the path to a GCS directory containing saved model
artifacts.
Module Contents
---------------
.. py:data:: PROJECT_ID
.. py:data:: REGION
.. py:data:: BUCKET
.. py:data:: STAGING_BUCKET
.. py:data:: DISPLAY_NAME
.. py:data:: CONTAINER_URI
:annotation: = gcr.io/cloud-aiplatform/training/tf-cpu.2-2:latest
.. py:data:: CUSTOM_CONTAINER_URI
.. py:data:: MODEL_SERVING_CONTAINER_URI
:annotation: = gcr.io/cloud-aiplatform/prediction/tf2-cpu.2-2:latest
.. py:data:: REPLICA_COUNT
:annotation: = 1
.. py:data:: MACHINE_TYPE
:annotation: = n1-standard-4
.. py:data:: ACCELERATOR_TYPE
:annotation: = ACCELERATOR_TYPE_UNSPECIFIED
.. py:data:: ACCELERATOR_COUNT
:annotation: = 0
.. py:data:: TRAINING_FRACTION_SPLIT
:annotation: = 0.7
.. py:data:: TEST_FRACTION_SPLIT
:annotation: = 0.15
.. py:data:: VALIDATION_FRACTION_SPLIT
:annotation: = 0.15
.. py:data:: PYTHON_PACKAGE_GCS_URI
.. py:data:: PYTHON_MODULE_NAME
:annotation: = aiplatform_custom_trainer_script.task
.. py:data:: LOCAL_TRAINING_SCRIPT_PATH
.. py:data:: TRAINING_PIPELINE_ID
:annotation: = test-training-pipeline-id
.. py:data:: CUSTOM_JOB_ID
:annotation: = test-custom-job-id
.. py:data:: IMAGE_DATASET
.. py:data:: TABULAR_DATASET
.. py:data:: TEXT_DATASET
.. py:data:: VIDEO_DATASET
.. py:data:: TIME_SERIES_DATASET
.. py:data:: DATASET_ID
.. py:data:: TEST_EXPORT_CONFIG
.. py:data:: TEST_IMPORT_CONFIG
.. py:data:: DATASET_TO_UPDATE
.. py:data:: TEST_UPDATE_MASK
.. py:data:: TEST_TIME_COLUMN
:annotation: = date
.. py:data:: TEST_TIME_SERIES_IDENTIFIER_COLUMN
:annotation: = store_name
.. py:data:: TEST_TARGET_COLUMN
:annotation: = sale_dollars
.. py:data:: COLUMN_SPECS
.. py:data:: COLUMN_TRANSFORMATIONS
.. py:data:: MODEL_ID
.. py:data:: MODEL_ARTIFACT_URI
.. py:data:: MODEL_NAME
.. py:data:: JOB_DISPLAY_NAME
.. py:data:: BIGQUERY_SOURCE
.. py:data:: GCS_DESTINATION_PREFIX
:annotation: = gs://test-vertex-ai-bucket-us/output
.. py:data:: MODEL_PARAMETERS
.. py:data:: ENDPOINT_CONF
.. py:data:: DEPLOYED_MODEL
.. py:data:: MODEL_OUTPUT_CONFIG
.. py:data:: MODEL_OBJ
.. py:data:: create_custom_container_training_job
.. py:data:: create_image_dataset_job
.. py:data:: create_auto_ml_forecasting_training_job
.. py:data:: create_batch_prediction_job
.. py:data:: create_endpoint
.. py:data:: create_hyperparameter_tuning_job
.. py:data:: upload_model