| :mod:`airflow.contrib.operators.bigquery_get_data` |
| ================================================== |
| |
| .. py:module:: airflow.contrib.operators.bigquery_get_data |
| |
| |
| Module Contents |
| --------------- |
| |
| .. py:class:: BigQueryGetDataOperator(dataset_id, table_id, max_results=100, selected_fields=None, bigquery_conn_id='bigquery_default', delegate_to=None, *args, **kwargs) |
| |
| Bases: :class:`airflow.models.BaseOperator` |
| |
| Fetches the data from a BigQuery table (alternatively fetch data for selected columns) |
| and returns data in a python list. The number of elements in the returned list will |
| be equal to the number of rows fetched. Each element in the list will again be a list |
| where element would represent the columns values for that row. |
| |
| **Example Result**: ``[['Tony', '10'], ['Mike', '20'], ['Steve', '15']]`` |
| |
| .. note:: |
| If you pass fields to ``selected_fields`` which are in different order than the |
| order of columns already in |
| BQ table, the data will still be in the order of BQ table. |
| For example if the BQ table has 3 columns as |
| ``[A,B,C]`` and you pass 'B,A' in the ``selected_fields`` |
| the data would still be of the form ``'A,B'``. |
| |
| **Example**: :: |
| |
| get_data = BigQueryGetDataOperator( |
| task_id='get_data_from_bq', |
| dataset_id='test_dataset', |
| table_id='Transaction_partitions', |
| max_results=100, |
| selected_fields='DATE', |
| bigquery_conn_id='airflow-service-account' |
| ) |
| |
| :param dataset_id: The dataset ID of the requested table. (templated) |
| :type dataset_id: str |
| :param table_id: The table ID of the requested table. (templated) |
| :type table_id: str |
| :param max_results: The maximum number of records (rows) to be fetched |
| from the table. (templated) |
| :type max_results: int |
| :param selected_fields: List of fields to return (comma-separated). If |
| unspecified, all fields are returned. |
| :type selected_fields: str |
| :param bigquery_conn_id: reference to a specific BigQuery hook. |
| :type bigquery_conn_id: str |
| :param delegate_to: The account to impersonate, if any. |
| For this to work, the service account making the request must have domain-wide |
| delegation enabled. |
| :type delegate_to: str |
| |
| .. attribute:: template_fields |
| :annotation: = ['dataset_id', 'table_id', 'max_results'] |
| |
| |
| |
| .. attribute:: ui_color |
| :annotation: = #e4f0e8 |
| |
| |
| |
| |
| .. method:: execute(self, context) |
| |
| |
| |
| |