| Importing and Exporting Datasources |
| =================================== |
| |
| The superset cli allows you to import and export datasources from and to YAML. |
| Datasources include both databases and druid clusters. The data is expected to be organized in the following hierarchy: :: |
| |
| . |
| ├──databases |
| | ├──database_1 |
| | | ├──table_1 |
| | | | ├──columns |
| | | | | ├──column_1 |
| | | | | ├──column_2 |
| | | | | └──... (more columns) |
| | | | └──metrics |
| | | | ├──metric_1 |
| | | | ├──metric_2 |
| | | | └──... (more metrics) |
| | | └── ... (more tables) |
| | └── ... (more databases) |
| └──druid_clusters |
| ├──cluster_1 |
| | ├──datasource_1 |
| | | ├──columns |
| | | | ├──column_1 |
| | | | ├──column_2 |
| | | | └──... (more columns) |
| | | └──metrics |
| | | ├──metric_1 |
| | | ├──metric_2 |
| | | └──... (more metrics) |
| | └── ... (more datasources) |
| └── ... (more clusters) |
| |
| |
| Exporting Datasources to YAML |
| ----------------------------- |
| You can print your current datasources to stdout by running: :: |
| |
| superset export_datasources |
| |
| |
| To save your datasources to a file run: :: |
| |
| superset export_datasources -f <filename> |
| |
| |
| By default, default (null) values will be omitted. Use the ``-d`` flag to include them. |
| If you want back references to be included (e.g. a column to include the table id |
| it belongs to) use the ``-b`` flag. |
| |
| Alternatively you can export datasources using the UI: :: |
| |
| 1. Open **Sources** -> **Databases** to export all tables associated to a single or multiple databases. (**Tables** for one or more tables, **Druid Clusters** for clusters, **Druid Datasources** for datasources) |
| 2. Select the items you would like to export |
| 3. Click **Actions** -> **Export to YAML** |
| 4. If you want to import an item that you exported through the UI, you will need to nest it inside its parent element, e.g. a `database` needs to be nested under `databases` a `table` needs to be nested inside a `database` element. |
| |
| Exporting the complete supported YAML schema |
| -------------------------------------------- |
| In order to obtain an exhaustive list of all fields you can import using the YAML import run: :: |
| |
| superset export_datasource_schema |
| |
| Again, you can use the ``-b`` flag to include back references. |
| |
| |
| Importing Datasources from YAML |
| ------------------------------- |
| In order to import datasources from a YAML file(s), run: :: |
| |
| superset import_datasources -p <path or filename> |
| |
| If you supply a path all files ending with ``*.yaml`` or ``*.yml`` will be parsed. |
| You can apply additional flags e.g.: :: |
| |
| superset import_datasources -p <path> -r |
| |
| Will search the supplied path recursively. |
| |
| The sync flag ``-s`` takes parameters in order to sync the supplied elements with |
| your file. Be careful this can delete the contents of your meta database. Example: |
| |
| superset import_datasources -p <path / filename> -s columns,metrics |
| |
| This will sync all ``metrics`` and ``columns`` for all datasources found in the |
| ``<path / filename>`` in the Superset meta database. This means columns and metrics |
| not specified in YAML will be deleted. If you would add ``tables`` to ``columns,metrics`` |
| those would be synchronised as well. |
| |
| |
| If you don't supply the sync flag (``-s``) importing will only add and update (override) fields. |
| E.g. you can add a ``verbose_name`` to the the column ``ds`` in the table ``random_time_series`` from the example datasets |
| by saving the following YAML to file and then running the ``import_datasources`` command. :: |
| |
| databases: |
| - database_name: main |
| tables: |
| - table_name: random_time_series |
| columns: |
| - column_name: ds |
| verbose_name: datetime |
| |