| .. Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| .. http://www.apache.org/licenses/LICENSE-2.0 |
| |
| .. Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| |
| Reference for Database Migrations |
| ''''''''''''''''''''''''''''''''' |
| |
| Here's the list of all the Database Migrations that are executed via when you run ``airflow db migrate``. |
| |
| .. warning:: |
| |
| Those migration details are mostly used here to make the users aware when and what kind of migrations |
| will be executed during migrations between specific Airflow versions. The intention here is that the |
| "DB conscious" users might perform an analysis on the migrations and draw conclusions about the impact |
| of the migrations on their Airflow database. Those users might also want to take a look at the |
| :doc:`database-erd-ref` document to understand how the internal DB of Airflow structure looks like. |
| However, you should be aware that the structure is internal and you should not access the DB directly |
| to retrieve or modify any data - you should use the :doc:`REST API <stable-rest-api-ref>` to do that instead. |
| |
| |
| |
| .. This table is automatically updated by prek hook by ``scripts/ci/prek/migration_reference.py`` |
| .. All table elements are scraped from migration files |
| .. Beginning of auto-generated table |
| |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | Revision ID | Revises ID | Airflow Version | Description | |
| +=========================+==================+===================+==============================================================+ |
| | ``cc92b33c6709`` (head) | ``eaf332f43c7c`` | ``3.1.0`` | Add backward compatibility for serialized DAG format v3 to | |
| | | | | v2. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``eaf332f43c7c`` | ``a3c7f2b18d4e`` | ``3.1.0`` | add last_parse_duration to dag model. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``a3c7f2b18d4e`` | ``7582ea3f3dd5`` | ``3.1.0`` | Add tables to store teams and associations with dag bundles. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``7582ea3f3dd5`` | ``a169942745c2`` | ``3.1.0`` | Make bundle_name not nullable. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``a169942745c2`` | ``808787349f22`` | ``3.1.0`` | Remove dag_id from Deadline. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``808787349f22`` | ``3bda03debd04`` | ``3.1.0`` | Modify deadline's callback schema. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``3bda03debd04`` | ``f56f68b9e02f`` | ``3.1.0`` | Add url template and template params to DagBundleModel. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``f56f68b9e02f`` | ``09fa89ba1710`` | ``3.1.0`` | Add callback_state to deadline. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``09fa89ba1710`` | ``40f7c30a228b`` | ``3.1.0`` | Add trigger_id to deadline. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``40f7c30a228b`` | ``ffdb0566c7c0`` | ``3.1.0`` | Add Human In the Loop Detail table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``ffdb0566c7c0`` | ``66a7743fe20e`` | ``3.1.0`` | Add dag_favorite table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``66a7743fe20e`` | ``583e80dfcef4`` | ``3.1.0`` | Add triggering user to dag_run. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``583e80dfcef4`` | ``3ac9e5732b1f`` | ``3.1.0`` | Add task_inlet_asset_reference table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``3ac9e5732b1f`` | ``0242ac120002`` | ``3.1.0`` | Change the on-delete behaviour of | |
| | | | | task_instance.dag_version_id foreign key constraint to | |
| | | | | RESTRICT. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``0242ac120002`` | ``dfee8bd5d574`` | ``3.1.0`` | Rename Deadline column in the Deadline table from deadline | |
| | | | | to deadline_time and change its type from DateTime to UTC | |
| | | | | DateTime. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``dfee8bd5d574`` | ``fe199e1abd77`` | ``3.1.0`` | Add Deadline to Dag. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``fe199e1abd77`` | ``29ce7909c52b`` | ``3.0.3`` | Delete import errors. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``29ce7909c52b`` | ``959e216a3abb`` | ``3.0.0`` | Change TI table to have unique UUID id/pk per attempt. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``959e216a3abb`` | ``0e9519b56710`` | ``3.0.0`` | Rename ``is_active`` to ``is_stale`` column in ``dag`` | |
| | | | | table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``0e9519b56710`` | ``ec62e120484d`` | ``3.0.0`` | Rename run_type from 'dataset_triggered' to | |
| | | | | 'asset_triggered' in dag_run table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``ec62e120484d`` | ``be2cc2f742cf`` | ``3.0.0`` | Add new otel span fields. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``be2cc2f742cf`` | ``d469d27e2a64`` | ``3.0.0`` | Support bundles in DagPriorityParsingRequest. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``d469d27e2a64`` | ``16f7f5ee874e`` | ``3.0.0`` | Use ti_id as FK to TaskReschedule. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``16f7f5ee874e`` | ``cf87489a35df`` | ``3.0.0`` | Remove dag.default_view column. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``cf87489a35df`` | ``7645189f3479`` | ``3.0.0`` | Use TI.id as primary key to TaskInstanceNote. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``7645189f3479`` | ``e00344393f31`` | ``3.0.0`` | Add try_id to TI and TIH. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``e00344393f31`` | ``6a9e7a527a88`` | ``3.0.0`` | remove external_trigger field. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``6a9e7a527a88`` | ``33b04e4bfa19`` | ``3.0.0`` | Add DagRun run_after. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``33b04e4bfa19`` | ``8ea135928435`` | ``3.0.0`` | add new task_instance field scheduled_dttm. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``8ea135928435`` | ``e39a26ac59f6`` | ``3.0.0`` | Add relative fileloc column. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``e39a26ac59f6`` | ``38770795785f`` | ``3.0.0`` | remove pickled data from dagrun table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``38770795785f`` | ``5c9c0231baa2`` | ``3.0.0`` | Add asset reference models. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``5c9c0231baa2`` | ``237cef8dfea1`` | ``3.0.0`` | Remove processor_subdir. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``237cef8dfea1`` | ``038dc8bc6284`` | ``3.0.0`` | Add deadline alerts table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``038dc8bc6284`` | ``e229247a6cb1`` | ``3.0.0`` | update trigger_timeout column in task_instance table to UTC. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``e229247a6cb1`` | ``eed27faa34e3`` | ``3.0.0`` | Add DagBundleModel. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``eed27faa34e3`` | ``9fc3fc5de720`` | ``3.0.0`` | Remove pickled data from xcom table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``9fc3fc5de720`` | ``2b47dc6bc8df`` | ``3.0.0`` | Add references between assets and triggers. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``2b47dc6bc8df`` | ``d03e4a635aa3`` | ``3.0.0`` | add dag versioning. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``d03e4a635aa3`` | ``d8cd3297971e`` | ``3.0.0`` | Drop DAG pickling. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``d8cd3297971e`` | ``5f57a45b8433`` | ``3.0.0`` | Add last_heartbeat_at directly to TI. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``5f57a45b8433`` | ``486ac7936b78`` | ``3.0.0`` | Drop task_fail table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``486ac7936b78`` | ``d59cbbef95eb`` | ``3.0.0`` | remove scheduler_lock column. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``d59cbbef95eb`` | ``05234396c6fc`` | ``3.0.0`` | Add UUID primary key to ``task_instance`` table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``05234396c6fc`` | ``3a8972ecb8f9`` | ``3.0.0`` | Rename dataset as asset. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``3a8972ecb8f9`` | ``fb2d4922cd79`` | ``3.0.0`` | Add exception_reason and logical_date to BackfillDagRun. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``fb2d4922cd79`` | ``5a5d66100783`` | ``3.0.0`` | Tweak AssetAliasModel to match AssetModel after AIP-76. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``5a5d66100783`` | ``c3389cd7793f`` | ``3.0.0`` | Add AssetActive to track orphaning instead of a flag. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``c3389cd7793f`` | ``0d9e73a75ee4`` | ``3.0.0`` | Add backfill to dag run model. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``0d9e73a75ee4`` | ``44eabb1904b4`` | ``3.0.0`` | Add name and group fields to DatasetModel. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``44eabb1904b4`` | ``16cbcb1c8c36`` | ``3.0.0`` | Update dag_run_note.user_id and task_instance_note.user_id | |
| | | | | columns to String. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``16cbcb1c8c36`` | ``522625f6d606`` | ``3.0.0`` | Remove redundant index. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``522625f6d606`` | ``1cdc775ca98f`` | ``3.0.0`` | Add tables for backfill. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``1cdc775ca98f`` | ``a2c32e6c7729`` | ``3.0.0`` | Rename execution_date to logical_date. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``a2c32e6c7729`` | ``0bfc26bc256e`` | ``3.0.0`` | Add triggered_by field to DagRun. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``0bfc26bc256e`` | ``d0f1c55954fa`` | ``3.0.0`` | Rename DagModel schedule_interval to timetable_summary. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``d0f1c55954fa`` | ``044f740568ec`` | ``3.0.0`` | Remove SubDAGs: ``is_subdag`` & ``root_dag_id`` columns from | |
| | | | | DAG table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``044f740568ec`` | ``5f2621c13b39`` | ``3.0.0`` | Drop ab_user.id foreign key. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``5f2621c13b39`` | ``22ed7efa9da2`` | ``2.10.3`` | Rename dag_schedule_dataset_alias_reference constraint | |
| | | | | names. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``22ed7efa9da2`` | ``8684e37832e6`` | ``2.10.0`` | Add dag_schedule_dataset_alias_reference table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``8684e37832e6`` | ``41b3bc7c0272`` | ``2.10.0`` | Add dataset_alias_dataset association table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``41b3bc7c0272`` | ``ec3471c1e067`` | ``2.10.0`` | Add try_number to audit log. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``ec3471c1e067`` | ``05e19f3176be`` | ``2.10.0`` | Add dataset_alias_dataset_event. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``05e19f3176be`` | ``d482b7261ff9`` | ``2.10.0`` | Add dataset_alias. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``d482b7261ff9`` | ``c4602ba06b4b`` | ``2.10.0`` | Add task_instance_history. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``c4602ba06b4b`` | ``677fdbb7fc54`` | ``2.10.0`` | Added DagPriorityParsingRequest table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``677fdbb7fc54`` | ``0fd0c178cbe8`` | ``2.10.0`` | add new executor field to db. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``0fd0c178cbe8`` | ``686269002441`` | ``2.10.0`` | Add indexes on dag_id column in referencing tables. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``686269002441`` | ``bff083ad727d`` | ``2.9.2`` | Fix inconsistency between ORM and migration files. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``bff083ad727d`` | ``1949afb29106`` | ``2.9.2`` | Remove ``idx_last_scheduling_decision`` index on | |
| | | | | last_scheduling_decision in dag_run table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``1949afb29106`` | ``ee1467d4aa35`` | ``2.9.0`` | update trigger kwargs type and encrypt. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``ee1467d4aa35`` | ``b4078ac230a1`` | ``2.9.0`` | add display name for dag and task instance. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``b4078ac230a1`` | ``8e1c784a4fc7`` | ``2.9.0`` | Change value column type to longblob in xcom table for | |
| | | | | mysql. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``8e1c784a4fc7`` | ``ab34f260b71c`` | ``2.9.0`` | Adding max_consecutive_failed_dag_runs column to dag_model | |
| | | | | table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``ab34f260b71c`` | ``d75389605139`` | ``2.9.0`` | add dataset_expression in DagModel. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``d75389605139`` | ``1fd565369930`` | ``2.9.0`` | Add run_id to (Audit) log table and increase event name | |
| | | | | length. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``1fd565369930`` | ``88344c1d9134`` | ``2.9.0`` | Add rendered_map_index to TaskInstance. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``88344c1d9134`` | ``10b52ebd31f7`` | ``2.8.1`` | Drop unused TI index. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``10b52ebd31f7`` | ``bd5dfbe21f88`` | ``2.8.0`` | Add processor_subdir to ImportError. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``bd5dfbe21f88`` | ``f7bf2a57d0a6`` | ``2.8.0`` | Make connection login/password TEXT. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``f7bf2a57d0a6`` | ``375a816bbbf4`` | ``2.8.0`` | Add owner_display_name to (Audit) Log table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``375a816bbbf4`` | ``405de8318b3a`` | ``2.8.0`` | add new field 'clear_number' to dagrun. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``405de8318b3a`` | ``788397e78828`` | ``2.7.0`` | add include_deferred column to pool. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``788397e78828`` | ``937cbd173ca1`` | ``2.7.0`` | Add custom_operator_name column. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| | ``937cbd173ca1`` (base) | ``None`` | ``2.7.0`` | Add index to task_instance table. | |
| +-------------------------+------------------+-------------------+--------------------------------------------------------------+ |
| |
| .. End of auto-generated table |
| |
| .. spelling:word-list:: |
| branchpoint |
| mergepoint |
