tree 4228b5b90708cf6ffb3c3ec2cd7c03eecd7d68f5
parent f36f9f4dc100986a32827b2ac3740ff56ed0a47c
author Daniel Standish <15932138+dstandish@users.noreply.github.com> 1715223415 -0700
committer GitHub <noreply@github.com> 1715223415 -0700
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsFcBAABCAAQBQJmPDt3CRC1aQ7uu5UhlAAAlxgQAC1YRgkcn6o+Nx3hoUqp5eMH
 9RYmLwrrn7iYFY41ctaT9kiZxs2BVBvRSyNVl+LSHvOyLlGH32/QFF79CZIG+k+E
 z/2u2gPtFeXQvGtNKkkY5fq8tFAf3Q4jTNtvL8o9binwra1sFpoQEiVLauk6I+xX
 iUIp6QwA42AVgkN+InSmQO9J9WnS7/kEHWhpZbxDsLqLLj0R3fUflm1hSq50B/U4
 muzNqzDh269y4wqzNrMcpa0+sbNrE7MnKia+d+LVSp1heuSm+KxgSbBTjF+rCPbP
 U0aZXZOLoJ0GaD8xGRKk7InLp/P2ExUcwbvbzk8gpaD5caRckscknav3wo3cGsIn
 2k2NKFXQEu9UlO6y9vyDduQdoP9Q1IzVa/A3PStnf3gXzfU/Pyqaq4abaejiPYnW
 /ZfZ3UztNtkvyenukdRUPKK2gac2nX0O+U4XNnW8AqzLlchl1fiMAKTggHoFSkcx
 885iMgoGiT/vK/hFzUzcxnhV0YY4atWzYpAtvcvOKNyCRAuxcyZ8/p49yRCqqbTi
 r6inF0hSxWvzLRWT6wutZ6ZPi41ILG0fJIcHEwo4p1avTdl9IgVMn67h2woTCjFj
 alBHj7Z5IQ005veMZfMNioBET1TR+mvXTqMg2n4j1Y0vZOlixku9Qgg0YmlODYd7
 +PtS7Qr0Wkh+XvtPeqi/
 =1aV9
 -----END PGP SIGNATURE-----
 

Scheduler to handle incrementing of try_number (#39336)

Previously, there was a lot of bad stuff happening around try_number.

We incremented it when task started running. And because of that, we had this logic to return "_try_number + 1" when task not running. But this gave the "right" try number before it ran, and the wrong number after it ran. And, since it was naively incremented when task starts running -- i.e. without regard to why it is running -- we decremented it when deferring or exiting on a reschedule.

What I do here is try to remove all of that stuff:

no more private _try_number attr
no more getter logic
no more decrementing
no more incrementing as part of task execution
Now what we do is increment only when the task is set to scheduled and only when it's not coming out of deferral or "up_for_reschedule". So the try_number will be more stable. It will not change throughout the course of task execution. The only time it will be incremented is when there's legitimately a new try.

One consequence of this is that try number will no longer be incremented if you run either airlfow tasks run or ti.run() in isolation. But because airflow assumes that all tasks runs are scheduled by the scheduler, I do not regard this to be a breaking change.

If user code or provider code has implemented hacks to get the "right" try_number when looking at it at the wrong time (because previously it gave the wrong answer), unfortunately that code will just have to be patched. There are only two cases I know of in the providers codebase -- openlineage listener, and dbt openlineage.

As a courtesy for backcompat we also add property _try_number which is just a proxy for try_number, so you'll still be able to access this attr. But, it will not behave the same as it did before.

---------

Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>