tree 70db6991b04ac0f1ab929f70fa0b23a05fc9f25b
parent 9e8dcbe787b2d331e731cdeb263f1d159959a925
author mynameborat <bharath.kumarasubramanian@gmail.com> 1617409026 -0700
committer GitHub <noreply@github.com> 1617409026 -0700
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsBcBAABCAAQBQJgZ7QCCRBK7hj4Ov3rIwAAdHIIACM/1SUSMDQO84YDQszsVKAv
 fYCXmwwxaVReizTA6ZOYhCquvwbOwMqBQlUOA4IyHhD7gSUUG0okIVrmPTXUnI7h
 NukUUaEeyI4AKJm50p4pSSSi3//NI5a46Ghf2ybsCrHCLTRTsDo4eAzDQibKRqCM
 hA48VKaunGdfeZy47+FPTcxHhoYnA2mGbv5OxeTPRLiNFjogfVU7nCMjvc32kn3m
 SdjXOuGVyOYj+On0KDxIO6IaPAX7+hqZGiyn/qdWwHKQ+BT8Bk1NXFHQ7d1/9Ze5
 h5GROKOue0UM6q9ssJ7N8LaRVXF0yNDn3HG2ukZJaWkSxqZWuql03xVHX0MHjIA=
 =pjw8
 -----END PGP SIGNATURE-----
 

SAMZA-2633: Rolling upgrades cause downtime to upgraded processors for the entire deployment window (#1484)

Description:
During rolling upgrades, the current debounce timer gets extended every time when there is a quorum change notification. As a result, processors that were upgraded earlier in the deployment window remain unavailable waiting for work assignment. In some scenarios, this cause processors to be unavailable for 20 minutes or so depending on the size of the quorum and the debounce time configuration. Refer to SAMZA-2633 for more information.

Changes:
Optimize the leader workflow to skip rebalance if there is no changes to work assignment
Make processors start with most recent agreed job model on startup
Leader persists the active job model version in ZK to enable change [2]
Introduce config for applications to opt-in for the optimization

Usage Instructions:
Set job.coordinator.zk.enable-startup-with-active-job-model to true as part of the application configuration to enable processor use the recent active job model during startup and also enable leader to skip rebalances if the work assignment remains the same.