tree ee85f4ba7476bcc7e7e6fcb518bbdd12a0f1783f
parent 6c9fdb0aa73bf7c50e131c99d5ff3704ca20f887
author Jiajun Wang <1803880+jiajunwang@users.noreply.github.com> 1588805295 -0700
committer GitHub <noreply@github.com> 1588805295 -0700
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsBcBAABCAAQBQJesz6vCRBK7hj4Ov3rIwAAdHIIABuNg2T7d+WMOThtujXJmMnW
 Aq0pWAgNygV2eyP1wwIEqBoQDAXqDgTGI8hIfn9ecKiIRS97ZRTb8BNprS8UlE8H
 Zy7fqaFGian6V3Bc4SxiMqvv4dt02OLl8iA0ARdnamw9f8PdVof8rn8c9CVTDjjK
 gVbmZffYb285Iu32G2mUhYd+xoNqXsVvUCD1N1cjc+Sz25cJSMpXWeaPSA+LCSly
 27+TCafjGLat+VYWBu+pQvCAZqsOeQfyJ0Q/nS70m5KhpeOdN4cINmeE9mFj5/NM
 QIn5v+L/ISUoGb223WeuqldUut5/Uw8YdcB3d3Ub/n8rfmVKXBwXsK4tR2bh1Vs=
 =LmY8
 -----END PGP SIGNATURE-----
 

Adjust the auto rebalancer state assignment logic to reduce top state transition. (#986)

The old state assignment logic assign the states to selected nodes according to the priority of the current replica state that is on the instance. Moreover, the sorting algorithm is designed to prioritize both current topstate and current secondary states equally. The result is that we will have premature mastership handoff to a current seconardy state host before the real desired master host is ready.
For example,
1. The current states are: [N1:M, N2:S, N3,S]
2. The desired states are: [N4:M, N2:S, N1:S]
3. Due to the sorting logic based on current states, we will have a transient preference list ordered like: [N2, N1, N4]. In which case, the controller will assign master to N2 before N4 has a slave state replica.
4. When N4 finishes the Offline to Slave transition, the same sorting logic will sort the preference list to be: [N4, N2, N1]. Then we have another mastership handoff.
To be clear, we don't want step 3. But only the state transition in step 4.

In this PR, we refactor the sorting logic so that it will only move the master whenever the candidate has a "ready" state replica, in which case, only one mastership handoff happens.