Improve the WAGED rebalancer performance. (#586)

This change improves the rebalance's speed by 2x to 5x depends on the host capacity.

Parallelism the loop processing whenever possible and help to improve the performance. This does not change the logic.
Avoid some duplicate logic in the loop. Put the calculation outside the loop and only do it once.
8 files changed
tree: 9b9fd7f3af624cfe788ba82a356c03e54d8eb34a
  1. helix-admin-webapp/
  2. helix-agent/
  3. helix-core/
  4. helix-front/
  5. helix-rest/
  6. recipes/
  7. scripts/
  8. website/
  9. .gitignore
  10. build
  11. bump-up.command
  12. deploySite.sh
  13. helix-style-intellij.xml
  14. helix-style.xml
  15. hpost-review.sh
  16. LICENSE
  17. NOTICE
  18. pom.xml
  19. README.md
README.md

Apache Helix

Helix is part of the Apache Software Foundation.

Project page: http://helix.apache.org/

Mailing list: http://helix.apache.org/mail-lists.html

Build

mvn clean install package -DskipTests

WHAT IS HELIX

Helix is a generic cluster management framework used for automatic management of partitioned, replicated and distributed resources hosted on a cluster of nodes. Helix provides the following features:

  1. Automatic assignment of resource/partition to nodes
  2. Node failure detection and recovery
  3. Dynamic addition of Resources
  4. Dynamic addition of nodes to the cluster
  5. Pluggable distributed state machine to manage the state of a resource via state transitions
  6. Automatic load balancing and throttling of transitions