Implementation of ZoneawareEnsemblePlacementPolicy


Descriptions of the changes in this PR:

Implementation of ZoneawareEnsemblePlacementPolicy

- This is meant to be used in public cloud infrastructure in which
Availability zones (AZs) are isolated locations within data center
regions from which public cloud services originate and operate.
- ZoneawareEnsemblePlacementPolicy is a placement policy implementation
which uses zone information for placing ensembles.
- In this PlacementPolicy, it is assumed the networktopology/networklocation
 is going to be two level tree, where first part indicates zoneid
and the second part indicates upgradedomainid.
- Here upgrade domain is a set of bookies in an AZ that can be brought
down together for short term maintenance with no impact to the service.
This would help in enabling parallel patching.
- Upgrade domain is a logic concept/division and it may be mapped to
cloud provider native cluster/grouping like Placement Groups in AWS.

Introduce soft PlacementPolicyAdherence

- for ZoneawarePlacementPolicy if in a writeset the number
of zones are in between minnumofzones and desirednumofzones
then it is considered MEETS_SOFT.
- if all of the writesets have zones >= desirednumofzones
then it is considered MEETS_STRICT
- if a writeset contains < minnumofzones then it is considered
FAIL
- add corresponding metric for PlacementPolicyAdherence.MEETS_SOFT


Reviewers: Enrico Olivelli <eolivelli@gmail.com>, Sijie Guo <sijie@apache.org>, Venkateswararao Jujjuri (JV) <None>

This closes #2113 from reddycharan/zoneawareplacementpolicycommunitybranch
21 files changed
tree: 9be07b4de9df910291d615137706297f253d9899
  1. .github/
  2. .test-infra/
  3. .travis_scripts/
  4. bin/
  5. bookkeeper-benchmark/
  6. bookkeeper-common/
  7. bookkeeper-common-allocator/
  8. bookkeeper-dist/
  9. bookkeeper-http/
  10. bookkeeper-proto/
  11. bookkeeper-server/
  12. bookkeeper-stats/
  13. bookkeeper-stats-providers/
  14. buildtools/
  15. circe-checksum/
  16. conf/
  17. cpu-affinity/
  18. deploy/
  19. dev/
  20. docker/
  21. metadata-drivers/
  22. microbenchmarks/
  23. shaded/
  24. site/
  25. stats/
  26. stream/
  27. tests/
  28. tools/
  29. .gitignore
  30. .travis.yml
  31. LICENSE
  32. NOTICE
  33. pom.xml
  34. README.md
README.md

Build Status Build Status Coverage Status Maven Central

Apache BookKeeper

Apache BookKeeper is a scalable, fault tolerant and low latency storage service optimized for append-only workloads.

It is suitable for being used in following scenarios:

  • WAL (Write-Ahead-Logging), e.g. HDFS NameNode.
  • Message Store, e.g. Apache Pulsar.
  • Offset/Cursor Store, e.g. Apache Pulsar.
  • Object/Blob Store, e.g. storing state machine snapshots.

Get Started

  • Concepts: Start with the basic concepts of Apache BookKeeper. This will help you to fully understand the other parts of the documentation.
  • Getting Started to setup BookKeeper to write logs.

Documentation

Developers

You can also read Turning Ledgers into Logs to learn how to turn ledgers into continuous log streams. If you are looking for a high level log stream API, you can checkout DistributedLog.

Administrators

Contributors

Get In Touch

Report a Bug

For filing bugs, suggesting improvements, or requesting new features, help us out by opening a Github issue or opening an Apache jira.

Need Help?

Subscribe or mail the user@bookkeeper.apache.org list - Ask questions, find answers, and also help other users.

Subscribe or mail the dev@bookkeeper.apache.org list - Join development discussions, propose new ideas and connect with contributors.

Join us on Slack - This is the most immediate way to connect with Apache BookKeeper committers and contributors.

Contributing

We feel that a welcoming open community is important and welcome contributions.

Contributing Code

  1. See Developer Setup to get your local environment setup.

  2. Take a look at our open issues: JIRA Issues Github Issues.

  3. Review our coding style and follow our pull requests to learn about our conventions.

  4. Make your changes according to our contribution guide.

Improving Website and Documentation

  1. See Building the website and documentation on how to build the website and documentation.