| commit | d4d0060bc38995777da5cc2363c7af897a908f27 | [log] [tgz] |
|---|---|---|
| author | Michael Marshall <47911938+michaeljmarshall@users.noreply.github.com> | Tue Feb 16 04:19:02 2021 -0700 |
| committer | GitHub <noreply@github.com> | Tue Feb 16 12:19:02 2021 +0100 |
| tree | e54e9a5892f20c0c845c9e45a24afe84a0d31873 | |
| parent | 92bca066c9eca88313b9ef89ff6c567b8f511a63 [diff] |
ISSUE #2506: Skip unavailable bookies during verifyLedgerFragment Descriptions of the changes in this PR: This PR improves the `verifyLedgerFragment` method in the `LedgerChecker` by skipping calls to bookies that are known to be unavailable. The "bad bookies" are calculated by using metadata available in ZK and accessed by the `BookKeeperAdmin`. Note that `verifyLedgerFragment` will still run checks on all other bookies that appear available. ### Motivation The motivation for this change is demonstrated in #2506. As this code currently works, there are a ton of calls made to unavailable bookies with the intent of calculating bad (unavailable) bookies. This proposed change would greatly decrease the number of calls that the auditor and the replicator need to make to calculate which ledgers need replicating. ### Changes 1. Added `getUnavailableBookies` method to the `BookKeeperAdmin`. This method could attempt to use caching, but it's not actually called that often, so I think caching might not add complexity without much value. 2. Updated `verifyLedgerFragment` method signature to take a collection of `unavailableBookies`. Master Issue: #2506 ### Testing If these changes are acceptable, I'd like some help identifying the best way to test these changes. I already added some coverage for the `getUnavailableBookies` method, but I haven't explicitly tested the fundamental change this PR proposes. Thanks! Reviewers: Enrico Olivelli <eolivelli@gmail.com>, Andrey Yegorov <andrey.yegorov@datastax.com> This closes #2597 from michaeljmarshall/leverage-available-bookies, closes #2506
Apache BookKeeper is a scalable, fault tolerant and low latency storage service optimized for append-only workloads.
It is suitable for being used in following scenarios:
Please visit the Documentation from the project website for more information.
For filing bugs, suggesting improvements, or requesting new features, help us out by opening a Github issue or opening an Apache jira.
Subscribe or mail the user@bookkeeper.apache.org list - Ask questions, find answers, and also help other users.
Subscribe or mail the dev@bookkeeper.apache.org list - Join development discussions, propose new ideas and connect with contributors.
Join us on Slack - This is the most immediate way to connect with Apache BookKeeper committers and contributors.
We feel that a welcoming open community is important and welcome contributions.
See Developer Setup to get your local environment setup.
Take a look at our open issues: JIRA Issues Github Issues.
Review our coding style and follow our pull requests to learn about our conventions.
Make your changes according to our contribution guide.