| commit | 4c8587715104cde6dbd70d1e7de0fc1853122eda | [log] [tgz] |
|---|---|---|
| author | Matteo Merli <mmerli@apache.org> | Tue Jan 08 16:12:48 2019 -0800 |
| committer | GitHub <noreply@github.com> | Tue Jan 08 16:12:48 2019 -0800 |
| tree | d5bcac98966098ac4409b1aeeed60e3cca6632aa | |
| parent | 1a66de0f1841309390261e90d8d0d0af7aa2ede6 [diff] |
Allow to configure sticky reads ### Motivation Currently the BK client is issuing the read requests in round-robin fashion across all the bookies in the write set. One issue with this approach is that it's not taking full advantage of the read-ahead cache, either explicit (like in `DbLedgerStorage`) or implicit (by reading data through Linux page cache which will do some prefetching). With `e=2`, `w=2`, when we read `e-0` from `bookie-1` and `e-1` from `bookie-2`, we fail to take advantage of the fact that `bookie-1` will have already `e-1` in memory. Effectively with `e-2`, `w-2` the disk read IO will be doubled, compared to the amount of data served to BK clients. The larger the quorum, the bigger will be overhead (eg: `e=5`, `w=5` will lead to 5x reads from disk). ### Changes Added a BK client flag for "sticky reads". When reading from a ledger that has `E=W` (every bookie has all the entries), the sticky reads will direct all read request to 1 single bookie in the ensemble. Reviewers: Enrico Olivelli <eolivelli@gmail.com>, Sijie Guo <sijie@apache.org> This closes #1808 from merlimat/single-bookie-per-ledger-read-master
Apache BookKeeper is a scalable, fault tolerant and low latency storage service optimized for append-only workloads.
It is suitable for being used in following scenarios:
You can also read Turning Ledgers into Logs to learn how to turn ledgers into continuous log streams. If you are looking for a high level log stream API, you can checkout DistributedLog.
For filing bugs, suggesting improvements, or requesting new features, help us out by opening a Github issue or opening an Apache jira.
Subscribe or mail the user@bookkeeper.apache.org list - Ask questions, find answers, and also help other users.
Subscribe or mail the dev@bookkeeper.apache.org list - Join development discussions, propose new ideas and connect with contributors.
Join us on Slack - This is the most immediate way to connect with Apache BookKeeper committers and contributors.
We feel that a welcoming open community is important and welcome contributions.
See Developer Setup to get your local environment setup.
Take a look at our open issues: JIRA Issues Github Issues.
Review our coding style and follow our pull requests to learn about our conventions.
Make your changes according to our contribution guide.