commit | 171e2f914c05357d9e81c7e951f5c86a0f463a75 | [log] [tgz] |
---|---|---|
author | Sijie Guo <guosijie@gmail.com> | Mon Aug 26 11:16:08 2019 -0700 |
committer | GitHub <noreply@github.com> | Mon Aug 26 11:16:08 2019 -0700 |
tree | d54fc0a24b7319ae802f63ba0a93a0eb46a58b3d | |
parent | f1df81f6590e048b93ec21776f931a28f0468a45 [diff] |
Improve journal throughput when journalSyncData is disabled. Descriptions of the changes in this PR: ### Motivation There was a weird behavior observed: when `journalSyncData` is disabled, the journal throughput decreased with multiple journal dirs on same journal disk. The problem is when `journalSyncData` is disabled, we still sync data to disk when rolling the journal file. The sync time increases to multiple seconds, because it has to flush all the files to disk and wait the whole sync operation to be completed. ### Changes when `journalSyncData` is disabled, we trigger callbacks when the data is flushed to filesystem but still enqueue a fsync request to force write queue to make sure we still fsync data frequently. Also added a journal perf tool to do performance testing on journal. Reviewers: Enrico Olivelli <eolivelli@gmail.com>, Jia Zhai <zhaijia@apache.org>, Matteo Merli <mmerli@apache.org> This closes #2150 from sijie/journal_perf
Apache BookKeeper is a scalable, fault tolerant and low latency storage service optimized for append-only workloads.
It is suitable for being used in following scenarios:
You can also read Turning Ledgers into Logs to learn how to turn ledgers into continuous log streams. If you are looking for a high level log stream API, you can checkout DistributedLog.
For filing bugs, suggesting improvements, or requesting new features, help us out by opening a Github issue or opening an Apache jira.
Subscribe or mail the user@bookkeeper.apache.org list - Ask questions, find answers, and also help other users.
Subscribe or mail the dev@bookkeeper.apache.org list - Join development discussions, propose new ideas and connect with contributors.
Join us on Slack - This is the most immediate way to connect with Apache BookKeeper committers and contributors.
We feel that a welcoming open community is important and welcome contributions.
See Developer Setup to get your local environment setup.
Take a look at our open issues: JIRA Issues Github Issues.
Review our coding style and follow our pull requests to learn about our conventions.
Make your changes according to our contribution guide.