Google Git
Sign in
apache / bookkeeper / 8201de86a6e04164efa4fabbf91d4c4a67bbd1fe
commit8201de86a6e04164efa4fabbf91d4c4a67bbd1fe[log] [tgz]
authorhangc0276 <hangc0276@163.com>Thu Mar 19 07:08:25 2020 +0800
committerGitHub <noreply@github.com>Wed Mar 18 16:08:25 2020 -0700
treebe82a0c915d6f7ece6318a547b7428e4862c827e
parentfbfd49a1e0307e5d490a2cdf8cbb840ef88cf628 [diff]
Add Journal PageCache flush interval control when journalSyncData is disabled

Descriptions of the changes in this PR:
### Motivation
There was a weird behavior observed: when journalSyncData is disabled, it will trigger callbacks when the data is flushed to filesystem(PageCache) but still enqueue a fsync request to force write queue to fsync PageCache data to disk frequently. Though the fsync thread dequeue fsync request and execute fsync operation asynchronously, it still cause disk io util increase when fsync frequency high enough. when disk io util increase, especially reach 100%, it will block new journal file creation and lead to journal sync time increases to mutiple seconds.

### Changes
when journalSyncData is disabled, we introduce `journalPageCacheFlushIntervalMSec` to control journal PageCache flush frequency to reduce disk io util, default value is 1000ms

Reviewers: Enrico Olivelli <eolivelli@gmail.com>, Jia Zhai <zhaijia@apache.org>, Sijie Guo <None>

This closes #2287 from hangc0276/bugfix
4 files changed
tree: be82a0c915d6f7ece6318a547b7428e4862c827e
  1. .github/
  2. .test-infra/
  3. bin/
  4. bookkeeper-benchmark/
  5. bookkeeper-common/
  6. bookkeeper-common-allocator/
  7. bookkeeper-dist/
  8. bookkeeper-http/
  9. bookkeeper-proto/
  10. bookkeeper-server/
  11. bookkeeper-stats/
  12. bookkeeper-stats-providers/
  13. buildtools/
  14. circe-checksum/
  15. conf/
  16. cpu-affinity/
  17. deploy/
  18. dev/
  19. docker/
  20. metadata-drivers/
  21. microbenchmarks/
  22. shaded/
  23. site/
  24. stats/
  25. stream/
  26. tests/
  27. tools/
  28. .gitignore
  29. LICENSE
  30. NOTICE
  31. pom.xml
  32. README.md
README.md

Build Status Build Status Coverage Status Maven Central

Apache BookKeeper

Apache BookKeeper is a scalable, fault tolerant and low latency storage service optimized for append-only workloads.

It is suitable for being used in following scenarios:

  • WAL (Write-Ahead-Logging), e.g. HDFS NameNode.
  • Message Store, e.g. Apache Pulsar.
  • Offset/Cursor Store, e.g. Apache Pulsar.
  • Object/Blob Store, e.g. storing state machine snapshots.

Get Started

  • Concepts: Start with the basic concepts of Apache BookKeeper. This will help you to fully understand the other parts of the documentation.
  • Getting Started to setup BookKeeper to write logs.

Documentation

Developers

You can also read Turning Ledgers into Logs to learn how to turn ledgers into continuous log streams. If you are looking for a high level log stream API, you can checkout DistributedLog.

Administrators

Contributors

Get In Touch

Report a Bug

For filing bugs, suggesting improvements, or requesting new features, help us out by opening a Github issue or opening an Apache jira.

Need Help?

Subscribe or mail the user@bookkeeper.apache.org list - Ask questions, find answers, and also help other users.

Subscribe or mail the dev@bookkeeper.apache.org list - Join development discussions, propose new ideas and connect with contributors.

Join us on Slack - This is the most immediate way to connect with Apache BookKeeper committers and contributors.

Contributing

We feel that a welcoming open community is important and welcome contributions.

Contributing Code

  1. See Developer Setup to get your local environment setup.

  2. Take a look at our open issues: JIRA Issues Github Issues.

  3. Review our coding style and follow our pull requests to learn about our conventions.

  4. Make your changes according to our contribution guide.

Improving Website and Documentation

  1. See Building the website and documentation on how to build the website and documentation.