Google Git
Sign in
apache / bookkeeper / ae023fd5fab1f015229e475de106df46144bd4d0
commitae023fd5fab1f015229e475de106df46144bd4d0[log] [tgz]
authorKevin Wilson <klwilson227@comcast.net>Mon Apr 05 22:02:43 2021 -0600
committerGitHub <noreply@github.com>Tue Apr 06 12:02:43 2021 +0800
treeb40bbe6a2d08da95fe374463fbfec7e1a00a1832
parent6eadbaac1806cebcf658a740b6dd7277a24e4fdb [diff]
Issue 2638: Compaction Limits (#2670)

This resolves issue 2638, by allowing a customer to set a limit on the duration of the major and minor compaction runs. This allows the customer to balance the needs for compaction against the need for normal garbage collection. Unbounded execution of compaction can starve the garbage collection process, and may lead to extended periods of high disk IO. By setting a upper bound on the execution duration for the garbage collection, we can ensure a balance is maintained between both GC and Compaction processes.

Logging has been improved to allow a user to see the complete usage of the buckets, vs. those that were compacted, so that they can decide on any further tuning that may be required.

Test cases have been added to ensure limits are enforced properly and that defaulting of the values is correctly implemented.

* 2638-CompactionLimits

* align config parameters

* Refactor Limit to MaxTimeMs
5 files changed
tree: b40bbe6a2d08da95fe374463fbfec7e1a00a1832
  1. .github/
  2. .test-infra/
  3. bin/
  4. bookkeeper-benchmark/
  5. bookkeeper-common/
  6. bookkeeper-common-allocator/
  7. bookkeeper-dist/
  8. bookkeeper-http/
  9. bookkeeper-proto/
  10. bookkeeper-server/
  11. bookkeeper-stats/
  12. bookkeeper-stats-providers/
  13. buildtools/
  14. circe-checksum/
  15. conf/
  16. cpu-affinity/
  17. deploy/
  18. dev/
  19. docker/
  20. gradle/
  21. metadata-drivers/
  22. microbenchmarks/
  23. shaded/
  24. site/
  25. site2/
  26. stats/
  27. stream/
  28. tests/
  29. tools/
  30. .gitignore
  31. build.gradle
  32. dependencies.gradle
  33. gradle.properties
  34. gradlew
  35. gradlew.bat
  36. Jenkinsfile
  37. LICENSE
  38. NOTICE
  39. pom.xml
  40. README.md
  41. settings.gradle
  42. version.gradle
README.md

Build Status Build Status Coverage Status Maven Central

Apache BookKeeper

Apache BookKeeper is a scalable, fault tolerant and low latency storage service optimized for append-only workloads.

It is suitable for being used in following scenarios:

  • WAL (Write-Ahead-Logging), e.g. HDFS NameNode.
  • Message Store, e.g. Apache Pulsar.
  • Offset/Cursor Store, e.g. Apache Pulsar.
  • Object/Blob Store, e.g. storing state machine snapshots.

Get Started

  • Checkout the project website.
  • Concepts: Start with the basic concepts of Apache BookKeeper. This will help you to fully understand the other parts of the documentation.
  • Follow the Install guide to setup BookKeeper.

Documentation

Please visit the Documentation from the project website for more information.

Get In Touch

Report a Bug

For filing bugs, suggesting improvements, or requesting new features, help us out by opening a Github issue or opening an Apache jira.

Need Help?

Subscribe or mail the user@bookkeeper.apache.org list - Ask questions, find answers, and also help other users.

Subscribe or mail the dev@bookkeeper.apache.org list - Join development discussions, propose new ideas and connect with contributors.

Join us on Slack - This is the most immediate way to connect with Apache BookKeeper committers and contributors.

Contributing

We feel that a welcoming open community is important and welcome contributions.

Contributing Code

  1. See Developer Setup to get your local environment setup.

  2. Take a look at our open issues: JIRA Issues Github Issues.

  3. Review our coding style and follow our pull requests to learn about our conventions.

  4. Make your changes according to our contribution guide.

Improving Website and Documentation

  1. See Building the website and documentation on how to build the website and documentation.