Google Git
Sign in
apache / bookkeeper / dc2bb1dfed38d7f9ad7a29fc92bd907e51af5b21
commitdc2bb1dfed38d7f9ad7a29fc92bd907e51af5b21[log] [tgz]
authorHang Chen <chenhang@apache.org>Thu Feb 01 14:49:07 2024 +0800
committerGitHub <noreply@github.com>Thu Feb 01 14:49:07 2024 +0800
tree4c3ca4738ad38d706a6ec815871987ff75bebc35
parent1eceb5dd2180ab57e5b21be968206292bb825177 [diff]
Enable reorder read sequence for bk client by default (#4139)

### Motivation

<!-- Explain here the context, and why you're making that change. What is the problem you're trying to solve. -->

If one ledger's ensemble is [bk0, bk1] and bk0 is down, the bookie client may send a read request to bk0 first then fail with the following errors, and resend the read request to bk1 in the end. 

```
2023-10-19T18:33:52,042 - ERROR - [BookKeeperClientWorker-OrderedExecutor-3-0:PerChannelBookieClient@563] - Cannot connect to 192.168.31.216:3181 as endpoint resolution failed (probably bookie is down) err org.apache.bookkeeper.proto.BookieAddressResolver$BookieIdNotResolvedException: Cannot resolve bookieId 192.168.31.216:3181, bookie does not exist or it is not running
2023-10-19T18:33:52,042 - INFO  - [BookKeeperClientWorker-OrderedExecutor-3-0:DefaultBookieAddressResolver@77] - Cannot resolve 192.168.31.216:3181, bookie is unknown org.apache.bookkeeper.client.BKException$BKBookieHandleNotAvailableException: Bookie handle is not available
2023-10-19T18:33:52,042 - INFO  - [BookKeeperClientWorker-OrderedExecutor-3-0:PendingReadOp$LedgerEntryRequest@223] - Error: Bookie handle is not available while reading L6 E40 from bookie: 192.168.31.216:3181
```
One of the related issues is in the auto-recovery decommission and there is one PR in the BookKeeper repo: https://github.com/apache/bookkeeper/pull/4113

However, the bookie client already knows the bk0 is down and we should send the read request to bk1 first. So we can reorder the read request based on the known bookie list. If one bookie is lost, it will reorder the lost bookie to the end of the read list.

### Modifications

<!-- Describe the modifications you've done. -->

Enable the `reorderReadSequence` by default for auto-recovery.
4 files changed
tree: 4c3ca4738ad38d706a6ec815871987ff75bebc35
  1. .github/
  2. .test-infra/
  3. bin/
  4. bookkeeper-benchmark/
  5. bookkeeper-common/
  6. bookkeeper-common-allocator/
  7. bookkeeper-dist/
  8. bookkeeper-http/
  9. bookkeeper-proto/
  10. bookkeeper-server/
  11. bookkeeper-slogger/
  12. buildtools/
  13. circe-checksum/
  14. conf/
  15. cpu-affinity/
  16. deploy/
  17. dev/
  18. docker/
  19. metadata-drivers/
  20. microbenchmarks/
  21. native-io/
  22. shaded/
  23. site3/
  24. src/
  25. stats/
  26. stream/
  27. tests/
  28. testtools/
  29. tools/
  30. .asf.yaml
  31. .dlc.json
  32. .gitignore
  33. CONTRIBUTING.md
  34. LICENSE
  35. NOTICE
  36. pom.xml
  37. README.md
README.md

Maven Central

Apache BookKeeper

Apache BookKeeper is a scalable, fault-tolerant and low latency storage service optimized for append-only workloads.

It is suitable for being used in following scenarios:

  • WAL (Write-Ahead-Logging), e.g. HDFS NameNode, Pravega.
  • Message Store, e.g. Apache Pulsar.
  • Offset/Cursor Store, e.g. Apache Pulsar.
  • Object/Blob Store, e.g. storing state machine snapshots.

Get Started

  • Checkout the project website.
  • Concepts: Start with the basic concepts of Apache BookKeeper. This will help you to fully understand the other parts of the documentation.
  • Follow the Installation guide to set up BookKeeper.

Documentation

Please visit the Documentation from the project website for more information.

Get In Touch

Report a Bug

For filing bugs, suggesting improvements, or requesting new features, help us out by opening a GitHub issue.

Need Help?

Subscribe or mail the user@bookkeeper.apache.org list - Ask questions, find answers, and also help other users.

Subscribe or mail the dev@bookkeeper.apache.org list - Join development discussions, propose new ideas and connect with contributors.

Join us on Slack - This is the most immediate way to connect with Apache BookKeeper committers and contributors.

Contributing

We feel that a welcoming open community is important and welcome contributions.

Contributing Code

  1. See our installation guide to get your local environment setup.

  2. Take a look at our open issues: GitHub Issues.

  3. Review our coding style and follow our pull requests to learn more about our conventions.

  4. Make your changes according to our contributing guide