commit | a659cf50fe46cfea801199df559663abad0eeaa5 | [log] [tgz] |
---|---|---|
author | karanmehta93 <k.mehta@salesforce.com> | Tue Nov 03 02:34:33 2020 -0800 |
committer | GitHub <noreply@github.com> | Tue Nov 03 18:34:33 2020 +0800 |
tree | 26550ab9d452aa67488b222a938047ed3c000410 | |
parent | 85cd302291784dc8502172303ad3a315a15566f3 [diff] |
Protobuf should be serialized to direct memory instead of heap memory (#2441) Descriptions of the changes in this PR: ### Motivation Protobuf serialization is the last step of the netty pipeline. Currently, we allocate a heap buffer while serializing and pass it down to netty library. In `AbstractChannel#filterOutboundMessage()`, netty copies that data to a direct buffer if it is currently in heap (otherwise skips it and uses it directly). This change helps in reducing unncessary CPU cycles for buffer copies and also helps alleviate pressure off the GC, since there is less churn of memory. In some stress tests, I see a reduction upto 33% in number of GC's with the same avg running duration in BK Client. The workload does writes of different sizes, ranging from 1 KB to 512 KB. Bookies aren't usually CPU bound. This change improves READ_ENTRY code paths by a small factor. ### Changes Used direct memory for buffer allocation versus heap memory * Added comments
Apache BookKeeper is a scalable, fault tolerant and low latency storage service optimized for append-only workloads.
It is suitable for being used in following scenarios:
You can also read Turning Ledgers into Logs to learn how to turn ledgers into continuous log streams. If you are looking for a high level log stream API, you can checkout DistributedLog.
For filing bugs, suggesting improvements, or requesting new features, help us out by opening a Github issue or opening an Apache jira.
Subscribe or mail the user@bookkeeper.apache.org list - Ask questions, find answers, and also help other users.
Subscribe or mail the dev@bookkeeper.apache.org list - Join development discussions, propose new ideas and connect with contributors.
Join us on Slack - This is the most immediate way to connect with Apache BookKeeper committers and contributors.
We feel that a welcoming open community is important and welcome contributions.
See Developer Setup to get your local environment setup.
Take a look at our open issues: JIRA Issues Github Issues.
Review our coding style and follow our pull requests to learn about our conventions.
Make your changes according to our contribution guide.