|author||Sijie Guo <firstname.lastname@example.org>||Tue Sep 12 15:16:49 2017 +0800|
|committer||Jia Zhai <email@example.com>||Tue Sep 12 15:16:49 2017 +0800|
ISSUE #185: Thrift version conflicts and use heap bytebuffer for thrift serialization Descriptions of the changes in this PR: - it seems that shade plugin doesn't work well with sub-modules. even the libthrift-9 was shaded, it is still imported/included in the sub-modules, causing the version conflict. Explicitly exclude the libthrift from distributedlog-core. - thrift serialization is using heap bytebuffer. so if a bytebuffer is direct, the serialization will fail. add a change to check if bytebuffer is heap, if bytebuffer is not heap, copy the content into a heap buffer. this is not good for performance, but we don't have any other choices because the limitation comes from libthrift. Author: Sijie Guo <firstname.lastname@example.org> Reviewers: Enrico Olivelli <email@example.com>, Jia Zhai <None>, Leigh Stewart <None> This closes #187 from sijie/use_heapbytebuffer, closes #185
Apache DistributedLog (DL) is a high-throughput, low-latency replicated log service, offering durability, replication and strong consistency as essentials for building reliable real-time applications.
Apache DistributedLog project graduated from Incubator at July 2017. It is now a sub-project of Apache BookKeeper.
DL is able to provide milliseconds latency on durable writes with a large number of concurrent logs, and handle high volume reads and writes per second from thousands of clients.
Messages are persisted on disk and replicated to store multiple copies to prevent data loss. They are guaranteed to be consistent among writers and readers in terms of strict ordering.
DL provides an efficient service layer that is optimized for running in a multi- tenant datacenter environment such as Mesos or Yarn. The service layer is able to support large scale writes (fan-in) and reads (fan-out).
DL supports various workloads from latency-sensitive online transaction processing (OLTP) applications (e.g. WAL for distributed database and in-memory replicated state machines), real-time stream ingestion and computing, to analytical processing.
To support a large number of logs for multi-tenants, DL is designed for I/O isolation in real-world workloads.
DL has a modern layered architecture design, which separates the stateless service tier from the stateful storage tier. To support large scale writes (fan- in) and reads (fan-out), DL allows scaling storage independent of scaling CPU and memory.
For filing bugs, suggesting improvements, or requesting new features, help us out by opening a jira.
Join us on Slack - This is the most immediate way to connect with DistributedLog committers and contributors.
We feel that a welcoming open community is important and welcome contributions.
See Developer Guide to get your local environment setup.
Take a look at our open issues.
Make your changes according to our code review workflow.
Checkout the list of project ideas to contribute more features or improvements.
See website/README.md on how to build the website.
See docs/README.md on how to build the documentation.
Apache DistributedLog is an open source project of The Apache Software Foundation (ASF). The Apache DistributedLog project originated from Twitter.