Google Git
Sign in
apache / distributedlog / 463f4972700d6389085544b19745e27fdaafd6d5
commit463f4972700d6389085544b19745e27fdaafd6d5[log] [tgz]
authorSijie Guo <sijie@apache.org>Thu Oct 13 00:38:25 2016 -0700
committerSijie Guo <sijieg@twitter.com>Thu Oct 13 00:38:25 2016 -0700
tree0dfd67a311a01f94cf04a5fffc00e39c3f73b483
parentae69fffd9bde9ca588fd008c2c70908b1ca024fb [diff]
[BLOG] A technical review of kafka and distributedlog

Author: Sijie Guo <sijie@apache.org>

Reviewers: Leigh Stewart <lstewart@twitter.com>

Closes #35 from sijie/sijie/kafka_dl_blog_post and squashes the following commits:

3d6bfd2 [Sijie Guo] Fix the format
a78175f [Sijie Guo] Add all references
74d538e [Sijie Guo] Use the old url as the permantlink
b760cb4 [Sijie Guo] Fix the broken link and the date
c170fdc [Sijie Guo] Change the table to html and set 1px border
4452709 [Sijie Guo] Add a blog post about "kafka vs distributedlog"
5 files changed
tree: 0dfd67a311a01f94cf04a5fffc00e39c3f73b483
  1. distributedlog-benchmark/
  2. distributedlog-client/
  3. distributedlog-core/
  4. distributedlog-example/
  5. distributedlog-protocol/
  6. distributedlog-service/
  7. distributedlog-tutorials/
  8. docker/
  9. docs/
  10. scripts/
  11. src/
  12. vagrant/
  13. website/
  14. .gitignore
  15. .travis.yml
  16. ChangeLog
  17. CONFIG.ini
  18. Dockerfile
  19. LICENSE
  20. NOTICE
  21. pom.xml
  22. README.md
  23. Vagrantfile
README.md

Build Status Coverage Status

Overview

DistributedLog (DL) is a high-performance, replicated log service, offering durability, replication and strong consistency as essentials for building reliable distributed systems.

High Performance

DL is able to provide milliseconds latency on durable writes with a large number of concurrent logs, and handle high volume reads and writes per second from thousands of clients.

Durable and Consistent

Messages are persisted on disk and replicated to store multiple copies to prevent data loss. They are guaranteed to be consistent among writers and readers in terms of strict ordering.

Efficient Fan-in and Fan-out

DL provides an efficient service layer that is optimized for running in a multi- tenant datacenter environment such as Mesos or Yarn. The service layer is able to support large scale writes (fan-in) and reads (fan-out).

Various Workloads

DL supports various workloads from latency-sensitive online transaction processing (OLTP) applications (e.g. WAL for distributed database and in-memory replicated state machines), real-time stream ingestion and computing, to analytical processing.

Multi Tenant

To support a large number of logs for multi-tenants, DL is designed for I/O isolation in real-world workloads.

Layered Architecture

DL has a modern layered architecture design, which separates the stateless service tier from the stateful storage tier. To support large scale writes (fan- in) and reads (fan-out), DL allows scaling storage independent of scaling CPU and memory.

Documentation and Getting Started

Getting involved

We feel that a welcoming community is important and we ask that you follow Twitter's Open Source Code of Conduct in all interactions with the community.

Authors

Thanks for assistance and contributions:

A full list of contributors can be found on GitHub.

License

Copyright 2016 Twitter, Inc.

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0