website/docs/latest/user_guide/api/core.rst - distributedlog - Git at Google

 ---
 title: "Core Library"
 layout: default

 # Sub-level navigation
 sub-nav-group: user-guide
 sub-nav-parent: apis
 sub-nav-pos: 3
 sub-nav-title: Core Library API

 ---

 .. contents:: Core Library API

 Core Library API
 ================

 The distributedlog core library interacts with namespaces and logs directly.
 It is written in Java.

 Namespace API
 -------------

 A DL namespace is a collection of *log streams*. Applications could *create*
 or *delete* logs under a DL namespace.

 Namespace URI
 ~~~~~~~~~~~~~

 An **URI** is used to locate the *namespace*. The *Namespace URI* is typically
 comprised of *3* components:

 * scheme: `distributedlog-<backend>`. The *backend* indicates what backend is used to store the log data.
 * domain name: the domain name that used to talk to the *backend*. In the example as below, the domain name part is *zookeeper server*, which is used to store log metadata in bookkeeper based backend implementation.
 * path: path points to the location that stores logs. In the example as below, it is a zookeeper path that points to the znode that stores all the logs metadata.

 ::

     distributedlog-bk://<zookeeper-server>/path/to/stream

 The available backend is only bookkeeper based backend.
 The default `distributedlog` scheme is aliased to `distributedlog-bk`.

 Building a Namespace
 ~~~~~~~~~~~~~~~~~~~~

 Once you have the *namespace uri*, you could build the namespace instance.
 The namespace instance will be used for operating streams under it.

 ::

     // DistributedLog Configuration
     DistributedLogConfiguration conf = new DistributedLogConfiguration();
     // Namespace URI
     URI uri = ...; // create the namespace uri
     // create a builder to build namespace instances
     DistributedLogNamespaceBuilder builder = DistributedLogNamespaceBuilder.newBuilder();
     DistributedLogNamespace namespace = builder
         .conf(conf)             // configuration that used by namespace
         .uri(uri)               // namespace uri
         .statsLogger(...)       // stats logger to log stats
         .featureProvider(...)   // feature provider on controlling features
         .build();

 Create a Log
 ~~~~~~~~~~~~

 Creating a log is pretty straight forward by calling `distributedlognamespace#createlog(logname)`.
 it only creates the log under the namespace but doesn't return any handle for operating the log.

 ::

     DistributedLogNamespace namespace = ...; // namespace
     try {
         namespace.createLog("test-log");
     } catch (IOException ioe) {
         // handling the exception on creating a log
     }

 Open a Log
 ~~~~~~~~~~

 A `DistributedLogManager` handle will be returned when opening a log by `#openLog(logName)`. The
 handle could be used for writing data to or reading data from the log. If the log doesn't exist
 and `createStreamIfNotExists` is set to true in the configuration, the log will be created
 automatically when writing first record.

 ::

     DistributedLogConfiguration conf = new DistributedLogConfiguration();
     conf.setCreateStreamIfNotExists(true);
     DistributedLogNamespace namespace = DistributedLogNamespace.newBuilder()
         .conf(conf)
         ...
         .build();
     DistributedLogManager logManager = namespace.openLog("test-log");
     // use the log manager to open writer to write data or open reader to read data
     ...

 Sometimes, applications may open a log with different configuration settings. It could be done via
 a overloaded `#openLog` method, as below:

 ::

     DistributedLogConfiguration conf = new DistributedLogConfiguration();
     // set the retention period hours to 24 hours.
     conf.setRetentionPeriodHours(24);
     URI uri = ...;
     DistributedLogNamespace namespace = DistributedLogNamespace.newBuilder()
         .conf(conf)
         .uri(uri)
         ...
         .build();

     // Per Log Configuration
     DistributedLogConfigration logConf = new DistributedLogConfiguration();
     // set the retention period hours to 12 hours for a single stream
     logConf.setRetentionPeriodHours(12);

     // open the log with overrided settings
     DistributedLogManager logManager = namespace.openLog("test-log",
         Optional.of(logConf),
         Optiona.absent());

 Delete a Log
 ~~~~~~~~~~~~

 `DistributedLogNamespace#deleteLog(logName)` will deletes the log from the namespace. Deleting a log
 will attempt acquiring a lock before deletion. If a log is writing by an active writer, the lock
 would be already acquired by the writer. so the deleting will fail.

 ::

     DistributedLogNamespace namespace = ...;
     try {
         namespace.deleteLog("test-log");
     } catch (IOException ioe) {
         // handle the exceptions
     }

 Log Existence
 ~~~~~~~~~~~~~

 Applications could check whether a log exists in a namespace by calling `DistributedLogNamespace#logExists(logName)`.

 ::

     DistributedLogNamespace namespace = ...;
     if (namespace.logExists("test-log")) {
         // actions when log exists
     } else {
         // actions when log doesn't exist
     }

 Get List of Logs
 ~~~~~~~~~~~~~~~~

 Applications could list the logs under a namespace by calling `DistributedLogNamespace#getLogs()`.

 ::

     DistributedLogNamespace namespace = ...;
     Iterator<String> logs = namespace.getLogs();
     while (logs.hasNext()) {
         String logName = logs.next();
         // ... process the log
     }

 Writer API
 ----------

 There are two ways to write records into a log stream, one is using 'synchronous' `LogWriter`, while the other one is using
 asynchronous `AsyncLogWriter`.

 LogWriter
 ~~~~~~~~~

 The first thing to write data into a log stream is to construct the writer instance. Please note that the distributedlog core library enforce single-writer
 semantic by deploying a zookeeper locking mechanism. If there is only an active writer, the subsequent calls to `#startLogSegmentNonPartitioned()` will
 fail with `OwnershipAcquireFailedException`.

 ::

     DistributedLogNamespace namespace = ....;
     DistributedLogManager dlm = namespace.openLog("test-log");
     LogWriter writer = dlm.startLogSegmentNonPartitioned();

 .. _Construct Log Record:

 Log records are constructed to represent the data written to a log stream. Each log record is associated with application defined transaction id.
 The transaction id has to be non-decreasing otherwise writing the record will be rejected with `TransactionIdOutOfOrderException`. Application is allowed to
 bypass the transaction id sanity checking by setting `maxIdSanityCheck` to false in configuration. System time and atomic numbers are good candicates used for
 transaction id.

 ::

     long txid = 1L;
     byte[] data = ...;
     LogRecord record = new LogRecord(txid, data);

 Application could either add a single record (via `#write(LogRecord)`) or a bunch of records (via `#writeBulk(List<LogRecord>)`) into the log stream.

 ::

     writer.write(record);
     // or
     List<LogRecord> records = Lists.newArrayList();
     records.add(record);
     writer.writeBulk(records);

 The write calls return immediately after the records are added into the output buffer of writer. So the data isn't guaranteed to be durable until writer
 explicitly calls `#setReadyToFlush()` and `#flushAndSync()`. Those two calls will first transmit buffered data to backend, wait for transmit acknowledges
 and commit the written data to make them visible to readers.

 ::

     // flush the records
     writer.setReadyToFlush();
     // commit the records to make them visible to readers
     writer.flushAndSync();

 The DL log streams are endless streams unless they are sealed. 'endless' means that writers keep writing records to the log streams, readers could keep
 tailing reading from the end of the streams and it never stops. Application could seal a log stream by calling `#markEndOfStream()`.

 ::

     // seal the log stream
     writer.markEndOfStream();


 The complete example of writing records is showed as below.

 ::

     DistributedLogNamespace namespace = ....;
     DistributedLogManager dlm = namespace.openLog("test-log");

     LogWriter writer = dlm.startLogSegmentNonPartitioned();
     for (long txid = 1L; txid <= 100L; txid++) {
         byte[] data = ...;
         LogRecord record = new LogRecord(txid, data);
         writer.write(record);
     }
     // flush the records
     writer.setReadyToFlush();
     // commit the records to make them visible to readers
     writer.flushAndSync();

     // seal the log stream
     writer.markEndOfStream();

 AsyncLogWriter
 ~~~~~~~~~~~~~~

 Constructing an asynchronous `AsyncLogWriter` is as simple as synchronous `LogWriter`.

 ::

     DistributedLogNamespace namespace = ....;
     DistributedLogManager dlm = namespace.openLog("test-log");
     AsyncLogWriter writer = dlm.startAsyncLogSegmentNonPartitioned();

 All the writes to `AsyncLogWriter` are asynchronous. The futures representing write results are only satisfied when the data are persisted in the stream durably.
 A DLSN (distributedlog sequence number) will be returned for each write, which is used to represent the position (aka offset) of the record in the log stream.
 All the records adding in order are guaranteed to be persisted in order.

 .. _Async Write Records:

 ::

     List<Future<DLSN>> addFutures = Lists.newArrayList();
     for (long txid = 1L; txid <= 100L; txid++) {
         byte[] data = ...;
         LogRecord record = new LogRecord(txid, data);
         addFutures.add(writer.write(record));
     }
     List<DLSN> addResults = Await.result(Future.collect(addFutures));

 The `AsyncLogWriter` also provides the method to truncate a stream to a given DLSN. This is super helpful for building replicated state machines, who need
 explicit controls on when the data could be deleted.

 ::

     DLSN truncateDLSN = ...;
     Future<DLSN> truncateFuture = writer.truncate(truncateDLSN);
     // wait for truncation result
     Await.result(truncateFuture);

 Reader API
 ----------

 Sequence Numbers
 ~~~~~~~~~~~~~~~~

 A log record is associated with sequence numbers. First of all, application can assign its own sequence number (called `TransactionID`)
 to the log record while writing it (see `Construct Log Record`_). Secondly, a log record will be assigned with an unique system generated sequence number
 `DLSN` (distributedlog sequence number) when it is written to a log (see `Async Write Records`_). Besides `DLSN` and `TransactionID`,
 a monotonically increasing 64-bits `SequenceId` is assigned to the record at read time, indicating its position within the log.

 :Transaction ID: Transaction ID is a positive 64-bits long number that is assigned by the application.
     Transaction ID is very helpful when application wants to organize the records and position the readers using their own sequencing method. A typical
     use case of `Transaction ID` is `DistributedLog Write Proxy`. The write proxy assigns non-decreasing timestamps to log records, which the timestamps
     could be used as `physical time` to implement `TTL` (Time To Live) feature in a strong consistent database.
 :DLSN: DLSN (DistributedLog Sequence Number) is the sequence number generated during written time.
     DLSN is comparable and could be used to figure out the order between records. A DLSN is comprised with 3 components. They are `Log Segment Sequence Number`,
     `Entry Id` and `Slot Id`. The DLSN is usually used for comparison, positioning and truncation.
 :Sequence ID: Sequence ID is introduced to address the drawback of `DLSN`, in favor of answering questions like `how many records written between two DLSNs`.
     Sequence ID is a 64-bits monotonic increasing number starting from zero. The sequence ids are computed during reading, and only accessible by readers.
     That means writers don't know the sequence ids of records at the point they wrote them.

 The readers could be positioned to start reading from any positions in the log, by using `DLSN` or `Transaction ID`.

 LogReader
 ~~~~~~~~~

 `LogReader` is a 'synchronous' sequential reader reading records from a log stream starting from a given position. The position could be
 `DLSN` (via `#getInputStream(DLSN)`) or `Transaction ID` (via `#getInputStream(long)`). After the reader is open, it could call either
 `#readNext(boolean)` or `#readBulk(boolean, int)` to read records out of the log stream sequentially. Closing the reader (via `#close()`)
 will release all the resources occupied by this reader instance.

 Exceptions could be thrown during reading records due to various issues. Once the exception is thrown, the reader is set to an error state
 and it isn't usable anymore. It is the application's responsibility to handle the exceptions and re-create readers if necessary.

 ::

     DistributedLogManager dlm = ...;
     long nextTxId = ...;
     LogReader reader = dlm.getInputStream(nextTxId);

     while (true) { // keep reading & processing records
         LogRecord record;
         try {
             record = reader.readNext(false);
             nextTxId = record.getTransactionId();
             // process the record
             ...
         } catch (IOException ioe) {
             // handle the exception
             ...
             reader = dlm.getInputStream(nextTxId + 1);
         }
     }

 Reading records from an endless log stream in `synchronous` way isn't as trivial as in `asynchronous` way. Because it lacks of callback mechanism.
 Instead, `LogReader` introduces a flag `nonBlocking` on controlling the waiting behavior on `synchronous` reads. Blocking (`nonBlocking = false`)
 means the reads will wait for records before returning read calls, while NonBlocking (`nonBlocking = true`) means the reads will only check readahead
 cache and return whatever records available in readahead cache.

 The `waiting` period varies in `blocking` mode. If the reader is catching up with writer (there are plenty of records in the log), the read call will
 wait until records are read and returned. If the reader is already caught up with writer (there are no more records in the log at read time), the read
 call will wait for a small period of time (defined in `DistributedLogConfiguration#getReadAheadWaitTime()`) and return whatever records available in
 readahead cache. In other words, if a reader sees no record on blocking reads, it means the reader is `caught-up` with the writer.

 See examples below on how to read records using `LogReader`.

 ::

     // Read individual records

     LogReader reader = ...;
     // keep reading records in blocking mode until no records available in the log
     LogRecord record = reader.readNext(false);
     while (null != record) {
         // process the record
         ...
         // read next record
         record = reader.readNext(false);
     }
     ...

     // reader is caught up with writer, doing non-blocking reads to tail the log
     while (true) {
         record = reader.readNext(true);
         if (null == record) {
             // no record available yet. backoff ?
             ...
         } else {
             // process the new record
             ...
         }
     }

 ::

     // Read records in batch

     LogReader reader = ...;
     int N = 10;

     // keep reading N records in blocking mode until no records available in the log
     List<LogRecord> records = reader.readBulk(false, N);
     while (!records.isEmpty()) {
         // process the list of records
         ...
         if (records.size() < N) { // no more records available in the log
             break;
         }
         // read next N records
         records = reader.readBulk(false, N);
     }

     ...

     // reader is caught up with writer, doing non-blocking reads to tail the log
     while (true) {
         records = reader.readBulk(true, N);
         // process the new records
         ...
     }


 AsyncLogReader
 ~~~~~~~~~~~~~~

 Similar as `LogReader`, applications could open an `AsyncLogReader` by positioning to different positions, either `DLSN` or `Transaction ID`.

 ::

     DistributedLogManager dlm = ...;

     Future<AsyncLogReader> openFuture;

     // position the reader to transaction id `999`
     openFuture = dlm.openAsyncLogReader(999L);

     // or position the reader to DLSN
     DLSN fromDLSN = ...;
     openFuture = dlm.openAsyncLogReader(fromDLSN);

     AsyncLogReader reader = Await.result(openFuture);

 Reading records from an `AsyncLogReader` is asynchronously. The future returned by `#readNext()`, `#readBulk(int)` or `#readBulk(int, long, TimeUnit)`
 represents the result of the read operation. The future is only satisfied when there are records available. Application could chain the futures
 to do sequential reads.

 Reading records one by one from an `AsyncLogReader`.

 ::

     void readOneRecord(AsyncLogReader reader) {
         reader.readNext().addEventListener(new FutureEventListener<LogRecordWithDLSN>() {
             public void onSuccess(LogRecordWithDLSN record) {
                 // process the record
                 ...
                 // read next
                 readOneRecord(reader);
             }
             public void onFailure(Throwable cause) {
                 // handle errors and re-create reader
                 ...
                 reader = ...;
                 // read next
                 readOneRecord(reader);
             }
         });
     }

     AsyncLogReader reader = ...;
     readOneRecord(reader);

 Reading records in batches from an `AsyncLogReader`.

 ::

     void readBulk(AsyncLogReader reader, int N) {
         reader.readBulk(N).addEventListener(new FutureEventListener<List<LogRecordWithDLSN>>() {
             public void onSuccess(List<LogRecordWithDLSN> records) {
                 // process the records
                 ...
                 // read next
                 readBulk(reader, N);
             }
             public void onFailure(Throwable cause) {
                 // handle errors and re-create reader
                 ...
                 reader = ...;
                 // read next
                 readBulk(reader, N);
             }
         });
     }

     AsyncLogReader reader = ...;
     readBulk(reader, N);
	---
	title: "Core Library"
	layout: default

	# Sub-level navigation
	sub-nav-group: user-guide
	sub-nav-parent: apis
	sub-nav-pos: 3
	sub-nav-title: Core Library API

	---

	.. contents:: Core Library API

	Core Library API
	================

	The distributedlog core library interacts with namespaces and logs directly.
	It is written in Java.

	Namespace API
	-------------

	A DL namespace is a collection of log streams. Applications could create
	or delete logs under a DL namespace.

	Namespace URI
	~~~~~~~~~~~~~

	An URI is used to locate the namespace. The Namespace URI is typically
	comprised of 3 components:

	* scheme: `distributedlog-<backend>`. The backend indicates what backend is used to store the log data.
	* domain name: the domain name that used to talk to the backend. In the example as below, the domain name part is zookeeper server, which is used to store log metadata in bookkeeper based backend implementation.
	* path: path points to the location that stores logs. In the example as below, it is a zookeeper path that points to the znode that stores all the logs metadata.

	::

	distributedlog-bk://<zookeeper-server>/path/to/stream

	The available backend is only bookkeeper based backend.
	The default `distributedlog` scheme is aliased to `distributedlog-bk`.

	Building a Namespace
	~~~~~~~~~~~~~~~~~~~~

	Once you have the namespace uri, you could build the namespace instance.
	The namespace instance will be used for operating streams under it.

	::

	// DistributedLog Configuration
	DistributedLogConfiguration conf = new DistributedLogConfiguration();
	// Namespace URI
	URI uri = ...; // create the namespace uri
	// create a builder to build namespace instances
	DistributedLogNamespaceBuilder builder = DistributedLogNamespaceBuilder.newBuilder();
	DistributedLogNamespace namespace = builder
	.conf(conf) // configuration that used by namespace
	.uri(uri) // namespace uri
	.statsLogger(...) // stats logger to log stats
	.featureProvider(...) // feature provider on controlling features
	.build();

	Create a Log
	~~~~~~~~~~~~

	Creating a log is pretty straight forward by calling `distributedlognamespace#createlog(logname)`.
	it only creates the log under the namespace but doesn't return any handle for operating the log.

	::

	DistributedLogNamespace namespace = ...; // namespace
	try {
	namespace.createLog("test-log");
	} catch (IOException ioe) {
	// handling the exception on creating a log
	}

	Open a Log
	~~~~~~~~~~

	A `DistributedLogManager` handle will be returned when opening a log by `#openLog(logName)`. The
	handle could be used for writing data to or reading data from the log. If the log doesn't exist
	and `createStreamIfNotExists` is set to true in the configuration, the log will be created
	automatically when writing first record.

	::

	DistributedLogConfiguration conf = new DistributedLogConfiguration();
	conf.setCreateStreamIfNotExists(true);
	DistributedLogNamespace namespace = DistributedLogNamespace.newBuilder()
	.conf(conf)
	...
	.build();
	DistributedLogManager logManager = namespace.openLog("test-log");
	// use the log manager to open writer to write data or open reader to read data
	...

	Sometimes, applications may open a log with different configuration settings. It could be done via
	a overloaded `#openLog` method, as below:

	::

	DistributedLogConfiguration conf = new DistributedLogConfiguration();
	// set the retention period hours to 24 hours.
	conf.setRetentionPeriodHours(24);
	URI uri = ...;
	DistributedLogNamespace namespace = DistributedLogNamespace.newBuilder()
	.conf(conf)
	.uri(uri)
	...
	.build();

	// Per Log Configuration
	DistributedLogConfigration logConf = new DistributedLogConfiguration();
	// set the retention period hours to 12 hours for a single stream
	logConf.setRetentionPeriodHours(12);

	// open the log with overrided settings
	DistributedLogManager logManager = namespace.openLog("test-log",
	Optional.of(logConf),
	Optiona.absent());

	Delete a Log
	~~~~~~~~~~~~

	`DistributedLogNamespace#deleteLog(logName)` will deletes the log from the namespace. Deleting a log
	will attempt acquiring a lock before deletion. If a log is writing by an active writer, the lock
	would be already acquired by the writer. so the deleting will fail.

	::

	DistributedLogNamespace namespace = ...;
	try {
	namespace.deleteLog("test-log");
	} catch (IOException ioe) {
	// handle the exceptions
	}

	Log Existence
	~~~~~~~~~~~~~

	Applications could check whether a log exists in a namespace by calling `DistributedLogNamespace#logExists(logName)`.

	::

	DistributedLogNamespace namespace = ...;
	if (namespace.logExists("test-log")) {
	// actions when log exists
	} else {
	// actions when log doesn't exist
	}

	Get List of Logs
	~~~~~~~~~~~~~~~~

	Applications could list the logs under a namespace by calling `DistributedLogNamespace#getLogs()`.

	::

	DistributedLogNamespace namespace = ...;
	Iterator<String> logs = namespace.getLogs();
	while (logs.hasNext()) {
	String logName = logs.next();
	// ... process the log
	}

	Writer API
	----------

	There are two ways to write records into a log stream, one is using 'synchronous' `LogWriter`, while the other one is using
	asynchronous `AsyncLogWriter`.

	LogWriter
	~~~~~~~~~

	The first thing to write data into a log stream is to construct the writer instance. Please note that the distributedlog core library enforce single-writer
	semantic by deploying a zookeeper locking mechanism. If there is only an active writer, the subsequent calls to `#startLogSegmentNonPartitioned()` will
	fail with `OwnershipAcquireFailedException`.

	::

	DistributedLogNamespace namespace = ....;
	DistributedLogManager dlm = namespace.openLog("test-log");
	LogWriter writer = dlm.startLogSegmentNonPartitioned();

	.. _Construct Log Record:

	Log records are constructed to represent the data written to a log stream. Each log record is associated with application defined transaction id.
	The transaction id has to be non-decreasing otherwise writing the record will be rejected with `TransactionIdOutOfOrderException`. Application is allowed to
	bypass the transaction id sanity checking by setting `maxIdSanityCheck` to false in configuration. System time and atomic numbers are good candicates used for
	transaction id.

	::

	long txid = 1L;
	byte[] data = ...;
	LogRecord record = new LogRecord(txid, data);

	Application could either add a single record (via `#write(LogRecord)`) or a bunch of records (via `#writeBulk(List<LogRecord>)`) into the log stream.

	::

	writer.write(record);
	// or
	List<LogRecord> records = Lists.newArrayList();
	records.add(record);
	writer.writeBulk(records);

	The write calls return immediately after the records are added into the output buffer of writer. So the data isn't guaranteed to be durable until writer
	explicitly calls `#setReadyToFlush()` and `#flushAndSync()`. Those two calls will first transmit buffered data to backend, wait for transmit acknowledges
	and commit the written data to make them visible to readers.

	::

	// flush the records
	writer.setReadyToFlush();
	// commit the records to make them visible to readers
	writer.flushAndSync();

	The DL log streams are endless streams unless they are sealed. 'endless' means that writers keep writing records to the log streams, readers could keep
	tailing reading from the end of the streams and it never stops. Application could seal a log stream by calling `#markEndOfStream()`.

	::

	// seal the log stream
	writer.markEndOfStream();


	The complete example of writing records is showed as below.

	::

	DistributedLogNamespace namespace = ....;
	DistributedLogManager dlm = namespace.openLog("test-log");

	LogWriter writer = dlm.startLogSegmentNonPartitioned();
	for (long txid = 1L; txid <= 100L; txid++) {
	byte[] data = ...;
	LogRecord record = new LogRecord(txid, data);
	writer.write(record);
	}
	// flush the records
	writer.setReadyToFlush();
	// commit the records to make them visible to readers
	writer.flushAndSync();

	// seal the log stream
	writer.markEndOfStream();

	AsyncLogWriter
	~~~~~~~~~~~~~~

	Constructing an asynchronous `AsyncLogWriter` is as simple as synchronous `LogWriter`.

	::

	DistributedLogNamespace namespace = ....;
	DistributedLogManager dlm = namespace.openLog("test-log");
	AsyncLogWriter writer = dlm.startAsyncLogSegmentNonPartitioned();

	All the writes to `AsyncLogWriter` are asynchronous. The futures representing write results are only satisfied when the data are persisted in the stream durably.
	A DLSN (distributedlog sequence number) will be returned for each write, which is used to represent the position (aka offset) of the record in the log stream.
	All the records adding in order are guaranteed to be persisted in order.

	.. _Async Write Records:

	::

	List<Future<DLSN>> addFutures = Lists.newArrayList();
	for (long txid = 1L; txid <= 100L; txid++) {
	byte[] data = ...;
	LogRecord record = new LogRecord(txid, data);
	addFutures.add(writer.write(record));
	}
	List<DLSN> addResults = Await.result(Future.collect(addFutures));

	The `AsyncLogWriter` also provides the method to truncate a stream to a given DLSN. This is super helpful for building replicated state machines, who need
	explicit controls on when the data could be deleted.

	::

	DLSN truncateDLSN = ...;
	Future<DLSN> truncateFuture = writer.truncate(truncateDLSN);
	// wait for truncation result
	Await.result(truncateFuture);

	Reader API
	----------

	Sequence Numbers
	~~~~~~~~~~~~~~~~

	A log record is associated with sequence numbers. First of all, application can assign its own sequence number (called `TransactionID`)
	to the log record while writing it (see `Construct Log Record`_). Secondly, a log record will be assigned with an unique system generated sequence number
	`DLSN` (distributedlog sequence number) when it is written to a log (see `Async Write Records`_). Besides `DLSN` and `TransactionID`,
	a monotonically increasing 64-bits `SequenceId` is assigned to the record at read time, indicating its position within the log.

	:Transaction ID: Transaction ID is a positive 64-bits long number that is assigned by the application.
	Transaction ID is very helpful when application wants to organize the records and position the readers using their own sequencing method. A typical
	use case of `Transaction ID` is `DistributedLog Write Proxy`. The write proxy assigns non-decreasing timestamps to log records, which the timestamps
	could be used as `physical time` to implement `TTL` (Time To Live) feature in a strong consistent database.
	:DLSN: DLSN (DistributedLog Sequence Number) is the sequence number generated during written time.
	DLSN is comparable and could be used to figure out the order between records. A DLSN is comprised with 3 components. They are `Log Segment Sequence Number`,
	`Entry Id` and `Slot Id`. The DLSN is usually used for comparison, positioning and truncation.
	:Sequence ID: Sequence ID is introduced to address the drawback of `DLSN`, in favor of answering questions like `how many records written between two DLSNs`.
	Sequence ID is a 64-bits monotonic increasing number starting from zero. The sequence ids are computed during reading, and only accessible by readers.
	That means writers don't know the sequence ids of records at the point they wrote them.

	The readers could be positioned to start reading from any positions in the log, by using `DLSN` or `Transaction ID`.

	LogReader
	~~~~~~~~~

	`LogReader` is a 'synchronous' sequential reader reading records from a log stream starting from a given position. The position could be
	`DLSN` (via `#getInputStream(DLSN)`) or `Transaction ID` (via `#getInputStream(long)`). After the reader is open, it could call either
	`#readNext(boolean)` or `#readBulk(boolean, int)` to read records out of the log stream sequentially. Closing the reader (via `#close()`)
	will release all the resources occupied by this reader instance.

	Exceptions could be thrown during reading records due to various issues. Once the exception is thrown, the reader is set to an error state
	and it isn't usable anymore. It is the application's responsibility to handle the exceptions and re-create readers if necessary.

	::

	DistributedLogManager dlm = ...;
	long nextTxId = ...;
	LogReader reader = dlm.getInputStream(nextTxId);

	while (true) { // keep reading & processing records
	LogRecord record;
	try {
	record = reader.readNext(false);
	nextTxId = record.getTransactionId();
	// process the record
	...
	} catch (IOException ioe) {
	// handle the exception
	...
	reader = dlm.getInputStream(nextTxId + 1);
	}
	}

	Reading records from an endless log stream in `synchronous` way isn't as trivial as in `asynchronous` way. Because it lacks of callback mechanism.
	Instead, `LogReader` introduces a flag `nonBlocking` on controlling the waiting behavior on `synchronous` reads. Blocking (`nonBlocking = false`)
	means the reads will wait for records before returning read calls, while NonBlocking (`nonBlocking = true`) means the reads will only check readahead
	cache and return whatever records available in readahead cache.

	The `waiting` period varies in `blocking` mode. If the reader is catching up with writer (there are plenty of records in the log), the read call will
	wait until records are read and returned. If the reader is already caught up with writer (there are no more records in the log at read time), the read
	call will wait for a small period of time (defined in `DistributedLogConfiguration#getReadAheadWaitTime()`) and return whatever records available in
	readahead cache. In other words, if a reader sees no record on blocking reads, it means the reader is `caught-up` with the writer.

	See examples below on how to read records using `LogReader`.

	::

	// Read individual records

	LogReader reader = ...;
	// keep reading records in blocking mode until no records available in the log
	LogRecord record = reader.readNext(false);
	while (null != record) {
	// process the record
	...
	// read next record
	record = reader.readNext(false);
	}
	...

	// reader is caught up with writer, doing non-blocking reads to tail the log
	while (true) {
	record = reader.readNext(true);
	if (null == record) {
	// no record available yet. backoff ?
	...
	} else {
	// process the new record
	...
	}
	}

	::

	// Read records in batch

	LogReader reader = ...;
	int N = 10;

	// keep reading N records in blocking mode until no records available in the log
	List<LogRecord> records = reader.readBulk(false, N);
	while (!records.isEmpty()) {
	// process the list of records
	...
	if (records.size() < N) { // no more records available in the log
	break;
	}
	// read next N records
	records = reader.readBulk(false, N);
	}

	...

	// reader is caught up with writer, doing non-blocking reads to tail the log
	while (true) {
	records = reader.readBulk(true, N);
	// process the new records
	...
	}


	AsyncLogReader
	~~~~~~~~~~~~~~

	Similar as `LogReader`, applications could open an `AsyncLogReader` by positioning to different positions, either `DLSN` or `Transaction ID`.

	::

	DistributedLogManager dlm = ...;

	Future<AsyncLogReader> openFuture;

	// position the reader to transaction id `999`
	openFuture = dlm.openAsyncLogReader(999L);

	// or position the reader to DLSN
	DLSN fromDLSN = ...;
	openFuture = dlm.openAsyncLogReader(fromDLSN);

	AsyncLogReader reader = Await.result(openFuture);

	Reading records from an `AsyncLogReader` is asynchronously. The future returned by `#readNext()`, `#readBulk(int)` or `#readBulk(int, long, TimeUnit)`
	represents the result of the read operation. The future is only satisfied when there are records available. Application could chain the futures
	to do sequential reads.

	Reading records one by one from an `AsyncLogReader`.

	::

	void readOneRecord(AsyncLogReader reader) {
	reader.readNext().addEventListener(new FutureEventListener<LogRecordWithDLSN>() {
	public void onSuccess(LogRecordWithDLSN record) {
	// process the record
	...
	// read next
	readOneRecord(reader);
	}
	public void onFailure(Throwable cause) {
	// handle errors and re-create reader
	...
	reader = ...;
	// read next
	readOneRecord(reader);
	}
	});
	}

	AsyncLogReader reader = ...;
	readOneRecord(reader);

	Reading records in batches from an `AsyncLogReader`.

	::

	void readBulk(AsyncLogReader reader, int N) {
	reader.readBulk(N).addEventListener(new FutureEventListener<List<LogRecordWithDLSN>>() {
	public void onSuccess(List<LogRecordWithDLSN> records) {
	// process the records
	...
	// read next
	readBulk(reader, N);
	}
	public void onFailure(Throwable cause) {
	// handle errors and re-create reader
	...
	reader = ...;
	// read next
	readBulk(reader, N);
	}
	});
	}

	AsyncLogReader reader = ...;
	readBulk(reader, N);