hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/README.txt

This module provides a BookKeeper backend for HFDS Namenode write
ahead logging.  

BookKeeper is a highly available distributed write ahead logging
system. For more details, see
   
    http://zookeeper.apache.org/bookkeeper

-------------------------------------------------------------------------------
How do I build?

 To generate the distribution packages for BK journal, do the
 following.

   $ mvn clean package -Pdist

 This will generate a jar with all the dependencies needed by the journal
 manager, 

 target/hadoop-hdfs-bkjournal-<VERSION>.jar

 Note that the -Pdist part of the build command is important, as otherwise
 the dependencies would not be packaged in the jar. 

-------------------------------------------------------------------------------
How do I use the BookKeeper Journal?

 To run a HDFS namenode using BookKeeper as a backend, copy the bkjournal
 jar, generated above, into the lib directory of hdfs. In the standard 
 distribution of HDFS, this is at $HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/

  cp target/hadoop-hdfs-bkjournal-<VERSION>.jar \
    $HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/

 Then, in hdfs-site.xml, set the following properties.

   <property>
     <name>dfs.namenode.edits.dir</name>
     <value>bookkeeper://localhost:2181/bkjournal,file:///path/for/edits</value>
   </property>

   <property>
     <name>dfs.namenode.edits.journal-plugin.bookkeeper</name>
     <value>org.apache.hadoop.contrib.bkjournal.BookKeeperJournalManager</value>
   </property>

 In this example, the namenode is configured to use 2 write ahead
 logging devices. One writes to BookKeeper and the other to a local
 file system. At the moment is is not possible to only write to 
 BookKeeper, as the resource checker explicitly checked for local
 disks currently.

 The given example, configures the namenode to look for the journal
 metadata at the path /bkjournal on the a standalone zookeeper ensemble
 at localhost:2181. To configure a multiple host zookeeper ensemble,
 separate the hosts with semicolons. For example, if you have 3
 zookeeper servers, zk1, zk2 & zk3, each listening on port 2181, you
 would specify this with 
  
   bookkeeper://zk1:2181;zk2:2181;zk3:2181/bkjournal

 The final part /bkjournal specifies the znode in zookeeper where
 ledger metadata will be store. Administrators can set this to anything
 they wish.