Google Git
Sign in
apache / zookeeper / refs/heads/branch-3.3 / . / docs / zookeeperStarted.html
blob: 09504964810c2658c7e297f99bc95e239a1cc5d1 [file] [log] [blame]
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta content="Apache Forrest" name="Generator">
<meta name="Forrest-version" content="0.9">
<meta name="Forrest-skin-name" content="pelt">
<title>ZooKeeper Getting Started Guide</title>
<link type="text/css" href="skin/basic.css" rel="stylesheet">
<link media="screen" type="text/css" href="skin/screen.css" rel="stylesheet">
<link media="print" type="text/css" href="skin/print.css" rel="stylesheet">
<link type="text/css" href="skin/profile.css" rel="stylesheet">
<script src="skin/getBlank.js" language="javascript" type="text/javascript"></script><script src="skin/getMenu.js" language="javascript" type="text/javascript"></script><script src="skin/fontsize.js" language="javascript" type="text/javascript"></script>
<link rel="shortcut icon" href="images/favicon.ico">
</head>
<body onload="init()">
<script type="text/javascript">ndeSetTextSize();</script>
<div id="top">
<!--+
|breadtrail
+-->
<div class="breadtrail">
<a href="http://www.apache.org/">Apache</a> &gt; <a href="http://hadoop.apache.org/">Hadoop</a> &gt; <a href="http://hadoop.apache.org/zookeeper/">ZooKeeper</a><script src="skin/breadcrumbs.js" language="JavaScript" type="text/javascript"></script>
</div>
<!--+
|header
+-->
<div class="header">
<!--+
|start group logo
+-->
<div class="grouplogo">
<a href="http://hadoop.apache.org/"><img class="logoImage" alt="Hadoop" src="images/hadoop-logo.jpg" title="Apache Hadoop"></a>
</div>
<!--+
|end group logo
+-->
<!--+
|start Project Logo
+-->
<div class="projectlogo">
<a href="http://hadoop.apache.org/zookeeper/"><img class="logoImage" alt="ZooKeeper" src="images/zookeeper_small.gif" title="ZooKeeper: distributed coordination"></a>
</div>
<!--+
|end Project Logo
+-->
<!--+
|start Search
+-->
<div class="searchbox">
<form action="http://www.google.com/search" method="get" class="roundtopsmall">
<input value="hadoop.apache.org" name="sitesearch" type="hidden"><input onFocus="getBlank (this, 'Search the site with google');" size="25" name="q" id="query" type="text" value="Search the site with google">&nbsp;
<input name="Search" value="Search" type="submit">
</form>
</div>
<!--+
|end search
+-->
<!--+
|start Tabs
+-->
<ul id="tabs">
<li>
<a class="unselected" href="http://hadoop.apache.org/zookeeper/">Project</a>
</li>
<li>
<a class="unselected" href="http://wiki.apache.org/hadoop/ZooKeeper">Wiki</a>
</li>
<li class="current">
<a class="selected" href="index.html">ZooKeeper 3.3 Documentation</a>
</li>
</ul>
<!--+
|end Tabs
+-->
</div>
</div>
<div id="main">
<div id="publishedStrip">
<!--+
|start Subtabs
+-->
<div id="level2tabs"></div>
<!--+
|end Endtabs
+-->
<script type="text/javascript"><!--
document.write("Last Published: " + document.lastModified);
// --></script>
</div>
<!--+
|breadtrail
+-->
<div class="breadtrail">
&nbsp;
</div>
<!--+
|start Menu, mainarea
+-->
<!--+
|start Menu
+-->
<div id="menu">
<div onclick="SwitchMenu('menu_selected_1.1', 'skin/')" id="menu_selected_1.1Title" class="menutitle" style="background-image: url('skin/images/chapter_open.gif');">Overview</div>
<div id="menu_selected_1.1" class="selectedmenuitemgroup" style="display: block;">
<div class="menuitem">
<a href="index.html">Welcome</a>
</div>
<div class="menuitem">
<a href="zookeeperOver.html">Overview</a>
</div>
<div class="menupage">
<div class="menupagetitle">Getting Started</div>
</div>
<div class="menuitem">
<a href="releasenotes.html">Release Notes</a>
</div>
</div>
<div onclick="SwitchMenu('menu_1.2', 'skin/')" id="menu_1.2Title" class="menutitle">Developer</div>
<div id="menu_1.2" class="menuitemgroup">
<div class="menuitem">
<a href="api/index.html">API Docs</a>
</div>
<div class="menuitem">
<a href="zookeeperProgrammers.html">Programmer's Guide</a>
</div>
<div class="menuitem">
<a href="javaExample.html">Java Example</a>
</div>
<div class="menuitem">
<a href="zookeeperTutorial.html">Barrier and Queue Tutorial</a>
</div>
<div class="menuitem">
<a href="recipes.html">Recipes</a>
</div>
</div>
<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">BookKeeper</div>
<div id="menu_1.3" class="menuitemgroup">
<div class="menuitem">
<a href="bookkeeperStarted.html">Getting started</a>
</div>
<div class="menuitem">
<a href="bookkeeperOverview.html">Overview</a>
</div>
<div class="menuitem">
<a href="bookkeeperConfig.html">Setup guide</a>
</div>
<div class="menuitem">
<a href="bookkeeperProgrammer.html">Programmer's guide</a>
</div>
</div>
<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Admin &amp; Ops</div>
<div id="menu_1.4" class="menuitemgroup">
<div class="menuitem">
<a href="zookeeperAdmin.html">Administrator's Guide</a>
</div>
<div class="menuitem">
<a href="zookeeperQuotas.html">Quota Guide</a>
</div>
<div class="menuitem">
<a href="zookeeperJMX.html">JMX</a>
</div>
<div class="menuitem">
<a href="zookeeperObservers.html">Observers Guide</a>
</div>
</div>
<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Contributor</div>
<div id="menu_1.5" class="menuitemgroup">
<div class="menuitem">
<a href="zookeeperInternals.html">ZooKeeper Internals</a>
</div>
</div>
<div onclick="SwitchMenu('menu_1.6', 'skin/')" id="menu_1.6Title" class="menutitle">Miscellaneous</div>
<div id="menu_1.6" class="menuitemgroup">
<div class="menuitem">
<a href="http://wiki.apache.org/hadoop/ZooKeeper">Wiki</a>
</div>
<div class="menuitem">
<a href="http://wiki.apache.org/hadoop/ZooKeeper/FAQ">FAQ</a>
</div>
<div class="menuitem">
<a href="http://hadoop.apache.org/zookeeper/mailing_lists.html">Mailing Lists</a>
</div>
</div>
<div id="credit"></div>
<div id="roundbottom">
<img style="display: none" class="corner" height="15" width="15" alt="" src="skin/images/rc-b-l-15-1body-2menu-3menu.png"></div>
<!--+
|alternative credits
+-->
<div id="credit2"></div>
</div>
<!--+
|end Menu
+-->
<!--+
|start content
+-->
<div id="content">
<div title="Portable Document Format" class="pdflink">
<a class="dida" href="zookeeperStarted.pdf"><img alt="PDF -icon" src="skin/images/pdfdoc.gif" class="skin"><br>
PDF</a>
</div>
<h1>ZooKeeper Getting Started Guide</h1>
<div id="front-matter">
<div id="minitoc-area">
<ul class="minitoc">
<li>
<a href="#ch_GettingStarted">Getting Started: Coordinating Distributed Applications with
ZooKeeper</a>
<ul class="minitoc">
<li>
<a href="#sc_Prerequisites">Pre-requisites</a>
</li>
<li>
<a href="#sc_Download">Download</a>
</li>
<li>
<a href="#sc_InstallingSingleMode">Standalone Operation</a>
</li>
<li>
<a href="#sc_FileManagement">Managing ZooKeeper Storage</a>
</li>
<li>
<a href="#sc_ConnectingToZooKeeper">Connecting to ZooKeeper</a>
</li>
<li>
<a href="#sc_ProgrammingToZooKeeper">Programming to ZooKeeper</a>
</li>
<li>
<a href="#sc_RunningReplicatedZooKeeper">Running Replicated ZooKeeper</a>
</li>
<li>
<a href="#Other+Optimizations">Other Optimizations</a>
</li>
</ul>
</li>
</ul>
</div>
</div>
<a name="ch_GettingStarted"></a>
<h2 class="h3">Getting Started: Coordinating Distributed Applications with
ZooKeeper</h2>
<div class="section">
<p>This document contains information to get you started quickly with
ZooKeeper. It is aimed primarily at developers hoping to try it out, and
contains simple installation instructions for a single ZooKeeper server, a
few commands to verify that it is running, and a simple programming
example. Finally, as a convenience, there are a few sections regarding
more complicated installations, for example running replicated
deployments, and optimizing the transaction log. However for the complete
instructions for commercial deployments, please refer to the <a href="zookeeperAdmin.html">ZooKeeper
Administrator's Guide</a>.</p>
<a name="sc_Prerequisites"></a>
<h3 class="h4">Pre-requisites</h3>
<p>See <a href="zookeeperAdmin.html#sc_systemReq">
System Requirements</a> in the Admin guide.</p>
<a name="sc_Download"></a>
<h3 class="h4">Download</h3>
<p>To get a ZooKeeper distribution, download a recent
<a href="http://hadoop.apache.org/zookeeper/releases.html">
stable</a> release from one of the Apache Download
Mirrors.</p>
<a name="sc_InstallingSingleMode"></a>
<h3 class="h4">Standalone Operation</h3>
<p>Setting up a ZooKeeper server in standalone mode is
straightforward. The server is contained in a single JAR file,
so installation consists of creating a configuration.</p>
<p>Once you've downloaded a stable ZooKeeper release unpack
it and cd to the root</p>
<p>To start ZooKeeper you need a configuration file. Here is a sample,
create it in <strong>conf/zoo.cfg</strong>:</p>
<pre class="code">
tickTime=2000
dataDir=/var/zookeeper
clientPort=2181
</pre>
<p>This file can be called anything, but for the sake of this
discussion call
it <strong>conf/zoo.cfg</strong>. Change the
value of <strong>dataDir</strong> to specify an
existing (empty to start with) directory. Here are the meanings
for each of the fields:</p>
<dl>
<dt>
<term>
<strong>tickTime</strong>
</term>
</dt>
<dd>
<p>the basic time unit in milliseconds used by ZooKeeper. It is
used to do heartbeats and the minimum session timeout will be
twice the tickTime.</p>
</dd>
</dl>
<dl>
<dt>
<term>
<strong>dataDir</strong>
</term>
</dt>
<dd>
<p>the location to store the in-memory database snapshots and,
unless specified otherwise, the transaction log of updates to the
database.</p>
</dd>
<dt>
<term>
<strong>clientPort</strong>
</term>
</dt>
<dd>
<p>the port to listen for client connections</p>
</dd>
</dl>
<p>Now that you created the configuration file, you can start
ZooKeeper:</p>
<pre class="code">bin/zkServer.sh start</pre>
<p>ZooKeeper logs messages using log4j -- more detail
available in the
<a href="zookeeperProgrammers.html#Logging">Logging</a>
section of the Programmer's Guide. You will see log messages
coming to the console (default) and/or a log file depending on
the log4j configuration.</p>
<p>The steps outlined here run ZooKeeper in standalone mode. There is
no replication, so if ZooKeeper process fails, the service will go down.
This is fine for most development situations, but to run ZooKeeper in
replicated mode, please see <a href="#sc_RunningReplicatedZooKeeper">Running Replicated
ZooKeeper</a>.</p>
<a name="sc_FileManagement"></a>
<h3 class="h4">Managing ZooKeeper Storage</h3>
<p>For long running production systems ZooKeeper storage must
be managed externally (dataDir and logs). See the section on
<a href="zookeeperAdmin.html#sc_maintenance">maintenance</a> for
more details.</p>
<a name="sc_ConnectingToZooKeeper"></a>
<h3 class="h4">Connecting to ZooKeeper</h3>
<p>Once ZooKeeper is running, you have several options for connection
to it:</p>
<ul>
<li>
<p>
<strong>Java</strong>: Use</p>
<pre class="code">bin/zkCli.sh -server 127.0.0.1:2181</pre>
<p>This lets you perform simple, file-like operations.</p>
</li>
<li>
<p>
<strong>C</strong>: compile cli_mt
(multi-threaded) or cli_st (single-threaded) by running
<span class="codefrag command">make cli_mt</span> or <span class="codefrag command">make
cli_st</span> in
the <strong>src/c</strong> subdirectory in
the ZooKeeper sources. See the README contained within
<strong>src/c</strong> for full details.</p>
<p>You can run the program
from <strong>src/c</strong> using:</p>
<pre class="code">LD_LIBRARY_PATH=. cli_mt 127.0.0.1:2181</pre>
<p>or</p>
<pre class="code">LD_LIBRARY_PATH=. cli_st 127.0.0.1:2181</pre>
<p>This will give you a simple shell to execute file
system like operations on ZooKeeper.</p>
</li>
</ul>
<p>Once you have connected, you should see something like:
</p>
<pre class="code">
Connecting to localhost:2181
log4j:WARN No appenders could be found for logger (org.apache.zookeeper.ZooKeeper).
log4j:WARN Please initialize the log4j system properly.
Welcome to ZooKeeper!
JLine support is enabled
[zkshell: 0]
</pre>
<p>
From the shell, type <span class="codefrag command">help</span> to get a listing of commands that can be executed from the client, as in:
</p>
<pre class="code">
[zkshell: 0] help
ZooKeeper host:port cmd args
get path [watch]
ls path [watch]
set path data [version]
delquota [-n|-b] path
quit
printwatches on|off
createpath data acl
stat path [watch]
listquota path
history
setAcl path acl
getAcl path
sync path
redo cmdno
addauth scheme auth
delete path [version]
setquota -n|-b val path
</pre>
<p>From here, you can try a few simple commands to get a feel for this simple command line interface. First, start by issuing the list command, as
in <span class="codefrag command">ls</span>, yielding:
</p>
<pre class="code">
[zkshell: 8] ls /
[zookeeper]
</pre>
<p>Next, create a new znode by running <span class="codefrag command">create /zk_test my_data</span>. This creates a new znode and associates the string "my_data" with the node.
You should see:</p>
<pre class="code">
[zkshell: 9] create /zk_test my_data
Created /zk_test
</pre>
<p> Issue another <span class="codefrag command">ls /</span> command to see what the directory looks like:
</p>
<pre class="code">
[zkshell: 11] ls /
[zookeeper, zk_test]
</pre>
<p>
Notice that the zk_test directory has now been created.
</p>
<p>Next, verify that the data was associated with the znode by running the <span class="codefrag command">get</span> command, as in:
</p>
<pre class="code">
[zkshell: 12] get /zk_test
my_data
cZxid = 5
ctime = Fri Jun 05 13:57:06 PDT 2009
mZxid = 5
mtime = Fri Jun 05 13:57:06 PDT 2009
pZxid = 5
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0
dataLength = 7
numChildren = 0
</pre>
<p>We can change the data associated with zk_test by issuing the <span class="codefrag command">set</span> command, as in:
</p>
<pre class="code">
[zkshell: 14] set /zk_test junk
cZxid = 5
ctime = Fri Jun 05 13:57:06 PDT 2009
mZxid = 6
mtime = Fri Jun 05 14:01:52 PDT 2009
pZxid = 5
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0
dataLength = 4
numChildren = 0
[zkshell: 15] get /zk_test
junk
cZxid = 5
ctime = Fri Jun 05 13:57:06 PDT 2009
mZxid = 6
mtime = Fri Jun 05 14:01:52 PDT 2009
pZxid = 5
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0
dataLength = 4
numChildren = 0
</pre>
<p>
(Notice we did a <span class="codefrag command">get</span> after setting the data and it did, indeed, change.</p>
<p>Finally, let's <span class="codefrag command">delete</span> the node by issuing:
</p>
<pre class="code">
[zkshell: 16] delete /zk_test
[zkshell: 17] ls /
[zookeeper]
[zkshell: 18]
</pre>
<p>That's it for now. To explore more, continue with the rest of this document and see the <a href="zookeeperProgrammers.html">Programmer's Guide</a>. </p>
<a name="sc_ProgrammingToZooKeeper"></a>
<h3 class="h4">Programming to ZooKeeper</h3>
<p>ZooKeeper has a Java bindings and C bindings. They are
functionally equivalent. The C bindings exist in two variants: single
threaded and multi-threaded. These differ only in how the messaging loop
is done. For more information, see the <a href="zookeeperProgrammers.html#ch_programStructureWithExample">Programming
Examples in the ZooKeeper Programmer's Guide</a> for
sample code using of the different APIs.</p>
<a name="sc_RunningReplicatedZooKeeper"></a>
<h3 class="h4">Running Replicated ZooKeeper</h3>
<p>Running ZooKeeper in standalone mode is convenient for evaluation,
some development, and testing. But in production, you should run
ZooKeeper in replicated mode. A replicated group of servers in the same
application is called a <em>quorum</em>, and in replicated
mode, all servers in the quorum have copies of the same configuration
file. The file is similar to the one used in standalone mode, but with a
few differences. Here is an example:</p>
<pre class="code">
tickTime=2000
dataDir=/var/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3:2888:3888
</pre>
<p>The new entry, <strong>initLimit</strong> is
timeouts ZooKeeper uses to limit the length of time the ZooKeeper
servers in quorum have to connect to a leader. The entry <strong>syncLimit</strong> limits how far out of date a server can
be from a leader.</p>
<p>With both of these timeouts, you specify the unit of time using
<strong>tickTime</strong>. In this example, the timeout
for initLimit is 5 ticks at 2000 milleseconds a tick, or 10
seconds.</p>
<p>The entries of the form <em>server.X</em> list the
servers that make up the ZooKeeper service. When the server starts up,
it knows which server it is by looking for the file
<em>myid</em> in the data directory. That file has the
contains the server number, in ASCII.</p>
<p>Finally, note the two port numbers after each server
name: " 2888" and "3888". Peers use the former port to connect
to other peers. Such a connection is necessary so that peers
can communicate, for example, to agree upon the order of
updates. More specifically, a ZooKeeper server uses this port
to connect followers to the leader. When a new leader arises, a
follower opens a TCP connection to the leader using this
port. Because the default leader election also uses TCP, we
currently require another port for leader election. This is the
second port in the server entry.
</p>
<div class="note">
<div class="label">Note</div>
<div class="content">
<p>If you want to test multiple servers on a single
machine, specify the servername
as <em>localhost</em> with unique quorum &amp;
leader election ports (i.e. 2888:3888, 2889:3889, 2890:3890 in
the example above) for each server.X in that server's config
file. Of course separate <em>dataDir</em>s and
distinct <em>clientPort</em>s are also necessary
(in the above replicated example, running on a
single <em>localhost</em>, you would still have
three config files).</p>
</div>
</div>
<a name="Other+Optimizations"></a>
<h3 class="h4">Other Optimizations</h3>
<p>There are a couple of other configuration parameters that can
greatly increase performance:</p>
<ul>
<li>
<p>To get low latencies on updates it is important to
have a dedicated transaction log directory. By default
transaction logs are put in the same directory as the data
snapshots and <em>myid</em> file. The dataLogDir
parameters indicates a different directory to use for the
transaction logs.</p>
</li>
<li>
<p>
<em>[tbd: what is the other config param?]</em>
</p>
</li>
</ul>
</div>
<p align="right">
<font size="-2"></font>
</p>
</div>
<!--+
|end content
+-->
<div class="clearboth">&nbsp;</div>
</div>
<div id="footer">
<!--+
|start bottomstrip
+-->
<div class="lastmodified">
<script type="text/javascript"><!--
document.write("Last Published: " + document.lastModified);
// --></script>
</div>
<div class="copyright">
Copyright &copy;
2008-2013 <a href="http://www.apache.org/licenses/">The Apache Software Foundation.</a>
</div>
<!--+
|end bottomstrip
+-->
</div>
</body>
</html>