| <?xml version="1.0" encoding="UTF-8"?> |
| <!-- |
| Copyright 2002-2004 The Apache Software Foundation |
| |
| Licensed under the Apache License, Version 2.0 (the "License"); |
| you may not use this file except in compliance with the License. |
| You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| |
| <!DOCTYPE article PUBLIC "-//OASIS//DTD Simplified DocBook XML V1.0//EN" |
| "http://www.oasis-open.org/docbook/xml/simple/1.0/sdocbook.dtd"> |
| <article id="bk_GettStartedGuide"> |
| <title>ZooKeeper Getting Started Guide</title> |
| |
| <articleinfo> |
| <legalnotice> |
| <para>Licensed under the Apache License, Version 2.0 (the "License"); |
| you may not use this file except in compliance with the License. You may |
| obtain a copy of the License at <ulink |
| url="http://www.apache.org/licenses/LICENSE-2.0">http://www.apache.org/licenses/LICENSE-2.0</ulink>.</para> |
| |
| <para>Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an "AS IS" |
| BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or |
| implied. See the License for the specific language governing permissions |
| and limitations under the License.</para> |
| </legalnotice> |
| |
| <abstract> |
| <para>This guide contains detailed information about creating |
| distributed applications that use ZooKeeper. It discusses the basic |
| operations ZooKeeper supports, and how these can be used to build |
| higher-level abstractions. It contains solutions to common tasks, a |
| troubleshooting guide, and links to other information.</para> |
| </abstract> |
| </articleinfo> |
| |
| <section id="ch_GettingStarted"> |
| <title>Getting Started: Coordinating Distributed Applications with |
| ZooKeeper</title> |
| |
| <para>This document contains information to get you started quickly with |
| ZooKeeper. It is aimed primarily at developers hoping to try it out, and |
| contains simple installation instructions for a single ZooKeeper server, a |
| few commands to verify that it is running, and a simple programming |
| example. Finally, as a convenience, there are a few sections regarding |
| more complicated installations, for example running replicated |
| deployments, and optimizing the transaction log. However for the complete |
| instructions for commercial deployments, please refer to the <ulink |
| url="zookeeperAdmin.html">ZooKeeper |
| Administrator's Guide</ulink>.</para> |
| |
| <section id="sc_Prerequisites"> |
| <title>Pre-requisites</title> |
| |
| <para>See <ulink url="zookeeperAdmin.html#sc_systemReq"> |
| System Requirements</ulink> in the Admin guide.</para> |
| </section> |
| |
| <section id="sc_Download"> |
| <title>Download</title> |
| |
| <para>To get a ZooKeeper distribution, download a recent |
| <ulink url="http://zookeeper.apache.org/releases.html"> |
| stable</ulink> release from one of the Apache Download |
| Mirrors.</para> |
| </section> |
| |
| <section id="sc_InstallingSingleMode"> |
| <title>Standalone Operation</title> |
| |
| <para>Setting up a ZooKeeper server in standalone mode is |
| straightforward. The server is contained in a single JAR file, |
| so installation consists of creating a configuration.</para> |
| |
| <para>Once you've downloaded a stable ZooKeeper release unpack |
| it and cd to the root</para> |
| |
| <para>To start ZooKeeper you need a configuration file. Here is a sample, |
| create it in <emphasis role="bold">conf/zoo.cfg</emphasis>:</para> |
| |
| <programlisting> |
| tickTime=2000 |
| dataDir=/var/lib/zookeeper |
| clientPort=2181 |
| </programlisting> |
| |
| <para>This file can be called anything, but for the sake of this |
| discussion call |
| it <emphasis role="bold">conf/zoo.cfg</emphasis>. Change the |
| value of <emphasis role="bold">dataDir</emphasis> to specify an |
| existing (empty to start with) directory. Here are the meanings |
| for each of the fields:</para> |
| |
| <variablelist> |
| <varlistentry> |
| <term><emphasis role="bold">tickTime</emphasis></term> |
| |
| <listitem> |
| <para>the basic time unit in milliseconds used by ZooKeeper. It is |
| used to do heartbeats and the minimum session timeout will be |
| twice the tickTime.</para> |
| </listitem> |
| </varlistentry> |
| </variablelist> |
| |
| <variablelist> |
| <varlistentry> |
| <term><emphasis role="bold">dataDir</emphasis></term> |
| |
| <listitem> |
| <para>the location to store the in-memory database snapshots and, |
| unless specified otherwise, the transaction log of updates to the |
| database.</para> |
| </listitem> |
| </varlistentry> |
| |
| <varlistentry> |
| <term><emphasis role="bold">clientPort</emphasis></term> |
| |
| <listitem> |
| <para>the port to listen for client connections</para> |
| </listitem> |
| </varlistentry> |
| </variablelist> |
| |
| <para>Now that you created the configuration file, you can start |
| ZooKeeper:</para> |
| |
| <programlisting>bin/zkServer.sh start</programlisting> |
| |
| <para>ZooKeeper logs messages using log4j -- more detail |
| available in the |
| <ulink url="zookeeperProgrammers.html#Logging">Logging</ulink> |
| section of the Programmer's Guide. You will see log messages |
| coming to the console (default) and/or a log file depending on |
| the log4j configuration.</para> |
| |
| <para>The steps outlined here run ZooKeeper in standalone mode. There is |
| no replication, so if ZooKeeper process fails, the service will go down. |
| This is fine for most development situations, but to run ZooKeeper in |
| replicated mode, please see <ulink |
| url="#sc_RunningReplicatedZooKeeper">Running Replicated |
| ZooKeeper</ulink>.</para> |
| </section> |
| |
| <section id="sc_FileManagement"> |
| <title>Managing ZooKeeper Storage</title> |
| <para>For long running production systems ZooKeeper storage must |
| be managed externally (dataDir and logs). See the section on |
| <ulink |
| url="zookeeperAdmin.html#sc_maintenance">maintenance</ulink> for |
| more details.</para> |
| </section> |
| |
| <section id="sc_ConnectingToZooKeeper"> |
| <title>Connecting to ZooKeeper</title> |
| |
| <programlisting>$ bin/zkCli.sh -server 127.0.0.1:2181</programlisting> |
| |
| <para>This lets you perform simple, file-like operations.</para> |
| |
| <para>Once you have connected, you should see something like: |
| </para> |
| <programlisting> |
| <![CDATA[ |
| Connecting to localhost:2181 |
| log4j:WARN No appenders could be found for logger (org.apache.zookeeper.ZooKeeper). |
| log4j:WARN Please initialize the log4j system properly. |
| Welcome to ZooKeeper! |
| JLine support is enabled |
| [zkshell: 0] |
| ]]> </programlisting> |
| <para> |
| From the shell, type <command>help</command> to get a listing of commands that can be executed from the client, as in: |
| </para> |
| <programlisting> |
| <![CDATA[ |
| [zkshell: 0] help |
| ZooKeeper host:port cmd args |
| get path [watch] |
| ls path [watch] |
| set path data [version] |
| delquota [-n|-b] path |
| quit |
| printwatches on|off |
| create path data acl |
| stat path [watch] |
| listquota path |
| history |
| setAcl path acl |
| getAcl path |
| sync path |
| redo cmdno |
| addauth scheme auth |
| delete path [version] |
| deleteall path |
| setquota -n|-b val path |
| |
| ]]> </programlisting> |
| <para>From here, you can try a few simple commands to get a feel for this simple command line interface. First, start by issuing the list command, as |
| in <command>ls</command>, yielding: |
| </para> |
| <programlisting> |
| <![CDATA[ |
| [zkshell: 8] ls / |
| [zookeeper] |
| ]]> </programlisting> |
| <para>Next, create a new znode by running <command>create /zk_test my_data</command>. This creates a new znode and associates the string "my_data" with the node. |
| You should see:</para> |
| <programlisting> |
| <![CDATA[ |
| [zkshell: 9] create /zk_test my_data |
| Created /zk_test |
| ]]> </programlisting> |
| <para> Issue another <command>ls /</command> command to see what the directory looks like: |
| </para> |
| <programlisting> |
| <![CDATA[ |
| [zkshell: 11] ls / |
| [zookeeper, zk_test] |
| |
| ]]> </programlisting><para> |
| Notice that the zk_test directory has now been created. |
| </para> |
| <para>Next, verify that the data was associated with the znode by running the <command>get</command> command, as in: |
| </para> |
| <programlisting> |
| <![CDATA[ |
| [zkshell: 12] get /zk_test |
| my_data |
| cZxid = 5 |
| ctime = Fri Jun 05 13:57:06 PDT 2009 |
| mZxid = 5 |
| mtime = Fri Jun 05 13:57:06 PDT 2009 |
| pZxid = 5 |
| cversion = 0 |
| dataVersion = 0 |
| aclVersion = 0 |
| ephemeralOwner = 0 |
| dataLength = 7 |
| numChildren = 0 |
| ]]> </programlisting> |
| <para>We can change the data associated with zk_test by issuing the <command>set</command> command, as in: |
| </para> |
| <programlisting> |
| <![CDATA[ |
| [zkshell: 14] set /zk_test junk |
| cZxid = 5 |
| ctime = Fri Jun 05 13:57:06 PDT 2009 |
| mZxid = 6 |
| mtime = Fri Jun 05 14:01:52 PDT 2009 |
| pZxid = 5 |
| cversion = 0 |
| dataVersion = 1 |
| aclVersion = 0 |
| ephemeralOwner = 0 |
| dataLength = 4 |
| numChildren = 0 |
| [zkshell: 15] get /zk_test |
| junk |
| cZxid = 5 |
| ctime = Fri Jun 05 13:57:06 PDT 2009 |
| mZxid = 6 |
| mtime = Fri Jun 05 14:01:52 PDT 2009 |
| pZxid = 5 |
| cversion = 0 |
| dataVersion = 1 |
| aclVersion = 0 |
| ephemeralOwner = 0 |
| dataLength = 4 |
| numChildren = 0 |
| ]]> </programlisting> |
| <para> |
| (Notice we did a <command>get</command> after setting the data and it did, indeed, change.</para> |
| <para>Finally, let's <command>delete</command> the node by issuing: |
| </para> |
| <programlisting> |
| <![CDATA[ |
| [zkshell: 16] delete /zk_test |
| [zkshell: 17] ls / |
| [zookeeper] |
| [zkshell: 18] |
| ]]></programlisting> |
| <para>That's it for now. To explore more, continue with the rest of this document and see the <ulink url="zookeeperProgrammers.html">Programmer's Guide</ulink>. </para> |
| </section> |
| |
| <section id="sc_ProgrammingToZooKeeper"> |
| <title>Programming to ZooKeeper</title> |
| |
| <para>ZooKeeper has a Java bindings and C bindings. They are |
| functionally equivalent. The C bindings exist in two variants: single |
| threaded and multi-threaded. These differ only in how the messaging loop |
| is done. For more information, see the <ulink |
| url="zookeeperProgrammers.html#ch_programStructureWithExample">Programming |
| Examples in the ZooKeeper Programmer's Guide</ulink> for |
| sample code using of the different APIs.</para> |
| </section> |
| |
| <section id="sc_RunningReplicatedZooKeeper"> |
| <title>Running Replicated ZooKeeper</title> |
| |
| <para>Running ZooKeeper in standalone mode is convenient for evaluation, |
| some development, and testing. But in production, you should run |
| ZooKeeper in replicated mode. A replicated group of servers in the same |
| application is called a <emphasis>quorum</emphasis>, and in replicated |
| mode, all servers in the quorum have copies of the same configuration |
| file.</para> |
| <note> |
| <para> |
| For replicated mode, a minimum of three servers are required, |
| and it is strongly recommended that you have an odd number of |
| servers. If you only have two servers, then you are in a |
| situation where if one of them fails, there are not enough |
| machines to form a majority quorum. Two servers is inherently |
| <emphasis role="bold">less</emphasis> |
| stable than a single server, because there are two single |
| points of failure. |
| </para> |
| </note> |
| <para> |
| The required |
| <emphasis role="bold">conf/zoo.cfg</emphasis> |
| file for replicated mode is similar to the one used in standalone |
| mode, but with a few differences. Here is an example: |
| </para> |
| |
| <programlisting> |
| tickTime=2000 |
| dataDir=/var/lib/zookeeper |
| clientPort=2181 |
| initLimit=5 |
| syncLimit=2 |
| server.1=zoo1:2888:3888 |
| server.2=zoo2:2888:3888 |
| server.3=zoo3:2888:3888 |
| </programlisting> |
| |
| <para>The new entry, <emphasis role="bold">initLimit</emphasis> is |
| timeouts ZooKeeper uses to limit the length of time the ZooKeeper |
| servers in quorum have to connect to a leader. The entry <emphasis |
| role="bold">syncLimit</emphasis> limits how far out of date a server can |
| be from a leader.</para> |
| |
| <para>With both of these timeouts, you specify the unit of time using |
| <emphasis role="bold">tickTime</emphasis>. In this example, the timeout |
| for initLimit is 5 ticks at 2000 milleseconds a tick, or 10 |
| seconds.</para> |
| |
| <para>The entries of the form <emphasis>server.X</emphasis> list the |
| servers that make up the ZooKeeper service. When the server starts up, |
| it knows which server it is by looking for the file |
| <emphasis>myid</emphasis> in the data directory. That file has the |
| contains the server number, in ASCII.</para> |
| |
| <para>Finally, note the two port numbers after each server |
| name: " 2888" and "3888". Peers use the former port to connect |
| to other peers. Such a connection is necessary so that peers |
| can communicate, for example, to agree upon the order of |
| updates. More specifically, a ZooKeeper server uses this port |
| to connect followers to the leader. When a new leader arises, a |
| follower opens a TCP connection to the leader using this |
| port. Because the default leader election also uses TCP, we |
| currently require another port for leader election. This is the |
| second port in the server entry. |
| </para> |
| |
| <note> |
| <para>If you want to test multiple servers on a single |
| machine, specify the servername |
| as <emphasis>localhost</emphasis> with unique quorum & |
| leader election ports (i.e. 2888:3888, 2889:3889, 2890:3890 in |
| the example above) for each server.X in that server's config |
| file. Of course separate <emphasis>dataDir</emphasis>s and |
| distinct <emphasis>clientPort</emphasis>s are also necessary |
| (in the above replicated example, running on a |
| single <emphasis>localhost</emphasis>, you would still have |
| three config files).</para> |
| <para>Please be aware that setting up multiple servers on a single |
| machine will not create any redundancy. If something were to |
| happen which caused the machine to die, all of the zookeeper |
| servers would be offline. Full redundancy requires that each |
| server have its own machine. It must be a completely separate |
| physical server. Multiple virtual machines on the same physical |
| host are still vulnerable to the complete failure of that host.</para> |
| </note> |
| </section> |
| |
| <section> |
| <title>Other Optimizations</title> |
| |
| <para>There are a couple of other configuration parameters that can |
| greatly increase performance:</para> |
| |
| <itemizedlist> |
| <listitem> |
| <para>To get low latencies on updates it is important to |
| have a dedicated transaction log directory. By default |
| transaction logs are put in the same directory as the data |
| snapshots and <emphasis>myid</emphasis> file. The dataLogDir |
| parameters indicates a different directory to use for the |
| transaction logs.</para> |
| </listitem> |
| |
| <listitem> |
| <para><emphasis>[tbd: what is the other config param?]</emphasis></para> |
| </listitem> |
| </itemizedlist> |
| </section> |
| </section> |
| </article> |
