If you are feeling adventurous, you can setup ozone in a real cluster. Setting up a real cluster requires us to understand the components of Ozone. Ozone is designed to work concurrently with HDFS. However, Ozone is also capable of running independently. The components of ozone are the same in both approaches.
Please untar the ozone-0.2.1-SNAPSHOT to the directory where you are going to run Ozone from. We need Ozone jars on all machines in the cluster. So you need to do this on all machines in the cluster.
Ozone relies on a configuration file called ozone-site.xml
. To generate a template that you can replace with proper values, please run the following command. This will generate a template called ozone-site.xml
at the specified path (directory).
{{< highlight bash >}} ozone genconf {{< /highlight >}}
Let us look at the settings inside the generated file (ozone-site.xml) and how they control ozone. Once the right values are defined, this file needs to be copied to ozone directory/etc/Hadoop
.
true
enables ozone in the HDFS or Ozone cluster.Here is an example,
{{< highlight xml >}} ozone.enabled True {{< /highlight >}}
ozone.metadata.dirs Allows Administrators to specify where the metadata must reside. Usually you pick your fastest disk (SSD if you have them on your nodes). OzoneManager, SCM and datanode will write the metadata to this path. This is a required setting, if this is missing Ozone will fail to come up.
Here is an example,
{{< highlight xml >}} ozone.metadata.dirs /data/disk1/meta {{< /highlight >}}
Here is an example,
{{< highlight xml >}} ozone.scm.names scm.hadoop.apache.org {{< /highlight >}}
Here is an example, {{< highlight xml >}} ozone.scm.datanode.id /data/disk1/meta/node/datanode.id {{< /highlight >}}
Here is an example, {{< highlight xml >}} ozone.om.address ozonemanager.hadoop.apache.org {{< /highlight >}}
Setting | Value | Comment |
---|---|---|
ozone.enabled | true | This enables SCM and containers in HDFS cluster. |
ozone.metadata.dirs | file path | The metadata will be stored here. |
ozone.scm.names | SCM server name | Hostname:port or IP:port address of SCM. |
ozone.scm.block.client.address | SCM server name and port | Used by services like OM |
ozone.scm.client.address | SCM server name and port | Used by client-side |
ozone.scm.datanode.address | SCM server name and port | Used by datanode to talk to SCM |
ozone.om.address | OM server name | Used by Ozone handler and Ozone file system. |