| |
| |
| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| |
| <meta name="description" content="Hadoop Ozone Documentation"> |
| |
| <title>Documentation for Apache Hadoop Ozone</title> |
| |
| |
| <link href="css/bootstrap.min.css" rel="stylesheet"> |
| |
| |
| <link href="css/ozonedoc.css" rel="stylesheet"> |
| |
| </head> |
| |
| |
| <body> |
| |
| |
| <nav class="navbar navbar-inverse navbar-fixed-top"> |
| <div class="container-fluid"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a class="navbar-brand" href="#">Apache Hadoop Ozone/HDDS documentation</a> |
| </div> |
| <div id="navbar" class="navbar-collapse collapse"> |
| <ul class="nav navbar-nav navbar-right"> |
| <li><a href="https://github.com/apache/hadoop">Source</a></li> |
| <li><a href="https://hadoop.apache.org">Apache Hadoop</a></li> |
| <li><a href="https://apache.org">ASF</a></li> |
| </ul> |
| </div> |
| </div> |
| </nav> |
| |
| |
| <div class="container-fluid"> |
| <div class="row"> |
| |
| <div class="col-sm-3 col-md-2 sidebar"> |
| <img src="ozone-logo.png" style="max-width: 100%;"/> |
| <ul class="nav nav-sidebar"> |
| |
| |
| |
| <li class=""> |
| |
| <a href="index.html"> |
| |
| |
| |
| <span>Ozone Overview</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="runningviadocker.html"> |
| |
| <span>Getting Started</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| <a href="./runningviadocker.html">Alpha Cluster</a> |
| </li> |
| |
| <li class=""> |
| <a href="./settings.html">Configuration</a> |
| </li> |
| |
| <li class=""> |
| <a href="./realcluster.html">Starting an Ozone Cluster</a> |
| </li> |
| |
| <li class=""> |
| <a href="./runningwithhdfs.html">Running concurrently with HDFS</a> |
| </li> |
| |
| <li class=""> |
| <a href="./buildingsources.html">Building from Sources</a> |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="commandshell.html"> |
| |
| <span>Client</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| <a href="./commandshell.html">Ozone CLI</a> |
| </li> |
| |
| <li class=""> |
| <a href="./volumecommands.html">Volume Commands</a> |
| </li> |
| |
| <li class=""> |
| <a href="./bucketcommands.html">Bucket Commands</a> |
| </li> |
| |
| <li class=""> |
| <a href="./keycommands.html">Key Commands</a> |
| </li> |
| |
| <li class=""> |
| <a href="./javaapi.html">Java API</a> |
| </li> |
| |
| <li class=""> |
| <a href="./ozonefs.html">Ozone File System</a> |
| </li> |
| |
| <li class=""> |
| <a href="./rest.html">REST API</a> |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="dozone.html"> |
| |
| <span>Tools</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| <a href="./dozone.html">Dozone & Dev Tools</a> |
| </li> |
| |
| <li class=""> |
| <a href="./freon.html">Freon</a> |
| </li> |
| |
| <li class=""> |
| <a href="./scmcli.html">SCMCLI</a> |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="./concepts.html"> |
| |
| <span>Architecture</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| <a href="./hdds.html">Hadoop Distributed Data Store</a> |
| </li> |
| |
| <li class="active"> |
| <a href="./ozonemanager.html">Ozone Manager</a> |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| </ul> |
| |
| </div> |
| |
| <div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main"> |
| <h1>Ozone Manager</h1> |
| <div class="col-md-9"> |
| |
| |
| <!--- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| |
| <h2 id="om-overview">OM Overview</h2> |
| |
| <p>Ozone Manager or OM is the namespace manager for Ozone. The clients (RPC clients, Rest proxy, Ozone file system, etc.) communicate with OM to create and delete various ozone objects.</p> |
| |
| <p>Each ozone volume is the root of a namespace under OM. This is very different from HDFS which provides a single rooted file system.</p> |
| |
| <p>Ozone’s namespace is a collection of volumes or is a forest instead of a |
| single rooted tree as in HDFS. This property makes it easy to deploy multiple |
| OMs for scaling, this feature is under development.</p> |
| |
| <h2 id="om-metadata">OM Metadata</h2> |
| |
| <p>Conceptually, OM maintains a list of volumes, buckets, and keys. For each user, it maintains a list of volumes. For each volume, the list of buckets and for each bucket the list of keys.</p> |
| |
| <p>Right now, OM is a single instance service. Ozone already relies on Apache Ratis (A Replicated State Machine based on Raft protocol). OM will be extended to replicate all its metadata via Ratis. With that, OM will be highly available.</p> |
| |
| <h2 id="om-ui">OM UI</h2> |
| |
| <p>OM supports a simple UI for the time being. The default port of OM is 9874. To access the OM UI, the user can connect to <a href="http://OM:port">http://OM:port</a> or for a concrete example,</p> |
| |
| <pre><code>http://omserver:9874/ |
| </code></pre> |
| |
| <p>OM UI primarily tries to measure load and latency of OM. The first section of OM UI relates to the number of operations seen by the cluster broken down by the object, operation and whether the operation was successful.</p> |
| |
| <p>The latter part of the UI is focused on latency and number of operations that OM is performing.</p> |
| |
| <p>One of the hardest problems in HDFS world is discovering the numerous settings offered to tune HDFS. Ozone solves that problem by tagging the configs. To discover settings, click on “Common Tools”->Config. This will take you to the ozone config UI.</p> |
| |
| <h2 id="config-ui">Config UI</h2> |
| |
| <p>The ozone config UI is a matrix with row representing the tags, and columns representing All, OM and SCM.</p> |
| |
| <p>Suppose a user wanted to discover the required settings for ozone. Then the user can tick the checkbox that says “Required.” |
| This will filter out all “Required” settings along with the description of what each setting does.</p> |
| |
| <p>The user can combine different checkboxes and UI will combine the results. That is, If you have more than one row selected, then all keys for those chosen tags are displayed together.</p> |
| |
| <p>We are hopeful that this leads to a more straightforward way of discovering settings that manage ozone.</p> |
| |
| <h2 id="om-and-scm">OM and SCM</h2> |
| |
| <p><a href="./hdds.html">Storage container manager</a> or (SCM) is the block manager |
| for ozone. When a client requests OM for a set of data nodes to write data, OM talk to SCM and gets a block.</p> |
| |
| <p>A block returned by SCM contains a pipeline, which is a set of nodes that we participate in that block replication.</p> |
| |
| <p>So OM is dependent on SCM for reading and writing of Keys. However, OM is independent of SCM while doing metadata operations like ozone volume or bucket operations.</p> |
| |
| </div> |
| </div> |
| </div> |
| </div> |
| |
| |
| |
| |
| <script src="./js/jquery.min.js"></script> |
| <script src="./js/ozonedoc.js"></script> |
| <script src="./js/bootstrap.min.js"></script> |
| |
| |
| </body> |
| </html> |
