| |
| |
| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| |
| <meta name="description" content="Hadoop Ozone Documentation"> |
| |
| <title>Documentation for Apache Hadoop Ozone</title> |
| |
| |
| <link href="./css/bootstrap.min.css" rel="stylesheet"> |
| |
| |
| <link href="./css/ozonedoc.css" rel="stylesheet"> |
| |
| </head> |
| |
| |
| <body> |
| |
| |
| <nav class="navbar navbar-inverse navbar-fixed-top"> |
| <div class="container-fluid"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#sidebar" aria-expanded="false" aria-controls="navbar"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a href="#" class="navbar-left" style="height: 50px; padding: 5px 5px 5px 0;"> |
| <img src="./ozone-logo-small.png" width="40"/> |
| </a> |
| <a class="navbar-brand hidden-xs" href="#"> |
| Apache Hadoop Ozone/HDDS documentation |
| </a> |
| <a class="navbar-brand visible-xs-inline" href="#">Hadoop Ozone</a> |
| </div> |
| <div id="navbar" class="navbar-collapse collapse"> |
| <ul class="nav navbar-nav navbar-right"> |
| <li><a href="https://github.com/apache/hadoop">Source</a></li> |
| <li><a href="https://hadoop.apache.org">Apache Hadoop</a></li> |
| <li><a href="https://apache.org">ASF</a></li> |
| </ul> |
| </div> |
| </div> |
| </nav> |
| |
| |
| <div class="container-fluid"> |
| <div class="row"> |
| |
| <div class="col-sm-2 col-md-2 sidebar" id="sidebar"> |
| <ul class="nav nav-sidebar"> |
| |
| |
| |
| <li class=""> |
| |
| <a href="./index.html"> |
| |
| |
| |
| <span>Overview</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="./start.html"> |
| |
| |
| |
| <span>Getting Started</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="./shell.html"> |
| |
| |
| |
| <span>Command Line Interface</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="./interface.html"> |
| |
| |
| |
| <span>Programming Interfaces</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="./security.html"> |
| |
| |
| |
| <span>Security</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class="active"> |
| |
| <a href="./concept.html"> |
| |
| |
| |
| <span>Concepts</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="./beyond.html"> |
| |
| |
| |
| <span>Beyond Basics</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="./tools.html"> |
| |
| |
| |
| <span>Tools</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="./recipe.html"> |
| |
| |
| |
| <span>Recipes</span> |
| </a> |
| </li> |
| |
| |
| <li class="visible-xs"><a href="#">References</a> |
| <ul class="nav"> |
| <li><a href="https://github.com/apache/hadoop"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> Source</a></li> |
| <li><a href="https://hadoop.apache.org"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> Apache Hadoop</a></li> |
| <li><a href="https://apache.org"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> ASF</a></li> |
| </ul></li> |
| </ul> |
| |
| </div> |
| |
| <div class="col-sm-10 col-sm-offset-2 col-md-10 col-md-offset-2 main"> |
| <div class="col-md-9"> |
| <h1>Concepts</h1> |
| </div> |
| <div class="col-md-9"> |
| <!--- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| |
| |
| |
| <div class="jumbotron jumbotron-fluid"> |
| <div class="container"> |
| <h3 class="display-4">Ozone Architecture </h3> |
| <p class="lead"> |
| |
| |
| Ozone's architectural elements are explained in the following pages. The |
| metadata layer, data layer, protocol bus, replication layer and Recon are |
| discussed in the following pages. These concepts are useful if you want to |
| understand how ozone works in depth. |
| |
| |
| </p> |
| </div> |
| </div> |
| |
| |
| |
| |
| |
| |
| |
| <div class="row"> |
| |
| <div class="col-sm-6"> |
| <div class="card"> |
| <div class="card-body"> |
| <h2 class="card-title"> |
| |
| Overview |
| </h2> |
| <p class="card-text">Ozone is a redundant, distributed object store optimized for Big data workloads. The primary design point of ozone is scalability, and it aims to scale to billions of objects. |
| Ozone separates namespace management and block space management; this helps ozone to scale much better. The namespace is managed by a daemon called Ozone Manager (OM), and block space is managed by Storage Container Manager (SCM). |
| Ozone consists of volumes, buckets, and keys.</p> |
| <a href="./concept/overview.html" |
| class=" btn btn-primary btn-lg">Overview</a> |
| </div> |
| </div> |
| </div> |
| |
| |
| |
| |
| |
| |
| <div class="col-sm-6"> |
| <div class="card"> |
| <div class="card-body"> |
| <h2 class="card-title"> |
| |
| Ozone Manager |
| </h2> |
| <p class="card-text">Ozone Manager (OM) is the namespace manager for Ozone. |
| This means that when you want to write some data, you ask Ozone Manager for a block and Ozone Manager gives you a block and remembers that information. When you want to read that file back, you need to find the address of the block and Ozone Manager returns it you. |
| Ozone Manager also allows users to organize keys under a volume and bucket.</p> |
| <a href="./concept/ozonemanager.html" |
| class=" btn btn-primary btn-lg">Ozone Manager</a> |
| </div> |
| </div> |
| </div> |
| |
| |
| </div> |
| |
| |
| |
| |
| |
| <div class="row"> |
| |
| <div class="col-sm-6"> |
| <div class="card"> |
| <div class="card-body"> |
| <h2 class="card-title"> |
| |
| Storage Container Manager |
| </h2> |
| <p class="card-text">Storage container manager provides multiple critical functions for the Ozone cluster. SCM acts as the cluster manager, Certificate authority, Block manager and the Replica manager. |
| Cluster Management SCM is in charge of creating an Ozone cluster. When an SCM is booted up via init command, SCM creates the cluster identity and root certificates needed for the SCM certificate authority. SCM manages the life cycle of a data node in the cluster.</p> |
| <a href="./concept/hdds.html" |
| class=" btn btn-primary btn-lg">Storage Container Manager</a> |
| </div> |
| </div> |
| </div> |
| |
| |
| |
| |
| |
| |
| <div class="col-sm-6"> |
| <div class="card"> |
| <div class="card-body"> |
| <h2 class="card-title"> |
| |
| Datanodes |
| </h2> |
| <p class="card-text">Datanodes are the worker bees of Ozone. All data is stored on data nodes. Clients write data in terms of blocks. Datanode aggregates these blocks into a storage container. A storage container is the data streams and metadata about the blocks written by the clients. |
| Storage Containers A storage container is a self-contained super block. It has a list of Ozone blocks that reside inside it, as well as on-disk files which contain the actual data streams.</p> |
| <a href="./concept/datanodes.html" |
| class=" btn btn-primary btn-lg">Datanodes</a> |
| </div> |
| </div> |
| </div> |
| |
| |
| </div> |
| |
| |
| |
| </div> |
| </div> |
| </div> |
| </div> |
| |
| |
| |
| |
| <script src="./js/jquery-3.4.1.min.js"></script> |
| <script src="./js/ozonedoc.js"></script> |
| <script src="./js/bootstrap.min.js"></script> |
| |
| |
| </body> |
| |
| </html> |
