| |
| |
| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| |
| <meta name="description" content="Hadoop Ozone Documentation"> |
| |
| <title>Documentation for Apache Hadoop Ozone</title> |
| |
| |
| <link href="../../css/bootstrap.min.css" rel="stylesheet"> |
| |
| |
| <link href="../../css/ozonedoc.css" rel="stylesheet"> |
| |
| </head> |
| |
| |
| <body> |
| |
| |
| <nav class="navbar navbar-inverse navbar-fixed-top"> |
| <div class="container-fluid"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#sidebar" aria-expanded="false" aria-controls="navbar"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a href="#" class="navbar-left" style="height: 50px; padding: 5px 5px 5px 0;"> |
| <img src="../../ozone-logo-small.png" width="40"/> |
| </a> |
| <a class="navbar-brand hidden-xs" href="#"> |
| Apache Hadoop Ozone/HDDS documentation |
| </a> |
| <a class="navbar-brand visible-xs-inline" href="#">Hadoop Ozone</a> |
| </div> |
| <div id="navbar" class="navbar-collapse collapse"> |
| <ul class="nav navbar-nav navbar-right"> |
| <li><a href="https://github.com/apache/hadoop-ozone">Source</a></li> |
| <li><a href="https://hadoop.apache.org">Apache Hadoop</a></li> |
| <li><a href="https://apache.org">ASF</a></li> |
| </ul> |
| </div> |
| </div> |
| </nav> |
| |
| |
| <div class="container-fluid"> |
| <div class="row"> |
| |
| <div class="col-sm-2 col-md-2 sidebar" id="sidebar"> |
| <ul class="nav nav-sidebar"> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/"> |
| |
| |
| |
| <span>概述</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/start.html"> |
| |
| |
| |
| <span>快速入门</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/interface.html"> |
| |
| |
| |
| <span>编程接口</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/feature.html"> |
| |
| |
| |
| <span>GDPR</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/security.html"> |
| |
| |
| |
| <span>安全</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/concept.html"> |
| |
| |
| |
| <span>概念</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/tools.html"> |
| |
| |
| |
| <span>工具</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/recipe.html"> |
| |
| |
| |
| <span>使用配方</span> |
| </a> |
| </li> |
| |
| |
| <li><a href="../../design.html"><span><b>Design docs</b></span></a></li> |
| <li class="visible-xs"><a href="#">References</a> |
| <ul class="nav"> |
| <li><a href="https://github.com/apache/hadoop"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> Source</a></li> |
| <li><a href="https://hadoop.apache.org"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> Apache Hadoop</a></li> |
| <li><a href="https://apache.org"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> ASF</a></li> |
| </ul></li> |
| </ul> |
| |
| </div> |
| |
| <div class="col-sm-10 col-sm-offset-2 col-md-10 col-md-offset-2 main"> |
| |
| |
| |
| <div class="col-md-9"> |
| <nav aria-label="breadcrumb"> |
| <ol class="breadcrumb"> |
| <li class="breadcrumb-item"><a href="../../">Home</a></li> |
| <li class="breadcrumb-item" aria-current="page"><a href="../../zh/concept.html">概念</a></li> |
| <li class="breadcrumb-item active" aria-current="page">数据节点</li> |
| </ol> |
| </nav> |
| |
| |
| |
| <div class="pull-right"> |
| |
| |
| <a href="../../concept/datanodes.html"><span class="label label-success">English</span></a> |
| |
| |
| |
| |
| </div> |
| |
| |
| <div class="col-md-9"> |
| <h1>数据节点</h1> |
| |
| <!--- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| <p>数据节点是 Ozone 中的 worker,所有的数据都存储在数据节点上,用户以块的方式写数据,数据节点将多个块聚合成一个存储容器,存储容器中包含用户写入的数据块和这些块的元数据。</p> |
| <h2 id="存储容器">存储容器</h2> |
| <p><img src="ContainerMetadata.png" alt="FunctionalOzone"></p> |
| <p>Ozone 的存储容器是一个自包含的超级块,容器中包含一系列的 Ozone 块,以及存储实际数据的磁盘文件,这是默认的存储容器格式。对于 Ozone 来说,容器只是提供了一个协议规范,它独立于具体的存储格式实现,换句话说,我们可以很容易扩展或引入新的容器实现格式。因此,上述格式应当被看作是 Ozone 存储容器的参考实现。</p> |
| <h2 id="理解-ozone-中的块和容器">理解 Ozone 中的块和容器</h2> |
| <p>当用户想要从 Ozone 中读取一个键时,用户向 OM 发送该键名,OM 会返回组成该键的块列表。</p> |
| <p>每个 Ozone 块包含一个容器 ID 和一个本地 ID,下图展示了 Ozone 块的逻辑组成:</p> |
| <p><img src="OzoneBlock.png" alt="OzoneBlock"></p> |
| <p>容器 ID 用来让用户发现容器的位置,容器位置的权威信息存储在 SCM 中。大部分情况下,OM 会缓存容器的位置信息,并会将它们随 Ozone 块一起返回给用户。</p> |
| <p>当用户定位到容器的位置,即知道哪些数据节点包含这个容器后,用户会连接数据节点,然后读取由 <em>容器ID:本地ID</em> 指定的数据流,换句话说,本地 ID 相当于容器内的索引,描述了我们应该读取哪个数据流。</p> |
| <h3 id="容器的位置发现">容器的位置发现</h3> |
| <p>SCM 如何获得容器的位置?这一点和现有的 HDFS 十分相似。数据节点会定期发送类似于块报告的容器报告,容器报告比块报告的内容简洁的多,比如,对于一个存储容量为 196 TB 的集群,Ozone 大概会拥有四万个容器,相比于 HDFS 的一百五十万个块,块报告数量缩减为四十分之一。</p> |
| <p>这种间接管理的方式大大地提高了 Ozone 的扩展性,因为 SCM 需要处理的块数据大大减少,且命名服务(OM)作为一个独特的服务主体对于扩展 Ozone 具有重要意义。</p> |
| |
| |
| |
| </div> |
| |
| </div> |
| </div> |
| </div> |
| </div> |
| |
| |
| |
| |
| <script src="../../js/jquery-3.5.1.min.js"></script> |
| <script src="../../js/ozonedoc.js"></script> |
| <script src="../../js/bootstrap.min.js"></script> |
| |
| |
| </body> |
| |
| </html> |