| |
| |
| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| |
| <meta name="description" content="Apache Ozone Documentation"> |
| |
| <title>Documentation for Apache Ozone</title> |
| |
| |
| <link href="../../css/bootstrap.min.css" rel="stylesheet"> |
| |
| |
| <link href="../../css/ozonedoc.css" rel="stylesheet"> |
| |
| </head> |
| |
| |
| <body> |
| |
| |
| <nav class="navbar navbar-inverse navbar-fixed-top"> |
| <div class="container-fluid"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#sidebar" aria-expanded="false" aria-controls="navbar"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a href="../../zh/index.html" class="navbar-left ozone-logo"> |
| <img src="../../ozone-logo-small.png"/> |
| </a> |
| <a class="navbar-brand hidden-xs" href="../../zh/index.html"> |
| Apache Ozone/HDDS documentation |
| </a> |
| <a class="navbar-brand visible-xs-inline" href="#">Apache Ozone</a> |
| </div> |
| <div id="navbar" class="navbar-collapse collapse"> |
| <ul class="nav navbar-nav navbar-right"> |
| <li><a href="https://github.com/apache/hadoop-ozone">Source</a></li> |
| <li><a href="https://hadoop.apache.org">Apache Hadoop</a></li> |
| <li><a href="https://apache.org">ASF</a></li> |
| </ul> |
| </div> |
| </div> |
| </nav> |
| |
| |
| <div class="wrapper"> |
| <div class="container-fluid"> |
| <div class="row"> |
| |
| <div class="col-sm-2 col-md-2 sidebar" id="sidebar"> |
| <ul class="nav nav-sidebar"> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/index.html"> |
| |
| |
| |
| <span>概述</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/start.html"> |
| |
| |
| |
| <span>快速入门</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="../../zh/concept.html"> |
| |
| <span>概念</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| |
| <a href="../../zh/concept/overview.html">概览</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/concept/ozonemanager.html">Ozone Manager</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/concept/storagecontainermanager.html">Storage Container Manager</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/concept/datanodes.html">数据节点</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/concept/containers.html">Containers</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/concept/recon.html">Recon</a> |
| |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="../../zh/feature.html"> |
| |
| <span>特点</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| |
| <a href="../../zh/feature/ha.html">高可用</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/security/gdpr.html">Ozone 中的 GDPR</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/feature/quota.html">Ozone中的配额</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/feature/recon.html">Recon 服务器</a> |
| |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="../../zh/security.html"> |
| |
| <span>安全</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| |
| <a href="../../zh/security/secureozone.html">安全化 Ozone</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/security/securingtde.html">透明数据加密</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/security/securings3.html">安全化 S3</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/security/securitywithranger.html">Apache Ranger</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/security/securityacls.html">Ozone 访问控制列表</a> |
| |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="../../zh/interface.html"> |
| |
| <span>编程接口</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| |
| <a href="../../zh/interface/javaapi.html">Java API</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/interface/o3fs.html">Ozone 文件系统</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/interface/csi.html">CSI 协议</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/interface/s3.html">S3 协议接口</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/interface/reconapi.html">Recon API</a> |
| |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/tools.html"> |
| |
| |
| |
| <span>工具</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/recipe.html"> |
| |
| |
| |
| <span>使用配方</span> |
| </a> |
| </li> |
| |
| |
| <li><a href="../../design.html"><span><b>Design docs</b></span></a></li> |
| <li class="visible-xs"><a href="#">References</a> |
| <ul class="nav"> |
| <li><a href="https://github.com/apache/hadoop"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> Source</a></li> |
| <li><a href="https://hadoop.apache.org"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> Apache Hadoop</a></li> |
| <li><a href="https://apache.org"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> ASF</a></li> |
| </ul></li> |
| </ul> |
| |
| </div> |
| |
| <div class="col-sm-10 col-sm-offset-2 col-md-10 col-md-offset-2 main"> |
| |
| |
| |
| <div class="col-md-9"> |
| <nav aria-label="breadcrumb"> |
| <ol class="breadcrumb"> |
| <li class="breadcrumb-item"><a href="../../zh/index.html">Home</a></li> |
| <li class="breadcrumb-item" aria-current="page"><a href="../../zh/start.html">快速入门</a></li> |
| <li class="breadcrumb-item active" aria-current="page">物理集群上 Ozone 的安装</li> |
| </ol> |
| </nav> |
| |
| |
| |
| <div class="pull-right"> |
| |
| |
| |
| <a href="../../start/onprem.html"><span class="label label-success">English</span></a> |
| |
| |
| |
| |
| </div> |
| |
| |
| <div class="col-md-9"> |
| <h1>物理集群上 Ozone 的安装</h1> |
| |
| <!--- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| <p>如果你想要有点挑战性，你可以在物理集群上安装 ozone。搭建一个 Ozone 集群需要了解它的各个组件，Ozone 既能和现有的 HDFS 集群并存运行，也可以独立运行。在这两种模式下，需要运行的 Ozone 组件是相同的。</p> |
| <h2 id="ozone-组件">Ozone 组件</h2> |
| <ol> |
| <li>Ozone Manager - 管理 Ozone 命名空间的服务，负责所有对卷、桶和键的操作。</li> |
| <li>Storage Container Manager - Ozone 中块的管理者，Ozone Manager 从 SCM 请求块，然后用户向块写入数据。</li> |
| <li>Datanodes - Ozone 的 Datanode 代码既可以运行在 HDFS 的 Datanode 内，也可以独立部署成单独的进程。</li> |
| </ol> |
| <h2 id="搭建一个独立-ozone-集群">搭建一个独立 Ozone 集群</h2> |
| <ul> |
| <li> |
| <p>将 ozone-<version> 安装包解压到目标目录，因为 Ozone 的 jar 包需要部署到集群的所有机器上，所以你需要在所有机器上进行此操作。</p> |
| </li> |
| <li> |
| <p>Ozone 依赖名为 <code>ozone-site.xml</code> 的配置文件， 运行下面的命令可以在指定目录生成名为 <code>ozone-site.xml</code> 的配置文件模板，然后你可以将参数替换为合适的值。</p> |
| </li> |
| </ul> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone genconf <path></code></pre></div> |
| <p>我们来看看生成的文件（ozone-site.xml）中都有哪些参数，以及它们是如何影响 ozone 的。当各个参数都配置了合适的值之后，需要把该文件拷贝到 <code>ozone directory/etc/hadoop</code>。</p> |
| <ul> |
| <li><strong>ozone.metadata.dirs</strong> 管理员通过此参数指定元数据的存储位置，通常应该选择最快的磁盘（比如 SSD，如果节点上有的话），OM、SCM 和 Datanode |
| 会将元数据写入此路径。这是个必需的参数，如果不配置它，Ozone 会启动失败。</li> |
| </ul> |
| <p>示例如下：</p> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span> |
| <span style="color:#f92672"><name></span>ozone.metadata.dirs<span style="color:#f92672"></name></span> |
| <span style="color:#f92672"><value></span>/data/disk1/meta<span style="color:#f92672"></value></span> |
| <span style="color:#f92672"></property></span></code></pre></div> |
| <ul> |
| <li><strong>ozone.scm.names</strong> Storage container manager(SCM) 提供 ozone 使用的分布式块服务，Datanode 通过这个参数来连接 SCM 并向 SCM 发送心跳。Ozone |
| 目前尚未支持 SCM 的 HA，ozone.scm.names 只需配置单个 SCM 地址即可。</li> |
| </ul> |
| <p>示例如下：</p> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span> |
| <span style="color:#f92672"><name></span>ozone.scm.names<span style="color:#f92672"></name></span> |
| <span style="color:#f92672"><value></span>scm.hadoop.apache.org<span style="color:#f92672"></value></span> |
| <span style="color:#f92672"></property></span> |
| </code></pre></div> |
| <ul> |
| <li><strong>ozone.scm.datanode.id.dir</strong> 每个 Datanode 会生成一个唯一 ID，叫做 Datanode ID。Datanode ID 会被写入此参数所指定路径下名为 datanode.id |
| 的文件中，如果该路径不存在，Datanode 会自动创建。</li> |
| </ul> |
| <p>示例如下：</p> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span> |
| <span style="color:#f92672"><name></span>ozone.scm.datanode.id.dir<span style="color:#f92672"></name></span> |
| <span style="color:#f92672"><value></span>/data/disk1/meta/node<span style="color:#f92672"></value></span> |
| <span style="color:#f92672"></property></span></code></pre></div> |
| <ul> |
| <li><strong>ozone.om.address</strong> OM 服务地址，OzoneClient 和 Ozone 文件系统需要使用此地址。</li> |
| </ul> |
| <p>示例如下：</p> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span> |
| <span style="color:#f92672"><name></span>ozone.om.address<span style="color:#f92672"></name></span> |
| <span style="color:#f92672"><value></span>ozonemanager.hadoop.apache.org<span style="color:#f92672"></value></span> |
| <span style="color:#f92672"></property></span></code></pre></div> |
| <h2 id="ozone-参数汇总">Ozone 参数汇总</h2> |
| <table> |
| <thead> |
| <tr> |
| <th>Setting</th> |
| <th>Value</th> |
| <th>Comment</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td>ozone.metadata.dirs</td> |
| <td>文件路径</td> |
| <td>元数据存储位置</td> |
| </tr> |
| <tr> |
| <td>ozone.scm.names</td> |
| <td>SCM 服务地址</td> |
| <td>SCM的主机名:端口，或者IP:端口</td> |
| </tr> |
| <tr> |
| <td>ozone.scm.block.client.address</td> |
| <td>SCM 服务地址和端口</td> |
| <td>Ozone 内部服务使用（如 OM）</td> |
| </tr> |
| <tr> |
| <td>ozone.scm.client.address</td> |
| <td>SCM 服务地址和端口</td> |
| <td>客户端使用</td> |
| </tr> |
| <tr> |
| <td>ozone.scm.datanode.address</td> |
| <td>SCM 服务地址和端口</td> |
| <td>Datanode 使用</td> |
| </tr> |
| <tr> |
| <td>ozone.om.address</td> |
| <td>OM 服务地址</td> |
| <td>Ozone handler 和 Ozone 文件系统使用</td> |
| </tr> |
| </tbody> |
| </table> |
| <h2 id="启动集群">启动集群</h2> |
| <p>在启动 Ozone 集群之前，需要依次初始化 SCM 和 OM。</p> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone scm --init</code></pre></div> |
| <p>这条命令会使 SCM 创建集群 ID 并初始化它的状态。 |
| <code>init</code> 命令和 Namenode 的 <code>format</code> 命令类似，只需要执行一次，SCM 就可以在磁盘上准备好正常运行所需的数据结构。</p> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone --daemon start scm</code></pre></div> |
| <p>SCM 启动之后，我们就可以创建对象存储空间，命令如下：</p> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone om --init</code></pre></div> |
| <p>OM 初始化完成之后，就可以启动 OM 服务了：</p> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone --daemon start om</code></pre></div> |
| <p>此时 Ozone 的命名服务 OM 和 块服务 SCM 都已运行。<br> |
| <strong>注意</strong>: 如果 SCM 未启动，<code>om --init</code> 命令会失败，同样，如果磁盘上的元数据缺失，SCM 也无法启动，所以请确保 <code>scm --init</code> 和 <code>om --init</code> 两条命令都成功执行了。</p> |
| <p>接下来启动 Datanode，在每个 Datanode 上运行下面的命令：</p> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone --daemon start datanode</code></pre></div> |
| <p>现在 SCM、OM 和所有的 Datanode 都已启动并运行。</p> |
| <p><em><strong>恭喜！你成功地搭建了一个完整的 ozone 集群。</strong></em></p> |
| <h2 id="捷径">捷径</h2> |
| <p>如果你想简化操作，可以直接运行：</p> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone scm --init |
| ozone om --init |
| start-ozone.sh</code></pre></div> |
| <p>这么做的前提是，<code>workers</code> 文件已经正确编写，并且配置好了到各个 Datanode 的 ssh，这和 HDFS 的配置方式相同，具体方法请查看 HDFS 文档。</p> |
| |
| |
| |
| <a class="btn btn-success btn-lg" href="../../zh/start/minikube.html">Next >></a> |
| |
| </div> |
| |
| </div> |
| </div> |
| </div> |
| </div> |
| <div class="push"></div> |
| </div> |
| |
| |
| |
| <footer class="footer"> |
| <div class="container"> |
| <span class="small text-muted"> |
| Version: 1.1.0, Last Modified: July 31, 2020 <a class="hide-child link primary-color" href="https://github.com/apache/ozone/commit/0651be8ebddeaefa884a5194303696d1384f144d">0651be8eb</a> |
| </span> |
| </div> |
| </footer> |
| |
| |
| |
| <script src="../../js/jquery-3.5.1.min.js"></script> |
| <script src="../../js/ozonedoc.js"></script> |
| <script src="../../js/bootstrap.min.js"></script> |
| |
| |
| </body> |
| |
| </html> |
