| |
| |
| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| |
| <meta name="description" content="Hadoop Ozone Documentation"> |
| |
| <title>Documentation for Apache Hadoop Ozone</title> |
| |
| |
| <link href="../../css/bootstrap.min.css" rel="stylesheet"> |
| |
| |
| <link href="../../css/ozonedoc.css" rel="stylesheet"> |
| |
| </head> |
| |
| |
| <body> |
| |
| |
| <nav class="navbar navbar-inverse navbar-fixed-top"> |
| <div class="container-fluid"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#sidebar" aria-expanded="false" aria-controls="navbar"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a href="#" class="navbar-left" style="height: 50px; padding: 5px 5px 5px 0;"> |
| <img src="../../ozone-logo-small.png" width="40"/> |
| </a> |
| <a class="navbar-brand hidden-xs" href="#"> |
| Apache Hadoop Ozone/HDDS documentation |
| </a> |
| <a class="navbar-brand visible-xs-inline" href="#">Hadoop Ozone</a> |
| </div> |
| <div id="navbar" class="navbar-collapse collapse"> |
| <ul class="nav navbar-nav navbar-right"> |
| <li><a href="https://github.com/apache/hadoop-ozone">Source</a></li> |
| <li><a href="https://hadoop.apache.org">Apache Hadoop</a></li> |
| <li><a href="https://apache.org">ASF</a></li> |
| </ul> |
| </div> |
| </div> |
| </nav> |
| |
| |
| <div class="container-fluid"> |
| <div class="row"> |
| |
| <div class="col-sm-2 col-md-2 sidebar" id="sidebar"> |
| <ul class="nav nav-sidebar"> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/"> |
| |
| |
| |
| <span>概述</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/start.html"> |
| |
| |
| |
| <span>快速入门</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/shell.html"> |
| |
| |
| |
| <span>命令行接口</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/interface.html"> |
| |
| |
| |
| <span>编程接口</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/gdpr.html"> |
| |
| |
| |
| <span>GDPR</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/concept.html"> |
| |
| |
| |
| <span>概念</span> |
| </a> |
| </li> |
| |
| |
| <li class="visible-xs"><a href="#">References</a> |
| <ul class="nav"> |
| <li><a href="https://github.com/apache/hadoop"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> Source</a></li> |
| <li><a href="https://hadoop.apache.org"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> Apache Hadoop</a></li> |
| <li><a href="https://apache.org"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> ASF</a></li> |
| </ul></li> |
| </ul> |
| |
| </div> |
| |
| <div class="col-sm-10 col-sm-offset-2 col-md-10 col-md-offset-2 main"> |
| |
| |
| |
| <div class="col-md-9"> |
| <nav aria-label="breadcrumb"> |
| <ol class="breadcrumb"> |
| <li class="breadcrumb-item"><a href="../../">Home</a></li> |
| <li class="breadcrumb-item" aria-current="page"><a href="../../zh/start.html">快速入门</a></li> |
| <li class="breadcrumb-item active" aria-current="page">物理集群上 Ozone 的安装</li> |
| </ol> |
| </nav> |
| |
| |
| |
| <div class="pull-right"> |
| |
| |
| <a href="../../start/onprem.html"><span class="label label-success">English</span></a> |
| |
| |
| |
| |
| </div> |
| |
| |
| <div class="col-md-9"> |
| <h1>物理集群上 Ozone 的安装</h1> |
| |
| |
| |
| <!--- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| |
| <p>如果你想要有点挑战性,你可以在物理集群上安装 ozone。搭建一个 Ozone 集群需要了解它的各个组件,Ozone 既能和现有的 HDFS 集群并存运行,也可以独立运行。在这两种模式下,需要运行的 Ozone 组件是相同的。</p> |
| |
| <h2 id="ozone-组件">Ozone 组件</h2> |
| |
| <ol> |
| <li>Ozone Manager - 管理 Ozone 命名空间的服务,负责所有对卷、桶和键的操作。</li> |
| <li>Storage Container Manager - Ozone 中块的管理者,Ozone Manager 从 SCM 请求块,然后用户向块写入数据。</li> |
| <li>Datanodes - Ozone 的 Datanode 代码既可以运行在 HDFS 的 Datanode 内,也可以独立部署成单独的进程。</li> |
| </ol> |
| |
| <h2 id="搭建一个独立-ozone-集群">搭建一个独立 Ozone 集群</h2> |
| |
| <ul> |
| <li><p>将 ozone-<version> 安装包解压到目标目录,因为 Ozone 的 jar 包需要部署到集群的所有机器上,所以你需要在所有机器上进行此操作。</p></li> |
| |
| <li><p>Ozone 依赖名为 <code>ozone-site.xml</code> 的配置文件, 运行下面的命令可以在指定目录生成名为 <code>ozone-site.xml</code> 的配置文件模板,然后你可以将参数替换为合适的值。</p></li> |
| </ul> |
| |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone genconf <path></code></pre></div> |
| |
| <p>我们来看看生成的文件(ozone-site.xml)中都有哪些参数,以及它们是如何影响 ozone 的。当各个参数都配置了合适的值之后,需要把该文件拷贝到 <code>ozone directory/etc/hadoop</code>。</p> |
| |
| <ul> |
| <li><strong>ozone.metadata.dirs</strong> 管理员通过此参数指定元数据的存储位置,通常应该选择最快的磁盘(比如 SSD,如果节点上有的话),OM、SCM 和 Datanode |
| 会将元数据写入此路径。这是个必需的参数,如果不配置它,Ozone 会启动失败。</li> |
| </ul> |
| |
| <p>示例如下:</p> |
| |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span> |
| <span style="color:#f92672"><name></span>ozone.metadata.dirs<span style="color:#f92672"></name></span> |
| <span style="color:#f92672"><value></span>/data/disk1/meta<span style="color:#f92672"></value></span> |
| <span style="color:#f92672"></property></span></code></pre></div> |
| |
| <ul> |
| <li><strong>ozone.scm.names</strong> Storage container manager(SCM) 提供 ozone 使用的分布式块服务,Datanode 通过这个参数来连接 SCM 并向 SCM 发送心跳。Ozone |
| 目前尚未支持 SCM 的 HA,ozone.scm.names 只需配置单个 SCM 地址即可。 |
| <br /></li> |
| </ul> |
| |
| <p>示例如下:</p> |
| |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span> |
| <span style="color:#f92672"><name></span>ozone.scm.names<span style="color:#f92672"></name></span> |
| <span style="color:#f92672"><value></span>scm.hadoop.apache.org<span style="color:#f92672"></value></span> |
| <span style="color:#f92672"></property></span> |
| </code></pre></div> |
| |
| <ul> |
| <li><strong>ozone.scm.datanode.id.dir</strong> 每个 Datanode 会生成一个唯一 ID,叫做 Datanode ID。Datanode ID 会被写入此参数所指定路径下名为 datanode.id |
| 的文件中,如果该路径不存在,Datanode 会自动创建。</li> |
| </ul> |
| |
| <p>示例如下:</p> |
| |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span> |
| <span style="color:#f92672"><name></span>ozone.scm.datanode.id.dir<span style="color:#f92672"></name></span> |
| <span style="color:#f92672"><value></span>/data/disk1/meta/node<span style="color:#f92672"></value></span> |
| <span style="color:#f92672"></property></span></code></pre></div> |
| |
| <ul> |
| <li><strong>ozone.om.address</strong> OM 服务地址,OzoneClient 和 Ozone 文件系统需要使用此地址。</li> |
| </ul> |
| |
| <p>示例如下:</p> |
| |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"> <span style="color:#f92672"><property></span> |
| <span style="color:#f92672"><name></span>ozone.om.address<span style="color:#f92672"></name></span> |
| <span style="color:#f92672"><value></span>ozonemanager.hadoop.apache.org<span style="color:#f92672"></value></span> |
| <span style="color:#f92672"></property></span></code></pre></div> |
| |
| <h2 id="ozone-参数汇总">Ozone 参数汇总</h2> |
| |
| <table> |
| <thead> |
| <tr> |
| <th>Setting</th> |
| <th>Value</th> |
| <th>Comment</th> |
| </tr> |
| </thead> |
| |
| <tbody> |
| <tr> |
| <td>ozone.metadata.dirs</td> |
| <td>文件路径</td> |
| <td>元数据存储位置</td> |
| </tr> |
| |
| <tr> |
| <td>ozone.scm.names</td> |
| <td>SCM 服务地址</td> |
| <td>SCM的主机名:端口,或者IP:端口</td> |
| </tr> |
| |
| <tr> |
| <td>ozone.scm.block.client.address</td> |
| <td>SCM 服务地址和端口</td> |
| <td>Ozone 内部服务使用(如 OM)</td> |
| </tr> |
| |
| <tr> |
| <td>ozone.scm.client.address</td> |
| <td>SCM 服务地址和端口</td> |
| <td>客户端使用</td> |
| </tr> |
| |
| <tr> |
| <td>ozone.scm.datanode.address</td> |
| <td>SCM 服务地址和端口</td> |
| <td>Datanode 使用</td> |
| </tr> |
| |
| <tr> |
| <td>ozone.om.address</td> |
| <td>OM 服务地址</td> |
| <td>Ozone handler 和 Ozone 文件系统使用</td> |
| </tr> |
| </tbody> |
| </table> |
| |
| <h2 id="启动集群">启动集群</h2> |
| |
| <p>在启动 Ozone 集群之前,需要依次初始化 SCM 和 OM。</p> |
| |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone scm --init</code></pre></div> |
| |
| <p>这条命令会使 SCM 创建集群 ID 并初始化它的状态。 |
| <code>init</code> 命令和 Namenode 的 <code>format</code> 命令类似,只需要执行一次,SCM 就可以在磁盘上准备好正常运行所需的数据结构。</p> |
| |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone --daemon start scm</code></pre></div> |
| |
| <p>SCM 启动之后,我们就可以创建对象存储空间,命令如下:</p> |
| |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone om --init</code></pre></div> |
| |
| <p>OM 初始化完成之后,就可以启动 OM 服务了:</p> |
| |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone --daemon start om</code></pre></div> |
| |
| <p>此时 Ozone 的命名服务 OM 和 块服务 SCM 都已运行。<br /> |
| <strong>注意</strong>: 如果 SCM 未启动,<code>om --init</code> 命令会失败,同样,如果磁盘上的元数据缺失,SCM 也无法启动,所以请确保 <code>scm --init</code> 和 <code>om --init</code> 两条命令都成功执行了。</p> |
| |
| <p>接下来启动 Datanode,在每个 Datanode 上运行下面的命令:</p> |
| |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone --daemon start datanode</code></pre></div> |
| |
| <p>现在 SCM、OM 和所有的 Datanode 都已启动并运行。</p> |
| |
| <p><strong><em>恭喜!你成功地搭建了一个完整的 ozone 集群。</em></strong></p> |
| |
| <h2 id="捷径">捷径</h2> |
| |
| <p>如果你想简化操作,可以直接运行:</p> |
| |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">ozone scm --init |
| ozone om --init |
| start-ozone.sh</code></pre></div> |
| |
| <p>这么做的前提是,slaves 文件已经正确编写,并且配置好了到各个 Datanode 的 ssh,这和 HDFS 的配置方式相同,具体方法请查看 HDFS 文档。</p> |
| |
| |
| |
| <a class="btn btn-success btn-lg" href="../../zh/start/minikube.html">Next >></a> |
| |
| </div> |
| |
| </div> |
| </div> |
| </div> |
| </div> |
| |
| |
| |
| |
| <script src="../../js/jquery-3.4.1.min.js"></script> |
| <script src="../../js/ozonedoc.js"></script> |
| <script src="../../js/bootstrap.min.js"></script> |
| |
| |
| </body> |
| |
| </html> |