| |
| |
| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| |
| <meta name="description" content="Apache Ozone Documentation"> |
| |
| <title>Documentation for Apache Ozone</title> |
| |
| |
| <link href="../../css/bootstrap.min.css" rel="stylesheet"> |
| |
| |
| <link href="../../css/ozonedoc.css" rel="stylesheet"> |
| |
| |
| |
| <link href="../../swagger-resources/swagger-ui.css" rel="stylesheet"> |
| |
| |
| <script> |
| var _paq = window._paq = window._paq || []; |
| |
| |
| |
| _paq.push(['disableCookies']); |
| |
| |
| _paq.push(['trackPageView']); |
| _paq.push(['enableLinkTracking']); |
| (function() { |
| var u="//analytics.apache.org/"; |
| _paq.push(['setTrackerUrl', u+'matomo.php']); |
| _paq.push(['setSiteId', '34']); |
| var d=document, g=d.createElement('script'), |
| s=d.getElementsByTagName('script')[0]; |
| g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s); |
| })(); |
| </script> |
| |
| |
| </head> |
| |
| |
| <body> |
| |
| |
| <nav class="navbar navbar-inverse navbar-fixed-top"> |
| <div class="container-fluid"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#sidebar" aria-expanded="false" aria-controls="navbar"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a href="../../zh/index.html" class="navbar-left ozone-logo"> |
| <img src="../../ozone-logo-small.png"/> |
| </a> |
| <a class="navbar-brand hidden-xs" href="../../zh/index.html"> |
| Apache Ozone/HDDS Documentation |
| </a> |
| <a class="navbar-brand visible-xs-inline" href="#">Apache Ozone</a> |
| </div> |
| <div id="navbar" class="navbar-collapse collapse"> |
| <ul class="nav navbar-nav navbar-right"> |
| <li><a href="https://github.com/apache/ozone">Source</a></li> |
| <li><a href="https://ozone.apache.org">Apache Ozone</a></li> |
| <li><a href="https://apache.org">ASF</a></li> |
| </ul> |
| </div> |
| </div> |
| </nav> |
| |
| |
| <div class="wrapper"> |
| <div class="container-fluid"> |
| <div class="row"> |
| |
| <div class="col-sm-2 col-md-2 sidebar" id="sidebar"> |
| <ul class="nav nav-sidebar"> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/index.html"> |
| |
| |
| |
| <span>概述</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/start.html"> |
| |
| |
| |
| <span>快速入门</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="../../zh/concept.html"> |
| |
| <span>概念</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| |
| <a href="../../zh/concept/overview.html">概览</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/concept/ozonemanager.html">Ozone Manager</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/concept/storagecontainermanager.html">Storage Container Manager</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/concept/datanodes.html">数据节点</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/concept/containers.html">Containers</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/concept/recon.html">Recon</a> |
| |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="../../zh/feature.html"> |
| |
| <span>特性</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| |
| <a href="../../zh/feature/decommission.html">Decommissioning</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/feature/erasurecoding.html">纠删码</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/feature/om-ha.html">高可用 OM</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/feature/scm-ha.html">高可用 SCM</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/feature/dn-merge-rocksdb.html">在DataNode上合并Container的RocksDB</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/feature/prefixfso.html">基于前缀的文件系统优化</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/feature/topology.html">拓扑感知能力</a> |
| |
| </li> |
| |
| <li class="active"> |
| |
| <a href="../../zh/feature/quota.html">Ozone 中的配额</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/feature/recon.html">Recon 服务器</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/feature/reconfigurability.html">动态加载配置</a> |
| |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="../../zh/security.html"> |
| |
| <span>安全</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| |
| <a href="../../zh/security/secureozone.html">安全化 Ozone</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/security/securingtde.html">透明数据加密</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/security/gdpr.html">Ozone 中的 GDPR</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/security/securingdatanodes.html">安全化 Datanode</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/security/securings3.html">安全化 S3</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/security/securityacls.html">Ozone 访问控制列表</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/security/securitywithranger.html">Apache Ranger</a> |
| |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="../../zh/interface.html"> |
| |
| <span>编程接口</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| |
| <a href="../../zh/interface/javaapi.html">Java API</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/interface/o3fs.html">Ozone 文件系统</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/interface/csi.html">CSI 协议</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/interface/s3.html">S3 协议接口</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../../zh/interface/reconapi.html">Recon API</a> |
| |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/tools.html"> |
| |
| |
| |
| <span>工具</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../../zh/recipe.html"> |
| |
| |
| |
| <span>使用配方</span> |
| </a> |
| </li> |
| |
| |
| <li><a href="../../design.html"><span><b>Design docs</b></span></a></li> |
| <li class="visible-xs"><a href="#">References</a> |
| <ul class="nav"> |
| <li><a href="https://github.com/apache/ozone"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> Source</a></li> |
| <li><a href="https://ozone.apache.org"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> Apache Ozone</a></li> |
| <li><a href="https://apache.org"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> ASF</a></li> |
| </ul></li> |
| </ul> |
| |
| </div> |
| |
| <div class="col-sm-10 col-sm-offset-2 col-md-10 col-md-offset-2 main-content"> |
| |
| |
| |
| <div class="col-md-9"> |
| <nav aria-label="breadcrumb"> |
| <ol class="breadcrumb"> |
| <li class="breadcrumb-item"><a href="../../zh/index.html">Home</a></li> |
| <li class="breadcrumb-item" aria-current="page"><a href="../../zh/feature.html">特性</a></li> |
| <li class="breadcrumb-item active" aria-current="page">Ozone 中的配额</li> |
| </ol> |
| </nav> |
| |
| |
| |
| <div class="pull-right"> |
| |
| |
| |
| <a href="../../feature/quota.html"><span class="label label-success">English</span></a> |
| |
| |
| |
| |
| </div> |
| |
| |
| <div class="col-md-9"> |
| <h1>Ozone 中的配额</h1> |
| |
| <!--- |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| <p>目前，我们知道Ozone允许用户创建Volume，Bucket和Key。一个Volume通常包含几个Bucket，每个Bucket也包含一定数量的Key。显然，它应该允许用户定义配额（例如，在一个Volume下可以创建多少个Bucket或一个Bucket可以使用多少空间），这是存储系统的常见要求。</p> |
| <h2 id="目前支持的">目前支持的</h2> |
| <ol> |
| <li>Storage space级别配额</li> |
| </ol> |
| <p>管理员应该能够定义一个Volume或Bucket可以使用多少存储空间。目前支持以下storage space quota的设置:</p> |
| <p>a. 默认情况下volume和bucket的quota不启用。</p> |
| <p>b. 当volume quota启用时，bucket quota的总大小不能超过volume。</p> |
| <p>c. 可以在不启用volume quota的情况下单独给bucket设置quota。此时bucket quota的大小是不受限制的。</p> |
| <p>d. 目前不支持单独设置volume quota，只有在设置了bucket quota的情况下volume quota才会生效。因为ozone在写入key时只检查bucket的usedBytes。</p> |
| <p>e. 如果集群从小于1.1.0的旧版本升级而来，则不建议在旧volume和bucket(可以通过查看volume或者bucket的info确认，如果quota值是-2，那么这个volume或者bucket就是旧的)上使用配额。由于旧的key没有计算到bucket的usedBytes中，所以此时配额设置是不准确的。</p> |
| <p>f. 如果volume quota被启用，那么bucket quota将不能被清除。</p> |
| <p>g. Volume内的linked bucket不消耗Volume的空间配额。linked bcuket将消耗源Volume和源Bucket的空间配额。</p> |
| <ol start="2"> |
| <li>命名空间配额</li> |
| </ol> |
| <p>管理员应当能够定义一个Volume或Bucket可以使用多少命名空间。目前支持命名空间的配额设置为：</p> |
| <p>a. 默认情况下volume和bucket的命名空间配额不启用(即无限配额)。</p> |
| <p>b. 当volume命名空间配额启用时，该volume的bucket数目不能超过此配额。</p> |
| <p>c. 当bucket的命名空间配额启用时，该bucket的key数目不能超过此配额。</p> |
| <p>d. Linked bucket 上不支持定义单独的命名空间配额，它直接引用源bucket的命名空间配额。</p> |
| <p>e. Linked bucket将消耗卷的命名空间配额。</p> |
| <p>f. 如果集群从小于1.1.0的旧版本升级而来，则不建议在旧volume和bucket(可以通过查看volume或者bucket的info确认，如果quota值是-2，那么这个volume或者bucket就是旧的)上使用配额。由于旧的key没有计算到bucket的命名空间配额中，所以此时配额设置是不准确的。</p> |
| <p>g. 对于FSO bucket, 对移动到垃圾桶的文件和目录，垃圾桶将在以下情况下占用额外的命名空间：</p> |
| <ul> |
| <li>对于桶中垃圾的内部目录路径，即 <code>/.trash/<user>/<current or timestamp></code></li> |
| <li>用于将文件/目录移动到垃圾桶中对应层次结构时创建的额外路径。例如：</li> |
| </ul> |
| <pre><code>- 来源：/<vol>/<bucket>/dir1/dir2/file.txt |
| 场景 1： |
| - 将 file.txt 移至回收站（删除操作时） |
| - 使用“dir1”和“dir2” 在垃圾桶中创建与来源相同的路径 |
| /<vol>/<bucket>/.trash/<user>/current/dir1/dir2/file.txt |
| 所以这将消耗“2”的额外名称空间 |
| |
| 场景 2： |
| - 将 dir2 移至垃圾箱（删除操作时） |
| - 使用“dir1”在垃圾桶中创建与来源相同的路径 |
| /<vol>/<bucket>/.trash/<user>/current/dir1/dir2/file.txt |
| 所以这将为 dir1 消耗额外的命名空间“1” |
| |
| 场景 3： |
| - 将 dir1 移动到垃圾桶（删除操作时），在这种情况下，不需要额外的命名空间 |
| /<vol>/<bucket>/.trash/<user>/current/dir1/dir2/file.txt |
| |
| </code></pre><h3 id="笔记">笔记</h3> |
| <ul> |
| <li>对于FSO bucket，当递归删除目录时，配额将在子目录和文件被异步删除后释放（当目录被删除时，递归删除在后台进行）。</li> |
| <li>当配额即将达到限制，多个ozone 客户端（并行）提交文件时，后端按先到先得的顺序进行验证，满足配额的文件将成功提交。</li> |
| </ul> |
| <h2 id="客户端用法">客户端用法</h2> |
| <h3 id="storage-space级别配额">Storage space级别配额</h3> |
| <p>Storage space级别配额允许使用 B, KB ，MB ，GB ，TB 等单位。表示将使用多少个存储空间。</p> |
| <h4 id="注意">注意:</h4> |
| <ul> |
| <li>Volume 和 Bucket 不支持设置带小数点的配额值，例如 1.5 TB.</li> |
| <li>最小的有效空间配额，是一个数据块需要的存储空间，即默认块大小 * 副本数. 请确保设置的空间配额不小于这个数值，不然对象/文件写入操作，会失败。</li> |
| </ul> |
| <h4 id="volume-space-quota用法">Volume Space quota用法</h4> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">bin/ozone sh volume create --space-quota 5MB /volume1 |
| </code></pre></div><p>这意味着将volume1的存储空间设置为5MB</p> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">bin/ozone sh volume setquota --space-quota 10GB /volume1 |
| </code></pre></div><p>此行为将volume1的配额更改为10GB。</p> |
| <h4 id="bucket-space-quota-用法">Bucket Space quota 用法</h4> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">bin/ozone sh bucket create --space-quota 5MB /volume1/bucket1 |
| </code></pre></div><p>这意味着bucket1允许我们使用5MB的存储空间。</p> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">bin/ozone sh bucket setquota --space-quota 10GB /volume1/bucket1 |
| </code></pre></div><p>该行为将bucket1的配额更改为10GB</p> |
| <p>bucket的总配额 不应大于其Volume的配额。让我们看一个例子，如果我们有一个10MB的Volume，该volume下所有bucket的大小之和不能超过10MB，否则设置bucket quota将失败。</p> |
| <h4 id="清除volume和bucket的配额">清除volume和bucket的配额</h4> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">bin/ozone sh volume clrquota --space-quota /volume1 |
| bin/ozone sh bucket clrquota --space-quota /volume1/bucket1 |
| </code></pre></div><h4 id="查看volume和bucket的quota值以及usedbytes">查看volume和bucket的quota值以及usedBytes</h4> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">bin/ozone sh volume info /volume1 |
| bin/ozone sh bucket info /volume1/bucket1 |
| </code></pre></div><p>我们能够在volume和bucket的info中查看quota及usedBytes的值</p> |
| <h3 id="namespace-quota">Namespace quota</h3> |
| <p>命名空间配额是一个数字，其代表由多少个名字能够使用。这个数字不能超过Java long数据类型的最大值。</p> |
| <h4 id="volume-namespace-quota">Volume Namespace quota</h4> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">bin/ozone sh volume create --namespace-quota <span style="color:#ae81ff">100</span> /volume1 |
| </code></pre></div><p>这意味着将volume1的命名空间配额设置为100。</p> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">bin/ozone sh volume setquota --namespace-quota <span style="color:#ae81ff">1000</span> /volume1 |
| </code></pre></div><p>此行为将volume1的命名空间配额更改为1000。</p> |
| <h4 id="bucket-namespace-quota">Bucket Namespace quota</h4> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">bin/ozone sh bucket create --namespace-quota <span style="color:#ae81ff">100</span> /volume1/bucket1 |
| </code></pre></div><p>这意味着bucket1允许我们使用100的命名空间。</p> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">bin/ozone sh bucket setquota --namespace-quota <span style="color:#ae81ff">1000</span> /volume1/bucket1 |
| </code></pre></div><p>该行为将bucket1的命名空间配额更改为1000。</p> |
| <h4 id="清除volume和bucket的配额-1">清除volume和bucket的配额</h4> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">bin/ozone sh volume clrquota --namespace-quota /volume1 |
| bin/ozone sh bucket clrquota --namespace-quota /volume1/bucket1 |
| </code></pre></div><h4 id="查看volume和bucket的quota值以及usednamespace">查看volume和bucket的quota值以及usedNamespace</h4> |
| <div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">bin/ozone sh volume info /volume1 |
| bin/ozone sh bucket info /volume1/bucket1 |
| </code></pre></div><p>我们能够在volume和bucket的info中查看quota及usedNamespace的值</p> |
| |
| |
| |
| <a class="btn btn-success btn-lg" href="../../zh/feature/recon.html">Next >></a> |
| |
| </div> |
| |
| </div> |
| </div> |
| </div> |
| </div> |
| <div class="push"></div> |
| </div> |
| |
| |
| |
| <footer class="footer"> |
| <div class="container"> |
| <span class="small text-muted"> |
| Version: 1.5.0-SNAPSHOT, Last Modified: February 27, 2024 <a class="hide-child link primary-color" href="https://github.com/apache/ozone/commit/7939faf7d6c904bf1e4ad32baa5d6d0c1de19003">7939faf</a> |
| </span> |
| </div> |
| </footer> |
| |
| |
| |
| <script src="../../js/jquery-3.5.1.min.js"></script> |
| <script src="../../js/ozonedoc.js"></script> |
| <script src="../../js/bootstrap.min.js"></script> |
| |
| |
| </body> |
| |
| </html> |
