| |
| |
| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| |
| <meta name="description" content="Apache Ozone Documentation"> |
| |
| <title>Documentation for Apache Ozone</title> |
| |
| |
| <link href="../css/bootstrap.min.css" rel="stylesheet"> |
| |
| |
| <link href="../css/ozonedoc.css" rel="stylesheet"> |
| |
| |
| |
| <link href="../swagger-resources/swagger-ui.css" rel="stylesheet"> |
| |
| |
| <script> |
| var _paq = window._paq = window._paq || []; |
| |
| |
| |
| _paq.push(['disableCookies']); |
| |
| |
| _paq.push(['trackPageView']); |
| _paq.push(['enableLinkTracking']); |
| (function() { |
| var u="//analytics.apache.org/"; |
| _paq.push(['setTrackerUrl', u+'matomo.php']); |
| _paq.push(['setSiteId', '34']); |
| var d=document, g=d.createElement('script'), |
| s=d.getElementsByTagName('script')[0]; |
| g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s); |
| })(); |
| </script> |
| |
| |
| </head> |
| |
| |
| <body> |
| |
| <nav class="navbar navbar-inverse navbar-fixed-top"> |
| <div class="container-fluid"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#sidebar" aria-expanded="false" aria-controls="navbar"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a href="../index.html" class="navbar-left ozone-logo"> |
| <img src="../ozone-logo-small.png"/> |
| </a> |
| <a class="navbar-brand hidden-xs" href="../index.html"> |
| Apache Ozone/HDDS Documentation |
| </a> |
| <a class="navbar-brand visible-xs-inline" href="#">Apache Ozone</a> |
| </div> |
| <div id="navbar" class="navbar-collapse collapse"> |
| <ul class="nav navbar-nav navbar-right"> |
| <li><a href="https://github.com/apache/ozone">Source</a></li> |
| <li><a href="https://ozone.apache.org">Apache Ozone</a></li> |
| <li><a href="https://apache.org">ASF</a></li> |
| </ul> |
| </div> |
| </div> |
| </nav> |
| |
| <div class="container-fluid"> |
| <div class="row"> |
| |
| <div class="col-sm-2 col-md-2 sidebar" id="sidebar"> |
| <ul class="nav nav-sidebar"> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../index.html"> |
| |
| |
| |
| <span>Overview</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../start.html"> |
| |
| |
| |
| <span>Getting Started</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="../concept.html"> |
| |
| <span>Architecture</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| |
| <a href="../concept/overview.html">Overview</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../concept/ozonemanager.html">Ozone Manager</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../concept/storagecontainermanager.html">Storage Container Manager</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../concept/containers.html">Containers</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../concept/datanodes.html">Datanodes</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../concept/recon.html">Recon</a> |
| |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="../feature.html"> |
| |
| <span>Features</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| |
| <a href="../feature/decommission.html">Decommissioning</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../feature/om-ha.html">OM High Availability</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../feature/erasurecoding.html">Ozone Erasure Coding</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../feature/snapshot.html">Ozone Snapshot</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../feature/scm-ha.html">SCM High Availability</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../feature/streaming-write-pipeline.html">Streaming Write Pipeline</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../feature/dn-merge-rocksdb.html">Merge Container RocksDB in DN</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../feature/prefixfso.html">Prefix based File System Optimization</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../feature/topology.html">Topology awareness</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../feature/quota.html">Quota in Ozone</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../feature/recon.html">Recon Server</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../feature/observability.html">Observability</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../feature/nonrolling-upgrade.html">Non-Rolling Upgrades and Downgrades</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../feature/s3-multi-tenancy.html"> |
| |
| <span>S3 Multi-Tenancy</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| <a href="../feature/s3-multi-tenancy-setup.html">Setup</a> |
| </li> |
| |
| <li class=""> |
| <a href="../feature/s3-tenant-commands.html">Tenant commands</a> |
| </li> |
| |
| <li class=""> |
| <a href="../feature/s3-multi-tenancy-access-control.html">Access Control</a> |
| </li> |
| |
| </ul> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../feature/reconfigurability.html">Reconfigurability</a> |
| |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="../interface.html"> |
| |
| <span>Client Interfaces</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| |
| <a href="../interface/ofs.html">Ofs (Hadoop compatible)</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../interface/o3fs.html">O3fs (Hadoop compatible)</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../interface/s3.html">S3 Protocol</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../interface/cli.html">Command Line Interface</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../interface/reconapi.html">Recon API</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../interface/javaapi.html">Java API</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../interface/csi.html">CSI Protocol</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../interface/httpfs.html">HttpFS Gateway</a> |
| |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| |
| <li class=""> |
| <a href="../security.html"> |
| |
| <span>Security</span> |
| </a> |
| <ul class="nav"> |
| |
| <li class=""> |
| |
| <a href="../security/secureozone.html">Securing Ozone</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../security/securingtde.html">Transparent Data Encryption</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../security/gdpr.html">GDPR in Ozone</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../security/securingdatanodes.html">Securing Datanodes</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../security/securingozonehttp.html">Securing HTTP</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../security/securings3.html">Securing S3</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../security/securityacls.html">Ozone ACLs</a> |
| |
| </li> |
| |
| <li class=""> |
| |
| <a href="../security/securitywithranger.html">Apache Ranger</a> |
| |
| </li> |
| |
| </ul> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../tools.html"> |
| |
| |
| |
| <span>Tools</span> |
| </a> |
| </li> |
| |
| |
| |
| <li class=""> |
| |
| <a href="../recipe.html"> |
| |
| |
| |
| <span>Recipes</span> |
| </a> |
| </li> |
| |
| |
| <li><a href="../design.html"><span><b>Design docs</b></span></a></li> |
| <li class="visible-xs"><a href="#">References</a> |
| <ul class="nav"> |
| <li><a href="https://github.com/apache/ozone"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> Source</a></li> |
| <li><a href="https://ozone.apache.org"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> Apache Ozone</a></li> |
| <li><a href="https://apache.org"><span class="glyphicon glyphicon-new-window" aria-hidden="true"></span> ASF</a></li> |
| </ul></li> |
| </ul> |
| |
| </div> |
| |
| <div class="col-sm-9 col-sm-offset-3 col-md-10 col-md-offset-2 main-content"> |
| <div class="col-md-9"> |
| <h1><a href="https://issues.apache.org/jira/browse/">[]</a> Upgrade - Developer Primer () </h1> |
| <div><i>Authors: Aravindan Vijayan</i><div class="pull-right">2021-02-15</div></div> |
| <p> </p> |
| |
| <div class="panel panel-success"> |
| <div class="panel-heading">Summary</div> |
| <div class="panel-body"> |
| Helpful resources for those who are bringing layout changes. |
| </div> |
| </div> |
| |
| <!-- |
| Licensed under the Apache License, Version 2.0 (the "License"); |
| you may not use this file except in compliance with the License. |
| You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. See accompanying LICENSE file. |
| --> |
| <h1 id="terminologies">Terminologies</h1> |
| <h2 id="layout-feature">Layout Feature</h2> |
| <p>A layout feature is any new Ozone feature that makes a backward incompatible change to the on disk layout. Each layout feature is associated with a layout version that it defines. A component has a list of monotonically increasing layout features (versions) that it supports.</p> |
| <h2 id="finalizing--pre-finalized-state">Finalizing & Pre-Finalized State</h2> |
| <p>When upgrading a component from an older version to a newer version which has a higher layout version, the component automatically goes into a pre-finalized state after which an explicit ‘finalize’ action is required from the user to finalize it. In the pre-finalized state, commands/APIs/on disk structures used and created by newer layout features are meant to be unsupported or unused. After finalizing, the newer layout feature APIs are supported.</p> |
| <h2 id="downgrade">Downgrade</h2> |
| <p>Downgrade to a lower version is allowed from the pre-finalized state. This involves stopping the component, replacing the artifacts to the older version, and then starting it up again.</p> |
| <h1 id="useful-framework-tools-to-use">Useful framework tools to use</h1> |
| <h2 id="layoutfeature">LayoutFeature</h2> |
| <pre><code>org.apache.hadoop.ozone.om.upgrade.OMLayoutFeature |
| org.apache.hadoop.hdds.upgrade.HDDSLayoutFeature |
| </code></pre> |
| <p>Class to add a new layout feature being brought in. Layout version is typically 1 + last layout feature in that catalog.</p> |
| <h2 id="layoutversionmanager">LayoutVersionManager</h2> |
| <pre><code>org.apache.hadoop.ozone.om.upgrade.OMLayoutVersionManager |
| org.apache.hadoop.hdds.upgrade.HDDSLayoutVersionManager |
| </code></pre> |
| <p>Every component carries an instance of this interface, which provides APIs to get runtime layout version, and if a feature is allowed based on that or not.</p> |
| <p>The LayoutVersionManager interface carries an API that can be used to check if a feature is allowed in the current layout version. |
| org.apache.hadoop.ozone.upgrade.LayoutVersionManager#isAllowed(org.apache.hadoop.ozone.upgrade.LayoutFeature)</p> |
| <h2 id="disalloweduntillayoutversion-annotation">@DisallowedUntilLayoutVersion Annotation</h2> |
| <p>Method level annotation used to “disallow” an API if current layout version does not include the associated layout feature. Currently it is added only to the OM module, but can easily be moved down to a common module based on need on the HDDS layer.</p> |
| <h2 id="belongstolayoutversion-annotation">@BelongsToLayoutVersion Annotation</h2> |
| <p>Annotation to mark an OM request class that it belongs to a specific Layout Version. Until that version is available post finalize, this request will not be supported. A newer version of an existing OM request can be created (by inheritance or a fully new class) and marked with a newer layout version. Until finalizing this layout version, the older request class is used. Post finalizing, the newer version of the request class is used.</p> |
| <h2 id="layoutversioninstancefactoryt">LayoutVersionInstanceFactory<T></h2> |
| <p>Generic factory which stores different instances of Type ‘T’ sharded by a key & version. A single key can be associated with different versions of ‘T’.</p> |
| <h3 id="why-does-this-class-exist">Why does this class exist?</h3> |
| <p>A typical use case during upgrade is to have multiple versions of a class / method / object and choose them based on the current layout version at runtime. Before finalizing, an older version is typically needed, and after finalization, a newer version is needed. This class serves this purpose in a generic way. For example, we can create a Factory to create multiple versions of OMRequests sharded by Request Type & Layout Version Supported.</p> |
| <h2 id="upgrade-action-upgradeactionom--upgradeactionhdds">Upgrade Action (UpgradeActionOm & UpgradeActionHdds)</h2> |
| <p>Annotation to specify upgrade action run during specific upgrade phases. Each layout feature can optionally define an upgrade action for every supported phase. These are the supported phases of action callbacks.</p> |
| <h4 id="validate_in_prefinalize">VALIDATE_IN_PREFINALIZE</h4> |
| <p>A ‘validation’ action run every time a component is started up with this layout feature being unfinalized.</p> |
| <ul> |
| <li> |
| <p>Example: Stopping a component if a new configuration is used prior to it being finalized.</p> |
| </li> |
| <li> |
| <p>Example: Cleaning up from a failed ON_FINALIZE action that may have left on disk data in an inoperable state.</p> |
| <ul> |
| <li>Note that because the ON_FINALIZE action failed, the feature remains pre-finalized.</li> |
| </ul> |
| </li> |
| </ul> |
| <h4 id="on_first_upgrade_start">ON_FIRST_UPGRADE_START</h4> |
| <p>A backward compatible action run once when an upgraded cluster is detected with this new layout version. This differs from VALIDATE_IN_PREFINALIZE because it will not be run again once it completes successfully. |
| This action will be run again if it fails partway through, and may be run again if another error occurs during the upgrade. The action must always leave on disk data in a backwards compatible state, even if it fails partway through, since it is being executed before finalization.</p> |
| <ul> |
| <li>Example: The new version expects data in a different location even when it is pre-finalized. The action creates a symlink in the new location pointing to the old location.</li> |
| </ul> |
| <h4 id="on_finalize">ON_FINALIZE</h4> |
| <p>An action run once during finalization of layout version (feature). This action will be run again if it fails partway through, and may be run again if another error occurs during the upgrade. This is the only action permitted to make backwards incompatible changes to on disk structures, since finalization has been initiated by the time it is run. If a failure partway through could leave the component in an inoperable state, a cleanup action should be used in VALIDATE_IN_PREFINALIZE, which will be run when the component is restarted after a failure.</p> |
| <ul> |
| <li> |
| <p>Example: Adding a new RocksDB column family.</p> |
| </li> |
| <li> |
| <p>Example: Logging a message saying a feature is being finalized.</p> |
| </li> |
| </ul> |
| <h2 id="prepare-the-ozone-manager">‘Prepare’ the Ozone Manager</h2> |
| <p>Used to flush all transactions to disk, take a DB snapshot, and purge the logs, leaving Ratis in a clean state without unapplied log entries. This prepares the OM for upgrades/downgrades so that no request in the log is applied to the database in the old version of the code in one OM, and the new version of the code in another OM.</p> |
| <p>To prepare an OM quorum, run</p> |
| <pre><code>ozone admin om -id=<om-sevice-id> prepare |
| </code></pre> |
| <p>To cancel preparation of an OM quorum, run</p> |
| <pre><code>ozone admin om -id=<om-sevice-id> cancelprepare |
| </code></pre> |
| <h2 id="when-do-you-bring-in-a-change-as-a-layout-feature">When do you bring in a change as a Layout feature?</h2> |
| <p>By using the new feature, if it creates a change in disk layout (RocksDB, HDDS Volume etc) that is incompatible with the older version, then that qualifies as a layout feature.</p> |
| <h4 id="what-are-some-examples-of-changes-that-need-not-be-layout-features">What are some examples of changes that need not be layout features?</h4> |
| <ul> |
| <li>A feature that creates a table and writes into it. On downgrade, the table |
| is no longer accessed (as expected), but does not interfere with the existing |
| functionality.</li> |
| <li>A change to an existing OM request that does not change on disk layout.</li> |
| </ul> |
| <h2 id="testing-layout-feature-onboarding">Testing layout feature onboarding</h2> |
| <ul> |
| <li>Ability to deploy Mini ozone cluster with any layout version |
| <ul> |
| <li>org.apache.hadoop.ozone.MiniOzoneCluster.Builder#setScmLayoutVersion</li> |
| <li>org.apache.hadoop.ozone.MiniOzoneCluster.Builder#setOmLayoutVersion</li> |
| <li>org.apache.hadoop.ozone.MiniOzoneCluster.Builder#setDnLayoutVersion</li> |
| </ul> |
| </li> |
| <li>Acceptance test framework provides callbacks at various upgrade stages |
| (older, pre-finalize, on downgrade, on finalize) to plug in data ingestion / verification logic. |
| <ul> |
| <li>Please refer to hadoop-ozone/dist/src/main/compose/upgrade/README.md</li> |
| </ul> |
| </li> |
| </ul> |
| |
| </div> |
| |
| </div> |
| </div> |
| </div> |
| |
| |
| |
| <footer class="footer"> |
| <div class="container"> |
| <span class="small text-muted"> |
| Version: 1.5.0-SNAPSHOT, Last Modified: February 26, 2024 <a class="hide-child link primary-color" href="https://github.com/apache/ozone/commit/1b48186a0107711235abcd2636977ae0242f6be8">1b48186</a> |
| </span> |
| </div> |
| </footer> |
| |
| |
| |
| <script src="../js/jquery-3.5.1.min.js"></script> |
| <script src="../js/ozonedoc.js"></script> |
| <script src="../js/bootstrap.min.js"></script> |
| |
| |
| </body> |
| </html> |
