| <!DOCTYPE html> |
| <html> |
| <head> |
| <meta charset="utf-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| <title>Pegasus | Table Soft Delete</title> |
| <link rel="stylesheet" href="/zh/assets/css/app.css"> |
| <link rel="shortcut icon" href="/zh/assets/images/favicon.ico"> |
| <link rel="stylesheet" href="/zh/assets/css/utilities.min.css"> |
| <link rel="stylesheet" href="/zh/assets/css/docsearch.v3.css"> |
| <script src="/assets/js/jquery.min.js"></script> |
| <script src="/assets/js/all.min.js"></script> |
| <script src="/assets/js/docsearch.v3.js"></script> |
| <!-- Begin Jekyll SEO tag v2.8.0 --> |
| <title>Table Soft Delete | Pegasus</title> |
| <meta name="generator" content="Jekyll v4.3.3" /> |
| <meta property="og:title" content="Table Soft Delete" /> |
| <meta property="og:locale" content="en_US" /> |
| <meta name="description" content="功能目标 软删除主要用于防范数据被错误的永久删除。具体而言，软删除应该具有如下几个功能： 当用户删除某个指定的表后，该表变得不可访问。 被删除的表其数据不会立即物理清除，而是会保留一段时间。 被删除的表过期（超过保留时间）后，数据会从物理上做彻底的删除，但是数据真正删除的时间还取决于其他因素，参见垃圾文件夹管理。 未过期的表可以通过一定的方式恢复，恢复成功后又可以像正常的表一样进行读写访问。" /> |
| <meta property="og:description" content="功能目标 软删除主要用于防范数据被错误的永久删除。具体而言，软删除应该具有如下几个功能： 当用户删除某个指定的表后，该表变得不可访问。 被删除的表其数据不会立即物理清除，而是会保留一段时间。 被删除的表过期（超过保留时间）后，数据会从物理上做彻底的删除，但是数据真正删除的时间还取决于其他因素，参见垃圾文件夹管理。 未过期的表可以通过一定的方式恢复，恢复成功后又可以像正常的表一样进行读写访问。" /> |
| <meta property="og:site_name" content="Pegasus" /> |
| <meta property="og:type" content="article" /> |
| <meta property="article:published_time" content="2024-04-22T06:39:52+00:00" /> |
| <meta name="twitter:card" content="summary" /> |
| <meta property="twitter:title" content="Table Soft Delete" /> |
| <script type="application/ld+json"> |
| {"@context":"https://schema.org","@type":"BlogPosting","dateModified":"2024-04-22T06:39:52+00:00","datePublished":"2024-04-22T06:39:52+00:00","description":"功能目标 软删除主要用于防范数据被错误的永久删除。具体而言，软删除应该具有如下几个功能： 当用户删除某个指定的表后，该表变得不可访问。 被删除的表其数据不会立即物理清除，而是会保留一段时间。 被删除的表过期（超过保留时间）后，数据会从物理上做彻底的删除，但是数据真正删除的时间还取决于其他因素，参见垃圾文件夹管理。 未过期的表可以通过一定的方式恢复，恢复成功后又可以像正常的表一样进行读写访问。","headline":"Table Soft Delete","mainEntityOfPage":{"@type":"WebPage","@id":"/administration/table-soft-delete"},"url":"/administration/table-soft-delete"}</script> |
| <!-- End Jekyll SEO tag --> |
| </head> |
| |
| |
| <body> |
| <div class="dashboard is-full-height"> |
| <!-- left panel --> |
| <div class="dashboard-panel is-medium is-hidden-mobile pl-0"> |
| <div class="dashboard-panel-header has-text-centered"> |
| <a href="/zh/"> |
| <img src="/assets/images/pegasus-logo-inv.png" style="width: 80%;"> |
| </a> |
| |
| </div> |
| <div class="dashboard-panel-main is-scrollable pl-6"> |
| |
| |
| <aside class="menu"> |
| |
| <p class="menu-label">Pegasus 产品文档</p> |
| <ul class="menu-list"> |
| |
| <li> |
| <a href="/zh/docs/downloads" |
| class=""> |
| 下载 |
| </a> |
| </li> |
| |
| </ul> |
| |
| <p class="menu-label">编译构建</p> |
| <ul class="menu-list"> |
| |
| <li> |
| <a href="/zh/docs/build/compile-by-docker" |
| class=""> |
| 使用 Docker 完成编译（推荐） |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/docs/build/compile-from-source" |
| class=""> |
| 从源码编译 |
| </a> |
| </li> |
| |
| </ul> |
| |
| <p class="menu-label">客户端库</p> |
| <ul class="menu-list"> |
| |
| <li> |
| <a href="/zh/clients/java-client" |
| class=""> |
| Java 客户端 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/clients/cpp-client" |
| class=""> |
| C++ 客户端 |
| </a> |
| </li> |
| |
| <li> |
| <a href="https://github.com/apache/incubator-pegasus/tree/master/go-client" |
| class=""> |
| Golang 客户端 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/clients/python-client" |
| class=""> |
| Python 客户端 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/clients/node-client" |
| class=""> |
| NodeJS 客户端 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/clients/scala-client" |
| class=""> |
| Scala 客户端 |
| </a> |
| </li> |
| |
| </ul> |
| |
| <p class="menu-label">生态工具</p> |
| <ul class="menu-list"> |
| |
| <li> |
| <a href="/zh/docs/tools/shell" |
| class=""> |
| Pegasus Shell 工具 |
| </a> |
| </li> |
| |
| <li> |
| <a href="https://github.com/pegasus-kv/admin-cli" |
| class=""> |
| 集群管理命令行 |
| </a> |
| </li> |
| |
| <li> |
| <a href="https://github.com/pegasus-kv/pegic" |
| class=""> |
| 数据访问命令行 |
| </a> |
| </li> |
| |
| </ul> |
| |
| <p class="menu-label">用户接口</p> |
| <ul class="menu-list"> |
| |
| <li> |
| <a href="/zh/api/ttl" |
| class=""> |
| TTL |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/api/single-atomic" |
| class=""> |
| 单行原子操作 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/api/redis" |
| class=""> |
| Redis 适配 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/api/geo" |
| class=""> |
| GEO 支持 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/api/http" |
| class=""> |
| HTTP 接口 |
| </a> |
| </li> |
| |
| </ul> |
| |
| <p class="menu-label">高效运维</p> |
| <ul class="menu-list"> |
| |
| <li> |
| <a href="/zh/administration/deployment" |
| class=""> |
| 集群部署 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/config" |
| class=""> |
| 配置说明 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/rebalance" |
| class=""> |
| 负载均衡 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/monitoring" |
| class=""> |
| 可视化监控 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/rolling-update" |
| class=""> |
| 集群重启和升级 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/scale-in-out" |
| class=""> |
| 集群扩容缩容 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/resource-management" |
| class=""> |
| 资源管理 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/cold-backup" |
| class=""> |
| 冷备份 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/meta-recovery" |
| class=""> |
| 元数据恢复 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/replica-recovery" |
| class=""> |
| Replica 数据恢复 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/zk-migration" |
| class=""> |
| Zookeeper 迁移 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/table-migration" |
| class=""> |
| Table 迁移 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/table-soft-delete" |
| class="is-active"> |
| Table 软删除 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/table-env" |
| class=""> |
| Table 环境变量 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/remote-commands" |
| class=""> |
| 远程命令 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/partition-split" |
| class=""> |
| Partition-Split |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/duplication" |
| class=""> |
| 跨机房同步 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/compression" |
| class=""> |
| 数据压缩 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/throttling" |
| class=""> |
| 流量控制 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/experiences" |
| class=""> |
| 运维经验 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/manual-compact" |
| class=""> |
| Manual Compact 功能 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/usage-scenario" |
| class=""> |
| Usage Scenario 功能 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/bad-disk" |
| class=""> |
| 坏盘检修 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/whitelist" |
| class=""> |
| Replica Server 白名单 |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/backup-request" |
| class=""> |
| Backup Request |
| </a> |
| </li> |
| |
| <li> |
| <a href="/zh/administration/hotspot-detection" |
| class=""> |
| 热点检测 |
| </a> |
| </li> |
| |
| </ul> |
| |
| </aside> |
| </div> |
| </div> |
| |
| <!-- main section --> |
| <div class="dashboard-main is-scrollable"> |
| <nav class="navbar is-hidden-desktop"> |
| <div class="navbar-brand"> |
| <a href="/zh/" class="navbar-item"> |
| <!-- Pegasus Icon --> |
| <img src="/assets/images/pegasus-square.png"> |
| </a> |
| <div class="navbar-item"> |
| |
| |
| <!--A simple language switch button that only supports zh and en.--> |
| <!--IF its language is zh, then switches to en.--> |
| |
| <!--If you don't want a url to be relativized, you can add a space explicitly into the href to |
| prevents a url from being relativized by polyglot.--> |
| <a class="button is-light is-outlined is-inverted" href=" /administration/table-soft-delete"><strong>En</strong></a> |
| |
| </div> |
| <a role="button" class="navbar-burger burger" aria-label="menu" aria-expanded="false" data-target="navMenu"> |
| <!-- Appears in mobile mode only --> |
| <span aria-hidden="true"></span> |
| <span aria-hidden="true"></span> |
| <span aria-hidden="true"></span> |
| </a> |
| </div> |
| <div class="navbar-menu" id="navMenu"> |
| <div class="navbar-end"> |
| |
| <!--dropdown--> |
| <div class="navbar-item has-dropdown is-hoverable"> |
| <a href="" |
| class="navbar-link "> |
| <span> |
| Pegasus 产品文档 |
| </span> |
| </a> |
| <div class="navbar-dropdown"> |
| |
| <a href="/zh/docs/downloads" |
| class="navbar-item "> |
| 下载 |
| </a> |
| |
| </div> |
| </div> |
| |
| <!--dropdown--> |
| <div class="navbar-item has-dropdown is-hoverable"> |
| <a href="" |
| class="navbar-link "> |
| <span> |
| 编译构建 |
| </span> |
| </a> |
| <div class="navbar-dropdown"> |
| |
| <a href="/zh/docs/build/compile-by-docker" |
| class="navbar-item "> |
| 使用 Docker 完成编译（推荐） |
| </a> |
| |
| <a href="/zh/docs/build/compile-from-source" |
| class="navbar-item "> |
| 从源码编译 |
| </a> |
| |
| </div> |
| </div> |
| |
| <!--dropdown--> |
| <div class="navbar-item has-dropdown is-hoverable"> |
| <a href="" |
| class="navbar-link "> |
| <span> |
| 客户端库 |
| </span> |
| </a> |
| <div class="navbar-dropdown"> |
| |
| <a href="/zh/clients/java-client" |
| class="navbar-item "> |
| Java 客户端 |
| </a> |
| |
| <a href="/zh/clients/cpp-client" |
| class="navbar-item "> |
| C++ 客户端 |
| </a> |
| |
| <a href="https://github.com/apache/incubator-pegasus/tree/master/go-client" |
| class="navbar-item "> |
| Golang 客户端 |
| </a> |
| |
| <a href="/zh/clients/python-client" |
| class="navbar-item "> |
| Python 客户端 |
| </a> |
| |
| <a href="/zh/clients/node-client" |
| class="navbar-item "> |
| NodeJS 客户端 |
| </a> |
| |
| <a href="/zh/clients/scala-client" |
| class="navbar-item "> |
| Scala 客户端 |
| </a> |
| |
| </div> |
| </div> |
| |
| <!--dropdown--> |
| <div class="navbar-item has-dropdown is-hoverable"> |
| <a href="" |
| class="navbar-link "> |
| <span> |
| 生态工具 |
| </span> |
| </a> |
| <div class="navbar-dropdown"> |
| |
| <a href="/zh/docs/tools/shell" |
| class="navbar-item "> |
| Pegasus Shell 工具 |
| </a> |
| |
| <a href="https://github.com/pegasus-kv/admin-cli" |
| class="navbar-item "> |
| 集群管理命令行 |
| </a> |
| |
| <a href="https://github.com/pegasus-kv/pegic" |
| class="navbar-item "> |
| 数据访问命令行 |
| </a> |
| |
| </div> |
| </div> |
| |
| <!--dropdown--> |
| <div class="navbar-item has-dropdown is-hoverable"> |
| <a href="" |
| class="navbar-link "> |
| <span> |
| 用户接口 |
| </span> |
| </a> |
| <div class="navbar-dropdown"> |
| |
| <a href="/zh/api/ttl" |
| class="navbar-item "> |
| TTL |
| </a> |
| |
| <a href="/zh/api/single-atomic" |
| class="navbar-item "> |
| 单行原子操作 |
| </a> |
| |
| <a href="/zh/api/redis" |
| class="navbar-item "> |
| Redis 适配 |
| </a> |
| |
| <a href="/zh/api/geo" |
| class="navbar-item "> |
| GEO 支持 |
| </a> |
| |
| <a href="/zh/api/http" |
| class="navbar-item "> |
| HTTP 接口 |
| </a> |
| |
| </div> |
| </div> |
| |
| <!--dropdown--> |
| <div class="navbar-item has-dropdown is-hoverable"> |
| <a href="" |
| class="navbar-link "> |
| <span> |
| 高效运维 |
| </span> |
| </a> |
| <div class="navbar-dropdown"> |
| |
| <a href="/zh/administration/deployment" |
| class="navbar-item "> |
| 集群部署 |
| </a> |
| |
| <a href="/zh/administration/config" |
| class="navbar-item "> |
| 配置说明 |
| </a> |
| |
| <a href="/zh/administration/rebalance" |
| class="navbar-item "> |
| 负载均衡 |
| </a> |
| |
| <a href="/zh/administration/monitoring" |
| class="navbar-item "> |
| 可视化监控 |
| </a> |
| |
| <a href="/zh/administration/rolling-update" |
| class="navbar-item "> |
| 集群重启和升级 |
| </a> |
| |
| <a href="/zh/administration/scale-in-out" |
| class="navbar-item "> |
| 集群扩容缩容 |
| </a> |
| |
| <a href="/zh/administration/resource-management" |
| class="navbar-item "> |
| 资源管理 |
| </a> |
| |
| <a href="/zh/administration/cold-backup" |
| class="navbar-item "> |
| 冷备份 |
| </a> |
| |
| <a href="/zh/administration/meta-recovery" |
| class="navbar-item "> |
| 元数据恢复 |
| </a> |
| |
| <a href="/zh/administration/replica-recovery" |
| class="navbar-item "> |
| Replica 数据恢复 |
| </a> |
| |
| <a href="/zh/administration/zk-migration" |
| class="navbar-item "> |
| Zookeeper 迁移 |
| </a> |
| |
| <a href="/zh/administration/table-migration" |
| class="navbar-item "> |
| Table 迁移 |
| </a> |
| |
| <a href="/zh/administration/table-soft-delete" |
| class="navbar-item is-active"> |
| Table 软删除 |
| </a> |
| |
| <a href="/zh/administration/table-env" |
| class="navbar-item "> |
| Table 环境变量 |
| </a> |
| |
| <a href="/zh/administration/remote-commands" |
| class="navbar-item "> |
| 远程命令 |
| </a> |
| |
| <a href="/zh/administration/partition-split" |
| class="navbar-item "> |
| Partition-Split |
| </a> |
| |
| <a href="/zh/administration/duplication" |
| class="navbar-item "> |
| 跨机房同步 |
| </a> |
| |
| <a href="/zh/administration/compression" |
| class="navbar-item "> |
| 数据压缩 |
| </a> |
| |
| <a href="/zh/administration/throttling" |
| class="navbar-item "> |
| 流量控制 |
| </a> |
| |
| <a href="/zh/administration/experiences" |
| class="navbar-item "> |
| 运维经验 |
| </a> |
| |
| <a href="/zh/administration/manual-compact" |
| class="navbar-item "> |
| Manual Compact 功能 |
| </a> |
| |
| <a href="/zh/administration/usage-scenario" |
| class="navbar-item "> |
| Usage Scenario 功能 |
| </a> |
| |
| <a href="/zh/administration/bad-disk" |
| class="navbar-item "> |
| 坏盘检修 |
| </a> |
| |
| <a href="/zh/administration/whitelist" |
| class="navbar-item "> |
| Replica Server 白名单 |
| </a> |
| |
| <a href="/zh/administration/backup-request" |
| class="navbar-item "> |
| Backup Request |
| </a> |
| |
| <a href="/zh/administration/hotspot-detection" |
| class="navbar-item "> |
| 热点检测 |
| </a> |
| |
| </div> |
| </div> |
| |
| </div> |
| </div> |
| </nav> |
| |
| <nav class="navbar is-hidden-mobile"> |
| <div class="navbar-start w-full"> |
| <div class="navbar-item pl-0 w-full"> |
| <!--TODO(wutao): Given the limitation of docsearch that couldn't handle multiple input, |
| I make searchbox only shown in desktop. Fix this issue when docsearch.js v3 released. |
| Related issue: https://github.com/algolia/docsearch/issues/230--> |
| <div id="docsearch"></div> |
| </div> |
| </div> |
| <div class="navbar-end"> |
| <div class="navbar-item"> |
| |
| |
| <!--A simple language switch button that only supports zh and en.--> |
| <!--IF its language is zh, then switches to en.--> |
| |
| <!--If you don't want a url to be relativized, you can add a space explicitly into the href to |
| prevents a url from being relativized by polyglot.--> |
| <a class="button is-light is-outlined is-inverted" href=" /administration/table-soft-delete"><strong>En</strong></a> |
| |
| </div> |
| </div> |
| </nav> |
| |
| <section class="hero is-info lg:mr-3"> |
| <div class="hero-body"> |
| |
| <p class="title is-size-2 is-centered">Table 软删除</p> |
| </div> |
| </section> |
| <section class="section" style="padding-top: 2rem;"> |
| <div class="content"> |
| <h1 id="功能目标">功能目标</h1> |
| <p>软删除主要用于防范数据被错误的永久删除。具体而言，软删除应该具有如下几个功能：</p> |
| <ul> |
| <li>当用户删除某个指定的表后，该表变得不可访问。</li> |
| <li>被删除的表其数据不会立即物理清除，而是会保留一段时间。</li> |
| <li>被删除的表过期（超过保留时间）后，数据会从物理上做彻底的删除，但是数据真正删除的时间还取决于其他因素，参见<a href="resource-management#垃圾文件夹管理">垃圾文件夹管理</a>。</li> |
| <li>未过期的表可以通过一定的方式恢复，恢复成功后又可以像正常的表一样进行读写访问。</li> |
| </ul> |
| |
| <h1 id="操作命令">操作命令</h1> |
| <p>shell端提供了<code class="language-plaintext highlighter-rouge">drop</code>和<code class="language-plaintext highlighter-rouge">recall</code>命令支持软删除。</p> |
| |
| <h2 id="使用drop命令删除表">使用drop命令删除表</h2> |
| <p>使用方式：</p> |
| <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>drop <app_name> [-r|--reserve_seconds num] |
| </code></pre></div></div> |
| |
| <p>drop命令用于删除一个表，通过<code class="language-plaintext highlighter-rouge">-r</code>选项指定数据的保留时间，从删除时间开始计算，单位为秒。如果不指定，则使用配置文件<code class="language-plaintext highlighter-rouge">hold_seconds_for_dropped_app</code>指定的值，默认为7天。</p> |
| |
| <p>表删除成功后：</p> |
| <ul> |
| <li>再次访问该表，将会返回<code class="language-plaintext highlighter-rouge">ERR_OBJECT_NOT_FOUND</code>，提示表不存在。</li> |
| <li>通过shell的<code class="language-plaintext highlighter-rouge">ls</code>命令看不到被删除的表。</li> |
| <li>通过shell的<code class="language-plaintext highlighter-rouge">ls -a</code>命令可以看到被删除的表。</li> |
| <li>被删除的表的ID不会被重用，以防止恢复时出现ID冲突。</li> |
| <li>表删除后，可以新建同名表，当客户端访问这个表名时，访问的是新表。</li> |
| </ul> |
| |
| <h2 id="过期表数据的物理删除">过期表数据的物理删除</h2> |
| <p>过期表的数据在各个replica server上也未必能立即被物理删除，因为：</p> |
| <ul> |
| <li>表的保留时间过期后，只有通过shell设置了<code class="language-plaintext highlighter-rouge">set_meta_level lively</code>，使meta server进入<a href="rebalance#控制集群的负载均衡">负载均衡模式</a>，meta server才会通过<code class="language-plaintext highlighter-rouge">config_sync</code>RPC通知replica server删除相关的replica。而replica server在收到meta server的通知后，就会将需删除的replica文件夹通过添加<code class="language-plaintext highlighter-rouge">.gar</code>后缀进行重命名，表示这是可以被删除的垃圾数据。但此时数据仍未被真正物理删除。</li> |
| <li>replica server会定期（配置文件<code class="language-plaintext highlighter-rouge">disk_stat_interval_seconds</code>）扫描各个数据文件夹（配置文件<code class="language-plaintext highlighter-rouge">data_dirs</code>），统计文件夹的使用情况。对于标记为<code class="language-plaintext highlighter-rouge">.gar</code>后缀的replica文件夹，获取其最后修改时间，并和当前时间进行比较，只有当两者时间差超过了阈值（配置文件<code class="language-plaintext highlighter-rouge">gc_disk_garbage_replica_interval_seconds</code>）后，在会将文件夹删除掉。此时数据才算被真正物理删除。</li> |
| </ul> |
| |
| <p>所以综上所述，能够影响表被删除后进行物理删除的时间点的配置项包括：</p> |
| <ul> |
| <li><code class="language-plaintext highlighter-rouge">[meta_server] hold_seconds_for_dropped_app</code>：当drop表没有指定<code class="language-plaintext highlighter-rouge">-r</code>选项时，决定该表的保留时间。</li> |
| <li><code class="language-plaintext highlighter-rouge">[replication] disk_stat_interval_seconds</code>：replica server定期扫描各个数据文件夹的时间间隔。</li> |
| <li><code class="language-plaintext highlighter-rouge">[replication] gc_disk_garbage_replica_interval_seconds</code>：垃圾replica文件夹的最后修改时间距离当前时间超过这个阈值，文件夹才会被删除。</li> |
| </ul> |
| |
| <p>如果遇到需要紧急删除数据以释放磁盘空间，但是又不方便重启replica server更新配置的情况，可以根据表ID进行手工暴力删除，但是千万注意：</p> |
| <ul> |
| <li>不到万不得已，不要进行手工暴力删表，避免误操作。</li> |
| <li>坚决只能删除<strong>过期表</strong>的数据。</li> |
| <li>不要误删其他表的数据。</li> |
| </ul> |
| |
| <h2 id="使用recall命令恢复表">使用recall命令恢复表</h2> |
| <p>使用方式</p> |
| <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>recall <app_id> [new_app_name] |
| </code></pre></div></div> |
| |
| <p>只要表的保留时间还没有过期，就能执行恢复：</p> |
| <ul> |
| <li>恢复时需指定表ID。</li> |
| <li>可以指定新表名，如果不指定新表名，则会使用原表名。</li> |
| <li>如果原表名已存在（删表后新建了同名表），则必须指定另外一个不同的新表名，否则会失败。</li> |
| <li>恢复过程可能需要花费一段时间。</li> |
| </ul> |
| |
| <h2 id="示例">示例</h2> |
| <p>以下是使用示例：删除mytable表，然后恢复成新表名mytable2</p> |
| <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>>>> ls |
| app_id status app_name app_type partition_count replica_count is_stateful drop_expire_time envs_count |
| 1 AVAILABLE temp pegasus 8 3 true - 0 |
| 2 AVAILABLE mytable pegasus 8 3 true - 0 |
| |
| list apps succeed |
| |
| >>> drop mytable |
| reserve_seconds = 0 |
| drop app mytable succeed |
| |
| >>> ls |
| app_id status app_name app_type partition_count replica_count is_stateful drop_expire_time envs_count |
| 1 AVAILABLE temp pegasus 8 3 true - 0 |
| |
| list apps succeed |
| |
| >>> ls -a |
| app_id status app_name app_type partition_count replica_count is_stateful drop_expire_time envs_count |
| 1 AVAILABLE temp pegasus 8 3 true - 0 |
| 2 DROPPED mytable pegasus 8 3 true 2018-07-28 19:07:21 0 |
| |
| list apps succeed |
| |
| >>> recall 2 mytable2 |
| recall app ok, id(2), name(mytable2), partition_count(8), wait it ready |
| mytable2 not ready yet, still waiting... (0/8) |
| mytable2 not ready yet, still waiting... (0/8) |
| mytable2 not ready yet, still waiting... (0/8) |
| mytable2 not ready yet, still waiting... (0/8) |
| mytable2 not ready yet, still waiting... (0/8) |
| mytable2 not ready yet, still waiting... (0/8) |
| mytable2 not ready yet, still waiting... (0/8) |
| mytable2 not ready yet, still waiting... (0/8) |
| mytable2 not ready yet, still waiting... (7/8) |
| mytable2 is ready now: (8/8) |
| recall app 2 succeed |
| |
| >>> ls |
| app_id status app_name app_type partition_count replica_count is_stateful drop_expire_time envs_count |
| 1 AVAILABLE temp pegasus 8 3 true - 0 |
| 2 AVAILABLE mytable2 pegasus 8 3 true - 0 |
| |
| list apps succeed |
| </code></pre></div></div> |
| |
| <h1 id="设计与实现">设计与实现</h1> |
| <p>关键点：</p> |
| <ul> |
| <li>表的生命周期定义要清晰：对于正在删除/召回的表，其他的create/recall/drop的操作要禁止。</li> |
| <li>表的过期时间要在各个meta server之间达成一致，这需要各个meta server做时钟同步。</li> |
| <li>如果表格被多次的删除和召回，当这些消息以乱序的方式送达到replica server时，一定要保证replica server能处理这些情况。最好能把drop操作映射成replica configuration元数据的变更。</li> |
| </ul> |
| |
| <p>实现要点简述：</p> |
| <ul> |
| <li>meta server对删除动作的响应：当收到客户端响应时，meta server需要把信息更新到zookeeper: (1) app的状态改为dropped, 记录过期时间 (2) 升级configuration的状态，并记录到zookeeper。</li> |
| <li>replica server的replica生命周期变化：replica server通过定期和meta server交换心跳获取自己所服务的replica，如果发现本地的replica已经在远端不存在，则把本地的replica清除掉。</li> |
| <li>过期数据的删除：replica server和meta交换心跳时，汇报自己所存储的replica。当meta判定一个replica已经无效时，命令replica将其删掉。</li> |
| </ul> |
| |
| </div> |
| </section> |
| <footer class="footer"> |
| <div class="container"> |
| <div class="content is-small has-text-centered"> |
| <div style="margin-bottom: 20px;"> |
| <a href="http://incubator.apache.org"> |
| <img src="/assets/images/egg-logo.png" |
| width="15%" |
| alt="Apache Incubator"/> |
| </a> |
| </div> |
| Copyright © 2023 <a href="http://www.apache.org">The Apache Software Foundation</a>. |
| Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version |
| 2.0</a>. |
| <br><br> |
| |
| Apache Pegasus is an effort undergoing incubation at The Apache Software Foundation (ASF), |
| sponsored by the Apache Incubator. Incubation is required of all newly accepted projects |
| until a further review indicates that the infrastructure, communications, and decision making process |
| have stabilized in a manner consistent with other successful ASF projects. While incubation status is |
| not necessarily a reflection of the completeness or stability of the code, it does indicate that the |
| project has yet to be fully endorsed by the ASF. |
| |
| <br><br> |
| Apache Pegasus, Pegasus, Apache, the Apache feather logo, and the Apache Pegasus project logo are either |
| registered trademarks or trademarks of The Apache Software Foundation in the United States and other |
| countries. |
| </div> |
| </div> |
| </footer> |
| </div> |
| |
| <!-- right panel --> |
| <div class="dashboard-panel is-small is-scrollable is-hidden-mobile"> |
| <p class="menu-label"> |
| <span class="icon"> |
| <i class="fa fa-bars" aria-hidden="true"></i> |
| </span> |
| 本页导航 |
| </p> |
| <ul class="menu-list"> |
| <li><a href="#功能目标">功能目标</a></li> |
| <li><a href="#操作命令">操作命令</a> |
| <ul> |
| <li><a href="#使用drop命令删除表">使用drop命令删除表</a></li> |
| <li><a href="#过期表数据的物理删除">过期表数据的物理删除</a></li> |
| <li><a href="#使用recall命令恢复表">使用recall命令恢复表</a></li> |
| <li><a href="#示例">示例</a></li> |
| </ul> |
| </li> |
| <li><a href="#设计与实现">设计与实现</a></li> |
| </ul> |
| |
| </div> |
| </div> |
| |
| <script src="/assets/js/app.js" type="text/javascript"></script> |
| <script> |
| docsearch({ |
| container: '#docsearch', |
| appId: 'QRN30RBW0S', |
| indexName: 'pegasus-apache', |
| apiKey: 'd3a3252fa344359766707a106c4ed88f', |
| debug: true |
| }); |
| </script> |
| |
| </body> |
| |
| </html> |
