| <?xml version="1.0" encoding="UTF-8"?> |
| <!DOCTYPE html |
| PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> |
| <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
| <head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> |
| |
| <meta name="copyright" content="(C) Copyright 2025" /> |
| <meta name="DC.rights.owner" content="(C) Copyright 2025" /> |
| <meta name="DC.Type" content="concept" /> |
| <meta name="DC.Title" content="Metadata Management" /> |
| <meta name="DC.Relation" scheme="URI" content="../topics/impala_scalability.html" /> |
| <meta name="prodname" content="Impala" /> |
| <meta name="version" content="Impala 3.4.x" /> |
| <meta name="DC.Format" content="XHTML" /> |
| <meta name="DC.Identifier" content="impala_metadata" /> |
| <link rel="stylesheet" type="text/css" href="../commonltr.css" /> |
| <title>Metadata Management</title> |
| </head> |
| <body id="impala_metadata"> |
| |
| |
| <h1 class="title topictitle1" id="ariaid-title1">Metadata Management</h1> |
| |
| |
| |
| |
| <div class="body conbody"> |
| |
| <p class="p"> |
| This topic describes various knobs you can use to control how Impala manages its metadata |
| in order to improve performance and scalability. |
| </p> |
| |
| |
| <p class="p toc inpage"></p> |
| |
| |
| </div> |
| |
| |
| <div class="related-links"> |
| <div class="familylinks"> |
| <div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_scalability.html">Scalability Considerations for Impala</a></div> |
| </div> |
| </div><div class="topic concept nested1" aria-labelledby="ariaid-title2" id="on_demand_metadata"> |
| |
| <h2 class="title topictitle2" id="ariaid-title2">On-demand Metadata</h2> |
| |
| |
| <div class="body conbody"> |
| |
| <p class="p"> |
| In previous versions of Impala, every coordinator kept a replica of all the cache in |
| <code class="ph codeph">catalogd</code>, consuming large memory on each coordinator with no option to |
| evict. Metadata always propagated through the <code class="ph codeph">statestored</code> and suffers |
| from head-of-line blocking, for example, one user loading a big table blocking another |
| user loading a small table. |
| </p> |
| |
| |
| <p class="p"> |
| With this new feature, the coordinators pull metadata as needed from |
| <code class="ph codeph">catalogd</code> and cache it locally. The cached metadata gets evicted |
| automatically under memory pressure. |
| </p> |
| |
| |
| <p class="p"> |
| The granularity of on-demand metadata fetches is now at the partition level between the |
| coordinator and <code class="ph codeph">catalogd</code>. Common use cases like add/drop partitions do |
| not trigger unnecessary serialization/deserialization of large metadata. |
| </p> |
| |
| |
| <p class="p"> |
| This feature is disabled by default. |
| </p> |
| |
| |
| <div class="p"> |
| The feature can be used in either of the following modes. |
| <dl class="dl"> |
| |
| |
| <dt class="dt dlterm"> |
| Metadata on-demand mode |
| </dt> |
| |
| |
| <dd class="dd"> |
| In this mode, all coordinators use the metadata on-demand. |
| </dd> |
| |
| |
| <dd class="dd ddexpand"> |
| Set the following on <code class="ph codeph">catalogd</code>: |
| <pre class="pre codeblock"><code>--catalog_topic_mode=minimal</code></pre> |
| </dd> |
| |
| |
| <dd class="dd ddexpand"> |
| Set the following on all <code class="ph codeph">impalad</code> coordinators: |
| <pre class="pre codeblock"><code>--use_local_catalog=true</code></pre> |
| </dd> |
| |
| |
| |
| |
| |
| |
| <dt class="dt dlterm"> |
| Mixed mode |
| </dt> |
| |
| |
| <dd class="dd"> |
| In this mode, only some coordinators are enabled to use the metadata on-demand. |
| </dd> |
| |
| |
| <dd class="dd ddexpand"> |
| We recommend that you use the mixed mode only for testing local catalog’s impact |
| on heap usage. |
| </dd> |
| |
| |
| <dd class="dd ddexpand"> |
| Set the following on <code class="ph codeph">catalogd</code>: |
| <pre class="pre codeblock"><code>--catalog_topic_mode=mixed</code></pre> |
| </dd> |
| |
| |
| <dd class="dd ddexpand"> |
| Set the following on <code class="ph codeph">impalad</code> coordinators with metdadata |
| on-demand: |
| <pre class="pre codeblock"><code>--use_local_catalog=true </code></pre> |
| </dd> |
| |
| |
| |
| |
| <dt class="dt dlterm">Flags related to <code class="ph codeph">use_local_catalog</code></dt> |
| |
| <dd class="dd">When <code class="ph codeph">use_local_catalog</code> is enabled or set to <code class="ph codeph">True</code> on the impalad |
| coordinators the following list of flags configure various parameters as described below. It is not |
| recommended to change the default values on these flags. |
| </dd> |
| |
| <dd class="dd ddexpand"> |
| <ul class="ul"> |
| <li class="li">The flag <code class="ph codeph">local_catalog_cache_mb</code> (defaults to -1) configures the |
| size of the catalog cache within each coordinator. With the default set to -1, the |
| cache is auto-configured to 60% of the configured Java heap size. Note that the |
| Java heap size is distinct from and typically smaller than the overall Impala |
| memory limit.</li> |
| |
| <li class="li">The flag <code class="ph codeph">local_catalog_cache_expiration_s</code> (defaults to 3600) configures the |
| expiration time of the catalog cache within each impalad. Even if the configured |
| cache capacity has not been reached, items are removed from the cache if they have not |
| been accessed in the defined amount of time.</li> |
| |
| <li class="li">The flag <code class="ph codeph">local_catalog_max_fetch_retries</code> (defaults to 40) configures |
| the maximum number of retries needed for queries to fetch a metadata object from the impalad |
| coordinator's local catalog cache.</li> |
| |
| </ul> |
| |
| </dd> |
| |
| |
| </dl> |
| |
| </div> |
| |
| |
| </div> |
| |
| |
| </div> |
| |
| |
| <div class="topic concept nested1" aria-labelledby="ariaid-title3" id="auto_invalidate_metadata"> |
| |
| <h2 class="title topictitle2" id="ariaid-title3">Automatic Invalidation of Metadata Cache</h2> |
| |
| |
| <div class="body conbody"> |
| |
| <p class="p"> |
| To keep the size of metadata bounded, <code class="ph codeph">catalogd</code> periodically scans all |
| the tables and invalidates those not recently used. There are two types of |
| configurations for <code class="ph codeph">catalogd</code> and <code class="ph codeph">impalad</code>. |
| </p> |
| |
| |
| <dl class="dl"> |
| |
| |
| <dt class="dt dlterm"> |
| Time-based cache invalidation |
| </dt> |
| |
| |
| <dd class="dd"> |
| <code class="ph codeph">Catalogd</code> invalidates tables that are not recently used in the |
| specified time period (in seconds). |
| </dd> |
| |
| |
| <dd class="dd ddexpand"> |
| The <code class="ph codeph">‑‑invalidate_tables_timeout_s</code> flag needs to be |
| applied to both <code class="ph codeph">impalad</code> and <code class="ph codeph">catalogd</code>. |
| </dd> |
| |
| |
| |
| |
| |
| |
| <dt class="dt dlterm"> |
| Memory-based cache invalidation |
| </dt> |
| |
| |
| <dd class="dd"> |
| When the memory pressure reaches 60% of JVM heap size after a Java garbage |
| collection in <code class="ph codeph">catalogd</code>, Impala invalidates 10% of the least |
| recently used tables. |
| </dd> |
| |
| |
| <dd class="dd ddexpand"> |
| The <code class="ph codeph">‑‑invalidate_tables_on_memory_pressure</code> flag needs |
| to be applied to both <code class="ph codeph">impalad</code> and <code class="ph codeph">catalogd</code>. |
| </dd> |
| |
| |
| |
| </dl> |
| |
| |
| <p class="p"> |
| Automatic invalidation of metadata provides more stability with lower chances of running |
| out of memory, but the feature could potentially cause performance issues and may |
| require tuning. |
| </p> |
| |
| |
| </div> |
| |
| |
| </div> |
| |
| |
| <div class="topic concept nested1" aria-labelledby="ariaid-title4" id="auto_poll_hms_notification"> |
| |
| <h2 class="title topictitle2" id="ariaid-title4">Automatic Invalidation/Refresh of Metadata</h2> |
| |
| |
| <div class="body conbody"> |
| |
| <p class="p"> |
| When tools such as Hive and Spark are used to process the raw data ingested into Hive |
| tables, new HMS metadata (database, tables, partitions) and filesystem metadata (new |
| files in existing partitions/tables) is generated. In previous versions of Impala, in |
| order to pick up this new information, Impala users needed to manually issue an |
| <code class="ph codeph">INVALIDATE</code> or <code class="ph codeph">REFRESH</code> commands. |
| </p> |
| |
| |
| <p class="p"> |
| When automatic invalidate/refresh of metadata is enabled, <code class="ph codeph">catalogd</code> |
| polls Hive Metastore (HMS) notification events at a configurable interval and processes |
| the following changes: |
| </p> |
| |
| |
| <div class="note note"><span class="notetitle">Note:</span> |
| This is a preview feature in <span class="keyword">Impala 3.3</span> and <span class="keyword">Impala 4.0</span> |
| It is generally available and enabled by default from <span class="keyword">Impala 4.1</span> onwards. |
| </div> |
| |
| |
| <ul class="ul"> |
| <li class="li"> |
| Invalidates the tables when it receives the <code class="ph codeph">ALTER TABLE</code> event. |
| </li> |
| |
| |
| <li class="li"> |
| Refreshes the partition when it receives the <code class="ph codeph">ALTER</code>, |
| <code class="ph codeph">ADD</code>, or <code class="ph codeph">DROP</code> partitions. |
| </li> |
| |
| |
| <li class="li"> |
| Adds the tables or databases when it receives the <code class="ph codeph">CREATE TABLE</code> or |
| <code class="ph codeph">CREATE DATABASE</code> events. |
| </li> |
| |
| |
| <li class="li"> |
| Removes the tables from <code class="ph codeph">catalogd</code> when it receives the <code class="ph codeph">DROP |
| TABLE</code> or <code class="ph codeph">DROP DATABASE</code> events. |
| </li> |
| |
| |
| <li class="li"> |
| Refreshes the table and partitions when it receives the <code class="ph codeph">INSERT</code> |
| events. |
| <p class="p"> |
| If the table is not loaded at the time of processing the <code class="ph codeph">INSERT</code> |
| event, the event processor does not need to refresh the table and skips it. |
| </p> |
| |
| </li> |
| |
| |
| <li class="li"> |
| Changes the database and updates <code class="ph codeph">catalogd</code> when it receives the |
| <code class="ph codeph">ALTER DATABASE</code> events. The following changes are supported. This |
| event does not invalidate the tables in the database. |
| <ul class="ul"> |
| <li class="li"> |
| Change the database properties |
| </li> |
| |
| |
| <li class="li"> |
| Change the comment on the database |
| </li> |
| |
| |
| <li class="li"> |
| Change the owner of the database |
| </li> |
| |
| |
| <li class="li"> |
| Change the default location of the database |
| <p class="p"> |
| Changing the default location of the database does not move the tables of that |
| database to the new location. Only the new tables which are created subsequently |
| use the default location of the database in case it is not provided in the |
| create table statement. |
| </p> |
| |
| </li> |
| |
| </ul> |
| |
| </li> |
| |
| </ul> |
| |
| |
| <p class="p"> |
| This feature is controlled by the |
| <code class="ph codeph">‑‑hms_event_polling_interval_s</code> flag. Start the |
| <code class="ph codeph">catalogd</code> with the <code class="ph codeph">‑‑hms_event_polling_interval_s</code> |
| flag set to a positive integer to enable the feature and set the polling frequency in |
| seconds. We recommend the value to be less than 5 seconds. |
| </p> |
| |
| |
| <p class="p"> |
| The following use cases are not supported: |
| </p> |
| |
| |
| <ul class="ul"> |
| <li class="li"> |
| When you bypass HMS and add or remove data into table by adding files directly on the |
| filesystem, HMS does not generate the <code class="ph codeph">INSERT</code> event, and the event |
| processor will not invalidate the corresponding table or refresh the corresponding |
| partition. |
| <p class="p"> |
| It is recommended that you use the <code class="ph codeph">LOAD DATA</code> command to do the data |
| load in such cases, so that event processor can act on the events generated by the |
| <code class="ph codeph">LOAD</code> command. |
| </p> |
| |
| </li> |
| |
| |
| <li class="li"> |
| The Spark API that saves data to a specified location does not generate events in HMS, |
| thus is not supported. For example: |
| <pre class="pre codeblock"><code>Seq((1, 2)).toDF("i", "j").write.save("/user/hive/warehouse/spark_etl.db/customers/date=01012019")</code></pre> |
| </li> |
| |
| </ul> |
| |
| |
| <p class="p"> |
| This feature is turned off by default with the |
| <code class="ph codeph">‑‑hms_event_polling_interval_s</code> flag set to |
| <code class="ph codeph">0</code>. |
| </p> |
| |
| |
| </div> |
| |
| |
| <div class="topic concept nested2" aria-labelledby="ariaid-title5" id="configure_event_based_metadata_sync"> |
| |
| <h3 class="title topictitle3" id="ariaid-title5">Configure HMS for Event Based Automatic Metadata Sync</h3> |
| |
| |
| <div class="body conbody"> |
| |
| <p class="p"> |
| To use the HMS event based metadata sync: |
| </p> |
| |
| |
| <ol class="ol"> |
| <li class="li"> |
| Add the following entries to the <code class="ph codeph">hive-site.xml</code> of the Hive |
| Metastore service. |
| <pre class="pre codeblock"><code> <property> |
| <name>hive.metastore.transactional.event.listeners</name> |
| <value>org.apache.hive.hcatalog.listener.DbNotificationListener</value> |
| </property> |
| <property> |
| <name>hive.metastore.dml.events</name> |
| <value>true</true> |
| </property></code></pre> |
| </li> |
| |
| |
| <li class="li"> |
| Save <code class="ph codeph">hive-site.xml</code>. |
| </li> |
| |
| |
| <li class="li"> |
| Set the <code class="ph codeph">hive.metastore.dml.events</code> configuration key to |
| <code class="ph codeph">true</code> in HiveServer2 service's <code class="ph codeph">hive-site.xml</code>. This |
| configuration key needs to be set to <code class="ph codeph">true</code> in both Hive services, |
| HiveServer2 and Hive Metastore. |
| </li> |
| |
| |
| <li class="li"> |
| If applicable, set the <code class="ph codeph">hive.metastore.dml.events</code> configuration key |
| to <code class="ph codeph">true</code> in <code class="ph codeph">hive-site.xml</code> used by the Spark |
| applications (typically, <code class="ph codeph">/etc/hive/conf/hive-site.xml</code>) so that the |
| <code class="ph codeph">INSERT</code> events are generated when the Spark application inserts data |
| into existing tables and partitions. |
| </li> |
| |
| |
| <li class="li"> |
| Restart the HiveServer2, Hive Metastore, and Spark (if applicable) services. |
| </li> |
| |
| </ol> |
| |
| |
| </div> |
| |
| |
| </div> |
| |
| |
| <div class="topic concept nested2" aria-labelledby="ariaid-title6" id="disable_event_based_metadata_sync"> |
| |
| <h3 class="title topictitle3" id="ariaid-title6">Disable Event Based Automatic Metadata Sync</h3> |
| |
| |
| <div class="body conbody"> |
| |
| <p class="p"> |
| When the <code class="ph codeph">‑‑hms_event_polling_interval_s</code> flag is set to a non-zero |
| value for your <code class="ph codeph">catalogd</code>, the event-based automatic invalidation is |
| enabled for all databases and tables. If you wish to have the fine-grained control on |
| which tables or databases need to be synced using events, you can use the |
| <code class="ph codeph">impala.disableHmsSync</code> property to disable the event processing at the |
| table or database level. |
| </p> |
| |
| |
| <p class="p"> |
| When you add the <code class="ph codeph">DBPROPERTIES</code> or <code class="ph codeph">TBLPROPERTIES</code> with |
| the <code class="ph codeph">impala.disableHmsSync</code> key, the HMS event based sync is turned on |
| or off. The value of the <code class="ph codeph">impala.disableHmsSync</code> property determines if |
| the event processing needs to be disabled for a particular table or database. |
| </p> |
| |
| |
| <ul class="ul"> |
| <li class="li"> |
| If <code class="ph codeph">'impala.disableHmsSync'='true'</code>, the events for that table or |
| database are ignored and not synced with HMS. |
| </li> |
| |
| |
| <li class="li"> |
| If <code class="ph codeph">'impala.disableHmsSync'='false'</code> or if |
| <code class="ph codeph">impala.disableHmsSync</code> is not set, the automatic sync with HMS is |
| enabled if the <code class="ph codeph">‑‑hms_event_polling_interval_s</code> global flag is |
| set to non-zero. |
| </li> |
| |
| </ul> |
| |
| |
| <ul class="ul"> |
| <li class="li"> |
| To disable the event based HMS sync for a new database, set the |
| <code class="ph codeph">impala.disableHmsSync</code> database properties in Hive as currently, |
| Impala does not support setting database properties: |
| <pre class="pre codeblock"><code>CREATE DATABASE <name> WITH DBPROPERTIES ('impala.disableHmsSync'='true');</code></pre> |
| </li> |
| |
| |
| <li class="li"> |
| To enable or disable the event based HMS sync for a table: |
| <pre class="pre codeblock"><code>CREATE TABLE <name> WITH TBLPROPERTIES ('impala.disableHmsSync'='true' | 'false');</code></pre> |
| </li> |
| |
| |
| <li class="li"> |
| To change the event based HMS sync at the table level: |
| <pre class="pre codeblock"><code>ALTER TABLE <name> WITH TBLPROPERTIES ('impala.disableHmsSync'='true' | 'false');</code></pre> |
| </li> |
| |
| </ul> |
| |
| |
| <p class="p"> |
| When both table and database level properties are set, the table level property takes |
| precedence. If the table level property is not set, then the database level property |
| is used to evaluate if the event needs to be processed or not. |
| </p> |
| |
| |
| <p class="p"> |
| If the property is changed from <code class="ph codeph">true</code> (meaning events are skipped) to |
| <code class="ph codeph">false</code> (meaning events are not skipped), you need to issue a manual |
| <code class="ph codeph">INVALIDATE METADATA</code> command to reset event processor because it |
| doesn't know how many events have been skipped in the past and cannot know if the |
| object in the event is the latest. In such a case, the status of the event processor |
| changes to <code class="ph codeph">NEEDS_INVALIDATE</code>. |
| </p> |
| |
| |
| </div> |
| |
| |
| </div> |
| |
| |
| <div class="topic concept nested2" aria-labelledby="ariaid-title7" id="event_processor_metrics"> |
| |
| <h3 class="title topictitle3" id="ariaid-title7">Metrics for Event Based Automatic Metadata Sync</h3> |
| |
| |
| <div class="body conbody"> |
| |
| <p class="p"> |
| You can use the web UI of the <code class="ph codeph">catalogd</code> to check the state of the |
| automatic invalidate event processor. |
| </p> |
| |
| |
| <div class="p"> |
| Under the web UI, there are two pages that presents the metrics for HMS event |
| processor that is responsible for the event based automatic metadata sync. |
| <ul class="ul"> |
| <li class="li"> |
| <strong class="ph b">/metrics#events</strong> |
| </li> |
| |
| |
| <li class="li"> |
| <strong class="ph b">/events</strong> |
| <p class="p"> |
| This provides a detailed view of the metrics of the event processor, including |
| min, max, mean, median, of the durations and rate metrics for all the counters |
| listed on the <strong class="ph b">/metrics#events</strong> page. |
| </p> |
| |
| </li> |
| |
| </ul> |
| |
| </div> |
| |
| |
| </div> |
| |
| |
| <div class="topic concept nested3" aria-labelledby="ariaid-title8" id="concept_gch_xzm_1hb"> |
| |
| <h4 class="title topictitle4" id="ariaid-title8">/metrics#events Page</h4> |
| |
| |
| <div class="body conbody"> |
| |
| <p class="p"> |
| The <strong class="ph b">/metrics#events</strong> page provides the following metrics about the HMS event |
| processor. |
| </p> |
| |
| |
| |
| <div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="concept_gch_xzm_1hb__events-tbl" class="table" frame="border" border="1" rules="all"><colgroup><col style="width:27.932960893854748%" /><col style="width:72.06703910614524%" /></colgroup><thead class="thead" style="text-align:left;"> |
| <tr class="row"> |
| <th class="entry nocellnorowborder" style="vertical-align:top;" id="d135928e659"> |
| Name |
| </th> |
| |
| <th class="entry cell-norowborder" style="vertical-align:top;" id="d135928e662"> |
| Description |
| </th> |
| |
| </tr> |
| |
| </thead> |
| <tbody class="tbody"> |
| <tr class="row"> |
| <td class="entry nocellnorowborder" style="vertical-align:top;" headers="d135928e659 "> |
| events-processor.avg-events-fetch-duration |
| </td> |
| |
| <td class="entry cell-norowborder" style="vertical-align:top;" headers="d135928e662 "> |
| Average duration to fetch a batch of events and process it. |
| </td> |
| |
| </tr> |
| |
| <tr class="row"> |
| <td class="entry nocellnorowborder" style="vertical-align:top;" headers="d135928e659 "> |
| events-processor.avg-events-process-duration |
| </td> |
| |
| <td class="entry cell-norowborder" style="vertical-align:top;" headers="d135928e662 "> |
| Average time taken to process a batch of events received from the Metastore. |
| </td> |
| |
| </tr> |
| |
| <tr class="row"> |
| <td class="entry nocellnorowborder" style="vertical-align:top;" headers="d135928e659 "> |
| events-processor.events-received |
| </td> |
| |
| <td class="entry cell-norowborder" style="vertical-align:top;" headers="d135928e662 "> |
| Total number of the Metastore events received. |
| </td> |
| |
| </tr> |
| |
| <tr class="row"> |
| <td class="entry nocellnorowborder" style="vertical-align:top;" headers="d135928e659 "> |
| events-processor.events-received-15min-rate |
| </td> |
| |
| <td class="entry cell-norowborder" style="vertical-align:top;" headers="d135928e662 "> |
| Exponentially weighted moving average (EWMA) of number of events received in |
| last 15 min. |
| |
| <p class="p"> |
| This rate of events can be used to determine if there are spikes in event |
| processor activity during certain hours of the day. |
| </p> |
| |
| </td> |
| |
| </tr> |
| |
| <tr class="row"> |
| <td class="entry nocellnorowborder" style="vertical-align:top;" headers="d135928e659 "> |
| events-processor.events-received-1min-rate |
| </td> |
| |
| <td class="entry cell-norowborder" style="vertical-align:top;" headers="d135928e662 "> |
| Exponentially weighted moving average (EWMA) of number of events received in |
| last 1 min. |
| |
| <p class="p"> |
| This rate of events can be used to determine if there are spikes in event |
| processor activity during certain hours of the day. |
| </p> |
| |
| </td> |
| |
| </tr> |
| |
| <tr class="row"> |
| <td class="entry nocellnorowborder" style="vertical-align:top;" headers="d135928e659 "> |
| events-processor.events-received-5min-rate |
| </td> |
| |
| <td class="entry cell-norowborder" style="vertical-align:top;" headers="d135928e662 "> |
| Exponentially weighted moving average (EWMA) of number of events received in |
| last 5 min. |
| |
| <p class="p"> |
| This rate of events can be used to determine if there are spikes in event |
| processor activity during certain hours of the day. |
| </p> |
| |
| </td> |
| |
| </tr> |
| |
| <tr class="row"> |
| <td class="entry nocellnorowborder" style="vertical-align:top;" headers="d135928e659 "> |
| events-processor.events-skipped |
| </td> |
| |
| <td class="entry cell-norowborder" style="vertical-align:top;" headers="d135928e662 "> |
| Total number of the Metastore events skipped. |
| |
| <div class="p"> |
| Events can be skipped based on certain flags are table and database level. |
| You can use this metric to make decisions, such as: |
| <ul class="ul"> |
| <li class="li"> |
| If most of the events are being skipped, see if you might just turn |
| off the event processing. |
| </li> |
| |
| |
| <li class="li"> |
| If most of the events are not skipped, see if you need to add flags on |
| certain databases. |
| </li> |
| |
| </ul> |
| |
| </div> |
| |
| </td> |
| |
| </tr> |
| |
| <tr class="row"> |
| <td class="entry row-nocellborder" style="vertical-align:top;" headers="d135928e659 "> |
| events-processor.status |
| </td> |
| |
| <td class="entry cellrowborder" style="vertical-align:top;" headers="d135928e662 "> |
| Metastore event processor status to see if there are events being received |
| or not. Possible states are: |
| |
| <ul class="ul"> |
| <li class="li"> |
| <code class="ph codeph">PAUSED</code> |
| <p class="p"> |
| The event processor is paused because catalog is being reset |
| concurrently. |
| </p> |
| |
| </li> |
| |
| |
| <li class="li"> |
| <code class="ph codeph">ACTIVE</code> |
| <p class="p"> |
| The event processor is scheduled at a given frequency. |
| </p> |
| |
| </li> |
| |
| |
| <li class="li"> |
| <code class="ph codeph">ERROR</code> |
| </li> |
| |
| |
| <li class="li"> |
| The event processor is in error state and event processing has stopped. |
| </li> |
| |
| |
| <li class="li"> |
| <code class="ph codeph">NEEDS_INVALIDATE</code> |
| <p class="p"> |
| The event processor could not resolve certain events and needs a |
| manual <code class="ph codeph">INVALIDATE</code> command to reset the state. |
| </p> |
| |
| </li> |
| |
| |
| <li class="li"> |
| <code class="ph codeph">STOPPED</code> |
| <p class="p"> |
| The event processing has been shutdown. No events will be processed. |
| </p> |
| |
| </li> |
| |
| |
| <li class="li"> |
| <code class="ph codeph">DISABLED</code> |
| <p class="p"> |
| The event processor is not configured to run. |
| </p> |
| |
| </li> |
| |
| </ul> |
| |
| </td> |
| |
| </tr> |
| |
| </tbody> |
| </table> |
| </div> |
| |
| |
| </div> |
| |
| |
| </div> |
| |
| |
| </div> |
| |
| |
| </div> |
| |
| |
| </body> |
| </html> |
