HBASE-30348 Throttles should work for system tables (excluding quotas, meta, namespace)

RegionServerRpcQuotaManager#getQuota returned NoopOperationQuota for any table
where TableName#isSystemTable() is true, and QuotaCache#getUserLimiter returned
NoopQuotaLimiter under the same condition. That covers every table in the hbase
namespace and every table in the backup namespace, so no quota of any kind could
throttle them: the quota was accepted, stored, and displayed by list_quotas, but
the request path never consulted a limiter and nothing warned the operator.

Only hbase:meta, hbase:quota and hbase:namespace are exempt now. All three are
read while a master is still initializing, before quotas can be served, so
throttling them could prevent a throttle from ever being lifted: quotas load
from hbase:quota, clients resolve region locations through hbase:meta, and
TableNamespaceManager reads hbase:namespace during initClusterSchemaService,
which runs before initQuotaManager creates hbase:quota. An earlier attempt at
this change omitted hbase:namespace and deadlocked master startup on branch-2.x.

QuotaCache#ensureInitialized loaded the cache on the calling RPC handler, and
that load both reads hbase:quota and asks the master for cluster metrics. A read
of a table which is not exempt could therefore wait on a master that cannot
answer until the read itself completes. The load now runs on a separate thread
and the request waits for it with a timeout, configurable via
hbase.quota.cache.initial.load.timeout.ms and defaulting to ten seconds. A
request that waits out the timeout proceeds with default quota state, which does
not throttle, and the load is picked up once it finishes.

This keeps the load synchronous in effect, so a quota which was just set is
still visible to the next lookup and cluster scope factors are still computed
before the first cluster scope check. It also means requests no longer queue on
the monitor that the refresh holds while it talks to the master.

Adds TestQuotaUtil covering the exemption, including that hbase:namespace is
exempt while the backup tables and the other hbase namespace tables are not.
4 files changed
tree: 59ab8fa6a7746e64b24dc52a0ff53b4c367262b7
  1. .github/
  2. bin/
  3. conf/
  4. dev-support/
  5. hbase-annotations/
  6. hbase-archetypes/
  7. hbase-assembly/
  8. hbase-assembly-byo-hadoop/
  9. hbase-asyncfs/
  10. hbase-backup/
  11. hbase-balancer/
  12. hbase-build-configuration/
  13. hbase-checkstyle/
  14. hbase-client/
  15. hbase-common/
  16. hbase-compression/
  17. hbase-dev-generate-classpath/
  18. hbase-diagnostics/
  19. hbase-endpoint/
  20. hbase-examples/
  21. hbase-extensions/
  22. hbase-external-blockcache/
  23. hbase-hadoop-compat/
  24. hbase-hbtop/
  25. hbase-http/
  26. hbase-it/
  27. hbase-logging/
  28. hbase-mapreduce/
  29. hbase-metrics/
  30. hbase-metrics-api/
  31. hbase-procedure/
  32. hbase-protocol-shaded/
  33. hbase-replication/
  34. hbase-resource-bundle/
  35. hbase-rest/
  36. hbase-server/
  37. hbase-shaded/
  38. hbase-shell/
  39. hbase-testing-util/
  40. hbase-thrift/
  41. hbase-website/
  42. hbase-zookeeper/
  43. .asf.yaml
  44. .editorconfig
  45. .git-blame-ignore-revs
  46. .gitattributes
  47. .gitignore
  48. .pylintrc
  49. .rubocop.yml
  50. AGENTS.md
  51. CHANGES.txt
  52. LICENSE.txt
  53. NOTICE.txt
  54. pom.xml
  55. README.md
  56. SECURITY.md
README.md

hbase-logo

Apache HBase is an open-source, distributed, versioned, column-oriented store modeled after Google's Bigtable: A Distributed Storage System for Structured Data by Chang et al. Just as Bigtable leverages the distributed data storage provided by the Google File System, HBase provides Bigtable-like capabilities on top of Apache Hadoop.

Getting Started

To get started using HBase, visit the project home page. The HBase Reference Guide has a ‘Quick Start’ section and is where you should begin your exploration of the HBase project.

The latest HBase can be downloaded from the download page.

We use mailing lists to send notices and discuss. See the mailing lists and archives for more information.

We use the #hbase channel on the official ASF Slack Workspace for real time questions and discussions. Please mail dev@hbase.apache.org to request an invite.

How to Contribute

The source code can be found at https://hbase.apache.org/source-repository

The HBase issue tracker is at https://issues.apache.org/jira/browse/HBASE

Notice that, the public registration for https://issues.apache.org/ has been disabled due to spam. If you want to contribute to HBase, please visit the Request a jira account page to submit your request. Please make sure to select hbase as the ‘ASF project you want to file a ticket’ so we can receive your request and process it.

NOTE: we need to process the requests manually so it may take sometime, for example, up to a week, for us to respond to your request.

About

Apache HBase is made available under the Apache License, Version 2.0.

The HBase distribution includes cryptographic software. See the export control notice for more information.