| Overview |
| ======== |
| hbase-metrics and hbase-metrics-api are two modules that define and implement the "new" metric |
| system used internally within HBase. These two modules (and some other code in hbase-hadoop2-compat) |
| module are referred as "HBase metrics framework". |
| |
| HBase-metrics-api Module |
| ======================== |
| HBase Metrics API (hbase-metrics-api) contains the interface |
| that HBase exposes internally and to third party code (including coprocessors). It is a thin |
| abstraction over the actual implementation for backwards compatibility guarantees. The source |
| / binary and other compatibility guarantees are for "LimitedPrivate API" (see [1] for an |
| explanation). |
| |
| The metrics API in this hbase-metrics-api module is inspired by the Dropwizard metrics 3.1 API |
| (See [2]). It is a subset of the API only containing metrics collection. However, the implementation |
| is HBase-specific and provided in hbase-metrics module. All of the classes in this module is |
| HBase-internal. See the latest documentation of Dropwizard metrics for examples of defining / using |
| metrics. |
| |
| |
| HBase-metrics Module |
| ==================== |
| hbase-metrics module contains implementation of the "HBase Metrics API", including MetricRegistry, |
| Counter, Histogram, etc. These are highly concurrent implementations of the Metric interfaces. |
| Metrics in HBase are grouped into different sets (like WAL, RPC, RegionServer, etc). Each group of |
| metrics should be tracked via a MetricRegistry specific to that group. Metrics can be dynamically |
| added or removed from the registry with a name. Each Registry is independent of the other |
| registries and will have it's own JMX context and MetricRecord (when used with Metrics2). |
| |
| |
| MetricRegistry's themselves are tracked via a global registry (of MetricRegistries) called |
| MetricRegistries. MetricRegistries.global() can be used to obtain the global instance. |
| MetricRegistry instances can also be dynamically registered and removed. However, unlike the |
| MetricRegistry, MetricRegistries does reference counting of the MetricRegistry instances. Only |
| Metrics in the MetricRegistry instances that are in the global MetricRegistry are exported to the |
| metric sinks or JMX. |
| |
| |
| Coprocessor Metrics |
| =================== |
| HBase allows custom coprocessors to track and export metrics using the new framework. |
| Coprocessors and other third party code should only use the classes and interfaces from |
| hbase-metrics-api module and only the classes that are marked with InterfaceAudience.LimitedPrivate |
| annotation. There is no guarantee on the compatibility requirements for other classes. |
| |
| Coprocessors can obtain the MetricRegistry to register their custom metrics via corresponding |
| CoprocessorEnvironment context. See ExampleRegionObserverWithMetrics and |
| ExampleMasterObserverWithMetrics classes in hbase-examples module for usage. |
| |
| |
| Developer Notes |
| =============== |
| Historically, HBase has been using Hadoop's Metrics2 framework [3] for collecting and reporting the |
| metrics internally. However, due to the difficultly of dealing with the Metrics2 framework, HBase is |
| moving away from Hadoop's metrics implementation to its custom implementation. The move will happen |
| incrementally, and during the time, both Hadoop Metrics2-based metrics and hbase-metrics module |
| based classes will be in the source code. All new implementations for metrics SHOULD use the new |
| API and framework. |
| |
| Examples of the new framework can be found in MetricsCoprocessor and MetricsRegionServerSourceImpl |
| classes. See HBASE-9774 [4] for more context. |
| |
| hbase-metrics module right now only deals with metrics tracking and collection. It does not do JMX |
| reporting or reporting to console, ganglia, opentsdb, etc. We use Hadoop's Metrics2 for reporting |
| metrics to different sinks or exporting via JMX. However, this is contained within the |
| hbase-hadoop2-compat module completely, so that rest of the code does not know anything about the |
| Metrics2 dependency. HBaseMetrics2HadoopMetricsAdapter is the adapter that can collect metrics |
| in a MetricRegistry using the metric2 MetricsCollector / MetricRecordBuilder interfaces. |
| GlobalMetricRegistriesSource is the global Metrics2 Source that collects all of the metrics in all |
| of the metric registries in the MetricRegistries.global() instance. |
| |
| |
| References |
| 1. https://hbase.apache.org/book.html#hbase.versioning |
| 2. http://metrics.dropwizard.io/ |
| 3. https://hadoop.apache.org/docs/r2.7.2/api/org/apache/hadoop/metrics2/package-summary.html |
| 4. https://issues.apache.org/jira/browse/HBASE-9774 |