---
layout: post
status: PUBLISHED
published: true
title: Migration to the New Metrics Hotness &ndash; Metrics2
author:
  display_name: Michael Stack
  login: stack
  email: stack@apache.org
author_login: stack
author_email: stack@apache.org
id: 84f4fd20-7ca7-46fb-a80a-4ce1afee9fec
date: '2019-03-02 21:41:51 -0500'
categories:
- General
tags:
- hadoop
- apache
- metrics
- hbase
- metrics2
comments: []
permalink: hbase/entry/migration_to_the_new_metrics
---
<p><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-family: Times; font-weight: normal;"> </p>
<p style="font-size: medium;">by Elliott Clark</p>
<p><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"> </strong></p>
<p style="display: inline !important;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><em><font size="2">HBase Committer and Cloudera Engineer&nbsp;</font></em></strong></p></p>
<p style="font-size: medium;">
<p></strong> </p>
<p dir="ltr" style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt; display: inline !important;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-family: Times; font-size: medium; font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="line-height: normal; font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">NOTE: This blog post describes the server code of HBase. It assumes a general knowledge of the system. You can read more about HBase in the blog posts </span><a href="http://blog.cloudera.com/blog/category/hbase"><span style="font-size: 16px; font-family: Cambria; color: #0000ff; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">here</span></a><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">.</span></strong></strong></strong></strong></strong></strong></strong></strong></strong></strong></strong></strong></strong></strong></strong></strong></p>
<p><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><span style="color: #4f81bd; font-family: Calibri; font-size: 17px; font-weight: bold; letter-spacing: -0.018em; line-height: 1; white-space: pre-wrap; background-color: transparent;">Introduction</span></strong></strong></strong></strong></strong></p>
<p><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><strong id="docs-internal-guid-2b6d978d-5e63-0634-270e-7df7748990e7" style="font-weight: normal;"><span style="color: #4f81bd; font-family: Calibri; font-size: 17px; font-weight: bold; letter-spacing: -0.018em; line-height: 1; white-space: pre-wrap; background-color: transparent;"></span></strong></strong></strong></strong></strong><span style="font-family: Cambria; font-size: 16px; white-space: pre-wrap; line-height: 1; text-indent: 36pt; background-color: transparent;">HBase is a distributed big data store modeled after Google&rsquo;s Bigtable paper. As with all distributed systems, knowing what&rsquo;s happening at a given time can help &nbsp;spot problems before they arise, debug on-going issues, evaluate new usage patterns, and provide insight into capacity planning. </span></p>
<p><span style="font-family: Cambria; font-size: 16px; white-space: pre-wrap; line-height: 1; text-indent: 36pt; background-color: transparent;"></span><span style="text-indent: 36pt; font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">Since October 2008, version 0.19.0 , (</span><a href="https://issues.apache.org/jira/browse/HBASE-625" style="line-height: 1; text-indent: 36pt;"><span style="font-size: 16px; font-family: Cambria; color: #0000ff; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">HBASE-625</span></a><span style="text-indent: 36pt; font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">) HBase has been using Hadoop&rsquo;s metrics system to export metrics to JMX, Ganglia, and other metrics sinks. As the code base grew, more and more metrics were added by different developers. New features got metrics. When users needed more data on issues, they added more metrics. These new metrics were not always consistently named, and some were not well documented. </span></p>
<p><span style="text-indent: 36pt; font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">As HBase&rsquo;s metrics system grew organically, Hadoop developers were making a new version of the Metrics system called Metrics2. In </span><a href="https://issues.apache.org/jira/browse/HADOOP-6728" style="line-height: 1; text-indent: 36pt;"><span style="font-size: 16px; font-family: Cambria; color: #0000ff; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">HADOOP-6728</span></a><span style="text-indent: 36pt; font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"> and subsequent JIRAs, a new version of the metrics system was created. This new subsystem has a new name space, different sinks, different sources, more features, and is more complete than the old metrics. When the Metrics2 system was completed, the old system (aka Metrics1) was deprecated. With all of these things in mind, it was time to update HBase&rsquo;s metrics system so </span><a href="https://issues.apache.org/jira/browse/HBASE-4050" style="line-height: 1; text-indent: 36pt;"><span style="font-size: 16px; font-family: Cambria; color: #0000ff; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">HBASE-4050</span></a><span style="text-indent: 36pt; font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"> was started. &nbsp;I also wanted to clean up the implementation cruft that had accumulated.</span></p>
<h2 dir="ltr" style="line-height: 1; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-size: 17px; font-family: Calibri; color: #4f81bd; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">Definitions</span></h2>
<h2 dir="ltr" style="line-height: 1; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-size: 17px; font-family: Calibri; color: #4f81bd; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"></span><span style="letter-spacing: -0.018em; line-height: 1; font-family: Cambria; font-weight: normal; white-space: pre-wrap; background-color: transparent;">The implementation details are pretty dense on terminology so lets make sure everything is defined:</span></h2>
<p><em style="color: #222222; font-family: "Helvetica Neue", "Lucida Grande", Helvetica, Arial, Verdana, sans-serif; font-style: normal; background-color: #f9f7ed;"> </p>
<ul style="margin-top: 0pt; margin-bottom: 0pt;">
<li dir="ltr" style="list-style-type: disc; font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">
<p dir="ltr" style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">Metric: A measurement of a property in the system.</span></p>
</li>
<li dir="ltr" style="list-style-type: disc; font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">
<p dir="ltr" style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">Snapshot: A set of metrics at a given point in time.</span></p>
</li>
<li dir="ltr" style="list-style-type: disc; font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">
<p dir="ltr" style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">Metrics1: The old Apache Hadoop metrics system.</span></p>
</li>
<li dir="ltr" style="list-style-type: disc; font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">
<p dir="ltr" style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">Metrics2: The new overhauled Apache Hadoop Metrics system.</span></p>
</li>
<li dir="ltr" style="list-style-type: disc; font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">
<p dir="ltr" style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">Source: A class that exposes metrics to the Hadoop metrics system.</span></p>
</li>
<li dir="ltr" style="list-style-type: disc; font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">
<p dir="ltr" style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">Sink: A class that receives metrics snapshots from the Hadoop metrics system.</span></p>
</li>
<li dir="ltr" style="list-style-type: disc; font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">
<p dir="ltr" style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">JMX: Java Management Extension. A system built into java that facilitates the management of java processes over a network; it includes the ability to expose metrics.</span></p>
</li>
<li dir="ltr" style="list-style-type: disc; font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">
<p dir="ltr" style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">Dynamic Metrics: Metrics that come and go. These metrics are not all known at compile time; instead they are discovered at runtime.</span></p>
</li>
</ul>
<p></em> </p>
<h2 dir="ltr" style="line-height: 1; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-size: 17px; font-family: Calibri; color: #4f81bd; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">Implementation</span></h2>
<h2 dir="ltr" style="line-height: 1; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-size: 17px; font-family: Calibri; color: #4f81bd; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"></span><span style="font-size: medium; font-weight: normal; line-height: normal; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">The Hadoop Metrics2 system implementations in branch-1 and branch-2 have diverged pretty drastically. This means that a single implementation of the code to move metrics from HBase to metrics2 sinks would not be performant or easy. As a result I created different hadoop compatibility shims and a system to load a version at runtime. This led to using </span><a href="http://docs.oracle.com/javase/6/docs/api/java/util/ServiceLoader.html" style="font-size: medium; font-family: verdana, arial, "Bitstream Vera Sans", helvetica, sans-serif; font-weight: normal; line-height: 1;"><span style="font-family: Cambria; color: #0000ff; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">ServiceLoader</span></a><span style="font-size: medium; font-weight: normal; line-height: normal; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"> to create an instance of any class that touched parts of Hadoop that had changed between branch-1 and branch-2.</span></h2>
<h2 dir="ltr" style="line-height: 1; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-size: medium; font-weight: normal; line-height: normal; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"></span><span style="font-family: Cambria; font-size: medium; font-weight: normal; line-height: 1; white-space: pre-wrap; letter-spacing: -0.018em; background-color: transparent;">Here is an example of how a region server could request a Hadoop 2 version of the shim for exposing metrics about the HRegionServer. (Hadoop 1&rsquo;s compatibility jar is shown in dotted lines to indicate that it could be swapped in if Hadoop 1 was being used)</span></h2>
<p dir="ltr" style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"></p>
<p dir="ltr" style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"></span><img src="https://lh3.googleusercontent.com/YoYo9OswOJXVF7g62Yc8hnR-4QG8U9icOWVky41zHQHyenQb4MlozCTrWGM8LQXP6o8aJbqqAJKaMMAdvWvjzOz_Rdxsp-X97wuSw8pxu0ZNql6AKDouaA_w" /><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"></span></p>
<p><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"></span> </p>
<p dir="ltr" style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">This system allows HBase to support both Hadoop 1.x and Hadoop 2.x implementations without using reflection or other tricks to get around differences in API, usage, and naming.</span></p>
<p> <span style="font-family: Cambria; font-size: 16px; line-height: 1; white-space: pre-wrap; background-color: transparent;">Now that HBase can use either the Hadoop 1 or Hadoop 2 versions of the metrics 2 systems, I set about cleaning up what metrics HBase exposes, how those metrics are exposed, naming, and performance of gathering the data. </span></p>
<p><span style="font-family: Cambria; font-size: 16px; line-height: 1; text-indent: 36pt; white-space: pre-wrap; background-color: transparent;">Metrics2 uses either annotations or sources to expose metrics. Since HBase can&rsquo;t require any part of the metrics2 system in the core classes I exposed all metrics from HBase by creating sources. For metrics that are known ahead of time I created wrappers around classes in the core of HBase that the metrics2 shims could interrogate for values. Here is an example on how HRegionServer&rsquo;s metrics(the non-dynamic metrics) are exposed :</span></p>
<p><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"></span><img src="https://lh5.googleusercontent.com/T7WMyUKl_Yz_mvnd5HS3GjSmnpWp6XKs-oDAX0ZW5xtCowt6BNKbUgkUgjkMTGtc4_jWk9QmX7qcJn0op7vZvcYLvFfMo3KwWekiHmULcPFgPQM9LXJk5iHG" /><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"></span><br /><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"></span> </p>
<p dir="ltr" style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">The above pattern can be repeated to expose a great deal of the metrics that HBase has. However metrics about specific regions are still very interesting but can&rsquo;t be exposed following the above pattern. So a new solution that would allow metrics about regions to be exposed by whichever HRegionServer is hosting that region was needed. To complicate things further Hadoop&rsquo;s metrics2 system needs one MetricsSource to be responsible for all metrics that are going to be exposed through a JMX mbean. In order for metrics about regions to be well laid out, HBase needs a way to aggregate metrics from multiple regions into one source. This source will then be responsible for knowing what regions are assigned to the regionserver. These requirements led me to have one aggregation source that contains pseudo-sources for each region. These pseudo-sources each contain a wrapper around the region. This </span><span style="font-family: Cambria; font-size: 16px; line-height: 16px; white-space: pre-wrap;">leads to something that looks like this:</span></p></p>
<p dir="ltr" style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><img src="https://lh4.googleusercontent.com/6XcdhbfZjlneSWFF8pNHULQVKrEfxZ7nknAHdTJstuuhaK3NrAkws-Ug0Cx_iAozwfzqAtyAxQ8zp6Ud5UemXTd-JMBl5AtGGNrVKbYe9aPdkYwZupi5yNMD" /><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"></span></p>
<h2 dir="ltr" style="line-height: 1; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-size: 17px; font-family: Calibri; color: #4f81bd; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span></h2>
<h2 dir="ltr" style="line-height: 1; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-size: 17px; font-family: Calibri; color: #4f81bd; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">Benefits</span></h2>
<h2 dir="ltr" style="line-height: 1; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-size: 17px; font-family: Calibri; color: #4f81bd; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"></span><span style="font-family: Cambria; font-weight: normal; line-height: 1; white-space: pre-wrap; font-size: medium; background-color: transparent;">That&rsquo;s a lot of work to re-do a previously working metrics system, so what was gained by all this work? The entire system is much easier to test in unit and systems tests. The whole system has been made more regular; that is everything follows the same patterns and naming conventions. Finally everything has been rewritten to be faster.</span></h2>
<h2 dir="ltr" style="line-height: 1; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-family: Cambria; font-weight: normal; line-height: 1; white-space: pre-wrap; font-size: medium; background-color: transparent;"></span><span style="font-family: Cambria; font-size: medium; font-weight: normal; letter-spacing: -0.018em; line-height: 1; white-space: pre-wrap; background-color: transparent;">Since the previous metrics have all been added on as needed they were not all named well. Some metrics were named following the pattern: &ldquo;metricNameCount&rdquo; others were named following &ldquo;numMetricName&rdquo; while still others were named like &ldquo;metricName_Count&rdquo;. This made parsing hard and gave a generally chaotic feel. After the overhaul metrics that are a counter start with the camel cased metric name followed by the suffix &ldquo;Count.&rdquo; The mbeans were poorly laid out. Some metrics we spread out between two mbeans. Metrics about a region were under an mbean named Dynamic, not the most descriptive name. Now mbeans are much better organized and have better descriptions.</span></h2>
<h2 dir="ltr" style="line-height: 1; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-family: Cambria; font-size: medium; font-weight: normal; line-height: 1; white-space: pre-wrap; letter-spacing: -0.018em; background-color: transparent;"></span><span style="font-size: medium; letter-spacing: -0.018em; font-weight: normal; line-height: normal; text-indent: 36pt; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">Tests have found that single threaded scans run as much as 9% faster after HBase&rsquo;s old metrics system has been replaced. The previous system used lots of </span><a href="http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ConcurrentHashMap.html" style="font-size: medium; letter-spacing: -0.018em; line-height: 1; font-family: verdana, arial, "Bitstream Vera Sans", helvetica, sans-serif; font-weight: normal; text-indent: 36pt;"><span style="font-family: Cambria; color: #1155cc; text-decoration: underline; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">ConcurrentHashMap&rsquo;s</span></a><span style="font-size: medium; letter-spacing: -0.018em; font-weight: normal; line-height: normal; text-indent: 36pt; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"> to store dynamic metrics. All accesses to mutate these metrics required a lookup into these large hash maps. The new system minimizes the use of maps. Instead every region or server exports metrics to one pseudo source. The only changes to hashmaps in the metrics system occurs on region close or open.</span></h2>
<p><font size="3"><span style="text-indent: 36pt; font-weight: normal; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;"></span></font><span style="color: #4f81bd; font-family: Calibri; font-size: 17px; white-space: pre-wrap; font-weight: bold; letter-spacing: -0.018em; line-height: 1; background-color: transparent;">Conclusions</span></p>
<p><span style="font-size: 16px; font-family: Cambria; vertical-align: baseline; white-space: pre-wrap; background-color: transparent;">Overall the whole system is just better. The process was long and laborious, but worth it to make sure that HBase&rsquo;s metrics system is in a good state. HBase 0.95, and later 0.96, will have the new metrics system. &nbsp;There&rsquo;s still more work to be completed but great strides have been made.</span></p>