| --- |
| layout: post |
| status: PUBLISHED |
| published: true |
| title: Saving CPU! Using Native Hadoop Libraries for CRC computation in HBase |
| id: 06a13081-1422-4706-8682-849f8b67991f |
| date: '2015-06-06 01:51:31 -0400' |
| categories: hbase |
| tags: |
| - checksums |
| - native |
| - hadoop |
| - crc |
| - hbase |
| permalink: hbase/entry/saving_cpu_using_native_hadoop |
| --- |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;" id="docs-internal-guid-01082555-c635-1396-6a69-6a06df3b7a46"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">by Apekshit Sharma, HBase contributor and Cloudera Engineer</span></p> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"><b>TL;DR</b> Use Hadoop Native Library calculating CRC and save CPU!</span></p> |
| <h1 dir="ltr" style="line-height: 1.38; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-family: 'Trebuchet MS'; font-weight: normal; vertical-align: baseline; background-color: transparent;">Checksums in HBase</span></h1> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Checksum are used to check data integrity. HDFS computes and stores checksums for all files on write. One checksum is written per chunk of data (size can be configured using </span><span style="font-size: 15px; font-family: Arial; font-style: italic; vertical-align: baseline; background-color: transparent;">bytes.per.checksum</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">) in a separate, companion checksum file. When data is read back, the file with corresponding checksums is read back as well and is used to ensure data integrity. However, having two files results in two disk seeks reading any chunk of data. For HBase, the extra seek while reading HFileBlock results in extra latency. To work around the extra seek, HBase inlines checksums. HBase calculates checksums for the data in a HFileBlock and appends them to the end of the block itself on write to HDFS (HDFS then checksums the HBase data+inline checksums). On read, by default HDFS checksum verification is turned off, and HBase itself verifies data integrity.</span></p> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt; text-align: center;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"><img src="https://docs.google.com/a/cloudera.com/drawings/d/siWw7jzHFxaiiu2sClidr8A/image?w=407&h=207&rev=473&ac=1" width="407" height="207" style="transform: rotate(0rad); -webkit-transform: rotate(0rad);" /></span></p> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Can we then get rid of HDFS checksum altogether? Unfortunately no. While HBase can detect corruptions, it can’t fix them, whereas HDFS uses replication and a background process to detect and *fix* data corruptions if and when they happen. Since HDFS checksums generated at write-time are also available, we fall back to them when HBase verification fails for any reason. If the HDFS check fails too, the data is reported as corrupt.</span></p> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">The related hbase configurations are </span><span style="font-size: 15px; font-family: Arial; font-style: italic; vertical-align: baseline; background-color: transparent;">hbase.hstore.checksum.algorithm</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">, </span><span style="font-size: 15px; font-family: Arial; font-style: italic; vertical-align: baseline; background-color: transparent;">hbase.hstore.bytes.per.checksum</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> and </span><span style="font-size: 15px; font-family: Arial; font-style: italic; vertical-align: baseline; background-color: transparent;">hbase.regionserver.checksum.verify.</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> HBase inline checksums are enabled by default.</span></p> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Calculating checksums is computationally expensive and requires lots of CPU. When HDFS switched over to JNI + C for computing checksums, they witnessed big gains in CPU usage.</span></p> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">This post is about replicating those gains in HBase by using Native Hadoop Libraries (NHL). See </span><a href="https://issues.apache.org/jira/browse/HBASE-11927"><span style="font-size: 15px; font-family: Arial; color: #1155cc; text-decoration: underline; vertical-align: baseline; background-color: transparent;">HBASE-11927</span></a></p> |
| <p> </p> |
| <h1 dir="ltr" style="line-height: 1.38; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-family: 'Trebuchet MS'; font-weight: normal; vertical-align: baseline; background-color: transparent;">Survey</span></h1> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">We switched to use the Hadoop DataChecksum library which under-the-hood uses NHL if available, else we fall back to use the Java CRC implementation. Another alternative considered was the ‘</span><a href="https://github.com/trevorr/circe"><span style="font-size: 15px; font-family: Arial; color: #1155cc; text-decoration: underline; vertical-align: baseline; background-color: transparent;">Circe</span></a><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">’ library. The following table highlights the differences with NHL and makes the reasoning for our choice clear.</span></p> |
| <p> </p> |
| <div dir="ltr" style="margin-left: 0pt;"> |
| <table style="border: none; border-collapse: collapse; width: 624px;"> |
| <colgroup> |
| <col width="*" /> |
| <col width="*" /></colgroup> |
| <tbody> |
| <tr style="height: 0px;"> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><a href="http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/NativeLibraries.html"><span style="font-size: 15px; font-family: Arial; color: #1155cc; text-decoration: underline; vertical-align: baseline; background-color: transparent;">Hadoop Native Library</span></a></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><a href="https://github.com/trevorr/circe"><span style="font-size: 15px; font-family: Arial; color: #1155cc; text-decoration: underline; vertical-align: baseline; background-color: transparent;">Circe</span></a></p> |
| </td> |
| </tr> |
| <tr style="height: 0px;"> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Native code supports both crc32 and crc32c</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Native code supports only crc32c</span></p> |
| </td> |
| </tr> |
| <tr style="height: 0px;"> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Adds dependency on hadoop-common which is reliable and actively developed</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Adds dependency on external project</span></p> |
| </td> |
| </tr> |
| <tr style="height: 0px;"> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Interface supports taking in stream of data, stream of checksums, chunk size as parameters and compute/verify checksums considering data in chunks.</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Only supports calculation of single checksum for all input data.</span></p> |
| </td> |
| </tr> |
| </tbody> |
| </table></div> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Both libraries supported use of the special x86 instruction for hardware calculation of CRC32C if available (defined in </span><a href="http://en.wikipedia.org/wiki/SSE4#SSE4.2"><span style="font-size: 15px; font-family: Arial; color: #1155cc; text-decoration: underline; vertical-align: baseline; background-color: transparent;">SSE4.2 instruction set</span></a><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">). In the case of NHL, hadoop-2.6.0 or newer version is required for HBase to get the native checksum benefit.</span></p> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">However, based on the data layout of HFileBlock, which has ‘real data’ followed by checksums on the end, only NHL supported the interface we wanted. Implementing the same in Circe would have been significant effort. So we chose to go with NHL. </span></p> |
| <h1 dir="ltr" style="line-height: 1.38; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-family: 'Trebuchet MS'; font-weight: normal; vertical-align: baseline; background-color: transparent;">Setup</span></h1> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Since the metric to be evaluated was CPU usage, a simple configuration of two nodes was used. Node1 was configured to be the NameNode, Zookeeper and HBase master. Node2 was configured to be DataNode and RegionServer. All real computational work was done on Node2 while Node1 remained idle most of the time. This isolation of work on a single node made it easier to measure impact on CPU usage.</span></p> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 16px; font-family: Arial; font-weight: bold; vertical-align: baseline; background-color: transparent;">Configuration</span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Ubuntu 14.04.2 LTS (GNU/Linux 3.13.0-24-generic x86_64)</span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz </span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Socket(s) : 1</span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Core(s) per socket : 1</span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Thread(s) per core : 4</span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Logical CPU(s) : 4</span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Number of disks : 1</span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Memory : 8 GB</span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">HBase Version/Distro *: 1.0.0 / CDH 5.4.0</span></p> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">*Since trunk resolves to hadoop-2.5.1 which does not have </span><a href="https://github.com/apache/hadoop/commit/ab638e77b811d9592470f7d342cd11a66efbbf0d#diff-7ac71c978eb2f964eb66793f21f807be"><span style="font-size: 15px; font-family: Arial; color: #1155cc; text-decoration: underline; vertical-align: baseline; background-color: transparent;">HDFS-6865</span></a><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">, it was easier to use a CDH distro which already has HDFS-6865 backported.</span></p> |
| <h3 dir="ltr" style="line-height: 1.38; margin-top: 8pt; margin-bottom: 0pt;"><span style="font-size: 16px; font-family: 'Trebuchet MS'; color: #666666; vertical-align: baseline; background-color: transparent;">Procedure</span></h3> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">We chose to study the impact on major compactions, mainly because of the presence of </span><a href="https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java"><span style="font-size: 15px; font-family: Arial; color: #1155cc; text-decoration: underline; vertical-align: baseline; background-color: transparent;">CompactionTool</span></a><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> which a) can be used offline, b) allowed us to profile only the relevant workload. PerformanceEvaluation (bin/hbase pe) was used to build a test table which was then copied to local disk for reuse.</span></p> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; font-style: italic; vertical-align: baseline; background-color: transparent;">% ./bin/hbase pe --nomapred --rows=150000 --table="t1" --valueSize=10240 --presplit=10 sequentialWrite 10</span></p> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Table size: 14.4G</span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Number of rows: 1.5M</span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Number of regions: 10</span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Row size: 10K</span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Total store files across regions: 67</span></p> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">For profiling, </span><a href="https://code.google.com/p/lightweight-java-profiler/"><span style="font-size: 15px; font-family: Arial; color: #1155cc; text-decoration: underline; vertical-align: baseline; background-color: transparent;">Lightweight-java-profiler</span></a><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> was used and </span><a href="https://github.com/brendangregg/FlameGraph"><span style="font-size: 15px; font-family: Arial; color: #1155cc; text-decoration: underline; vertical-align: baseline; background-color: transparent;">FlameGraph</span></a><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> was used to generate graphs.</span></p> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">For benchmarking, the linux ‘</span><span style="font-size: 15px; font-family: Arial; font-style: italic; vertical-align: baseline; background-color: transparent;">time’</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> command was used. Profiling was disabled during these runs. A script repeatedly executed following in order:</span></p> |
| <ol style="margin-top: 0pt; margin-bottom: 0pt;"> |
| <li dir="ltr" style="list-style-type: decimal; font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="vertical-align: baseline; background-color: transparent;">delete </span><span style="font-style: italic; vertical-align: baseline; background-color: transparent;">hdfs:///hbase</span></p> |
| </li> |
| <li dir="ltr" style="list-style-type: decimal; font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="vertical-align: baseline; background-color: transparent;">copy t1 from local disk </span><span style="font-style: italic; vertical-align: baseline; background-color: transparent;">hdfs:///hbase/data/default</span></p> |
| </li> |
| <li dir="ltr" style="list-style-type: decimal; font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="vertical-align: baseline; background-color: transparent;">run compaction tool on t1 and time it</span></p> |
| </li> |
| </ol> |
| <h1 dir="ltr" style="line-height: 1.38; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-family: 'Trebuchet MS'; font-weight: normal; vertical-align: baseline; background-color: transparent;">Observations</span></h1> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; font-weight: bold; vertical-align: baseline; background-color: transparent;">Profiling</span></p> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">CPU profiling of HBase not using NHL (figure 1) shows that about 22% cpu is used for generating and validating checksums, whereas, while using NHL (figure 2) it takes only about 3%.</span></p> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"><img src="https://lh6.googleusercontent.com/FjbuvCYql6Jc-aNXzl1FHO4828kJfeppkDFhD-rcPPul0GirpFJzPIowuvJPxB5mnxNbSWHextDXCzMMbaAUjJIhXF-7na4_HFmcN-JtE9_Xi8MzoR7Nf1BhcE-6elPnEzlUbhQ" width="624" height="403" style="transform: rotate(0rad); -webkit-transform: rotate(0rad);" alt="Screen Shot 2015-06-02 at 8.14.07 PM.png" /></span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt; text-align: center;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Figure 1: CPU profile - HBase not using NHL (</span><a href="https://drive.google.com/file/d/0BxfFHM6D5k9AMXI5M0pNWnNoSGc/view?usp=sharing"><span style="font-size: 15px; font-family: Arial; color: #1155cc; text-decoration: underline; vertical-align: baseline; background-color: transparent;">svg</span></a><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">)</span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"><img src="https://lh5.googleusercontent.com/Hxd-7ZZtjMJ65TtGloarURWHVdqJNoSVpDA-5vJ-oDKy4Ir5ukvsW6dHF4bRgldjkEzIe_IZyq2vNvTeAdZjpyL-h00doxT_yJ44526sRyUWjd1YZzCVBIYg5i9DZFMr2130ao8" width="624" height="401" style="transform: rotate(0rad); -webkit-transform: rotate(0rad);" alt="Screen Shot 2015-06-02 at 8.03.39 PM.png" /></span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt; text-align: center;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Figure 2: CPU profile - HBase using NHL (</span><a href="https://drive.google.com/file/d/0BxfFHM6D5k9AZUU4ZmM2ZjhoNnc/view?usp=sharing"><span style="font-size: 15px; font-family: Arial; color: #1155cc; text-decoration: underline; vertical-align: baseline; background-color: transparent;">svg</span></a><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">)</span></p> |
| <p> </p> |
| <h3 dir="ltr" style="line-height: 1.38; margin-top: 8pt; margin-bottom: 0pt;"><span style="font-size: 16px; font-family: 'Trebuchet MS'; color: #666666; vertical-align: baseline; background-color: transparent;">Benchmarking</span></h3> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Benchmarking was done for three different cases: (a) neither HBase nor HDFS use NHL, (b) HDFS uses NHL but not HBase, and (c) both HDFS and HBase use NHL. For each case, we did 5 runs. Observations from table 1:</span></p> |
| <ol style="margin-top: 0pt; margin-bottom: 0pt;"> |
| <li dir="ltr" style="list-style-type: decimal; font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="vertical-align: baseline; background-color: transparent;">Within a case, while real time fluctuates across runs, user and sys times remain same. This is expected as compactions are IO bound.</span></p> |
| </li> |
| <li dir="ltr" style="list-style-type: decimal; font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="vertical-align: baseline; background-color: transparent;">Using NHL only for HDFS reduces CPU usage by about 10% (A vs B)</span></p> |
| </li> |
| <li dir="ltr" style="list-style-type: decimal; font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="vertical-align: baseline; background-color: transparent;">Further, using NHL for HBase checksums reduces CPU usage by about 23% (B vs C).</span></p> |
| </li> |
| </ol> |
| <p> </p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">All times are in seconds. </span><a href="http://stackoverflow.com/questions/556405/what-do-real-user-and-sys-mean-in-the-output-of-time1"><span style="font-size: 15px; font-family: Arial; color: #1155cc; text-decoration: underline; vertical-align: baseline; background-color: transparent;">This</span></a><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> stackoverflow answer provides a good explaination of real, user and sys times.</span></p> |
| <p> </p> |
| <div dir="ltr" style="margin-left: 0pt;"> |
| <table style="border: none; border-collapse: collapse;"> |
| <colgroup> |
| <col width="65" /> |
| <col width="189" /> |
| <col width="177" /> |
| <col width="192" /></colgroup> |
| <tbody> |
| <tr style="height: 0px;"> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">run #</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">no native for HDFS and HBase (A)</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">no native for HBase (B)</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">native (C)</span></p> |
| </td> |
| </tr> |
| <tr style="height: 0px;"> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">1</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">real 469.4</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">user</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">110.8</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">sys</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">30.5</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">real 422.9</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">user 95.4</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">sys 30.5</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">real</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">414.6</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">user</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">67.5</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">sys</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">30.6</span></p> |
| </td> |
| </tr> |
| <tr style="height: 0px;"> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">2</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">real</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">384.3</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">user</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">111.4</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">sys</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">30.4</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">real</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">400.5</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">user</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">96.7</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">sys</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">30.5</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">real</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">393.8</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">user 67.6</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">sys</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">30.6</span></p> |
| </td> |
| </tr> |
| <tr style="height: 0px;"> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">3</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">real</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">400.7</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">user</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">111.5</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">sys</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">30.6</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">real</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">398.6</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">user</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">95.8</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">sys</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">30.6</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">real 392.0</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">user 66.9</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">sys 30.5</span></p> |
| </td> |
| </tr> |
| <tr style="height: 0px;"> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">4</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">real</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">396.8</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">user</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">111.1</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">sys</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">30.3</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">real</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">379.5</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">user</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">96.0</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">sys</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">30.4</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">real 390.8</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">user 67.2</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">sys 30.5</span></p> |
| </td> |
| </tr> |
| <tr style="height: 0px;"> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">5</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">real</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">389.1</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">user</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">111.6</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">sys</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">30.3</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">real</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">377.4</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">user</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">96.5</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">sys</span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"> </span><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">30.4</span></p> |
| </td> |
| <td style="border: 1px solid #000000; vertical-align: top; padding: 7px;"> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">real 381.3</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">user 67.6</span></p> |
| <p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">sys 30.5</span></p> |
| </td> |
| </tr> |
| </tbody> |
| </table></div> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 10pt; margin-bottom: 0pt; text-align: center;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Table 1</span></p> |
| <p dir="ltr" style="line-height: 1.38; margin-top: 10pt; margin-bottom: 0pt; text-align: center;"><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;"><img src="https://lh4.googleusercontent.com/5LiMLhloVoGrSKHKYbhjCqUOKlV8Uwq8u-zlKmYVzCQHn8C61mHTcQJ3kGhGX4FPOm57Z8Vgc_IJLVj7J0n7fzr-XvmoIudkGroHGMvTwBKQyw0dyvlLdhXCf2d_qo_I81LJ34A" width="624" height="524" style="transform: rotate(0rad); -webkit-transform: rotate(0rad);" alt="times.png" /></span></p> |
| <h1 dir="ltr" style="line-height: 1.38; margin-top: 10pt; margin-bottom: 0pt;"><span style="font-family: 'Trebuchet MS'; font-weight: normal; vertical-align: baseline; background-color: transparent;">Conclusion</span></h1> |
| <p><span style="font-size: 15px; font-family: Arial; vertical-align: baseline; background-color: transparent;">Native Hadoop Library leverages the special processor instruction (if available) that does pipelining and other low level optimizations when performing CRC calculations. Using NHL in HBase for heavy checksum computation, allows HBase make use of this facility, saving significant amounts of CPU time checksumming.</span></p> |
