blob: 09e3d767ee0f9b47e74f5e73c06aca68f8becc78 [file] [log] [blame]
<!DOCTYPE html>
<html lang="en">
<meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
<meta name="description" content="A new open source Apache Hadoop ecosystem project, Apache Kudu completes Hadoop's storage layer to enable fast analytics on fast data" />
<meta name="author" content="Cloudera" />
<title>Apache Kudu - Apache Kudu Weekly Update August 16th, 2016</title>
<!-- Bootstrap core CSS -->
<link rel="stylesheet" href=""
<!-- Custom styles for this template -->
<link href="/css/kudu.css" rel="stylesheet"/>
<link href="/css/asciidoc.css" rel="stylesheet"/>
<link rel="shortcut icon" href="/img/logo-favicon.ico" />
<link rel="stylesheet" href="" />
<link rel="alternate" type="application/atom+xml"
title="RSS Feed for Apache Kudu blog"
href="/feed.xml" />
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
<!--[if lt IE 9]>
<script src=""></script>
<script src=""></script>
<div class="kudu-site container-fluid">
<!-- Static navbar -->
<nav class="navbar navbar-default">
<div class="container-fluid">
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<a class="logo" href="/"><img
srcset="// 1x, // 2x"
alt="Apache Kudu"/></a>
<div id="navbar" class="collapse navbar-collapse">
<ul class="nav navbar-nav navbar-right">
<li >
<a href="/">Home</a>
<li >
<a href="/overview.html">Overview</a>
<li >
<a href="/docs/">Documentation</a>
<li >
<a href="/releases/">Releases</a>
<li class="active">
<a href="/blog/">Blog</a>
<!-- NOTE: this dropdown menu does not appear on Mobile, so don't add anything here
that doesn't also appear elsewhere on the site. -->
<li class="dropdown">
<a href="/community.html" role="button" aria-haspopup="true" aria-expanded="false">Community <span class="caret"></span></a>
<ul class="dropdown-menu">
<li class="dropdown-header">GET IN TOUCH</li>
<li><a class="icon email" href="/community.html">Mailing Lists</a></li>
<li><a class="icon slack" href="">Slack Channel</a></li>
<li role="separator" class="divider"></li>
<li><a href="/community.html#meetups-user-groups-and-conference-presentations">Events and Meetups</a></li>
<li><a href="/committers.html">Project Committers</a></li>
<!--<li><a href="/roadmap.html">Roadmap</a></li>-->
<li><a href="/community.html#contributions">How to Contribute</a></li>
<li role="separator" class="divider"></li>
<li class="dropdown-header">DEVELOPER RESOURCES</li>
<li><a class="icon github" href="">GitHub</a></li>
<li><a class="icon gerrit" href="">Gerrit Code Review</a></li>
<li><a class="icon jira" href="">JIRA Issue Tracker</a></li>
<li role="separator" class="divider"></li>
<li class="dropdown-header">SOCIAL MEDIA</li>
<li><a class="icon twitter" href="">Twitter</a></li>
<li><a href="">Reddit</a></li>
<li role="separator" class="divider"></li>
<li class="dropdown-header">APACHE SOFTWARE FOUNDATION</li>
<li><a href="" target="_blank">Security</a></li>
<li><a href="" target="_blank">Sponsorship</a></li>
<li><a href="" target="_blank">Thanks</a></li>
<li><a href="" target="_blank">License</a></li>
<li >
<a href="/faq.html">FAQ</a>
</ul><!-- /.nav -->
</div><!-- /#navbar -->
</div><!-- /.container-fluid -->
<div class="row header">
<div class="col-lg-12">
<h2><a href="/blog">Apache Kudu Blog</a></h2>
<div class="row-fluid">
<div class="col-lg-9">
<h1 class="entry-title">Apache Kudu Weekly Update August 16th, 2016</h1>
<p class="meta">Posted 16 Aug 2016 by Todd Lipcon</p>
<div class="entry-content">
<p>Welcome to the twentieth edition of the Kudu Weekly Update. This weekly blog post
covers ongoing development and news in the Apache Kudu project.</p>
<h2 id="project-news">Project news</h2>
<p>The first release candidate for the 0.10.0 is <a href="">now available</a></p>
<p>Community developers and users are encouraged to download the source
tarball and vote on the release.</p>
<p>For information on what’s new, check out the
<a href="">release notes</a>.
<em>Note:</em> some links from these in-progress release notes will not be live until the
release itself is published.</p>
<h2 id="development-discussions-and-code-in-progress">Development discussions and code in progress</h2>
<p>Will Berkeley spent some time working on the Spark integration this week
to add support for UPSERT as well as other operations.
Dan Burkert pitched in a bit with some <a href="">suggestions</a>
which were then integrated in a <a href="">patch</a>
provided by Will.</p>
<p>After some reviews by Dan, Chris George, and Ram Mettu, the patch was committed
in time for the upcoming 0.10.0 release.</p>
<p>Dan Burkert also completed work for the new <a href="">manual partitioning APIs</a>
in the Java client. After finishing up the basic implementation, Dan also made some
cleanups to the related APIs in both the <a href="">Java</a>
and <a href="">C++</a> clients.</p>
<p>Dan and Misty Stanley-Jones also collaborated to finish the
<a href="">documentation</a>
for this new feature.</p>
<p>Adar Dembo worked on some tooling to allow users to migrate their Kudu clusters
from a single-master configuration to a multi-master one. Along the way, he
started building some common infrastructure for command-line tooling.</p>
<p>Since Kudu’s initial release, it has included separate binaries for different
administrative or operational tools (e.g. <code class="language-plaintext highlighter-rouge">kudu-ts-cli</code>, <code class="language-plaintext highlighter-rouge">kudu-ksck</code>, <code class="language-plaintext highlighter-rouge">kudu-fs_dump</code>,
<code class="language-plaintext highlighter-rouge">log-dump</code>, etc). Despite having similar usage, these tools don’t share much code,
and the separate statically linked binaries make the Kudu packages take more disk
space than strictly necessary.</p>
<p>Adar’s work has introduced a new top-level <code class="language-plaintext highlighter-rouge">kudu</code> binary which exposes a set of subcommands,
much like the <code class="language-plaintext highlighter-rouge">git</code> and <code class="language-plaintext highlighter-rouge">docker</code> binaries with which readers may be familiar.
For example, a new tool he has built for dumping peer identifiers from a tablet’s
consensus metadata is triggered using <code class="language-plaintext highlighter-rouge">kudu tablet cmeta print_replica_uuids</code>.</p>
<p>This new tool will be available in the upcoming 0.10.0 release; however, migration
of the existing tools to the new infrastructure has not yet been completed. We
expect that by Kudu 1.0, the old tools will be removed in favor of more subcommands
of the <code class="language-plaintext highlighter-rouge">kudu</code> tool.</p>
<p>Todd Lipcon picked up the work started by David Alves in July to provide
<a href="">“exactly-once” semantics</a> for write operations.
Todd carried the patch series through review and also completed integration of the
feature into the Kudu server processes.</p>
<p>After testing the feature for several days on a large cluster under load,
the team decided to enable this new feature by default in Kudu 0.10.0.</p>
<p>Mike Percy resumed working on garbage collection of <a href="">past versions of
updated and deleted rows</a>. His <a href="">main
patch for the feature</a> went through
several rounds of review and testing, but unfortunately missed the cut-off
for 0.10.0.</p>
<p>Alexey Serbin’s work to add doxygen-based documentation for the C++ Client API
was <a href="">committed</a> this week. These
docs will be published as part of the 0.10.0 release.</p>
<p>Alexey also continued work on implementing the <code class="language-plaintext highlighter-rouge">AUTO_FLUSH_BACKGROUND</code> write
mode for the C++ client. This feature makes it easier to implement high-throughput
ingest using the C++ API by automatically handling the batching and flushing of writes
based on a configurable buffer size.</p>
<p>Alexey’s <a href="">patch</a> has received several
rounds of review and looks likely to be committed soon. Detailed performance testing
will follow.</p>
<p>Congratulations to Ram Mettu for committing his first patch to Kudu this week!
Ram fixed a <a href="">bug in handling Alter Table with TIMESTAMP columns</a>.</p>
<h2 id="upcoming-talks">Upcoming talks</h2>
<li>Mike Percy will be speaking about Kudu this Wednesday at the
<a href="">Denver Cloudera User Group</a>
and on Thursday at the
<a href="">Boulder/Denver Big Data Meetup</a>.
If you’re based in the Boulder/Denver area, be sure not to miss these talks!</li>
<p>Want to learn more about a specific topic from this blog post? Shoot an email to the
<a href="">kudu-user mailing list</a> or
tweet at <a href="">@ApacheKudu</a>. Similarly, if you’re
aware of some Kudu news we missed, let us know so we can cover it in
a future post.</p>
<div class="col-lg-3 recent-posts">
<h3>Recent posts</h3>
<li> <a href="/2020/07/30/building-near-real-time-big-data-lake.html">Building Near Real-time Big Data Lake</a> </li>
<li> <a href="/2020/05/18/apache-kudu-1-12-0-release.html">Apache Kudu 1.12.0 released</a> </li>
<li> <a href="/2019/11/20/apache-kudu-1-11-1-release.html">Apache Kudu 1.11.1 released</a> </li>
<li> <a href="/2019/11/20/apache-kudu-1-10-1-release.html">Apache Kudu 1.10.1 released</a> </li>
<li> <a href="/2019/07/09/apache-kudu-1-10-0-release.html">Apache Kudu 1.10.0 Released</a> </li>
<li> <a href="/2019/04/30/location-awareness.html">Location Awareness in Kudu</a> </li>
<li> <a href="/2019/04/22/fine-grained-authorization-with-apache-kudu-and-impala.html">Fine-Grained Authorization with Apache Kudu and Impala</a> </li>
<li> <a href="/2019/03/19/testing-apache-kudu-applications-on-the-jvm.html">Testing Apache Kudu Applications on the JVM</a> </li>
<li> <a href="/2019/03/15/apache-kudu-1-9-0-release.html">Apache Kudu 1.9.0 Released</a> </li>
<li> <a href="/2019/03/05/transparent-hierarchical-storage-management-with-apache-kudu-and-impala.html">Transparent Hierarchical Storage Management with Apache Kudu and Impala</a> </li>
<li> <a href="/2018/12/11/call-for-posts.html">Call for Posts</a> </li>
<li> <a href="/2018/10/26/apache-kudu-1-8-0-released.html">Apache Kudu 1.8.0 Released</a> </li>
<li> <a href="/2018/09/26/index-skip-scan-optimization-in-kudu.html">Index Skip Scan Optimization in Kudu</a> </li>
<li> <a href="/2018/09/11/simplified-pipelines-with-kudu.html">Simplified Data Pipelines with Kudu</a> </li>
<li> <a href="/2018/08/06/getting-started-with-kudu-an-oreilly-title.html">Getting Started with Kudu - an O'Reilly Title</a> </li>
<footer class="footer">
<div class="row">
<div class="col-md-9">
<p class="small">
Copyright &copy; 2019 The Apache Software Foundation.
<p class="small">
Apache Kudu, Kudu, Apache, the Apache feather logo, and the Apache Kudu
project logo are either registered trademarks or trademarks of The
Apache Software Foundation in the United States and other countries.
<div class="col-md-3">
<a class="pull-right" href="">
<img src=""/>
<script src=""></script>
// Try to detect touch-screen devices. Note: Many laptops have touch screens.
$(document).ready(function() {
if ("ontouchstart" in document.documentElement) {
} else {
<script src=""
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
ga('create', 'UA-68448017-1', 'auto');
ga('send', 'pageview');
<script src=""></script>
anchors.options = {
placement: 'right',
visible: 'touch',