Publish commit(s) from site source repo:
1c4d974 Fix syntax highlighting on new Jekyll
f70ddde Add blog post about multi-master support in Kudu 1.0
Site-Repo-Commit: 1c4d9744361658ff2a34ec37071425368be55dc8
diff --git a/2016/02/26/apache-kudu-0-7-0-released.html b/2016/02/26/apache-kudu-0-7-0-released.html
index b445d47..b615fda 100644
--- a/2016/02/26/apache-kudu-0-7-0-released.html
+++ b/2016/02/26/apache-kudu-0-7-0-released.html
@@ -121,6 +121,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -149,8 +151,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/03/10/apache-kudu-0-7-1-released.html b/2016/03/10/apache-kudu-0-7-1-released.html
index 1b1226e..0b1fd8e 100644
--- a/2016/03/10/apache-kudu-0-7-1-released.html
+++ b/2016/03/10/apache-kudu-0-7-1-released.html
@@ -120,6 +120,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -148,8 +150,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/03/20/weekly-update.html b/2016/03/20/weekly-update.html
index aec1aec..25dc240 100644
--- a/2016/03/20/weekly-update.html
+++ b/2016/03/20/weekly-update.html
@@ -180,6 +180,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -208,8 +210,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/03/28/weekly-update.html b/2016/03/28/weekly-update.html
index e7bf795..674b682 100644
--- a/2016/03/28/weekly-update.html
+++ b/2016/03/28/weekly-update.html
@@ -171,6 +171,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -199,8 +201,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/04/04/weekly-update.html b/2016/04/04/weekly-update.html
index b73a1db..814c768 100644
--- a/2016/04/04/weekly-update.html
+++ b/2016/04/04/weekly-update.html
@@ -190,6 +190,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -218,8 +220,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/04/11/weekly-update.html b/2016/04/11/weekly-update.html
index 0397f0f..cc142d9 100644
--- a/2016/04/11/weekly-update.html
+++ b/2016/04/11/weekly-update.html
@@ -190,6 +190,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -218,8 +220,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/04/12/apache-kudu-0-8-0-released.html b/2016/04/12/apache-kudu-0-8-0-released.html
index 8d3691f..a9a301f 100644
--- a/2016/04/12/apache-kudu-0-8-0-released.html
+++ b/2016/04/12/apache-kudu-0-8-0-released.html
@@ -111,6 +111,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -139,8 +141,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/04/14/ingesting-json-apache-kudu-streamsets-data-collector.html b/2016/04/14/ingesting-json-apache-kudu-streamsets-data-collector.html
index 226a99d..f0936df 100644
--- a/2016/04/14/ingesting-json-apache-kudu-streamsets-data-collector.html
+++ b/2016/04/14/ingesting-json-apache-kudu-streamsets-data-collector.html
@@ -134,6 +134,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -162,8 +164,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/04/18/weekly-update.html b/2016/04/18/weekly-update.html
index 84c968c..9864267 100644
--- a/2016/04/18/weekly-update.html
+++ b/2016/04/18/weekly-update.html
@@ -188,6 +188,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -216,8 +218,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/04/19/kudu-0-8-0-predicate-improvements.html b/2016/04/19/kudu-0-8-0-predicate-improvements.html
index 45c92b4..28a48b4 100644
--- a/2016/04/19/kudu-0-8-0-predicate-improvements.html
+++ b/2016/04/19/kudu-0-8-0-predicate-improvements.html
@@ -110,15 +110,13 @@
query Kudu will now be able to skip scanning 15 out of the 16 tablets in the
table:</p>
-<p>```SQL
-– create a table with 16 tablets
-CREATE TABLE users (id INT64, name STRING, address STRING)
-DISTRIBUTE BY HASH (id) INTO 16 BUCKETS;</p>
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="c1">-- create a table with 16 tablets</span>
+<span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">users</span> <span class="p">(</span><span class="n">id</span> <span class="n">INT64</span><span class="p">,</span> <span class="n">name</span> <span class="n">STRING</span><span class="p">,</span> <span class="n">address</span> <span class="n">STRING</span><span class="p">)</span>
+<span class="n">DISTRIBUTE</span> <span class="k">BY</span> <span class="n">HASH</span> <span class="p">(</span><span class="n">id</span><span class="p">)</span> <span class="k">INTO</span> <span class="mi">16</span> <span class="n">BUCKETS</span><span class="p">;</span>
-<p>– scan over a single tablet
-SELECT id, name, address FROM users
-WHERE id = 876932;
-```</p>
+<span class="c1">-- scan over a single tablet</span>
+<span class="k">SELECT</span> <span class="n">id</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">address</span> <span class="k">FROM</span> <span class="n">users</span>
+<span class="k">WHERE</span> <span class="n">id</span> <span class="o">=</span> <span class="mi">876932</span><span class="p">;</span></code></pre></div>
<p>For a deeper look at the newly implemented scan and partition pruning
optimizations, see the associated <a href="https://github.com/apache/incubator-kudu/blob/master/docs/design-docs/scan-optimization-partition-pruning.md">design
@@ -138,19 +136,17 @@
API has been deprecated, and will be removed soon. Example of transitioning from
the old to new API:</p>
-<p>```java
-ColumnSchema myIntColumnSchema = …;
-KuduScanner.KuduScannerBuilder scannerBuilder = …;</p>
+<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="n">ColumnSchema</span> <span class="n">myIntColumnSchema</span> <span class="o">=</span> <span class="o">...;</span>
+<span class="n">KuduScanner</span><span class="o">.</span><span class="na">KuduScannerBuilder</span> <span class="n">scannerBuilder</span> <span class="o">=</span> <span class="o">...;</span>
-<p>// Old predicate API
-ColumnRangePredicate predicate = new ColumnRangePredicate(myIntColumnSchema);
-predicate.setLowerBound(20);
-scannerBuilder.addColumnRangePredicate(predicate);</p>
+<span class="c1">// Old predicate API</span>
+<span class="n">ColumnRangePredicate</span> <span class="n">predicate</span> <span class="o">=</span> <span class="k">new</span> <span class="nf">ColumnRangePredicate</span><span class="o">(</span><span class="n">myIntColumnSchema</span><span class="o">);</span>
+<span class="n">predicate</span><span class="o">.</span><span class="na">setLowerBound</span><span class="o">(</span><span class="mi">20</span><span class="o">);</span>
+<span class="n">scannerBuilder</span><span class="o">.</span><span class="na">addColumnRangePredicate</span><span class="o">(</span><span class="n">predicate</span><span class="o">);</span>
-<p>// New predicate API
-scannerBuilder.newPredicate(
- KuduPredicate.newComparisonPredicate(myIntColumnSchema, ComparisonOp.GREATER_EQUAL, 20));
-```</p>
+<span class="c1">// New predicate API</span>
+<span class="n">scannerBuilder</span><span class="o">.</span><span class="na">newPredicate</span><span class="o">(</span>
+ <span class="n">KuduPredicate</span><span class="o">.</span><span class="na">newComparisonPredicate</span><span class="o">(</span><span class="n">myIntColumnSchema</span><span class="o">,</span> <span class="n">ComparisonOp</span><span class="o">.</span><span class="na">GREATER_EQUAL</span><span class="o">,</span> <span class="mi">20</span><span class="o">));</span></code></pre></div>
<h2 id="under-the-covers-changes">Under the Covers Changes</h2>
@@ -172,6 +168,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -200,8 +198,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/04/25/weekly-update.html b/2016/04/25/weekly-update.html
index 5a32d4b..3835323 100644
--- a/2016/04/25/weekly-update.html
+++ b/2016/04/25/weekly-update.html
@@ -187,6 +187,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -215,8 +217,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/04/26/ycsb.html b/2016/04/26/ycsb.html
index 1d8698a..8776997 100644
--- a/2016/04/26/ycsb.html
+++ b/2016/04/26/ycsb.html
@@ -387,6 +387,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -415,8 +417,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/05/03/weekly-update.html b/2016/05/03/weekly-update.html
index 97df784..43bf2ad 100644
--- a/2016/05/03/weekly-update.html
+++ b/2016/05/03/weekly-update.html
@@ -162,6 +162,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -190,8 +192,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/05/09/weekly-update.html b/2016/05/09/weekly-update.html
index 836774e..c7ce93a 100644
--- a/2016/05/09/weekly-update.html
+++ b/2016/05/09/weekly-update.html
@@ -152,6 +152,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -180,8 +182,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/05/16/weekly-update.html b/2016/05/16/weekly-update.html
index e2f009f..c35aff7 100644
--- a/2016/05/16/weekly-update.html
+++ b/2016/05/16/weekly-update.html
@@ -187,6 +187,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -215,8 +217,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/05/23/weekly-update.html b/2016/05/23/weekly-update.html
index 99a352e..dc7659a 100644
--- a/2016/05/23/weekly-update.html
+++ b/2016/05/23/weekly-update.html
@@ -176,6 +176,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -204,8 +206,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/06/01/weekly-update.html b/2016/06/01/weekly-update.html
index e17674e..bdca9e4 100644
--- a/2016/06/01/weekly-update.html
+++ b/2016/06/01/weekly-update.html
@@ -142,6 +142,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -170,8 +172,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/06/02/no-default-partitioning.html b/2016/06/02/no-default-partitioning.html
index 88afbce..d1a54bc 100644
--- a/2016/06/02/no-default-partitioning.html
+++ b/2016/06/02/no-default-partitioning.html
@@ -136,26 +136,22 @@
calling <code>KuduTableCreator:add_hash_partitions</code> with the columns to hash and the
number of buckets to use:</p>
-<p><code>cpp
-unique_ptr<KuduTableCreator> table_creator(my_client->NewTableCreator());
-Status create_status = table_creator->table_name("my-table")
- .schema(my_schema)
- .add_hash_partitions({ "key_column_a", "key_column_b" }, 16)
- .Create();
-if (!create_status.ok() { /* handle error */ }
-</code></p>
+<div class="highlight"><pre><code class="language-cpp" data-lang="cpp"><span class="n">unique_ptr</span><span class="o"><</span><span class="n">KuduTableCreator</span><span class="o">></span> <span class="n">table_creator</span><span class="p">(</span><span class="n">my_client</span><span class="o">-></span><span class="n">NewTableCreator</span><span class="p">());</span>
+<span class="n">Status</span> <span class="n">create_status</span> <span class="o">=</span> <span class="n">table_creator</span><span class="o">-></span><span class="n">table_name</span><span class="p">(</span><span class="s">"my-table"</span><span class="p">)</span>
+ <span class="p">.</span><span class="n">schema</span><span class="p">(</span><span class="n">my_schema</span><span class="p">)</span>
+ <span class="p">.</span><span class="n">add_hash_partitions</span><span class="p">({</span> <span class="s">"key_column_a"</span><span class="p">,</span> <span class="s">"key_column_b"</span> <span class="p">},</span> <span class="mi">16</span><span class="p">)</span>
+ <span class="p">.</span><span class="n">Create</span><span class="p">();</span>
+<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">create_status</span><span class="p">.</span><span class="n">ok</span><span class="p">()</span> <span class="p">{</span> <span class="cm">/* handle error */</span> <span class="p">}</span></code></pre></div>
<h1 id="java-client">Java Client</h1>
<p>And similarly, in Java:</p>
-<p><code>java
-List<String> hashColumns = new ArrayList<>();
-hashColumns.add("key_column_a");
-hashColumn.add("key_column_b");
-CreateTableOptions options = new CreateTableOptions().addHashPartitions(hashColumns, 16);
-myClient.createTable("my-table", my_schema, options);
-</code></p>
+<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="n">List</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">hashColumns</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o"><>();</span>
+<span class="n">hashColumns</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="s">"key_column_a"</span><span class="o">);</span>
+<span class="n">hashColumn</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="s">"key_column_b"</span><span class="o">);</span>
+<span class="n">CreateTableOptions</span> <span class="n">options</span> <span class="o">=</span> <span class="k">new</span> <span class="nf">CreateTableOptions</span><span class="o">().</span><span class="na">addHashPartitions</span><span class="o">(</span><span class="n">hashColumns</span><span class="o">,</span> <span class="mi">16</span><span class="o">);</span>
+<span class="n">myClient</span><span class="o">.</span><span class="na">createTable</span><span class="o">(</span><span class="s">"my-table"</span><span class="o">,</span> <span class="n">my_schema</span><span class="o">,</span> <span class="n">options</span><span class="o">);</span></code></pre></div>
<p>In the examples above, if the hash partition configuration is omitted the create
table operation will fail with the error <code>Table partitioning must be specified
@@ -168,16 +164,15 @@
<p>When creating Kudu tables with Impala, the formerly optional <code>DISTRIBUTE BY</code>
clause is now required:</p>
-<p><code>SQL
-CREATE TABLE my_table (key_column_a STRING, key_column_b STRING, other_column STRING)
-DISTRIBUTE BY HASH (key_column_a, key_column_b) INTO 16 BUCKETS
-TBLPROPERTIES(
- 'storage_handler' = 'com.cloudera.kudu.hive.KuduStorageHandler',
- 'kudu.table_name' = 'my_table',
- 'kudu.master_addresses' = 'kudu-master.example.com:7051',
- 'kudu.key_columns' = 'key_column_a,key_column_b'
-);
-</code></p>
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">my_table</span> <span class="p">(</span><span class="n">key_column_a</span> <span class="n">STRING</span><span class="p">,</span> <span class="n">key_column_b</span> <span class="n">STRING</span><span class="p">,</span> <span class="n">other_column</span> <span class="n">STRING</span><span class="p">)</span>
+<span class="n">DISTRIBUTE</span> <span class="k">BY</span> <span class="n">HASH</span> <span class="p">(</span><span class="n">key_column_a</span><span class="p">,</span> <span class="n">key_column_b</span><span class="p">)</span> <span class="k">INTO</span> <span class="mi">16</span> <span class="n">BUCKETS</span>
+<span class="n">TBLPROPERTIES</span><span class="p">(</span>
+ <span class="s1">'storage_handler'</span> <span class="o">=</span> <span class="s1">'com.cloudera.kudu.hive.KuduStorageHandler'</span><span class="p">,</span>
+ <span class="s1">'kudu.table_name'</span> <span class="o">=</span> <span class="s1">'my_table'</span><span class="p">,</span>
+ <span class="s1">'kudu.master_addresses'</span> <span class="o">=</span> <span class="s1">'kudu-master.example.com:7051'</span><span class="p">,</span>
+ <span class="s1">'kudu.key_columns'</span> <span class="o">=</span> <span class="s1">'key_column_a,key_column_b'</span>
+<span class="p">);</span></code></pre></div>
+
</div>
</article>
@@ -188,6 +183,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -216,8 +213,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/06/06/weekly-update.html b/2016/06/06/weekly-update.html
index fed6135..22eb9e5 100644
--- a/2016/06/06/weekly-update.html
+++ b/2016/06/06/weekly-update.html
@@ -137,6 +137,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -165,8 +167,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/06/10/apache-kudu-0-9-0-released.html b/2016/06/10/apache-kudu-0-9-0-released.html
index 83b7e9d..b424568 100644
--- a/2016/06/10/apache-kudu-0-9-0-released.html
+++ b/2016/06/10/apache-kudu-0-9-0-released.html
@@ -112,6 +112,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -140,8 +142,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/06/13/weekly-update.html b/2016/06/13/weekly-update.html
index 254e774..a4fbdb1 100644
--- a/2016/06/13/weekly-update.html
+++ b/2016/06/13/weekly-update.html
@@ -145,6 +145,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -173,8 +175,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/06/17/raft-consensus-single-node.html b/2016/06/17/raft-consensus-single-node.html
index 509d8e7..049c18b 100644
--- a/2016/06/17/raft-consensus-single-node.html
+++ b/2016/06/17/raft-consensus-single-node.html
@@ -187,6 +187,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -215,8 +217,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/06/21/weekly-update.html b/2016/06/21/weekly-update.html
index e2a2fa6..8b74868 100644
--- a/2016/06/21/weekly-update.html
+++ b/2016/06/21/weekly-update.html
@@ -93,6 +93,8 @@
<p>Welcome to the fourteenth edition of the Kudu Weekly Update. This weekly blog post
covers ongoing development and news in the Apache Kudu (incubating) project.</p>
+<!--more-->
+
<h2 id="development-discussions-and-code-in-progress">Development discussions and code in progress</h2>
<ul>
@@ -121,8 +123,6 @@
uses Raft consensus on a single node, and some changes we’re making as Kudu is getting more mature.</li>
</ul>
-<!--more-->
-
<p>Want to learn more about a specific topic from this blog post? Shoot an email to the
<a href="mailto:user@kudu.incubator.apache.org">kudu-user mailing list</a> or
tweet at <a href="https://twitter.com/ApacheKudu">@ApacheKudu</a>. Similarly, if you’re
@@ -138,6 +138,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -166,8 +168,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/2016/06/24/multi-master-1-0-0.html b/2016/06/24/multi-master-1-0-0.html
new file mode 100644
index 0000000..9b4aa8b
--- /dev/null
+++ b/2016/06/24/multi-master-1-0-0.html
@@ -0,0 +1,277 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8" />
+ <meta http-equiv="X-UA-Compatible" content="IE=edge" />
+ <meta name="viewport" content="width=device-width, initial-scale=1" />
+ <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
+ <meta name="description" content="A new open source Apache Hadoop ecosystem project, Apache Kudu (incubating) completes Hadoop's storage layer to enable fast analytics on fast data" />
+ <meta name="author" content="Cloudera" />
+ <title>Apache Kudu (incubating) - Master fault tolerance in Kudu 1.0</title>
+ <!-- Bootstrap core CSS -->
+ <link href="/css/bootstrap.min.css" rel="stylesheet" />
+
+ <!-- Custom styles for this template -->
+ <link href="/css/justified-nav.css" rel="stylesheet" />
+
+ <link href="/css/kudu.css" rel="stylesheet"/>
+ <link href="/css/asciidoc.css" rel="stylesheet"/>
+ <link rel="shortcut icon" href="/img/logo-favicon.ico" />
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.6.1/css/font-awesome.min.css" />
+
+
+ <link rel="alternate" type="application/atom+xml"
+ title="RSS Feed for Apache Kudu blog"
+ href="/feed.xml" />
+
+
+ <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+ <!-- Fork me on GitHub -->
+ <a class="fork-me-on-github" href="https://github.com/apache/incubator-kudu"><img src="//aral.github.io/fork-me-on-github-retina-ribbons/right-cerulean@2x.png" alt="Fork me on GitHub" /></a>
+
+ <div class="kudu-site container-fluid">
+ <!-- Static navbar -->
+ <nav class="container-fluid navbar-default">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+
+ <a class="logo" href="/"><img src="/img/logo_small.png" width="80" /></a>
+
+ </div>
+ <div id="navbar" class="navbar-collapse collapse navbar-right">
+ <ul class="nav navbar-nav">
+ <li >
+ <a href="/">Home</a>
+ </li>
+ <li >
+ <a href="/overview.html">Overview</a>
+ </li>
+ <li >
+ <a href="/docs/">Documentation</a>
+ </li>
+ <li >
+ <a href="/releases/">Download</a>
+ </li>
+ <li class="active">
+ <a href="/blog/">Blog</a>
+ </li>
+ <li >
+ <a href="/community.html">Community</a>
+ </li>
+ <li >
+ <a href="/faq.html">FAQ</a>
+ </li>
+ </ul>
+ </div><!--/.nav-collapse -->
+ </nav>
+
+<div class="row header">
+ <div class="col-lg-12">
+ <h2><a href="/blog">Apache Kudu (incubating) Blog</a></h2>
+ </div>
+</div>
+
+<div class="row-fluid">
+ <div class="col-lg-9">
+ <article>
+ <header>
+ <h1 class="entry-title">Master fault tolerance in Kudu 1.0</h1>
+ <p class="meta">Posted 24 Jun 2016 by Adar Dembo</p>
+ </header>
+ <div class="entry-content">
+ <p>This blog post describes how the 1.0 release of Apache Kudu (incubating) will
+support fault tolerance for the Kudu master, finally eliminating Kudu’s last
+single point of failure.</p>
+
+<!--more-->
+
+<p>As those of you who follow this blog know by now, replication is a signature
+feature in Kudu. Replication is used to provide fault tolerance for all loaded
+data. By implementing the Raft consensus protocol, Kudu guarantees that a tablet
+replicated <strong>2N+1</strong> times can tolerate up to <strong>N</strong> failures.</p>
+
+<p>What you may not know is that Kudu replicates its metadata, too. That is, the
+Kudu master stores all table and tablet metadata in a single “master” tablet.
+As a regular Kudu tablet itself, this master tablet may be replicated with
+Raft. As such, the Kudu master is a special kind of tablet server whose primary
+job is to host a single replica of the master tablet.</p>
+
+<p>When we launched Kudu’s first beta, support for replicated masters had been
+implemented but was too fragile to be anything but experimental. One of our
+goals for Kudu’s 1.0 release is to improve replicated master support so that it
+can be safely enabled in production clusters.</p>
+
+<h1 id="how-master-replication-works">How master replication works</h1>
+
+<p>To use replicated masters, a Kudu operator must deploy some number of Kudu
+masters, providing the hostname and port number of each master in the group via
+the <code>--master_address</code> command line option. For example, each master in a
+three-node deployment should be started with
+<code>--master_address=<host1:port1>,<host2:port2><host3:port3></code>. In Raft parlance,
+this group of masters is known as a <em>Raft configuration</em>.</p>
+
+<p>At startup, a Raft configuration of masters will hold a leader election and
+elect one master as the leader. The leader master is responsible for servicing
+both tablet server heartbeats as well as client requests. The remaining masters
+are followers: they participate in Raft consensus and replicate writes sent by
+the leader, but are otherwise idle. Any client requests they receive are
+rejected. Likewise, all tablet server heartbeats they receive are ignored. If
+the leader master ever dies or steps down, the remaining replicas hold an
+election to determine the new leader.</p>
+
+<p>All persistent master metadata is stored in the single replicated “master”
+tablet. Every row in this tablet represents either a table or a tablet. Table
+records include unique table identifiers, the table’s schema, and other bits of
+information. Tablet records include a unique identifier, the tablet’s Raft
+configuration, and other information.</p>
+
+<p>What master metadata is replicated?</p>
+
+<ol>
+ <li>Table and tablet existence, via <strong>CreateTable()</strong> and <strong>DeleteTable()</strong>.
+Every new tablet record also includes an initial Raft configuration.</li>
+ <li>Schema changes, via <strong>AlterTable()</strong> and tablet server heartbeats.</li>
+ <li>Tablet server Raft configuration changes, via tablet server heartbeats.
+These include both the list of Raft peers (may have changed due to
+under-replication) as well as the current leader (may have changed due to
+an election).</li>
+</ol>
+
+<p>Scanning the master tablet to service every heartbeat or client request would be
+slow, so the leader master caches all master metadata in memory. The caches are
+only updated after a metadata change is successfully replicated; in this way
+they are always consistent with the on-disk tablet. When a new leader master is
+elected, it scans the entire master tablet and uses the metadata to rebuild its
+in-memory caches.</p>
+
+<h1 id="communication-with-replicated-masters">Communication with replicated masters</h1>
+
+<p>All tablet servers start up with location information for the entire master Raft
+configuration and will periodically heartbeat to every master. Similarly,
+clients are also configured with the locations of all masters. Unlike tablet
+servers, they always communicate with the leader master as follower masters will
+reject client requests. To do this, clients must determine which master is the
+leader before sending the first request as well as whenever any request fails
+with a <code>NOT_THE_LEADER</code> error.</p>
+
+<h1 id="remaining-work-for-kudu-10">Remaining work for Kudu 1.0</h1>
+
+<p><a href="https://issues.apache.org/jira/browse/KUDU-422">KUDU-422</a> tracks the remaining
+master replication work. The guts of this feature have been implemented as far
+back as early 2015; the remaining work has been focused on fixing bugs that
+manifest only under specific conditions. For example, we’ve observed failures in
+DDL operations (e.g. <strong>CreateTable()</strong>) that only materialize upon the
+completion of a master leader election. These failures highlight some of the
+gaps in our testing regimen: we need a robust stress test that repeatedly
+performs such operations while holding master leader elections.</p>
+
+<p>That said, there is one remaining work item of larger scope: there’s no
+mechanism with which to perform a Raft configuration change for replicated
+masters. Such a mechanism would have multiple uses:</p>
+
+<ol>
+ <li>Migrating from a single-node master deployment to a fully replicated
+three-node (or five-node) deployment.</li>
+ <li>Replacing a failed master with a new one.</li>
+</ol>
+
+<p>This is being tracked by
+<a href="https://issues.apache.org/jira/browse/KUDU-1474">KUDU-1474</a>, and there’s been
+<a href="http://gerrit.cloudera.org:8080/3393">some discussion</a> around a design, but
+nothing has been implemented yet. Stay tuned!</p>
+
+ </div>
+</article>
+
+
+ </div>
+ <div class="col-lg-3 recent-posts">
+ <h3>Recent posts</h3>
+ <ul>
+
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
+ <li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
+
+ <li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
+
+ <li> <a href="/2016/06/13/weekly-update.html">Apache Kudu (incubating) Weekly Update June 13, 2016</a> </li>
+
+ <li> <a href="/2016/06/10/apache-kudu-0-9-0-released.html">Apache Kudu (incubating) 0.9.0 released</a> </li>
+
+ <li> <a href="/2016/06/06/weekly-update.html">Apache Kudu (incubating) Weekly Update June 6, 2016</a> </li>
+
+ <li> <a href="/2016/06/02/no-default-partitioning.html">Default Partitioning Changes Coming in Kudu 0.9</a> </li>
+
+ <li> <a href="/2016/06/01/weekly-update.html">Apache Kudu (incubating) Weekly Update June 1, 2016</a> </li>
+
+ <li> <a href="/2016/05/23/weekly-update.html">Apache Kudu (incubating) Weekly Update May 23, 2016</a> </li>
+
+ <li> <a href="/2016/05/16/weekly-update.html">Apache Kudu (incubating) Weekly Update May 16, 2016</a> </li>
+
+ <li> <a href="/2016/05/09/weekly-update.html">Apache Kudu (incubating) Weekly Update May 9, 2016</a> </li>
+
+ <li> <a href="/2016/05/03/weekly-update.html">Apache Kudu (incubating) Weekly Update May 3, 2016</a> </li>
+
+ <li> <a href="/2016/04/26/ycsb.html">Benchmarking and Improving Kudu Insert Performance with YCSB</a> </li>
+
+ <li> <a href="/2016/04/25/weekly-update.html">Apache Kudu (incubating) Weekly Update April 25, 2016</a> </li>
+
+ <li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
+
+ </ul>
+ </div>
+</div>
+
+ <footer class="footer">
+ <p class="pull-left">
+ <a href="http://incubator.apache.org"><img src="/img/apache-incubator.png" width="225" height="53" align="right"/></a>
+ </p>
+ <p class="small">
+ Apache Kudu (incubating) is an effort undergoing incubation at the Apache Software
+ Foundation (ASF), sponsored by the Apache Incubator PMC. Incubation is
+ required of all newly accepted projects until a further review
+ indicates that the infrastructure, communications, and decision making
+ process have stabilized in a manner consistent with other successful
+ ASF projects. While incubation status is not necessarily a reflection
+ of the completeness or stability of the code, it does indicate that the
+ project has yet to be fully endorsed by the ASF.
+
+ Copyright © 2016 The Apache Software Foundation.
+ </p>
+ </footer>
+ </div>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
+ <script src="/js/bootstrap.js"></script>
+ <script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','//www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-68448017-1', 'auto');
+ ga('send', 'pageview');
+
+ </script>
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/anchor-js/3.1.0/anchor.js"></script>
+ <script>
+ anchors.options = {
+ placement: 'right',
+ visible: 'touch',
+ };
+ anchors.add();
+ </script>
+ </body>
+</html>
+
diff --git a/blog/index.html b/blog/index.html
index 12b26ec..e99c55b 100644
--- a/blog/index.html
+++ b/blog/index.html
@@ -89,6 +89,28 @@
<!-- Articles -->
<article>
<header>
+ <h1 class="entry-title"><a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a></h1>
+ <p class="meta">Posted 24 Jun 2016 by Adar Dembo</p>
+ </header>
+ <div class="entry-content">
+
+ <p>This blog post describes how the 1.0 release of Apache Kudu (incubating) will
+support fault tolerance for the Kudu master, finally eliminating Kudu’s last
+single point of failure.</p>
+
+
+
+ </div>
+ <div class="read-full">
+ <a class="btn btn-info" href="/2016/06/24/multi-master-1-0-0.html">Read full post...</a>
+ </div>
+</article>
+
+
+
+<!-- Articles -->
+<article>
+ <header>
<h1 class="entry-title"><a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a></h1>
<p class="meta">Posted 21 Jun 2016 by Jean-Daniel Cryans</p>
</header>
@@ -97,34 +119,6 @@
<p>Welcome to the fourteenth edition of the Kudu Weekly Update. This weekly blog post
covers ongoing development and news in the Apache Kudu (incubating) project.</p>
-<h2 id="development-discussions-and-code-in-progress">Development discussions and code in progress</h2>
-
-<ul>
- <li>
- <p>Dan Burkert posted a series of patches to <a href="https://gerrit.cloudera.org/#/c/3388/">add support in the Java client</a>
-for non-covering range partitions. At the same time he improved how that client locates tables by
-leveraging the tablets cache.</p>
- </li>
- <li>
- <p>In the context of making multi-master reliable in 1.0, Adar Dembo posted a <a href="https://gerrit.cloudera.org/#/c/3393/">design document</a>
-on how to handle permanent master failures. Currently the master’s code is missing some features
-like <code>remote bootstrap</code> which makes it possible for a new replica to download a snapshot of the data
-from the leader replica.</p>
- </li>
- <li>
- <p>Tsuyoshi Ozawa refreshed <a href="https://gerrit.cloudera.org/#/c/2162/">a patch</a> posted in February that
-makes it easier to get started contributing to Kudu by providing a Dockerfile with the right
-environment.</p>
- </li>
-</ul>
-
-<h2 id="on-the-blog">On the blog</h2>
-
-<ul>
- <li>Mike Percy <a href="http://getkudu.io/2016/06/17/raft-consensus-single-node.html">wrote</a> about how Kudu
-uses Raft consensus on a single node, and some changes we’re making as Kudu is getting more mature.</li>
-</ul>
-
</div>
@@ -211,27 +205,6 @@
-<!-- Articles -->
-<article>
- <header>
- <h1 class="entry-title"><a href="/2016/06/06/weekly-update.html">Apache Kudu (incubating) Weekly Update June 6, 2016</a></h1>
- <p class="meta">Posted 06 Jun 2016 by Jean-Daniel Cryans</p>
- </header>
- <div class="entry-content">
-
- <p>Welcome to the twelfth edition of the Kudu Weekly Update. This weekly blog post
-covers ongoing development and news in the Apache Kudu (incubating) project.</p>
-
-
-
- </div>
- <div class="read-full">
- <a class="btn btn-info" href="/2016/06/06/weekly-update.html">Read full post...</a>
- </div>
-</article>
-
-
-
<!-- Pagination links -->
<nav>
@@ -250,6 +223,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -278,8 +253,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/blog/page/2/index.html b/blog/page/2/index.html
index 6073859..a732c9d 100644
--- a/blog/page/2/index.html
+++ b/blog/page/2/index.html
@@ -89,6 +89,27 @@
<!-- Articles -->
<article>
<header>
+ <h1 class="entry-title"><a href="/2016/06/06/weekly-update.html">Apache Kudu (incubating) Weekly Update June 6, 2016</a></h1>
+ <p class="meta">Posted 06 Jun 2016 by Jean-Daniel Cryans</p>
+ </header>
+ <div class="entry-content">
+
+ <p>Welcome to the twelfth edition of the Kudu Weekly Update. This weekly blog post
+covers ongoing development and news in the Apache Kudu (incubating) project.</p>
+
+
+
+ </div>
+ <div class="read-full">
+ <a class="btn btn-info" href="/2016/06/06/weekly-update.html">Read full post...</a>
+ </div>
+</article>
+
+
+
+<!-- Articles -->
+<article>
+ <header>
<h1 class="entry-title"><a href="/2016/06/02/no-default-partitioning.html">Default Partitioning Changes Coming in Kudu 0.9</a></h1>
<p class="meta">Posted 02 Jun 2016 by Dan Burkert</p>
</header>
@@ -172,27 +193,6 @@
-<!-- Articles -->
-<article>
- <header>
- <h1 class="entry-title"><a href="/2016/05/09/weekly-update.html">Apache Kudu (incubating) Weekly Update May 9, 2016</a></h1>
- <p class="meta">Posted 09 May 2016 by Jean-Daniel Cryans</p>
- </header>
- <div class="entry-content">
-
- <p>Welcome to the eighth edition of the Kudu Weekly Update. This weekly blog post
-covers ongoing development and news in the Apache Kudu (incubating) project.</p>
-
-
-
- </div>
- <div class="read-full">
- <a class="btn btn-info" href="/2016/05/09/weekly-update.html">Read full post...</a>
- </div>
-</article>
-
-
-
<!-- Pagination links -->
<nav>
@@ -213,6 +213,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -241,8 +243,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/blog/page/3/index.html b/blog/page/3/index.html
index 6f16baf..45e30fe 100644
--- a/blog/page/3/index.html
+++ b/blog/page/3/index.html
@@ -89,6 +89,27 @@
<!-- Articles -->
<article>
<header>
+ <h1 class="entry-title"><a href="/2016/05/09/weekly-update.html">Apache Kudu (incubating) Weekly Update May 9, 2016</a></h1>
+ <p class="meta">Posted 09 May 2016 by Jean-Daniel Cryans</p>
+ </header>
+ <div class="entry-content">
+
+ <p>Welcome to the eighth edition of the Kudu Weekly Update. This weekly blog post
+covers ongoing development and news in the Apache Kudu (incubating) project.</p>
+
+
+
+ </div>
+ <div class="read-full">
+ <a class="btn btn-info" href="/2016/05/09/weekly-update.html">Read full post...</a>
+ </div>
+</article>
+
+
+
+<!-- Articles -->
+<article>
+ <header>
<h1 class="entry-title"><a href="/2016/05/03/weekly-update.html">Apache Kudu (incubating) Weekly Update May 3, 2016</a></h1>
<p class="meta">Posted 03 May 2016 by Todd Lipcon</p>
</header>
@@ -171,27 +192,6 @@
-<!-- Articles -->
-<article>
- <header>
- <h1 class="entry-title"><a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a></h1>
- <p class="meta">Posted 18 Apr 2016 by Todd Lipcon</p>
- </header>
- <div class="entry-content">
-
- <p>Welcome to the fifth edition of the Kudu Weekly Update. This weekly blog post
-covers ongoing development and news in the Apache Kudu (incubating) project.</p>
-
-
-
- </div>
- <div class="read-full">
- <a class="btn btn-info" href="/2016/04/18/weekly-update.html">Read full post...</a>
- </div>
-</article>
-
-
-
<!-- Pagination links -->
<nav>
@@ -212,6 +212,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -240,8 +242,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/blog/page/4/index.html b/blog/page/4/index.html
index e7da898..9f7f115 100644
--- a/blog/page/4/index.html
+++ b/blog/page/4/index.html
@@ -89,6 +89,27 @@
<!-- Articles -->
<article>
<header>
+ <h1 class="entry-title"><a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a></h1>
+ <p class="meta">Posted 18 Apr 2016 by Todd Lipcon</p>
+ </header>
+ <div class="entry-content">
+
+ <p>Welcome to the fifth edition of the Kudu Weekly Update. This weekly blog post
+covers ongoing development and news in the Apache Kudu (incubating) project.</p>
+
+
+
+ </div>
+ <div class="read-full">
+ <a class="btn btn-info" href="/2016/04/18/weekly-update.html">Read full post...</a>
+ </div>
+</article>
+
+
+
+<!-- Articles -->
+<article>
+ <header>
<h1 class="entry-title"><a href="/2016/04/14/ingesting-json-apache-kudu-streamsets-data-collector.html">Ingesting JSON Data Into Apache Kudu with StreamSets Data Collector</a></h1>
<p class="meta">Posted 14 Apr 2016 by Pat Patterson</p>
</header>
@@ -189,28 +210,6 @@
-<!-- Articles -->
-<article>
- <header>
- <h1 class="entry-title"><a href="/2016/03/28/weekly-update.html">Apache Kudu (incubating) Weekly Update March 28, 2016</a></h1>
- <p class="meta">Posted 28 Mar 2016 by Todd Lipcon</p>
- </header>
- <div class="entry-content">
-
- <p>Welcome to the second edition of the Kudu Weekly Update. As with last week’s
-inaugural post, we’ll cover ongoing development and news in the Apache Kudu
-project on a weekly basis.</p>
-
-
-
- </div>
- <div class="read-full">
- <a class="btn btn-info" href="/2016/03/28/weekly-update.html">Read full post...</a>
- </div>
-</article>
-
-
-
<!-- Pagination links -->
<nav>
@@ -231,6 +230,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -259,8 +260,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/blog/page/5/index.html b/blog/page/5/index.html
index aefbb2b..c5d7545 100644
--- a/blog/page/5/index.html
+++ b/blog/page/5/index.html
@@ -89,6 +89,28 @@
<!-- Articles -->
<article>
<header>
+ <h1 class="entry-title"><a href="/2016/03/28/weekly-update.html">Apache Kudu (incubating) Weekly Update March 28, 2016</a></h1>
+ <p class="meta">Posted 28 Mar 2016 by Todd Lipcon</p>
+ </header>
+ <div class="entry-content">
+
+ <p>Welcome to the second edition of the Kudu Weekly Update. As with last week’s
+inaugural post, we’ll cover ongoing development and news in the Apache Kudu
+project on a weekly basis.</p>
+
+
+
+ </div>
+ <div class="read-full">
+ <a class="btn btn-info" href="/2016/03/28/weekly-update.html">Read full post...</a>
+ </div>
+</article>
+
+
+
+<!-- Articles -->
+<article>
+ <header>
<h1 class="entry-title"><a href="/2016/03/20/weekly-update.html">Apache Kudu (incubating) Weekly Update March 21, 2016</a></h1>
<p class="meta">Posted 20 Mar 2016 by Todd Lipcon</p>
</header>
@@ -179,6 +201,8 @@
<h3>Recent posts</h3>
<ul>
+ <li> <a href="/2016/06/24/multi-master-1-0-0.html">Master fault tolerance in Kudu 1.0</a> </li>
+
<li> <a href="/2016/06/21/weekly-update.html">Apache Kudu (incubating) Weekly Update June 21, 2016</a> </li>
<li> <a href="/2016/06/17/raft-consensus-single-node.html">Using Raft Consensus on a Single Node</a> </li>
@@ -207,8 +231,6 @@
<li> <a href="/2016/04/19/kudu-0-8-0-predicate-improvements.html">Predicate Improvements in Kudu 0.8</a> </li>
- <li> <a href="/2016/04/18/weekly-update.html">Apache Kudu (incubating) Weekly Update April 18, 2016</a> </li>
-
</ul>
</div>
</div>
diff --git a/css/kudu.css b/css/kudu.css
index 7d5b8b9..cad01f8 100644
--- a/css/kudu.css
+++ b/css/kudu.css
@@ -229,4 +229,68 @@
padding: 0.5em 0;
margin: 0;
border-bottom: 1px solid #eee;
-}
\ No newline at end of file
+}
+
+/* ------------------------------------------
+ Syntax highlighting for Jekyll
+ ------------------------------------- */
+.highlight { background: #ffffff; }
+.highlight .c { color: #999988; font-style: italic } /* Comment */
+.highlight .err { color: #a61717; background-color: #e3d2d2 } /* Error */
+.highlight .k { font-weight: bold } /* Keyword */
+.highlight .o { font-weight: bold } /* Operator */
+.highlight .cm { color: #999988; font-style: italic } /* Comment.Multiline */
+.highlight .cp { color: #999999; font-weight: bold } /* Comment.Preproc */
+.highlight .c1 { color: #999988; font-style: italic } /* Comment.Single */
+.highlight .cs { color: #999999; font-weight: bold; font-style: italic } /* Comment.Special */
+.highlight .gd { color: #000000; background-color: #ffdddd } /* Generic.Deleted */
+.highlight .gd .x { color: #000000; background-color: #ffaaaa } /* Generic.Deleted.Specific */
+.highlight .ge { font-style: italic } /* Generic.Emph */
+.highlight .gr { color: #aa0000 } /* Generic.Error */
+.highlight .gh { color: #999999 } /* Generic.Heading */
+.highlight .gi { color: #000000; background-color: #ddffdd } /* Generic.Inserted */
+.highlight .gi .x { color: #000000; background-color: #aaffaa } /* Generic.Inserted.Specific */
+.highlight .go { color: #888888 } /* Generic.Output */
+.highlight .gp { color: #555555 } /* Generic.Prompt */
+.highlight .gs { font-weight: bold } /* Generic.Strong */
+.highlight .gu { color: #aaaaaa } /* Generic.Subheading */
+.highlight .gt { color: #aa0000 } /* Generic.Traceback */
+.highlight .kc { font-weight: bold } /* Keyword.Constant */
+.highlight .kd { font-weight: bold } /* Keyword.Declaration */
+.highlight .kp { font-weight: bold } /* Keyword.Pseudo */
+.highlight .kr { font-weight: bold } /* Keyword.Reserved */
+.highlight .kt { color: #445588; font-weight: bold } /* Keyword.Type */
+.highlight .m { color: #009999 } /* Literal.Number */
+.highlight .s { color: #d14 } /* Literal.String */
+.highlight .na { color: #008080 } /* Name.Attribute */
+.highlight .nb { color: #0086B3 } /* Name.Builtin */
+.highlight .nc { color: #445588; font-weight: bold } /* Name.Class */
+.highlight .no { color: #008080 } /* Name.Constant */
+.highlight .ni { color: #800080 } /* Name.Entity */
+.highlight .ne { color: #990000; font-weight: bold } /* Name.Exception */
+.highlight .nf { color: #990000; font-weight: bold } /* Name.Function */
+.highlight .nn { color: #555555 } /* Name.Namespace */
+.highlight .nt { color: #000080 } /* Name.Tag */
+.highlight .nv { color: #008080 } /* Name.Variable */
+.highlight .ow { font-weight: bold } /* Operator.Word */
+.highlight .w { color: #bbbbbb } /* Text.Whitespace */
+.highlight .mf { color: #009999 } /* Literal.Number.Float */
+.highlight .mh { color: #009999 } /* Literal.Number.Hex */
+.highlight .mi { color: #009999 } /* Literal.Number.Integer */
+.highlight .mo { color: #009999 } /* Literal.Number.Oct */
+.highlight .sb { color: #d14 } /* Literal.String.Backtick */
+.highlight .sc { color: #d14 } /* Literal.String.Char */
+.highlight .sd { color: #d14 } /* Literal.String.Doc */
+.highlight .s2 { color: #d14 } /* Literal.String.Double */
+.highlight .se { color: #d14 } /* Literal.String.Escape */
+.highlight .sh { color: #d14 } /* Literal.String.Heredoc */
+.highlight .si { color: #d14 } /* Literal.String.Interpol */
+.highlight .sx { color: #d14 } /* Literal.String.Other */
+.highlight .sr { color: #009926 } /* Literal.String.Regex */
+.highlight .s1 { color: #d14 } /* Literal.String.Single */
+.highlight .ss { color: #990073 } /* Literal.String.Symbol */
+.highlight .bp { color: #999999 } /* Name.Builtin.Pseudo */
+.highlight .vc { color: #008080 } /* Name.Variable.Class */
+.highlight .vg { color: #008080 } /* Name.Variable.Global */
+.highlight .vi { color: #008080 } /* Name.Variable.Instance */
+.highlight .il { color: #009999 } /* Literal.Number.Integer.Long */
diff --git a/feed.xml b/feed.xml
index 724e9dd..50496a7 100644
--- a/feed.xml
+++ b/feed.xml
@@ -1,6 +1,109 @@
-<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom"><generator uri="http://jekyllrb.com" version="2.5.3">Jekyll</generator><link href="/feed.xml" rel="self" type="application/atom+xml" /><link href="/" rel="alternate" type="text/html" /><updated>2016-06-22T20:12:13-07:00</updated><id>/</id><entry><title>Apache Kudu (incubating) Weekly Update June 21, 2016</title><link href="/2016/06/21/weekly-update.html" rel="alternate" type="text/html" title="Apache Kudu (incubating) Weekly Update June 21, 2016" /><published>2016-06-21T00:00:00-07:00</published><updated>2016-06-21T00:00:00-07:00</updated><id>/2016/06/21/weekly-update</id><content type="html" xml:base="/2016/06/21/weekly-update.html"><p>Welcome to the fourteenth edition of the Kudu Weekly Update. This weekly blog post
+<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom"><generator uri="http://jekyllrb.com" version="2.5.3">Jekyll</generator><link href="/feed.xml" rel="self" type="application/atom+xml" /><link href="/" rel="alternate" type="text/html" /><updated>2016-06-24T08:38:41-07:00</updated><id>/</id><entry><title>Master fault tolerance in Kudu 1.0</title><link href="/2016/06/24/multi-master-1-0-0.html" rel="alternate" type="text/html" title="Master fault tolerance in Kudu 1.0" /><published>2016-06-24T00:00:00-07:00</published><updated>2016-06-24T00:00:00-07:00</updated><id>/2016/06/24/multi-master-1-0-0</id><content type="html" xml:base="/2016/06/24/multi-master-1-0-0.html"><p>This blog post describes how the 1.0 release of Apache Kudu (incubating) will
+support fault tolerance for the Kudu master, finally eliminating Kudu’s last
+single point of failure.</p>
+
+<!--more-->
+
+<p>As those of you who follow this blog know by now, replication is a signature
+feature in Kudu. Replication is used to provide fault tolerance for all loaded
+data. By implementing the Raft consensus protocol, Kudu guarantees that a tablet
+replicated <strong>2N+1</strong> times can tolerate up to <strong>N</strong> failures.</p>
+
+<p>What you may not know is that Kudu replicates its metadata, too. That is, the
+Kudu master stores all table and tablet metadata in a single “master” tablet.
+As a regular Kudu tablet itself, this master tablet may be replicated with
+Raft. As such, the Kudu master is a special kind of tablet server whose primary
+job is to host a single replica of the master tablet.</p>
+
+<p>When we launched Kudu’s first beta, support for replicated masters had been
+implemented but was too fragile to be anything but experimental. One of our
+goals for Kudu’s 1.0 release is to improve replicated master support so that it
+can be safely enabled in production clusters.</p>
+
+<h1 id="how-master-replication-works">How master replication works</h1>
+
+<p>To use replicated masters, a Kudu operator must deploy some number of Kudu
+masters, providing the hostname and port number of each master in the group via
+the <code>--master_address</code> command line option. For example, each master in a
+three-node deployment should be started with
+<code>--master_address=&lt;host1:port1&gt;,&lt;host2:port2&gt;&lt;host3:port3&gt;</code>. In Raft parlance,
+this group of masters is known as a <em>Raft configuration</em>.</p>
+
+<p>At startup, a Raft configuration of masters will hold a leader election and
+elect one master as the leader. The leader master is responsible for servicing
+both tablet server heartbeats as well as client requests. The remaining masters
+are followers: they participate in Raft consensus and replicate writes sent by
+the leader, but are otherwise idle. Any client requests they receive are
+rejected. Likewise, all tablet server heartbeats they receive are ignored. If
+the leader master ever dies or steps down, the remaining replicas hold an
+election to determine the new leader.</p>
+
+<p>All persistent master metadata is stored in the single replicated “master”
+tablet. Every row in this tablet represents either a table or a tablet. Table
+records include unique table identifiers, the table’s schema, and other bits of
+information. Tablet records include a unique identifier, the tablet’s Raft
+configuration, and other information.</p>
+
+<p>What master metadata is replicated?</p>
+
+<ol>
+ <li>Table and tablet existence, via <strong>CreateTable()</strong> and <strong>DeleteTable()</strong>.
+Every new tablet record also includes an initial Raft configuration.</li>
+ <li>Schema changes, via <strong>AlterTable()</strong> and tablet server heartbeats.</li>
+ <li>Tablet server Raft configuration changes, via tablet server heartbeats.
+These include both the list of Raft peers (may have changed due to
+under-replication) as well as the current leader (may have changed due to
+an election).</li>
+</ol>
+
+<p>Scanning the master tablet to service every heartbeat or client request would be
+slow, so the leader master caches all master metadata in memory. The caches are
+only updated after a metadata change is successfully replicated; in this way
+they are always consistent with the on-disk tablet. When a new leader master is
+elected, it scans the entire master tablet and uses the metadata to rebuild its
+in-memory caches.</p>
+
+<h1 id="communication-with-replicated-masters">Communication with replicated masters</h1>
+
+<p>All tablet servers start up with location information for the entire master Raft
+configuration and will periodically heartbeat to every master. Similarly,
+clients are also configured with the locations of all masters. Unlike tablet
+servers, they always communicate with the leader master as follower masters will
+reject client requests. To do this, clients must determine which master is the
+leader before sending the first request as well as whenever any request fails
+with a <code>NOT_THE_LEADER</code> error.</p>
+
+<h1 id="remaining-work-for-kudu-10">Remaining work for Kudu 1.0</h1>
+
+<p><a href="https://issues.apache.org/jira/browse/KUDU-422">KUDU-422</a> tracks the remaining
+master replication work. The guts of this feature have been implemented as far
+back as early 2015; the remaining work has been focused on fixing bugs that
+manifest only under specific conditions. For example, we’ve observed failures in
+DDL operations (e.g. <strong>CreateTable()</strong>) that only materialize upon the
+completion of a master leader election. These failures highlight some of the
+gaps in our testing regimen: we need a robust stress test that repeatedly
+performs such operations while holding master leader elections.</p>
+
+<p>That said, there is one remaining work item of larger scope: there’s no
+mechanism with which to perform a Raft configuration change for replicated
+masters. Such a mechanism would have multiple uses:</p>
+
+<ol>
+ <li>Migrating from a single-node master deployment to a fully replicated
+three-node (or five-node) deployment.</li>
+ <li>Replacing a failed master with a new one.</li>
+</ol>
+
+<p>This is being tracked by
+<a href="https://issues.apache.org/jira/browse/KUDU-1474">KUDU-1474</a>, and there’s been
+<a href="http://gerrit.cloudera.org:8080/3393">some discussion</a> around a design, but
+nothing has been implemented yet. Stay tuned!</p></content><author><name>Adar Dembo</name></author><summary>This blog post describes how the 1.0 release of Apache Kudu (incubating) will
+support fault tolerance for the Kudu master, finally eliminating Kudu&#8217;s last
+single point of failure.</summary></entry><entry><title>Apache Kudu (incubating) Weekly Update June 21, 2016</title><link href="/2016/06/21/weekly-update.html" rel="alternate" type="text/html" title="Apache Kudu (incubating) Weekly Update June 21, 2016" /><published>2016-06-21T00:00:00-07:00</published><updated>2016-06-21T00:00:00-07:00</updated><id>/2016/06/21/weekly-update</id><content type="html" xml:base="/2016/06/21/weekly-update.html"><p>Welcome to the fourteenth edition of the Kudu Weekly Update. This weekly blog post
covers ongoing development and news in the Apache Kudu (incubating) project.</p>
+<!--more-->
+
<h2 id="development-discussions-and-code-in-progress">Development discussions and code in progress</h2>
<ul>
@@ -29,41 +132,12 @@
uses Raft consensus on a single node, and some changes we’re making as Kudu is getting more mature.</li>
</ul>
-<!--more-->
-
<p>Want to learn more about a specific topic from this blog post? Shoot an email to the
<a href="&#109;&#097;&#105;&#108;&#116;&#111;:&#117;&#115;&#101;&#114;&#064;&#107;&#117;&#100;&#117;&#046;&#105;&#110;&#099;&#117;&#098;&#097;&#116;&#111;&#114;&#046;&#097;&#112;&#097;&#099;&#104;&#101;&#046;&#111;&#114;&#103;">kudu-user mailing list</a> or
tweet at <a href="https://twitter.com/ApacheKudu">@ApacheKudu</a>. Similarly, if you’re
aware of some Kudu news we missed, let us know so we can cover it in
a future post.</p></content><author><name>Jean-Daniel Cryans</name></author><summary>Welcome to the fourteenth edition of the Kudu Weekly Update. This weekly blog post
-covers ongoing development and news in the Apache Kudu (incubating) project.
-
-Development discussions and code in progress
-
-
-
- Dan Burkert posted a series of patches to add support in the Java client
-for non-covering range partitions. At the same time he improved how that client locates tables by
-leveraging the tablets cache.
-
-
- In the context of making multi-master reliable in 1.0, Adar Dembo posted a design document
-on how to handle permanent master failures. Currently the master&#8217;s code is missing some features
-like remote bootstrap which makes it possible for a new replica to download a snapshot of the data
-from the leader replica.
-
-
- Tsuyoshi Ozawa refreshed a patch posted in February that
-makes it easier to get started contributing to Kudu by providing a Dockerfile with the right
-environment.
-
-
-
-On the blog
-
-
- Mike Percy wrote about how Kudu
-uses Raft consensus on a single node, and some changes we&#8217;re making as Kudu is getting more mature.</summary></entry><entry><title>Using Raft Consensus on a Single Node</title><link href="/2016/06/17/raft-consensus-single-node.html" rel="alternate" type="text/html" title="Using Raft Consensus on a Single Node" /><published>2016-06-17T00:00:00-07:00</published><updated>2016-06-17T00:00:00-07:00</updated><id>/2016/06/17/raft-consensus-single-node</id><content type="html" xml:base="/2016/06/17/raft-consensus-single-node.html"><p>As Kudu marches toward its 1.0 release, which will include support for
+covers ongoing development and news in the Apache Kudu (incubating) project.</summary></entry><entry><title>Using Raft Consensus on a Single Node</title><link href="/2016/06/17/raft-consensus-single-node.html" rel="alternate" type="text/html" title="Using Raft Consensus on a Single Node" /><published>2016-06-17T00:00:00-07:00</published><updated>2016-06-17T00:00:00-07:00</updated><id>/2016/06/17/raft-consensus-single-node</id><content type="html" xml:base="/2016/06/17/raft-consensus-single-node.html"><p>As Kudu marches toward its 1.0 release, which will include support for
multi-master operation, we are working on removing old code that is no longer
needed. One such piece of code is called LocalConsensus. Once LocalConsensus is
removed, we will be using Raft consensus even on Kudu tables that have a
@@ -302,26 +376,22 @@
calling <code>KuduTableCreator:add_hash_partitions</code> with the columns to hash and the
number of buckets to use:</p>
-<p><code>cpp
-unique_ptr&lt;KuduTableCreator&gt; table_creator(my_client-&gt;NewTableCreator());
-Status create_status = table_creator-&gt;table_name("my-table")
- .schema(my_schema)
- .add_hash_partitions({ "key_column_a", "key_column_b" }, 16)
- .Create();
-if (!create_status.ok() { /* handle error */ }
-</code></p>
+<div class="highlight"><pre><code class="language-cpp" data-lang="cpp"><span class="n">unique_ptr</span><span class="o">&lt;</span><span class="n">KuduTableCreator</span><span class="o">&gt;</span> <span class="n">table_creator</span><span class="p">(</span><span class="n">my_client</span><span class="o">-&gt;</span><span class="n">NewTableCreator</span><span class="p">());</span>
+<span class="n">Status</span> <span class="n">create_status</span> <span class="o">=</span> <span class="n">table_creator</span><span class="o">-&gt;</span><span class="n">table_name</span><span class="p">(</span><span class="s">&quot;my-table&quot;</span><span class="p">)</span>
+ <span class="p">.</span><span class="n">schema</span><span class="p">(</span><span class="n">my_schema</span><span class="p">)</span>
+ <span class="p">.</span><span class="n">add_hash_partitions</span><span class="p">({</span> <span class="s">&quot;key_column_a&quot;</span><span class="p">,</span> <span class="s">&quot;key_column_b&quot;</span> <span class="p">},</span> <span class="mi">16</span><span class="p">)</span>
+ <span class="p">.</span><span class="n">Create</span><span class="p">();</span>
+<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">create_status</span><span class="p">.</span><span class="n">ok</span><span class="p">()</span> <span class="p">{</span> <span class="cm">/* handle error */</span> <span class="p">}</span></code></pre></div>
<h1 id="java-client">Java Client</h1>
<p>And similarly, in Java:</p>
-<p><code>java
-List&lt;String&gt; hashColumns = new ArrayList&lt;&gt;();
-hashColumns.add("key_column_a");
-hashColumn.add("key_column_b");
-CreateTableOptions options = new CreateTableOptions().addHashPartitions(hashColumns, 16);
-myClient.createTable("my-table", my_schema, options);
-</code></p>
+<div class="highlight"><pre><code class="language-java" data-lang="java"><span class="n">List</span><span class="o">&lt;</span><span class="n">String</span><span class="o">&gt;</span> <span class="n">hashColumns</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o">&lt;&gt;();</span>
+<span class="n">hashColumns</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="s">&quot;key_column_a&quot;</span><span class="o">);</span>
+<span class="n">hashColumn</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="s">&quot;key_column_b&quot;</span><span class="o">);</span>
+<span class="n">CreateTableOptions</span> <span class="n">options</span> <span class="o">=</span> <span class="k">new</span> <span class="nf">CreateTableOptions</span><span class="o">().</span><span class="na">addHashPartitions</span><span class="o">(</span><span class="n">hashColumns</span><span class="o">,</span> <span class="mi">16</span><span class="o">);</span>
+<span class="n">myClient</span><span class="o">.</span><span class="na">createTable</span><span class="o">(</span><span class="s">&quot;my-table&quot;</span><span class="o">,</span> <span class="n">my_schema</span><span class="o">,</span> <span class="n">options</span><span class="o">);</span></code></pre></div>
<p>In the examples above, if the hash partition configuration is omitted the create
table operation will fail with the error <code>Table partitioning must be specified
@@ -334,16 +404,14 @@
<p>When creating Kudu tables with Impala, the formerly optional <code>DISTRIBUTE BY</code>
clause is now required:</p>
-<p><code>SQL
-CREATE TABLE my_table (key_column_a STRING, key_column_b STRING, other_column STRING)
-DISTRIBUTE BY HASH (key_column_a, key_column_b) INTO 16 BUCKETS
-TBLPROPERTIES(
- 'storage_handler' = 'com.cloudera.kudu.hive.KuduStorageHandler',
- 'kudu.table_name' = 'my_table',
- 'kudu.master_addresses' = 'kudu-master.example.com:7051',
- 'kudu.key_columns' = 'key_column_a,key_column_b'
-);
-</code></p></content><author><name>Dan Burkert</name></author><summary>The upcoming Apache Kudu (incubating) 0.9 release is changing the default
+<div class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">my_table</span> <span class="p">(</span><span class="n">key_column_a</span> <span class="n">STRING</span><span class="p">,</span> <span class="n">key_column_b</span> <span class="n">STRING</span><span class="p">,</span> <span class="n">other_column</span> <span class="n">STRING</span><span class="p">)</span>
+<span class="n">DISTRIBUTE</span> <span class="k">BY</span> <span class="n">HASH</span> <span class="p">(</span><span class="n">key_column_a</span><span class="p">,</span> <span class="n">key_column_b</span><span class="p">)</span> <span class="k">INTO</span> <span class="mi">16</span> <span class="n">BUCKETS</span>
+<span class="n">TBLPROPERTIES</span><span class="p">(</span>
+ <span class="s1">&#39;storage_handler&#39;</span> <span class="o">=</span> <span class="s1">&#39;com.cloudera.kudu.hive.KuduStorageHandler&#39;</span><span class="p">,</span>
+ <span class="s1">&#39;kudu.table_name&#39;</span> <span class="o">=</span> <span class="s1">&#39;my_table&#39;</span><span class="p">,</span>
+ <span class="s1">&#39;kudu.master_addresses&#39;</span> <span class="o">=</span> <span class="s1">&#39;kudu-master.example.com:7051&#39;</span><span class="p">,</span>
+ <span class="s1">&#39;kudu.key_columns&#39;</span> <span class="o">=</span> <span class="s1">&#39;key_column_a,key_column_b&#39;</span>
+<span class="p">);</span></code></pre></div></content><author><name>Dan Burkert</name></author><summary>The upcoming Apache Kudu (incubating) 0.9 release is changing the default
partitioning configuration for new tables. This post will introduce the change,
explain the motivations, and show examples of how code can be updated to work
with the new release.</summary></entry><entry><title>Apache Kudu (incubating) Weekly Update June 1, 2016</title><link href="/2016/06/01/weekly-update.html" rel="alternate" type="text/html" title="Apache Kudu (incubating) Weekly Update June 1, 2016" /><published>2016-06-01T00:00:00-07:00</published><updated>2016-06-01T00:00:00-07:00</updated><id>/2016/06/01/weekly-update</id><content type="html" xml:base="/2016/06/01/weekly-update.html"><p>Welcome to the eleventh edition of the Kudu Weekly Update. This weekly blog post
@@ -550,56 +618,4 @@
<a href="http://www.meetup.com/Big-Data-Montreal/events/230879277/?eventId=230879277">Big Data Montreal</a>
meetup.</li>
</ul></content><author><name>Todd Lipcon</name></author><summary>Welcome to the ninth edition of the Kudu Weekly Update. This weekly blog post
-covers ongoing development and news in the Apache Kudu (incubating) project.</summary></entry><entry><title>Apache Kudu (incubating) Weekly Update May 9, 2016</title><link href="/2016/05/09/weekly-update.html" rel="alternate" type="text/html" title="Apache Kudu (incubating) Weekly Update May 9, 2016" /><published>2016-05-09T00:00:00-07:00</published><updated>2016-05-09T00:00:00-07:00</updated><id>/2016/05/09/weekly-update</id><content type="html" xml:base="/2016/05/09/weekly-update.html"><p>Welcome to the eighth edition of the Kudu Weekly Update. This weekly blog post
-covers ongoing development and news in the Apache Kudu (incubating) project.</p>
-
-<!--more-->
-
-<p>If you find this post useful, please let us know by emailing the
-<a href="&#109;&#097;&#105;&#108;&#116;&#111;:&#117;&#115;&#101;&#114;&#064;&#107;&#117;&#100;&#117;&#046;&#105;&#110;&#099;&#117;&#098;&#097;&#116;&#111;&#114;&#046;&#097;&#112;&#097;&#099;&#104;&#101;&#046;&#111;&#114;&#103;">kudu-user mailing list</a> or
-tweeting at <a href="https://twitter.com/ApacheKudu">@ApacheKudu</a>. Similarly, if you’re
-aware of some Kudu news we missed, let us know so we can cover it in
-a future post.</p>
-
-<h2 id="development-discussions-and-code-in-progress">Development discussions and code in progress</h2>
-
-<ul>
- <li>
- <p>Sameer Abhyankar posted a <a href="http://gerrit.cloudera.org:8080/#/c/2986/">patch</a>
-for <a href="https://issues.apache.org/jira/browse/KUDU-1363">KUDU-1363</a>
-that adds the ability to specify
-<a href="http://www.w3schools.com/sql/sql_in.asp">IN</a>-like predicates on column values.</p>
- </li>
- <li>
- <p>Chris George and Andy Grove have both been adding new features in Kudu’s
-Spark module such as methods to <a href="http://gerrit.cloudera.org:8080/#/c/2981/">create/delete tables</a>
-and <a href="http://gerrit.cloudera.org:8080/#/c/2992/">insert/update rows in the DataSource</a>.</p>
- </li>
- <li>
- <p>Todd Lipcon <a href="https://issues.apache.org/jira/browse/KUDU-1437">fixed a bug</a> in RLE
-encoding that was reported by Przemyslaw Maciolek. Thank you Przemyslaw for
-reporting it and providing an easy way to reproduce it!</p>
- </li>
- <li>
- <p>Adar Dembo is currently working on addressing the
-<a href="https://github.com/cloudera/kudu/blob/master/docs/design-docs/multi-master-1.0.md">issues with multi-master</a>
-and early last week he got <a href="http://gerrit.cloudera.org:8080/2879">a</a>
-<a href="http://gerrit.cloudera.org:8080/2928">few</a> <a href="http://gerrit.cloudera.org:8080/2891">patches</a>
-in that address some race conditions.</p>
- </li>
- <li>
- <p>Zhen Zhang got <a href="http://gerrit.cloudera.org:8080/#/c/2858/">a first contribution</a>
-in with a patch that adds statistics in the Java client. In 0.9.0 it will be
-possible to query the client to get things like the number of bytes written or
-how many write operations were sent.</p>
- </li>
-</ul>
-
-<h2 id="upcoming-talks-and-meetups">Upcoming talks and meetups</h2>
-
-<ul>
- <li>Dan Burkert and Mike Percy will present Kudu at the
-<a href="http://www.meetup.com/Vancouver-Spark/events/229692936/">Vancouver Spark Meetup</a> in Vancouver, BC,
-on May 10.</li>
-</ul></content><author><name>Jean-Daniel Cryans</name></author><summary>Welcome to the eighth edition of the Kudu Weekly Update. This weekly blog post
covers ongoing development and news in the Apache Kudu (incubating) project.</summary></entry></feed>
