_posts/2022-04-28-ignite.markdown

---
layout: post
title: 'Apache Ignite 2.13.0: new Apache Calcite-based SQL engine'
date: '2022-04-28T00:00:00+00:00'
categories: ignite
---
<p>As of April 26, 2022, <a href="https://ignite.apache.org/">Apache Ignite</a> 2.13 has been released. You can directly check the full list of resolved <a href="https://s.apache.org/x8u49">Important JIRA tasks</a> but here let&#39;s briefly overview some valuable improvements.</p>
<h4>This is a breaking change release: The legacy service grid implementation was removed.</h4>
<h3 id="new-apache-calcite-based-sql-engine">New Apache Calcite-based SQL engine</h3>
<p>We&#39;ve implemented a new experimental SQL engine based on Apache Calcite. Now it&#39;s possible to:</p>
<ul>
<li>Get rid of some <a href="https://cwiki.apache.org/confluence/display/IGNITE/IEP-37%3A+New+query+execution+engine#IEP37:Newqueryexecutionengine-Motivation">H2 limitations</a>;</li>
<li><a href="https://cwiki.apache.org/confluence/display/IGNITE/IEP-37%3A+New+query+execution+engine#IEP37:Newqueryexecutionengine-Implementationdetails">Optimize</a> some query execution.</li>
</ul>
<p>The current H2-based engine has fundamental limitations. For example:</p>
<ul>
<li>some queries should be splitted into 2 phases (map subquery and reduce subquery), but some of them cannot be effectively executed in 2 phases.</li>
<li>H2 is a third-party database product with not-ASF license.</li>
<li>The optimizer and other internal things are not supposed to work in a distributed environment.</li>
<li>It&#39;s hard to make Ignite-specific changes to the H2 code, patches are often declined.</li>
</ul>
<p>The Apache Calcite is a SQL engine with customizable modules. Requests can be splitted into more than 2 phases.</p>
<p>A query engine can be set before query execution. Here is an example for SQL:</p>
<pre><code class="lang-java"><span class="hljs-keyword">SELECT</span> <span class="hljs-comment">/*+ QUERY_ENGINE('h2') */</span> fld <span class="hljs-keyword">FROM</span> <span class="hljs-keyword">table</span>;
or
<span class="hljs-keyword">SELECT</span> <span class="hljs-comment">/*+ QUERY_ENGINE('calcite') */</span> fld <span class="hljs-keyword">FROM</span> <span class="hljs-keyword">table</span>;
</code></pre>
<p>See JDBC and ODBC examples <a href="https://github.com/apache/ignite/blob/master/modules/calcite/README.txt">here</a>.</p>
<p>The new engine requires the ignite-indexing module (which depends on H2) to be included to the classpath to support queries infrastructure.</p>
<p>See more technical details about the new engine in the <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=130028084">IEP-37</a>.</p>
<h3 id="read-repair-strategies">Read Repair strategies</h3>
<p><a href="https://ignite.apache.org/docs/latest/read-repair">&quot;Read Repair&quot;</a> refers to a technique of repairing inconsistencies between primary and backup copies of data during normal read operations. When a specific key (or keys) is read by a user operation, Ignite checks the values for the given key in all backup copies.</p>
<p>We&#39;ve implemented the new Read Repair strategies as follows:</p>
<ul>
<li>LWW (Last Write Wins) - Last write (the newest entry) wins.</li>
<li>PRIMARY - Value from the primary node wins.</li>
<li>RELATIVE_MAJORITY - The relative majority: a value found more often than any other wins.</li>
<li>REMOVE - Inconsistent entries will be removed.</li>
<li>CHECK_ONLY - Only check will be performed.</li>
</ul>
<h3 id="array-type-in-binary-object">Array type in Binary Object</h3>
<p>In previous versions Ignite did not save information about array type. Now it can be stored in a binary object:</p>
<pre><code class="lang-java">cache.put(1, new <span class="hljs-keyword">Person</span><span class="hljs-comment">[]</span> {new <span class="hljs-keyword">Person</span>(1), new <span class="hljs-keyword">Person</span>(2)});

<span class="hljs-keyword">Person</span><span class="hljs-comment">[]</span> obj = cache.get(1);

assertEquals(<span class="hljs-keyword">Person</span><span class="hljs-comment">[]</span>.class, obj.getClass());
</code></pre>
<p>The feature is disabled by default due to compatibility issues. Set the IGNITE_USE_BINARY_ARRAYS system property to true to enable it.</p>
<h3 id="cdc-for-in-memory-caches">CDC for in-memory caches</h3>
<p>The <a href="https://ignite.apache.org/docs/latest/persistence/change-data-capture">Change Data Capture</a> now can be configured for in-memory caches. From now on, only CDC needed records for such caches will be logged to WAL.</p>
<h3 id="other-improvements-and-changes">Other improvements and changes</h3>
<ul>
<li>The C++ thin client implemented continuous queries and asynchronous network events handling. See the updated thin clients features list <a href="https://cwiki.apache.org/confluence/display/IGNITE/Thin+clients+features">here</a>;</li>
<li>Implemented NUMA-aware <a href="https://github.com/apache/ignite/tree/master/modules/numa-allocator">allocator</a> for data regions;</li>
<li>Ignite maven BOM;</li>
<li>Removed the legacy service grid implementation;</li>
<li>100+ small improvements and bug fixes.</li>
</ul>
<h3 id="anything-else-">Anything else?</h3>
<p>See the <a href="https://ignite.apache.org/releases/2.13.0/release_notes.html">release notes</a> to learn about all of the new features and improvements.</p>
<p>Reach out to us on the community user list for more questions, details, and feedback.</p>
<p>Sincerely yours,
Ignite contributors and committers</p>