blob: cc1c2483408367d79d237f4e1dffc827d4ac08bb [file] [log] [blame]
<!--
▄▄▄ ██▓███ ▄▄▄ ▄████▄ ██░ ██ ▓█████ ██▓ ▄████ ███▄ █ ██▓▄▄▄█████▓▓█████
▒████▄ ▓██░ ██▒▒████▄ ▒██▀ ▀█ ▓██░ ██▒▓█ ▀ ▓██▒ ██▒ ▀█▒ ██ ▀█ █ ▓██▒▓ ██▒ ▓▒▓█ ▀
▒██ ▀█▄ ▓██░ ██▓▒▒██ ▀█▄ ▒▓█ ▄ ▒██▀▀██░▒███ ▒██▒▒██░▄▄▄░▓██ ▀█ ██▒▒██▒▒ ▓██░ ▒░▒███
░██▄▄▄▄██ ▒██▄█▓▒ ▒░██▄▄▄▄██ ▒▓▓▄ ▄██▒░▓█ ░██ ▒▓█ ▄ ░██░░▓█ ██▓▓██▒ ▐▌██▒░██░░ ▓██▓ ░ ▒▓█ ▄
▓█ ▓██▒▒██▒ ░ ░ ▓█ ▓██▒▒ ▓███▀ ░░▓█▒░██▓░▒████▒ ░██░░▒▓███▀▒▒██░ ▓██░░██░ ▒██▒ ░ ░▒████▒
▒▒ ▓▒█░▒▓▒░ ░ ░ ▒▒ ▓▒█░░ ░▒ ▒ ░ ▒ ░░▒░▒░░ ▒░ ░ ░▓ ░▒ ▒ ░ ▒░ ▒ ▒ ░▓ ▒ ░░ ░░ ▒░ ░
▒ ▒▒ ░░▒ ░ ▒ ▒▒ ░ ░ ▒ ▒ ░▒░ ░ ░ ░ ░ ▒ ░ ░ ░ ░ ░░ ░ ▒░ ▒ ░ ░ ░ ░ ░
░ ▒ ░░ ░ ▒ ░ ░ ░░ ░ ░ ▒ ░░ ░ ░ ░ ░ ░ ▒ ░ ░ ░
░ ░ ░ ░░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░
-->
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!DOCTYPE html>
<html>
<head>
<link rel="canonical" href="https://ignite.apache.org/features/datagrid.html" />
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate" />
<meta http-equiv="Pragma" content="no-cache" />
<meta http-equiv="Expires" content="0" />
<meta name="description" content="Apache Ignite In-Memory Data Grid is a distributed transactional key-value store that stores data
in-memory and on-disk and supports key-value, SQL, and processing APIs"/>
<title>In-Memory Data Grid - Apache Ignite</title>
<link media="all" rel="stylesheet" href="/css/all.css?v=1514336027"/>
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.css" rel="stylesheet"/>
<link href='https://fonts.googleapis.com/css?family=Open+Sans:400,300,300italic,400italic,600,600italic,700,700italic,800,800italic' rel='stylesheet' type='text/css'/>
<!--#include virtual="/includes/sh.html" -->
</head>
<body>
<div id="wrapper">
<!--#include virtual="/includes/header.html" -->
<main id="main" role="main" class="container">
<section id="datagrid" class="page-section">
<h1 class="first">In-Memory Data Grid</h1>
<div class="col-sm-12 col-md-12 col-xs-12" style="padding:0 0 20px 0;">
<div class="col-sm-6 col-md-6 col-xs-12" style="padding-left:0; padding-right:0">
<p>
Ignite data grid is a <code>distributed, transactional key-value store</code>. Unlike other
in-memory data grids (IMDG), Ignite enables storing data both, in memory and on disk, and
therefore is able to store more data than can fit in physical memory.
On top of the standard key-value data operations, including ACID transactions and all the
JCache (JSR 107) functionality, Ignite also supports <nobr><code>SQL queries</code></nobr>
over its data with the support for <nobr><code>distributed joins</code></nobr>.
</p>
</div>
<div class="col-sm-4 col-md-6 col-xs-12" style="padding-right:0">
<img class="img-responsive" src="/images/data_grid.png" width="500px" style="float:right;"/>
</div>
</div>
<p>
The data grid has been built from the ground up to linearly scale to hundreds of nodes with strong
semantics for data locality and affinity data routing to reduce redundant data noise. It can be
viewed as a distributed partitioned hash map with every cluster node owning a portion of the
overall data. This way the more cluster nodes we add, the more data we can cache.
</p>
<p>
Ignite data grid is one of the fastest implementations of ACID transactions or
atomic data updates in distributed clusters today. We know it because we constantly benchmark it ourselves.
</p>
<p>
The data in the data grid can be stored either purely in memory, or can also be persisted to disk.
If <a href="/features/persistence.html" target="_blank">Ignite native persistence</a> is enabled,
then data and indexes will be persisted natively by Ignite on every cluster node. In this case,
memory only serves as a smaller caching layer of the overall persisted data set. All queries and transactions
span the whole data set stored on disk.
</p>
<p>
The in-memory data grid can also improve performance and scalability by integrating with existing 3rd party
databases, like RDBMS, NoSQL, or Hadoop-based storages. This approach does not require
rip-and-replace of the existing data, but has its limitations. For example, SQL or scan queries will only include the
results stored in memory, and not in the external database, since Ignite does not have any knowledge of the external data.
</p>
<p>
<a href="https://apacheignite.readme.io/docs/distributed-persistent-store" target="_blank">Read more</a>
about different types of persistence in Ignite.
</p>
<div class="videos">
<div class="page-heading">Videos:</div>
<ul class="page-list">
<li>
<i class="fa fa-lg fa-play-circle-o"></i>
<span class="video-title">
<a target="youtube" href="https://www.youtube.com/watch?v=pFbDWpOiMOU">Getting Started with Data Grid</a>
</span>
<span class="video-duration">03:49</span>
</li>
</ul>
</div>
<div class="code-examples">
<div class="page-heading">Code Examples:</div>
<!-- Nav tabs -->
<ul id="datagrid-examples" class="nav nav-tabs">
<li class="active"><a href="#datagrid-example-basic" role="tab" data-toggle="tab">Put and Get</a></li>
<li><a href="#datagrid-example-transactions" role="tab" data-toggle="tab">Transactions</a></li>
<li><a href="#datagrid-example-locks" role="tab" data-toggle="tab">Locks</a></li>
<li><a href="#datagrid-example-sqlquery" role="tab" data-toggle="tab">SQL Query</a></li>
<li><a href="#datagrid-example-sqljoin" role="tab" data-toggle="tab">SQL Join</a></li>
<li><a href="#datagrid-example-sqlaggregation" role="tab" data-toggle="tab">SQL Aggregation</a></li>
</ul>
<!-- Tab panes -->
<div class="tab-content">
<div role="tabpanel" class="tab-pane active" id="datagrid-example-basic">
<pre class="brush:java">
Ignite ignite = Ignition.ignite();
// Get an instance of named cache.
final IgniteCache&lt;Integer, String&gt; cache = ignite.cache("cacheName");
// Store keys in cache.
for (int i = 0; i < 10; i++)
cache.put(i, Integer.toString(i));
// Retrieve values from cache.
for (int i = 0; i < 10; i++)
System.out.println("Got [key=" + i + ", val=" + cache.get(i) + ']');
// Remove objects from cache.
for (int i = 0; i < 10; i++)
cache.remove(i);
// Atomic put-if-absent.
cache.putIfAbsent(1, "1");
// Atomic replace.
cache.replace(1, "1", "2");
</pre>
</div>
<div role="tabpanel" class="tab-pane" id="datagrid-example-transactions">
<pre class="brush:java">
Ignite ignite = Ignition.ignite();
// Clone every object we get from cache, so we can freely update it.
IgniteCache&lt;Integer, Account&gt; cache = ignite.cache("cacheName");
try (IgniteTx tx = Ignition.ignite().transactions().txStart()) {
Account acct = cache.get(acctId);
assert acct != null;
// Deposit $20 into account.
acct.setBalance(acct.getBalance() + 20);
// Store updated account in cache.
cache.put(acctId, acct);
tx.commit();
}
</pre>
</div>
<div role="tabpanel" class="tab-pane" id="datagrid-example-locks">
<pre class="brush:java">
Ignite ignite = Ignition.ignite();
// Get an instance of named cache.
final GridCache&lt;String, Integer&gt; cache = ignite.cache("cacheName");
// Lock cache key "Hello".
Lock lock = cache.lock("Hello");
lock.lock();
try {
cache.put("Hello", 11);
cache.put("World", 22);
}
finally {
lock.unlock();
}
</pre>
</div>
<div role="tabpanel" class="tab-pane" id="datagrid-example-sqlquery">
<pre class="brush:java">
IgniteCache&lt;Long, Person&gt; cache = ignite.cache("mycache");
SqlFieldsQuery sql = new SqlFieldsQuery(
"select concat(firstName, ' ', lastName) from Person");
// Select concatinated first and last name for all persons.
try (QueryCursor&lt;List&lt;?&gt;&gt; cursor = cache.query(sql)) {
for (List&lt;?&gt; row : cursor)
System.out.println("Full name: " + row.get(0));
}
</pre>
</div>
<div role="tabpanel" class="tab-pane" id="datagrid-example-sqljoin">
<pre class="brush:java">
IgniteCache&lt;Long, Person&gt; personCache = ignite.cache("personCache");
// Select with join between Person and Organization to
// get the names of all the employees of a specific organization.
SqlFieldsQuery sql = new SqlFieldsQuery(
"select p.name "
+ "from Person p, \"orgCache\".Organization o where "
+ "p.orgId = o.id "
+ "and o.name = ?");
// Execute the query and obtain the query result cursor.
try (QueryCursor&lt;List&lt;?&gt;&gt; cursor = personCache.query(sql.setArgs("Ignite"))) {
for (List&lt;?&gt; row : cursor)
System.out.println("Person name=" + row);
}
</pre>
</div>
<div role="tabpanel" class="tab-pane" id="datagrid-example-sqlaggregation">
<pre class="brush:java">
IgniteCache&lt;Long, Person&gt; personCache = ignite.cache("personCache");
// Select average age of people working within different departments.
SqlFieldsQuery sql = new SqlFieldsQuery(
"select avg(p.age) as avg_age, d.name as dpmt_name, o.name as org_name "
+ "from Person p, \"depCache\".Department d, \"orgCache\".Organization o "
+ "where p.depid = d.id and d.orgid = o.id "
+ "group by d.name, o.name "
+ "order by avg_age";
// Execute the query and obtain the query result cursor.
try (QueryCursor&lt;List&lt;?&gt;&gt; cursor = personCache.query(sql.setArgs("Ignite"))) {
for (List&lt;?&gt; row : cursor)
System.out.println("Average age by department and organization: " + row);
}
</pre>
</div>
</div>
</div>
<div class="page-heading github-examples">GitHub Examples:</div>
<p>
Also see <a href="https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples/datagrid" target="github">data grid examples</a>
available on GitHub.
</p>
</section>
<section id="key-features" class="page-section">
<h2>Data Grid Features</h2>
<table class="formatted" name="Data Grid Features">
<thead>
<tr>
<th width="35%" class="left">Feature</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="left">Key-Value Store</td>
<td>
<p>
Ignite data grid is a <code>key-value store</code> which can store data both, in-memory
and on-disk. It can be viewed as a distributed partitioned hash map, with every cluster
node owning a portion of the overall data. This way the more cluster nodes we add,
the more data we can store.
</p>
<p>
Unlike other key-value stores, Ignite determines data locality using a pluggable hashing
algorithm. Every client can determine which node a key belongs to by plugging it into a
hashing function, without a need for any special mapping servers or name nodes.
</p>
<div class="page-links">
<a href="http://apacheignite.readme.io/docs/data-grid" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">SQL Support</td>
<td>
<p>
In addition to key-value API, Ignite data grid has complete SQL support, including primary
and secondary indexes and distributed JOINs.
</p>
<div class="page-links">
<a href="https://apacheignite-sql.readme.io/docs/how-ignite-sql-works" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">Durable Memory</td>
<td>
<p>
Ignite <code>Durable Memory</code> allows storing and processing data and indexes
both, in memory and on disk. The in-memory data, including indexes, is always
stored and managed <code>off-heap</code>, completely removing any type of Garbage
Collection overhead.
</p>
<div class="page-links">
<a href="https://apacheignite.readme.io/docs/durable-memory" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">JCache (JSR 107)</td>
<td>
<p>
Ignite is a 100% compliant implementation of <span style="white-space: nowrap">JCache (JSR 107)</span> specification.
JCache provides a very simple to use, yet very powerful API for data caching.
</p>
<p>
Some of the JCache API features include:
<ul class="page-list">
<li>Basic Cache Operations</li>
<li>ConcurrentMap APIs</li>
<li>Collocated Processing (EntryProcessor)</li>
<li>Events and Metrics</li>
<li>Pluggable Persistence</li>
</ul>
</p>
<div class="page-links">
<a href="http://apacheignite.readme.io/docs/jcache" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">Partitioning & Replication</td>
<td>
<p>
Depending on the configuration, Ignite can either <i>partition</i> or <i>replicate</i>
data. Unlike <code>REPLICATED</code> mode, where data is fully replicated across
all nodes in the cluster, in <code>PARTITIONED</code> mode Ignite will equally split the data
across multiple cluster nodes.
</p>
<p>
Ignite also allows to configure multiple <b>backup copies</b> to guarantee data resiliency
in case of node failures.
</p>
<div class="page-links">
<a href="http://apacheignite.readme.io/docs/cache-modes" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">Collocated Processing</td>
<td>
<p>
Ignite allows executing any native Java, C++, and .NET/C# code directly on the server-side, close to the data, in collocated fashion.
</p>
<div class="page-links">
<a href="/collocatedprocessing.html" target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">Self-Healing Cluster</td>
<td>
<p>
Ignite cluster can self-heal, where clients automatically reconnect in case of failures,
slow clients are automatically kicked out, and data from failed nodes
is automatically propagated to other nodes in the grid.
</p>
<div class="page-links">
<a href="http://apacheignite.readme.io/docs/clients-vs-servers" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">Client-side Near Caches</td>
<td>
<p>
Near cache is local client-side cache that stores the most recently and most frequently accessed data.
</p>
<div class="page-links">
<a href="http://apacheignite.readme.io/docs/near-caches" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">ACID Transactions</td>
<td>
<p>
Ignite provides fully ACID compliant distributed transactions that ensure guaranteed consistency.</p>
<p>
Ignite supports <code>OPTIMISTIC</code> and <code>PESSIMISTIC</code> concurrency modes
as well as <code>READ_COMMITTED</code>, <code>REPEATABLE_READ</code>, and <code>SERIALIZABLE</code> isolation levels.</p>
<p>
Ignite transactions utilize 2PC protocol with many one-phase-commit optimizations whenever applicable.
</p>
<div class="page-links">
<a href="http://apacheignite.readme.io/docs/transactions" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left" width="35%">Deadlock-Free Transactions</td>
<td>
<p>
Ignite supports deadlock-free, optimistic transactions, which do not acquire any locks, and free users from worrying about the lock order. Such transactions also provide much better performance.
</p>
<div class="page-links">
<a href="https://apacheignite.readme.io/docs/transactions" target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left" width="35%">Transactional Entry Processor</td>
<td>
<p>
Ignite transactional entry processor allows executing collocated user logic on the server side within a transaction.
</p>
<div class="page-links">
<a href="https://apacheignite.readme.io/docs/affinity-collocation#ignitecompute-vs-entryprocessor" target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left" width="35%">Cross-Partition Transactions</td>
<td>
<p>
In Ignite, transactions can be performed on all partitions of a cache across the whole cluster.
</p>
<div class="page-links">
<a href="https://apacheignite.readme.io/docs/transactions#ignitetransactions" target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left" width="35%">Locks</td>
<td>
<p>
Ignite allows developers to define explicit locks enforcing mutual exclusion on cached objects.
</p>
<div class="page-links">
<a href="https://apacheignite.readme.io/docs/distributed-locks" target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left" width="35%">Continuous Queries</td>
<td>
<p>
Continuous queries are useful for cases when you want to execute a query and then continue to get notified about the data changes that fall into your query filter.
</p>
<div class="page-links">
<a href="https://apacheignite.readme.io/docs/continuous-queries" target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">Native Persistence</td>
<td>
<p>
Ignite <code>Native Persistence</code> is a distributed ACID and SQL-compliant disk
store that transparently integrates with Ignite's Durable Memory as an optional
persistence layer, storing data and indexes on SSD, Flash, 3D XPoint, or any other
types of storages.
</p>
<div class="page-links">
<a href="https://apacheignite.readme.io/docs/distributed-persistent-store" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">
<b>Database Integration</b>
</td>
<td>
<p>
Ignite can automatically integrate with external databases, like RDBMS, NoSQL, HDFS and others.
</p>
<div class="page-links">
<a href="https://apacheignite.readme.io/docs/3rd-party-store" target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left" width="35%">Write-Through</td>
<td>
<p>
Write-Through mode allows updating the data in the database.
</p>
<div class="page-links">
<a href="https://apacheignite.readme.io/docs/3rd-party-store#read-through-and-write-through" target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">Read-Through</td>
<td>
<p>
Read-Through mode allows reading the data from the database.
</p>
<div class="page-links">
<a href="https://apacheignite.readme.io/docs/3rd-party-store#read-through-and-write-through" target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">Write-Behind Caching</td>
<td>
<p>
Ignite provides an option to asynchronously perform updates to the database via Write-Behind Caching.
</p>
<div class="page-links">
<a href="https://apacheignite.readme.io/docs/3rd-party-store#section-write-behind-caching" target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">Web Session Clustering</td>
<td>
<p>
Ignite data grid is capable of caching web sessions of all Java Servlet
containers that follow Java Servlet 3.0 Specification, including Apache Tomcat,
Eclipse Jetty, Oracle WebLogic, and others.
</p>
<p>
Web sessions caching becomes useful when running a cluster of app servers to improve
performance and scalability of the servlet container.
</p>
<div class="page-links">
<a href="https://apacheignite-mix.readme.io/docs/web-session-clustering" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">Hibernate L2 Caching</td>
<td>
<p>
Ignite data grid can be used as <code>Hibernate Second-Level Cache</code> (or L2 cache),
which can significantly speed-up the persistence layer of your application.
</p>
<div class="page-links">
<a href="https://apacheignite-mix.readme.io/docs/hibernate-l2-cache" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">Spring Caching</td>
<td>
<p>
Ignite provides Spring-annotation-based way to enable caching for Java methods so that
the result of a method execution is stored in the Ignite cache. If later the same
method is called with the same set of parameters, the result will be retrieved from
the cache instead of actually executing the method.
</p>
<div class="page-links">
<a href="http://apacheignite-mix.readme.io/docs/spring-caching" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">Spring Data</td>
<td>
<p>
Apache Ignite implements Spring Data <code>CrudRepository</code> interface that not only supports basic CRUD operations but also provides access to the Apache Ignite SQL capabilities via the unified Spring Data API.
</p>
<div class="page-links">
<a href="https://apacheignite-mix.readme.io/docs/spring-data" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">XA/JTA</td>
<td>
<p>
Ignite can be configured with a Java Transaction API (JTA) transaction manager lookup class.
</p>
<div class="page-links">
<a href="https://apacheignite.readme.io/docs/transactions#integration-with-jta" target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">OSGI Support</td>
<td>
<p></p> &nbsp;
<div class="page-links">
<a href="https://apacheignite-mix.readme.io/docs/starting-inside-an-osgi-container" target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
<tr>
<td class="left">Binary Protocol</td>
<td>
<p>
Apache Ignite stores data in caches as <code>BinaryObjects</code> that allows you to:
</p>
<ul class="page-list">
<li>Read a serialized object's field without full object deserialization.</li>
<li>Dynamically change an object's structure.</li>
<li>Dynamically create an object.</li>
</ul>
<div class="page-links">
<a href="https://apacheignite.readme.io/docs/binary-marshaller" target="docs">Docs for this feature <i class="fa fa-angle-double-right"></i></a>
</div>
</td>
</tr>
</tbody>
</table>
</section>
</main>
<!--#include virtual="/includes/footer.html" -->
</div>
<!--#include virtual="/includes/scripts.html" -->
</body>
</html>