| <!-- |
| ▄▄▄ ██▓███ ▄▄▄ ▄████▄ ██░ ██ ▓█████ ██▓ ▄████ ███▄ █ ██▓▄▄▄█████▓▓█████ |
| ▒████▄ ▓██░ ██▒▒████▄ ▒██▀ ▀█ ▓██░ ██▒▓█ ▀ ▓██▒ ██▒ ▀█▒ ██ ▀█ █ ▓██▒▓ ██▒ ▓▒▓█ ▀ |
| ▒██ ▀█▄ ▓██░ ██▓▒▒██ ▀█▄ ▒▓█ ▄ ▒██▀▀██░▒███ ▒██▒▒██░▄▄▄░▓██ ▀█ ██▒▒██▒▒ ▓██░ ▒░▒███ |
| ░██▄▄▄▄██ ▒██▄█▓▒ ▒░██▄▄▄▄██ ▒▓▓▄ ▄██▒░▓█ ░██ ▒▓█ ▄ ░██░░▓█ ██▓▓██▒ ▐▌██▒░██░░ ▓██▓ ░ ▒▓█ ▄ |
| ▓█ ▓██▒▒██▒ ░ ░ ▓█ ▓██▒▒ ▓███▀ ░░▓█▒░██▓░▒████▒ ░██░░▒▓███▀▒▒██░ ▓██░░██░ ▒██▒ ░ ░▒████▒ |
| ▒▒ ▓▒█░▒▓▒░ ░ ░ ▒▒ ▓▒█░░ ░▒ ▒ ░ ▒ ░░▒░▒░░ ▒░ ░ ░▓ ░▒ ▒ ░ ▒░ ▒ ▒ ░▓ ▒ ░░ ░░ ▒░ ░ |
| ▒ ▒▒ ░░▒ ░ ▒ ▒▒ ░ ░ ▒ ▒ ░▒░ ░ ░ ░ ░ ▒ ░ ░ ░ ░ ░░ ░ ▒░ ▒ ░ ░ ░ ░ ░ |
| ░ ▒ ░░ ░ ▒ ░ ░ ░░ ░ ░ ▒ ░░ ░ ░ ░ ░ ░ ▒ ░ ░ ░ |
| ░ ░ ░ ░░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ |
| --> |
| |
| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| --> |
| |
| <!DOCTYPE html> |
| <html> |
| <head> |
| <link rel="canonical" href="https://ignite.apache.org/arch/memorycentric.html" /> |
| <meta charset="utf-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| <meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate" /> |
| <meta http-equiv="Pragma" content="no-cache" /> |
| <meta http-equiv="Expires" content="0" /> |
| <title>Memory Centric - Apache Ignite</title> |
| <link media="all" rel="stylesheet" href="/css/all.css?v=1538416900"> |
| <link href="https://netdna.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.css" rel="stylesheet"> |
| <link href='https://fonts.googleapis.com/css?family=Open+Sans:400,300,300italic,400italic,600,600italic,700,700italic,800,800italic' rel='stylesheet' type='text/css'> |
| |
| <!--#include virtual="/includes/sh.html" --> |
| </head> |
| <body> |
| <div id="wrapper"> |
| <!--#include virtual="/includes/header.html" --> |
| |
| <main id="main" role="main" class="container"> |
| <section id="memory-centric" class="page-section"> |
| <h1 class="first">Memory-Centric Storage</h1> |
| <div class="col-sm-12 col-md-12 col-xs-12" style="padding:0 0 20px 0;"> |
| <div class="col-sm-6 col-md-6 col-xs-12" style="padding-left:0; padding-right:0"> |
| <p> |
| Apache Ignite is based on distributed <i>memory-centric architecture</i> that combines the |
| performance and scale of in-memory computing together with the disk durability and strong |
| consistency in one system. |
| </p> |
| <p> |
| When native persistence is turned on, Ignite functions as a <nobr><i>memory-centric system-of-record</i></nobr>, |
| where most of the processing happens in memory on cached data, but the superset of data and indexes gets persisted to disk. |
| </p> |
| </div> |
| <div class="col-sm-6 col-md-6 col-xs-12" style="padding-right:0"> |
| <img class="img-responsive" src="/images/durable_memory.png" width="440px" style="float: right;"/> |
| </div> |
| </div> |
| <p> |
| When persistence is turned off, Ignite functions as a memory-only store, in which case it can be treated as a Distributed Cache, |
| In-Memory Database (IMDB) or In-Memory Data Grid (IMDG). |
| </p> |
| |
| <div class="page-heading" id="db-and-caching-in-one"> |
| Database and Caching in One |
| <a href="/arch/memorycentric.html#db-and-caching-in-one"><i class="fa fa-anchor"></i></a> |
| </div> |
| <p> |
| One of the main advantages of Ignite is that it comes with a distributed in-memory cache and a |
| distributed on-disk storage in one platform. In other words, Ignite users get both, a |
| distributed cache and a distributed database together. |
| </p> |
| <p> |
| In partitioned (not replicated) mode, the data is partitioned across multiple servers, with each server responsible |
| for a subset of the data. Collectively, the full data set is stored across all servers. Each server |
| has its subset persisted on disk. Depending on how much memory is available, |
| each server also has either the whole subset or a portion of it cached in memory. Such combination |
| of memory and disk creates a distributed memory-centric storage. |
| </p> |
| <p> |
| The following <i>memory and disk usage modes</i> are supported: |
| </p> |
| <table class="formatted" name="Deployment Options Features"> |
| <thead> |
| <tr> |
| <th width="35%" class="left">Mode</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td class="left">In-Memory</td> |
| <td> |
| <p> |
| The whole data set is stored in memory. In order to survive |
| node failures, it is recommended to configure a number of redundant backup |
| copies (aka. replication factor) across the cluster. |
| </p> |
| |
| <p> |
| <strong>Use cases</strong>: in-memory caches, in-memory data grids, in-memory computations, |
| web-session caching, real-time processing of continuous data streams. |
| </p> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">In-Memory + 3rd party database</td> |
| <td> |
| <p> |
| Ignite can be used as a caching layer (aka. data grid) over any existing 3rd party |
| database. This mode is used to accelerate and scale the underlying database. |
| Automatic integration is provided with most known databases, like Oracle, MySQL, |
| PostgreSQL, Apache Cassandra, etc. |
| </p> |
| |
| <p> |
| <strong>Use cases</strong>: Ignite as <a href="/features/datagrid.html">In-Memory Data Grid</a> |
| - adds acceleration and scale to existing database deployments (RDBMS, NoSQL, etc). |
| </p> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">In-Memory + Full Copy on Disk</td> |
| <td> |
| <p> |
| The whole data set is stored in memory and on disk. The disk is used as a memory-offload for data recovery |
| purposes, in case of full cluster crashes and restarts. |
| </p> |
| |
| <p> |
| <strong>Use cases</strong>: Ignite as an |
| <a href="/use-cases/database/in-memory-database.html"><nobr>In-Memory Database</nobr></a> - |
| provides SQL, key-value and collocated processing APIs over in-memory data. |
| </p> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">100% on Disk + In-Memory Cache</td> |
| <td> |
| <p> |
| 100% of data is persisted to disk and the same or smaller amount is cached in memory. |
| The more data is cached, the faster is the performance. The disk serves as the primary storage that survives |
| any type of cluster failures and restarts. |
| </p> |
| |
| <p> |
| <strong>Use cases</strong>: Ignite as a |
| <a href="/use-cases/database/distributed-database.html">Memory-Centric Distributed Database</a> |
| - provides distributed database with SQL, key-value and collocated processing APIs. |
| </p> |
| </td> |
| </tr> |
| </tbody> |
| </table> |
| |
| <h2 style="padding-top: 10px;" id="collocated-processing"> |
| Collocated vs Client-Server Processing |
| <a href="/arch/memorycentric.html#collocated-processing"><i class="fa fa-anchor"></i></a> |
| </h2> |
| <p> |
| The disk-centric systems, like RDBMS or NoSQL, generally utilize the classic client-server approach, where |
| the data is brought from the server to the client side where it gets processed and then is usually discarded. |
| This approach does not scale well as moving the data over the network is the most expensive operation in a distributed system. |
| </p> |
| <p> |
| A much more scalable approach is <code>collocated processing</code> that reverses the flow by bringing the computations to the |
| servers where the data actually resides. This approach allows you to execute advanced logic or distributed SQL with JOINs |
| exactly where the data is stored avoiding expensive serialization and network trips. |
| </p> |
| |
| <h2 style="padding-top: 10px;" id="partitioning"> |
| Partitioning & Replication |
| <a href="/arch/memorycentric.html#partitioning"><i class="fa fa-anchor"></i></a> |
| </h2> |
| <p> |
| Depending on the configuration, Ignite can either partition or replicate data across its memory-centric |
| storage. Unlike <code>REPLICATED</code> mode, where data is fully replicated across all nodes |
| in the cluster, in <code>PARTITIONED</code> mode Ignite will equally split the data across |
| multiple cluster nodes, allowing for storing TBs of data both in memory and on disk. |
| </p> |
| |
| <h2 style="padding-top: 10px;" id="durability"> |
| Durability |
| <a href="/arch/memorycentric.html#durability"><i class="fa fa-anchor"></i></a> |
| </h2> |
| <p> |
| Ignite provides strong ACID durability guarantees to the data: |
| |
| <ul class="page-list" style="margin-bottom: 20px;"> |
| <li>Committed transactions will always survive any failures.</li> |
| <li> |
| The cluster can always be recovered to the latest successfully committed transaction. |
| </li> |
| <li> |
| The cluster restarts are very fast. |
| </li> |
| </ul> |
| </p> |
| |
| <h2 style="padding-top: 10px;" id="redundancy"> |
| Redundancy |
| <a href="/arch/memorycentric.html#redundancy"><i class="fa fa-anchor"></i></a> |
| </h2> |
| <p> |
| Ignite also allows to configure multiple <b>backup copies</b> to guarantee data resiliency |
| in case of failures. |
| </p> |
| |
| <h2 style="padding-top: 10px;" id="consistency"> |
| Consistency |
| <a href="/arch/memorycentric.html#consistency"><i class="fa fa-anchor"></i></a> |
| </h2> |
| <p> |
| Regardless of which replication scheme is used, Ignite guarantees data consistency across |
| all cluster members. |
| </p> |
| |
| <h2 style="padding-top: 10px;" id="write-ahead-log"> |
| Write-Ahead Log |
| <a href="/arch/memorycentric.html#write-ahead-log"><i class="fa fa-anchor"></i></a> |
| </h2> |
| <p> |
| Every time the data is updated in memory, the update will be appended to the tail of |
| the write-ahead log (WAL). The purpose of the WAL is to propagate updates to disk in |
| the fastest way possible and provide a consistent recovery mechanism that supports full cluster failures. |
| </p> |
| |
| <h2 style="padding-top: 10px;" id="checkpointing"> |
| Checkpointing |
| <a href="/arch/memorycentric.html#checkpointing"><i class="fa fa-anchor"></i></a> |
| </h2> |
| <p> |
| As WAL grows, it periodically gets <i>checkpointed</i> to the main storage. |
| Checkpointing is the process of copying <i>dirty pages</i> from memory to the partition files on disk. |
| A dirty page is a page that was updated in memory, was appended to WAL, but was not written to a respective partition |
| file on disk yet. |
| </p> |
| |
| <h2 style="padding-top: 10px;" id="peristence-config"> |
| Persistence Configuration |
| <a href="/arch/memorycentric.html#peristence-config"><i class="fa fa-anchor"></i></a> |
| </h2> |
| <p> |
| To enable Ignite persistence, add the following configuration parameter to the cluster's |
| node configuration: |
| </p> |
| <div class="tab-content"> |
| <div class="tab-pane active" id="configuration"> |
| <pre class="brush:xml"> |
| <bean class="org.apache.ignite.configuration.IgniteConfiguration"> |
| <!-- Enabling Apache Ignite native persistence. --> |
| <property name="dataStorageConfiguration"> |
| <bean class="org.apache.ignite.configuration.DataStorageConfiguration"> |
| <property name="defaultDataRegionConfiguration"> |
| <bean class="org.apache.ignite.configuration.DataRegionConfiguration"> |
| <property name="persistenceEnabled" value="true"/> |
| </bean> |
| </property> |
| </bean> |
| </property> |
| |
| <!-- Additional setting. --> |
| |
| </bean> |
| </pre> |
| </div> |
| </div><br/> |
| |
| <h2 style="padding-top: 10px;" id="more"> |
| More on Memory-Centric Storage |
| <a href="/arch/memorycentric.html#more"><i class="fa fa-anchor"></i></a> |
| </h2> |
| <table class="formatted" name="Deployment Options Features"> |
| <thead> |
| <tr> |
| <th width="35%" class="left">Feature</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td class="left">Persistence</td> |
| <td> |
| <p> |
| Ignite native persistence is a distributed, ACID, and <nobr>SQL-compliant</nobr> disk store |
| that transparently integrates with Ignite memory-centric storage: |
| </p> |
| <div class="page-links"> |
| <a href="/arch/persistence.html">Docs for this feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">Partitioning & Replication</td> |
| <td> |
| <p> |
| Depending on the configuration, Ignite can either <i>partition</i> or <i>replicate</i> |
| data. Unlike <code>REPLICATED</code> mode, where data is fully replicated across |
| all nodes in the cluster, in <code>PARTITIONED</code> mode Ignite will equally split the data |
| across multiple cluster nodes. |
| </p> |
| <div class="page-links"> |
| <a href="http://apacheignite.readme.io/docs/cache-modes" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">Distributed Database</td> |
| <td> |
| <p> |
| Apache Ignite can be used as all-in-one distributed database that supports SQL, key-value, |
| compute, machine learning and other data processing APIs: |
| </p> |
| <div class="page-links"> |
| <a href="/use-cases/database/distributed-database.html">Docs for this feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">In-Memory Database</td> |
| <td> |
| <p> |
| Apache Ignite can be used as a distributed and horizontally scalable in-memory database (IMDB): |
| </p> |
| <div class="page-links"> |
| <a href="/use-cases/database/in-memory-database.html">Docs for this feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">Data Grid</td> |
| <td> |
| <p> |
| Ignite can act as a data grid that is a distributed, transactional key-value store. Unlike |
| other in-memory data grids (IMDG), Ignite enables storing data both, in memory and on disk, |
| and therefore is able to store more data than can fit in physical memory: |
| </p> |
| <div class="page-links"> |
| <a href="/features/datagrid.html">Docs for this Feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">Database Caching</td> |
| <td> |
| <p> |
| Ignite is used as a caching layer (aka. data grid) above 3rd party databases such as RDBMS, |
| Apache Cassandra, MongoDB: |
| </p> |
| <div class="page-links"> |
| <a href="/use-cases/caching/database-caching.html">Docs for this Feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| </tbody> |
| </table> |
| </section> |
| </main> |
| |
| <!--#include virtual="/includes/footer.html" --> |
| </div> |
| <!--#include virtual="/includes/scripts.html" --> |
| </body> |
| </html> |
