arch/persistence.html - ignite-website - Git at Google

 <!--
 ▄▄▄       ██▓███   ▄▄▄       ▄████▄   ██░ ██ ▓█████     ██▓  ▄████  ███▄    █  ██▓▄▄▄█████▓▓█████
 ▒████▄    ▓██░  ██▒▒████▄    ▒██▀ ▀█  ▓██░ ██▒▓█   ▀    ▓██▒ ██▒ ▀█▒ ██ ▀█   █ ▓██▒▓  ██▒ ▓▒▓█   ▀
 ▒██  ▀█▄  ▓██░ ██▓▒▒██  ▀█▄  ▒▓█    ▄ ▒██▀▀██░▒███      ▒██▒▒██░▄▄▄░▓██  ▀█ ██▒▒██▒▒ ▓██░ ▒░▒███
 ░██▄▄▄▄██ ▒██▄█▓▒ ▒░██▄▄▄▄██ ▒▓▓▄ ▄██▒░▓█ ░██ ▒▓█  ▄    ░██░░▓█  ██▓▓██▒  ▐▌██▒░██░░ ▓██▓ ░ ▒▓█  ▄
 ▓█   ▓██▒▒██▒ ░  ░ ▓█   ▓██▒▒ ▓███▀ ░░▓█▒░██▓░▒████▒   ░██░░▒▓███▀▒▒██░   ▓██░░██░  ▒██▒ ░ ░▒████▒
 ▒▒   ▓▒█░▒▓▒░ ░  ░ ▒▒   ▓▒█░░ ░▒ ▒  ░ ▒ ░░▒░▒░░ ▒░ ░   ░▓   ░▒   ▒ ░ ▒░   ▒ ▒ ░▓    ▒ ░░   ░░ ▒░ ░
  ▒   ▒▒ ░░▒ ░       ▒   ▒▒ ░  ░  ▒    ▒ ░▒░ ░ ░ ░  ░    ▒ ░  ░   ░ ░ ░░   ░ ▒░ ▒ ░    ░     ░ ░  ░
  ░   ▒   ░░         ░   ▒   ░         ░  ░░ ░   ░       ▒ ░░ ░   ░    ░   ░ ░  ▒ ░  ░         ░
      ░  ░               ░  ░░ ░       ░  ░  ░   ░  ░    ░        ░          ░  ░              ░  ░
 -->

 <!--
 Licensed to the Apache Software Foundation (ASF) under one
 or more contributor license agreements.  See the NOTICE file
 distributed with this work for additional information
 regarding copyright ownership.  The ASF licenses this file
 to you under the Apache License, Version 2.0 (the
 "License"); you may not use this file except in compliance
 with the License.  You may obtain a copy of the License at

 http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing,
 software distributed under the License is distributed on an
 "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 KIND, either express or implied.  See the License for the
 specific language governing permissions and limitations
 under the License.
 -->

 <!DOCTYPE html>
 <html lang="en">
 <head>
     <link rel="canonical" href="https://ignite.apache.org/arch/persistence.html"/>
     <meta charset="utf-8">
     <meta name="viewport" content="width=device-width, initial-scale=1.0">
     <meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate"/>
     <meta http-equiv="Pragma" content="no-cache"/>
     <meta http-equiv="Expires" content="0"/>

     <title>Distributed Persistence - Apache Ignite</title>

     <meta name="description"
           content="The native persistence integrates into the Ignite multi-tier storage as a disk tier that can
      be turned on to let Ignite store more data on disk than it can cache in memory and to enable fast cluster restarts."/>

     <!--#include virtual="/includes/styles.html" -->

     <!--#include virtual="/includes/sh.html" -->
 </head>
 <body>
     <!--#include virtual="/includes/header.html" -->
     <article>
       <header>
         <div class="container">
           <h1>Apache Ignite <strong>Native Persistence</strong></h1>
         </div>
       </header>
       <div class="container">

           <p>
             Even though Apache Ignite® is broadly used as a caching layer on top of external databases, it
             comes with its native persistence - a distributed, ACID, and SQL-compliant disk-based
             store. The native persistence integrates into the Ignite multi-tier storage as a disk tier that
             can be turned on to let Ignite store more data on disk than it can cache in memory and to enable
             fast cluster restarts.
         </p>
         <img class="img-responsive diagram-right" src="/images/svg-diagrams/native_persistence.svg" alt="Apache Ignite Native Persistence"/></a>
         <p>
             When the native persistence enabled, Ignite stores a superset of data on disk and caches as
             much as it can in memory. For example, if your application needs to store 200 records in an
             Ignite cluster and the memory capacity allows caching only 150 records, then all 200 will be
             stored on disk, out of which 150 will be served from memory while the rest 50 from disk whenever
             the application requests them.
         </p>

           <h2>Ignite Persistence vs. External Databases</h2>
           <p>
             The native persistence has the following advantages over external databases:
           </p>
           <ul>
             <li>
                 The ability to cache a subset of the data - Ignite stores 100% of data on disk and lets you cache as
                 much as required in memory.
             </li>
             <li>
                 The ability to query data from disk - if any record is missing in memory, then Ignite reads it from
                 disk. This is supported for all the APIs including SQL.
             </li>
             <li>
                 Instantaneous cluster restarts - Ignite becomes fully operational from disk upon a cluster
                 startup or restarts without requiring to preload or warm up the memory tier.
             </li>
           </ul>

           <h2>Write-Ahead Logging and Checkpointing</h2>
           <p>
             If Ignite native persistence is selected as a disk tier, then every time a record is updated in memory,
             the change is added to the write-ahead log (WAL). The purpose of the WAL is to propagate updates to disk
             in the fastest way possible and provide a consistent recovery mechanism that supports full cluster
             failures.

             As the WAL grows, it periodically gets checkpointed to the main storage. Checkpointing is the process of
             copying dirty pages from the memory tier to the partition files on disk. A dirty page is a page that was
             updated in memory, was appended to the WAL, but was not written to the respective partition file on disk yet.
           </p>

           <h2>Durability</h2>
           <p>
             Ignite native persistence provides the following ACID guarantees across the cluster:
           </p>

           <ul>
             <li>
                 Committed transactions always survive failures.
             </li>
             <li>
                 The cluster can always be recovered to the latest successfully committed transaction.
             </li>
           </ul>

       <div class="jumbotron jumbotron-fluid">
         <div class="container">
           <div class="display-6 title">Learn More</div>
           <hr class="my-4">
           <div class="row">
             <div class="col-sm-6">
               <ul>
                 <li>
                   <p><a href="/arch/multi-tier-storage.html">Multi-Tier Storage <i class="fa fa-angle-double-right"></i></a></p>
                 </li>
                 <li>
                   <p><a href="https://apacheignite.readme.io/docs/distributed-persistent-store" target="docs">Native Persistence Usage and Configuration <i class="fa fa-angle-double-right"></i></a></p>
                 </li>
                 <li>
                   <p><a href="https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood" target="docs">Native Persistence Implementation Details <i class="fa fa-angle-double-right"></i></a></p>
                 </li>
               </ul>
             </div>
             <div class="col-sm-6">
               <ul>
                 <li>
                   <p><a href="/features/sql.html">Distributed SQL <i class="fa fa-angle-double-right"></i></a></p>
                 </li>
                 <li>
                   <p><a href="/features/collocatedprocessing.html">Co-located Processing <i class="fa fa-angle-double-right"></i></a></p>
                 </li>
                 <li>
                 <p><a href="/features/transactions.html">ACID Transactions <i class="fa fa-angle-double-right"></i></a></p>
                 </li>
               </ul>
             </div>
           </div>
         </div>
       </div>
   </div><!-- end .container -->
 </article>
 <!--#include virtual="/includes/footer.html" -->
 <!--#include virtual="/includes/scripts.html" -->
 </body>
 </html>
	<!--
	▄▄▄ ██▓███ ▄▄▄ ▄████▄ ██░ ██ ▓█████ ██▓ ▄████ ███▄ █ ██▓▄▄▄█████▓▓█████
	▒████▄ ▓██░ ██▒▒████▄ ▒██▀ ▀█ ▓██░ ██▒▓█ ▀ ▓██▒ ██▒ ▀█▒ ██ ▀█ █ ▓██▒▓ ██▒ ▓▒▓█ ▀
	▒██ ▀█▄ ▓██░ ██▓▒▒██ ▀█▄ ▒▓█ ▄ ▒██▀▀██░▒███ ▒██▒▒██░▄▄▄░▓██ ▀█ ██▒▒██▒▒ ▓██░ ▒░▒███
	░██▄▄▄▄██ ▒██▄█▓▒ ▒░██▄▄▄▄██ ▒▓▓▄ ▄██▒░▓█ ░██ ▒▓█ ▄ ░██░░▓█ ██▓▓██▒ ▐▌██▒░██░░ ▓██▓ ░ ▒▓█ ▄
	▓█ ▓██▒▒██▒ ░ ░ ▓█ ▓██▒▒ ▓███▀ ░░▓█▒░██▓░▒████▒ ░██░░▒▓███▀▒▒██░ ▓██░░██░ ▒██▒ ░ ░▒████▒
	▒▒ ▓▒█░▒▓▒░ ░ ░ ▒▒ ▓▒█░░ ░▒ ▒ ░ ▒ ░░▒░▒░░ ▒░ ░ ░▓ ░▒ ▒ ░ ▒░ ▒ ▒ ░▓ ▒ ░░ ░░ ▒░ ░
	▒ ▒▒ ░░▒ ░ ▒ ▒▒ ░ ░ ▒ ▒ ░▒░ ░ ░ ░ ░ ▒ ░ ░ ░ ░ ░░ ░ ▒░ ▒ ░ ░ ░ ░ ░
	░ ▒ ░░ ░ ▒ ░ ░ ░░ ░ ░ ▒ ░░ ░ ░ ░ ░ ░ ▒ ░ ░ ░
	░ ░ ░ ░░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░
	-->

	<!--
	Licensed to the Apache Software Foundation (ASF) under one
	or more contributor license agreements. See the NOTICE file
	distributed with this work for additional information
	regarding copyright ownership. The ASF licenses this file
	to you under the Apache License, Version 2.0 (the
	"License"); you may not use this file except in compliance
	with the License. You may obtain a copy of the License at

	http://www.apache.org/licenses/LICENSE-2.0

	Unless required by applicable law or agreed to in writing,
	software distributed under the License is distributed on an
	"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	KIND, either express or implied. See the License for the
	specific language governing permissions and limitations
	under the License.
	-->

	<!DOCTYPE html>
	<html lang="en">
	<head>
	<link rel="canonical" href="https://ignite.apache.org/arch/persistence.html"/>
	<meta charset="utf-8">
	<meta name="viewport" content="width=device-width, initial-scale=1.0">
	<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate"/>
	<meta http-equiv="Pragma" content="no-cache"/>
	<meta http-equiv="Expires" content="0"/>

	<title>Distributed Persistence - Apache Ignite</title>

	<meta name="description"
	content="The native persistence integrates into the Ignite multi-tier storage as a disk tier that can
	be turned on to let Ignite store more data on disk than it can cache in memory and to enable fast cluster restarts."/>

	<!--#include virtual="/includes/styles.html" -->

	<!--#include virtual="/includes/sh.html" -->
	</head>
	<body>
	<!--#include virtual="/includes/header.html" -->
	<article>
	<header>
	<div class="container">
	<h1>Apache Ignite <strong>Native Persistence</strong></h1>
	</div>
	</header>
	<div class="container">

	<p>
	Even though Apache Ignite® is broadly used as a caching layer on top of external databases, it
	comes with its native persistence - a distributed, ACID, and SQL-compliant disk-based
	store. The native persistence integrates into the Ignite multi-tier storage as a disk tier that
	can be turned on to let Ignite store more data on disk than it can cache in memory and to enable
	fast cluster restarts.
	</p>
	<img class="img-responsive diagram-right" src="/images/svg-diagrams/native_persistence.svg" alt="Apache Ignite Native Persistence"/></a>
	<p>
	When the native persistence enabled, Ignite stores a superset of data on disk and caches as
	much as it can in memory. For example, if your application needs to store 200 records in an
	Ignite cluster and the memory capacity allows caching only 150 records, then all 200 will be
	stored on disk, out of which 150 will be served from memory while the rest 50 from disk whenever
	the application requests them.
	</p>

	<h2>Ignite Persistence vs. External Databases</h2>
	<p>
	The native persistence has the following advantages over external databases:
	</p>
	<ul>
	<li>
	The ability to cache a subset of the data - Ignite stores 100% of data on disk and lets you cache as
	much as required in memory.
	</li>
	<li>
	The ability to query data from disk - if any record is missing in memory, then Ignite reads it from
	disk. This is supported for all the APIs including SQL.
	</li>
	<li>
	Instantaneous cluster restarts - Ignite becomes fully operational from disk upon a cluster
	startup or restarts without requiring to preload or warm up the memory tier.
	</li>
	</ul>

	<h2>Write-Ahead Logging and Checkpointing</h2>
	<p>
	If Ignite native persistence is selected as a disk tier, then every time a record is updated in memory,
	the change is added to the write-ahead log (WAL). The purpose of the WAL is to propagate updates to disk
	in the fastest way possible and provide a consistent recovery mechanism that supports full cluster
	failures.

	As the WAL grows, it periodically gets checkpointed to the main storage. Checkpointing is the process of
	copying dirty pages from the memory tier to the partition files on disk. A dirty page is a page that was
	updated in memory, was appended to the WAL, but was not written to the respective partition file on disk yet.
	</p>

	<h2>Durability</h2>
	<p>
	Ignite native persistence provides the following ACID guarantees across the cluster:
	</p>

	<ul>
	<li>
	Committed transactions always survive failures.
	</li>
	<li>
	The cluster can always be recovered to the latest successfully committed transaction.
	</li>
	</ul>

	<div class="jumbotron jumbotron-fluid">
	<div class="container">
	<div class="display-6 title">Learn More</div>
	<hr class="my-4">
	<div class="row">
	<div class="col-sm-6">
	<ul>
	<li>
	<p><a href="/arch/multi-tier-storage.html">Multi-Tier Storage <i class="fa fa-angle-double-right"></i></a></p>
	</li>
	<li>
	<p><a href="https://apacheignite.readme.io/docs/distributed-persistent-store" target="docs">Native Persistence Usage and Configuration <i class="fa fa-angle-double-right"></i></a></p>
	</li>
	<li>
	<p><a href="https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood" target="docs">Native Persistence Implementation Details <i class="fa fa-angle-double-right"></i></a></p>
	</li>
	</ul>
	</div>
	<div class="col-sm-6">
	<ul>
	<li>
	<p><a href="/features/sql.html">Distributed SQL <i class="fa fa-angle-double-right"></i></a></p>
	</li>
	<li>
	<p><a href="/features/collocatedprocessing.html">Co-located Processing <i class="fa fa-angle-double-right"></i></a></p>
	</li>
	<li>
	<p><a href="/features/transactions.html">ACID Transactions <i class="fa fa-angle-double-right"></i></a></p>
	</li>
	</ul>
	</div>
	</div>
	</div>
	</div>
	</div><!-- end .container -->
	</article>
	<!--#include virtual="/includes/footer.html" -->
	<!--#include virtual="/includes/scripts.html" -->
	</body>
	</html>