| <!-- |
| ▄▄▄ ██▓███ ▄▄▄ ▄████▄ ██░ ██ ▓█████ ██▓ ▄████ ███▄ █ ██▓▄▄▄█████▓▓█████ |
| ▒████▄ ▓██░ ██▒▒████▄ ▒██▀ ▀█ ▓██░ ██▒▓█ ▀ ▓██▒ ██▒ ▀█▒ ██ ▀█ █ ▓██▒▓ ██▒ ▓▒▓█ ▀ |
| ▒██ ▀█▄ ▓██░ ██▓▒▒██ ▀█▄ ▒▓█ ▄ ▒██▀▀██░▒███ ▒██▒▒██░▄▄▄░▓██ ▀█ ██▒▒██▒▒ ▓██░ ▒░▒███ |
| ░██▄▄▄▄██ ▒██▄█▓▒ ▒░██▄▄▄▄██ ▒▓▓▄ ▄██▒░▓█ ░██ ▒▓█ ▄ ░██░░▓█ ██▓▓██▒ ▐▌██▒░██░░ ▓██▓ ░ ▒▓█ ▄ |
| ▓█ ▓██▒▒██▒ ░ ░ ▓█ ▓██▒▒ ▓███▀ ░░▓█▒░██▓░▒████▒ ░██░░▒▓███▀▒▒██░ ▓██░░██░ ▒██▒ ░ ░▒████▒ |
| ▒▒ ▓▒█░▒▓▒░ ░ ░ ▒▒ ▓▒█░░ ░▒ ▒ ░ ▒ ░░▒░▒░░ ▒░ ░ ░▓ ░▒ ▒ ░ ▒░ ▒ ▒ ░▓ ▒ ░░ ░░ ▒░ ░ |
| ▒ ▒▒ ░░▒ ░ ▒ ▒▒ ░ ░ ▒ ▒ ░▒░ ░ ░ ░ ░ ▒ ░ ░ ░ ░ ░░ ░ ▒░ ▒ ░ ░ ░ ░ ░ |
| ░ ▒ ░░ ░ ▒ ░ ░ ░░ ░ ░ ▒ ░░ ░ ░ ░ ░ ░ ▒ ░ ░ ░ |
| ░ ░ ░ ░░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ |
| --> |
| |
| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| --> |
| |
| <!DOCTYPE html> |
| <html> |
| <head> |
| <link rel="canonical" href="https://ignite.apache.org/features/igfs.html" /> |
| <meta charset="utf-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| <meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate" /> |
| <meta http-equiv="Pragma" content="no-cache" /> |
| <meta http-equiv="Expires" content="0" /> |
| <title>In-Memory File System - Apache Ignite</title> |
| <link media="all" rel="stylesheet" href="/css/all.css?v=1514336028"> |
| <link href="https://netdna.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.css" rel="stylesheet"> |
| <link href='https://fonts.googleapis.com/css?family=Open+Sans:400,300,300italic,400italic,600,600italic,700,700italic,800,800italic' rel='stylesheet' type='text/css'> |
| |
| <!--#include virtual="/includes/sh.html" --> |
| </head> |
| <body> |
| <div id="wrapper"> |
| <!--#include virtual="/includes/header.html" --> |
| |
| <main id="main" role="main" class="container"> |
| <section id="igfs" class="page-section"> |
| <h1 class="first">In-Memory File System</h1> |
| <div class="col-sm-12 col-md-12 col-xs-12" style="padding:0 0 10px 0;"> |
| <div class="col-sm-6 col-md-6 col-xs-12" style="padding-left:0; padding-right:0"> |
| |
| <p> |
| One of the unique capabilities of Apache Ignite is a distributed in-memory file system called |
| Ignite File System (IGFS). IGFS delivers a similar functionality to Hadoop HDFS, but |
| only in-memory. In fact, in addition to its own APIs, IGFS implements Hadoop FileSystem API |
| and can be transparently plugged into Hadoop or Spark deployments. |
| </p> |
| <p> |
| IGFS splits the data from each file into separate data blocks and stores them in |
| a distributed in-memory cache. However, unlike Hadoop HDFS, IGFS does not need a name node |
| and automatically determines file data locality using a hashing function. |
| </p> |
| </div> |
| <div class="col-sm-6 col-md-6 col-xs-12" style="padding-right:0"> |
| <img class="img-responsive" src="/images/spark-igfs.png" width="540px" style="float:right;"/> |
| </div> |
| </div> |
| <p> |
| IGFS can be deployed stand alone, as well as on top of HDFS in which case it becomes |
| a transparent caching layer for the files stored in HDFS. |
| </p> |
| |
| <p>IGFS integrates with native Apache Hadoop distribution, Cloudera CDH, and Hortonworks HDP.</p> |
| <div class="page-heading">Tachyon Replacement</div> |
| <p> |
| IGFS can transparently replace Tachyon file system in Spark deployments. Given that IGFS is based |
| on battle-tested Ignite data grid technology, it exhibits much better write and read performance |
| than Tachyon, and is more stable. |
| </p> |
| <div class="page-heading">Hadoop File System</div> |
| <p> |
| See <a href="http://apacheignite-fs.readme.io/docs/file-system" target="docs">Hadoop integration documentation</a> if |
| you plan to use IGFS as Hadoop file system. In this case working with IGFS is no different than working |
| with HDFS. |
| </p> |
| <div class="page-heading">GitHub Native API Examples</div> |
| <p> |
| Also see <a href="https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples/igfs" target="github">IGFS native API examples</a> |
| available on GitHub. |
| </p> |
| </section> |
| <section id="key-features" class="page-section"> |
| <h2>Ignite File System Features</h2> |
| <table class="formatted" name="Ignite File System Features"> |
| <thead> |
| <tr> |
| <th width="35%" class="left">Feature</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td class="left">On-Heap and Off-Heap</td> |
| <td> |
| <p> |
| IGFS allows to store files either on-heap or off-heap. For larger memory spaces |
| it is critical to use off-heap to avoid JVM lengthy garbage collection pauses. |
| </p> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">IGFS as Hadoop FileSystem</td> |
| <td> |
| <p> |
| IGFS implements Hadoop <code>FileSystem</code> API and can be deployed as a native |
| Hadoop file system, just like HDFS. This allows to natively deploy IGFS into Hadoop or |
| Spark installations in a plug-n-play fashion. |
| </p> |
| <div class="page-links"> |
| <a href="http://apacheignite-fs.readme.io/docs/file-system" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">Hadoop FileSystem Cache</td> |
| <td> |
| <p> |
| IGFS can also be deployed as a <code>caching layer</code> over another Hadoop File System. In this |
| case, if a file is updated IGFS, then update will automatically be written through to |
| HDFS. Also, if a file is read and is not currently in IGFS, it will be automatically |
| loaded from HDFS. |
| </p> |
| <div class="page-links"> |
| <a href="http://apacheignite-fs.readme.io/docs/secondary-file-system" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">Any Hadoop Distribution</td> |
| <td> |
| <p> |
| IGFS integrates with native Apache Hadoop distribution, as well as Cloudera CDH, and |
| Hortonworks HDP. |
| </p> |
| <div class="page-links" style="line-height: 1.3;"> |
| <a href="http://apacheignite-fs.readme.io/docs/installing-on-apache-hadoop" target="docs">IGFS on Apache Hadoop <i class="fa fa-angle-double-right"></i></a><br/> |
| <a href="http://apacheignite-fs.readme.io/docs/installing-on-cloudera-cdh" target="docs">IGFS on Cloudera CDH <i class="fa fa-angle-double-right"></i></a><br/> |
| <a href="http://apacheignite-fs.readme.io/docs/installing-on-hortonworks-hdp" target="docs">IGFS on Hortonworks HDP <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| </tbody> |
| </table> |
| </section> |
| </main> |
| |
| <!--#include virtual="/includes/footer.html" --> |
| </div> |
| <!--#include virtual="/includes/scripts.html" --> |
| </body> |
| </html> |
