use-cases/hadoop/mapreduce.html - ignite-website - Git at Google

 <!--
 ▄▄▄       ██▓███   ▄▄▄       ▄████▄   ██░ ██ ▓█████     ██▓  ▄████  ███▄    █  ██▓▄▄▄█████▓▓█████
 ▒████▄    ▓██░  ██▒▒████▄    ▒██▀ ▀█  ▓██░ ██▒▓█   ▀    ▓██▒ ██▒ ▀█▒ ██ ▀█   █ ▓██▒▓  ██▒ ▓▒▓█   ▀
 ▒██  ▀█▄  ▓██░ ██▓▒▒██  ▀█▄  ▒▓█    ▄ ▒██▀▀██░▒███      ▒██▒▒██░▄▄▄░▓██  ▀█ ██▒▒██▒▒ ▓██░ ▒░▒███
 ░██▄▄▄▄██ ▒██▄█▓▒ ▒░██▄▄▄▄██ ▒▓▓▄ ▄██▒░▓█ ░██ ▒▓█  ▄    ░██░░▓█  ██▓▓██▒  ▐▌██▒░██░░ ▓██▓ ░ ▒▓█  ▄
 ▓█   ▓██▒▒██▒ ░  ░ ▓█   ▓██▒▒ ▓███▀ ░░▓█▒░██▓░▒████▒   ░██░░▒▓███▀▒▒██░   ▓██░░██░  ▒██▒ ░ ░▒████▒
 ▒▒   ▓▒█░▒▓▒░ ░  ░ ▒▒   ▓▒█░░ ░▒ ▒  ░ ▒ ░░▒░▒░░ ▒░ ░   ░▓   ░▒   ▒ ░ ▒░   ▒ ▒ ░▓    ▒ ░░   ░░ ▒░ ░
  ▒   ▒▒ ░░▒ ░       ▒   ▒▒ ░  ░  ▒    ▒ ░▒░ ░ ░ ░  ░    ▒ ░  ░   ░ ░ ░░   ░ ▒░ ▒ ░    ░     ░ ░  ░
  ░   ▒   ░░         ░   ▒   ░         ░  ░░ ░   ░       ▒ ░░ ░   ░    ░   ░ ░  ▒ ░  ░         ░
      ░  ░               ░  ░░ ░       ░  ░  ░   ░  ░    ░        ░          ░  ░              ░  ░
 -->

 <!--
 Licensed to the Apache Software Foundation (ASF) under one
 or more contributor license agreements.  See the NOTICE file
 distributed with this work for additional information
 regarding copyright ownership.  The ASF licenses this file
 to you under the Apache License, Version 2.0 (the
 "License"); you may not use this file except in compliance
 with the License.  You may obtain a copy of the License at

 http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing,
 software distributed under the License is distributed on an
 "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 KIND, either express or implied.  See the License for the
 specific language governing permissions and limitations
 under the License.
 -->

 <!DOCTYPE html>
 <html>
 <head>
     <meta charset="utf-8">
     <meta name="viewport" content="width=device-width, initial-scale=1.0">
     <meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate" />
     <meta http-equiv="Pragma" content="no-cache" />
     <meta http-equiv="Expires" content="0" />
     <title>Apache Ignite - Faster In-Memory MapReduce</title>
     <link media="all" rel="stylesheet" href="/css/all.css">
     <link href="https://netdna.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.css" rel="stylesheet">
     <link href='https://fonts.googleapis.com/css?family=Open+Sans:400,300,300italic,400italic,600,600italic,700,700italic,800,800italic' rel='stylesheet' type='text/css'>

     <!--#include virtual="/includes/sh.html" -->
 </head>
 <body>
 <div id="wrapper">
     <!--#include virtual="/includes/header.html" -->

     <main id="main" role="main" class="container">
         <section id="mapreduce" class="page-section">
             <div class="col-sm-12 col-md-12 col-xs-12" style="padding:0 0 20px 0;">
                 <div class="col-sm-6 col-md-6 col-xs-12" style="padding-left:0;">
                     <h2 class="first">Faster In-Memory MapReduce</h2>
                     <p>
                         Apache Ignite In-Memory MapReduce allows to effectively parallelize the processing of data stored in
                         any Hadoop file system, including the In-Memory File System provided by Ignite.
                     </p>
                     <p>
                         It eliminates the overhead associated with name-node, job tracker, and task trackers in a
                         standard Hadoop architecture while providing low-latency distributed processing.
                     </p>

                     <div class="page-heading">Faster Scheduling</div>
                     <p>
                         Ignite MapReduce performs much better than Hadoop due to push-based resource allocation
                         (vs. pull-based in Hadoop), as well as in-process collocation of computations with data.
                     </p>
                 </div>

                 <div class="col-sm-6 col-md-6 col-xs-12" style="padding-right:0;">
                     <img class="first img-responsive" src="/images/hadoop_sequence.png"  style="float:right;"/>
                 </div>
             </div>

             <div class="page-heading">Data Locality</div>
             <p>
                 In HDFS, <i>NameNode</i> stores all the metadata and can be a single point of failure. In Ignite, every client
                 can determine which node a key belongs to by plugging it into a hashing function, without a need for any special mapping servers or name nodes.
             </p>

             <div class="page-heading">Reduced Network Overhead</div>
             <p>
                 Since <a href="/features/igfs.html">Ignite File System (IGFS)</a> does not need a
                 <i>NameNode</i>, when used with <i>IGFS</i>, Ignite MapReduce jobs go directly to the
                 IGFS data nodes in a single <span style="white-space: nowrap">round-trip</span>.
             </p>

             <p>
                 <a href="https://apacheignite-fs.readme.io/docs/map-reduce" target="docs">
                     <b>Configure In-Memory MapReduce <i class="fa fa-angle-double-right"></i></b>
                 </a>
             </p>
         </section>

     </main>

     <!--#include virtual="/includes/footer.html" -->
 </div>
 <!--#include virtual="/includes/scripts.html" -->
 </body>
 </html>
	<!--
	▄▄▄ ██▓███ ▄▄▄ ▄████▄ ██░ ██ ▓█████ ██▓ ▄████ ███▄ █ ██▓▄▄▄█████▓▓█████
	▒████▄ ▓██░ ██▒▒████▄ ▒██▀ ▀█ ▓██░ ██▒▓█ ▀ ▓██▒ ██▒ ▀█▒ ██ ▀█ █ ▓██▒▓ ██▒ ▓▒▓█ ▀
	▒██ ▀█▄ ▓██░ ██▓▒▒██ ▀█▄ ▒▓█ ▄ ▒██▀▀██░▒███ ▒██▒▒██░▄▄▄░▓██ ▀█ ██▒▒██▒▒ ▓██░ ▒░▒███
	░██▄▄▄▄██ ▒██▄█▓▒ ▒░██▄▄▄▄██ ▒▓▓▄ ▄██▒░▓█ ░██ ▒▓█ ▄ ░██░░▓█ ██▓▓██▒ ▐▌██▒░██░░ ▓██▓ ░ ▒▓█ ▄
	▓█ ▓██▒▒██▒ ░ ░ ▓█ ▓██▒▒ ▓███▀ ░░▓█▒░██▓░▒████▒ ░██░░▒▓███▀▒▒██░ ▓██░░██░ ▒██▒ ░ ░▒████▒
	▒▒ ▓▒█░▒▓▒░ ░ ░ ▒▒ ▓▒█░░ ░▒ ▒ ░ ▒ ░░▒░▒░░ ▒░ ░ ░▓ ░▒ ▒ ░ ▒░ ▒ ▒ ░▓ ▒ ░░ ░░ ▒░ ░
	▒ ▒▒ ░░▒ ░ ▒ ▒▒ ░ ░ ▒ ▒ ░▒░ ░ ░ ░ ░ ▒ ░ ░ ░ ░ ░░ ░ ▒░ ▒ ░ ░ ░ ░ ░
	░ ▒ ░░ ░ ▒ ░ ░ ░░ ░ ░ ▒ ░░ ░ ░ ░ ░ ░ ▒ ░ ░ ░
	░ ░ ░ ░░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░
	-->

	<!--
	Licensed to the Apache Software Foundation (ASF) under one
	or more contributor license agreements. See the NOTICE file
	distributed with this work for additional information
	regarding copyright ownership. The ASF licenses this file
	to you under the Apache License, Version 2.0 (the
	"License"); you may not use this file except in compliance
	with the License. You may obtain a copy of the License at

	http://www.apache.org/licenses/LICENSE-2.0

	Unless required by applicable law or agreed to in writing,
	software distributed under the License is distributed on an
	"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	KIND, either express or implied. See the License for the
	specific language governing permissions and limitations
	under the License.
	-->

	<!DOCTYPE html>
	<html>
	<head>
	<meta charset="utf-8">
	<meta name="viewport" content="width=device-width, initial-scale=1.0">
	<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate" />
	<meta http-equiv="Pragma" content="no-cache" />
	<meta http-equiv="Expires" content="0" />
	<title>Apache Ignite - Faster In-Memory MapReduce</title>
	<link media="all" rel="stylesheet" href="/css/all.css">
	<link href="https://netdna.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.css" rel="stylesheet">
	<link href='https://fonts.googleapis.com/css?family=Open+Sans:400,300,300italic,400italic,600,600italic,700,700italic,800,800italic' rel='stylesheet' type='text/css'>

	<!--#include virtual="/includes/sh.html" -->
	</head>
	<body>
	<div id="wrapper">
	<!--#include virtual="/includes/header.html" -->

	<main id="main" role="main" class="container">
	<section id="mapreduce" class="page-section">
	<div class="col-sm-12 col-md-12 col-xs-12" style="padding:0 0 20px 0;">
	<div class="col-sm-6 col-md-6 col-xs-12" style="padding-left:0;">
	<h2 class="first">Faster In-Memory MapReduce</h2>
	<p>
	Apache Ignite In-Memory MapReduce allows to effectively parallelize the processing of data stored in
	any Hadoop file system, including the In-Memory File System provided by Ignite.
	</p>
	<p>
	It eliminates the overhead associated with name-node, job tracker, and task trackers in a
	standard Hadoop architecture while providing low-latency distributed processing.
	</p>

	<div class="page-heading">Faster Scheduling</div>
	<p>
	Ignite MapReduce performs much better than Hadoop due to push-based resource allocation
	(vs. pull-based in Hadoop), as well as in-process collocation of computations with data.
	</p>
	</div>

	<div class="col-sm-6 col-md-6 col-xs-12" style="padding-right:0;">
	<img class="first img-responsive" src="/images/hadoop_sequence.png" style="float:right;"/>
	</div>
	</div>

	<div class="page-heading">Data Locality</div>
	<p>
	In HDFS, <i>NameNode</i> stores all the metadata and can be a single point of failure. In Ignite, every client
	can determine which node a key belongs to by plugging it into a hashing function, without a need for any special mapping servers or name nodes.
	</p>

	<div class="page-heading">Reduced Network Overhead</div>
	<p>
	Since <a href="/features/igfs.html">Ignite File System (IGFS)</a> does not need a
	<i>NameNode</i>, when used with <i>IGFS</i>, Ignite MapReduce jobs go directly to the
	IGFS data nodes in a single <span style="white-space: nowrap">round-trip</span>.
	</p>

	<p>
	<a href="https://apacheignite-fs.readme.io/docs/map-reduce" target="docs">
	<b>Configure In-Memory MapReduce <i class="fa fa-angle-double-right"></i></b>
	</a>
	</p>
	</section>

	</main>

	<!--#include virtual="/includes/footer.html" -->
	</div>
	<!--#include virtual="/includes/scripts.html" -->
	</body>
	</html>