| <!-- |
| ▄▄▄ ██▓███ ▄▄▄ ▄████▄ ██░ ██ ▓█████ ██▓ ▄████ ███▄ █ ██▓▄▄▄█████▓▓█████ |
| ▒████▄ ▓██░ ██▒▒████▄ ▒██▀ ▀█ ▓██░ ██▒▓█ ▀ ▓██▒ ██▒ ▀█▒ ██ ▀█ █ ▓██▒▓ ██▒ ▓▒▓█ ▀ |
| ▒██ ▀█▄ ▓██░ ██▓▒▒██ ▀█▄ ▒▓█ ▄ ▒██▀▀██░▒███ ▒██▒▒██░▄▄▄░▓██ ▀█ ██▒▒██▒▒ ▓██░ ▒░▒███ |
| ░██▄▄▄▄██ ▒██▄█▓▒ ▒░██▄▄▄▄██ ▒▓▓▄ ▄██▒░▓█ ░██ ▒▓█ ▄ ░██░░▓█ ██▓▓██▒ ▐▌██▒░██░░ ▓██▓ ░ ▒▓█ ▄ |
| ▓█ ▓██▒▒██▒ ░ ░ ▓█ ▓██▒▒ ▓███▀ ░░▓█▒░██▓░▒████▒ ░██░░▒▓███▀▒▒██░ ▓██░░██░ ▒██▒ ░ ░▒████▒ |
| ▒▒ ▓▒█░▒▓▒░ ░ ░ ▒▒ ▓▒█░░ ░▒ ▒ ░ ▒ ░░▒░▒░░ ▒░ ░ ░▓ ░▒ ▒ ░ ▒░ ▒ ▒ ░▓ ▒ ░░ ░░ ▒░ ░ |
| ▒ ▒▒ ░░▒ ░ ▒ ▒▒ ░ ░ ▒ ▒ ░▒░ ░ ░ ░ ░ ▒ ░ ░ ░ ░ ░░ ░ ▒░ ▒ ░ ░ ░ ░ ░ |
| ░ ▒ ░░ ░ ▒ ░ ░ ░░ ░ ░ ▒ ░░ ░ ░ ░ ░ ░ ▒ ░ ░ ░ |
| ░ ░ ░ ░░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ |
| --> |
| |
| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| --> |
| |
| <!DOCTYPE html> |
| <html> |
| <head> |
| <link rel="canonical" href="https://ignite.apache.org/features/computegrid.html" /> |
| <meta charset="utf-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| <meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate" /> |
| <meta http-equiv="Pragma" content="no-cache" /> |
| <meta http-equiv="Expires" content="0" /> |
| <title>In-Memory Compute Grid - Apache Ignite</title> |
| <link media="all" rel="stylesheet" href="/css/all.css?v=1514336028"> |
| <link href="https://netdna.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.css" rel="stylesheet"> |
| <link href='https://fonts.googleapis.com/css?family=Open+Sans:400,300,300italic,400italic,600,600italic,700,700italic,800,800italic' rel='stylesheet' type='text/css'> |
| |
| <!--#include virtual="/includes/sh.html" --> |
| </head> |
| <body> |
| <div id="wrapper"> |
| <!--#include virtual="/includes/header.html" --> |
| |
| <main id="main" role="main" class="container"> |
| <section id="computegrid" class="page-section"> |
| <h1 class="first">In-Memory Compute Grid</h1> |
| <div class="col-sm-12 col-md-12 col-xs-12" style="padding:0px;"> |
| <div class="col-sm-6 col-md-7 col-xs-12" style="padding-left:0; padding-right:0"> |
| <p> |
| Ignite In-Memory Compute Grid allows executing distributed computations in a parallel fashion to gain high performance, |
| low latency, and linear scalability. Ignite compute grid provides a set of simple APIs |
| that allow users distribute computations and data processing across multiple computers |
| in the cluster. |
| </p> |
| <p> |
| Distributed parallel processing is based on the ability to take any computation and |
| execute it on any set of cluster nodes and return the results back. |
| </p> |
| <div class="videos"> |
| <div class="page-heading">Videos:</div> |
| <ul> |
| <li> |
| <i class="fa fa-lg fa-play-circle-o"></i> |
| <span class="video-title"> |
| <a target="youtube" href="https://www.youtube.com/watch?v=J775J77_-B8">Getting Started with Compute Grid</a> |
| </span> |
| <span class="video-duration">03:50</span> |
| </li> |
| </ul> |
| </div> |
| </div> |
| <br/> |
| <div class="col-sm-6 col-md-5 col-xs-12" style="padding-right:0"> |
| <img class="img-responsive" src="/images/collocated_processing.png" width="400px" style="float:right;"/> |
| </div> |
| </div> |
| <div class="code-examples"> |
| <div class="page-heading">Code Examples:</div> |
| <!-- Nav tabs --> |
| <ul id="compute-examples" class="nav nav-tabs"> |
| <li class="active"><a href="#compute-example-broadcast" aria-controls="home" data-toggle="tab">Broadcast</a></li> |
| <li><a href="#compute-example-countchars" aria-controls="profile" data-toggle="tab">Count Characters with Callable</a></li> |
| <li><a href="#compute-example-closure" aria-controls="profile" data-toggle="tab">Count Characters with Closure</a></li> |
| </ul> |
| |
| <!-- Tab panes --> |
| <div class="tab-content"> |
| <div class="tab-pane active" id="compute-example-broadcast"> |
| <pre class="brush:java"> |
| Ignite ignite = Ignition.ignite(); |
| |
| // Print out hello message on all cluster nodes. |
| ignite.compute().broadcast(() -> System.out.println("Hello Node!")); |
| </pre> |
| </div> |
| <div class="tab-pane" id="compute-example-countchars"> |
| <pre class="brush:java"> |
| Collection<IgniteCallable<Integer>> calls = new ArrayList<>(); |
| |
| // Iterate through all words in the sentence and create callable jobs. |
| for (String word : "How Many Characters".split(" ")) |
| calls.add(word::length); |
| |
| // Execute collection of callables on the Ignite cluster. |
| Collection<Integer> res = ignite.compute().call(calls); |
| |
| // Add all the word lengths received from cluster nodes. |
| int total = res.stream().mapToInt(Integer::intValue).sum(); |
| </pre> |
| </div> |
| <div class="tab-pane" id="compute-example-closure"> |
| <pre class="brush:java"> |
| IgniteCompute compute = ignite.compute(); |
| |
| // Execute closure on all cluster nodes. |
| Collection<Integer> res = ignite.compute().apply( |
| String::length, |
| Arrays.asList("How Many Characters".split(" ")) |
| ); |
| |
| // Add all the word lengths received from cluster nodes. |
| int total = res.stream().mapToInt(Integer::intValue).sum(); |
| </pre> |
| </div> |
| </div> |
| </div> |
| |
| <div class="page-heading github-examples">GitHub Examples:</div> |
| <p> |
| Also see <a href="https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples/computegrid" target="github">Java 7 examples</a> |
| and <a href="https://github.com/apache/ignite/tree/master/examples/src/main/java8/org/apache/ignite/examples/java8/computegrid" target="github">Java 8 examples</a> available on GitHub. |
| </p> |
| </section> |
| <section id="key-features" class="page-section"> |
| <h2>Compute Grid Features</h2> |
| <table class="formatted" name="Compute Grid Features"> |
| <thead> |
| <tr> |
| <th width="35%" class="left">Feature</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td class="left">Distributed Closure Execution</td> |
| <td> |
| <p> |
| Ignite compute grid allows to broadcast and load-balance any closure within the cluster |
| or a cluster group, including Java 8 lambdas, as well as plain Java runnables and callables. |
| </p> |
| <div class="page-links"> |
| <a href="http://apacheignite.readme.io/docs/distributed-closures" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">ForkJoin Processing</td> |
| <td> |
| <p> |
| <code>ComputeTask</code> is Ignite abstraction for the in-memory <code>ForkJoin</code>, |
| paradigm, which is a light-weight form of MapReduce. Pure |
| MapReduce was never built for performance and only works well when dealing with |
| off-line batch oriented processing (e.g. Hadoop MapReduce). |
| </p> |
| <p> |
| However, when computing on data that resides in-memory, real-time low latencies |
| and high throughput usually take the highest priority. Also, simplicity of the API |
| becomes very important as well. With that in mind, Ignite introduced the |
| <code>ComputeTask</code> API, which is Ignite <code>ForkJoin</code> (or a |
| light-weight MapReduce) implementation. |
| </p> |
| <div class="page-links"> |
| <a href="http://apacheignite.readme.io/docs/compute-tasks" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">Clustered ExecutorService</td> |
| <td> |
| <p> |
| Ignite provides a cluster-enabled implementation of standard JDK |
| <code>ExecutorService</code> and automatically executes all the computations in |
| load-balanced fashion within the cluster. Your computations also become fault-tolerant |
| and are guaranteed to execute as long as there is at least one node left. You can |
| think of it as a distributed cluster-enabled thread pool. |
| </p> |
| <div class="page-links"> |
| <a href="http://apacheignite.readme.io/docs/executor-service" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">Collocation of Compute & Data</td> |
| <td> |
| <p> |
| Collocation of computations with data allow for minimizing data serialization within the |
| network and can significantly improve performance and scalability of your application. |
| Whenever possible, you should always make best effort to collocate your computations |
| with the cluster nodes caching the data that needs to be processed. |
| </p> |
| <p> |
| Ignite provides various ways to collocate compute with data either automatically |
| or manually as needed. |
| </p> |
| <div class="page-links"> |
| <a href="http://apacheignite.readme.io/docs/collocate-compute-and-data" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">Fault Tolerance</td> |
| <td> |
| <p> |
| Ignite supports automatic job failover. In case of a node crash or any other error, |
| jobs are automatically transferred to other available nodes for re-execution. |
| The pluggable <code>FailoverSpi</code> is responsible for handling the selection of a |
| new node for the execution of a failed job. |
| </p> |
| <p> |
| <b>At Least Once Guarantee</b> - Ignite guarantees that as long as there is at |
| least one node standing, no job will ever be lost. |
| </p> |
| <div class="page-links"> |
| <a href="http://apacheignite.readme.io/docs/fault-tolerance" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">Load Balancing</td> |
| <td> |
| <p> |
| Load balancing component balances job distribution among cluster nodes. In Ignite |
| load balancing is achieved via a pluggable <code>LoadBalancingSpi</code> which controls |
| load on all nodes and makes sure that every node in the cluster is equally loaded. |
| </p> |
| <p> |
| In homogeneous environments with homogeneous tasks load balancing is achieved by random or |
| round-robin policies. However, in many other use cases, especially under uneven load, |
| more complex adaptive load-balancing policies are provided. |
| </p> |
| <div class="page-links"> |
| <a href="http://apacheignite.readme.io/docs/load-balancing" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">Job Checkpointing</td> |
| <td> |
| <p> |
| Checkpointing is supported via a pluggable <code>CheckpointSpi</code>. It |
| provides ability to save an intermediate job state. Checkpointing can be useful |
| when long running jobs need to store some intermediate state to protect from node |
| failures. Then on restart of a failed node, a job would load the saved checkpoint |
| and continue from where it left off. |
| </p> |
| <div class="page-links"> |
| <a href="http://apacheignite.readme.io/docs/checkpointing" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| <tr> |
| <td class="left">Job Scheduling</td> |
| <td> |
| <p> |
| Pluggable <code>CollisionSpi</code> provides a fine-grained control over how the jobs |
| are scheduled once they arrive to a specific cluster node for execution. Various |
| policies are supported, including FIFO, Priority, or even job stealing. |
| </p> |
| <div class="page-links"> |
| <a href="http://apacheignite.readme.io/docs/job-scheduling" target="docs">Docs for this Feature <i class="fa fa-angle-double-right"></i></a> |
| </div> |
| </td> |
| </tr> |
| </tbody> |
| </table> |
| </section> |
| </main> |
| |
| <!--#include virtual="/includes/footer.html" --> |
| </div> |
| <!--#include virtual="/includes/scripts.html" --> |
| </body> |
| </html> |
