features/machinelearning.html - ignite-website - Git at Google

 <!--
 ▄▄▄       ██▓███   ▄▄▄       ▄████▄   ██░ ██ ▓█████     ██▓  ▄████  ███▄    █  ██▓▄▄▄█████▓▓█████
 ▒████▄    ▓██░  ██▒▒████▄    ▒██▀ ▀█  ▓██░ ██▒▓█   ▀    ▓██▒ ██▒ ▀█▒ ██ ▀█   █ ▓██▒▓  ██▒ ▓▒▓█   ▀
 ▒██  ▀█▄  ▓██░ ██▓▒▒██  ▀█▄  ▒▓█    ▄ ▒██▀▀██░▒███      ▒██▒▒██░▄▄▄░▓██  ▀█ ██▒▒██▒▒ ▓██░ ▒░▒███
 ░██▄▄▄▄██ ▒██▄█▓▒ ▒░██▄▄▄▄██ ▒▓▓▄ ▄██▒░▓█ ░██ ▒▓█  ▄    ░██░░▓█  ██▓▓██▒  ▐▌██▒░██░░ ▓██▓ ░ ▒▓█  ▄
 ▓█   ▓██▒▒██▒ ░  ░ ▓█   ▓██▒▒ ▓███▀ ░░▓█▒░██▓░▒████▒   ░██░░▒▓███▀▒▒██░   ▓██░░██░  ▒██▒ ░ ░▒████▒
 ▒▒   ▓▒█░▒▓▒░ ░  ░ ▒▒   ▓▒█░░ ░▒ ▒  ░ ▒ ░░▒░▒░░ ▒░ ░   ░▓   ░▒   ▒ ░ ▒░   ▒ ▒ ░▓    ▒ ░░   ░░ ▒░ ░
  ▒   ▒▒ ░░▒ ░       ▒   ▒▒ ░  ░  ▒    ▒ ░▒░ ░ ░ ░  ░    ▒ ░  ░   ░ ░ ░░   ░ ▒░ ▒ ░    ░     ░ ░  ░
  ░   ▒   ░░         ░   ▒   ░         ░  ░░ ░   ░       ▒ ░░ ░   ░    ░   ░ ░  ▒ ░  ░         ░
      ░  ░               ░  ░░ ░       ░  ░  ░   ░  ░    ░        ░          ░  ░              ░  ░
 -->

 <!--
 Licensed to the Apache Software Foundation (ASF) under one
 or more contributor license agreements.  See the NOTICE file
 distributed with this work for additional information
 regarding copyright ownership.  The ASF licenses this file
 to you under the Apache License, Version 2.0 (the
 "License"); you may not use this file except in compliance
 with the License.  You may obtain a copy of the License at

 http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing,
 software distributed under the License is distributed on an
 "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 KIND, either express or implied.  See the License for the
 specific language governing permissions and limitations
 under the License.
 -->

 <!DOCTYPE html>
 <html lang="en">
 <head>
 <meta charset="utf-8">

     <link rel="canonical" href="https://ignite.apache.org/features/machinelearning.html" />
     <!--#include virtual="/includes/scriptshead.html" -->

     <meta name="viewport" content="width=device-width, initial-scale=1.0">
     <meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate" />
     <meta http-equiv="Pragma" content="no-cache" />
     <meta http-equiv="Expires" content="0" />

     <title>Machine Learning - Apache Ignite</title>

     <meta name="description"
           content="Apache Ignite Machine Learning is a set of simple, scalable, and efficient APIs that
                         allow building predictive machine learning models at scale and to enable continuous learning."/>

     <!--#include virtual="/includes/styles.html" -->


 </head>
 <body>

     <!--#include virtual="/includes/header.html" -->
 <article>
     <header>
         <div class="container">
             <h1>Apache Ignite <strong>Machine Learning</strong></h1>
         </div>
     </header>
     <div class="container">
             <p>
                 Apache Ignite® Machine Learning (ML) is a set of simple, scalable, and efficient tools that
                 allow building predictive machine learning models without costly data transfers. The rationale for
                 adding machine and deep learning (DL) to Apache Ignite is quite simple.
                 Today's data scientists have to deal with two major factors that keep ML from mainstream adoption.
             </p>
             <h2>Problem #1: Constant Data Movement (ETL)</h2>

             <img class="diagram-right img-responsive" src="/images/svg-diagrams/machine_learning.svg" alt="Apache Ignite Machine Learning" />
             <p>
                 First, the models are trained and deployed (after the training is over) in different systems.
                 The data scientists have to wait for ETL or some other data transfer process to move the data
                 into a system like Apache Mahout or Apache Spark for a training purpose. Then they have to wait
                 while this process completes and redeploy the models in a production environment. The whole
                 process can take hours moving terabytes of data from one system to another. Moreover, the
                 training part usually happens over the old data set.
             </p>


             <h2>Problem #2: Lack of Horizontal Scalability</h2>

             <p>
                 The second factor relates to scalability. ML and DL algorithms have to process data sets that no
                 longer fit within a single server unit are continually growing. This requires data scientists to come
                 up with sophisticated solutions or turn to distributed computing platforms such as Apache Spark and
                 TensorFlow. However, those platforms mostly solve only a part of the puzzle, which is the models
                 training, making it a burden for the developers to decide how to deploy the models in production later.
             </p>

             <h2>Zero ETL and Massive Scalability</h2>

             <p>
                 Ignite Machine Learning relies on Ignite's multi-tier storage that brings massive scalability
                 for ML and DL tasks and eliminates the wait imposed by ETL between the different systems.
                 For instance, it allows users to run ML/DL training and inference directly on the data stored across
                 memory and disk in an Ignite cluster. Next, Ignite provides a host
                 of ML and DL algorithms that are optimized for Ignite's collocated distributed processing.
                 These implementations deliver in-memory speed and unlimited horizontal scalability when running
                 in place against massive data sets or incrementally against incoming data streams, without
                 requiring the data to be moved into another store. By eliminating the data movement and the
                 lengthy processing wait times, Ignite Machine learning enables continuous learning that can
                 improve decisions based on the latest data as it arrives in real-time.
             </p>

             <h2>Fault Tolerance and Continuous Learning</h2>
             <p>
                 Ignite Machine Learning is tolerant to node failures. This means that in the case of node
                 failures during the learning process, all recovery procedures will be transparent to the user,
                 learning processes won't be interrupted, and you will get results in the time similar to the case when
                 all nodes are up and running.
             </p>

             <div class="jumbotron jumbotron-fluid">
                 <div class="container">
                   <div class="title display-6">Learn More</div>
                   <hr class="my-4">
                   <div class="row">
                     <div class="col-sm-6">
                         <ul>
                             <li><a href="/docs/latest/machine-learning/machine-learning">Ignite Machine Learning Documentation <i class="fas fa-angle-double-right"></i></a></li>
                             <li><a href="/docs/latest/machine-learning/partition-based-dataset">Partition-Based Data Sets <i class="fas fa-angle-double-right"></i></a></li>
                         </ul>
                     </div>
                     <div class="col-sm-6">
                         <ul>
                             <li><a href="/features/tensorflow.html">Apache Ignite integration for TensorFlow <i class="fas fa-angle-double-right"></i></a></li>
                         </ul>
                     </div>
                 </div>
             </div>
         </div>

     </div>

 </article>

     <!--#include virtual="/includes/footer.html" -->

 <!--#include virtual="/includes/scripts.html" -->
 </body>
 </html>
	<!--
	▄▄▄ ██▓███ ▄▄▄ ▄████▄ ██░ ██ ▓█████ ██▓ ▄████ ███▄ █ ██▓▄▄▄█████▓▓█████
	▒████▄ ▓██░ ██▒▒████▄ ▒██▀ ▀█ ▓██░ ██▒▓█ ▀ ▓██▒ ██▒ ▀█▒ ██ ▀█ █ ▓██▒▓ ██▒ ▓▒▓█ ▀
	▒██ ▀█▄ ▓██░ ██▓▒▒██ ▀█▄ ▒▓█ ▄ ▒██▀▀██░▒███ ▒██▒▒██░▄▄▄░▓██ ▀█ ██▒▒██▒▒ ▓██░ ▒░▒███
	░██▄▄▄▄██ ▒██▄█▓▒ ▒░██▄▄▄▄██ ▒▓▓▄ ▄██▒░▓█ ░██ ▒▓█ ▄ ░██░░▓█ ██▓▓██▒ ▐▌██▒░██░░ ▓██▓ ░ ▒▓█ ▄
	▓█ ▓██▒▒██▒ ░ ░ ▓█ ▓██▒▒ ▓███▀ ░░▓█▒░██▓░▒████▒ ░██░░▒▓███▀▒▒██░ ▓██░░██░ ▒██▒ ░ ░▒████▒
	▒▒ ▓▒█░▒▓▒░ ░ ░ ▒▒ ▓▒█░░ ░▒ ▒ ░ ▒ ░░▒░▒░░ ▒░ ░ ░▓ ░▒ ▒ ░ ▒░ ▒ ▒ ░▓ ▒ ░░ ░░ ▒░ ░
	▒ ▒▒ ░░▒ ░ ▒ ▒▒ ░ ░ ▒ ▒ ░▒░ ░ ░ ░ ░ ▒ ░ ░ ░ ░ ░░ ░ ▒░ ▒ ░ ░ ░ ░ ░
	░ ▒ ░░ ░ ▒ ░ ░ ░░ ░ ░ ▒ ░░ ░ ░ ░ ░ ░ ▒ ░ ░ ░
	░ ░ ░ ░░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░
	-->

	<!--
	Licensed to the Apache Software Foundation (ASF) under one
	or more contributor license agreements. See the NOTICE file
	distributed with this work for additional information
	regarding copyright ownership. The ASF licenses this file
	to you under the Apache License, Version 2.0 (the
	"License"); you may not use this file except in compliance
	with the License. You may obtain a copy of the License at

	http://www.apache.org/licenses/LICENSE-2.0

	Unless required by applicable law or agreed to in writing,
	software distributed under the License is distributed on an
	"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	KIND, either express or implied. See the License for the
	specific language governing permissions and limitations
	under the License.
	-->

	<!DOCTYPE html>
	<html lang="en">
	<head>
	<meta charset="utf-8">

	<link rel="canonical" href="https://ignite.apache.org/features/machinelearning.html" />
	<!--#include virtual="/includes/scriptshead.html" -->

	<meta name="viewport" content="width=device-width, initial-scale=1.0">
	<meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate" />
	<meta http-equiv="Pragma" content="no-cache" />
	<meta http-equiv="Expires" content="0" />

	<title>Machine Learning - Apache Ignite</title>

	<meta name="description"
	content="Apache Ignite Machine Learning is a set of simple, scalable, and efficient APIs that
	allow building predictive machine learning models at scale and to enable continuous learning."/>

	<!--#include virtual="/includes/styles.html" -->


	</head>
	<body>

	<!--#include virtual="/includes/header.html" -->
	<article>
	<header>
	<div class="container">
	<h1>Apache Ignite <strong>Machine Learning</strong></h1>
	</div>
	</header>
	<div class="container">
	<p>
	Apache Ignite® Machine Learning (ML) is a set of simple, scalable, and efficient tools that
	allow building predictive machine learning models without costly data transfers. The rationale for
	adding machine and deep learning (DL) to Apache Ignite is quite simple.
	Today's data scientists have to deal with two major factors that keep ML from mainstream adoption.
	</p>
	<h2>Problem #1: Constant Data Movement (ETL)</h2>

	<img class="diagram-right img-responsive" src="/images/svg-diagrams/machine_learning.svg" alt="Apache Ignite Machine Learning" />
	<p>
	First, the models are trained and deployed (after the training is over) in different systems.
	The data scientists have to wait for ETL or some other data transfer process to move the data
	into a system like Apache Mahout or Apache Spark for a training purpose. Then they have to wait
	while this process completes and redeploy the models in a production environment. The whole
	process can take hours moving terabytes of data from one system to another. Moreover, the
	training part usually happens over the old data set.
	</p>


	<h2>Problem #2: Lack of Horizontal Scalability</h2>

	<p>
	The second factor relates to scalability. ML and DL algorithms have to process data sets that no
	longer fit within a single server unit are continually growing. This requires data scientists to come
	up with sophisticated solutions or turn to distributed computing platforms such as Apache Spark and
	TensorFlow. However, those platforms mostly solve only a part of the puzzle, which is the models
	training, making it a burden for the developers to decide how to deploy the models in production later.
	</p>

	<h2>Zero ETL and Massive Scalability</h2>

	<p>
	Ignite Machine Learning relies on Ignite's multi-tier storage that brings massive scalability
	for ML and DL tasks and eliminates the wait imposed by ETL between the different systems.
	For instance, it allows users to run ML/DL training and inference directly on the data stored across
	memory and disk in an Ignite cluster. Next, Ignite provides a host
	of ML and DL algorithms that are optimized for Ignite's collocated distributed processing.
	These implementations deliver in-memory speed and unlimited horizontal scalability when running
	in place against massive data sets or incrementally against incoming data streams, without
	requiring the data to be moved into another store. By eliminating the data movement and the
	lengthy processing wait times, Ignite Machine learning enables continuous learning that can
	improve decisions based on the latest data as it arrives in real-time.
	</p>

	<h2>Fault Tolerance and Continuous Learning</h2>
	<p>
	Ignite Machine Learning is tolerant to node failures. This means that in the case of node
	failures during the learning process, all recovery procedures will be transparent to the user,
	learning processes won't be interrupted, and you will get results in the time similar to the case when
	all nodes are up and running.
	</p>

	<div class="jumbotron jumbotron-fluid">
	<div class="container">
	<div class="title display-6">Learn More</div>
	<hr class="my-4">
	<div class="row">
	<div class="col-sm-6">
	<ul>
	<li><a href="/docs/latest/machine-learning/machine-learning">Ignite Machine Learning Documentation <i class="fas fa-angle-double-right"></i></a></li>
	<li><a href="/docs/latest/machine-learning/partition-based-dataset">Partition-Based Data Sets <i class="fas fa-angle-double-right"></i></a></li>
	</ul>
	</div>
	<div class="col-sm-6">
	<ul>
	<li><a href="/features/tensorflow.html">Apache Ignite integration for TensorFlow <i class="fas fa-angle-double-right"></i></a></li>
	</ul>
	</div>
	</div>
	</div>
	</div>

	</div>

	</article>

	<!--#include virtual="/includes/footer.html" -->

	<!--#include virtual="/includes/scripts.html" -->
	</body>
	</html>