blob: 2d3dcd0088aabaff9d40f0d04275119392aaef9d [file] [log] [blame]
extend ../_components/base.pug
block pagetitle
| Accelerate Existing Hadoop Deployments
block css
link(rel="stylesheet", href="../css/native-persistence.css?ver=" + config.version)
link(rel="stylesheet", href="../css/compute-apis.css?ver=" + config.version)
link(rel="stylesheet", href="../css/digital-hub.css?ver=" + config.version)
link(rel="stylesheet", href="../css/hadoop.css?ver=" + config.version)
block main
- global.pageHref = "usecases"
- config.hdrClassName = "hdr__blue"
include ../_components/header.pug
h1.h1.innerhero__h1 Accelerate Existing Hadoop Deployments
span.with-apache With Apache Ignite
Achieve the performance acceleration of Hadoop-based<br> application with Ignite as a high-performance data access layer
a.button.innerhero__button(href="") Start Coding
img.innerhero__pic.innerhero__pic--hadoop(src="/public/img/usecases/hadoop/hero-image.svg", alt="hero-image")
// /.innerhero
h2.compute2__h2 Benefits Of Using Apache Ignite
h3.fz20.compute2item__title Real-time analytics
p.compute2__text.base2__text Apache Ignite enables real-time analytics across Apache Hadoop operational and historical data silos.
h3.fz20.compute2item__title Low-latency and high-throughput operations
p.compute2__text.base2__text Ignite enables low-latency and high-throughput access while Hadoop continues to be used for long-running OLAP workloads.
// /.compute2
h2.doop3__h2.h5 How Does Apache Ignite Acceleration Work?
p.doop3__text To achieve the performance acceleration of Hadoop-based systems, deploy Ignite as a separate distributed storage that maintains the data sets required for your low-latency operations or real-time reports
h2.doop3__h2.h5 There are 3 basic steps:
.fz20.doop3__number 01
p.doop3__subtext Depending on the data volume and available memory capacity, you can enable<a href="/arch/native-persistence.html" target="_blank"> Ignite native persistence</a> to store historical data sets on disk while dedicating a memory space for operational records. You can continue to use Hadoop as storage for less frequently used data or for long-running and ad-hoc analytical queries.
.fz20.doop3__number 02
p.doop3__subtext Your applications and services should use Ignite native APIs to process the data residing in the in-memory cluster. Ignite provides SQL, compute (aka. map-reduce), and machine learning APIs for various data processing needs.
.fz20.doop3__number 03
p.doop3__subtext Consider using Apache Spark DataFrames APIs if an application needs to run federated or cross-database queries across Ignite and Hadoop clusters. Ignite is <a href="/use-cases/spark-acceleration.html" target="_blank">integrated with Spark</a>, which natively supports Hive/Hadoop. Cross-database queries should be considered only for a limited number of scenarios when neither Ignite nor Hadoop contains the entire data set.
img.doop3__image(src="/public/img/usecases/hadoop/image.svg", alt="image")
// /.doop3
h2.doop4__h2.h4 How Can You Split Data And Operations Between Ignite And Hadoop?
p.doop4__text Use Apache Ignite for tasks that require:<br> Low-latency response time <span class="doop4__grey">(microseconds, milliseconds, seconds)</span> High throughput operations <span class="doop4__grey">(thousands and millions of operations per second)</span> <br> Real-time processing.
p.doop4__text Continue using Apache Hadoop for: <br> High-latency operations <span class="doop4__grey">(dozens of seconds, minutes, hours)</span><br>— Batch processing
// /.doop4
h2.h4.doop5__h2 5 Steps To Implement The Architecture In Practice
.doop5__number.h4 01
h4.doop5__title Download and install Apache Ignite to your system.
.doop5__number.h4 02
h4.doop5__title Select a list of operations for Ignite. The best operations are those that require low-latency response time, high-throughput, and real-time analytics.
.doop5__number.h4 03
p.doop5__text <span class="doop5__title">Consider enabling Ignite native persistence,</span> or use Ignite as a pure in-memory cache, or in-memory data grid that persists changes to Hadoop or another external database.
.doop5__number.h4 04
h4.doop5__title Update your applications Ensure they use Ignite native APIs to process Ignite data and Spark for federated queries.
.doop5__number.h4 05
.doop5__titleend If you need to replicate changes between Ignite and Hadoop clusters, use existing change-data-capture solutions:
p Debezium<br>Kafka
p.doop5__middle GridGain Data Lake Accelerator<br>Oracle GoldenGate
p.doop5__end To write-through changes to Hadoop directly,<br> implement <a href="" target="_blank">Ignite's CacheStore</a> interface.
img(src="/public/img/features/native-rocket.svg", alt="").nativebotblock__icon
span Ready to Start?
p.nativebotblock__text Discover our quick start guide and build your first<br> application in 5-10 minutes
a.nativebotblock__link.arrowlink(href="", target="_blank") Quick Start Guide
img(src="/public/img/features/native-docs.svg", alt="").nativebotblock__icon
span Want to Learn More?
p.nativebotblock__text Read the Apache Spark acceleration article
a.nativebotblock__link.arrowlink(href="/use-cases/spark-acceleration.html") Apache Spark Acceleration Article