| extend ../_components/base.pug |
| |
| block pagetitle |
| title Apache Hadoop Performance Acceleration |
| meta(name="description", content="Achieve the performance acceleration of Hadoop-based systems by deploying Ignite as an in-memory computing platform designated for low-latency, high-throughput and real-time operations while Hadoop continues to be used for long-running OLAP workloads.") |
| link(rel="canonical", href="https://ignite.apache.org/use-cases/hadoop-acceleration.html") |
| |
| meta(property="og:title", content="Apache Hadoop Performance Acceleration") |
| meta(property="og:type", content="article") |
| meta(property="og:url", content="https://ignite.apache.org/use-cases/hadoop-acceleration.html") |
| meta(property="og:image", content="/img/og-pic.png") |
| meta(property="og:description", content="Achieve the performance acceleration of Hadoop-based systems by deploying Ignite as an in-memory computing platform designated for low-latency, high-throughput and real-time operations while Hadoop continues to be used for long-running OLAP workloads.") |
| |
| block css |
| link(rel="stylesheet", href="../css/native-persistence.css?ver=" + config.version) |
| link(rel="stylesheet", href="../css/compute-apis.css?ver=" + config.version) |
| link(rel="stylesheet", href="../css/digital-hub.css?ver=" + config.version) |
| link(rel="stylesheet", href="../css/hadoop.css?ver=" + config.version) |
| |
| |
| |
| |
| |
| block main |
| - global.pageHref = "usecases" |
| - config.hdrClassName = "hdr__blue" |
| include ../_components/header.pug |
| |
| |
| section.innerhero |
| .container.innerhero__cont |
| .innerhero__main |
| h1.h1.innerhero__h1 Accelerate Existing Hadoop Deployments |
| <br> |
| span.with-apache With Apache Ignite |
| .innerhero__descr.pt-2.h5. |
| Accelerate the performance of Hadoop-based applications with Ignite <br>as a high-performance data access layer |
| .innerhero__action |
| a.button.innerhero__button(href="https://ignite.apache.org/docs/latest/index") Start Coding |
| img.innerhero__pic.innerhero__pic--hadoop(src="/img/usecases/hadoop/hero-image.svg", alt="hero-image") |
| // /.innerhero |
| |
| |
| |
| |
| section.compute2 |
| .container |
| .doop2__block |
| h2.compute2__h2 Benefits Of Using Apache Ignite |
| .compute2__grid.flexi.hub2__grid.doop2__grid |
| .compute2item.hub2item.doop2__item |
| .compute2-points__item.fz20 |
| .compute2item__block |
| h3.fz20.compute2item__title Real-time analytics |
| p.compute2__text.base2__text Apache Ignite enables real-time analytics across Apache Hadoop operational and historical data silos. |
| |
| .compute2item.hub2item.doop2__item |
| .compute2-points__item.fz20 |
| .compute2item__block |
| h3.fz20.compute2item__title Low-latency and high-throughput operations |
| p.compute2__text.base2__text Ignite enables low-latency and high-throughput access while Hadoop continues to be used for long-running OLAP workloads. |
| |
| |
| // /.compute2 |
| |
| section.doop3 |
| .container |
| .doop3__block.flexi |
| .doop3__info |
| h2.doop3__h2.h5 How Does Apache Ignite Acceleration Work? |
| p.doop3__text To achieve the performance acceleration of Hadoop-based systems, deploy Ignite as a separate distributed storage that maintains the data sets required for your low-latency operations or real-time reports |
| h2.doop3__h2.h5 There are 3 basic steps: |
| .fz20.doop3__number 01 |
| p.doop3__subtext Depending on the data volume and available memory capacity, you can enable<a href="/arch/native-persistence.html" target="_blank"> Ignite native persistence</a> to store historical data sets on disk while dedicating a memory space for operational records. |
| p.doop3__subtext.pt-1 You can continue to use Hadoop as storage for less frequently-used data or for long-running and ad-hoc analytical queries. |
| .fz20.doop3__number 02 |
| p.doop3__subtext Your applications and services should use Ignite native APIs to process the data residing in the in-memory cluster. Ignite provides SQL, compute (aka. map-reduce), and machine learning APIs for various data processing needs. |
| .fz20.doop3__number 03 |
| p.doop3__subtext Consider using Apache Spark DataFrames APIs if an application needs to run federated or cross-database queries across Ignite and Hadoop clusters. |
| p.doop3__subtext.pt-1 Ignite is <a href="/use-cases/spark-acceleration.html" target="_blank">integrated with Spark</a>, which natively supports Hive/Hadoop. Cross-database queries should be considered only for a limited number of scenarios when neither Ignite nor Hadoop contains the entire data set. |
| img.doop3__image(src="/img/usecases/hadoop/image.svg", alt="image") |
| // /.doop3 |
| |
| section.doop4 |
| .container |
| h2.doop4__h2.h4 How Can You Split Data And Operations Between Ignite And Hadoop? |
| .doop4__block |
| .doop4__item |
| p.doop4__text Use Apache Ignite for tasks that require:<br> – Low-latency response time <span class="doop4__grey">(microseconds, milliseconds, seconds)</span> |
| p.doop4__text.pt-1 – High-throughput operations <span class="doop4__grey">(thousands and millions of operations per second)</span> <br>– Real-time processing |
| .doop4__item |
| p.doop4__text Continue using Apache Hadoop for: <br>— High-latency operations <span class="doop4__grey">(dozens of seconds, minutes, hours)</span><br>— Batch processing |
| // /.doop4 |
| |
| section.doop5 |
| .container |
| h2.h4.doop5__h2 5 Steps To Implement The Architecture In Practice |
| .doop5__blocks |
| .doop5__block |
| .doop5__item.post1 |
| .doop5__number.h4 01 |
| .doop5__title Download and install Apache Ignite to your system. |
| .doop5__item.post2 |
| .doop5__number.h4 02 |
| .doop5__title Select a list of operations for Ignite. |
| p.doop5__text.pt-2 The best operations are those that require low-latency response time, high-throughput, and real-time analytics. |
| .doop5__item.post3 |
| .doop5__number.h4 03 |
| p.doop5__text <span class="doop5__title">Consider enabling Ignite native persistence,</span> or use Ignite as a pure in-memory cache, or in-memory data grid that persists changes to Hadoop or another external database. |
| .doop5__item.post4 |
| .doop5__number.h4 04 |
| .doop5__title Update your applications |
| p.doop5__text.pt-2 Ensure they use Ignite native APIs to process Ignite data and Spark for federated queries. |
| .doop5__item.post5 |
| .doop5__number.h4 05 |
| .doop5__titleend If you need to replicate changes between Ignite and Hadoop clusters, use existing change-data-capture solutions: |
| .doop5__part.flexi |
| p Debezium<br>Kafka |
| p.doop5__middle GridGain Data Lake Accelerator<br>Oracle GoldenGate |
| p.doop5__end To write-through changes to Hadoop directly,<br> implement <a href="https://ignite.apache.org/docs/latest/persistence/external-storage" target="_blank">Ignite's CacheStore</a> interface. |
| |
| |
| |
| |
| |
| |
| |
| |
| section.native-bottom.container |
| .native-bottom__grid |
| article.nativebotblock |
| .h4.nativebotblock__title |
| img(src="/img/features/native-rocket.svg", alt="").nativebotblock__icon |
| span Ready to Start? |
| p.nativebotblock__text Discover our quick start guide and build your first<br> application in 5-10 minutes |
| a.nativebotblock__link.arrowlink(href="https://ignite.apache.org/docs/latest/", target="_blank") Quick Start Guide |
| article.nativebotblock.nativebotblock--learn |
| .h4.nativebotblock__title |
| img(src="/img/features/native-docs.svg", alt="").nativebotblock__icon |
| span Want to Learn More? |
| p.nativebotblock__text Read the Apache Spark acceleration article |
| a.nativebotblock__link.arrowlink(href="/use-cases/spark-acceleration.html") Apache Spark Acceleration Article |
| |
| |
| |
| |
| |
| |
