| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> |
| <title>Apache Flink: Ecosystem</title> |
| <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"> |
| <link rel="icon" href="/favicon.ico" type="image/x-icon"> |
| |
| <!-- Bootstrap --> |
| <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.4.1/css/bootstrap.min.css"> |
| <link rel="stylesheet" href="/css/flink.css"> |
| <link rel="stylesheet" href="/css/syntax.css"> |
| |
| <!-- Blog RSS feed --> |
| <link href="/blog/feed.xml" rel="alternate" type="application/rss+xml" title="Apache Flink Blog: RSS feed" /> |
| |
| <!-- jQuery (necessary for Bootstrap's JavaScript plugins) --> |
| <!-- We need to load Jquery in the header for custom google analytics event tracking--> |
| <script src="/js/jquery.min.js"></script> |
| |
| <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries --> |
| <!-- WARNING: Respond.js doesn't work if you view the page via file:// --> |
| <!--[if lt IE 9]> |
| <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script> |
| <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script> |
| <![endif]--> |
| </head> |
| <body> |
| |
| |
| <!-- Main content. --> |
| <div class="container"> |
| <div class="row"> |
| |
| |
| <div id="sidebar" class="col-sm-3"> |
| |
| |
| <!-- Top navbar. --> |
| <nav class="navbar navbar-default"> |
| <!-- The logo. --> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1"> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <div class="navbar-logo"> |
| <a href="/"> |
| <img alt="Apache Flink" src="/img/flink-header-logo.svg" width="147px" height="73px"> |
| </a> |
| </div> |
| </div><!-- /.navbar-header --> |
| |
| <!-- The navigation links. --> |
| <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1"> |
| <ul class="nav navbar-nav navbar-main"> |
| |
| <!-- First menu section explains visitors what Flink is --> |
| |
| <!-- What is Stream Processing? --> |
| <!-- |
| <li><a href="/streamprocessing1.html">What is Stream Processing?</a></li> |
| --> |
| |
| <!-- What is Flink? --> |
| <li><a href="/flink-architecture.html">What is Apache Flink?</a></li> |
| |
| |
| |
| <!-- Use cases --> |
| <li><a href="/usecases.html">Use Cases</a></li> |
| |
| <!-- Powered by --> |
| <li><a href="/poweredby.html">Powered By</a></li> |
| |
| <!-- FAQ --> |
| <li><a href="/faq.html">FAQ</a></li> |
| |
| |
| <!-- Second menu section aims to support Flink users --> |
| |
| <!-- Downloads --> |
| <li><a href="/downloads.html">Downloads</a></li> |
| |
| <!-- Quickstart --> |
| <li> |
| <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.9/getting-started/tutorials/local_setup.html" target="_blank">Tutorials <small><span class="glyphicon glyphicon-new-window"></span></small></a> |
| </li> |
| |
| <!-- Documentation --> |
| <li class="dropdown"> |
| <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation<span class="caret"></span></a> |
| <ul class="dropdown-menu"> |
| <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.9" target="_blank">1.9 (Latest stable release) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| <li><a href="https://ci.apache.org/projects/flink/flink-docs-master" target="_blank">Master (Latest Snapshot) <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| </ul> |
| </li> |
| |
| <!-- getting help --> |
| <li><a href="/gettinghelp.html">Getting Help</a></li> |
| |
| <!-- Blog --> |
| <li><a href="/blog/"><b>Flink Blog</b></a></li> |
| |
| <!-- Ecosystem --> |
| <li class="active"><a href="/ecosystem.html">Ecosystem</a></li> |
| |
| |
| |
| <!-- Third menu section aim to support community and contributors --> |
| |
| <!-- Community --> |
| <li><a href="/community.html">Community & Project Info</a></li> |
| |
| <!-- Roadmap --> |
| <li><a href="/roadmap.html">Roadmap</a></li> |
| |
| <!-- Contribute --> |
| <li><a href="/contributing/how-to-contribute.html">How to Contribute</a></li> |
| |
| |
| <!-- GitHub --> |
| <li> |
| <a href="https://github.com/apache/flink" target="_blank">Flink on GitHub <small><span class="glyphicon glyphicon-new-window"></span></small></a> |
| </li> |
| |
| |
| |
| <!-- Language Switcher --> |
| <li> |
| |
| |
| <a href="/zh/ecosystem.html">中文版</a> |
| |
| |
| </li> |
| |
| </ul> |
| |
| <ul class="nav navbar-nav navbar-bottom"> |
| <hr /> |
| |
| <!-- Twitter --> |
| <li><a href="https://twitter.com/apacheflink" target="_blank">@ApacheFlink <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| |
| <!-- Visualizer --> |
| <li class=" hidden-md hidden-sm"><a href="/visualizer/" target="_blank">Plan Visualizer <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| |
| <hr /> |
| |
| <li><a href="https://apache.org" target="_blank">Apache Software Foundation <small><span class="glyphicon glyphicon-new-window"></span></small></a></li> |
| |
| <li> |
| <style> |
| .smalllinks:link { |
| display: inline-block !important; background: none; padding-top: 0px; padding-bottom: 0px; padding-right: 0px; min-width: 75px; |
| } |
| </style> |
| |
| <a class="smalllinks" href="https://www.apache.org/licenses/" target="_blank">License</a> <small><span class="glyphicon glyphicon-new-window"></span></small> |
| |
| <a class="smalllinks" href="https://www.apache.org/security/" target="_blank">Security</a> <small><span class="glyphicon glyphicon-new-window"></span></small> |
| |
| <a class="smalllinks" href="https://www.apache.org/foundation/sponsorship.html" target="_blank">Donate</a> <small><span class="glyphicon glyphicon-new-window"></span></small> |
| |
| <a class="smalllinks" href="https://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a> <small><span class="glyphicon glyphicon-new-window"></span></small> |
| </li> |
| |
| </ul> |
| </div><!-- /.navbar-collapse --> |
| </nav> |
| |
| </div> |
| <div class="col-sm-9"> |
| <div class="row-fluid"> |
| <div class="col-sm-12"> |
| <h1>Ecosystem</h1> |
| |
| <p><br /> |
| Apache Flink supports a broad ecosystem and works seamlessly with |
| many other data processing projects and frameworks. |
| <br /></p> |
| <div class="page-toc"> |
| <ul id="markdown-toc"> |
| <li><a href="#connectors" id="markdown-toc-connectors">Connectors</a></li> |
| <li><a href="#third-party-projects" id="markdown-toc-third-party-projects">Third-Party Projects</a></li> |
| </ul> |
| |
| </div> |
| |
| <h2 id="connectors">Connectors</h2> |
| |
| <p>Connectors provide code for interfacing with various third-party systems.</p> |
| |
| <p>Currently these systems are supported:</p> |
| |
| <ul> |
| <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/connectors/kafka.html" target="_blank">Apache Kafka</a> (sink/source)</li> |
| <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/connectors/elasticsearch.html" target="_blank">Elasticsearch 1.x / 2.x / 5.x / 6.x</a> (sink)</li> |
| <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/connectors/filesystem_sink.html" target="_blank">HDFS</a> (sink)</li> |
| <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/connectors/rabbitmq.html" target="_blank">RabbitMQ</a> (sink/source)</li> |
| <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/connectors/kinesis.html" target="_blank">Amazon Kinesis Streams</a> (sink/source)</li> |
| <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/connectors/twitter.html" target="_blank">Twitter</a> (source)</li> |
| <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/connectors/nifi.html" target="_blank">Apache NiFi</a> (sink/source)</li> |
| <li><a href="https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/connectors/cassandra.html" target="_blank">Apache Cassandra</a> (sink)</li> |
| <li><a href="https://github.com/apache/bahir-flink" target="_blank">Redis, Flume, and ActiveMQ (via Apache Bahir)</a> (sink)</li> |
| </ul> |
| |
| <p>To run an application using one of these connectors, additional third party |
| components are usually required to be installed and launched, e.g., the servers |
| for the message queues. Further instructions for these can be found in the |
| corresponding subsections.</p> |
| |
| <h2 id="third-party-projects">Third-Party Projects</h2> |
| |
| <p>This is a list of third party packages (i.e., libraries, system extensions, or examples) built on Flink. |
| The Flink community collects links to these packages but does not maintain them. |
| Thus, they do not belong to the Apache Flink project, and the community cannot give any support for them. |
| <strong>Is your project missing?</strong> |
| Please let us know on the <a href="/community.html#mailing-lists">user/dev mailing list</a>.</p> |
| |
| <p><strong>Apache Zeppelin</strong></p> |
| |
| <p><a href="https://zeppelin.apache.org/">Apache Zeppelin</a> is a web-based notebook that enables interactive data analytics and can be used with |
| <a href="https://zeppelin.apache.org/docs/latest/interpreter/flink.html">Flink as an execution engine</a>. |
| See also Jim Dowling’s <a href="http://www.slideshare.net/FlinkForward/jim-dowling-interactive-flink-analytics-with-hopsworks-and-zeppelin">Flink Forward talk</a> about Zeppelin on Flink.</p> |
| |
| <p><strong>Apache Mahout</strong></p> |
| |
| <p><a href="https://mahout.apache.org/">Apache Mahout</a> is a machine learning library that will feature Flink as an execution engine soon. |
| Check out Sebastian Schelter’s <a href="http://www.slideshare.net/FlinkForward/sebastian-schelter-distributed-machine-learing-with-the-samsara-dsl">Flink Forward talk</a> about Mahout-Samsara DSL.</p> |
| |
| <p><strong>Cascading</strong></p> |
| |
| <p><a href="http://www.cascading.org/cascading-flink/">Cascading</a> enables a user to build complex workflows easily on Flink and other execution engines. |
| <a href="https://github.com/dataArtisans/cascading-flink">Cascading on Flink</a> is built by <a href="http://data-artisans.com/">dataArtisans</a> and <a href="http://www.driven.io/">Driven, Inc</a>. |
| See Fabian Hueske’s <a href="http://www.slideshare.net/FlinkForward/fabian-hueske-training-cascading-on-flink">Flink Forward talk</a> for more details.</p> |
| |
| <p><strong>Apache Beam</strong></p> |
| |
| <p><a href="https://beam.apache.org/">Apache Beam</a> is an open-source, unified programming model that you can use to create a data processing pipeline. Flink is one of the back-ends supported by the Beam programming model.</p> |
| |
| <p><strong>GRADOOP</strong></p> |
| |
| <p><a href="http://dbs.uni-leipzig.de/en/research/projects/gradoop">GRADOOP</a> enables scalable graph analytics on top of Flink and is developed at Leipzig University. Check out Martin Junghanns’ <a href="http://www.slideshare.net/FlinkForward/martin-junghans-gradoop-scalable-graph-analytics-with-apache-flink">Flink Forward talk</a>.</p> |
| |
| <p><strong>BigPetStore</strong></p> |
| |
| <p><a href="https://github.com/apache/bigtop/tree/master/bigtop-bigpetstore">BigPetStore</a> is a benchmarking suite including a data generator and will be available for Flink soon. |
| See Suneel Marthi’s <a href="http://www.slideshare.net/FlinkForward/suneel-marthi-bigpetstore-flink-a-comprehensive-blueprint-for-apache-flink?ref=http://flink-forward.org/?session=tbd-3">Flink Forward talk</a> as preview.</p> |
| |
| <p><strong>FastR</strong></p> |
| |
| <p><a href="https://github.com/oracle/fastr">FastR</a> is an implemenation of the R language in Java. <a href="https://bitbucket.org/allr/fastr-flink/src/3535a9b7c7f208508d6afbcdaf1de7d04fa2bf79/README_FASTR_FLINK.md?at=default&fileviewer=file-view-default">FastR Flink</a> executes R workloads on top of Flink.</p> |
| |
| <p><strong>Apache SAMOA</strong></p> |
| |
| <p><a href="https://samoa.incubator.apache.org/">Apache SAMOA (incubating)</a> is a streaming ML library featuring Flink as an execution engine soon. Albert Bifet introduced SAMOA on Flink at his <a href="http://www.slideshare.net/FlinkForward/albert-bifet-apache-samoa-mining-big-data-streams-with-apache-flink?ref=http://flink-forward.org/?session=apache-samoa-mining-big-data-streams-with-apache-flink">Flink Forward talk</a>.</p> |
| |
| <p><strong>Alluxio</strong></p> |
| |
| <p><a href="http://www.alluxio.org/">Alluxio</a> is an open-source memory-speed virtual distributed storage that enables applications to efficiently share data and access data across different storage systems in a <a href="http://www.alluxio.org/docs/master/en/Unified-and-Transparent-Namespace.html">unified namespace</a>. Here is an example of <a href="http://www.alluxio.org/docs/master/en/Running-Flink-on-Alluxio.html">using Flink to access data through Alluxio</a>.</p> |
| |
| <p><strong>Python Examples on Flink</strong></p> |
| |
| <p>A <a href="https://github.com/wdm0006/flink-python-examples">collection of examples</a> using Apache Flink’s Python API.</p> |
| |
| <p><strong>WordCount Example in Clojure</strong></p> |
| |
| <p>A small <a href="https://github.com/mjsax/flink-external/tree/master/flink-clojure">WordCount example</a> on how to write a Flink program in Clojure.</p> |
| |
| <p><strong>Anomaly Detection and Prediction in Flink</strong></p> |
| |
| <p><a href="https://github.com/nupic-community/flink-htm">flink-htm</a> is a library for anomaly detection and prediction in Apache Flink. The algorithms are based on Hierarchical Temporal Memory (HTM) as implemented by the Numenta Platform for Intelligent Computing (NuPIC).</p> |
| |
| <p><strong>Apache Ignite</strong></p> |
| |
| <p><a href="https://ignite.apache.org">Apache Ignite</a> is a high-performance, integrated and distributed in-memory platform for computing and transacting on large-scale data sets in real-time. See <a href="https://github.com/apache/ignite/tree/master/modules/flink">Flink sink streaming connector</a> to inject data into Ignite cache.</p> |
| |
| <p><strong>Tink temporal graph library</strong></p> |
| |
| <p><a href="https://github.com/otherwise777/Temporal_Graph_library">Tink</a> is a temporal graph library built on top of Flink. It allows for temporal graph analytics like different interpretations of the shortest temporal path algorithm and metrics like temporal betweenness and temporal closeness. This library was the result of the <a href="http://www.win.tue.nl/~gfletche/ligtenberg2017.pdf">Thesis</a> of Wouter Ligtenberg.</p> |
| |
| <p><strong>FlinkK8sOperator</strong></p> |
| |
| <p><a href="https://github.com/lyft/flinkk8soperator">FlinkK8sOperator</a> is a Kubernetes operator that manages Flink applications on Kubernetes. The operator acts as control plane to manage the complete deployment lifecycle of the application.</p> |
| |
| |
| </div> |
| </div> |
| |
| </div> |
| </div> |
| |
| <hr /> |
| |
| <div class="row"> |
| <div class="footer text-center col-sm-12"> |
| <p>Copyright © 2014-2019 <a href="http://apache.org">The Apache Software Foundation</a>. All Rights Reserved.</p> |
| <p>Apache Flink, Flink®, Apache®, the squirrel logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.</p> |
| <p><a href="/privacy-policy.html">Privacy Policy</a> · <a href="/blog/feed.xml">RSS feed</a></p> |
| </div> |
| </div> |
| </div><!-- /.container --> |
| |
| <!-- Include all compiled plugins (below), or include individual files as needed --> |
| <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></script> |
| <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery.matchHeight/0.7.0/jquery.matchHeight-min.js"></script> |
| <script src="/js/codetabs.js"></script> |
| <script src="/js/stickysidebar.js"></script> |
| |
| <!-- Google Analytics --> |
| <script> |
| (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ |
| (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), |
| m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) |
| })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); |
| |
| ga('create', 'UA-52545728-1', 'auto'); |
| ga('send', 'pageview'); |
| </script> |
| </body> |
| </html> |
