| <!DOCTYPE html> |
| <html> |
| |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| |
| <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"> |
| <link rel="icon" href="/favicon.ico" type="image/x-icon"> |
| |
| <title>Apache Storm 2.0.0 Released</title> |
| |
| <!-- Bootstrap core CSS --> |
| <link href="/assets/css/bootstrap.min.css" rel="stylesheet"> |
| <!-- Bootstrap theme --> |
| <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet"> |
| |
| <!-- Custom styles for this template --> |
| <link rel="stylesheet" href="http://fortawesome.github.io/Font-Awesome/assets/font-awesome/css/font-awesome.css"> |
| <link href="/css/style.css" rel="stylesheet"> |
| <link href="/assets/css/owl.theme.css" rel="stylesheet"> |
| <link href="/assets/css/owl.carousel.css" rel="stylesheet"> |
| <script type="text/javascript" src="/assets/js/jquery.min.js"></script> |
| <script type="text/javascript" src="/assets/js/bootstrap.min.js"></script> |
| <script type="text/javascript" src="/assets/js/owl.carousel.min.js"></script> |
| <script type="text/javascript" src="/assets/js/storm.js"></script> |
| <!-- Just for debugging purposes. Don't actually copy these 2 lines! --> |
| <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]--> |
| |
| <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries --> |
| <!--[if lt IE 9]> |
| <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script> |
| <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script> |
| <![endif]--> |
| </head> |
| |
| |
| |
| <body> |
| |
| <header> |
| <div class="container-fluid"> |
| <div class="row"> |
| <div class="col-md-5"> |
| <a href="/index.html"><img src="/images/logo.png" class="logo" /></a> |
| </div> |
| <div class="col-md-5"> |
| |
| </div> |
| <div class="col-md-2"> |
| <a href="/downloads.html" class="btn-std btn-block btn-download">Download</a> |
| </div> |
| </div> |
| </div> |
| </header> |
| <!--Header End--> |
| <!--Navigation Begin--> |
| <div class="navbar" role="banner"> |
| <div class="container-fluid"> |
| <div class="navbar-header"> |
| <button class="navbar-toggle" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse"> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| </div> |
| <nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation"> |
| <ul class="nav navbar-nav"> |
| <li><a href="/index.html" id="home">Home</a></li> |
| <li><a href="/getting-help.html" id="getting-help">Getting Help</a></li> |
| <li><a href="/about/integrates.html" id="project-info">Project Information</a></li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="documentation">Documentation <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| |
| |
| <li><a href="/releases/2.3.0/index.html">2.3.0</a></li> |
| |
| |
| |
| <li><a href="/releases/2.2.0/index.html">2.2.0</a></li> |
| |
| |
| |
| <li><a href="/releases/2.1.0/index.html">2.1.0</a></li> |
| |
| |
| |
| <li><a href="/releases/2.0.0/index.html">2.0.0</a></li> |
| |
| |
| |
| <li><a href="/releases/1.2.3/index.html">1.2.3</a></li> |
| |
| |
| </ul> |
| </li> |
| <li><a href="/talksAndVideos.html">Talks and Slideshows</a></li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="contribute">Community <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li><a href="/contribute/Contributing-to-Storm.html">Contributing</a></li> |
| <li><a href="/contribute/People.html">People</a></li> |
| <li><a href="/contribute/BYLAWS.html">ByLaws</a></li> |
| </ul> |
| </li> |
| <li><a href="/2021/09/27/storm230-released.html" id="news">News</a></li> |
| </ul> |
| </nav> |
| </div> |
| </div> |
| |
| |
| |
| <div class="container-fluid"> |
| <div class="row"> |
| <div class="col-md-12"> |
| <div class="row"> |
| <div class="col-md-3"> |
| <ul class="news" id="news-list"> |
| |
| <li><a href="/2021/09/27/storm230-released.html">Apache Storm 2.3.0 Released</a></li> |
| |
| <li><a href="/2020/06/30/storm220-released.html">Apache Storm 2.2.0 Released</a></li> |
| |
| <li><a href="/2019/10/31/storm210-released.html">Apache Storm 2.1.0 Released</a></li> |
| |
| <li><a href="/2019/07/18/storm123-released.html">Apache Storm 1.2.3 Released</a></li> |
| |
| <li><a href="/2019/05/30/storm200-released.html">Apache Storm 2.0.0 Released</a></li> |
| |
| <li><a href="/2018/06/04/storm122-released.html">Apache Storm 1.2.2 Released</a></li> |
| |
| <li><a href="/2018/06/04/storm113-released.html">Apache Storm 1.1.3 Released</a></li> |
| |
| <li><a href="/2018/02/19/storm121-released.html">Apache Storm 1.2.1 Released</a></li> |
| |
| <li><a href="/2018/02/15/storm120-released.html">Apache Storm 1.2.0 Released</a></li> |
| |
| <li><a href="/2018/02/15/storm112-released.html">Apache Storm 1.1.2 Released</a></li> |
| |
| <li><a href="/2018/02/14/storm106-released.html">Apache Storm 1.0.6 Released</a></li> |
| |
| <li><a href="/2017/09/15/storm105-released.html">Apache Storm 1.0.5 Released</a></li> |
| |
| <li><a href="/2017/08/01/storm111-released.html">Apache Storm 1.1.1 Released</a></li> |
| |
| <li><a href="/2017/07/28/storm104-released.html">Apache Storm 1.0.4 Released</a></li> |
| |
| <li><a href="/2017/03/29/storm110-released.html">Apache Storm 1.1.0 released</a></li> |
| |
| <li><a href="/2017/02/14/storm103-released.html">Apache Storm 1.0.3 Released</a></li> |
| |
| <li><a href="/2016/09/14/storm0102-released.html">Apache Storm 0.10.2 Released</a></li> |
| |
| <li><a href="/2016/09/07/storm097-released.html">Apache Storm 0.9.7 Released</a></li> |
| |
| <li><a href="/2016/08/10/storm102-released.html">Apache Storm 1.0.2 Released</a></li> |
| |
| <li><a href="/2016/05/06/storm101-released.html">Apache Storm 1.0.1 Released</a></li> |
| |
| <li><a href="/2016/05/05/storm0101-released.html">Apache Storm 0.10.1 Released</a></li> |
| |
| <li><a href="/2016/04/12/storm100-released.html">Apache Storm 1.0.0 released</a></li> |
| |
| <li><a href="/2015/11/05/storm096-released.html">Apache Storm 0.9.6 released</a></li> |
| |
| <li><a href="/2015/11/05/storm0100-released.html">Apache Storm 0.10.0 released</a></li> |
| |
| <li><a href="/2015/06/15/storm0100-beta-released.html">Apache Storm 0.10.0 Beta Released</a></li> |
| |
| <li><a href="/2015/06/04/storm095-released.html">Apache Storm 0.9.5 released</a></li> |
| |
| <li><a href="/2015/03/25/storm094-released.html">Apache Storm 0.9.4 released</a></li> |
| |
| <li><a href="/2014/11/25/storm093-released.html">Apache Storm 0.9.3 released</a></li> |
| |
| <li><a href="/2014/10/20/storm093-release-candidate.html">Apache Storm 0.9.3 release candidate 1 available</a></li> |
| |
| <li><a href="/2014/06/25/storm092-released.html">Apache Storm 0.9.2 released</a></li> |
| |
| <li><a href="/2014/06/17/contest-results.html">Apache Storm Logo Contest Results</a></li> |
| |
| <li><a href="/2014/05/27/round1-results.html">Logo Contest - Round 1 Results</a></li> |
| |
| <li><a href="/2014/04/29/logo-jlee3.html">Logo Entry No. 11 - Jennifer Lee</a></li> |
| |
| <li><a href="/2014/04/29/logo-jlee2.html">Logo Entry No. 10 - Jennifer Lee</a></li> |
| |
| <li><a href="/2014/04/29/logo-jlee1.html">Logo Entry No. 9 - Jennifer Lee</a></li> |
| |
| <li><a href="/2014/04/27/logo-sasili.html">Logo Entry No. 8 - Stefano Asili</a></li> |
| |
| <li><a href="/2014/04/27/logo-cboustead.html">Logo Entry No. 7 - Calum Boustead</a></li> |
| |
| <li><a href="/2014/04/23/logo-abartos.html">Logo Entry No. 6 - Alec Bartos</a></li> |
| |
| <li><a href="/2014/04/22/logo-zsayari.html">Logo Entry No. 5 - Ziba Sayari</a></li> |
| |
| <li><a href="/2014/04/21/logo-rmarshall.html">Logo Entry No. 4 - Richard Brownlie-Marshall</a></li> |
| |
| <li><a href="/2014/04/19/logo-ssuleman.html">Logo Entry No. 3- Shaan Shiv Suleman</a></li> |
| |
| <li><a href="/2014/04/17/logo-squinones.html">Logo Entry No. 2 - Samuel Quiñones</a></li> |
| |
| <li><a href="/2014/04/17/logo-pforrest.html">Logo Entry No. 1 - Patricia Forrest</a></li> |
| |
| <li><a href="/2014/04/10/storm-logo-contest.html">Apache Storm Logo Contest</a></li> |
| |
| <li><a href="/2013/12/08/storm090-released.html">Apache Storm 0.9.0 Released</a></li> |
| |
| <li><a href="/2013/01/11/storm082-released.html">Apache Storm 0.8.2 released</a></li> |
| |
| <li><a href="/2012/09/06/storm081-released.html">Apache Storm 0.8.1 released</a></li> |
| |
| <li><a href="/2012/08/02/storm080-released.html">Apache Storm 0.8.0 and Trident released</a></li> |
| |
| </ul> |
| </div> |
| <div class="col-md-9" id="news-content"> |
| <h1 class="page-title"> |
| Apache Storm 2.0.0 Released |
| </h1> |
| |
| <div class="row" style="margin: -15px;"> |
| <div class="col-md-12"> |
| <p class="text-muted credit pull-left">Posted on May 30, 2019 by Apache Storm PMC</p> |
| <div class="pull-right"> |
| <a |
| href="https://twitter.com/share" |
| class="twitter-share-button" |
| data-count=none |
| >Tweet</a> |
| <script> !function(d,s,id){ |
| var js, |
| fjs=d.getElementsByTagName(s)[0], |
| p=/^http:/.test(d.location)?'http':'https'; |
| if(!d.getElementById(id)){ |
| js=d.createElement(s); |
| js.id=id; |
| js.src=p+'://platform.twitter.com/widgets.js'; |
| fjs.parentNode.insertBefore(js,fjs); |
| } |
| }(document, 'script', 'twitter-wjs'); |
| </script> |
| </div> |
| </div> |
| </div> |
| <div> |
| <p>The Apache Storm community is pleased to announce that version 2.0.0 has been released and is available from <a href="/downloads.html">the downloads page</a>. This release represents a major milestone and accomplishment by the Apache Storm community.</p> |
| |
| <p>Apache Storm 2.0.0 includes significant improvements in terms of performance, new features, and integration with external systems. In the coming weeks members will post a series of deep dive articles covering new features improvements. In this post we'll highlight some of the key features and changes in this release.</p> |
| |
| <p>The full list of changes in this release can be found <a href="https://www.apache.org/dist/storm/apache-storm-2.0.0/RELEASE_NOTES.html">here</a>.</p> |
| |
| <h2 id="new-architecture-implemented-in-java">New Architecture Implemented in Java</h2> |
| |
| <p>In previous releases a large part of Apache Storm's core functionality was implemented in Clojure. Apache Storm 2.0.0 has been rearchitected with it's core functionality implemented in pure Java. The new Java-based implementation has improved performance significantly, and made Apache Storm's internal APIs more maintainable and extensible. While Apache Storm's Clojure implementation served it well for many years, it was often cited as a barrier for entry to new contributors. Apache Storm's codebase is now more accessible to developers who don't want to learn Clojure in order to contribute.</p> |
| |
| <h2 id="new-high-performance-core">New High Performance Core:</h2> |
| |
| <p>Apache Storm 2.0.0 introduces a new core featuring a leaner threading model, a blazing fast messaging subsystem and a lightweight back pressure model. It is designed to push boundaries on throughput, latency and energy consumption while maintaining backward compatibility. The design was motivated by the observation that existing hardware remains capable of much more than what the best streaming engines can deliver. Apache Storm 2.0 is the first streaming engine to break the 1 microsecond latency barrier.</p> |
| |
| <h2 id="new-streams-api">New Streams API</h2> |
| |
| <p>Apache Storm 2.0.0 introduces a new typed API for expressing streaming computations more easily using functional style operations. It builds on top of the Apache Storm's core spouts and bolt APIs and automatically fuses multiple operations together to optimize the pipeline.</p> |
| |
| <p>For more details and examples see the <a href="https://github.com/apache/storm/blob/master/docs/Stream-API.md">Stream API documentation</a>.</p> |
| |
| <h2 id="windowing-enhancements">Windowing Enhancements</h2> |
| |
| <p>Apache Storm 2.0.0's Windowing API can save/restore the window state to the configured state backend so that larger continuous windows can be supported. The window boundaries can now be accessed via the APIs.</p> |
| |
| <p>For more details see <a href="https://github.com/apache/storm/blob/master/docs/Windowing.md#stateful-windowing">stateful windowing documentation</a>.</p> |
| |
| <h2 id="kafka-integration-changes">Kafka Integration Changes</h2> |
| |
| <h3 id="removal-of-storm-kafka">Removal of Storm-Kafka</h3> |
| |
| <p>The most significant change to Apache Storm's Kafka integration since 1.x, is that storm-kafka has been removed. The module was deprecated a while back, due to Kafka's deprecation of the underlying client library. Users will have to move to the storm-kafka-client module, which uses Kafka's ´kafka-clients´ library for integration.</p> |
| |
| <p>For the most part, the migration to storm-kafka-client is straightforward. The documentation for storm-kafka-client contains a helpful mapping between the old and new spout configurations. If you are using any of the storm-kafka spouts, you will need to migrate offset checkpoints to the new spout, to avoid the new spout starting from scratch on your partitions. Apache Storm provides a helper tool to do this which can be found <a href="https://github.com/apache/storm/tree/master/external/storm-kafka-migration">here</a>.</p> |
| |
| <p>When performing a migration, you should stop your topology, run the migration tool, then redeploy your topology with the storm-kafka-client spout.</p> |
| |
| <h3 id="move-to-using-the-kafkaconsumer-assign-api">Move to Using the KafkaConsumer.assign API</h3> |
| |
| <p>Storm-kafka-client in Apache Storm 1.x allowed you to use Kafka's own mechanism to manage which spout tasks were responsible for which partitions. This mechanism was a poor fit for Apache Storm, and was deprecated in 1.2.0. It has been <a href="https://issues.apache.org/jira/browse/STORM-2542">removed entirely in 2.0</a>.</p> |
| |
| <p>The storm-kafka-client Subscription interface has also been removed. It offered too limited control over the subscription behavior. It has been replaced with the TopicFilter and ManualPartitioner interfaces. Unless you were using a custom Subscription implementation, this will likely not |
| affect you. If you were using a custom Subscription, <a href="https://github.com/apache/storm/blob/master/docs/storm-kafka-client.md#manual-partition-assigment-advanced">the storm-kafka-client documentation describes how to customize assignment</a>.</p> |
| |
| <h3 id="other-kafka-highlights">Other Kafka Highlights</h3> |
| |
| <ul> |
| <li>The KafkaBolt now allows you to <a href="https://issues.apache.org/jira/browse/STORM-3175">specify a callback that will be called when a batch is written to Kafka</a>.</li> |
| <li>The FirstPollOffsetStrategy behavior has been made consistent between the non-Trident and Trident spouts. It is now always the case that EARLIEST/LATEST only take effect on topology redeploy, and not when a worker restarts <a href="https://issues.apache.org/jira/browse/STORM-2990">https://issues.apache.org/jira/browse/STORM-2990</a>.</li> |
| <li>Storm-kafka-client now has a transactional non-opaque Trident spout <a href="https://issues.apache.org/jira/browse/STORM-2974">https://issues.apache.org/jira/browse/STORM-2974</a>.</li> |
| <li>There are new example modules for storm-kafka-client. You can find them <a href="https://github.com/apache/storm/tree/master/examples/storm-kafka-client-examples">here</a>.</li> |
| <li>Deprecated methods in KafkaSpoutConfig have been removed. If you are using one of the deprecated methods, check the Javadoc for the latest 1.2.x release, which describes the replacement for each method.</li> |
| </ul> |
| |
| <h2 id="eol-for-1-0-x">EOL for 1.0.x</h2> |
| |
| <p>With the release of 2.0.0 the 1.0.x version line will no longer be maintained. 1.0.x users are strongly encouraged to upgrade to a more recent release.</p> |
| |
| <h2 id="move-to-java-8">Move to Java 8</h2> |
| |
| <p>Java 7 support has been dropped, and Apache Storm 2.0.0 requires Java 8.</p> |
| |
| <h2 id="reorganization-of-apache-storm-maven-artifacts">Reorganization of Apache Storm Maven artifacts</h2> |
| |
| <p>The storm-core artifact has <a href="https://issues.apache.org/jira/browse/STORM-2441">been split</a> into client and server-facing parts. Topology jars should depend on the following artifact as of Apache Storm 2.0.0:</p> |
| <div class="highlight"><pre><code class="language-" data-lang=""><groupId>org.apache.storm</groupId> |
| <artifactId>storm-client</artifactId> |
| <version>2.0.0</version> |
| <scope>provided</scope> |
| </code></pre></div> |
| <p>Projects using <code>LocalCluster</code> for testing will additionally need to depend on the Apache Storm server jar:</p> |
| <div class="highlight"><pre><code class="language-" data-lang=""><groupId>org.apache.storm</groupId> |
| <artifactId>storm-server</artifactId> |
| <version>2.0.0</version> |
| <scope>test</scope> |
| </code></pre></div> |
| <h2 id="stay-tuned">Stay Tuned</h2> |
| |
| <p>Keep an eye on the Apache Storm blog for additional posts by Apache Storm contributors for more in-depth discussions of new features in Apache Storm 2.0.0 including:</p> |
| |
| <ul> |
| <li>SQL enhancements</li> |
| <li>Metrics improvements</li> |
| <li>New security features such as nimbus admin groups, delegation tokens, and optional impersonation</li> |
| <li>Module restructuring & dependency resolution improvements</li> |
| <li>API improvements</li> |
| <li>Lambda support</li> |
| <li>Resource Aware Scheduler enhancements</li> |
| <li>New admin commands for debugging cluster state</li> |
| </ul> |
| |
| <h2 id="thanks">Thanks</h2> |
| |
| <p>Special thanks are due to all those who have contributed to Apache Storm -- whether through direct code contributions, documentation, bug reports, or helping other users on the mailing lists. Your efforts are much appreciated.</p> |
| |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| <footer> |
| <div class="container-fluid"> |
| <div class="row"> |
| <div class="col-md-3"> |
| <div class="footer-widget"> |
| <h5>Meetups</h5> |
| <ul class="latest-news"> |
| |
| <li><a href="http://www.meetup.com/Apache-Storm-Apache-Kafka/">Apache Storm & Apache Kafka</a> <span class="small">(Sunnyvale, CA)</span></li> |
| |
| <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Apache Storm & Kafka Users</a> <span class="small">(Seattle, WA)</span></li> |
| |
| <li><a href="http://www.meetup.com/New-York-City-Storm-User-Group/">NYC Storm User Group</a> <span class="small">(New York, NY)</span></li> |
| |
| <li><a href="http://www.meetup.com/Bay-Area-Stream-Processing">Bay Area Stream Processing</a> <span class="small">(Emeryville, CA)</span></li> |
| |
| <li><a href="http://www.meetup.com/Boston-Storm-Users/">Boston Realtime Data</a> <span class="small">(Boston, MA)</span></li> |
| |
| <li><a href="http://www.meetup.com/storm-london">London Storm User Group</a> <span class="small">(London, UK)</span></li> |
| |
| <!-- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Seatle, WA</a> <span class="small">(27 Jun 2015)</span></li> --> |
| </ul> |
| </div> |
| </div> |
| <div class="col-md-3"> |
| <div class="footer-widget"> |
| <h5>About Apache Storm</h5> |
| <p>Apache Storm integrates with any queueing system and any database system. Apache Storm's spout abstraction makes it easy to integrate a new queuing system. Likewise, integrating Apache Storm with database systems is easy.</p> |
| </div> |
| </div> |
| <div class="col-md-3"> |
| <div class="footer-widget"> |
| <h5>First Look</h5> |
| <ul class="footer-list"> |
| <li><a href="/releases/current/Rationale.html">Rationale</a></li> |
| <li><a href="/releases/current/Tutorial.html">Tutorial</a></li> |
| <li><a href="/releases/current/Setting-up-development-environment.html">Setting up development environment</a></li> |
| <li><a href="/releases/current/Creating-a-new-Storm-project.html">Creating a new Apache Storm project</a></li> |
| </ul> |
| </div> |
| </div> |
| <div class="col-md-3"> |
| <div class="footer-widget"> |
| <h5>Documentation</h5> |
| <ul class="footer-list"> |
| <li><a href="/releases/current/index.html">Index</a></li> |
| <li><a href="/releases/current/javadocs/index.html">Javadoc</a></li> |
| <li><a href="/releases/current/FAQ.html">FAQ</a></li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| <hr/> |
| <div class="row"> |
| <div class="col-md-12"> |
| <p align="center">Copyright © 2019 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. |
| <br>Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. |
| <br>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p> |
| </div> |
| </div> |
| </div> |
| </footer> |
| <!--Footer End--> |
| <!-- Scroll to top --> |
| <span class="totop"><a href="#"><i class="fa fa-angle-up"></i></a></span> |
| |
| </body> |
| |
| </html> |
| |
