| <!DOCTYPE html> |
| <html> |
| |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| |
| <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"> |
| <link rel="icon" href="/favicon.ico" type="image/x-icon"> |
| |
| <title>Storm 0.9.0 Released</title> |
| |
| <!-- Bootstrap core CSS --> |
| <link href="/assets/css/bootstrap.min.css" rel="stylesheet"> |
| <!-- Bootstrap theme --> |
| <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet"> |
| |
| <!-- Custom styles for this template --> |
| <link rel="stylesheet" href="http://fortawesome.github.io/Font-Awesome/assets/font-awesome/css/font-awesome.css"> |
| <link href="/css/style.css" rel="stylesheet"> |
| <link href="/assets/css/owl.theme.css" rel="stylesheet"> |
| <link href="/assets/css/owl.carousel.css" rel="stylesheet"> |
| <script type="text/javascript" src="/assets/js/jquery.min.js"></script> |
| <script type="text/javascript" src="/assets/js/bootstrap.min.js"></script> |
| <script type="text/javascript" src="/assets/js/owl.carousel.min.js"></script> |
| <script type="text/javascript" src="/assets/js/storm.js"></script> |
| <!-- Just for debugging purposes. Don't actually copy these 2 lines! --> |
| <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]--> |
| |
| <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries --> |
| <!--[if lt IE 9]> |
| <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script> |
| <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script> |
| <![endif]--> |
| </head> |
| |
| |
| |
| <body> |
| |
| <header> |
| <div class="container-fluid"> |
| <div class="row"> |
| <div class="col-md-10"> |
| <a href="/index.html"><img src="/images/logo.png" class="logo" /></a> |
| </div> |
| <div class="col-md-2"> |
| <a href="/downloads.html" class="btn-std btn-block btn-download">Download</a> |
| </div> |
| </div> |
| </div> |
| </header> |
| <!--Header End--> |
| <!--Navigation Begin--> |
| <div class="navbar" role="banner"> |
| <div class="container-fluid"> |
| <div class="navbar-header"> |
| <button class="navbar-toggle" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse"> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| </div> |
| <nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation"> |
| <ul class="nav navbar-nav"> |
| <li><a href="/index.html" id="home">Home</a></li> |
| <li><a href="/getting-help.html" id="getting-help">Getting Help</a></li> |
| <li><a href="/about/integrates.html" id="project-info">Project Information</a></li> |
| <li><a href="/documentation.html" id="documentation">Documentation</a></li> |
| <li><a href="/talksAndVideos.html">Talks and Slideshows</a></li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="contribute">Community <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li><a href="/contribute/Contributing-to-Storm.html">Contributing</a></li> |
| <li><a href="/contribute/People.html">People</a></li> |
| <li><a href="/contribute/BYLAWS.html">ByLaws</a></li> |
| </ul> |
| </li> |
| <li><a href="/2015/11/05/storm096-released.html" id="news">News</a></li> |
| </ul> |
| </nav> |
| </div> |
| </div> |
| |
| |
| |
| <div class="container-fluid"> |
| <div class="row"> |
| <div class="col-md-12"> |
| <div class="row"> |
| <div class="col-md-3"> |
| <ul class="news" id="news-list"> |
| |
| <li><a href="/2015/11/05/storm096-released.html">Storm 0.9.6 released</a></li> |
| |
| <li><a href="/2015/11/05/storm0100-released.html">Storm 0.10.0 released</a></li> |
| |
| <li><a href="/2015/06/15/storm0100-beta-released.html">Storm 0.10.0 Beta Released</a></li> |
| |
| <li><a href="/2015/06/04/storm095-released.html">Storm 0.9.5 released</a></li> |
| |
| <li><a href="/2015/03/25/storm094-released.html">Storm 0.9.4 released</a></li> |
| |
| <li><a href="/2014/11/25/storm093-released.html">Storm 0.9.3 released</a></li> |
| |
| <li><a href="/2014/10/20/storm093-release-candidate.html">Storm 0.9.3 release candidate 1 available</a></li> |
| |
| <li><a href="/2014/06/25/storm092-released.html">Storm 0.9.2 released</a></li> |
| |
| <li><a href="/2014/06/17/contest-results.html">Storm Logo Contest Results</a></li> |
| |
| <li><a href="/2014/04/10/storm-logo-contest.html">Apache Storm Logo Contest</a></li> |
| |
| <li><a href="/2013/12/08/storm090-released.html">Storm 0.9.0 Released</a></li> |
| |
| <li><a href="/2013/01/11/storm082-released.html">Storm 0.8.2 released</a></li> |
| |
| <li><a href="/2012/09/06/storm081-released.html">Storm 0.8.1 released</a></li> |
| |
| <li><a href="/2012/08/02/storm080-released.html">Storm 0.8.0 and Trident released</a></li> |
| |
| </ul> |
| </div> |
| <div class="col-md-9" id="news-content"> |
| <h1 class="page-title"> |
| Storm 0.9.0 Released |
| </h1> |
| |
| <div class="row" style="margin: -15px;"> |
| <div class="col-md-12"> |
| <p class="text-muted credit pull-left">Posted on Dec 8, 2013 by P. Taylor Goetz</p> |
| <div class="pull-right"> |
| <a |
| href="https://twitter.com/share" |
| class="twitter-share-button" |
| data-count=none |
| >Tweet</a> |
| <script> !function(d,s,id){ |
| var js, |
| fjs=d.getElementsByTagName(s)[0], |
| p=/^http:/.test(d.location)?'http':'https'; |
| if(!d.getElementById(id)){ |
| js=d.createElement(s); |
| js.id=id; |
| js.src=p+'://platform.twitter.com/widgets.js'; |
| fjs.parentNode.insertBefore(js,fjs); |
| } |
| }(document, 'script', 'twitter-wjs'); |
| </script> |
| </div> |
| </div> |
| </div> |
| <div> |
| <p>We are pleased to announce that Storm 0.9.0 has been released and is available from <a href="/downloads.html">the downloads page</a>. This release represents an important milestone in the evolution of Storm.</p> |
| |
| <p>While a number of new features have been added, a key focus area for this release has been stability-related fixes. Though many users are successfully running work-in-progress builds for Storm 0.9.x in production, this release represents the most stable version to-date, and is highly recommended for everyone, especially users of 0.8.x versions.</p> |
| |
| <h2 id="netty-transport">Netty Transport</h2> |
| |
| <p>The first hightlight of this release is the new <a href="http://netty.io/index.html">Netty</a> Transport contributed by <a href="http://yahooeng.tumblr.com/">Yahoo! Engineering</a>. Storm's core network transport mechanism is now plugable, and Storm now comes with two implementations: The original 0MQ transport, and a new Netty-based implementation.</p> |
| |
| <p>In earlier versions, Storm relied solely on 0MQ for transport. Since 0MQ is a native library, it was highly platform-dependent and, at times, challenging to install properly. In addition, stability between versions varied widely between versions and only a relatively old 0MQ version (2.1.7) was certified to work with Storm.</p> |
| |
| <p>The Netty transport offers a pure Java alternative that eliminates Storm's dependency on native libraries. The Netty transport's performance is up to twice as fast as 0MQ, and it will open the door for authorization and authentication between worker processes. For an in-depth performance comparison of the 0MQ and Netty transports, see <a href="http://yahooeng.tumblr.com/post/64758709722/making-storm-fly-with-netty">this blog post</a> by Storm contributor <a href="https://github.com/revans2">Bobby Evans</a>.</p> |
| |
| <p>To configure Storm to use the Netty transport simply add the following to your <code>storm.yaml</code> configuration and adjust the values to best suit your use-case:</p> |
| <div class="highlight"><pre><code class="language-" data-lang="">storm.messaging.transport: "backtype.storm.messaging.netty.Context" |
| storm.messaging.netty.server_worker_threads: 1 |
| storm.messaging.netty.client_worker_threads: 1 |
| storm.messaging.netty.buffer_size: 5242880 |
| storm.messaging.netty.max_retries: 100 |
| storm.messaging.netty.max_wait_ms: 1000 |
| storm.messaging.netty.min_wait_ms: 100 |
| </code></pre></div> |
| <p>You can also write your own transport implementation by implementing the <a href="https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/backtype/storm/messaging/IContext.java"><code>backtype.storm.messaging.IContext</code></a> interface.</p> |
| |
| <h2 id="log-viewer-ui">Log Viewer UI</h2> |
| |
| <p>Storm now includes a helpful new feature for debugging and monitoring topologies: The <code>logviewer</code> daemon.</p> |
| |
| <p>In earlier versions of Storm, viewing worker logs involved determining a worker's location (host/port), typically through Storm UI, then <code>ssh</code>ing to that host and <code>tail</code>ing the corresponding worker log file. With the new log viewer. You can now easily access a specific worker's log in a web browser by clicking on a worker's port number right from Storm UI.</p> |
| |
| <p>The <code>logviewer</code> daemon runs as a separate process on Storm supervisor nodes. To enable the <code>logviewer</code> run the following command (under supervision) on your cluster's supervisor nodes:</p> |
| <div class="highlight"><pre><code class="language-" data-lang="">$ storm logviewer |
| </code></pre></div> |
| <h2 id="improved-windows-support">Improved Windows Support</h2> |
| |
| <p>In previous versions, running Storm on Microsoft Windows required installing third-party binaries (0MQ), modifying Storm's source, and adding Windows-specific scripts. With the addition of the platform-independent Netty transport, as well as numerous enhancements to make Storm more platform-independent, running Storm on Windows is easier than ever.</p> |
| |
| <h2 id="security-improvements">Security Improvements</h2> |
| |
| <p>Security, Authentication, and Authorization have been and will continue to be important focus areas for upcoming features. Storm 0.9.0 introduces an API for pluggable tuple serialization and a blowfish encryption based implementation for encrypting tuple data for sensitive use cases.</p> |
| |
| <h2 id="api-compatibility-and-upgrading">API Compatibility and Upgrading</h2> |
| |
| <p>For most Storm topology developers, upgrading to 0.9.0 is simply a matter of updating the <a href="https://clojars.org/storm">dependency</a>. Storm's core API has changed very little since the 0.8.2 release.</p> |
| |
| <p>On the devops side, when upgrading to a new Storm release, it is safest to clear any existing state (Zookeeper, <code>storm.local.dir</code>), prior to upgrading.</p> |
| |
| <h2 id="logging-changes">Logging Changes</h2> |
| |
| <p>Another important change in 0.9.0 has to do with logging. Storm has largely switched over to the <a href="http://www.slf4j.org">slf4j API</a> (backed by a <a href="http://logback.qos.ch">logback</a> logger implementation). Some Storm dependencies rely on the log4j API, so Storm currently depends on <a href="http://www.slf4j.org/legacy.html#log4j-over-slf4j">log4j-over-slf4j</a>.</p> |
| |
| <p>These changes have implications for existing topologies and topology components that use the log4j API.</p> |
| |
| <p>In general, and when possible, Storm topologies and topology components should use the <a href="http://www.slf4j.org">slf4j API</a> for logging.</p> |
| |
| <h2 id="thanks">Thanks</h2> |
| |
| <p>Special thanks are due to all those who have contributed to Storm -- whether through direct code contributions, documentation, bug reports, or helping other users on the mailing lists. Your efforts are much appreciated.</p> |
| |
| <h2 id="changelog">Changelog</h2> |
| |
| <ul> |
| <li>Update build configuration to force compatibility with Java 1.6</li> |
| <li>Fixed a netty client issue where sleep times for reconnection could be negative (thanks brndnmtthws)</li> |
| <li>Fixed an issue that would cause storm-netty unit tests to fail</li> |
| <li>Added configuration to limit ShellBolt internal _pendingWrites queue length (thanks xiaokang)</li> |
| <li>Fixed a a netty client issue where sleep times for reconnection could be negative (thanks brndnmtthws)</li> |
| <li>Fixed a display issue with system stats in Storm UI (thanks d2r)</li> |
| <li>Nimbus now does worker heartbeat timeout checks as soon as heartbeats are updated (thanks d2r)</li> |
| <li>The logviewer now determines log file location by examining the logback configuration (thanks strongh)</li> |
| <li>Allow tick tuples to work with the system bolt (thanks xumingming)</li> |
| <li>Add default configuration values for the netty transport and the ability to configure the number of worker threads (thanks revans2)</li> |
| <li>Added timeout to unit tests to prevent a situation where tests would hang indefinitely (thanks d2r)</li> |
| <li>Fixed an issue in the system bolt where local mode would not be detected accurately (thanks miofthena)</li> |
| <li>Fixed <code>storm jar</code> command to work properly when STORM_JAR_JVM_OPTS is not specified (thanks roadkill001)</li> |
| <li>All logging now done with slf4j</li> |
| <li>Replaced log4j logging system with logback</li> |
| <li>Logs are now limited to 1GB per worker (configurable via logging configuration file)</li> |
| <li>Build upgraded to leiningen 2.0</li> |
| <li>Revamped Trident spout interfaces to support more dynamic spouts, such as a spout who reads from a changing set of brokers</li> |
| <li>How tuples are serialized is now pluggable (thanks anfeng)</li> |
| <li>Added blowfish encryption based tuple serialization (thanks anfeng)</li> |
| <li>Have storm fall back to installed storm.yaml (thanks revans2)</li> |
| <li>Improve error message when Storm detects bundled storm.yaml to show the URL's for offending resources (thanks revans2)</li> |
| <li>Nimbus throws NotAliveException instead of FileNotFoundException from various query methods when topology is no longer alive (thanks revans2)</li> |
| <li>Escape HTML and Javascript appropriately in Storm UI (thanks d2r)</li> |
| <li>Storm's Zookeeper client now uses bounded exponential backoff strategy on failures</li> |
| <li>Automatically drain and log error stream of multilang subprocesses</li> |
| <li>Append component name to thread name of running executors so that logs are easier to read</li> |
| <li>Messaging system used for passing messages between workers is now pluggable (thanks anfeng)</li> |
| <li>Netty implementation of messaging (thanks anfeng)</li> |
| <li>Include topology id, worker port, and worker id in properties for worker processes, useful for logging (thanks d2r)</li> |
| <li>Tick tuples can now be scheduled using floating point seconds (thanks tscurtu)</li> |
| <li>Added log viewer daemon and links from UI to logviewers (thanks xiaokang)</li> |
| <li>DRPC server childopts now configurable (thanks strongh)</li> |
| <li>Default number of ackers to number of workers, instead of just one (thanks lyogavin)</li> |
| <li>Validate that Storm configs are of proper types/format/structure (thanks d2r)</li> |
| <li>FixedBatchSpout will now replay batches appropriately on batch failure (thanks ptgoetz)</li> |
| <li>Can set JAR_JVM_OPTS env variable to add jvm options when calling 'storm jar' (thanks srmelody)</li> |
| <li>Throw error if batch id for transaction is behind the batch id in the opaque value (thanks mrflip)</li> |
| <li>Sort topologies by name in UI (thanks jaked)</li> |
| <li>Added LoggingMetricsConsumer to log all metrics to a file, by default not enabled (thanks mrflip)</li> |
| <li>Add prepare(Map conf) method to TopologyValidator (thanks ankitoshniwal)</li> |
| <li>Bug fix: Supervisor provides full path to workers to logging config rather than relative path (thanks revans2) </li> |
| <li>Bug fix: Call ReducerAggregator#init properly when used within persistentAggregate (thanks lorcan)</li> |
| <li>Bug fix: Set component-specific configs correctly for Trident spouts</li> |
| </ul> |
| |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| <footer> |
| <div class="container-fluid"> |
| <div class="row"> |
| <div class="col-md-3"> |
| <div class="footer-widget"> |
| <h5>Meetups</h5> |
| <ul class="latest-news"> |
| |
| <li><a href="http://www.meetup.com/Apache-Storm-Apache-Kafka/">Apache Storm & Apache Kafka</a> <span class="small">(Sunnyvale, CA)</span></li> |
| |
| <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Apache Storm & Kafka Users</a> <span class="small">(Seattle, WA)</span></li> |
| |
| <li><a href="http://www.meetup.com/New-York-City-Storm-User-Group/">NYC Storm User Group</a> <span class="small">(New York, NY)</span></li> |
| |
| <li><a href="http://www.meetup.com/Bay-Area-Stream-Processing">Bay Area Stream Processing</a> <span class="small">(Emeryville, CA)</span></li> |
| |
| <li><a href="http://www.meetup.com/Boston-Storm-Users/">Boston Realtime Data</a> <span class="small">(Boston, MA)</span></li> |
| |
| <li><a href="http://www.meetup.com/storm-london">London Storm User Group</a> <span class="small">(London, UK)</span></li> |
| |
| <!-- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Seatle, WA</a> <span class="small">(27 Jun 2015)</span></li> --> |
| </ul> |
| </div> |
| </div> |
| <div class="col-md-3"> |
| <div class="footer-widget"> |
| <h5>About Storm</h5> |
| <p>Storm integrates with any queueing system and any database system. Storm's spout abstraction makes it easy to integrate a new queuing system. Likewise, integrating Storm with database systems is easy.</p> |
| </div> |
| </div> |
| <div class="col-md-3"> |
| <div class="footer-widget"> |
| <h5>First Look</h5> |
| <ul class="footer-list"> |
| <li><a href="/documentation/Rationale.html">Rationale</a></li> |
| <li><a href="/tutorial.html">Tutorial</a></li> |
| <li><a href="/documentation/Setting-up-development-environment.html">Setting up development environment</a></li> |
| <li><a href="/documentation/Creating-a-new-Storm-project.html">Creating a new Storm project</a></li> |
| </ul> |
| </div> |
| </div> |
| <div class="col-md-3"> |
| <div class="footer-widget"> |
| <h5>Documentation</h5> |
| <ul class="footer-list"> |
| <li><a href="/doc-index.html">Index</a></li> |
| <li><a href="/documentation.html">Manual</a></li> |
| <li><a href="https://storm.apache.org/javadoc/apidocs/index.html">Javadoc</a></li> |
| <li><a href="/documentation/FAQ.html">FAQ</a></li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| <hr/> |
| <div class="row"> |
| <div class="col-md-12"> |
| <p align="center">Copyright © 2015 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. |
| <br>Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. |
| <br>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p> |
| </div> |
| </div> |
| </div> |
| </footer> |
| <!--Footer End--> |
| <!-- Scroll to top --> |
| <span class="totop"><a href="#"><i class="fa fa-angle-up"></i></a></span> |
| |
| </body> |
| |
| </html> |
| |
