| <!DOCTYPE html> |
| <html> |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| |
| <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"> |
| <link rel="icon" href="/favicon.ico" type="image/x-icon"> |
| |
| <title>Command Line Client</title> |
| |
| <!-- Bootstrap core CSS --> |
| <link href="/assets/css/bootstrap.min.css" rel="stylesheet"> |
| <!-- Bootstrap theme --> |
| <link href="/assets/css/bootstrap-theme.min.css" rel="stylesheet"> |
| |
| <!-- Custom styles for this template --> |
| <link rel="stylesheet" href="http://fortawesome.github.io/Font-Awesome/assets/font-awesome/css/font-awesome.css"> |
| <link href="/css/style.css" rel="stylesheet"> |
| <link href="/assets/css/owl.theme.css" rel="stylesheet"> |
| <link href="/assets/css/owl.carousel.css" rel="stylesheet"> |
| <script type="text/javascript" src="/assets/js/jquery.min.js"></script> |
| <script type="text/javascript" src="/assets/js/bootstrap.min.js"></script> |
| <script type="text/javascript" src="/assets/js/owl.carousel.min.js"></script> |
| <script type="text/javascript" src="/assets/js/storm.js"></script> |
| <!-- Just for debugging purposes. Don't actually copy these 2 lines! --> |
| <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]--> |
| |
| <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries --> |
| <!--[if lt IE 9]> |
| <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script> |
| <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script> |
| <![endif]--> |
| </head> |
| |
| |
| <body> |
| <header> |
| <div class="container-fluid"> |
| <div class="row"> |
| <div class="col-md-5"> |
| <a href="/index.html"><img src="/images/logo.png" class="logo" /></a> |
| </div> |
| <div class="col-md-5"> |
| |
| <h1>Version: 1.2.3</h1> |
| |
| </div> |
| <div class="col-md-2"> |
| <a href="/downloads.html" class="btn-std btn-block btn-download">Download</a> |
| </div> |
| </div> |
| </div> |
| </header> |
| <!--Header End--> |
| <!--Navigation Begin--> |
| <div class="navbar" role="banner"> |
| <div class="container-fluid"> |
| <div class="navbar-header"> |
| <button class="navbar-toggle" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse"> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| </div> |
| <nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation"> |
| <ul class="nav navbar-nav"> |
| <li><a href="/index.html" id="home">Home</a></li> |
| <li><a href="/getting-help.html" id="getting-help">Getting Help</a></li> |
| <li><a href="/about/integrates.html" id="project-info">Project Information</a></li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="documentation">Documentation <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| |
| |
| <li><a href="/releases/2.3.0/index.html">2.3.0</a></li> |
| |
| |
| |
| <li><a href="/releases/2.2.0/index.html">2.2.0</a></li> |
| |
| |
| |
| <li><a href="/releases/2.1.0/index.html">2.1.0</a></li> |
| |
| |
| |
| <li><a href="/releases/2.0.0/index.html">2.0.0</a></li> |
| |
| |
| |
| <li><a href="/releases/1.2.3/index.html">1.2.3</a></li> |
| |
| |
| </ul> |
| </li> |
| <li><a href="/talksAndVideos.html">Talks and Slideshows</a></li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown" id="contribute">Community <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li><a href="/contribute/Contributing-to-Storm.html">Contributing</a></li> |
| <li><a href="/contribute/People.html">People</a></li> |
| <li><a href="/contribute/BYLAWS.html">ByLaws</a></li> |
| </ul> |
| </li> |
| <li><a href="/2021/09/27/storm230-released.html" id="news">News</a></li> |
| </ul> |
| </nav> |
| </div> |
| </div> |
| |
| |
| |
| <div class="container-fluid"> |
| <h1 class="page-title">Command Line Client</h1> |
| <div class="row"> |
| <div class="col-md-12"> |
| <!-- Documentation --> |
| |
| <p class="post-meta"></p> |
| |
| <div class="documentation-content"><p>This page describes all the commands that are possible with the "storm" command line client. To learn how to set up your "storm" client to talk to a remote cluster, follow the instructions in <a href="Setting-up-development-environment.html">Setting up development environment</a>. See <a href="Classpath-handling.html">Classpath handling</a> for details on using external libraries in these commands.</p> |
| |
| <p>These commands are:</p> |
| |
| <ol> |
| <li>jar</li> |
| <li>sql</li> |
| <li>kill</li> |
| <li>activate</li> |
| <li>deactivate</li> |
| <li>rebalance</li> |
| <li>repl</li> |
| <li>classpath</li> |
| <li>localconfvalue</li> |
| <li>remoteconfvalue</li> |
| <li>nimbus</li> |
| <li>supervisor</li> |
| <li>ui</li> |
| <li>drpc</li> |
| <li>blobstore</li> |
| <li>dev-zookeeper</li> |
| <li>get-errors</li> |
| <li>heartbeats</li> |
| <li>kill_workers</li> |
| <li>list</li> |
| <li>logviewer</li> |
| <li>monitor</li> |
| <li>node-health-check</li> |
| <li>pacemaker</li> |
| <li>set_log_level</li> |
| <li>shell</li> |
| <li>upload-credentials</li> |
| <li>version</li> |
| <li>help</li> |
| </ol> |
| |
| <h3 id="jar">jar</h3> |
| |
| <p>Syntax: <code>storm jar topology-jar-path class ...</code></p> |
| |
| <p>Runs the main method of <code>class</code> with the specified arguments. The storm jars and configs in <code>~/.storm</code> are put on the classpath. The process is configured so that <a href="javadocs/org/apache/storm/StormSubmitter.html">StormSubmitter</a> will upload the jar at <code>topology-jar-path</code> when the topology is submitted.</p> |
| |
| <p>When you want to ship other jars which is not included to application jar, you can pass them to <code>--jars</code> option with comma-separated string. |
| For example, --jars "your-local-jar.jar,your-local-jar2.jar" will load your-local-jar.jar and your-local-jar2.jar. |
| And when you want to ship maven artifacts and its transitive dependencies, you can pass them to <code>--artifacts</code> with comma-separated string. You can also exclude some dependencies like what you're doing in maven pom. Please add exclusion artifacts with '^' separated string after the artifact. For example, <code>--artifacts "redis.clients:jedis:2.9.0,org.apache.kafka:kafka_2.10:0.8.2.2^org.slf4j:slf4j-log4j12"</code> will load jedis and kafka artifact and all of transitive dependencies but exclude slf4j-log4j12 from kafka.</p> |
| |
| <p>When you need to pull the artifacts from other than Maven Central, you can pass remote repositories to --artifactRepositories option with comma-separated string. Repository format is "<name>^<url>". '^' is taken as separator because URL allows various characters. For example, --artifactRepositories "jboss-repository^<a href="http://repository.jboss.com/maven2,HDPRepo%5Ehttp://repo.hortonworks.com/content/groups/public/">http://repository.jboss.com/maven2,HDPRepo^http://repo.hortonworks.com/content/groups/public/</a>" will add JBoss and HDP repositories for dependency resolver.</p> |
| |
| <p>Complete example of both options is here: <code>./bin/storm jar example/storm-starter/storm-starter-topologies-*.jar org.apache.storm.starter.RollingTopWords blobstore-remote2 remote --jars "./external/storm-redis/storm-redis-1.1.0.jar,./external/storm-kafka/storm-kafka-1.1.0.jar" --artifacts "redis.clients:jedis:2.9.0,org.apache.kafka:kafka_2.10:0.8.2.2^org.slf4j:slf4j-log4j12" --artifactRepositories "jboss-repository^http://repository.jboss.com/maven2,HDPRepo^http://repo.hortonworks.com/content/groups/public/"</code></p> |
| |
| <p>When you pass jars and/or artifacts options, StormSubmitter will upload them when the topology is submitted, and they will be included to classpath of both the process which runs the class, and also workers for that topology.</p> |
| |
| <h3 id="sql">sql</h3> |
| |
| <p>Syntax: <code>storm sql sql-file topology-name</code></p> |
| |
| <p>Compiles the SQL statements into a Trident topology and submits it to Storm.</p> |
| |
| <p><code>--jars</code> and <code>--artifacts</code>, and <code>--artifactRepositories</code> options available for jar are also applied to sql command. Please refer "help jar" to see how to use --jars and --artifacts, and --artifactRepositories options. You normally want to pass these options since you need to set data source to your sql which is an external storage in many cases.</p> |
| |
| <h3 id="kill">kill</h3> |
| |
| <p>Syntax: <code>storm kill topology-name [-w wait-time-secs]</code></p> |
| |
| <p>Kills the topology with the name <code>topology-name</code>. Storm will first deactivate the topology's spouts for the duration of the topology's message timeout to allow all messages currently being processed to finish processing. Storm will then shutdown the workers and clean up their state. You can override the length of time Storm waits between deactivation and shutdown with the -w flag.</p> |
| |
| <h3 id="activate">activate</h3> |
| |
| <p>Syntax: <code>storm activate topology-name</code></p> |
| |
| <p>Activates the specified topology's spouts.</p> |
| |
| <h3 id="deactivate">deactivate</h3> |
| |
| <p>Syntax: <code>storm deactivate topology-name</code></p> |
| |
| <p>Deactivates the specified topology's spouts.</p> |
| |
| <h3 id="rebalance">rebalance</h3> |
| |
| <p>Syntax: <code>storm rebalance topology-name [-w wait-time-secs] [-n new-num-workers] [-e component=parallelism]*</code></p> |
| |
| <p>Sometimes you may wish to spread out where the workers for a topology are running. For example, let's say you have a 10 node cluster running 4 workers per node, and then let's say you add another 10 nodes to the cluster. You may wish to have Storm spread out the workers for the running topology so that each node runs 2 workers. One way to do this is to kill the topology and resubmit it, but Storm provides a "rebalance" command that provides an easier way to do this.</p> |
| |
| <p>Rebalance will first deactivate the topology for the duration of the message timeout (overridable with the -w flag) and then redistribute the workers evenly around the cluster. The topology will then return to its previous state of activation (so a deactivated topology will still be deactivated and an activated topology will go back to being activated).</p> |
| |
| <p>The rebalance command can also be used to change the parallelism of a running topology. Use the -n and -e switches to change the number of workers or number of executors of a component respectively.</p> |
| |
| <h3 id="repl">repl</h3> |
| |
| <p>Syntax: <code>storm repl</code></p> |
| |
| <p>Opens up a Clojure REPL with the storm jars and configuration on the classpath. Useful for debugging.</p> |
| |
| <h3 id="classpath">classpath</h3> |
| |
| <p>Syntax: <code>storm classpath</code></p> |
| |
| <p>Prints the classpath used by the storm client when running commands.</p> |
| |
| <h3 id="localconfvalue">localconfvalue</h3> |
| |
| <p>Syntax: <code>storm localconfvalue conf-name</code></p> |
| |
| <p>Prints out the value for <code>conf-name</code> in the local Storm configs. The local Storm configs are the ones in <code>~/.storm/storm.yaml</code> merged in with the configs in <code>defaults.yaml</code>.</p> |
| |
| <h3 id="remoteconfvalue">remoteconfvalue</h3> |
| |
| <p>Syntax: <code>storm remoteconfvalue conf-name</code></p> |
| |
| <p>Prints out the value for <code>conf-name</code> in the cluster's Storm configs. The cluster's Storm configs are the ones in <code>$STORM-PATH/conf/storm.yaml</code> merged in with the configs in <code>defaults.yaml</code>. This command must be run on a cluster machine.</p> |
| |
| <h3 id="nimbus">nimbus</h3> |
| |
| <p>Syntax: <code>storm nimbus</code></p> |
| |
| <p>Launches the nimbus daemon. This command should be run under supervision with a tool like <a href="http://cr.yp.to/daemontools.html">daemontools</a> or <a href="http://mmonit.com/monit/">monit</a>. See <a href="Setting-up-a-Storm-cluster.html">Setting up a Storm cluster</a> for more information.</p> |
| |
| <h3 id="supervisor">supervisor</h3> |
| |
| <p>Syntax: <code>storm supervisor</code></p> |
| |
| <p>Launches the supervisor daemon. This command should be run under supervision with a tool like <a href="http://cr.yp.to/daemontools.html">daemontools</a> or <a href="http://mmonit.com/monit/">monit</a>. See <a href="Setting-up-a-Storm-cluster.html">Setting up a Storm cluster</a> for more information.</p> |
| |
| <h3 id="ui">ui</h3> |
| |
| <p>Syntax: <code>storm ui</code></p> |
| |
| <p>Launches the UI daemon. The UI provides a web interface for a Storm cluster and shows detailed stats about running topologies. This command should be run under supervision with a tool like <a href="http://cr.yp.to/daemontools.html">daemontools</a> or <a href="http://mmonit.com/monit/">monit</a>. See <a href="Setting-up-a-Storm-cluster.html">Setting up a Storm cluster</a> for more information.</p> |
| |
| <h3 id="drpc">drpc</h3> |
| |
| <p>Syntax: <code>storm drpc</code></p> |
| |
| <p>Launches a DRPC daemon. This command should be run under supervision with a tool like <a href="http://cr.yp.to/daemontools.html">daemontools</a> or <a href="http://mmonit.com/monit/">monit</a>. See <a href="Distributed-RPC.html">Distributed RPC</a> for more information.</p> |
| |
| <h3 id="blobstore">blobstore</h3> |
| |
| <p>Syntax: <code>storm blobstore cmd</code></p> |
| |
| <p>list [KEY...] - lists blobs currently in the blob store</p> |
| |
| <p>cat [-f FILE] KEY - read a blob and then either write it to a file, or STDOUT (requires read access).</p> |
| |
| <p>create [-f FILE] [-a ACL ...] [--replication-factor NUMBER] KEY - create a new blob. Contents comes from a FILE or STDIN. ACL is in the form [uo]:[username]:[r-][w-][a-] can be comma separated list.</p> |
| |
| <p>update [-f FILE] KEY - update the contents of a blob. Contents comes from a FILE or STDIN (requires write access).</p> |
| |
| <p>delete KEY - delete an entry from the blob store (requires write access).</p> |
| |
| <p>set-acl [-s ACL] KEY - ACL is in the form [uo]:[username]:[r-][w-][a-] can be comma separated list (requires admin access).</p> |
| |
| <p>replication --read KEY - Used to read the replication factor of the blob.</p> |
| |
| <p>replication --update --replication-factor NUMBER KEY where NUMBER > 0. It is used to update the replication factor of a blob.</p> |
| |
| <p>For example, the following would create a mytopo:data.tgz key using the data stored in data.tgz. User alice would have full access, bob would have read/write access and everyone else would have read access.</p> |
| |
| <p>storm blobstore create mytopo:data.tgz -f data.tgz -a u:alice:rwa,u:bob:rw,o::r</p> |
| |
| <p>See <a href="distcache-blobstore.html">Blobstore(Distcahce)</a> for more information.</p> |
| |
| <h3 id="dev-zookeeper">dev-zookeeper</h3> |
| |
| <p>Syntax: <code>storm dev-zookeeper</code></p> |
| |
| <p>Launches a fresh Zookeeper server using "dev.zookeeper.path" as its local dir and "storm.zookeeper.port" as its port. This is only intended for development/testing, the Zookeeper instance launched is not configured to be used in production.</p> |
| |
| <h3 id="get-errors">get-errors</h3> |
| |
| <p>Syntax: <code>storm get-errors topology-name</code></p> |
| |
| <p>Get the latest error from the running topology. The returned result contains the key value pairs for component-name and component-error for the components in error. The result is returned in json format.</p> |
| |
| <h3 id="heartbeats">heartbeats</h3> |
| |
| <p>Syntax: <code>storm heartbeats [cmd]</code></p> |
| |
| <p>list PATH - lists heartbeats nodes under PATH currently in the ClusterState. |
| get PATH - Get the heartbeat data at PATH</p> |
| |
| <h3 id="kill_workers">kill_workers</h3> |
| |
| <p>Syntax: <code>storm kill_workers</code></p> |
| |
| <p>Kill the workers running on this supervisor. This command should be run on a supervisor node. If the cluster is running in secure mode, then user needs to have admin rights on the node to be able to successfully kill all workers.</p> |
| |
| <h3 id="list">list</h3> |
| |
| <p>Syntax: <code>storm list</code></p> |
| |
| <p>List the running topologies and their statuses.</p> |
| |
| <h3 id="logviewer">logviewer</h3> |
| |
| <p>Syntax: <code>storm logviewer</code></p> |
| |
| <p>Launches the log viewer daemon. It provides a web interface for viewing storm log files. This command should be run under supervision with a tool like daemontools or monit.</p> |
| |
| <p>See Setting up a Storm cluster for more information.(<a href="http://storm.apache.org/documentation/Setting-up-a-Storm-cluster">http://storm.apache.org/documentation/Setting-up-a-Storm-cluster</a>)</p> |
| |
| <h3 id="monitor">monitor</h3> |
| |
| <p>Syntax: <code>storm monitor topology-name [-i interval-secs] [-m component-id] [-s stream-id] [-w [emitted | transferred]]</code></p> |
| |
| <p>Monitor given topology's throughput interactively. |
| One can specify poll-interval, component-id, stream-id, watch-item[emitted | transferred] |
| By default, |
| poll-interval is 4 seconds; |
| all component-ids will be list; |
| stream-id is 'default'; |
| watch-item is 'emitted';</p> |
| |
| <h3 id="node-health-check">node-health-check</h3> |
| |
| <p>Syntax: <code>storm node-health-check</code></p> |
| |
| <p>Run health checks on the local supervisor.</p> |
| |
| <h3 id="pacemaker">pacemaker</h3> |
| |
| <p>Syntax: <code>storm pacemaker</code></p> |
| |
| <p>Launches the Pacemaker daemon. This command should be run under |
| supervision with a tool like daemontools or monit.</p> |
| |
| <p>See Setting up a Storm cluster for more information.(<a href="http://storm.apache.org/documentation/Setting-up-a-Storm-cluster">http://storm.apache.org/documentation/Setting-up-a-Storm-cluster</a>)</p> |
| |
| <h3 id="set_log_level">set_log_level</h3> |
| |
| <p>Syntax: <code>storm set_log_level -l [logger name]=[log level][:optional timeout] -r [logger name] topology-name</code></p> |
| |
| <p>Dynamically change topology log levels</p> |
| |
| <p>where log level is one of: ALL, TRACE, DEBUG, INFO, WARN, ERROR, FATAL, OFF |
| and timeout is integer seconds.</p> |
| |
| <p>e.g. |
| ./bin/storm set_log_level -l ROOT=DEBUG:30 topology-name</p> |
| |
| <p>Set the root logger's level to DEBUG for 30 seconds</p> |
| |
| <p>./bin/storm set_log_level -l com.myapp=WARN topology-name</p> |
| |
| <p>Set the com.myapp logger's level to WARN for 30 seconds</p> |
| |
| <p>./bin/storm set_log_level -l com.myapp=WARN -l com.myOtherLogger=ERROR:123 topology-name</p> |
| |
| <p>Set the com.myapp logger's level to WARN indifinitely, and com.myOtherLogger to ERROR for 123 seconds</p> |
| |
| <p>./bin/storm set_log_level -r com.myOtherLogger topology-name</p> |
| |
| <p>Clears settings, resetting back to the original level</p> |
| |
| <h3 id="shell">shell</h3> |
| |
| <p>Syntax: <code>storm shell resourcesdir command args</code></p> |
| |
| <p>Makes constructing jar and uploading to nimbus for using non JVM languages</p> |
| |
| <p>eg: <code>storm shell resources/ python topology.py arg1 arg2</code></p> |
| |
| <h3 id="upload-credentials">upload-credentials</h3> |
| |
| <p>Syntax: <code>storm upload_credentials topology-name [credkey credvalue]*</code></p> |
| |
| <p>Uploads a new set of credentials to a running topology</p> |
| |
| <h3 id="version">version</h3> |
| |
| <p>Syntax: <code>storm version</code></p> |
| |
| <p>Prints the version number of this Storm release.</p> |
| |
| <h3 id="help">help</h3> |
| |
| <p>Syntax: <code>storm help [command]</code></p> |
| |
| <p>Print one help message or list of available commands</p> |
| </div> |
| |
| |
| </div> |
| </div> |
| </div> |
| <footer> |
| <div class="container-fluid"> |
| <div class="row"> |
| <div class="col-md-3"> |
| <div class="footer-widget"> |
| <h5>Meetups</h5> |
| <ul class="latest-news"> |
| |
| <li><a href="http://www.meetup.com/Apache-Storm-Apache-Kafka/">Apache Storm & Apache Kafka</a> <span class="small">(Sunnyvale, CA)</span></li> |
| |
| <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Apache Storm & Kafka Users</a> <span class="small">(Seattle, WA)</span></li> |
| |
| <li><a href="http://www.meetup.com/New-York-City-Storm-User-Group/">NYC Storm User Group</a> <span class="small">(New York, NY)</span></li> |
| |
| <li><a href="http://www.meetup.com/Bay-Area-Stream-Processing">Bay Area Stream Processing</a> <span class="small">(Emeryville, CA)</span></li> |
| |
| <li><a href="http://www.meetup.com/Boston-Storm-Users/">Boston Realtime Data</a> <span class="small">(Boston, MA)</span></li> |
| |
| <li><a href="http://www.meetup.com/storm-london">London Storm User Group</a> <span class="small">(London, UK)</span></li> |
| |
| <!-- <li><a href="http://www.meetup.com/Apache-Storm-Kafka-Users/">Seatle, WA</a> <span class="small">(27 Jun 2015)</span></li> --> |
| </ul> |
| </div> |
| </div> |
| <div class="col-md-3"> |
| <div class="footer-widget"> |
| <h5>About Apache Storm</h5> |
| <p>Apache Storm integrates with any queueing system and any database system. Apache Storm's spout abstraction makes it easy to integrate a new queuing system. Likewise, integrating Apache Storm with database systems is easy.</p> |
| </div> |
| </div> |
| <div class="col-md-3"> |
| <div class="footer-widget"> |
| <h5>First Look</h5> |
| <ul class="footer-list"> |
| <li><a href="/releases/current/Rationale.html">Rationale</a></li> |
| <li><a href="/releases/current/Tutorial.html">Tutorial</a></li> |
| <li><a href="/releases/current/Setting-up-development-environment.html">Setting up development environment</a></li> |
| <li><a href="/releases/current/Creating-a-new-Storm-project.html">Creating a new Apache Storm project</a></li> |
| </ul> |
| </div> |
| </div> |
| <div class="col-md-3"> |
| <div class="footer-widget"> |
| <h5>Documentation</h5> |
| <ul class="footer-list"> |
| <li><a href="/releases/current/index.html">Index</a></li> |
| <li><a href="/releases/current/javadocs/index.html">Javadoc</a></li> |
| <li><a href="/releases/current/FAQ.html">FAQ</a></li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| <hr/> |
| <div class="row"> |
| <div class="col-md-12"> |
| <p align="center">Copyright © 2019 <a href="http://www.apache.org">Apache Software Foundation</a>. All Rights Reserved. |
| <br>Apache Storm, Apache, the Apache feather logo, and the Apache Storm project logos are trademarks of The Apache Software Foundation. |
| <br>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p> |
| </div> |
| </div> |
| </div> |
| </footer> |
| <!--Footer End--> |
| <!-- Scroll to top --> |
| <span class="totop"><a href="#"><i class="fa fa-angle-up"></i></a></span> |
| |
| </body> |
| |
| </html> |
| |
