Storm has a variety of configurations for tweaking the behavior of nimbus, supervisors, and running topologies. Some configurations are system configurations and cannot be modified on topology by topology basis, whereas other configurations can be modified per topology.
Every configuration has a default value defined in defaults.yaml in the Storm codebase. You can override these configurations by defining a storm.yaml in the classpath of Nimbus and the supervisors. Finally, you can define a topology-specific configuration that you submit along with your topology when using StormSubmitter. However, the topology-specific configuration can only override configs prefixed with “TOPOLOGY”.
Storm 0.7.0 and onwards lets you override configuration on a per-bolt/per-spout basis. The only configurations that can be overridden this way are:
The Java API lets you specify component specific configurations in two ways:
getComponentConfiguration
in any spout or bolt and return the component-specific configuration map.setSpout
and setBolt
in TopologyBuilder
return an object with methods addConfiguration
and addConfigurations
that can be used to override the configurations for the component.The preference order for configuration values is defaults.yaml < storm.yaml < topology specific configuration < internal component specific configuration < external component specific configuration.
In almost all cases configuration for a bolt or a spout should be done through setters on the bolt or spout implementation and not the topology conf. In some rare cases, it may make sense to expose topology wide configurations that are not currently a part of Config or DaemonConfig such as when writing a custom scheduler or a plugin to some part of storm. In those cases you can create your own class like Config but implements Validated. Any public static final String
field declared in this class will be treated as a config and annotations from the org.apache.storm.validation.ConfigValidationAnnotations
class can be used to enforce what is stored in that config. To let the validator know about this class you need to treat the class like a service that will be loaded through a ServiceLoader for the Validated class and include a META-INF/services/org.apache.storm.validation.Validated
file in your jar that holds the name of your Config class.
Resources: