From the beginning, Heron was envisioned as a new kind of stream processing system, built to meet the most demanding of technological requirements, to handle even the most massive of workloads, and to meet the needs of organizations of all sizes and degrees of complexity. Amongst these requirements:
To meet these requirements, a few core design goals have guided---and continue to guide---Heron's development:
Heron was designed to serve a wide range of requirements, use cases, platforms, programming languages and so on. In order to suit varying---and often unforeseeable---needs, Heron provides support for mulitple:
Due to its fundamentally modular character, Heron is remarkably easy to extend to meet your needs, with simple APIs that you can use to add support for new schedulers, programming languages (for topologies), topology uploaders, etc.
Heron topologies should be process based rather than thread based, and each process should run in isolation for the sake of easy debugging, profiling, and troubleshooting.
Heron topologies should use only those resources that they are initially allocated and never exceed those bounds. This makes Heron safe to run in shared infrastructure.
Although Heron has a Functional API that we recommend for all future topology development, Heron is fully API and data model compatible with Apache Storm, making it easy for developers to transition from Storm to Heron.
In a distributed system like Heron, there are no guarantees that all system components will execute at the same speed. Heron has built-in back pressure mechanisms to ensure that topologies can self-adjust in case components lag.
Heron provides support for at-most-once, at-least-once, and effectively-once processing semantics.