Heron is realtime analytics platform developed by Twitter. It is the direct successor of Apache Storm, built to be backwards compatible with Storm‘s topology API but with a wide array of architectural improvements. For more on the architecture and design goals of Heron, see Heron’s documentation, especially The Architecture of Heron and Heron Topologies.
For information beyond the documentation:
Currently, we are working on ensuring that Heron is easy to install and run in a Mesos Cluster in AWS, Mesos/Aurora in AWS and in local laptop. Will post updates as we progress.
Heron's OSS documentation is not yet publicly hosted. You can view the documentation locally, however, using the instructions below.
Heron‘s OSS documentation is currently built using GitBook. To build and run the docs locally, you’ll need to install it. GitBook depends on NodeJS and npm.
Once you've installed GitBook, you should run gitbook install
to make sure that all the required modules are fetched. The documentation for GitBook can be found here.
The Heron docs are currently under very active development. Any help with correcting factual errors, adding missing material, improving examples, and more would be greatly appreciated.
To build the docs, simply run gitbook build
in the docs
directory. The resulting artifacts are in docs/_book
. The _book
directory is ignored by Git.
To run the docs locally, navigate to the docs
directory in the Heron repo and run gitbook serve
. This will start up a local web server serving the docs on localhost:4000
. You can specify a different port using the --port
flag.
You can build a PDF of the docs by running gitbook pdf
in the docs
directory. This will produce a PDF called book.pdf
.
Note: To build PDFs locally, you will need to install Calibre.
If you'd like to add a custom cover page for the PDF version of the docs, create a JPEG and save it as cover.jpg
.
GitBook enables you to set project-wide variables that can be embedded into text. These variables can be set in docs/book.json
, within the variables
object. Currently, there is one variable for the general version of the Heron docs (version
) and three variables corresponding to versions of the Heron API docs: scheduler_api_version
, topology_api_version
, and metrics_api_version
. They are currently all set to 0.1.0
, which will need to change when versioning gets settled.
You can then use those variables in text like this:
The current version of the metrics API docs is {{book.metrics_api_version}}.
Please note that project-level variables don‘t really work in normal code blocks. You’ll have to embed special HTML to make them work. This is an unfortunate but currently unavoidable limitation of GitBook. Examples of a workaround can be found in docs/contributors/custom-{scheduler,sink}.md
.
There are some basic components of the docs that you should be aware of:
index.md
is the main page of the docs.SUMMARY.md
is the index for the docs. If you'd like to add new pages to the docs or change how things are structured in the left-hand nav, make those changes here and GitBook will take care of the rest.book.json
provides project-level configurationnode_modules
directory houses fetched dependencies (and is ignored by Git)._book
directory houses all generated artifacts. The docs
directory holds generated HTML, while the gitbook
directory holds all static assets.concepts
, contributors
, developers
, and operators
).