| <!DOCTYPE html> |
| <!-- |
| | Generated by Apache Maven Doxia Site Renderer 1.7.4 at 2018-04-02 |
| | Rendered using Apache Maven Fluido Skin 1.5 |
| --> |
| <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
| <head> |
| <meta charset="UTF-8" /> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0" /> |
| <meta name="Date-Revision-yyyymmdd" content="20180402" /> |
| <meta http-equiv="Content-Language" content="en" /> |
| <title>Apache Streams – Architecture</title> |
| <link rel="stylesheet" href="./css/apache-maven-fluido-1.5.min.css" /> |
| <link rel="stylesheet" href="./css/site.css" /> |
| <link rel="stylesheet" href="./css/print.css" media="print" /> |
| |
| |
| <script type="text/javascript" src="./js/apache-maven-fluido-1.5.min.js"></script> |
| |
| <script> |
| (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ |
| (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), |
| m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) |
| })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); |
| |
| ga('create', 'UA-77087101-1', 'auto'); |
| ga('send', 'pageview'); |
| </script> |
| </head> |
| <body class="topBarEnabled"> |
| |
| |
| |
| |
| |
| |
| <a href="https://github.com/apache/streams"> |
| <img style="position: absolute; top: 0; right: 0; border: 0; z-index: 10000;" |
| src="https://s3.amazonaws.com/github/ribbons/forkme_right_darkblue_121621.png" |
| alt="Fork me on GitHub"> |
| </a> |
| |
| |
| |
| |
| |
| <div id="topbar" class="navbar navbar-fixed-top navbar-inverse"> |
| <div class="navbar-inner"> |
| <div class="container"><div class="nav-collapse"> |
| |
| |
| <ul class="nav"> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown">Overview <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| |
| <li> <a href="index.html" title="Overview">Overview</a> |
| </li> |
| |
| <li> <a href="architecture.html" title="Architecture">Architecture</a> |
| </li> |
| |
| <li> <a href="concepts.html" title="Concepts">Concepts</a> |
| </li> |
| |
| <li> <a href="downloads.html" title="Downloads">Downloads</a> |
| </li> |
| |
| <li> <a href="streams-examples/index.html" title="Examples">Examples</a> |
| </li> |
| |
| <li> <a href="faq.html" title="Frequently Asked Questions">Frequently Asked Questions</a> |
| </li> |
| |
| <li> <a href="modules.html" title="Modules">Modules</a> |
| </li> |
| </ul> |
| </li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown">Users <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| |
| <li> <a href="install/java.html" title="Install Java">Install Java</a> |
| </li> |
| |
| <li> <a href="install/docker.html" title="Install Docker">Install Docker</a> |
| </li> |
| |
| <li> <a href="install/sbt.html" title="Install SBT">Install SBT</a> |
| </li> |
| |
| <li class="dropdown-submenu"> |
| <a href="" title="Credentials">Credentials</a> |
| <ul class="dropdown-menu"> |
| <li> <a href="credentials/twitter.html" title="Twitter">Twitter</a> |
| </li> |
| </ul> |
| </li> |
| |
| <li class="dropdown-submenu"> |
| <a href="" title="Services">Services</a> |
| <ul class="dropdown-menu"> |
| <li> <a href="services/cassandra.html" title="Cassandra">Cassandra</a> |
| </li> |
| <li> <a href="services/elasticsearch.html" title="Elasticsearch">Elasticsearch</a> |
| </li> |
| <li> <a href="services/mongo.html" title="Mongo">Mongo</a> |
| </li> |
| <li> <a href="services/neo4j.html" title="Neo4j">Neo4j</a> |
| </li> |
| <li> <a href="services/riak.html" title="Riak">Riak</a> |
| </li> |
| </ul> |
| </li> |
| </ul> |
| </li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown">Developers <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| |
| <li> <a href="install/git.html" title="Install Git">Install Git</a> |
| </li> |
| |
| <li> <a href="install/maven.html" title="Install Maven">Install Maven</a> |
| </li> |
| |
| <li> <a href="changelog.html" title="Changelog">Changelog</a> |
| </li> |
| |
| <li> <a href="release-setup.html" title="Release Setup">Release Setup</a> |
| </li> |
| |
| <li> <a href="release.html" title="Release Process">Release Process</a> |
| </li> |
| |
| <li> <a href="website.html" title="Website Management">Website Management</a> |
| </li> |
| |
| <li> <a href="code-conventions.html" title="Coding Conventions">Coding Conventions</a> |
| </li> |
| </ul> |
| </li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown">Reports <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| |
| <li class="dropdown-submenu"> |
| <a href="project-info.html" title="Project Information">Project Information</a> |
| <ul class="dropdown-menu"> |
| <li> <a href="index.html" title="About">About</a> |
| </li> |
| <li> <a href="modules.html" title="Project Modules">Project Modules</a> |
| </li> |
| <li> <a href="license.html" title="Licenses">Licenses</a> |
| </li> |
| <li> <a href="mail-lists.html" title="Mailing Lists">Mailing Lists</a> |
| </li> |
| <li> <a href="team-list.html" title="Team">Team</a> |
| </li> |
| <li> <a href="source-repository.html" title="Source Code Management">Source Code Management</a> |
| </li> |
| <li> <a href="issue-tracking.html" title="Issue Management">Issue Management</a> |
| </li> |
| <li> <a href="dependencies.html" title="Dependencies">Dependencies</a> |
| </li> |
| <li> <a href="dependency-info.html" title="Dependency Information">Dependency Information</a> |
| </li> |
| <li> <a href="dependency-management.html" title="Dependency Management">Dependency Management</a> |
| </li> |
| <li> <a href="distribution-management.html" title="Distribution Management">Distribution Management</a> |
| </li> |
| <li> <a href="plugin-management.html" title="Plugin Management">Plugin Management</a> |
| </li> |
| </ul> |
| </li> |
| |
| <li class="dropdown-submenu"> |
| <a href="project-reports.html" title="Project Reports">Project Reports</a> |
| <ul class="dropdown-menu"> |
| <li> <a href="checkstyle.html" title="Checkstyle">Checkstyle</a> |
| </li> |
| <li> <a href="checkstyle-aggregate.html" title="Checkstyle">Checkstyle</a> |
| </li> |
| <li> <a href="aggregate-third-party-report.html" title="Third Parties (Aggregate mode)">Third Parties (Aggregate mode)</a> |
| </li> |
| <li> <a href="rat-report.html" title="Rat Report">Rat Report</a> |
| </li> |
| </ul> |
| </li> |
| </ul> |
| </li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown">Other Resources <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| |
| <li> <a href="http://activitystrea.ms" title="ActivityStreams Homepage">ActivityStreams Homepage</a> |
| </li> |
| |
| <li> <a href="http://activitystrea.ms/specs/" title="ActivityStreams Specifications">ActivityStreams Specifications</a> |
| </li> |
| |
| <li> <a href="http://blogs.apache.org/streams/" title="Streams Blog">Streams Blog</a> |
| </li> |
| |
| <li> <a href="http://wiki.apache.org/incubator/StreamsProposal" title="Streams Proposal Wiki Page">Streams Proposal Wiki Page</a> |
| </li> |
| |
| <li> <a href="http://www.apache.org/" title="Apache Foundation Info">Apache Foundation Info</a> |
| </li> |
| |
| <li> <a href="http://www.apache.org/foundation/sponsorship.html" title="Sponsorship">Sponsorship</a> |
| </li> |
| |
| <li> <a href="http://www.apache.org/foundation/thanks.html" title="Thanks">Thanks</a> |
| </li> |
| </ul> |
| </li> |
| </ul> |
| |
| |
| |
| |
| |
| <ul class="nav pull-right"><li> |
| |
| <a href="https://twitter.com/ApacheStreams" class="twitter-follow-button" data-show-count="true" data-align="right" data-size="large" data-show-screen-name="true" data-lang="en">Follow ApacheStreams</a> |
| <script type="text/javascript">!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script> |
| |
| </li></ul> |
| |
| |
| </div> |
| |
| </div> |
| </div> |
| </div> |
| |
| <div class="container"> |
| <div id="banner"> |
| <div class="pull-left"> |
| <a href="http://streams.apache.org" id="bannerLeft"> |
| <img src="http://streams.apache.org/images/streams_logo.jpg" alt="Apache Streams" width="150" height="100"/> |
| </a> |
| </div> |
| <div class="pull-right"> </div> |
| <div class="clear"><hr/></div> |
| </div> |
| |
| <div id="breadcrumbs"> |
| <ul class="breadcrumb"> |
| |
| <li class=""> |
| <a href="http://streams.apache.org" class="externalLink" title="Streams"> |
| Streams</a> |
| <span class="divider">/</span> |
| </li> |
| <li class="active ">Architecture</li> |
| |
| |
| |
| </ul> |
| </div> |
| |
| |
| |
| <div id="bodyColumn" > |
| |
| <div class="section"> |
| <h2><a name="Architecture"></a>Architecture</h2> |
| <p>Apache Streams consists of a loosely coupled set of modules with specific capabilities, such as:</p> |
| |
| <ul> |
| |
| <li>collecting data.</li> |
| |
| <li>transforming or filter data</li> |
| |
| <li>storing and retrieving documents and metadata from databases</li> |
| |
| <li>binding streams components to other systems</li> |
| |
| <li>facilitating starting and stopping of streams.</li> |
| </ul> |
| <p><img src="architecture.dot.svg" alt="Architecture" /></p> |
| <div class="section"> |
| <div class="section"> |
| <h4><a name="Modules"></a>Modules</h4> |
| <p>Each module has it’s own POM and dependency tree. Each stream deployment needs to import only the modules it needs for what it wants to do.</p></div> |
| <div class="section"> |
| <h4><a name="Schemas"></a>Schemas</h4> |
| <p>Streams also contains libraries and patterns for specifying document schemas, converting documents to and from ActivityStreams format, and generating source and resource files for binding to data objects in those formats.</p></div> |
| <div class="section"> |
| <h4><a name="Pipelines"></a>Pipelines</h4> |
| <p>A Pipeline is a set of collection, processing, and storage components structured in a directed graph (cycles may be permitted) which is packaged, deployed, started, and stopped together.</p></div> |
| <div class="section"> |
| <h4><a name="Runtimes"></a>Runtimes</h4> |
| <p>A Runtime is a module containing bindings that help setup and run a pipeline. Runtimes may submit pipeline binaries to an existing cluster, or may launch the process(es) to execute the stream directly. </p></div></div> |
| <div class="section"> |
| <h3><a name="Example"></a>Example</h3> |
| <p>A standard usage of Apache Streams is to collect, normalize, and archive activity across multiple networks.</p> |
| <p><img src="example.dot.svg" alt="Example" /></p> |
| <div class="section"> |
| <div class="section"> |
| <div class="section"> |
| <h6><a name="Licensed_under_Apache_License_2.0_-_http:www.apache.orglicensesLICENSE-2.0"></a>Licensed under Apache License 2.0 - <a class="externalLink" href="http://www.apache.org/licenses/LICENSE-2.0">http://www.apache.org/licenses/LICENSE-2.0</a></h6></div></div></div></div></div> |
| </div> |
| </div> |
| |
| <hr/> |
| |
| <footer> |
| <div class="container"> |
| <div class="row"> |
| <p >Copyright © 2018 |
| <a href="https://www.apache.org/">The Apache Software Foundation</a>. |
| All rights reserved. |
| <li id="publishDate" class="pull-right">Last Published: 2018-04-02</li> |
| <li id="projectVersion" class="pull-right"> |
| Version: 0.5.2-SNAPSHOT |
| </li> |
| </p> |
| </div> |
| |
| <p id="poweredBy" class="pull-right"> |
| <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"> |
| <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> |
| </a> |
| </p> |
| |
| </div> |
| </footer> |
| </body> |
| </html> |
