blob: 4e382b7dc523740508a685e87b34a176d4c1ccfc [file] [log] [blame] [view]
Apache S4
=========
Overview
--------
S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable platform that allows programmers to easily develop applications for processing continuous unbounded streams of data.
S4 0.5.0 is a complete refactoring of the previous version of S4. It grounds on the same concepts (partitioning inspired by map-reduce, actors-like distribution model), but with the following objectives:
- cleaner and simpler API
- robust configuration through statically defined modules
- cleaner architecture
- robust codebase
- easier to develop S4 apps, to test, and to use the platform
We added the following core features:
- TCP-based communications
- state recovery through a flexible checkpointing mechanism
- inter-cluster/app communications through a pub-sub model
- dynamic application deployment
- toolset for creating nodes, adding
Documentation
-------------
For the latest information about S4, please visit our website at:
http://inbubator.apache.org/s4
and our wiki, at:
https://cwiki.apache.org/confluence/display/S4/S4+Wiki
Currently the wiki contains the most relevant and up-to-date documentation.
Source code is available here: https://git-wip-us.apache.org/repos/asf?p=incubator-s4.git
How to build
------------
This only applies if you checkout from the source repository or if you download a released source package.
You need a 6+ JDK.
We use gradle http://gradle.org as the build system.
* From the root directory of the S4 project:
./gradlew install
This will build the packages and install the artefacts in the local maven repository.
* Then, build the tools:
./gradlew s4-tools:installApp
This will build the tools so that you can drive the platform through the "s4" command.
Directory structure
-------------------
* If you have a source package:
- root directory contains build and utility scripts, as well as informative files
- config/ directory contains configuration files for source code formatting
- doc/ directory contains the javadoc
- gradle/ directory contains libraries used by the gradle build tool
- lib/ directory contains some other gradle libraries
- subproject/ directory contains the plaftorm subprojects: base, comm, core, tools, as well as example (example is not fully ported to 0.5.0)
- test-apps/ directory contains some examples (some of them very trivial are used in regression tests)
* If you have a binary package:
- root directory contains the s4 utility script as well as informative files
- doc/ directory contains the javadoc
- lib/ directory contains :
* the platform libraries (prefixed with "s4")
* the dependencies