Google Git
Sign in
apache / incubator-retired-s4 / refs/tags/0.5.0
tag70806aa1ee0b9154d36fd834dc4907cd8d3eb791
taggerMatthieu Morel <mmorel@apache.org>Tue Jul 31 13:19:51 2012 +0200
object53e0d34fb40df384efeb03d47c8a965f4bb4c0cc 
0.5.0 release
commit53e0d34fb40df384efeb03d47c8a965f4bb4c0cc[log] [tgz]
authorMatthieu Morel <mmorel@apache.org>Tue Jul 31 13:11:13 2012 +0200
committerMatthieu Morel <mmorel@apache.org>Tue Jul 31 13:11:13 2012 +0200
treeb5b76ee705cc6e5ace5a2bd0687a9e6446210307
parentade0e115f9336dd110ced74fbce639a6f502c8d3 [diff]
Fix release notes suffix
1 file changed
tree: b5b76ee705cc6e5ace5a2bd0687a9e6446210307
  1. config/
  2. lib/
  3. subprojects/
  4. test-apps/
  5. .gitignore
  6. build.gradle
  7. DISCLAIMER
  8. gradlew
  9. gradlew.bat
  10. LICENSE
  11. NOTICE.txt
  12. README.md
  13. RELEASE_NOTES.html
  14. s4
  15. settings.gradle
README.md

Apache S4

Overview

S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable platform that allows programmers to easily develop applications for processing continuous unbounded streams of data.

S4 0.5.0 is a complete refactoring of the previous version of S4. It grounds on the same concepts (partitioning inspired by map-reduce, actors-like distribution model), but with the following objectives:

  • cleaner and simpler API
  • robust configuration through statically defined modules
  • cleaner architecture
  • robust codebase
  • easier to develop S4 apps, to test, and to use the platform

We added the following core features:

  • TCP-based communications
  • state recovery through a flexible checkpointing mechanism
  • inter-cluster/app communications through a pub-sub model
  • dynamic application deployment
  • toolset for easily starting S4 nodes, testing, packaging, deploying and monitoring S4 apps

Documentation

For the latest information about S4, please visit our website at:

http://inbubator.apache.org/s4

and our wiki, at:

https://cwiki.apache.org/confluence/display/S4/S4+Wiki

Currently the wiki contains the most relevant and up-to-date documentation.

Source code is available here: https://git-wip-us.apache.org/repos/asf?p=incubator-s4.git

Requirements

  • JDK 6 or higher
  • *nix or macosx (you may build the project and develop S4 applications with microsoft windows though, the only limitation is that the “s4” script has not been ported to windows yet)

How to build

This only applies if you checkout from the source repository or if you download a released source package.

We use gradle http://gradle.org as the build system.

  • From the root directory of the S4 project:

./gradlew install

This will build the packages and install the artifacts in the local maven repository.

  • Then, build the tools:

./gradlew s4-tools:installApp

This will build the tools so that you can drive the platform through the “s4” command.

Directory structure

  • If you have a source package:

    • root directory contains build and utility scripts, as well as informative files
    • config/ directory contains configuration files for source code formatting
    • doc/ directory contains the javadoc
    • gradle/ directory contains libraries used by the gradle build tool
    • lib/ directory contains some other gradle libraries
    • subproject/ directory contains the plaftorm subprojects: base, comm, core, tools, as well as example (example is not fully ported to 0.5.0)
    • test-apps/ directory contains some examples (some of them very trivial are used in regression tests)

  • If you have a binary package:

    • root directory contains the s4 utility script as well as informative files
    • doc/ directory contains the javadoc
    • lib/ directory contains :
      • the platform libraries (prefixed with “s4”)
      • the dependencies
    • bin/ directory contains some scripts that are used by the s4 script
    • gradle/ directory contains libraries used for building S4 projects