Google Git
Sign in
apache / streampipes-extensions / 046e239f381b513073a5a276dca8d2f4bfaed0c2
commit046e239f381b513073a5a276dca8d2f4bfaed0c2[log] [tgz]
authorzehnder <zehnder@fzi.de>Tue Mar 19 21:28:10 2019 +0000
committerzehnder <zehnder@fzi.de>Tue Mar 19 21:28:10 2019 +0000
tree1de4bbf335f0d897ef5951e075b30260680f49d0
parent7b0370d31760d4ff0e0aab3efae843630506c0e4 [diff]
parent9112c1b0d3ff0e439c8d68963c5c3143ff45ea98 [diff]
[RELEASE] [skip-ci]merging 'release-0.61.0' into 'master'
tree: 1de4bbf335f0d897ef5951e075b30260680f49d0
  1. streampipes-pipeline-elements-shared/
  2. streampipes-processors-aggregation-flink/
  3. streampipes-processors-enricher-flink/
  4. streampipes-processors-filters-jvm/
  5. streampipes-processors-filters-siddhi/
  6. streampipes-processors-geo-flink/
  7. streampipes-processors-geo-jvm/
  8. streampipes-processors-image-processing-jvm/
  9. streampipes-processors-pattern-detection-flink/
  10. streampipes-processors-statistics-flink/
  11. streampipes-processors-text-mining-flink/
  12. streampipes-processors-transformation-flink/
  13. streampipes-processors-transformation-jvm/
  14. streampipes-sinks-brokers-jvm/
  15. streampipes-sinks-databases-flink/
  16. streampipes-sinks-databases-jvm/
  17. streampipes-sinks-internal-jvm/
  18. streampipes-sinks-notifications-jvm/
  19. streampipes-sources-random-data-generator/
  20. streampipes-sources-vehicle-simulator/
  21. streampipes-sources-watertank-simulator/
  22. .gitignore
  23. .gitlab-ci.yml
  24. LICENSE
  25. pom.xml
  26. README.md
README.md

StreamPipes

StreamPipes enables flexible modeling of stream processing pipelines by providing a graphical modeling editor on top of existing stream processing frameworks.

It leverages non-technical users to quickly define and execute processing pipelines based on an easily extensible toolbox of data sources, data processors and data sinks.

Learn more about StreamPipes at https://www.streampipes.org/

Read the full documentation at https://docs.streampipes.org

StreamPipes Pipeline Elements

This project provides a library of several pipeline elements that can be used within the StreamPipes toolbox.

Currently, the following pipeline elements are available:

Data Sources

  • Watertank Simulator: A data simulator that replays data from an Industrial IoT use case. Several streams are provided such as flow rate, water level and more.
  • Vehicle Simulator: Provides a simulated stream that replays location-based real-time data of a vehicle.
  • Random Data Generator: Several streams that produce randomly generated data in an endless stream.

Data Processors

  • Aggregation
  • Count Aggregation
  • Event Rate
  • Timestamp Enricher
  • Numerical Filter: Filters sensor values based on a configurable threshold value.
  • Text Filter: Filters text-based fields by a given string value.
  • Projection: Outputs a configurable subset of the fields available in an input event stream.
  • Geocoding
  • Google Routing
  • Increase
  • Peak Detection
  • Statistics Summary
  • Statistics Summary Window-Based
  • Field Converter
  • Field Hasher
  • Field Mapper
  • Measurement Unit Converter
  • Field Renamer

Data Sinks

  • CouchDB: Stores events in an Apache CouchDB database.
  • Dashboard: Can be used to display pipeline results in the real-time dashboard of the StreamPipes UI.
  • Kafka Publisher: Publishes events to an Apache Kafka broker.
  • Notification: Can be used to generate notifications that are shown in the notification center of the StreamPipes UI.
  • JMS
  • RabbitMQ
  • Elasticsearch
  • Dashboard
  • Notification
  • Email Notification
  • Slack Notification
  • OneSignal Notification

Contact us if you are missing some pipeline elements!

Getting started

All modules contain a docker-compose template in the deployment folder of each module. Copy this into your StreamPipes docker-compose file (see the docs for more info) and start StreamPipes! All pipeline elements are automatically registered (but do not forget to install them under Pipeline Element Installation).

It's easy to get started:

Feedback

We'd love to hear your feedback! Contact us at feedback@streampipes.org