Google Git
Sign in
apache / streampipes-extensions / refs/tags/0.62.0
taged98635fa440807727aea17c4bd2538ab750dd22
taggerzehnder <zehnder@fzi.de>Thu May 23 08:19:41 2019 +0000
objecta608ab44bb7f083db54a8d44c84a71ce46b9dfce 
[RELEASE] [skip-ci]tagging release 0.62.0
commita608ab44bb7f083db54a8d44c84a71ce46b9dfce[log] [tgz]
authorzehnder <zehnder@fzi.de>Thu May 23 08:19:40 2019 +0000
committerzehnder <zehnder@fzi.de>Thu May 23 08:19:40 2019 +0000
tree076da1482eafbce889e472d576b00e1827e39e38
parent046e239f381b513073a5a276dca8d2f4bfaed0c2 [diff]
parent0977cb43bbe90f04e705a84f9c458ec1617edd57 [diff]
[RELEASE] [skip-ci]merging 'release-0.62.0' into 'master'
tree: 076da1482eafbce889e472d576b00e1827e39e38
  1. streampipes-pipeline-elements-shared/
  2. streampipes-processors-aggregation-flink/
  3. streampipes-processors-enricher-flink/
  4. streampipes-processors-filters-jvm/
  5. streampipes-processors-filters-siddhi/
  6. streampipes-processors-geo-flink/
  7. streampipes-processors-geo-jvm/
  8. streampipes-processors-image-processing-jvm/
  9. streampipes-processors-pattern-detection-flink/
  10. streampipes-processors-statistics-flink/
  11. streampipes-processors-text-mining-flink/
  12. streampipes-processors-transformation-flink/
  13. streampipes-processors-transformation-jvm/
  14. streampipes-sinks-brokers-jvm/
  15. streampipes-sinks-databases-flink/
  16. streampipes-sinks-databases-jvm/
  17. streampipes-sinks-internal-jvm/
  18. streampipes-sinks-notifications-jvm/
  19. streampipes-sources-random-data-generator/
  20. streampipes-sources-vehicle-simulator/
  21. streampipes-sources-watertank-simulator/
  22. .gitignore
  23. .gitlab-ci.yml
  24. .spignore
  25. LICENSE
  26. pom.xml
  27. README.md
README.md

StreamPipes

StreamPipes enables flexible modeling of stream processing pipelines by providing a graphical modeling editor on top of existing stream processing frameworks.

It leverages non-technical users to quickly define and execute processing pipelines based on an easily extensible toolbox of data sources, data processors and data sinks.

Learn more about StreamPipes at https://www.streampipes.org/

Read the full documentation at https://docs.streampipes.org

StreamPipes Pipeline Elements

This project provides a library of several pipeline elements that can be used within the StreamPipes toolbox.

Currently, the following pipeline elements are available:

Data Sources

  • Watertank Simulator: A data simulator that replays data from an Industrial IoT use case. Several streams are provided such as flow rate, water level and more.
  • Vehicle Simulator: Provides a simulated stream that replays location-based real-time data of a vehicle.
  • Random Data Generator: Several streams that produce randomly generated data in an endless stream.

Data Processors

  • Aggregation
  • Count Aggregation
  • Event Rate
  • Timestamp Enricher
  • Numerical Filter: Filters sensor values based on a configurable threshold value.
  • Text Filter: Filters text-based fields by a given string value.
  • Projection: Outputs a configurable subset of the fields available in an input event stream.
  • Geocoding
  • Google Routing
  • Increase
  • Peak Detection
  • Statistics Summary
  • Statistics Summary Window-Based
  • Field Converter
  • Field Hasher
  • Field Mapper
  • Measurement Unit Converter
  • Field Renamer

Data Sinks

  • CouchDB: Stores events in an Apache CouchDB database.
  • Dashboard: Can be used to display pipeline results in the real-time dashboard of the StreamPipes UI.
  • Kafka Publisher: Publishes events to an Apache Kafka broker.
  • Notification: Can be used to generate notifications that are shown in the notification center of the StreamPipes UI.
  • JMS
  • RabbitMQ
  • Elasticsearch
  • Dashboard
  • Notification
  • Email Notification
  • Slack Notification
  • OneSignal Notification

Contact us if you are missing some pipeline elements!

Getting started

All modules contain a docker-compose template in the deployment folder of each module. Copy this into your StreamPipes docker-compose file (see the docs for more info) and start StreamPipes! All pipeline elements are automatically registered (but do not forget to install them under Pipeline Element Installation).

It's easy to get started:

Feedback

We'd love to hear your feedback! Contact us at feedback@streampipes.org