Google Git
Sign in
apache / streampipes-extensions / refs/heads/STREAMPIPES-451
commitcce581663b8db688be1967838453a119f1724456[log] [tgz]
authorPhilipp Zehnder <zehnder@fzi.de>Mon Nov 08 08:33:42 2021 +0100
committerPhilipp Zehnder <zehnder@fzi.de>Mon Nov 08 08:33:42 2021 +0100
tree0b0ab75b3d976fd29811c32beb6b909a9e15aa14
parent78dc072c297cf96edab7812417773efb9249413a [diff]
WIP working on STREAMPIPES-451
2 files changed
tree: 0b0ab75b3d976fd29811c32beb6b909a9e15aa14
  1. .github/
  2. .idea/
  3. streampipes-connect-adapters/
  4. streampipes-connect-adapters-iiot/
  5. streampipes-extensions-all-jvm/
  6. streampipes-pipeline-elements-all-flink/
  7. streampipes-pipeline-elements-all-jvm/
  8. streampipes-pipeline-elements-data-simulator/
  9. streampipes-pipeline-elements-shared/
  10. streampipes-processors-aggregation-flink/
  11. streampipes-processors-change-detection-jvm/
  12. streampipes-processors-enricher-flink/
  13. streampipes-processors-enricher-jvm/
  14. streampipes-processors-filters-jvm/
  15. streampipes-processors-filters-siddhi/
  16. streampipes-processors-geo-flink/
  17. streampipes-processors-geo-jvm/
  18. streampipes-processors-image-processing-jvm/
  19. streampipes-processors-pattern-detection-flink/
  20. streampipes-processors-statistics-flink/
  21. streampipes-processors-text-mining-flink/
  22. streampipes-processors-text-mining-jvm/
  23. streampipes-processors-transformation-flink/
  24. streampipes-processors-transformation-jvm/
  25. streampipes-sinks-brokers-jvm/
  26. streampipes-sinks-databases-flink/
  27. streampipes-sinks-databases-jvm/
  28. streampipes-sinks-internal-jvm/
  29. streampipes-sinks-notifications-jvm/
  30. streampipes-sources-vehicle-simulator/
  31. streampipes-sources-watertank-simulator/
  32. tools/
  33. .asf.yaml
  34. .gitignore
  35. .gitlab-ci.yml
  36. .spignore
  37. CONTRIBUTING.md
  38. DISCLAIMER
  39. docker-compose.yml
  40. LICENSE
  41. NOTICE
  42. pom.xml
  43. README.md
  44. RELEASE_NOTES.md
  45. RELEASE_VALIDATION.md
README.md

Github Actions

Apache StreamPipes

Apache StreamPipes (incubating) enables flexible modeling of stream processing pipelines by providing a graphical modeling editor on top of existing stream processing frameworks.

It empowers non-technical users to quickly define and execute processing pipelines based on an easily extensible toolbox of data sources, data processors and data sinks. StreamPipes has an exchangeable runtime execution layer and executes pipelines using one of the provided wrappers, e.g., standalone or distributed in Apache Flink.

Pipeline elements in StreamPipes can be installed at runtime - the built-in SDK allows to easily implement new pipeline elements according to your needs. Pipeline elements are standalone microservices that can run anywhere - centrally on your server, in a large-scale cluster or close at the edge.

Extensions: Connect Adapters and Pipeline Elements

This project provides a library of several Connect adapters and pipeline elements that can be used within the Apache StreamPipes toolbox.

See https://streampipes.apache.org/docs/docs/pipeline-elements/ for an overview of currently available pipeline elements.

Contact us if you are missing some pipeline elements!

Installation

The quickest way to run StreamPipes including the latest extensions (adapters, pipeline elements) is by using our Docker-based installation & operation options, namely:

NOTE: StreamPipes CLI & k8s are highly recommended for developers or operators. Standard users should stick to StreamPipes Compose.

Please follow the instructions provided in the corresponding README.md to get started.

For a more in-depth manual, read the installation guide at https://streampipes.apache.org/docs/docs/try-installation.html!

Building StreamPipes Extensions

To properly build the StreamPipes extensions project, the following tools should be installed:

Prerequisites

  • Java 8 JDK (minimum)
  • Maven (tested with 3.6)
  • NodeJS + NPM (tested with v12+/ v6+)
  • Docker + Docker-Compose

Building

To build the extensions project, do the following:

    mvn clean package

Starting

To start StreamPipes Extensions, run docker-compose up -d from the root directory. Make sure the core (incubator-streampipes) is already started.

You can also use the installer or CLI as described in the Getting Started section.

Bugs and Feature Requests

If you‘ve found a bug or have a feature that you’d love to see in StreamPipes, feel free to create an issue in our Jira: https://issues.apache.org/jira/projects/STREAMPIPES

Get help

If you have any problems during the installation or questions around StreamPipes, you'll get help through one of our community channels:

And don't forget to follow us on Twitter!

Contribute

We welcome contributions to StreamPipes. If you are interested in contributing to StreamPipes, let us know! You'll get to know an open-minded and motivated team working together to build the next IIoT analytics toolbox.

Here are some first steps in case you want to contribute:

Have fun!