In this section, we describe our recommended minimum setup for locally setting up a development instance of StreamPipes needed to develop, run and test new pipeline elements.
StreamPipes does not have specific requirements on the IDE - so feel free to choose the IDE of your choice. The only requirements in terms of development tools are that you have Java 8 and Maven installed.
In order to quickly test developed pipeline elements without needing to install all services required by StreamPipes, we provide a CLI tool that allows you to selectively start StreamPipes components. The CLI tool allows to switch to several templates (based on docker-compose) depending on the role. For instance, if you are developing a pipeline element, use the template pe-developer. This will start backend and ui components in a Docker container, while you can easily test your pipeline element in your IDE.
For now, we refer to the Github Readme for instructions on how to use the CLI tool: https://github.com/streampipes/streampipes-cli
Now, once you've started the development instance, you are ready to develop your very first pipeline element. Instead of starting from scratch, we recommend using our provided maven archetypes:
Create the Maven archetype as described in the Getting Started guide.
In order to develop a new pipeline element from scratch, you need to create a new Maven project and import the following dependencies:
The following three dependencies are mandatory:
streampipes-container-standalone, defines that we are going to create a new pipeline element where the description will be accessible through an embedded web server.streampipes-sdk imports the SDK which provides many convencience functions to create new pipeline elements.streampipes-vocabulary imports various RDF vocabularies which are used by the SDK to auto-generate the semantic description of pipeline elements.The following three dependencies might be optional depending on the pipeline element type you plan to create:
streampipes-wrapper-flink should be used in case you plan to connect a new data processor or data sink that uses Apache Flink for processing events at runtime.streampipes-wrapper-standalone should be used in case you plan to connect a new data processor or data sink that does not use an external processing engine. Events are directly processed in a single-host fashion.Finally, this dependency will provide abstract classes to define data sources and streams.