The pipeline editor module supports building pipelines that transform a data stream using a set of resuable data processors and data sinks. The empty pipeline editor looks similar to the illustration below after a new installation.
<img class=“docs-image” src=“/img/03_use-pipeline-editor/01_pipeline-editor-overview.png”alt=“StreamPipes Pipeline Editor Overview”/>
The four main concepts data sets, data streams, data processors and data sinks are available at the top of the pipeline editor. By switching the tabs, the individual pipeline elements for each category can be found. By clicking the questionmark symbol, which appears when hovering over an element, additional information can be viewed (e.g., for data streams a live preview of incoming data and the documentation of the pipeline element for data processors and sinks).
<img class=“docs-image” src=“/img/03_use-pipeline-editor/02_pipeline-element-info.png”alt=“StreamPipes Pipeline Element Info”/>
Pipelines are built by dragging data streams, processors and sinks into the pipeline assembly area. Typically, a pipeline is built step-by-step starting with a data soure (stream or set). Afterwards, data processors and sinks are subsequently added to the pipeline. Connections between pipeline elements are made by selecting the gray connector of the source and moving it to the target pipeline element. Once a connection is made, StreamPipes performs a quick validation step and, in case two pipeline elements are compatible, automatically opens a configuration window.
The configuration depends on the selected pipeline element and looks similar to the screenshot below. In general, pipeline elements are configured by providing the required values. Once the pipeline element is fully configured, the Save button activates and can be used to save the configuration for the pipeline element.
<img class=“docs-image” src=“/img/03_use-pipeline-editor/03_configure-pipeline-element.png”alt=“StreamPipes Pipeline Element Configuration”/>
In addition, the following options are available in the pipeline element configuration menu:
Further options for a pipeline element can be displayed by hovering over a pipeline element in the assembly area, so that additional buttons appear around the pipeline element:
Several pipeline editor options are available in the menu bar of the pipeline assembly:
<img class=“docs-image” src=“/img/03_use-pipeline-editor/05_pipeline-editor-options.png”alt=“StreamPipes Pipeline Editor Options”/>
To save a pipeline, press the save pipeline button. A dialog pops up where a name and description of the pipeline can be entered (only name is mandatory). Additionally, a pipeline can be directly started after it has been stored by checking the corresponding button.
<img class=“docs-image” src=“/img/03_use-pipeline-editor/06_save-pipeline.png”alt=“StreamPipes Save Pipeline Dialog”/>