5 minutes to read
We are proud to release StreamPipes version 0.61.0, which brings many improvements and some exciting new features that simplify pipeline development! The new version, which is already available on Docker Hub and can be downloaded using our installer brings over 20 improvements. Under the hood, a reworked event model simplifies the development of new pipeline elements using the built-in SDK.
Our main mission is to make IoT data management & analytics accessible to everyone. This release focuses on better integrating StreamPipes with other systems and data sources and contains UX improvements to ease the development of pipelines.
StreamPipes Connect, which aims to easily connect arbitrary data sources with StreamPipes, has been first introduced as a public beta in version 0.60.0. Version 0.61.0 brings many new features to Connect and also implements many bug fixes that now make Connect the recommended way to connect data sources.
StreamPipes Connect relies on an extensible architecture to connect either generic protocols or specific adapters.
In more detail, StreamPipes Connect supports the following protocols, with a completely code-free integration process:
Data Streams
Data Sets
The next version will also include an adapter for OPC UA, which is currently being tested in a real setting.
For protocols that are not yet supported by StreamPipes Connect, a new Maven archetype for easily connecting data sources using the SDK is provided.
On the UI side, we try to make the graphical modeling of pipelines as simple as possible. Besides some minor improvements (e.g., case-insensitive search for pipeline elements), pipelines are now constantly validated and users get feedback on steps that are still missing (e.g., missing pipeline elements). As many other features we are currently implementing, this feature directly comes from a user request - so if you have any suggestions for improvement, let us know!
One of the most important features of v0.61.0 is not directly visible to end users, but drastically eases the development of new pipeline elements with the SDK. From this release on, events at runtime are represented in form of a new Event Object instead of the former Map representation. This feature allows us for much better support of complex data processors which have more than one input data stream and comes with many convenience methods to better extrat fields from an event for processing.
For instance, prior to v0.61.0 you had to extract fields from an incoming events as follows:
@Override public void onEvent(Map<String, Object> in, String s, SpOutputCollector out) { Integer numberValue = Integer.parseInt(String.valueOf(in.get(MAPPING_PROPERTY_ID)); }
The new event model is much simpler and less error-prone:
public void onEvent(Event in, SpOutputCollector out) throws SpRuntimeException { Integer numberValue = in .getFieldBySelector(MAPPING_PROPERTY_SELECTOR) .getAsPrimitive() .getAsInteger(); }
The new event model is even more useful in cases where your data streams have a complex nested structure or contain list fields. If you connect two data streams containing the same fields to a processor, field names are automatically renamed in the background to avoid naming conflicts. A complete guide can be found in the developer guide.
The developer guide has been extended: Besides reworked tutorials on developing new data processors and data sources, the new documentation includes instructions on adding stream requirements, the new event model, and in-depth guides on defining static properties and output strategies.
See the release notes for more details.
We are absolutely open to your suggestions for further improvements! Let us know (by mail, slack or twitter) and we'll consider your feature request in the next release!