Apache InLong is a one-stop integration framework for massive data that provides automatic, secure and reliable data transmission capabilities. InLong supports both batch and stream data processing at the same time, which offers great power to build data analysis, modeling and other real-time applications based on streaming data.
The just-released 1.3.0 version closes about 410+ issues, contains 110+ features and 170+ optimizations. Mainly include the following:
Manager abstracts Load nodes to support easy expansion of Load node resources and greatly reduce the development time of a Load node This part of the feature was contributed by @ciscozhou
In the old version, the Manager Open Api can be accessed anonymously, and in the new version, it is implemented using the Apache Shiro framework. Login authentication method based on Basic Access Authentication, this part of the function was contributed by @woofyzhao
Version 1.3.0 enhances the collection of file data and k8s data, in which file collection supports regular expression custom line breaks, so that multiple lines can be merged and folded In addition, the new version of Agent supports full and incremental collection of file content. This part of the function was contributed by @ganfengtan
In addition to the current java client, DataProxy has added c++ client capabilities, which are provided by @pocozh
The new version of Sort supports three kinds of Temporal Join\ Lookup Join \ Interval Join, this part of the function was contributed by @yunqingmoswu Most community users mentioned the need for encryption and decryption and Json Udf, this part of the function was contributed by @Emsnap and @Emhui
The new version of Sort Connector supports Flink built-in indicator reporting of various Connectors. External indicator systems such as Prometheus can directly obtain the number and rate of task data read and write. In addition, the new version also supports InLong Audit Audit data reporting, which is contributed by @pacigong, @Emsnap, @thesumery @Oneal65 @yunqingmoswu
In version 1.3.0, Manager added the creation of some storage resources:
The above are all contributed by community member @haibo-duan, thanks
For related content, please refer to the Release Notes, which details the features, enhancements and bug fixes of this release.
In subsequent versions, we will expand more data sources and storages to cover more usage scenarios, and gradually improve the usability and robustness of the system, including: