Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log-like data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. The system is centrally managed and allows for intelligent dynamic management. It uses a simple extensible data model that allows for online analytic application.
The Apache Flume Spring Boot module provides the minimal framework required to allow Apache Flume to be created and configured as a “normal” Spring Boot applications.
Apache Flume Spring Boot is open-sourced under the Apache Software Foundation License v2.0.
Documentation is included in the binary distribution under the docs directory. In source form, it can be found in the flume-ng-doc directory.
In general, creating a Flume Spring Boot application requires creating a simple project that contains the creation of the Sources, Sinks, and Channels with the Sinks encapsulated in SinkRunners and the Sources in SourceRunners and then then providing the bootstrap.yml and application.yml files. The Java class containing the configuration must be named in the file: META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports where each line in the file should contain the fully qualified class name of the class to be autoconfigured.Note that Spring recommends that classes listed for AutoConfiguration should NOT specify packages for component scanning. Instead, the specific classes should be specified with @Import or they could also be added the the AutoConfiguration import file.
The Flume 2.x guide and FAQ are available here:
Bug and Issue tracker.
Compiling Flume Spring Boot requires the following tools: