Drop Jetty from flume-ng-core (#446)

* Move HTTPSource to its own flume-http-source module

Keeping HTTPSource in flume-ng-core forced the heavy Jetty/Gson HTTP
stack onto every core consumer. Extract it into a dedicated optional
source module under flume-ng-sources, like taildir, so it ships only
when needed.

The package (org.apache.flume.source.http) is unchanged, so the
SourceType.HTTP reflective mapping and existing "http" configs keep
working. Core retains Jetty/Gson for its metrics server and
HTTPServerConstraintUtil.

Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Make flume-http-source self-contained for HTTP constraints

HTTPSource was reaching into flume-ng-core for HTTPServerConstraintUtil.
Move that helper into the module next to its only caller and make it
package-private, so the HTTP source no longer depends on a core internal
for its Jetty constraint handling. Declare jetty-security directly here
(managed in flume-parent) since the helper uses it.

Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Move monitoring services to flume-ng-instrumentation modules

Split the Ganglia, HTTP and Prometheus MonitorService implementations
out of flume-ng-core into three modules under a new flume-ng-instrumentation
parent, each in its own package so there are no split packages.

Replace the hardcoded MonitoringType enum with ServiceLoader discovery:
MonitorService gains a getType() default method, each provider declares
itself via META-INF/services, and the node selects one by matching
flume.monitoring.type against getType() (FQCN fallback preserved). The
modules are wired through the BOM and bundled by flume-ng-dist only.

Core keeps the MonitorService interface and JMXPollUtil, and drops the now
unused gson and prometheus dependencies.

Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Declare gson dependency in flume-taildir-source

The taildir source uses gson for its position file but was getting it
transitively from flume-ng-core, which no longer provides it after the
monitoring split. Without an explicit dependency the position file
handling fails at runtime with NoClassDefFoundError.

Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Drop Jetty from flume-ng-core

With the HTTP source and the HTTP/Prometheus monitors moved to their own
modules, nothing in flume-ng-core's main code uses Jetty anymore. Remove
the four jetty dependencies from the module.

flume-ng-node's TestHttpConfigurationSource was relying on Jetty arriving
transitively through flume-ng-core, so declare jetty-server and
jetty-servlet as test dependencies there.

Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2 files changed
tree: 4b9936907a7447eb9990e1a80f5fc1a1f0739014
  1. .github/
  2. .mvn/
  3. bin/
  4. conf/
  5. dev-docs/
  6. dev-support/
  7. flume-bom/
  8. flume-ng-auth/
  9. flume-ng-channels/
  10. flume-ng-configfilters/
  11. flume-ng-configuration/
  12. flume-ng-core/
  13. flume-ng-dist/
  14. flume-ng-instrumentation/
  15. flume-ng-node/
  16. flume-ng-sdk/
  17. flume-ng-sources/
  18. flume-parent/
  19. flume-tools/
  20. .asf.yaml
  21. .gitattributes
  22. .gitignore
  23. .travis.yml.sav
  24. CHANGELOG
  25. CONTRIBUTING.md
  26. DEVNOTES
  27. doap_Flume.rdf
  28. LICENSE
  29. mvnw
  30. mvnw.cmd
  31. NOTICE
  32. pom.xml
  33. README.md
  34. RELEASE-NOTES
README.md

Project status

[!WARNING] As of May 2026 this project is undergoing significant rework! We do not advise using it until it is restablized and a formal release is announced. It has been marked as dormant by Apache Logging Services consensus on 2024-10-10. Users are advised to migrate to alternatives. For other inquiries, see the support policy.

Welcome to Apache Flume!

Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. The system is centrally managed and allows for intelligent dynamic management. It uses a simple extensible data model that allows for online analytic application.

The Apache Flume 1.x (NG) code line is a refactoring of the first generation Flume to solve certain known issues and limitations of the original design.

Apache Flume is open-sourced under the Apache Software Foundation License v2.0.

Documentation

Documentation is included in the binary distribution under the docs directory. In source form, it can be found in the flume-ng-doc directory.

The Flume 1.x guide and FAQ are available here:

Contact us!

Bug and Issue tracker.

Compiling Flume

Compiling Flume requires the following tools:

  • Oracle Java JDK 1.8
  • Apache Maven 3.x

Note: The Apache Flume build requires more memory than the default configuration. We recommend you set the following Maven options:

export MAVEN_OPTS="-Xms512m -Xmx1024m"

To compile Flume and build a distribution tarball, run mvn install from the top level directory. The artifacts will be placed under flume-ng-dist/target/.