blob: 3ab80d2fe37a8878134c5a2e74e33a4b11477f9c [file] [log] [blame]
Sample to connect to a Flume Agent via Avro using the FlumeAppender.
This sample uses the classes in the sample flume-common project. It will randomly pick from the events defined there
and send them to Flume. At the same time it will intermix some random non-audit events.
To run this sample:
1. Run "mvn install" on the flume-common project.
2. Download and install Flume.
3. Copy the in src/main/resources/flume/conf to the conf directory of where Flume was installed.
4. In a terminal window start flume using "bin/flume-ng agent --conf ./conf/ -f conf/ -Dflume.root.logger=DEBUG,console -n agent"
5. Verify Flume started and configured an Avro source, a memory channel and a logger sink by reviewing the startup log.
6. In a separate terminal window run "mvn jetty:run" in this project.
7. Verify the Flume appender connected to the Flume agent by finding "Starting FlumeAvroManager FlumeAvro[localhost:8800]"
in the jetty log and that there are no exceptions and also by seeing something like
"/ => /] OPEN" in the Flume log.
8. In a separate terminal window in the project directory run "tail -f target/logs/app.log" to see the application
generate non-audit logs.
9. In the browser go to url "http://localhost:8080/flumeAgent/". A started message should appear on the screen.
10. After verifying logs are being written click on the Stop button in the browser page.
Note that the Flume agent must be started for the FlumeAppender to successfully start. If the Flume agent is stopped
an exception will be thrown for each event that occurs while the agent is down. When the agent restarts the
appender will automatically reconnect.
The output from the Flume agent will include the generated Flume events. Since the events are sent individually
Avro will print the full event and Flume will print a hex dump of the first few bytes of the event.