Pycapa performs network packet capture, both off-the-wire and from Kafka, which is useful for the testing and development of Apache Metron. It is not intended for production use. The tool will capture packets from a specified interface and push them into a Kafka Topic.
pip install -r requirements.txt python setup.py install
$ pycapa --help
usage: pycapa [-h] [-p] [-c] [-k KAFKA_BROKERS] [-t TOPIC] [-n PACKET_COUNT]
[-d DEBUG] [-i INTERFACE]
optional arguments:
-h, --help show this help message and exit
-p, --producer sniff packets and send to kafka
-c, --consumer read packets from kafka
-k KAFKA_BROKERS, --kafka KAFKA_BROKERS
kafka broker(s)
-t TOPIC, --topic TOPIC
kafka topic
-n PACKET_COUNT, --number PACKET_COUNT
number of packets to consume
-d DEBUG, --debug DEBUG
debug every X packets
-i INTERFACE, --interface INTERFACE
interface to listen on
Pycapa has two primary runtime modes.
Pycapa can be configured to capture packets from a network interface and then forward those packets to a Kafka topic. The following example will capture packets from the eth0 network interface and forward those to a Kafka topic called pcap running on localhost.
pycapa --producer --kafka localhost:9092 --topic pcap -i eth0
To output debug messages every 100 captured packets, run the following.
pycapa --producer --kafka localhost:9092 --topic pcap -i eth0 --debug 100
Pycapa can be configured to consume packets from a Kafka topic and then write those packets to a libpcap-compliant file. To read 100 packets from a kafka topic and then write those to a libpcap-compliant file, run the following command. The file out.pcap can then be opened with a tool such as Wireshark for further validation.
pycapa --consumer --kafka localhost:9092 --topic pcap --n 100 > out.pcap
To consume packets from Kafka continuously and print debug messages every 10 packets, run the following command.
pycapa --consumer --kafka localhost:9092 --topic pcap --debug 10
When run in Producer Mode, Pycapa embeds the raw network packet data in the Kafka message. The message key contains the timestamp indicating when the packet was captured in microseconds from the epoch. This value is in network byte order.