| = Camel-Kafka-connector Docker Source Consuming Events |
| |
| This is an example for Camel-Kafka-connector Docker Source Consuming Events |
| |
| == Standalone |
| |
| === What is needed |
| |
| - A Docker daemon |
| |
| === Running Kafka |
| |
| [source] |
| ---- |
| $KAFKA_HOME/bin/zookeeper-server-start.sh $KAFKA_HOME/config/zookeeper.properties |
| $KAFKA_HOME/bin/kafka-server-start.sh $KAFKA_HOME/config/server.properties |
| $KAFKA_HOME/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic mytopic |
| ---- |
| |
| === Download the connector package |
| |
| Download the connector package tar.gz and extract the content to a directory. In this example we'll use `/home/oscerd/connectors/` |
| |
| [source] |
| ---- |
| > cd /home/oscerd/connectors/ |
| > wget https://repo1.maven.org/maven2/org/apache/camel/kafkaconnector/camel-docker-kafka-connector/0.11.5/camel-docker-kafka-connector-0.11.5-package.tar.gz |
| > untar.gz camel-docker-kafka-connector-0.11.5-package.tar.gz |
| ---- |
| |
| === Configuring Kafka Connect |
| |
| You'll need to set up the `plugin.path` property in your kafka |
| |
| Open the `$KAFKA_HOME/config/connect-standalone.properties` and set the `plugin.path` property to your choosen location: |
| |
| [source] |
| ---- |
| ... |
| plugin.path=/home/oscerd/connectors |
| ... |
| ---- |
| |
| === Setup the connectors |
| |
| Open the Docker configuration file at `$EXAMPLES/docker/docker-source-events/config/CamelDockerSourceConnector.properties` |
| |
| [source] |
| ---- |
| name=CamelDockerSourceConnector |
| connector.class=org.apache.camel.kafkaconnector.docker.CamelDockerSourceConnector |
| tasks.max=1 |
| |
| key.converter=org.apache.kafka.connect.storage.StringConverter |
| value.converter=org.apache.kafka.connect.storage.StringConverter |
| |
| topics=mytopic |
| |
| camel.component.docker.host=/var/run/docker.sock |
| camel.component.docker.socket=true |
| camel.source.path.operation=events |
| ---- |
| |
| In this case we are using local unix socket. |
| |
| === Running the example |
| |
| Run the kafka connect with the Docker Source connector: |
| |
| [source] |
| ---- |
| $KAFKA_HOME/bin/connect-standalone.sh $KAFKA_HOME/config/connect-standalone.properties $EXAMPLES/docker/docker-source-events/config/CamelDockerSourceConnector.properties |
| ---- |
| |
| In my local docker enviroment I have the Debian image for example |
| |
| [source] |
| ---- |
| > docker images |
| REPOSITORY TAG IMAGE ID CREATED SIZE |
| debian latest e7d08cddf791 8 days ago 114MB |
| ---- |
| |
| and I want to delete it and check the events in the mytopic Kafka topic |
| |
| [source] |
| ---- |
| > docker rmi e7d08cddf791 |
| Untagged: debian:latest |
| Untagged: debian@sha256:b16f66714660c4b3ea14d273ad8c35079b81b35d65d1e206072d226c7ff78299 |
| Deleted: sha256:e7d08cddf791fe3245267654331eb21b805458b3412d3680.11.59355855044a3 |
| Deleted: sha256:4762552ad7d851a9901571428078281985074e5ddb806979dd7ad24748db4ca0 |
| ---- |
| |
| I'm expecting 4 events in the kafka topic |
| |
| On a different terminal run the kafkacat consumer |
| |
| [source] |
| ---- |
| ./kafkacat -b localhost:9092 -t mytopic |
| % Auto-selecting Consumer mode (use -P or -C to override) |
| Event(status=destroy, id=1ef0f76d93ce49011ae134b4b60350d699c9b3c914f0b76f69d692aae9b5b2b7, from=debian, node=null, type=CONTAINER, action=destroy, actor=EventActor(id=1ef0f76d93ce49011ae134b4b60350d699c9b3c914f0b76f69d692aae9b5b2b7, attributes={image=debian, name=test}), time=1611124880, timeNano=1611124880722895527) |
| Event(status=untag, id=sha256:e7d08cddf791fe3245267654331eb21b805458b3412d3680.11.59355855044a3, from=null, node=null, type=IMAGE, action=untag, actor=EventActor(id=sha256:e7d08cddf791fe3245267654331eb21b805458b3412d3680.11.59355855044a3, attributes={name=sha256:e7d08cddf791fe3245267654331eb21b805458b3412d3680.11.59355855044a3}), time=1611124882, timeNano=1611124882129721717) |
| Event(status=untag, id=sha256:e7d08cddf791fe3245267654331eb21b805458b3412d3680.11.59355855044a3, from=null, node=null, type=IMAGE, action=untag, actor=EventActor(id=sha256:e7d08cddf791fe3245267654331eb21b805458b3412d3680.11.59355855044a3, attributes={name=sha256:e7d08cddf791fe3245267654331eb21b805458b3412d3680.11.59355855044a3}), time=1611124882, timeNano=1611124882131312712) |
| Event(status=delete, id=sha256:e7d08cddf791fe3245267654331eb21b805458b3412d3680.11.59355855044a3, from=null, node=null, type=IMAGE, action=delete, actor=EventActor(id=sha256:e7d08cddf791fe3245267654331eb21b805458b3412d3680.11.59355855044a3, attributes={name=sha256:e7d08cddf791fe3245267654331eb21b805458b3412d3680.11.59355855044a3}), time=1611124882, timeNano=1611124882347718909) |
| % Reached end of topic mytopic [0] at offset 4 |
| ---- |
