KafkaIO contains I/O transforms which allow you to read/write messages from/to Apache Kafka.
To use KafkaIO you must first add a dependency on beam-sdks-java-io-kafka
. KafkaIO supports multiple versions of Kafka clients at run time. It does not pull a specific version kafka-clients
transitively. You need to include a compatible version of kafka-clients
as runtime dependency. Usually current and recent versions of Kafka are supported, please see JavaDoc for KafkaIO for complete list.
<dependency> <groupId>org.apache.beam</groupId> <artifactId>beam-sdks-java-io-kafka</artifactId> <version>...</version> </dependency> <dependency> <groupId>org.apache.kafka</groupId> <artifactId>kafka-clients</artifactId> <version>a_recent_version</version> <scope>runtime</scope> </dependency>
The documentation is maintained in JavaDoc for KafkaIO class. It includes usage examples and primary concepts.