KafkaIO contains I/O transforms which allow you to read/write messages from/to Apache Kafka.

Dependencies

To use KafkaIO you must first add a dependency on beam-sdks-java-io-kafka. KafkaIO supports multiple versions of Kafka clients at run time. It does not pull a specific version kafka-clients transitively. You need to include a compatible version of kafka-clients as runtime dependency. Usually current and recent versions of Kafka are supported, please see JavaDoc for KafkaIO for complete list.

<dependency>
    <groupId>org.apache.beam</groupId>
    <artifactId>beam-sdks-java-io-kafka</artifactId>
    <version>...</version>
</dependency>

<dependency>
  <groupId>org.apache.kafka</groupId>
  <artifactId>kafka-clients</artifactId>
  <version>a_recent_version</version>
  <scope>runtime</scope>
</dependency>

Documentation

The documentation is maintained in JavaDoc for KafkaIO class. It includes usage examples and primary concepts.

• KafkaIO.java