Google Git
Sign in
apache / beam / refs/heads/zip_to_tar / . / sdks / java / io / kafka
tree: ca7367dd194ead2e1c7f6e4661b847dbd6e5ff9e [path history] [tgz]
  1. kafka-01103/
  2. kafka-100/
  3. kafka-111/
  4. kafka-201/
  5. kafka-211/
  6. kafka-222/
  7. kafka-231/
  8. kafka-241/
  9. kafka-251/
  10. src/
  11. build.gradle
  12. kafka-integration-test.gradle
  13. README.md
sdks/java/io/kafka/README.md

KafkaIO contains I/O transforms which allow you to read/write messages from/to Apache Kafka.

Dependencies

To use KafkaIO you must first add a dependency on beam-sdks-java-io-kafka. KafkaIO supports multiple versions of Kafka clients at run time. It does not pull a specific version kafka-clients transitively. You need to include a compatible version of kafka-clients as runtime dependency. Usually current and recent versions of Kafka are supported, please see JavaDoc for KafkaIO for complete list.

<dependency>
    <groupId>org.apache.beam</groupId>
    <artifactId>beam-sdks-java-io-kafka</artifactId>
    <version>...</version>
</dependency>

<dependency>
  <groupId>org.apache.kafka</groupId>
  <artifactId>kafka-clients</artifactId>
  <version>a_recent_version</version>
  <scope>runtime</scope>
</dependency>

Documentation

The documentation is maintained in JavaDoc for KafkaIO class. It includes usage examples and primary concepts.