This module provisions a Bitnami Kafka Cluster based on its helm chart. It uses the terraform helm provider Therefore, you DO NOT need helm to apply this module.
Simply follow standard terraform workflow to apply this module.
terraform init terraform apply
When applying this module to Google Kubernetes Engine (GKE) Autopilot you will see an “Unschedulable” status. This is because, GKE Autopilot needs time to scale up the node. After some time, the kubernetes cluster will provision the kafka cluster when these compute resources are available.
This module deploys a kafka client on the cluster to help with debugging and troubleshooting.
Run the following command to query the pod name.
kubectl get po -l app=kafka-client
You should see something similar to the following:
NAME READY STATUS RESTARTS AGE kafka-client-cdc7c8885-nmcjc 1/1 Running 0 4m12s
Run the following command to shell into the running container.
kubectl exec --stdin --tty kafka-client-cdc7c8885-nmcjc -- /bin/bash
The container executes using the latest bitnami/kafka image. It has installed all the necessary kafka-*.sh scripts in its path.
In all of the commands, you can use the flag: --bootstrap-server kafka:9092 because the pod is in the same kubernetes cluster and takes advantage of its domain name service (DNS). The bitnami helm operator creates a Kubernetes service called kafka that exposes port 9092.
The following command gives you the cluster ID and validates you can connect to the cluster.
kafka-cluster.sh cluster-id --bootstrap-server kafka:9092
The following command creates a Kafka topic.
kafka-topics.sh --create --topic some-topic --partitions 3 --replication-factor 3 --bootstrap-server kafka:9092
The following command queries information about a topic (assuming the name some-topic).
kafka-topics.sh --describe --topic some-topic --bootstrap-server kafka:9092
See https://kubernetes.io/docs/tasks/debug/debug-application/get-shell-running-container/