Installing Kafka and Kafdrop on Kubernetes in Less Than 5 Minutes

1. Set Up k3d Cluster
First, create a k3d cluster with the following command. This setup includes two agent nodes and a load balancer.

k3d cluster create –api-port 6550 -p “80:80@loadbalancer” –agents 2

2. Create Kafka Namespace
Next, we’ll create a dedicated namespace for Kafka.

kubectl create ns kafka

3. Install Kafka Using Helm
Now, let’s add the Bitnami Helm repository and deploy Kafka with three replicas and persistent storage.

helm repo add bitnami https://charts.bitnami.com/bitnami

helm install kafka bitnami/kafka –namespace kafka –set replicaCount=3 –set controller.persistence.size=1Gi –set listeners.client.protocol=PLAINTEXT –set listeners.controller.protocol=PLAINTEXT –set listeners.interbroker.protocol=PLAINTEXT –set listeners.external.protocol=PLAINTEXT –set extraConfig=”default.replication.factor=3″

After the deployment, monitor the Kafka pods using this command.

kubectl get po -n kafka -w

4. Clone and Configure Kafdrop
Next, we’ll deploy Kafdrop, a web UI for managing Kafka. Start by cloning the Kafdrop GitHub repository.

git clone https://github.com/obsidiandynamics/kafdrop && cd kafdrop

Edit the values-override.yaml file to configure Kafdrop. Here’s an example configuration:

image:
tag: 4.0.2
kafka:
brokerConnect: kafka:9092
server:
servlet:
contextPath: /
cmdArgs: “–message.format=AVRO –schemaregistry.connect=http://localhost:8080”
service:
type: ClusterIP
ingress:
enabled: true
hosts:
– kafdrop.devopstipstricks.local

PS: You check the latest version from the link below;
https://hub.docker.com/r/obsidiandynamics/kafdrop/tags

5. Deploy Kafdrop Using Helm
Deploy Kafdrop with the custom configuration using Helm.

helm upgrade -i kafdrop chart -n kafka -f values-override.yaml

6. Configure Local Hosts File
Finally, edit your /etc/hosts file to resolve the Kafdrop domain locally.

sudo vi /etc/hosts

Add the following entry:

127.0.0.1 kafdrop.devopstipstricks.local

7. Access Kafdrop

That’s it! Open your browser and navigate to http://kafdrop.devopstipstricks.local to access the Kafdrop UI.

#kafka #kubernetes #apachekafka