An example repository of using Kafka with Keda hosted on Kubernetes.
If you arrived at this repository from a blog post and would like to follow the example or just want a more wordy explanation for the code, you can find a list below with the respective link and tag you need to checkout:
- Exploring KEDA - A brief look at KEDA using Kafka triggers
Link: https://cottonglow.github.io/2019-12-02-exploring-keda/
Code: Checkout tagexploring-keda
- Download Kafka (2.3.1)
- Download the Strimzi Kafka operators (0.14.0)
- Clone the Keda repository (v1.0.0)
- Docker and Kubernetes
-
Start the Kafka and Zookeeper servers. The following script assumes that strimzi is located in the same directory as this project.
./start-kafka.bat
-
Deploy the consumer
./start-consumer.bat
-
Deploy Keda. The following script assumes that keda is located in the same
./start-keda.bat
-
Now everything should be setup. You'll notice that the consumer-service pod may not be there as Keda scaled the deployment down to 0 as it isn't being used.
-
Send messages to the consumer.
cd <KAFKA INSTALL DIRECTORY>\bin\windows ./kafka-console-producer.bat --broker-list localhost:32100 --topic messages
-
You should now see the consumer scaling up as messages come in and down when there are no messages in the queue.