Nestjs-kafkajs-integration

Demonstration of how to integrate kafkajs with nestjs and create a priority messaging queue.

Architecture

Fololowing is the architecture of the application.

• A topic with 10 partitions is created by the kafka-admin.service.ts file.
• The producer.service.ts has the logic to push messages as per their priority in their respective partitions.
• The app.service.ts produces messages with the correct key when localhost:3000 is queried.
• The test.consumer.ts file creates 10 consumers.
• The consumer.service.ts file maps the consumers to the respective partitions.

Environment setup

• Install JDK8
• Install and run the following
  • zookeeper - It keeps a track of kafka clusters and brokers within it.
  • kakfa - The message streaming service from apache.
• How to install kafka: https://kafka.apache.org/quickstart

Environment variables

• Copy the contents of .env.example file in .env file

Installation

Nestjs-kafkajs-integration requires Node.js v16+ to run.

Install the dependencies and devDependencies and start the server.

cd nestjs-kafkajs-integration
npm i
npm run start:dev

Author

Faizal Vasaya

License

MIT

Free Software, Hell Yeah!