Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: handle kafka transport with confluentkafka ✨ #1574

Merged
merged 4 commits into from
Jul 2, 2024

Conversation

meysam81
Copy link
Contributor

@meysam81 meysam81 commented Jul 14, 2022

Based on my local & confluent tests, the same transport class can be used for both confluentkafka and kafka itself. I've done countless tests and it works.

Based on the works on #1506

@@ -24,6 +24,7 @@ def supports_librabbitmq() -> bool | None:
'pyamqp': 'kombu.transport.pyamqp:Transport',
'librabbitmq': 'kombu.transport.librabbitmq:Transport',
'confluentkafka': 'kombu.transport.confluentkafka:Transport',
'kafka': 'kombu.transport.confluentkafka:Transport',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we might add another back end in near future, like aiokafka, so i am not in favor of adding this change right now :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well @auvipy , for the moment, you would want to give some API to the already-implemented logic. If in the future, you plan to change the logic, you'd just change one line here. No hard feelings :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because you see, at the moment, I'm giving confluentkafka://kafka.kafka.svc.cluster.local to my app. which is ugly when you think about it.

@auvipy auvipy merged commit 4fb7246 into celery:main Jul 2, 2024
15 checks passed
@meysam81 meysam81 deleted the meysam/kafka-transport branch July 2, 2024 09:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants