Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support MQTT message expiry in mqtt-kafka mapping #631

Closed
Tracked by #512
jfallows opened this issue Dec 11, 2023 · 0 comments · Fixed by #640
Closed
Tracked by #512

Support MQTT message expiry in mqtt-kafka mapping #631

jfallows opened this issue Dec 11, 2023 · 0 comments · Fixed by #640
Assignees
Labels
enhancement New feature or request

Comments

@jfallows
Copy link
Contributor

Kafka messages natively expire based on topic retention policy rather than any setting on individual messages, whereas individual MQTT messages can expire independently.

MQTT v3.1.1 has no support for message expiry, but message expiry should still be honored for messages published via MQTT v5.
MQTT v5 CONNECT will message expiry interval
MQTT v5 PUBLISH message expiry interval

When message expiry interval is omitted, the message does not expire.

When message is delivered by MQTT server to client, message expiry interval must be adjusted for any wait time.

Therefore, it may be helpful to compute expiry as an absolute value based on Kafka message timestamp plus expiry interval and adjust during delivery.

Note: if expiry interval has already elapsed then the message should be filtered out and not delivered to an MQTT subscriber.

Note: retained messages that have already expired should not be delivered to an MQTT subscriber.

@jfallows jfallows added the enhancement New feature or request label Dec 11, 2023
@bmaidics bmaidics linked a pull request Dec 22, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants