Support for 0 length messages- kafka limitation or librdkafka?

[winbatch]

There are cases where I want to send only the partition key but an empty payload. This gets rejected- is this a librdkafka limitation or a librdkafka enforcement of a kafka rule?

[edenhill]

Its an assert in librdkafka, but I dont think this is a kafka rule.
I will check if Kafka supports it and then remove the librdkafka limitation for it if so.
I'll get back to you.

[winbatch]

I guess 0 is ok? We should check that compression doesn't fail ;)

On Wednesday, January 15, 2014, Magnus Edenhill wrote:

Closed #60 #60.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/60
.

[edenhill]

payload length 0 is now okay, and yes, compression works for these empty messages :)

[winbatch]

I was surprised to get this in my callback. I'd zero is ok, it shoukdnt complain or at least should just be shown with debugging on.
16-Jan-2014 09:57:33:214 [T03][13156] kafka/kafka.cc:49 [TEST] fac:'GZIP' name:'LARA_replicator#producer-0' log:'emsdc136235-001.masked.com:9092/0: Failed to gzip-compress0 bytes for topic danP3R3 [2]: buffer error (1): sending uncompressed'

[winbatch]

Note I don't get this callback with snappy

[winbatch]

Looks like the rdkafka_example cannot handle an empty message. I tried a java consumer and it's fine.

[winbatch]

Never mind, mistake at my end

[edenhill]

The gzip warning is now gone.
Thanks!