Handle RecordTooLargeException #191
Conversation
Retrying will fail so catch the exception
Previous location was returning error
| @@ -262,6 +262,9 @@ def retrying_send(batch) | |||
| rescue org.apache.kafka.common.errors.InterruptException => e | |||
| failures << record | |||
| nil | |||
| rescue org.apache.kafka.common.errors.RecordTooLargeException => e | |||
There was a problem hiding this comment.
KafkaProducer::send cannot throw a RecordTooLargeException, so this PR does not make sense.
There was a problem hiding this comment.
It is a Runtime exception that is unchecked
There was a problem hiding this comment.
I would expect it to be thrown from the future.get() call a little down the code. Either way this PR does not help because the same (by default infinite) retry logic is still there; #193 proposes a better solution by only retrying sending events when a RetriableException is thrown.
|
I do agree that any However, this plugin currently does not have a way to distinguish between permanent and temporary failures. There is only a configurable retry limit ( #193 proposes a better solution to handle these kinds of failures. |
|
Closing as this has been fixed in logstash-plugins/logstash-integration-kafka#29, in Logstash Kafka Integration plugin version 10.5.0 |
RecordTooLargeException is similar to SerializationException. The retry will fail forever and will be caught in an endless loop.
See issue #190