[BUG] We sometimes (incorrectly) get "Error sending. Size of the payload exceeded maximum message size" error when scheduling a message #41605
Assignees
Labels
Client
This issue points to a problem in the data-plane of the library.
customer-reported
Issues that are reported by GitHub users external to the Azure organization.
needs-team-attention
Workflow: This issue needs attention from Azure service team or SDK team
question
The issue doesn't require a change to the product in order to be resolved. Most issues start as that
Service Bus
Comments
github-actions
bot
added
Client
|
Thank you for your feedback. Tagging and routing to the team member best able to assist. |
|
Hi @haraldk, could you help with a reference message that ran into this error? Basically, you would update code to print one of the impacted messages, which I can use to reproduce this. The code change look likes try {
serviceBusSenderClient.scheduleMessage(retryMessage, OffsetDateTime.now().plusSeconds(delay));
} catch (ServiceBusException sbe) {
if (sbe.getCause() != null && sbe.getCause() instanceof AmqpException) {
final AmqpException amqpException = (AmqpException) sbe.getCause();
if (amqpException.getErrorCondition() == AmqpErrorCondition.LINK_PAYLOAD_SIZE_EXCEEDED) {
final Throwable innerCause = amqpException.getCause();
if (innerCause != null && innerCause instanceof BufferOverflowException) {
System.out.println(innerCause.getMessage());
MessagePrinter.print(retryMessage);
}
}
}
throw sbe;
}The "MessagePrinter" utility class used above can be found here https://gist.github.com/anuchandy/5b08d59e2ac0528dc58b99541c8765f1 |
|
Thanks @anuchandy! I've modified your code slightly to use our logging, but it should otherwise output the same details. Will update the issue with the logged information as soon as possible. |
|
Here's the output of the extra output from a few occurrences extracted from our logs: The exception message wasn't very helpful in this case, but I hope the extra details can help in finding the issue! 😀 |
|
Hi @haraldk, thanks! it looks like the value of the field "Message.Properties.userId" is redacted, possibly a filter in the logging configuration? Could you double check and if so, can we output few new impacted messages again including this field value? |
|
Hi @@haraldk, I can repro using the shared messages, no need to pull another set of messages (with userId) |
|
Excellent news! I’ll have look, and will test as soon as possible. 😀 |
|
The version (7.17.14) with the fix should be available in September release. Closing, now that the changes are merged to the main. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Describe the bug
We sometimes (I believe incorrectly) get "Error sending. Size of the payload exceeded maximum message size: 256 kb" error when scheduling a message. The message is a copy of a received message, with a single added client property to control the number of times it is re-scheduled. The error message does not make sense to me, as the message was previously sent fine, and the full message body is at most a few hundred bytes (< 1 kb). It is nowhere near the 256 kb limit.
The scheduling (usually) also works after a few retries.
The cause of the exception, however, says "java.lang.IndexOutOfBoundsException: Requested min remaining bytes(155) exceeds remaining(137) in underlying ByteBuffer: java.nio.HeapByteBuffer[pos=636 lim=773 cap=773]". That makes more sense, but I don't know how/why the limit is set to 773 bytes, as that is outside the control of the client code.
Exception or Stack Trace
To Reproduce
Steps to reproduce the behavior:
This happens seemingly at random, and only a very small percentage of the times we invoke
com.azure.messaging.servicebus.ServiceBusSenderClient.scheduleMessage. I don't have a reproducer that can pinpoint the problem at this point, but the full method where the problem happens is attached below. I believe running the code below either by only using the exception path, or by replacingmessageConsumer.consume(message)with something likethrow new RuntimeException("Oh no")and running it enough times will show the problem.As mentioned, the message is (usually) scheduled after a few retries.
Code Snippet
Add the code snippet that causes the issue.
Our
processMessagemethod (passed on toServiceBusProcessorClientBuilder.processMessage()) looks like this:Expected behavior
The message should be scheduled without exception.
Screenshots
If applicable, add screenshots to help explain your problem.
Setup (please complete the following information):
Additional context
Add any other context about the problem here.
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report
The text was updated successfully, but these errors were encountered: