ServiceBus doesn't complete the messages from the prefetch buffer

[okyryliuk]

Describe the bug
If the lock expires in the prefetch buffer, the message is treated as abandoned and is again made available for retrieval from the queue. When there is more than a single failed message in the prefetch buffer, the client tries to process those messages again but doesn't complete any message even if they are processed successfully on the second attempt. If there is one failed message in the prefetch buffer, and it may be completed on the second attempt, all messages can be completed successfully.

To Reproduce

• The message lock duration is set to 10 seconds
• Prefetch count is set to 5 messages
• At least 2 messages should be processed more than the lock duration time

Code Snippet

ServiceBusReceiverAsyncClient receiver = new ServiceBusClientBuilder()
        .connectionString(CONNECTION_STRING)
        .receiver()
        .disableAutoComplete()
        .prefetchCount(5)
        .topicName(TOPIC)
        .subscriptionName(SUBSCRIPTION)
        .buildAsyncClient();

    AtomicInteger index = new AtomicInteger();
    receiver.receiveMessages()
        .flatMap(message -> {
          System.out.printf("Sequence #: %s. Contents: %s%n", message.getSequenceNumber(), message.getBody());
          try {
            int sleepTime = index.get() % 2 == 0 ? 12000 : 0;
            Thread.sleep(sleepTime);
            index.getAndIncrement();
          } catch (InterruptedException e) {
            e.printStackTrace();
          }
          return receiver
              .complete(message)
              .onErrorResume(e -> {
                e.printStackTrace();
                return Mono.empty();
              });
        })
        .doOnError(e -> System.out.println("ERROR: " + e.getMessage()))
        .doOnComplete(() -> System.out.println("DONE"))
        .subscribe();

Expected behavior
All messages from the prefetch buffer should be completed successfully after the successful attempt.

Setup (please complete the following information):

• OS: Windows
• IDE : IntelliJ
• Version of the Library used: 7.1.0

[joshfree]

@YijunXieMS could you please take a look?

/cc @hemanttanwar

[anuchandy]

Please refer to this git-ticket #26632 where the crew evaluated this. The decision is not to do anything on the client-side to handle such expired messages.

Also refer this section on general guideline https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-prefetch?tabs=dotnet#why-is-prefetch-not-the-default-option

The maximum prefetch count and the lock duration configured on the queue or subscription need to be balanced such that the lock timeout at least exceeds the cumulative expected message processing time for the maximum size of the prefetch buffer, plus one message. At the same time, the lock timeout shouldn't be so long that messages can exceed their maximum time to live when they're accidentally dropped, and so requiring their lock to expire before being redelivered.

Closing this ticket based on the above data.