Microsoft.Azure.ServiceBus.ServiceBusTimeoutException being recieved in bulk

[Vikhyat08]

I am using Microsoft.Azure.ServiceBus v 4.1.1 .NET Standard client library and stuck, running into some exceptions while using the SDK.
The exceptions are received in the exception handler registered with queueClient. These exceptions did not come up while using WindowsAzure.ServiceBus v 5.2.0
I have tried both queueclient with RetryPolicy.Default and RetryPolicy.NoRetry. Also I have tried both Amqp and AmqpWebSockets

I can reproduce this issue locally,
If i pause execution in message processing I am getting these exceptions in bulk. Not sure how to handle them.

Here is how I’m creating the queue and queueClient.

var serviceBusConnectionStringBuilder = new ServiceBusConnectionStringBuilder(namespaceConnectionString);
serviceBusConnectionStringBuilder.TransportType = TransportType.AmqpWebSockets; 
var namespaceConnectionStringWithTransportType = serviceBusConnectionStringBuilder.ToString();
var managementClient = new ManagementClient(namespaceConnectionStringWithTransportType);
if (!await managementClient.QueueExistsAsync(queueName))
{
       await managementClient.CreateQueueAsync(queueDescription);
}
QueueClient queueClient = new QueueClient(namespaceConnectionString, queueName, ReceiveMode.PeekLock);

Here is how I am registering the messageHandler.

Task ExceptionReceivedHandler(ExceptionReceivedEventArgs exceptionReceivedEventArgs)
{
       //Log Exception.
}

MessageHandlerOptions messageOptions = new MessageHandlerOptions(ExceptionReceivedHandler)
{
AutoComplete = false,
MaxConcurrentCalls = 20
};

async Task ProcessMessagesAsync(Message message, CancellationToken token)
{
       //ProcessMessage
}

queueClient.RegisterMessageHandler(ProcessMessagesAsync,messageOptions);

Attaching Exception trace -

1. Microsoft.Azure.ServiceBus.ServiceBusTimeoutException: The operation did not complete within the allocated time 00:00:00.0084284 for object out-connection83531. ---> System.TimeoutException: The operation did not complete within the allocated time 00:00:00.0084284 for object out-connection83531.
   Here the allocated time value is random.
   Full exception trace -
   Microsoft.Azure.ServiceBus.ServiceBusTimeoutException: The operation did not complete within the allocated time 00:00:00.0084284 for object out-connection83531. ---> System.TimeoutException: The operation did not complete within the allocated time 00:00:00.0084284 for object out-connection83531. at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result) at Microsoft.Azure.Amqp.AmqpObject.OpenAsyncResult.End(IAsyncResult result) at Microsoft.Azure.Amqp.AmqpObject.EndOpen(IAsyncResult result) at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.ServiceBus.ServiceBusConnection.d__45.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.Amqp.FaultTolerantAmqpObject1.<OnCreateAsync>d__6.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.Amqp.Singleton1.d__13.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.Amqp.Singleton1.<GetOrCreateAsync>d__13.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.ServiceBus.Amqp.AmqpLinkCreator.<CreateAndOpenAmqpLinkAsync>d__11.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.ServiceBus.Core.MessageReceiver.<CreateLinkAsync>d__101.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.Amqp.FaultTolerantAmqpObject1.d__6.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.Amqp.Singleton1.<GetOrCreateAsync>d__13.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.Amqp.Singleton1.d__13.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.ServiceBus.Core.MessageReceiver.d__86.MoveNext() --- End of inner exception stack trace --- at Microsoft.Azure.ServiceBus.Core.MessageReceiver.d__86.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.ServiceBus.Core.MessageReceiver.<>c__DisplayClass64_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.ServiceBus.RetryPolicy.d__19.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.Azure.ServiceBus.RetryPolicy.d__19.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.ServiceBus.Core.MessageReceiver.d__64.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.ServiceBus.Core.MessageReceiver.d__62.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.ServiceBus.MessageReceivePump.<b__11_0>d.MoveNext()

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jfggdl

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jfggdl

[jsquire]

@nemakam and @binzywu: Would you be so kind as to offer your thoughts?

[nemakam]

@Vikhyat08 , TimeoutExceptions can happen on service side, and the best way to handle this is to retry. When the process is paused in debugger, these are expected since the background threads cannot finish the operation within the time.
If you face several timeouts (without debugger), then please raise a support request through the portal and the support team will help with investigating why the operations are timing out on the service.