The operation was canceled #6097

ocengiz0 · 2019-05-03T17:39:34Z

Describe the bug
I often get this error even I updated the nuget package to latest
(regarding similar issue -> Azure/azure-service-bus-dotnet#639)

Exception or Stack Trace

---> System.OperationCanceledException: The operation was canceled. at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(
at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result
at Microsoft.Azure.Amqp.ReceivingAmqpLink.EndReceiveMessages(IAsyncResult result, IEnumerable1& messages
at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task
at Microsoft.Azure.ServiceBus.Core.MessageReceiver.d__86.MoveNext()
--- End of inner exception stack trace --- at Microsoft.Azure.ServiceBus.Core.MessageReceiver.d__86.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task
at Microsoft.Azure.ServiceBus.Core.MessageReceiver.<>c__DisplayClass64_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task
at Microsoft.Azure.ServiceBus.RetryPolicy.d__19.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(
at Microsoft.Azure.ServiceBus.RetryPolicy.d__19.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task
at Microsoft.Azure.ServiceBus.Core.MessageReceiver.d__64.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task
at Microsoft.Azure.ServiceBus.Core.MessageReceiver.d__62.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task
at Microsoft.Azure.ServiceBus.MessageReceivePump.d__11.MoveNext()```

**To Reproduce**
I can not reproduce it.

The text was updated successfully, but these errors were encountered:

kurtzeborn · 2019-05-03T18:36:52Z

Thank you for opening this issue! We are routing it to the appropriate team for follow up.

CC: @ramya-rao-a, @AlexGhiondea

ramya-rao-a · 2019-05-03T18:53:52Z

@kurtzeborn In this case, it would be @jsquire and @AlexGhiondea :)

ocengiz0 · 2019-05-20T08:02:59Z

Any update @jsquire and @AlexGhiondea ?

AlexGhiondea · 2019-05-20T17:01:52Z

@ocengiz0 from looking at the callstack there isn't sufficient information to try and figure out the root cause. Would you happen to have a repro that we can look at?

ocengiz0 · 2019-05-21T12:54:56Z

The only way I can reproduce is that while in a long debug session, I receive this error in my ExceptionReceivedEvent method in which implemented to MessageHandlerOptions
_subscriptionClient is ISubscriptionClient

 var messageHandlerOptions = new MessageHandlerOptions(ExceptionReceivedHandler)
            {
                MaxConcurrentCalls = 1,
                AutoComplete = false
            };
_subscriptionClient.RegisterMessageHandler(ProcessMessagesAsync, messageHandlerOptions);
     
private Task ExceptionReceivedHandler(ExceptionReceivedEventArgs exceptionReceivedEventArgs)
        {
            var context = exceptionReceivedEventArgs.ExceptionReceivedContext;
            var exception = exceptionReceivedEventArgs.Exception;
            _logger.LogError(exception, $"Message handler encountered an exception: {exception.Message}, Endpoint: {context.Endpoint}, Entity Path: {context.EntityPath}, Executing Action: {context.Action}");
            return Task.CompletedTask;
        }

dwaijam · 2019-05-31T20:28:34Z

We are getting this exception too in our production. However its difficult to repro locally

                messages = await eventHubReceiver.ReceiveAsync(64);

dwaijam · 2019-05-31T20:30:17Z

Our callstack is
System.OperationCanceledException: The operation was canceled. at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result) at Microsoft.Azure.Amqp.ReceivingAmqpLink.EndReceiveMessages(IAsyncResult result, IEnumerable1& messages) at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task`1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.EventHubs.Amqp.AmqpPartitionReceiver.OnReceiveAsync(Int32 maxMessageCount, TimeSpan waitTime) at Microsoft.Azure.EventHubs.Amqp.AmqpPartitionReceiver.OnReceiveAsync(Int32 maxMessageCount, TimeSpan waitTime) at Microsoft.Azure.EventHubs.PartitionReceiver.ReceiveAsync(Int32 maxMessageCount, TimeSpan waitTime)

ocengiz0 · 2019-06-04T07:01:28Z

Get similar error and new stacktrace:

Microsoft.Azure.ServiceBus.ServiceBusException: The operation was canceled. ---> System.OperationCanceledException: The operation was canceled.
   at Microsoft.Azure.Amqp.ExceptionDispatcher.Throw(Exception exception)
   at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result)
   at Microsoft.Azure.Amqp.ReceivingAmqpLink.ReceiveAsyncResult.End(IAsyncResult result, IEnumerable`1& messages)
   at Microsoft.Azure.Amqp.ReceivingAmqpLink.EndReceiveMessages(IAsyncResult result, IEnumerable`1& messages)
   at Microsoft.Azure.ServiceBus.Core.MessageReceiver.<>c__DisplayClass86_1.<OnReceiveAsync>b__1(IAsyncResult a) in C:\source\azure-service-bus-dotnet\src\Microsoft.Azure.ServiceBus\Core\MessageReceiver.cs:line 1037
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Azure.ServiceBus.Core.MessageReceiver.OnReceiveAsync(Int32 maxMessageCount, TimeSpan serverWaitTime) in C:\source\azure-service-bus-dotnet\src\Microsoft.Azure.ServiceBus\Core\MessageReceiver.cs:line 1035
   --- End of inner exception stack trace ---
   at Microsoft.Azure.ServiceBus.Core.MessageReceiver.OnReceiveAsync(Int32 maxMessageCount, TimeSpan serverWaitTime) in C:\source\azure-service-bus-dotnet\src\Microsoft.Azure.ServiceBus\Core\MessageReceiver.cs:line 1068
   at Microsoft.Azure.ServiceBus.Core.MessageReceiver.<>c__DisplayClass64_0.<<ReceiveAsync>b__0>d.MoveNext() in C:\source\azure-service-bus-dotnet\src\Microsoft.Azure.ServiceBus\Core\MessageReceiver.cs:line 375
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.Azure.ServiceBus.RetryPolicy.RunOperation(Func`1 operation, TimeSpan operationTimeout) in C:\source\azure-service-bus-dotnet\src\Microsoft.Azure.ServiceBus\RetryPolicy.cs:line 85
   at Microsoft.Azure.ServiceBus.RetryPolicy.RunOperation(Func`1 operation, TimeSpan operationTimeout) in C:\source\azure-service-bus-dotnet\src\Microsoft.Azure.ServiceBus\RetryPolicy.cs:line 108
   at Microsoft.Azure.ServiceBus.Core.MessageReceiver.ReceiveAsync(Int32 maxMessageCount, TimeSpan operationTimeout) in C:\source\azure-service-bus-dotnet\src\Microsoft.Azure.ServiceBus\Core\MessageReceiver.cs:line 380
   at Microsoft.Azure.ServiceBus.Core.MessageReceiver.ReceiveAsync(TimeSpan operationTimeout) in C:\source\azure-service-bus-dotnet\src\Microsoft.Azure.ServiceBus\Core\MessageReceiver.cs:line 323
   at Microsoft.Azure.ServiceBus.MessageReceivePump.MessagePumpTaskAsync() in C:\source\azure-service-bus-dotnet\src\Microsoft.Azure.ServiceBus\MessageReceivePump.cs:line 64```

AlexGhiondea · 2019-06-24T23:30:39Z

/cc @nemakam @binzywu

ocengiz0 · 2019-07-08T20:59:01Z

Hello, any update?
@nemakam, @binzywu, @kurtzeborn @jsquire and @AlexGhiondea

nemakam · 2019-07-08T21:05:09Z

I started a long running instance which captures the state when we run into this issue. Waiting for the bug to reproduce..

Multiple errors can happen when link/session is closing. Throw ServiceBusCommunicationException instead of ServiceBusException. Fixes #6097

Vikhyat08 · 2020-01-20T11:58:02Z

@ocengiz0 I a getting a similar issue #9543
I get into this state when message processing is taking a long time. I am getting Task canceled and service bus timeout exception. And getting these exceptions in bulk. How to handle these exceptions ?

Marusyk · 2020-11-16T21:26:37Z

I have the same #16975

triage-new-issues bot added the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label May 3, 2019

ocengiz0 mentioned this issue May 3, 2019

The operation was canceled Azure/azure-service-bus#282

Closed

kurtzeborn added Client This issue points to a problem in the data-plane of the library. Service Bus labels May 3, 2019

triage-new-issues bot removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label May 3, 2019

AlexGhiondea added bug This issue requires a change to an existing behavior in the product in order to be resolved. Service Attention Workflow: This issue is responsible by Azure service team. labels Jun 24, 2019

jfggdl assigned nemakam Jul 1, 2019

nemakam mentioned this issue Jul 16, 2019

Throwing communicationException when it is connectionError #6942

Merged

jfggdl added this to the Sprint 157 milestone Jul 16, 2019

nemakam closed this as completed in #6942 Jul 16, 2019

nemakam added a commit that referenced this issue Jul 16, 2019

Throwing communicationException when it is connectionError (#6942)

58b0c96

Multiple errors can happen when link/session is closing. Throw ServiceBusCommunicationException instead of ServiceBusException. Fixes #6097

github-actions bot locked and limited conversation to collaborators Mar 29, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The operation was canceled #6097

The operation was canceled #6097

ocengiz0 commented May 3, 2019 •

edited by AlexGhiondea

Loading

kurtzeborn commented May 3, 2019

ramya-rao-a commented May 3, 2019

ocengiz0 commented May 20, 2019

AlexGhiondea commented May 20, 2019

ocengiz0 commented May 21, 2019 •

edited

Loading

dwaijam commented May 31, 2019

dwaijam commented May 31, 2019

ocengiz0 commented Jun 4, 2019

AlexGhiondea commented Jun 24, 2019

ocengiz0 commented Jul 8, 2019

nemakam commented Jul 8, 2019

Vikhyat08 commented Jan 20, 2020

Marusyk commented Nov 16, 2020

The operation was canceled #6097

The operation was canceled #6097

Comments

ocengiz0 commented May 3, 2019 • edited by AlexGhiondea Loading

kurtzeborn commented May 3, 2019

ramya-rao-a commented May 3, 2019

ocengiz0 commented May 20, 2019

AlexGhiondea commented May 20, 2019

ocengiz0 commented May 21, 2019 • edited Loading

dwaijam commented May 31, 2019

dwaijam commented May 31, 2019

ocengiz0 commented Jun 4, 2019

AlexGhiondea commented Jun 24, 2019

ocengiz0 commented Jul 8, 2019

nemakam commented Jul 8, 2019

Vikhyat08 commented Jan 20, 2020

Marusyk commented Nov 16, 2020

ocengiz0 commented May 3, 2019 •

edited by AlexGhiondea

Loading

ocengiz0 commented May 21, 2019 •

edited

Loading