intercept Orchestrator before/after setting the runtimeStatus

[thdotnet]

When working with several sub activities on an orchestrator. If some activity exceeds the number of retries, it will set the orchestrator status as "runtimeStatus:Failed". I would like the ability to intercept this moment and perform some action.

[ConnorMcMahon]

You actually can already do this. These failed activities will throw a FunctionFailedException, which you can catch yourself and perform code in a try-catch block.

[thdotnet]

The retry logic is implemented by Durable Functions. I would like a point to intercept before / after it set this state.

In summary, some extensibility to enable AOP

[ConnorMcMahon]

@thdotnet

I'm not quite sure what you mean.

Let's say you have the following orchestration:

public static async Task<string> CatchException([OrchestrationTrigger] IDurableOrchestrationContext context)
{
    try
    {
         return await context.CallActivityWithRetryAsync<string>("PrimaryAction", new RetryOptions(TimeSpan.FromSeconds(3), 3), input);
    } catch (FunctionFailedException)
    {
         return await context.CallActivityAsync<string>("BackupAction", input);
    }
}

Now let's say that PrimaryAction fails 3 times. This replay, context.CallActivityWithRetryAsync will throw a FunctionFailedException, and therefore we will go into the catch block and execute BackupAction. The orchestrator will never go into the Failed state, as it never had an uncaught exception.

[thdotnet]

You got the idea. But I would like something like this:

public static async Task<string> CatchException([OrchestrationTrigger] IDurableOrchestrationContext context)
{
         context.OnFunctionFailedException("MyFunction"); 

         //or

         context.OnFunctionFailedException(s=> Foo()); 

         await context.CallActivityWithRetryAsync<string>("FunctionA", new RetryOptions(TimeSpan.FromSeconds(3), 3), input);
await context.CallActivityWithRetryAsync<string>("FunctionB", new RetryOptions(TimeSpan.FromSeconds(3), 3), input);
await context.CallActivityWithRetryAsync<string>("FunctionC", new RetryOptions(TimeSpan.FromSeconds(3), 3), input);
}

[ConnorMcMahon]

I'm not quite sure what utility that adds that catching the exception manually does not add, but I will leave it open for dicussion.

[thdotnet]

the benefit is quite simple. I can have one single orchestrator or multiple sub orchestrators. I want to setup a single entry point to capture failures, but this could be used for other purposes too. For example, in the past there was the Invocation Filters that was removed, but allow Cross Cutting Concerns. I would like the same ability, but in the orchestrator.

[ltouro]

I think the "before failure" part is easy to implement in user-land code.

You can provide an extension method to IDurableOrchestrationContext that accepts an additional delegate parameter "OnFailure" and wrap it around a try/catch block as Connor suggested.

public override System.Threading.Tasks.Task<TResult> CallActivityWithRetryAsyncAndFailureHandler<TResult> (string functionName, Microsoft.Azure.WebJobs.RetryOptions retryOptions, object input, Action<Exception> onError);

I think the "after failure" part is harder. But from your example, "just before failure" might be enough.

[thdotnet]

Agreed, in fact it should be:

Before / After -> activity
Before / After -> Orchestrator

[ltouro]

Oh, I see now. I think using the extension methods are viable for both before and after for activity functions (using 3 extra params, "before", "after", "onError").

The "before Orchestrator starts" is also viable (adding the extension method on IDurableOrchestrationClient interface, i.e, extending StartNewAsync).

The hard part is the "after orchestration is completed/failed". I suggest you to check WaitForCompletionOrCreateCheckStatusResponseAsync method. It uses some sort of pooling to keep checking the status on the orchestration. Not ideal given the amount of resource usage you would have.

Other option is to add extra activities to all your orchestrations. Altough not very "cross-cutting", these activities would be pretty generic and reusable.

[thdotnet]

as I said, I can implement everything by myself...but I think they would be a nice feature at the framework level