Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support repeatedly calling WhenAny on a progressively smaller list of already scheduled tasks #446

Merged
merged 2 commits into from
Aug 4, 2023

Conversation

davidmrdavid
Copy link
Collaborator

@davidmrdavid davidmrdavid commented Aug 2, 2023

Fixes: #349

Today, we following orchestrator code throws an exception.

def orchestrator_function(context: df.DurableOrchestrationContext):
    #  Create some tasks
    task1_until = context.current_utc_datetime + datetime.timedelta(seconds=30)
    task1 = context.create_timer(task1_until)
    task2_until = context.current_utc_datetime + datetime.timedelta(seconds=15)
    task2 = context.create_timer(task2_until)
    pending_tasks = [task1, task2]

    #  Yield until first task is completed
    finished_task1 = yield context.task_any(pending_tasks)

    #  Remove completed task from pending tasks
    pending_tasks.remove(finished_task1)

    #  Yield remaining task
    yield context.task_any(pending_tasks)

This exception comes from the incorrect assumption that given list of sub-tasks will not be yield'ed multiple times on different parent tasks. In this case, we have two WhenAny tasks that both operate on a shared subset of the pending_tasks list.

This PR addresses this by ensuring that the second WhenAny only schedules new actions for the set of sub-tasks that have not been already scheduled. Put more simply: we keep track of whether or not an action for a given task was already generated and, if so, we ignore it future attempts at scheduling it.

@davidmrdavid davidmrdavid marked this pull request as ready for review August 2, 2023 23:23
Copy link
Contributor

@nytian nytian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanations and LGTM!

@davidmrdavid davidmrdavid merged commit 5d30ae3 into dev Aug 4, 2023
@davidmrdavid davidmrdavid deleted the dajusto/support-reducing-when-any branch August 4, 2023 17:41
@nytian nytian added bug Something isn't working and removed bug Something isn't working labels Aug 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Reusing tasks makes a call to append attribute
2 participants