Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(engine): Improved DAG scheduling algorithm #827

Merged
merged 1 commit into from
Feb 5, 2025
Merged

Conversation

daryllimyt
Copy link
Contributor

@daryllimyt daryllimyt commented Feb 5, 2025

We don't just brainlessly tick and poll - instead we wait for tasks to complete to awake the scheduling coroutine.

Impact

  • Removes 1s asyncio.sleep() which creates Temporal timers
  • Cut tests duration by ~1.5min

Copy link

zeropath-ai bot commented Feb 5, 2025

No security or compliance issues detected. Reviewed everything up to fdc7f5e.

Security Overview
  • 🔎 Scanned files: 1 changed file(s)
Detected Code Changes
Change Type Relevant files
Enhancement ► scheduler.py
    Optimize task scheduling to avoid setting timers
    Add pending tasks management
    Improve task cleanup and cancellation

Reply to this PR with @zeropath-ai followed by a description of what change you want and we'll auto-submit a change to this PR to implement it.

@daryllimyt daryllimyt changed the title perf(engine): Avoid setting timers in scheduler perf(engine): Improved scheduling algorithm Feb 5, 2025
@daryllimyt daryllimyt changed the title perf(engine): Improved scheduling algorithm perf(engine): Improved DAG scheduling algorithm Feb 5, 2025
@daryllimyt daryllimyt merged commit 2fd1036 into main Feb 5, 2025
9 checks passed
@daryllimyt daryllimyt deleted the better-scheduler branch February 5, 2025 00:26
daryllimyt added a commit that referenced this pull request Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Changes that improve performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant