Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(autofix): Prevent long wait times for insights threadpool at end of run #1631

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

roaga
Copy link
Member

@roaga roaga commented Dec 14, 2024

Issue #1630

I discovered in profiling that when the root cause step completed, there was sometimes a huge wait (the case I saw was 6 seconds) for the thread pool to finish it's last insight card generation, before moving onto the root cause formatter. (similar for the coding step)

Why? Because we started a new insight sharing thread after every LLM call, including the final call of the agent run. Due to Zach's changes that make us wait for threads to complete before ending the run, this caused big delays.

Solutions in this PR:

  1. Don't generate an insight card on the final output of the run. That was somewhat redundant anyway if we're about to show the final output card, plus this avoids starting a new thread from scratch that blocks everything else.
  2. Adds a second thread worker. That way while running in parallel with the main agent, one insight doesn't have to wait for the previous to complete, and it's unlikely that insights will "fall behind" the main agent.

We could completely eliminate this delay if we didn't wait for the threads to be completed, and I'm not sure that's a bad idea, but I'm still assuming we don't want to sacrifice that safety. This PR mitigates most of the issue while maintaining thread safety.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant