Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Concurrency fixes #3642

Merged
merged 3 commits into from
Oct 1, 2024
Merged

Concurrency fixes #3642

merged 3 commits into from
Oct 1, 2024

Conversation

lihaoyi
Copy link
Member

@lihaoyi lihaoyi commented Oct 1, 2024

  • Swap from ForkJoinPool to ThreadPoolExecutor

    • ForkJoinPool does this weird thing where fork-join-tasks can be re-entrant at join points, resulting in weird scenarios where a mill-task that hits a yield point (e.g. inside Zinc / parallel-collections / FJP) can start running a second mill-task even before the first has finished, violating all sorts of invariants (# of running tasks exceeds --jobs, FixSizeCache semaphores get taken twice by the same thread, all sorts of craziness)
    • We replace ForkJoinPool#ManagedBlocker with our own manual logic increasing and decreasing the ThreadPoolExecutors maximumPoolSize and corePoolSize in our blocking{...} wrapper
  • We need to Thread#interrupt() the promptUpdaterThread thread when we close the PromptLogger, so we don't need to wait the promptUpdateInterval (0.1ms for interactive, 60s for non-interactive) before exiting

This should fix some of the flakiness we've been seeing in master, that seems to have started from 05bef7e (just eyeballing the CI history), and blocking our re-bootstrapping in #3637

@lihaoyi lihaoyi marked this pull request as ready for review October 1, 2024 14:08
@lihaoyi lihaoyi merged commit 26cbc8a into com-lihaoyi:main Oct 1, 2024
24 checks passed
@lefou lefou added this to the 0.12.0-RC3 milestone Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants