Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cancelable parallel #117

Merged
merged 13 commits into from
Sep 22, 2023
Merged

Cancelable parallel #117

merged 13 commits into from
Sep 22, 2023

Conversation

jelmervdl
Copy link
Collaborator

Fix for the non-quiting beviour of #116.

This adds a ThreadPool and a CancelableQueue.

The CancelableQueue combines a blocking queue with a stop signal. The problem is that the worker threads can only be interrupted when they're blocking on getting work from a queue, or putting results in a queue. This queue then has interruptible get() and put() methods.

The ThreadPool class is a better implementation of RaisingThread, where TheadPool.join() will behave more like a select() on sockets and raise an error as soon as any of the threads encountered an exception.

The general idea is that we start all threads as if everything will be allright, putting them in the thread pool. Then we wait for any of them to crash (or all of them to finish without exceptions). If any of them crashes, we call cancel() on the queues so that all the other threads also abort their loops. Then, because we're exiting the context of the ThreadPool, we'll wait for them to finish up nicely so that stuff like try ... finally still works as expected.

It is a contious decision that I don't care about exceptions in any of the other threads once I caught one in one of the threads in the threadpool. You can call join() multiple times until it exists without throwing, but in OpusCleaner just one error is already one too many or something.

@jelmervdl
Copy link
Collaborator Author

I'm waiting for CCmatrix en-ru to download so I can replicate and verify that this exact bug has now been fixed.

@jelmervdl jelmervdl merged commit e8408fd into main Sep 22, 2023
5 checks passed
@jelmervdl jelmervdl deleted the cancelable-parallel branch September 7, 2024 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant