Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Best multithreading setup for large projects #115

Open
kLy opened this issue Jun 24, 2022 · 1 comment
Open

Best multithreading setup for large projects #115

kLy opened this issue Jun 24, 2022 · 1 comment

Comments

@kLy
Copy link

kLy commented Jun 24, 2022

Hi. I have a pretty large project with thousands of audio files, but most of them are quite short. When I look at the documentation, it seems like rhubarb only properly utilises multi-threading with longer audio files. So I'm wondering what would be a good way to do batch processing of large numbers of audio files to most effectively use the multi-threading? I'm thinking:

  1. Use the existing setup, and batch the files through one at a time. According to the documentation, this seems like it will make least use of the multithreading built into the tool.
  2. Force thread count to 1 on rhubarb, but launch a pool of worker threads that call rhubarb in parallel, one for each processor core I have.
  3. Have some kind of functionality for rhubarb to input a list of files rather than just process one at a time. This would make rhubarb handle the multithreading in the way that it deems the most efficient, though of course that would mean adding support for this feature.

Curious to hear your thoughts. My current plan is to go with 2. Let me know if that's a good / bad idea. Thanks! :)

@DanielSWolf
Copy link
Owner

My thoughts on that topic are exactly the same. For Rhubarb 2, I'm thinking of implementing a built-in batch mode that will take a glob pattern of files and process them all using smart multi-threading.

Until then, your best bet when it comes to processing speed is to go with your option 2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants