Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add memory requirements for building with Sphinx to the README file #3239

Closed
aaronfranke opened this issue Mar 4, 2020 · 2 comments
Closed

Comments

@aaronfranke
Copy link
Member

#3175 added a note about out of memory errors, but I did not actually expect to encounter any on my Linux machine which has 24 GB RAM.

When I tried building, I very quickly ran out of memory, my system started swapping aggressively, it took me ~10 minutes to drop to a terminal to see that Sphinx spawned 4 processes which in total used about 30 GB RAM. To recover my system I disabled swap so that my OS would kill Sphinx.

There absolutely needs to be a mention of just how much memory is required to build the docs. If the README specifically said you need 64 GB RAM, I probably would have paid attention to the warning since that is more than I have in my system. But as-is, the warning is simply not enough, I do not expect to hit out of memory errors with any program on a machine with 24 GB RAM, and it's not acceptable for someone following build instructions to have their system unresponsive for 10 minutes.

I would submit a PR, but I don't know what the actual requirements are. How much memory do you need to build the docs? It's clearly more than 32 GB. So, 64 GB? Or 128 GB? Or more?

@YuriSizov
Copy link
Contributor

YuriSizov commented Mar 4, 2020

I'd built yesterday as well, and there were no memory issues, but the complete build took more than an hour on Windows 10 laptop with Core i7-6700HQ and 8 GB RAM. It looks like in your case multithreading was too aggressive and in my case did not happen at all.

Maybe -j auto option is not doing its best and should be always adjusted by user, like we do with scons.

Edit: Though, if each process requires somewhere between 7 and 8 GB RAM, I wouldn't want my builds to be multithreaded at all! This is simply inefficiency on Sphinx's part.

@Calinou
Copy link
Member

Calinou commented Mar 4, 2020

I can build the documentation successfully with 32 GB of RAM, but I guess you need to remove the classes/ folder if you want to build with multiple threads with less RAM.

I'll have to revert the -j auto thing 🙁

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants