Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error "Too many open files" when creating universal image #76

Closed
Sporif opened this issue Apr 6, 2021 · 3 comments
Closed

Error "Too many open files" when creating universal image #76

Sporif opened this issue Apr 6, 2021 · 3 comments

Comments

@Sporif
Copy link

Sporif commented Apr 6, 2021

I'm getting this error fairly often when creating a universal image with strip enabled, lines like:
/usr/lib/modules/rmd256.ko: pipe2: too many open files or
/usr/lib/modules/842_compress.ko: open /dev/null: too many open files

@anatol
Copy link
Owner

anatol commented Apr 6, 2021

It happens because you build a universal image (i.e. it contains a lot of modules) and strip option enabled. When many strip commands are executed in parallel then it requires many files (binaries, shared libraries, etc) to be open.

Current open file limit is ridiculously low, it is something like 1024 files (ulimit -n). Opening many files leads to this limit exhaustion.

To avoid this error message please increase the file limit https://wiki.archlinux.org/index.php/Limits.conf (option nofile) to something more reasonable (8192 at least).

Or you can disable strip option.

anatol added a commit that referenced this issue Oct 24, 2021
Opening a lot of modules for processing requires high open-file-limit
value. The default value is 1024 which is too low.

Try to increase process limit to infinity first. If it does not work
then try to increase soft limit to hard limit value.

Closes #76
@anatol anatol closed this as completed in a299aff Nov 2, 2021
anatol added a commit that referenced this issue Nov 13, 2021
Opening a lot of kernel modules for processing requires high open-file-limit
value. The default value is 1024 which is too low.

Try to increase process limit to infinity first. If it does not work
then try to increase soft limit to hard limit value.

Closes #76
anatol added a commit that referenced this issue Nov 14, 2021
Opening a lot of kernel modules for processing requires high open-file-limit
value. The default value is 1024 which is too low.

Try to increase process limit to infinity first. If it does not work
then try to increase soft limit to hard limit value.

Closes #76
@mvdan
Copy link
Contributor

mvdan commented May 6, 2022

FYI, Go 1.19 will do this for you automatically: golang/go#46279

Perhaps worth adding a TODO to remove your workaround in a year or two, once you can drop support for Go 1.18 and older. Then there's less code to maintain :)

@anatol
Copy link
Owner

anatol commented May 6, 2022

Good to know about the upcoming 1.19 change. Thanks for pointing it out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants