Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Respect ulimit #116

Closed
peterbraden opened this issue Sep 8, 2014 · 5 comments
Closed

Respect ulimit #116

peterbraden opened this issue Sep 8, 2014 · 5 comments

Comments

@peterbraden
Copy link
Contributor

I have run into the infamous 'EMFILE, too many open files' error several times while using glob, and while upping the ulimit does mitigate this temporarily, it doesn't really seem to address the problem that glob is opening way too many files at once.

 at Object.fs.readdirSync (fs.js:659:18)
    at Glob._readdir (.../node_modules/glob/glob.js:662:20)
    at Glob._process (.../node_modules/glob/glob.js:446:15)
    at Glob.iterator (.../node_modules/glob/glob.js:181:10)

Perhaps we can use https://github.com/isaacs/node-graceful-fs?

(more people with this error: https://www.google.ch/search?q=EMFILE%2C+too+many+open+files&oq=EMFILE%2C+too+many+open+files&aqs=chrome..69i57j0l5.262j0j7&sourceid=chrome&es_sm=91&ie=UTF-8#q=EMFILE%2C+too+many+open+files+glob)

@peterbraden
Copy link
Contributor Author

Of course, after looking at the code, graceful-fs is an optional dependency...

Without knowing any of the context behind this, I assume that graceful-fs can't be seamlessly dropped in (hence the optionalDep).

Is there any way to get glob to open less files?

@isaacs
Copy link
Owner

isaacs commented Sep 10, 2014

The latest version of graceful-fs is safe to drop in. We should just make
it a proper dep.

Patch welcome.

On Monday, September 8, 2014, Peter Braden [email protected] wrote:

Of course, after looking at the code, graceful-fs is an optional
dependency...

Without knowing any of the context behind this, I assume that graceful-fs
can't be seamlessly dropped in (hence the optionalDep).

Is there any way to get glob to open less files?


Reply to this email directly or view it on GitHub
#116 (comment).

@peterbraden
Copy link
Contributor Author

I'll make you a patch.

peterbraden added a commit to peterbraden/node-glob that referenced this issue Sep 17, 2014
@isaacs
Copy link
Owner

isaacs commented Oct 7, 2014

landed on master

@cletusw
Copy link

cletusw commented Apr 12, 2016

These issues are back due to #71

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants