Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix a bug that too many file are being watched when using hot-reloading #368

Closed
wants to merge 1 commit into from

Conversation

haixuanTao
Copy link
Collaborator

Currently there is an issue that the hot-reloading file watcher does not automatically notify the OS to stop watching files when the dataflow stop.

The error was as follows: https://stackoverflow.com/questions/53930305/nodemon-error-system-limit-for-number-of-file-watchers-reached

Currently there is an issue that the hot-reloading file watcher does not
automatically notify the OS to stop watching files when the dataflow stop.
@phil-opp
Copy link
Collaborator

I'm not sure whether I understand this issue. The watcher doesn't unwatch the files automatically when it's dropped?

Also, the watching seems to happen in our CLI process, which should exit when a dataflow finishes. Doesn't the OS clean up the file watchers when the corresponding process exits?

Base automatically changed from dora-record to main October 31, 2023 10:22
@haixuanTao
Copy link
Collaborator Author

Ok, I can't reproduce the error at the moment. Can we leave this open, and if I ever encounter the error again, i'll try to see if this fixes it. :)

Thanks for double checking

@haixuanTao haixuanTao marked this pull request as draft October 31, 2023 10:54
@haixuanTao
Copy link
Collaborator Author

Ok, so i double checked the number of files being watched and indeed. Inotify closes it normally without having to unsubscribe it.

If other people face the same issue:

OS file watch limit reached. about ["/home/dora/workspace/simulate/team_code/dora-drives/operators/obstacle_location_op.py"]

What fixed it for me was: https://stackoverflow.com/questions/53930305/nodemon-error-system-limit-for-number-of-file-watchers-reached

@haixuanTao haixuanTao closed this Nov 7, 2023
@haixuanTao haixuanTao deleted the fix-watch-file-limit-error branch November 7, 2023 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants