Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Notify fails to report file creation events if they happen too quickly #267

Closed
jimblandy opened this issue Nov 4, 2020 · 1 comment · Fixed by #268 or #269
Closed

Notify fails to report file creation events if they happen too quickly #267

jimblandy opened this issue Nov 4, 2020 · 1 comment · Fixed by #268 or #269

Comments

@jimblandy
Copy link
Contributor

When watching a directory, if many files are created in rapid succession, notify reports the first few creations but then stops reporting the rest. If, after a pause, another file is created in the directory, the missing reports do come in.

Linux, Fedora Core 32, x86_64
rustc 1.47.0 (18bf6b4f0 2020-10-07)
notify 5.0.0-pre.4

Test case:

use std::fs;
use notify::{Event, EventKind, RecommendedWatcher, RecursiveMode, Result, Watcher};

fn main() -> Result<()> {
    let mut watcher: RecommendedWatcher = Watcher::new_immediate(|res| {
        match res {
            Ok(Event { kind: EventKind::Create(_), paths, .. }) => {
                eprintln!("created {:?}", paths);
            }
            _ => (),
        }
    })?;

    fs::create_dir("notify-hang-test")?;
    watcher.watch("notify-hang-test", RecursiveMode::NonRecursive)?;

    for i in 0..30 {
        fs::File::create(format!("notify-hang-test/glurf{:02}", i))?;
    }

    // Only the first dozen or two files get reported.
    std::thread::sleep(std::time::Duration::from_secs(5));

    // Poking the directory again does seem to wake it up.
    eprintln!("poking");
    fs::File::create("notify-hang-test/glurf31")?;

    // Wait for reports to come in
    std::thread::sleep(std::time::Duration::from_secs(5));

    Ok(())
}

Sample output:

$ rm -r notify-hang-test
$ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.03s
     Running `target/debug/play-notify-hang`
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf00"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf01"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf02"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf03"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf04"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf05"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf06"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf07"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf08"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf09"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf10"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf11"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf12"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf13"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf14"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf15"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf16"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf17"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf18"]
poking
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf19"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf20"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf21"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf22"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf23"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf24"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf25"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf26"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf27"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf28"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf29"]
created ["/home/jimb/rust/play-notify-hang/notify-hang-test/glurf31"]
@jimblandy
Copy link
Contributor Author

This seems to be caused by hannobraun/inotify-rs#156: contrary to the documentation, Inotify::read_events returns an iterator over only one buffer's worth of events, not all available events. Because notify::inotify::EventLoop::new uses mio::PollOpt::edge() when it registers the inotify file descriptor, mio will only generate events when the inotify file descriptor state changes, not whenever data is available to read. From the mio docs:

With edge-triggered events, operations must be performed on the Evented type until WouldBlock is returned. In other words, after receiving an event indicating readiness for a certain operation, one should assume that Poll::poll may never return another event for the same token and readiness until the operation returns WouldBlock.

If Inotify::read_events worked as documented, then notify's use of edge-triggered events would make sense. However, changing read_events' behavior would be awkward; the interface would have to change in various ways (detailed in the issue linked). And Mio's level-triggered events are meant for use in exactly these situations; indeed, using PollOpt::level() for the inotify file descriptor seems to fix the bug.

jimblandy added a commit to jimblandy/notify that referenced this issue Nov 4, 2020
@jimblandy
EventLoop::new: Use level-triggered events for inotify fd.
0e49370 
The `inotify` crate's `Inotify::read_events` method does not read all available
events from inotify (see hannobraun/inotify-rs#156), only one buffer's worth. Using
level-triggered events tells Mio to report events on the inotify fd until all
events have been read.

Fixes notify-rs#267.
@jimblandy jimblandy mentioned this issue Nov 4, 2020
Merged
jimblandy added a commit to jimblandy/notify that referenced this issue Nov 5, 2020
@jimblandy
EventLoop::new: Use level-triggered events for inotify fd.
534c049 
The `inotify` crate's `Inotify::read_events` method does not read all available
events from inotify (see hannobraun/inotify-rs#156), only one buffer's worth. Using
level-triggered events tells Mio to report events on the inotify fd until all
events have been read.

Fixes notify-rs#267.
jimblandy added a commit to jimblandy/notify that referenced this issue Nov 5, 2020
@jimblandy
EventLoop::new: Use level-triggered events for inotify fd.
c5a1046 
The `inotify` crate's `Inotify::read_events` method does not read all available
events from inotify (see hannobraun/inotify-rs#156), only one buffer's worth. Using
level-triggered events tells Mio to report events on the inotify fd until all
events have been read.

Fixes notify-rs#267.
jimblandy added a commit to jimblandy/notify that referenced this issue Nov 5, 2020
@jimblandy
EventLoop::new: Use level-triggered events for inotify fd.
66a0a14 
The `inotify` crate's `Inotify::read_events` method does not read all available
events from inotify (see hannobraun/inotify-rs#156), only one buffer's worth. Using
level-triggered events tells Mio to report events on the inotify fd until all
events have been read.

Fixes notify-rs#267.
jimblandy added a commit to jimblandy/notify that referenced this issue Nov 5, 2020
@jimblandy
EventLoop::new: Use level-triggered events for inotify fd.
3389a1e 
The `inotify` crate's `Inotify::read_events` method does not read all available
events from inotify (see hannobraun/inotify-rs#156), only one buffer's worth. Using
level-triggered events tells Mio to report events on the inotify fd until all
events have been read.

Fixes notify-rs#267.
@jimblandy jimblandy mentioned this issue Nov 5, 2020
Merged
jimblandy added a commit to jimblandy/notify that referenced this issue Nov 5, 2020
@jimblandy
EventLoop::new: Use level-triggered events for inotify fd.
8d2706a 
The `inotify` crate's `Inotify::read_events` method does not read all available
events from inotify (see hannobraun/inotify-rs#156), only one buffer's worth. Using
level-triggered events tells Mio to report events on the inotify fd until all
events have been read.

Fixes notify-rs#267.
jimblandy added a commit to jimblandy/notify that referenced this issue Nov 5, 2020
@jimblandy
EventLoop::new: Use level-triggered events for inotify fd.
be04c3b 
The `inotify` crate's `Inotify::read_events` method does not read all available
events from inotify (see hannobraun/inotify-rs#156), only one buffer's worth. Using
level-triggered events tells Mio to report events on the inotify fd until all
events have been read.

Fixes notify-rs#267.
JohnTitor pushed a commit that referenced this issue Nov 6, 2020
@jimblandy
EventLoop::new: Use level-triggered events for inotify fd. (#268)
c36c241 
The `inotify` crate's `Inotify::read_events` method does not read all available
events from inotify (see hannobraun/inotify-rs#156), only one buffer's worth. Using
level-triggered events tells Mio to report events on the inotify fd until all
events have been read.

Fixes #267.
JohnTitor pushed a commit that referenced this issue Nov 6, 2020
@jimblandy
EventLoop::new: Use level-triggered events for inotify fd. (backport …
e2c9992 
…to v4) (#269)

The `inotify` crate's `Inotify::read_events` method does not read all available
events from inotify (see hannobraun/inotify-rs#156), only one buffer's worth. Using
level-triggered events tells Mio to report events on the inotify fd until all
events have been read.

Fixes #267.
roblabla added a commit to roblabla/notify that referenced this issue Jan 22, 2021
@roblabla
Fix notify-rs#267 by repeatedly calling INotify read_buffer
5c48373 
When INotify::read_buffer finally returns an empty iterator, we break
out of the loop. This should allow polling with edge triggering.
@roblabla roblabla mentioned this issue Jan 22, 2021
Merged
roblabla added a commit to roblabla/notify that referenced this issue Jan 22, 2021
@roblabla
Fix notify-rs#267 by repeatedly calling INotify read_buffer
9c0bd01 
When INotify::read_buffer finally returns an empty iterator, we break
out of the loop. This should allow polling with edge triggering.
@0xpr03 0xpr03 mentioned this issue Jan 26, 2021
Closed
@kjeremy kjeremy mentioned this issue Jan 28, 2021
Merged
@jiangliu jiangliu mentioned this issue Mar 22, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@jimblandy