Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add zero-compromise directory iteration using getdents64 #1856

Closed
wants to merge 1 commit into from

Conversation

SUPERCILEX
Copy link
Contributor

@SUPERCILEX SUPERCILEX commented Nov 7, 2022

Still need to add docs and tidy some stuff up. Plus #1854 will fix the extra formatting changes.

Benchmarks:

fn main() {
    let fd = open(
        env::args().nth(1).unwrap().as_str(),
        OFlag::O_RDONLY,
        Mode::empty(),
    )
    .unwrap();
    let fd = unsafe { OwnedFd::from_raw_fd(fd) };
    let mut buf = Vec::with_capacity(8192);

    // Do the same thing for Dir and read_dir
    let mut something: usize = 0;
    for file in RawDir::with_vec(fd.as_fd(), &mut buf) {
        something = something.wrapping_add(file.unwrap().name.to_bytes().len());
    }
    println!("{}", something);
}
$ ftzz g -n 1M -d 0 /tmp/ftzz-test
$ hyperfine "./dents /tmp/ftzz-test" "./dir /tmp/ftzz-test" "./stdlib /tmp/ftzz-test"                                                                               11/06/2022 04:43:44 PM
Benchmark 1: ./dents /tmp/ftzz-test
  Time (mean ± σ):     166.5 ms ±   1.4 ms    [User: 5.2 ms, System: 161.1 ms]
  Range (min … max):   164.2 ms … 169.4 ms    17 runs
 
Benchmark 2: ./dir /tmp/ftzz-test
  Time (mean ± σ):     202.7 ms ±   2.6 ms    [User: 42.2 ms, System: 159.9 ms]
  Range (min … max):   200.0 ms … 206.8 ms    14 runs
 
Benchmark 3: ./stdlib /tmp/ftzz-test
  Time (mean ± σ):     210.0 ms ±   3.9 ms    [User: 51.7 ms, System: 158.0 ms]
  Range (min … max):   204.0 ms … 220.0 ms    14 runs
 
Summary
  './dents /tmp/ftzz-test' ran
    1.22 ± 0.02 times faster than './dir /tmp/ftzz-test'
    1.26 ± 0.03 times faster than './stdlib /tmp/ftzz-test'

@SUPERCILEX SUPERCILEX marked this pull request as draft November 7, 2022 00:50
@SUPERCILEX SUPERCILEX marked this pull request as ready for review November 10, 2022 04:44
@SUPERCILEX
Copy link
Contributor Author

@asomers This should be ready to rock and roll.

@SUPERCILEX SUPERCILEX force-pushed the dents branch 2 times, most recently from 4635801 to 36f05fe Compare November 12, 2022 04:05
target_os = "haiku"
))]
None
pub fn file_type(&self) -> Type {
Copy link
Contributor Author

@SUPERCILEX SUPERCILEX Nov 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the policy on breaking changes like this? Option<Type> -> Type

I could undo this by matching on Unknown and returning None.

@SUPERCILEX SUPERCILEX force-pushed the dents branch 5 times, most recently from ff3e4e0 to 5d7ecc4 Compare November 15, 2022 01:18
@SUPERCILEX
Copy link
Contributor Author

Closing this as it's wrong anyway. See my rustix implementation for the latest and greatest: https://github.com/bytecodealliance/rustix/blob/main/src/fs/raw_dir.rs

@SUPERCILEX SUPERCILEX closed this Aug 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant