Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Too much useless io in chunk_write due to create_dir_all() #95

Closed
lukaslueg opened this issue Apr 20, 2017 · 2 comments
Closed

Too much useless io in chunk_write due to create_dir_all() #95

lukaslueg opened this issue Apr 20, 2017 · 2 comments

Comments

@lukaslueg
Copy link

ChunkWriterThread::run currently calls create_dir_all() twice, which is useless at best but causes quite a lot of IO.

We can reduce IO further by asking for forgivness instead of permission and moving the create_dir_all()-call into a new error-path for fs::File::create(&tmp_path).unwrap();. As the path will exist for almost all but one calls to fs::File::create() per path, we should just try to create the file directly and only create the path structure in case we fail. This saves us from a lot of stat64-calls.

@dpc
Copy link
Owner

dpc commented Apr 20, 2017

The reason it's called twice is: rust-lang/rust#33707 which I recently fixed with: rust-lang/rust#39799 . It is yet hit stable, That PR also improves the the performance of create_dir_all and makes it "try first and check later".

You're right that it would be better to just try File::create first and mkdir only if it failed. Please submit a PR!

@dpc
Copy link
Owner

dpc commented Jul 29, 2017

Done.

let mut chunk_file = match fs::File::create(&tmp_path) {

@dpc dpc closed this as completed Jul 29, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants