Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pre-allocate in fs::read and fs::read_string #47324

Merged
merged 1 commit into from
Jan 12, 2018
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions src/libstd/fs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,14 @@ pub struct DirBuilder {
recursive: bool,
}

/// How large a buffer to pre-allocate before reading the entire file at `path`.
fn initial_buffer_size<P: AsRef<Path>>(path: P) -> usize {
// Allocate one extra byte so the buffer doesn't need to grow before the
// final `read` call at the end of the file. Don't worry about `usize`
// overflow because reading will fail regardless in that case.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does "fail regardless" here mean Result::Err or panic?
m.len() as usize + 1 can panic.

Copy link
Contributor Author

@mbrubeck mbrubeck Jan 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will panic in RawVec code when requested capacity exceeds isize::MAX.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(or more likely, abort due to out-of-memory sometime before that)

metadata(path).map(|m| m.len() as usize + 1).unwrap_or(0)
}

/// Read the entire contents of a file into a bytes vector.
///
/// This is a convenience function for using [`File::open`] and [`read_to_end`]
Expand Down Expand Up @@ -246,7 +254,7 @@ pub struct DirBuilder {
/// ```
#[unstable(feature = "fs_read_write", issue = "46588")]
pub fn read<P: AsRef<Path>>(path: P) -> io::Result<Vec<u8>> {
let mut bytes = Vec::new();
let mut bytes = Vec::with_capacity(initial_buffer_size(&path));
File::open(path)?.read_to_end(&mut bytes)?;
Ok(bytes)
}
Expand Down Expand Up @@ -287,7 +295,7 @@ pub fn read<P: AsRef<Path>>(path: P) -> io::Result<Vec<u8>> {
/// ```
#[unstable(feature = "fs_read_write", issue = "46588")]
pub fn read_string<P: AsRef<Path>>(path: P) -> io::Result<String> {
let mut string = String::new();
let mut string = String::with_capacity(initial_buffer_size(&path));
File::open(path)?.read_to_string(&mut string)?;
Ok(string)
}
Expand Down