Skip to content

Commit

Permalink
fix: amplify io is too large to hold in fuse buffer (#1311)
Browse files Browse the repository at this point in the history
backport the logic of master commit c8a39c8

* fix: amplify io is too large to hold in fuse buffer

Fuse request buffer is fixed by `FUSE_KERN_BUF_SIZE * pagesize() + FUSE_HEADER_ SIZE`. When amplify io is larger than it, FuseDevWriter suffers from smaller buffer. As a result, invalid data error is returned.

Reproduction:
    run nydusd with 3MB amplify_io
    error from random io:
        reply error header OutHeader { len: 16, error: -5, unique: 108 }, error Custom { kind: InvalidData, error: "data out of range, available 1052656 requested 1250066" }

Details:
    size of fuse buffer = 1052656 + 16 (size of inner header) = 256(page number) * 4096(page size) + 4096(fuse header)
    let amplify_io = min(user_specified, fuseWriter.available_bytes())

Resolution:
    This pr is not best implements, but independent of modification to [fuse-backend-rs]("https://github.com/cloud-hypervisor/fuse-backend-rs").
    In future, evalucation of amplify_io will be replaced with [ZeroCopyWriter.available_bytes()]("cloud-hypervisor/fuse-backend-rs#135").

Signed-off-by: 泰友 <[email protected]>

* feat: e2e for amplify io larger than fuse buffer

Signed-off-by: 泰友 <[email protected]>

---------

Signed-off-by: 泰友 <[email protected]>
Co-authored-by: 泰友 <[email protected]>
Signed-off-by: Peng Tao <[email protected]>
  • Loading branch information
2 people authored and imeoer committed Jul 17, 2023
1 parent 7041460 commit 9727b85
Showing 1 changed file with 13 additions and 8 deletions.
21 changes: 13 additions & 8 deletions rafs/src/fs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,10 @@ pub type Handle = u64;
pub const RAFS_DEFAULT_ATTR_TIMEOUT: u64 = 1 << 32;
/// Rafs default entry timeout value.
pub const RAFS_DEFAULT_ENTRY_TIMEOUT: u64 = RAFS_DEFAULT_ATTR_TIMEOUT;
/// FUSE_KERN_BUF_SIZE * pagesize() + FUSE_HEADER_ SIZE
/// Use a const as stable v2.1 does not complie with latset fuse-backend-rs that
/// exports a writer's available buffer size.
pub const RAFS_MAX_AMPLIFIED_IO: usize = 1052656;

fn default_threads_count() -> usize {
8
Expand Down Expand Up @@ -765,18 +769,19 @@ impl FileSystem for Rafs {
debug_assert!(!descs.is_empty() && !descs[0].bi_vec.is_empty());

// Try to amplify user io for Rafs v5, to improve performance.
if self.sb.meta.is_v5() && size < self.amplify_io {
let amplify_io = cmp::min(self.amplify_io as usize, RAFS_MAX_AMPLIFIED_IO) as u32;
if self.sb.meta.is_v5() && size < amplify_io {
let all_chunks_ready = self.device.all_chunks_ready(&descs);
if !all_chunks_ready {
let chunk_size = self.metadata().chunk_size as u64;
let next_chunk_base = (offset + (size as u64) + chunk_size) & !chunk_size;
let window_base = std::cmp::min(next_chunk_base, inode_size);
let actual_size = window_base - (offset & !chunk_size);
if actual_size < self.amplify_io as u64 {
let window_size = self.amplify_io as u64 - actual_size;
let chunk_mask = self.metadata().chunk_size as u64 - 1;
let next_chunk_base = (offset + (size as u64) + chunk_mask) & !chunk_mask;
let window_base = cmp::min(next_chunk_base, inode_size);
let actual_size = window_base - (offset & !chunk_mask);
if actual_size < amplify_io as u64 {
let window_size = amplify_io as u64 - actual_size;
self.sb.amplify_io(
&self.device,
self.amplify_io,
amplify_io,
&mut descs,
&inode,
window_base,
Expand Down

0 comments on commit 9727b85

Please sign in to comment.