Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

backport #1311 # 1348 #1373

Merged
merged 2 commits into from
Jul 17, 2023
Merged

backport #1311 # 1348 #1373

merged 2 commits into from
Jul 17, 2023

Commits on Jul 13, 2023

  1. fix: large files broke prefetch 

    backport master commit 7e39a5d
Author: 泰友 <[email protected]>
Date:   Wed Jul 5 14:14:09 2023 +0800

    fix: large files broke prefetch

    Files larger than 4G leads to prefetch panic, because the max blob io
    range is smaller than 4G. This pr changes blob io max size from u32 to
    u64.

Signed-off-by: 泰友 <[email protected]>
Signed-off-by: Peng Tao <[email protected]>
    @bergwolf
    bergwolf committed Jul 13, 2023
    Configuration menu
    Copy the full SHA
    a70f36f View commit details
    Browse the repository at this point in the history

  2. fix: amplify io is too large to hold in fuse buffer (dragonflyoss#1311) 

    backport the logic of master commit c8a39c8

* fix: amplify io is too large to hold in fuse buffer

Fuse request buffer is fixed by `FUSE_KERN_BUF_SIZE * pagesize() + FUSE_HEADER_ SIZE`. When amplify io is larger than it, FuseDevWriter suffers from smaller buffer. As a result, invalid data error is returned.

Reproduction:
    run nydusd with 3MB amplify_io
    error from random io:
        reply error header OutHeader { len: 16, error: -5, unique: 108 }, error Custom { kind: InvalidData, error: "data out of range, available 1052656 requested 1250066" }

Details:
    size of fuse buffer = 1052656 + 16 (size of inner header) = 256(page number) * 4096(page size) + 4096(fuse header)
    let amplify_io = min(user_specified, fuseWriter.available_bytes())

Resolution:
    This pr is not best implements, but independent of modification to [fuse-backend-rs]("https://github.com/cloud-hypervisor/fuse-backend-rs").
    In future, evalucation of amplify_io will be replaced with [ZeroCopyWriter.available_bytes()]("cloud-hypervisor/fuse-backend-rs#135").

Signed-off-by: 泰友 <[email protected]>

* feat: e2e for amplify io larger than fuse buffer

Signed-off-by: 泰友 <[email protected]>

---------

Signed-off-by: 泰友 <[email protected]>
Co-authored-by: 泰友 <[email protected]>
Signed-off-by: Peng Tao <[email protected]>
    @ccx1024cc @bergwolf
    2 people authored and bergwolf committed Jul 13, 2023
    Configuration menu
    Copy the full SHA
    dfc08f1 View commit details
    Browse the repository at this point in the history