Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Builder: support tar build #351

Merged
merged 12 commits into from
Apr 7, 2022
Merged

Builder: support tar build #351

merged 12 commits into from
Apr 7, 2022

Commits on Apr 7, 2022

  1. Builder: support none whiteout-spec 

    Add `--whiteout-spec none` option for builer command, which will
build all `.wh.*` and `.wh..wh..opq` files into bootstrap.

This is used for tar build, packing blob and bootstrap into a tar
stream, so that correspond to the OCI tar layer one by one.

Signed-off-by: Yan Song <[email protected]>
    @imeoer
    imeoer committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    cbaa236 View commit details
    Browse the repository at this point in the history

  2. Builder: simplify call for bootstrap dump 

    Match blob table by fs version in `boostrap.dump()`.

Signed-off-by: Yan Song <[email protected]>
    @imeoer
    imeoer committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    5759787 View commit details
    Browse the repository at this point in the history

  3. Builder: support merge sub-command 

    The `merge` sub-command merges multiple nydus boostraps (from every layer
of image) to a final boostrap.

Signed-off-by: Yan Song <[email protected]>
    @imeoer
    imeoer committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    3cbf9da View commit details
    Browse the repository at this point in the history

  4. Builder: support compressed blob offset 

    Nydus tar build converts a OCI formatted tar stream to a nydus formatted tar stream.

The nydus blob tar stream contains blob and bootstrap files with the following
file tree structure:

/image
├── image.blob
├── image.boot

So for the chunk of files in the nydus boostreap, a blob compressed offset
of 1024 (size_of(tar_header) * 2) is required.

Add `--blob-offset` option to builder to support this case.

Signed-off-by: Yan Song <[email protected]>
    @imeoer
    imeoer committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    0c5e6f6 View commit details
    Browse the repository at this point in the history

  5. Builder: support chunk dict for merge 

    Add `--chunk-dict` option support for merge sub-command,
We need get blob list from chunk-dict bootstrap to fill
blob table in final bootstrap.

Signed-off-by: Yan Song <[email protected]>
    @imeoer
    imeoer committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    2b9c61f View commit details
    Browse the repository at this point in the history

  6. Builder: support --prefetch-policy option for merge 

    Add `--prefetch-policy` option support for merge sub-command, this
option has the same function as in the create sub-command, it is
used to enable prefetch to optimize inode/chunk layout.

Signed-off-by: Yan Song <[email protected]>
    @imeoer
    imeoer committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    d5a2527 View commit details
    Browse the repository at this point in the history

  7. Builder: tidy code for merge implementation 

    Signed-off-by: Yan Song <[email protected]>
    @imeoer
    imeoer committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    7811748 View commit details
    Browse the repository at this point in the history

  8. Builder: fix conflict ino cross layers in merge 

    Set node's layer index to distinguish same inode number (from bootstrap)
between different layers.

Signed-off-by: Yan Song <[email protected]>
    @imeoer
    imeoer committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    9171d39 View commit details
    Browse the repository at this point in the history

  9. Builder: small tidy for merge command 

    1. reuse options with create sub-command;
2. add context error message for user friendly;

Signed-off-by: Yan Song <[email protected]>
    @imeoer
    imeoer committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    b492144 View commit details
    Browse the repository at this point in the history

  10. Nydusify: check blob consistency without order 

    For the tar build scene, the regular layer represents blob+bootstrap,
for an empty data layer (for example `chmod +x`), the blob table in
bootstrap doesn't include the blob entry of this layer, so we need
remove the check of comparing the number of blobs and the blob order
between bootstrap and layers on the manifest, just ensure nydus blobs
in the blob table of bootstrap should all appear in the layers of manifest.

Signed-off-by: Yan Song <[email protected]>
    @imeoer
    imeoer committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    4d374c7 View commit details
    Browse the repository at this point in the history

  11. Builder: keep bootstrap following flags on merge 

    Keep final bootstrap following superblock flags of source bootstraps.

Signed-off-by: Yan Song <[email protected]>
    @imeoer
    imeoer committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    f23feca View commit details
    Browse the repository at this point in the history

  12. Builder: fix test for diff build 

    The behavior change of RafsSuper::walk_inodes api break diff
build, adjust it to ignore non-regular file handle.

Signed-off-by: Yan Song <[email protected]>
    @imeoer
    imeoer committed Apr 7, 2022
    Configuration menu
    Copy the full SHA
    fc72709 View commit details
    Browse the repository at this point in the history