Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import zdfs package to support a tricky OCIv1 tgz format #273

Merged
merged 1 commit into from
Apr 7, 2024

Conversation

BigVan
Copy link
Member

@BigVan BigVan commented Apr 5, 2024

What this PR does / why we need it:
Add zdfs package import.
Packing the reference files of the overlaybd blob into the layer of the OCIv1 image; This enables the old version of containerd or other runtime (such as docker) to run overlaybd image

reference files like:

ls /var/lib/containerd/io.containerd.snapshotter.v1.overlaybd/snapshots/283/fs -a
.aaaaaaaaaaaaaaaa.lsmt  .checksum_file  .data_size  .oss_url  .type

# .oss_url is the overlaybd blob uploaded address in remote storage
# .data_size is the blobSize record in the annotation of overlaybd image manifest.
# other files can be empty

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Please check the following list:

  • Does the affected code have corresponding tests, e.g. unit test, E2E test?
  • Does this change require a documentation update?
  • Does this introduce breaking changes that would require an announcement or bumping the major version?
  • Do all new files have an appropriate license header?

@BigVan BigVan requested a review from liulanzheng April 5, 2024 16:20
@BigVan BigVan force-pushed the zdfs branch 2 times, most recently from af91eda to 0d554ef Compare April 6, 2024 13:46
@liulanzheng liulanzheng added the ok-to-test Pull request is ok to run ci test label Apr 7, 2024
@BigVan BigVan changed the title Import zdfs package to support a tricky OCIv2 tgz format Import zdfs package to support a tricky OCIv1 tgz format Apr 7, 2024
pkg/snapshot/overlay.go Outdated Show resolved Hide resolved
@liulanzheng
Copy link
Member

LGTM @WaberZhuang cc

if ok {
log.G(ctx).Infof("found imageRef: %s", img)
if err := os.WriteFile(filepath.Join(td, "image_ref"), []byte(img), 0644); err != nil {
log.G(ctx).Errorf("LSMD ERROR write imageRef '%s'. path: %s, err: %s", img, filepath.Join(td, "image_ref"), err.Error())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: LSMD ERROR in the log is confusing, as is err.Error()

if isOverlaybd, err := zdfs.PrepareOverlayBDSpec(ctx, key, id, o.snPath(id), oinfo, o.snPath); isOverlaybd {
log.G(ctx).Infof("sn: %s is an overlaybd layer", id)
} else if err != nil {
log.G(ctx).Errorf("prepare overlaybd spec failed(sn: %s): %s", id, err.Error())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use ("%v", err) to print error?

if ok {
log.G(ctx).Infof("found imageRef: %s", img)
if err := os.WriteFile(filepath.Join(path, "image_ref"), []byte(img), 0644); err != nil {
log.G(ctx).Errorf("write imageRef '%s'. path: %s, err: %s", img, filepath.Join(path, "image_ref"), err.Error())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use ("%v", err) to print error?

@BigVan BigVan merged commit ac7c2db into containerd:main Apr 7, 2024
9 checks passed
github.com/containerd/continuity v0.4.3
github.com/containerd/go-cni v1.1.9
github.com/containerd/log v0.1.0
github.com/data-accelerator/zdfs v0.1.3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The package lacks license

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry...
I have added Apache license to the project.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ok-to-test Pull request is ok to run ci test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants