Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix crane digest for v1 and platform option #907

Merged
merged 3 commits into from
Jan 8, 2021
Merged

Conversation

dvob
Copy link
Contributor

@dvob dvob commented Jan 8, 2021

This allows to use crane.Digest on images with a v1 schema.

Fixes #906

@google-cla
Copy link

google-cla bot commented Jan 8, 2021

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

This allows to use crane.Digest on images with a v1 schema
@google-cla
Copy link

google-cla bot commented Jan 8, 2021

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.


What to do if you already signed the CLA

Individual signers
Corporate signers

ℹ️ Googlers: Go here for more info.

@dvob
Copy link
Contributor Author

dvob commented Jan 8, 2021

@googlebot I signed it!

pkg/crane/digest.go Outdated Show resolved Hide resolved
@codecov-io
Copy link

codecov-io commented Jan 8, 2021

Codecov Report

Merging #907 (d463476) into master (3b7741e) will increase coverage by 0.02%.
The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #907      +/-   ##
==========================================
+ Coverage   72.54%   72.57%   +0.02%     
==========================================
  Files         108      108              
  Lines        4673     4689      +16     
==========================================
+ Hits         3390     3403      +13     
- Misses        773      776       +3     
  Partials      510      510              
Impacted Files Coverage Δ
pkg/crane/digest.go 27.77% <0.00%> (-3.48%) ⬇️
pkg/crane/optimize.go 11.40% <0.00%> (+11.40%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b7741e...d463476. Read the comment docs.

@dvob
Copy link
Contributor Author

dvob commented Jan 8, 2021

How often do you release new versions (tags)?

@imjasonh
Copy link
Collaborator

imjasonh commented Jan 8, 2021

How often do you release new versions (tags)?

We tend to release as-needed whenever there are significant changes to make available.

cc @jonjohnsonjr to speak on whether/how we want crane to support v1 manifests. This change by itself seems innocuous to me at least 🤷

@jonjohnsonjr
Copy link
Collaborator

jonjohnsonjr commented Jan 8, 2021

How often do you release new versions (tags)?

Yeah, roughly as needed. Usually when I need to re-import this package into google or some large downstream dependent requests a fix be included in a release. We could have more discipline about this (cc @hasheddan).

This change by itself seems innocuous to me at least 🤷

This doesn't cover every base, but it's definitely an improvement, so I'm fine with it.

@dvob consider situations such as this: docker-library/golang#269 (comment)

$ crane manifest nginx:1.19.0-alpine | jq .
2021/01/08 09:18:20 No matching credentials were found for "index.docker.io/library/nginx", falling back on anonymous
{
  "manifests": [
    {
      "digest": "sha256:ee5a9b68e8d4a4b8b48318ff08ad5489bd1ce52b357bf48c511968a302bc347b",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      },
      "size": 1360
    },
    {
      "digest": "sha256:88fd834bc089b5dff1f27ab0315c3b0bbdfddfca3b2c1aeb9798944d179e441b",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "arm",
        "os": "linux",
        "variant": "v6"
      },
      "size": 1360
    },
    {
      "digest": "sha256:bf8ebb8068d7ff10711c1670398eeaa83283a7987cdc9b9b014560dbba71773c",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "arm",
        "os": "linux",
        "variant": "v7"
      },
      "size": 1360
    },
    {
      "digest": "sha256:eff196a3849ad6541fd3afe676113896be214753740e567575bb562986bd2cd4",
      "mediaType": "application/vnd.docker.distribution.manifest.v1+json",
      "platform": {
        "architecture": "arm64",
        "os": "linux",
        "variant": "v8"
      },
      "size": 13297
    },
    {
      "digest": "sha256:2fa12030ffb0224e0b2e17bc4b1f1479e191e2fd65869fc60d09a8efa5b6d879",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "386",
        "os": "linux"
      },
      "size": 1360
    },
    {
      "digest": "sha256:9012a1962d3201492ea815d0b395af623cf78e4b2187f9069d5cee0c0d1345c1",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "ppc64le",
        "os": "linux"
      },
      "size": 1360
    },
    {
      "digest": "sha256:d7b9368659ff1be2bbb4267c9d45e7aa201cb282ac9bc86e09bf390087df68b6",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "s390x",
        "os": "linux"
      },
      "size": 1360
    }
  ],
  "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
  "schemaVersion": 2
}

The linux/arm64/v8 image is a schema 1 image, so desc.Image() will fail, even though we could just pluck the digest from that descriptor. I think this is such a rare (and accidental) case that it's probably not worth fixing (and is currently broken anyway), but possibly add a TODO?

@dvob
Copy link
Contributor Author

dvob commented Jan 8, 2021

Or do you see an easy way to fix it for these cases as well?

@jonjohnsonjr
Copy link
Collaborator

Or do you see an easy way to fix it for these cases as well?

We'd need to move this logic somewhere else:

// matchesPlatform checks if the given platform matches the required platforms.
// The given platform matches the required platform if
// - architecture and OS are identical.
// - OS version and variant are identical if provided.
// - features and OS features of the required platform are subsets of those of the given platform.
func matchesPlatform(given, required v1.Platform) bool {
// Required fields that must be identical.
if given.Architecture != required.Architecture || given.OS != required.OS {
return false
}
// Optional fields that may be empty, but must be identical if provided.
if required.OSVersion != "" && given.OSVersion != required.OSVersion {
return false
}
if required.Variant != "" && given.Variant != required.Variant {
return false
}
// Verify required platform's features are a subset of given platform's features.
if !isSubset(given.OSFeatures, required.OSFeatures) {
return false
}
if !isSubset(given.Features, required.Features) {
return false
}
return true
}
// isSubset checks if the required array of strings is a subset of the given lst.
func isSubset(lst, required []string) bool {
set := make(map[string]bool)
for _, value := range lst {
set[value] = true
}
for _, value := range required {
if _, ok := set[value]; !ok {
return false
}
}
return true
}

(probably as a match.Matcher)

And use that inside crane.Digest to match the right platform. Like here:

index, err := r.IndexManifest()
if err != nil {
return nil, err
}
for _, childDesc := range index.Manifests {
// If platform is missing from child descriptor, assume it's amd64/linux.
p := defaultPlatform
if childDesc.Platform != nil {
p = *childDesc.Platform
}
if matchesPlatform(p, platform) {
return r.childDescriptor(childDesc, platform)
}
}

The TODO is fine for now, I think. There is some discussion around platform matching in #892 that we can tack this work onto.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

crane digest does not work with specific platform on v1 schema
4 participants