-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix podman load oci-archive's tar with name #7386
fix podman load oci-archive's tar with name #7386
Conversation
9d0fb2d
to
6013880
Compare
@mtrmac @vrothberg PTAL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for looking into this. Code is still broken (very unhappy tests), and I can't offer suggestions there, but here's one suggestion re: tests.
test/system/120-load.bats
Outdated
|
||
run_podman load -i $archive test.com/foo:test | ||
verify_name test.com/foo:test | ||
run_podman rmi test.com/foo:test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a nice set of tests, but they're actually unrelated to the purpose of this particular test block ("save to pipe"). They would belong better in a separate @test
. I would also like to see an explicit test for CAPS in the archive name. So maybe something like:
@test "podman save/load: oci-archive" {
# FIXME: remove this if #7371 gets fixed
safe_archive=$PODMAN_TMPDIR/restoreme.tar
run_podman save -o $safe_archive $IMAGE
archive=$PODMAN_TMPDIR/MYIMAGE-$(random_string 8).tar
run_podman save --format oci-archive -o $archive $IMAGE
run_podman rmi $IMAGE
function _load_and_verify() {
run_podman load -i $archive "$1"
run_podman images -a --format '{{.Repository}}:{{.Tag}}'
is "$output" "$2" "podman load, with tag $1"
run_podman rmi "$2"
}
_load_and_verify "foo" "localhost/foo:latest"
_load_and_verify "foo:test" "localhost/foo:test"
_load_and_verify "test.com/foo" "test.com/foo:latest"
_load_and_verify "test.com/foo:test" "test.com/foo:test"
# Restore original test image with real IID (FIXME: #7371)
run_podman load -i $safe_archive
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I got it
ci has some errors, I will try to fix it |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I really don’t understand what this is fixing and how. This is the third PR with, what seems to me, wildly different approach to supposedly the same problem.
You probably understand the situation, after spending days on this, but without writing it down we don’t know whether we are seeing the same thing or considering the same goal (and at least I can’t spend the same days to retrace all your steps now.)
Please make a clear record for the future:
- As a part of the code base:
- End-user documentation: what is the goal / intended behavior of the feature
- Programmer documentation: What is the semantics of the involved functions / data structures.
- (If it is not obvious, which should not be the case after the above:) How the implementation works and matches the function contracts.
- As part of this PR / process:
- What is failing. Was the previous goal of the code incorrect? Or just incorrectly developed?
- (podman load -i Path-With-Caps label : invalid reference format: repository name must be lowercase #7337 only lists an error reported by the
dir:
transport. If this is focusing onoci-archive:
, please write down what is the problem encountered when usingoci-archive:
that this is supposed to be fixing. So far I’m just guessing based on the the very different PRs and I can’t see any connection to upper-case file names.)
- (podman load -i Path-With-Caps label : invalid reference format: repository name must be lowercase #7337 only lists an error reported by the
- How is this PR fixing this
- What is failing. Was the previous goal of the code incorrect? Or just incorrectly developed?
(Others: See #6811 (comment) for some context around this, it’s not at all clear what we are doing here.)
libpod/image/pull.go
Outdated
@@ -176,6 +176,24 @@ func (ir *Runtime) pullGoalFromImageReference(ctx context.Context, srcRef types. | |||
}, nil | |||
|
|||
case OCIArchive: | |||
// imgName such as "oci-archive:/tmp/FOO.tar:localhost/foo:latest" | |||
// or "oci-archive:/tmp/FOO.tar:domain.com/foo:tag1", "oci-archive:/tmp/FOO.tar" | |||
imgNameSli := strings.SplitN(imgName, ":", 3) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Don’t use
imgName
, except maybe for the destination; start with the well-typedsrcRef
. oci-archive:
paths can include:
on Windows.
(Eventually this should get a docker/archive.Reader
-like interface similar to the concept of #6811 . But that’s much more work.)
// or "oci-archive:/tmp/FOO.tar:domain.com/foo:tag1", "oci-archive:/tmp/FOO.tar" | ||
imgNameSli := strings.SplitN(imgName, ":", 3) | ||
if len(imgNameSli) == 3 { | ||
// Fixes: https://github.com/containers/podman/issues/7337 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- How does this relate to podman load -i Path-With-Caps label : invalid reference format: repository name must be lowercase #7337 at all, when the previous version did not use the path component of
srcRef
fordest
either AFAICS? - Why is fixing the single-image case enough? Can’t multi-image archives use upper-case names as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Still no answer to this.
libpod/runtime_img.go
Outdated
@@ -283,14 +283,12 @@ func (r *Runtime) LoadImage(ctx context.Context, name, inputFile string, writer | |||
return dockerarchive.ParseReference(inputFile) // FIXME? We should add dockerarchive.NewReference() | |||
}, | |||
func() (types.ImageReference, error) { | |||
return ociarchive.NewReference(inputFile, name) // name may be "" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- This is incorrect. OCI reference annotations have no semantics other than a set of allowed characters, and it’s perfectly possible fo the annotation in the image to have no slash. It must be possible to refer to such an image.
oci:
andoci-archive:
should behave consistently; this changes the two in different ways.- I can’t see any relationship to podman load -i Path-With-Caps label : invalid reference format: repository name must be lowercase #7337 either way. One goal per PR, please; or at least one commit per concept; or at the very least write down in the PR description what else the PR does!
libpod/runtime_img.go
Outdated
} | ||
return nil, nil | ||
return layout.NewReference(inputFile, iName) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(In both the ociarchive
and layout
code above:)
- AFAICS this is unrelated to podman load -i Path-With-Caps label : invalid reference format: repository name must be lowercase #7337 , and probably belongs to a separate PR.
- If the separate
NewReference(inputFile, name)
attempt remains (as it should), this case should only try prependingDefaultLocalRegistry
. It’s a good point that in the""
case that shouldn’t happen, but in that case these functions should returnnil, nil
, not try to load the archive the second time using the same reference that has already failed.
pkg/domain/infra/abi/images.go
Outdated
@@ -452,7 +452,8 @@ func (ir *ImageEngine) Load(ctx context.Context, opts entities.ImageLoadOptions) | |||
if !opts.Quiet { | |||
writer = os.Stderr | |||
} | |||
name, err := ir.Libpod.LoadImage(ctx, opts.Name, opts.Input, writer, opts.SignaturePolicy) | |||
imgName := opts.Name + ":" + opts.Tag | |||
name, err := ir.Libpod.LoadImage(ctx, imgName, opts.Input, writer, opts.SignaturePolicy) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
⚠️ What’s the semantics ofImageLoadOptions
? We must establish that and document that. (See also podman load/save: support multi-image docker archive #6811 (comment) )- Why is it safe to change this behavior now?
pkg/domain/infra/tunnel.ImageEngine.Load
usesName
+Tag
in yet another different way. Which one is correct?
f893afc
to
483ccbc
Compare
@mtrmac there is strong precedent for the following to just work: # podman load -i FOO.tar myname
# podman load -i FOO.tar myname:mytag This is currently broken when |
I’m not disputing that things are broken, and it’s entirely fair to say that I should do my own research; I’m afraid I can’t spend that time this week; and looking at how all over the place the previous PRs have been, I’d like the PR author to show the analysis that has already been done. (It’s also quite possible I’ve just been an idiot and I’ve been overlooking an entirely obvious problem, and rejecting the clearly correct way to solve it. In that case, showing the existing analysis should hopefully also help.) |
Lines 285 to 318 in 516196f
When importing images, these methods will be tried from front to back In fact, when we import the oci-archive format and use name coverage, neither the second method nor the third method passed normally, so the error message is the e thrown by the last method. When the passed image name is not empty, the final method returns an error |
and the title of the #7337 is wrong,Actually fix this problem:
|
Yes… and what’s wrong about that? The Are we just back at #6811 (comment) ?
Even hypothetically granting the status quo of different transports behaving in different ways, and as a more limited case: either the |
@zhangguanzhang Are you still working on this one? |
|
@zhangguanzhang Thanks enjoy your vacation, I just returned. |
@zhangguanzhang What is going on with this? |
@QiWang19 Is there any progress on your side regarding this matter recently? |
Signed-off-by: zhangguanzhang <[email protected]>
483ccbc
to
13b36ee
Compare
/approve |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: rhatdan, zhangguanzhang The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
@@ -201,6 +201,24 @@ func (ir *Runtime) pullGoalFromImageReference(ctx context.Context, srcRef types. | |||
}, nil | |||
|
|||
case OCIArchive: | |||
// imgName such as "/tmp/FOO.tar:" or "/tmp/FOO.tar:domain.com/foo:tag1" | |||
imgName := strings.TrimSuffix(srcRef.StringWithinTransport(), ":") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What does the TrimSuffix
do?
// or "oci-archive:/tmp/FOO.tar:domain.com/foo:tag1", "oci-archive:/tmp/FOO.tar" | ||
imgNameSli := strings.SplitN(imgName, ":", 3) | ||
if len(imgNameSli) == 3 { | ||
// Fixes: https://github.com/containers/podman/issues/7337 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Still no answer to this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Even hypothetically granting the status quo of different transports behaving in different ways, and as a more limited case: either the
name
parameter toRuntime.LoadImage
foroci-archive:
images specifies what image inside the archive to use (and in that case it should be used for all OCI archives, whether single-image or multi-image), or it doesn’t (and in that case it should be ignored for all OCI archives, whether single-image or multi-image. It makes no sense to me to have a parameter that means different things for single-image and multi-image archives , when most of the call stack has no idea which one it is.
Please make a clear record for the future:
- As a part of the code base:
- End-user documentation: what is the goal / intended behavior of the feature
- Programmer documentation: What is the semantics of the involved functions / data structures.
- (If it is not obvious, which should not be the case after the above:) How the implementation works and matches the function contracts.
- As part of this PR / process:
- What is failing. Was the previous goal of the code incorrect? Or just incorrectly developed?
- (podman load -i Path-With-Caps label : invalid reference format: repository name must be lowercase #7337 only lists an error reported by the
dir:
transport. If this is focusing onoci-archive:
, please write down what is the problem encountered when usingoci-archive:
that this is supposed to be fixing. So far I’m just guessing based on the the very different PRs and I can’t see any connection to upper-case file names.)
- (podman load -i Path-With-Caps label : invalid reference format: repository name must be lowercase #7337 only lists an error reported by the
- How is this PR fixing this
- What is failing. Was the previous goal of the code incorrect? Or just incorrectly developed?
Is this feature needed or not planned to support this feature? |
Which “this”? |
podman load -i /tmp/FOO.tar foo:testTag |
#7337 seems not caused by Path-With-Caps. The issue can be regenerated even with a path using lower cases. |
For the name inconsistencies, I guess if no-one makes an executive decision one way or the other to use one of the semantics and possibly break users, we can always at least document the current behavior (both for the API and the CLI), and document both the behavior and that it is deprecated. In general I don’t think that changing small aspects of the inconsistent behavior would be materially better, as long as an inconsistency remains. OTOH… doesn’t some of the API come directly from Docker? At least that one should have clearer semantics, giving us a specific target that must work. That might involve actually implementing that in a reliable way (a |
Fixes: #7337
fixes this problem:
Signed-off-by: zhangguanzhang [email protected]