Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull cleanups #1361

Closed
wants to merge 15 commits into from
Closed

Pull cleanups #1361

wants to merge 15 commits into from

Conversation

mtrmac
Copy link
Contributor

@mtrmac mtrmac commented Feb 21, 2019

As a follow-up to #1319 , this cleans up some of the name handling to remove ambiguities and redundancies when parsing image names, e.g. correcting handling of docker://dir:localpath.

Most users should notice only that the debug log no longer contains various variants of

error parsing image name "localhost/fedora" as given, trying with transport "docker://": Invalid image name "localhost/fedora", expected colon-separated transport:reference

It would be possible to go even further with this, making ResolveName return complete ImageReference values for the source and destination (and perhaps fixing the differences in how various code paths do / don’t use localImageNameForReference), but this seemed like a reasonably compact unit of changes and a good place to stop.

See the individual commit messages for more details.

WARNING: To be honest, I did only a single smoke-test that all of this works at all; I’m mostly relying on existing tests to detect bugs.

@rhatdan
Copy link
Member

rhatdan commented Feb 22, 2019

LGTM
Nice job. I still want to get some of this in a separate place so we can share the code with podman.
You have a lint issue.

@mtrmac mtrmac force-pushed the pull-cleanups branch 2 times, most recently from a2eb352 to e9715b1 Compare February 25, 2019 12:32
Does not change behavior.

Signed-off-by: Miloslav Trmač <[email protected]>
No need to hard-code the :tag / @digest syntax when there
already is an API returning the string representation.

Should not change behavior.

Signed-off-by: Miloslav Trmač <[email protected]>
When ResolveName has already determined that the value is an
ID (prefix), and returned the full ID, rely on that knowledge
and don't try at all to pull the image from a 'remote transport ""';
also, don't try to match strings that are already known not to be
ID prefixes, or that are known to use a different transport, against
local storage.

Should not change behavior, except possibly in theoretical
inconsistency cases when store.Image(knownImageID) fails; the code
now does not report other unrelated errors on the transport == ""
path below.

Signed-off-by: Miloslav Trmač <[email protected]>
This case was originally here for options.Transport, which
no longer exists; and the previous commit has made it impossible
for transport == "" to reach this code path.

Should not change behavior.

Signed-off-by: Miloslav Trmač <[email protected]>
Use the right variable to make sure transport and image are
colon-separated in error reports.

Changes user-visible strings.

Signed-off-by: Miloslav Trmač <[email protected]>
ResolveName now guarantees that the transport, if it exists,
is not a part of the image name; the semantics is no longer
ambiguous, so use the value only as expected.

This could possibly fix incorrect handling of some strings
(pull docker://dir:localpath), and the debug log will no longer
contain "error parsing image name %q as given, trying with transport" for every
name parsing attempt.

Signed-off-by: Miloslav Trmač <[email protected]>
The result of ParseImageName("docker://"...) is not a storageRef.

Should not change behavior.

Signed-off-by: Miloslav Trmač <[email protected]>
pullImage has a dedicated transport: parameter, don't pass the transport
in the image name as well. The semantics of the imageName parameter to
pullImage is now unambiguous.

Should not change behavior, pullImage was trying
alltransports.ParseImageName(imageName) first.

Signed-off-by: Miloslav Trmač <[email protected]>
Should not change behavior, except possibly failing early if the server
returns an invalid tag name.

Signed-off-by: Miloslav Trmač <[email protected]>
Both callers now consistently pass the transport in the "transport"
parameter, so parsing imageName could only be incorrect.

This could possibly fix cases like pulling docker://dir:localpath,
and the debug log will no longer say
"error parsing image name %q, trying with transport %q: %v" on every pull attempt.

Signed-off-by: Miloslav Trmač <[email protected]>
…Reference

It should always be redundant with the reference itself; so,
use srcRef.StringWithinTransport() in the cases where we do
need to understand and hard-code the string syntax, after all.

Also improve the oci: format parsing a bit, to be robust
against including an image name.

NOTE: This might change the semantics a bit because StringWithinTransport
does not guarantee preserving the original string (e.g. paths
tend to be normalized not to contain symlinks).  Using local paths
as docker/distribution image names is conceptually so problematic
that this seems worth the code cleanup - but I might be wrong.

Signed-off-by: Miloslav Trmač <[email protected]>
Right now, we (conceptually unnecesarily) require an image with an existing
tag on the remote repository to list all other tags.

Given that, use the user-specified name:tag, if any, instead of discarding the
tag and requiring :latest to exist on the remote registry.

Signed-off-by: Miloslav Trmač <[email protected]>
This only moves the code, does not modify it at all; a separate
commit to make review easier.

pullImage eventually computes the same value anyway, so this
should not change behavior.  We will soon remove the redundant
value in pullImage.

Signed-off-by: Miloslav Trmač <[email protected]>
…llImage etc.

Use a typed value, to hopefully decrease further temptation to process strings
manually, and to avoid the unnecessary alltransports.ParseImageName which
resolveImage has already called.

This may change the strings used in some error/debug messages, which
now use transports.ImageName instead of the original input; the strings
should by definition have the same semantics.

Signed-off-by: Miloslav Trmač <[email protected]>
The code is already calling docker.GetRepositoryTags
immediately below, so the dependency already exists, and this
removes an unnecessary dependency on alltransports.

Should not change behavior.

Signed-off-by: Miloslav Trmač <[email protected]>
@rhatdan
Copy link
Member

rhatdan commented Feb 26, 2019

LGTM
@nalind @vrothberg @giuseppe @TomSweeneyRedHat PTAL

@vrothberg
Copy link
Member

bot, retest this please

@vrothberg
Copy link
Member

Red Hat CI is "Pending" a bit too long. If the retest shows no effect, @mtrmac might need to repush.

@vrothberg
Copy link
Member

Code LGTM

@TomSweeneyRedHat
Copy link
Member

bot, retest this please

@TomSweeneyRedHat
Copy link
Member

Changes LGTM assuming happy tests. I'm a little leery about merging this in before we spin 1.7-1 later today.

@rhatdan
Copy link
Member

rhatdan commented Feb 26, 2019

I agree.

@rhatdan
Copy link
Member

rhatdan commented Feb 27, 2019

@rh-atomic-bot r+

@rh-atomic-bot
Copy link
Collaborator

📌 Commit e9715b1 has been approved by rhatdan

@rh-atomic-bot
Copy link
Collaborator

⌛ Testing commit e9715b1 with merge e8a9185...

rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
Does not change behavior.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
No need to hard-code the :tag / @digest syntax when there
already is an API returning the string representation.

Should not change behavior.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
Should not change behavior, except possibly failing early if the server
returns an invalid tag name.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
Both callers now consistently pass the transport in the "transport"
parameter, so parsing imageName could only be incorrect.

This could possibly fix cases like pulling docker://dir:localpath,
and the debug log will no longer say
"error parsing image name %q, trying with transport %q: %v" on every pull attempt.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
…Reference

It should always be redundant with the reference itself; so,
use srcRef.StringWithinTransport() in the cases where we do
need to understand and hard-code the string syntax, after all.

Also improve the oci: format parsing a bit, to be robust
against including an image name.

NOTE: This might change the semantics a bit because StringWithinTransport
does not guarantee preserving the original string (e.g. paths
tend to be normalized not to contain symlinks).  Using local paths
as docker/distribution image names is conceptually so problematic
that this seems worth the code cleanup - but I might be wrong.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
Right now, we (conceptually unnecesarily) require an image with an existing
tag on the remote repository to list all other tags.

Given that, use the user-specified name:tag, if any, instead of discarding the
tag and requiring :latest to exist on the remote registry.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
This only moves the code, does not modify it at all; a separate
commit to make review easier.

pullImage eventually computes the same value anyway, so this
should not change behavior.  We will soon remove the redundant
value in pullImage.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
…llImage etc.

Use a typed value, to hopefully decrease further temptation to process strings
manually, and to avoid the unnecessary alltransports.ParseImageName which
resolveImage has already called.

This may change the strings used in some error/debug messages, which
now use transports.ImageName instead of the original input; the strings
should by definition have the same semantics.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
The code is already calling docker.GetRepositoryTags
immediately below, so the dependency already exists, and this
removes an unnecessary dependency on alltransports.

Should not change behavior.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
@rhatdan
Copy link
Member

rhatdan commented Feb 27, 2019

@rh-atomic-bot retry

@rh-atomic-bot
Copy link
Collaborator

⌛ Testing commit e9715b1 with merge 95a5089...

rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
No need to hard-code the :tag / @digest syntax when there
already is an API returning the string representation.

Should not change behavior.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
When ResolveName has already determined that the value is an
ID (prefix), and returned the full ID, rely on that knowledge
and don't try at all to pull the image from a 'remote transport ""';
also, don't try to match strings that are already known not to be
ID prefixes, or that are known to use a different transport, against
local storage.

Should not change behavior, except possibly in theoretical
inconsistency cases when store.Image(knownImageID) fails; the code
now does not report other unrelated errors on the transport == ""
path below.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
This case was originally here for options.Transport, which
no longer exists; and the previous commit has made it impossible
for transport == "" to reach this code path.

Should not change behavior.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
Use the right variable to make sure transport and image are
colon-separated in error reports.

Changes user-visible strings.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
ResolveName now guarantees that the transport, if it exists,
is not a part of the image name; the semantics is no longer
ambiguous, so use the value only as expected.

This could possibly fix incorrect handling of some strings
(pull docker://dir:localpath), and the debug log will no longer
contain "error parsing image name %q as given, trying with transport" for every
name parsing attempt.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
The result of ParseImageName("docker://"...) is not a storageRef.

Should not change behavior.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
pullImage has a dedicated transport: parameter, don't pass the transport
in the image name as well. The semantics of the imageName parameter to
pullImage is now unambiguous.

Should not change behavior, pullImage was trying
alltransports.ParseImageName(imageName) first.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
Should not change behavior, except possibly failing early if the server
returns an invalid tag name.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
Both callers now consistently pass the transport in the "transport"
parameter, so parsing imageName could only be incorrect.

This could possibly fix cases like pulling docker://dir:localpath,
and the debug log will no longer say
"error parsing image name %q, trying with transport %q: %v" on every pull attempt.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
…Reference

It should always be redundant with the reference itself; so,
use srcRef.StringWithinTransport() in the cases where we do
need to understand and hard-code the string syntax, after all.

Also improve the oci: format parsing a bit, to be robust
against including an image name.

NOTE: This might change the semantics a bit because StringWithinTransport
does not guarantee preserving the original string (e.g. paths
tend to be normalized not to contain symlinks).  Using local paths
as docker/distribution image names is conceptually so problematic
that this seems worth the code cleanup - but I might be wrong.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
Right now, we (conceptually unnecesarily) require an image with an existing
tag on the remote repository to list all other tags.

Given that, use the user-specified name:tag, if any, instead of discarding the
tag and requiring :latest to exist on the remote registry.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
This only moves the code, does not modify it at all; a separate
commit to make review easier.

pullImage eventually computes the same value anyway, so this
should not change behavior.  We will soon remove the redundant
value in pullImage.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
…llImage etc.

Use a typed value, to hopefully decrease further temptation to process strings
manually, and to avoid the unnecessary alltransports.ParseImageName which
resolveImage has already called.

This may change the strings used in some error/debug messages, which
now use transports.ImageName instead of the original input; the strings
should by definition have the same semantics.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
rh-atomic-bot pushed a commit that referenced this pull request Feb 27, 2019
The code is already calling docker.GetRepositoryTags
immediately below, so the dependency already exists, and this
removes an unnecessary dependency on alltransports.

Should not change behavior.

Signed-off-by: Miloslav Trmač <[email protected]>

Closes: #1361
Approved by: rhatdan
@rh-atomic-bot
Copy link
Collaborator

☀️ Test successful - status-papr, status-travis
Approved by: rhatdan
Pushing 95a5089 to master...

@mtrmac mtrmac deleted the pull-cleanups branch February 27, 2019 16:46
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 15, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants