Document docker transport is the only supported remote transport #8465

rhatdan · 2020-11-24T12:48:56Z

The goal is to improve errors when users use the wrong transport
in certain cases we stutter, in other cases we don't give enough
information.

Remove stutters when failing to pull remote images, because of
lack of support.

Fix errors returned by reference.Parse to wrap in image that was being
checked.

Fixes: #7116

Signed-off-by: Daniel J Walsh [email protected]

openshift-ci-robot · 2020-11-24T12:48:58Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rhatdan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [rhatdan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

mheon · 2020-11-24T16:28:16Z

docs/source/markdown/podman-create.1.md

@@ -22,6 +22,28 @@ Default settings for flags are defined in `containers.conf`. Most settings for
 remote connections use the server's containers.conf, except when documented in
 man pages.

+## IMAGE
+
+ The `image` uses a "transport":"details" format.


"The image is specified using transport:path format. If no transport is specified, the docker transport will be used by default. For remote Podman, docker is the only allowed transport."

mheon · 2020-11-24T16:30:04Z

docs/source/markdown/podman-create.1.md

+  An image in a registry implementing the "Docker Registry HTTP API V2". By default, uses the authorization state in `$XDG_RUNTIME_DIR/containers/auth.json`, which is set using `(podman login)`. If the authorization state is not found there, `$HOME/.docker/config.json` is checked, which is set using `(docker login)`.
+
+  **docker-archive:**_path_[**:**_docker-reference_]
+  An image is stored in the `docker save` formatted file.  _docker-reference_ is only used when creating such a file, and it must not contain a digest.


"Accepts images in archives created by the podman create command."

Also, please explain docker-reference better - I have no idea when and how I should be using it from this description.

Added an example.

mheon · 2020-11-24T16:32:36Z

docs/source/markdown/podman-create.1.md

+  An existing local directory _path_ storing the manifest, layer tarballs and signatures as individual files. This is a non-standardized format, primarily useful for debugging or noninvasive container inspection.
+
+  **docker://**_docker-reference_
+  An image in a registry implementing the "Docker Registry HTTP API V2". By default, uses the authorization state in `$XDG_RUNTIME_DIR/containers/auth.json`, which is set using `(podman login)`. If the authorization state is not found there, `$HOME/.docker/config.json` is checked, which is set using `(docker login)`.


"Accepts a reference to an image in a remote registry. The reference can include a path to a specific registry; if it does not, the registries listed in registries.conf will be queried to find a matching image. By default, credentials from podman login (stored at $XDG_RUNTIME_DIR/containers/auth.json by default) will be used to authenticate; if these cannot be found, we will fall back to using credentials in $HOME/.docker/config.json.

mheon · 2020-11-24T16:33:20Z

docs/source/markdown/podman-create.1.md

+ transport is allowed for remote access.
+
+  **dir:**_path_
+  An existing local directory _path_ storing the manifest, layer tarballs and signatures as individual files. This is a non-standardized format, primarily useful for debugging or noninvasive container inspection.


How do I make a directory like this? Providing a command would greatly help understanding.

podman push DIR
Would create this or podman pull DIR.

mheon · 2020-11-24T16:34:29Z

docs/source/markdown/podman-create.1.md

+  An image _docker-reference_ stored in the docker daemon internal storage.  _docker-reference_ must contain either a tag or a digest.  Alternatively, when reading images, the format can also be docker-daemon:algo:digest (an image ID).
+
+  **oci-archive:**_path_**:**_tag_
+  An image _tag_ in a directory compliant with "Open Container Image Layout Specification" at _path_.


How do I make such an archive?

Podman pull or podman push, podman save.
podman save IMAGE --format docker-dir

TomSweeneyRedHat · 2020-11-24T16:34:50Z

docs/source/markdown/podman-create.1.md

@@ -22,6 +22,28 @@ Default settings for flags are defined in `containers.conf`. Most settings for
 remote connections use the server's containers.conf, except when documented in
 man pages.

+## IMAGE
+
+ The `image` uses a "transport":"details" format.


Suggested change

The `image` uses a "transport":"details" format.

The specified `image` uses a "transport":"details" format.

mheon · 2020-11-24T16:35:13Z

docs/source/markdown/podman-create.1.md

+  An image is stored in the `docker save` formatted file.  _docker-reference_ is only used when creating such a file, and it must not contain a digest.
+
+  **docker-daemon:**_docker-reference_
+  An image _docker-reference_ stored in the docker daemon internal storage.  _docker-reference_ must contain either a tag or a digest.  Alternatively, when reading images, the format can also be docker-daemon:algo:digest (an image ID).


The last two sentences seem to be redundant - the format can be a tag, or a digest, or a digest?

mheon · 2020-11-24T16:35:32Z

docs/source/markdown/podman-create.1.md

@@ -22,6 +22,28 @@ Default settings for flags are defined in `containers.conf`. Most settings for
 remote connections use the server's containers.conf, except when documented in
 man pages.

+## IMAGE


Everything in this section needs examples - it's extremely unclear how any of this is used in practice.

Also, we need to make it clear that some of these transports (I think not all?) do not pull the image if it is present locally in c/storage.

Added examples.

mheon · 2020-11-24T16:36:11Z

docs/source/markdown/podman-pull.1.md

@@ -27,7 +27,8 @@ Images are stored in local image storage.
 ## SOURCE

 The SOURCE is the location from which the container images are pulled.
- The Image "SOURCE" uses a "transport":"details" format.
+ The Image "SOURCE" uses a "transport":"details" format.  Only the `docker` (container registry)


You should probably split what was written for podman run and podman create out into a separate manpage and have all of these reference that page.

mheon · 2020-11-24T16:36:41Z

pkg/api/handlers/libpod/images.go

@@ -423,7 +423,7 @@ func PushImage(w http.ResponseWriter, r *http.Request) {
 	source := strings.TrimSuffix(utils.GetName(r), "/push") // GetName returns the entire path
 	if _, err := utils.ParseStorageReference(source); err != nil {
 		utils.Error(w, http.StatusText(http.StatusBadRequest), http.StatusBadRequest,
-			errors.Wrapf(err, "image source %q is not a containers-storage-transport reference", source))
+			err)


If we don't have the Wrapf this can go on the previous line

mheon · 2020-11-24T16:36:49Z

pkg/api/handlers/libpod/images.go

@@ -434,7 +434,7 @@ func PushImage(w http.ResponseWriter, r *http.Request) {

 	if _, err := utils.ParseDockerReference(destination); err != nil {
 		utils.Error(w, http.StatusText(http.StatusBadRequest), http.StatusBadRequest,
-			errors.Wrapf(err, "image destination %q is not a docker-transport reference", destination))
+			err)


Move to previous line

mheon · 2020-11-24T16:36:58Z

pkg/api/handlers/libpod/images_pull.go

@@ -54,7 +54,7 @@ func ImagesPull(w http.ResponseWriter, r *http.Request) {
 	imageRef, err := utils.ParseDockerReference(query.Reference)
 	if err != nil {
 		utils.Error(w, http.StatusText(http.StatusBadRequest), http.StatusBadRequest,
-			errors.Wrapf(err, "image destination %q is not a docker-transport reference", query.Reference))
+			err)


Previous line

TomSweeneyRedHat · 2020-11-24T16:39:14Z

docs/source/markdown/podman-create.1.md

+  An existing local directory _path_ storing the manifest, layer tarballs and signatures as individual files. This is a non-standardized format, primarily useful for debugging or noninvasive container inspection.
+
+  **docker://**_docker-reference_
+  An image in a registry implementing the "Docker Registry HTTP API V2". By default, uses the authorization state in `$XDG_RUNTIME_DIR/containers/auth.json`, which is set using `(podman login)`. If the authorization state is not found there, `$HOME/.docker/config.json` is checked, which is set using `(docker login)`.


Suggested change

An image in a registry implementing the "Docker Registry HTTP API V2". By default, uses the authorization state in `$XDG_RUNTIME_DIR/containers/auth.json`, which is set using `(podman login)`. If the authorization state is not found there, `$HOME/.docker/config.json` is checked, which is set using `(docker login)`.

An image in a registry implementing the "Docker Registry HTTP API V2" format. By default, uses the authorization state in `$XDG_RUNTIME_DIR/containers/auth.json`, which is set using `(podman login)`. If the authorization state is not found there, `$HOME/.docker/config.json` is checked, which is set using `(docker login)`.

Wait to review rewrite by @mheon

TomSweeneyRedHat · 2020-11-24T16:41:19Z

docs/source/markdown/podman-create.1.md

+  An image is stored in the `docker save` formatted file.  _docker-reference_ is only used when creating such a file, and it must not contain a digest.
+
+  **docker-daemon:**_docker-reference_
+  An image _docker-reference_ stored in the docker daemon internal storage.  _docker-reference_ must contain either a tag or a digest.  Alternatively, when reading images, the format can also be docker-daemon:algo:digest (an image ID).


Suggested change

An image _docker-reference_ stored in the docker daemon internal storage. _docker-reference_ must contain either a tag or a digest. Alternatively, when reading images, the format can also be docker-daemon:algo:digest (an image ID).

An image in _docker-reference_ format stored in the docker daemon internal storage. _docker-reference_ must contain either a tag or a digest. Alternatively, when reading images, the format can also be docker-daemon:algo:digest (an image ID).

TomSweeneyRedHat · 2020-11-24T16:52:21Z

docs/source/markdown/podman-create.1.md

+  An image _docker-reference_ stored in the docker daemon internal storage.  _docker-reference_ must contain either a tag or a digest.  Alternatively, when reading images, the format can also be docker-daemon:algo:digest (an image ID).
+
+  **oci-archive:**_path_**:**_tag_
+  An image _tag_ in a directory compliant with "Open Container Image Layout Specification" at _path_.


Suggested change

An image _tag_ in a directory compliant with "Open Container Image Layout Specification" at _path_.

An image in a directory compliant with the "Open Container Image Layout Specification" at the specified _path_ and specified with a _tag_.

Do we have more info io the OCI Layout Spec that we should/can point them to?

TomSweeneyRedHat · 2020-11-24T16:57:34Z

docs/source/markdown/podman-run.1.md

+
+  **oci-archive:**_path_**:**_tag_
+  An image _tag_ in a directory compliant with "Open Container Image Layout Specification" at _path_.
+


Same as prior comments

mheon · 2020-11-24T18:15:28Z

docs/source/markdown/podman-create.1.md

+  will fall back to using credentials in $HOME/.docker/config.json.
+
+  **docker-archive:**_path_[**:**_docker-reference_]
+  An image stored in the `docker save` formatted file.  _docker-reference_ is only used when creating such a


This is still really unclear about docker-reference - if it's only used when it created, why do we want it here when we're only pulling/using such an image?

rhatdan · 2020-11-24T18:45:55Z

@mheon @TomSweeneyRedHat PTA Fresh Look.

TomSweeneyRedHat · 2020-11-24T20:00:12Z

docs/source/markdown/podman-create.1.md

+  An image reference stored in  a remote container image registry. Example: "quay.io/podman/stable:latest".
+The reference can include a path to a specific registry; if it does not, the registries listed in
+registries.conf will be queried to find a matching image. By default, credentials from podman login (stored at
+$XDG_RUNTIME_DIR/containers/auth.json by default) will  be used to authenticate; if these cannot be found, we


Suggested change

$XDG_RUNTIME_DIR/containers/auth.json by default) will be used to authenticate; if these cannot be found, we

$XDG_RUNTIME_DIR/containers/auth.json by default) will be used to authenticate; if these cannot be found,

TomSweeneyRedHat · 2020-11-24T20:01:17Z

docs/source/markdown/podman-create.1.md

+The reference can include a path to a specific registry; if it does not, the registries listed in
+registries.conf will be queried to find a matching image. By default, credentials from podman login (stored at
+$XDG_RUNTIME_DIR/containers/auth.json by default) will  be used to authenticate; if these cannot be found, we
+will fall back to using credentials in $HOME/.docker/config.json.


Suggested change

will fall back to using credentials in $HOME/.docker/config.json.

credentials in $HOME/.docker/config.json will be used if available.

TomSweeneyRedHat · 2020-11-24T20:03:09Z

docs/source/markdown/podman-create.1.md

+
+  **docker-daemon:**_docker-reference_
+  An image in _docker-reference_ format stored in the docker daemon internal storage.  _docker-reference_ must
+contain either a tag.  Alternatively, when reading images, the format can also be docker-daemon:algo:digest (an


I'm not sure what to do with "either a tag"? At the very least an "or" is missing or perhaps it's a really bad cut/paste?

TomSweeneyRedHat · 2020-11-24T20:04:44Z

docs/source/markdown/podman-pull.1.md


  **docker-daemon:**_docker-reference_
-  An image _docker-reference_ stored in the docker daemon internal storage.  _docker-reference_ must contain either a tag or a digest.  Alternatively, when reading images, the format can also be docker-daemon:algo:digest (an image ID).
+  An image _docker-reference_ stored in the docker daemon internal storage.  _docker-reference_ must contain either
+  a tag.  Alternatively, when reading images, the format can also be docker-daemon:algo:digest (an image ID).


same comments as previous.

docs/source/markdown/podman-pull.1.md

The goal is to improve errors when users use the wrong transport in certain cases we stutter, in other cases we don't give enough information. Remove stutters when failing to pull remote images, because of lack of support. Fix errors returned by reference.Parse to wrap in image that was being checked. Fixes: containers#7116 Signed-off-by: Daniel J Walsh <[email protected]>

rhatdan · 2020-11-29T11:30:00Z

@mheon @TomSweeneyRedHat PTAL again. Would be nice to get this in to 2.2 release.

mheon · 2020-11-30T18:16:46Z

docs/source/markdown/podman-create.1.md

+## IMAGE
+
+  The image is specified using transport:path format. If no transport is specified, the `docker` (container registry)
+transport will be used by default. For remote Podman, `docker` is the only allowed transport."


Extra " at end

mheon · 2020-11-30T18:16:59Z

One nit then LGTM

mheon · 2020-11-30T18:50:04Z

Merging as is, I will submit a fix PR later.

mheon · 2020-11-30T18:50:17Z

/lgtm

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 24, 2020

mheon reviewed Nov 24, 2020

View reviewed changes

TomSweeneyRedHat reviewed Nov 24, 2020

View reviewed changes

mheon reviewed Nov 24, 2020

View reviewed changes

TomSweeneyRedHat reviewed Nov 24, 2020

View reviewed changes

rhatdan force-pushed the pull branch 2 times, most recently from 7cead85 to 1ce978f Compare November 24, 2020 18:12

mheon reviewed Nov 24, 2020

View reviewed changes

rhatdan force-pushed the pull branch from 1ce978f to 342cc1e Compare November 24, 2020 18:44

TomSweeneyRedHat reviewed Nov 24, 2020

View reviewed changes

docs/source/markdown/podman-pull.1.md Show resolved Hide resolved

rhatdan force-pushed the pull branch from 8fc81a6 to e9b58b8 Compare November 24, 2020 21:21

rhatdan force-pushed the pull branch from e9b58b8 to 9ba52e8 Compare November 29, 2020 11:28

mheon reviewed Nov 30, 2020

View reviewed changes

openshift-ci-robot assigned mheon Nov 30, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Nov 30, 2020

openshift-merge-robot merged commit e2c406f into containers:master Nov 30, 2020

mheon mentioned this pull request Nov 30, 2020

[CI:DOCS] Fix extra quotation mark in manpages. #8518

Merged

github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 24, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 24, 2023

	The `image` uses a "transport":"details" format.
	The specified `image` uses a "transport":"details" format.

	An image in a registry implementing the "Docker Registry HTTP API V2". By default, uses the authorization state in `$XDG_RUNTIME_DIR/containers/auth.json`, which is set using `(podman login)`. If the authorization state is not found there, `$HOME/.docker/config.json` is checked, which is set using `(docker login)`.
	An image in a registry implementing the "Docker Registry HTTP API V2" format. By default, uses the authorization state in `$XDG_RUNTIME_DIR/containers/auth.json`, which is set using `(podman login)`. If the authorization state is not found there, `$HOME/.docker/config.json` is checked, which is set using `(docker login)`.

	An image _docker-reference_ stored in the docker daemon internal storage. _docker-reference_ must contain either a tag or a digest. Alternatively, when reading images, the format can also be docker-daemon:algo:digest (an image ID).
	An image in _docker-reference_ format stored in the docker daemon internal storage. _docker-reference_ must contain either a tag or a digest. Alternatively, when reading images, the format can also be docker-daemon:algo:digest (an image ID).

	An image _tag_ in a directory compliant with "Open Container Image Layout Specification" at _path_.
	An image in a directory compliant with the "Open Container Image Layout Specification" at the specified _path_ and specified with a _tag_.


		oci-archive:_path_:_tag_
		An image _tag_ in a directory compliant with "Open Container Image Layout Specification" at _path_.

	$XDG_RUNTIME_DIR/containers/auth.json by default) will be used to authenticate; if these cannot be found, we
	$XDG_RUNTIME_DIR/containers/auth.json by default) will be used to authenticate; if these cannot be found,

	will fall back to using credentials in $HOME/.docker/config.json.
	credentials in $HOME/.docker/config.json will be used if available.

Document docker transport is the only supported remote transport #8465

Document docker transport is the only supported remote transport #8465

Conversation

rhatdan commented Nov 24, 2020

openshift-ci-robot commented Nov 24, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rhatdan commented Nov 24, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rhatdan commented Nov 29, 2020

Choose a reason for hiding this comment

mheon commented Nov 30, 2020

mheon commented Nov 30, 2020

mheon commented Nov 30, 2020