podman pull shall accept https-input as oci-archive

[dilyanpalauzov]

podman import can download images over HTTPS, but later the podman images shows no repository and no tag:

$ podman import https://mail.aegee.org/oci-images/ldap-2021-09-11.tar
Downloading from "https://mail.aegee.org/oci-images/ldap-2021-09-11.tar"
Getting image source signatures
Copying blob 16446d688987 done  
Copying config a4b7d92986 done  
Writing manifest to image destination
Storing signatures
sha256:a4b7d92986aed414c56b3a7d05934bc1e5c9d80d4fe5e7a9abd68766f73f5106
d@d:~$ podman images
REPOSITORY  TAG         IMAGE ID      CREATED        SIZE
<none>      <none>      a4b7d92986ae  4 seconds ago  2.5 MB

I can fill the REPOSITORY and TAG column, if I first download the file and then use skope copy:

wget https://mail.aegee.org/oci-images/ldap-2021-09-11.tar
$ skopeo copy oci-archive:ldap-2021-09-11.tar:2021-09-11 containers-storage:localhost/openldap:2021-09-11
Getting image source signatures
Copying blob bc523d70bbe7 done  
Copying config f0937f9440 done  
Writing manifest to image destination
Storing signatures
$ podman images
REPOSITORY          TAG         IMAGE ID      CREATED        SIZE
<none>              <none>      a4b7d92986ae  2 minutes ago  2.5 MB
localhost/openldap  2021-09-11  f0937f9440d3  9 hours ago    6.34 MB

but the latter is more cumbersome. Please either extend skopeo copy to be able to download from HTTPS in the oci-archive: transport, extend podman import to be able to import the repository and tag, or both.

podman import has as extra, that it can import .tar.zstd files, (which can be compressed by zsd level 19, which is better than skopeo copy --dest-compress-format=zstd --dest-compress-level=20.

[mtrmac]

Thanks for your report.

First of all, consider using a registry; it’s pretty much certain to be more efficient than linear archive formats, both when publishing and when downloading images.

---

Skopeo is mostly a thin wrapper over c/image, and not really in the business of being an arbitrary HTTP client. It seems possible to do something like

curl 'https://mail.aegee.org/oci-images/ldap-2021-09-11.tar' | skopeo copy oci-archive:/dev/stdin $dest

and adding extra features to make that easier is not really a focus of Skopeo (especially for an approach that is somewhat discouraged anyway).

---

As for podman import, https://github.com/containers/podman/blob/main/docs/source/markdown/podman-import.1.md does document that there’s a second “reference” parameter; if that doesn’t work, please file a Podman issue.

[mtrmac]

Actually, if the input is an OCI image, you probably don’t want podman import at all; that should be loaded via podman load. There is some code in there that should be able to extract the tag from the OCI index.

[dilyanpalauzov]

podman import HTTP-file reference:tag does the right thing. podman load for the particular image sets as TAG as latest and puts inder REPOSITORY, what was originally a TAG.

It would be good, if command existed, which at the same time: consume the tag from the image and downloads the image over HTTP.

[mtrmac]

podman import HTTP-file reference:tag does the right thing.

Did you check the contents of the image? podman import does not interpret the OCI format; it just extracts a tarball into a container filesystem.

Anyway, please report Podman bugs in the Podman repo. That download RFE is also something that Skopeo will not do, but Podman might.

[dilyanpalauzov]

I have not checked the image. In any case, podman images shows the size of the HTTP file, not the size the original image had, before being converted from containers-storage:localhosh/… to oci-image:….tar It seems that running that image, does not find the files.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@dilyanpalauzov What is going on with this issue. Do you believe there is still a problem?

[dilyanpalauzov]

I think it would be more useful, if a single command exists, which downloads an image over HTTP and stores it under containers-storage: Currently this must be acomplished by two commands. Moreover there are so many commands to convert between oci-archive: and containers-storage: that it is hardly possible to remember which command is good when. Thus ideally all command serving the same purpose shall support the same set of options and features.

Apart from this, as described above, image export and import does increase the used size, which is a hint for some defect.

[rhatdan]

@dilyanpalauzov Could you change the title to identify what you would like Podman to do?

[mtrmac]

AFAICS, primarily, that tarball (an OCI archive) needs to be consumed using podman load, not podman import.

Then,

podman load for the particular image sets as TAG as latest and puts inder REPOSITORY, what was originally a TAG.

that does look correct/expected given the image as is: "org.opencontainers.image.ref.name": "2021-09-11" must be turned into a repository name, tags can’t exist without a repository. If it originally was tag somewhere, that’s a separate issue in the process turning the “original” into an OCI archive.

[dilyanpalauzov]

I adjusted the title. I am asking to be able to pass podman pull oci-archive:http://..... Ideally podmanpull https:// shall imply oci-archive:.

[mtrmac]

The semantics of oci-archive:… in podman pull are c/image-defined and shouldn’t be changed in Podman.

Doing this in podman load might make good sense.

[vrothberg]

I'll tackle it. Not a bug but easy to get done.