RFE: Save multiple manifests to a single file

[haircommander]

There have been some requests for podman to match docker functionality in a podman save:
containers/podman#2822
containers/podman#2669

A blocker on this is the fact we can't save multiple images to the same file with containers/image. There could be a hacky work around saving them to a temp file, then appending all the files at the end, but I'd rather this happen natively in c/image.

[haircommander]

@mtrmac PTAL

[mtrmac]

Yeah, as mentioned before, c/image does not have a concept of “a pending set of images”.

I’m not convinced it’s reasonable to make this a generic c/image concept, notably because the way how to designate images in that set, and the semantics of naming within the set, would be pretty tricky.

But specifically for docker-archive: (or, for any other single transport), it should be reasonably possible to implement a docker/archive.ArchiveWriter (with a better name), used something like:

wip := archive.NewArchiveWriter(destPath)
for /* sourceRef, destinationTag */ {
   destRef := wip.NewReference(destinationTag)
   copy.Image(…, sourceRef, destinationRef, …)
}
wip.Finish()

+ adapting the existing archive.archiveReference to internally create a single-image ArchiveWriter that is transparently Finished on Commit.

[haircommander]

@mtrmac Without even doing that, we could also have a Archive writer append to a file, no? That way, podman can just make consecutive image.Save() on the same destination and they will all end up in the same file.

[haircommander]

(note: this is me speaking with near zero understanding of how c/image works, so please excuse my inexperience :) )

[mtrmac]

@mtrmac Without even doing that, we could also have a Archive writer append to a file, no?

Not really, the archive must contain a single manifest.json file that describes all images.

Sure, each update could also append an updated version of that manifest.json — but I don’t know whether it is formally specified anywhere what a tar file with duplicates means, but it’s certainly unwise to rely these corner cases — and anyway, c/image itself, when reading from docker-archive: files, currently uses the first instance matching the requested file path.

[haircommander]

@mtrmac ah yeah okay. Are you able to implement the above solution on c/image side? I can handle adding it to podman, if so. We had another user ask on IRC today, which sets the request count to three. We can start with just the docker-archive: case right now, I think.

[rhatdan]

@haircommander @mtrmac What is the latest on this?

[rhatdan]

@mtrmac we need input on this.

[mtrmac]

*shrug* the technical design is clear enough, but I don’t really want to commit to a feature and a timeline to deliver it in a GitHub comment independently from the kanban board.

[haircommander]

@mtrmac I'll add the card and we can discuss timeline there :)

[ruckc]

this bit me today, and i've been trying to champion podman over docker, because of my faith in @rhatdan.

[rhatdan]

@haircommander any update on this?

[vrothberg]

Ping

[haircommander]

as reported here containers/podman#2669 (comment)
I don't currently have the bandwidth to take this on ☹️

[marcfiu]

I am trying to champion podman over docker; however, colleagues use "docker save" to air gap images to a docker-archive tarball. This translates into "podman" not being a drop-in replacement "docker", which is suboptimal. At the least, "podman save" should error out indicating "can't do multiple images" rather than silently succeeding with something that is simply not equivalent to "docker save".

[rhatdan]

@baude @vrothberg This needs to be a top priority once APIV2 is complete.

[vrothberg]

I believe there's already a card for it. After v2, I need to get back to systemd and auto updates first.

[marcfiu]

@baude @vrothberg This needs to be a top priority once APIV2 is complete.

@rhatdan: what's the timeline for APIV2?

[rhatdan]

Within the next month. If someone wanted to contribute a PR to this effort, that would certainly speed up the process.

[marcfiu]

@rhatdan: wish we were in a position to make such a PR contribution. Short of that maybe next time I'm in your neck of the woods (which I believe is 2-3 miles down the road from some of my colleagues' office in W***ford), I'll be happy to buy you a beer at the BBC.

[rhatdan]

Sure if we ever get out of lockdown.

[mtrmac]

#996 implements this.