build with --squash-all + --timestamp clobbers timestamps in preexisting layers

[edsantiago]

[filing under podman, not buildah, because --squash-all is a podman-only option]

$ printf "FROM alpine\nRUN echo hi\n" | bin/podman build --squash-all --timestamp 0 -t foo -
STEP 1/2: FROM alpine
STEP 2/2: RUN echo hi
hi
COMMIT foo
Getting image source signatures
Copying blob f89409ba1b89 skipped: already exists  
Copying config 7350b81f59 done  
Writing manifest to image destination
Storing signatures
--> 7350b81f59a
Successfully tagged localhost/foo:latest
7350b81f59a6cb83bd33e1a5f36a404655b947fa8b47f876dfc9b9e14b041ccc
$ for i in alpine foo;do bin/podman run --rm $i ls -l /etc/profile;done
-rw-r--r--    1 root     root           857 Mar  2 19:00 /etc/profile
-rw-r--r--    1 root     root           857 Jan  1  1970 /etc/profile         <<<<< unexpected

Documentation for --timestamp includes:

All files committed to the layers of the image will be created with the timestamp.

...but to me that applies to files added/created in the build, not files in the existing image. The current behavior violates POLA.

[edsantiago]

Here is my use case:

1. Build a base image in which /home/podman/testimage-id has a precisely tuned timestamp.
2. Build a new image based on the above image. This second image must use --timestamp, because of podman system prune support prune unused networks #14556. I would like to use --squash-all here, because otherwise I have to rework an existing image tree test.

--timestamp is a hard requirement, --squash-all is a wishlist. I will figure out a way to update the image tree test if necessary, I just don't think it should be, because I find it surprising that --squash-all clobbers timestamps on preexisting layers. My intuition is that those preexisting layers should be inviolate.

[flouthoc]

Hi @edsantiago , From last discussion I think fixing this and making --squash-all work with --timestamp would be hard since it also squashes all the upper layers including the base. Not sure if this solution is realistic but one very inefficient solution would be to create copy of the base as the most lower layer again ( without --timestamp and minus the diff from any other layers ) and include it in squashed result again.

@nalind @edsantiago If you are okay with it could we add a note for this behavior of --timestamp with --squash-all in the docs.

[nalind]

I thought about it some more, and I think that using the not-often-used storage Changes() method to gather a list of modified and added filepaths in the container, and using them to decide whether or not to reset the timestamp on an item that we read when we're tarring up the rootfs, could work. It would still be slower than the current implementation, but not prohibitively slower, and it would avoid the need for additional disk space that the approach I sketched out yesterday would have required.
I don't really have a strong preference between doing that and resolving this with a docs update, though, so whatever folks with stronger opinions prefer would be fine with me.

[edsantiago]

Speaking as someone with strong opinions :-), my intuition is that using --squash-all with --timestamp is going to be rare -- this is definitely an obscure corner case -- but, when someone does use it, precision is going to be more important than speed.

(Disclaimer: conviction is poorly correlated with actual truth. My "strong opinions" really should yield to careful consideration from those better informed).

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@nalind @flouthoc Did either of you look at this issue?

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@nalind @flouthoc Re-ping

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[edsantiago]

This is unlikely ever to get fixed. I have a suboptimal workaround and can live with the bug.