Nonvolatile Upper Overlay Mount

[p0da]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind feature

Description

Would it be possible expand the current overlay volume mount support to allow for modifications to the upper to not be discarded on container stop but instead cached to a specified location. Then upon the next start the mount could be reconstructed, ultimately allowing the overlay mount's state to be preserved throughout various container stops and starts. This would be immensely useful when working with rootless containers that are mounting paths owned by root that neither the user or container are able to write to.

I believe this is currently possible using oci-hooks but proper support would be appreciated.

[rhatdan]

@flouthoc WDYT? The issue is we would not want to save this content into an image. I think doing this would be fairly easy, figuring out a UI for the user to specify it could be difficult. Would :o versus :O work.
:O Overlay last while container is running (Does not get committed to image).
:o Overlay lasts while container exists (Does not get committed to image).

[flouthoc]

@rhatdan Could we have another extra flag --persist-overlay or something similar which could be specified as an additional flag with :O. Since :O and :o would become confusing with time. But overall I agree this could be useful. I'll check this issue.

Thanks @p0da for creating this issue

[rhatdan]

The problem with additional option is this is per volume. So theoretically I could have

-v /a:/a:O -v /b:/b:o

[rhatdan]

As far as confusion, I don't think it would be a huge issue. Only time :O would matter would be in the case of Builds, which is why this Overlay volume was first designed. In the case of --rootfs it might make more sense to default to persist.

We want :O behavior in builds, since we don't want

RUN dnf -y update; dnf -y clean all
To remove preexisting cache, We would rather it return to the underlying content.

[flouthoc]

ah i see makes sense.

[flouthoc]

@p0da Also i am interested in understanding your actual use case.

As far as i can see you want overlayed named volumes to persist while you also need changes to be reflected on from lower right ?

[p0da]

I would not need changes to be propagated to the lower directory, just that the upper directory's state is cached to a specified arbitrary location (such as a separate bind mount) allowing for both mount point reconstruction on container restart and backup/archival of the upper should it be necessary to destroy the initial container. Apologies if that was not clear.

[flouthoc]

My bad i did a typo: I meant you also wanted changes reflected from lower. Cause otherwise you could have just used named volumes with content copied. Nvm I understand your use-case. Thanks

[rhatdan]

I think we just add :o and keep it simple.

[p0da]

In that case would there be any way to extract the upper's cached content from the container if needed?

[rhatdan]

Only by sneaking under the covers.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[giuseppe]

can we add upperdir= and workdir= mount flags? If upperdir= is present, then we use it instead of creating a new one.

-v /foo:/bar:O,upperdir=/persistent/upperdir/content,workdir=/persistent/upperdir/wd

The nice thing is that in future we can think of adding new custom options (e.g. volatile)

[flouthoc]

I agree since this is already advanced use-case, I guess more verbose configuration options might be more helpful.

[rhatdan]

Are we opening ourselves up to potential weird errors. Is workdir allowed to be on different disk then upperdir?
@rhvgoyal WDYT?

[giuseppe]

They both must be on the same mount otherwise the overlay mount fails

[rhatdan]

Which we should probably check, otherwise this will end being a common failure

upperdir=$HOME/upper, WORKDIR=/tmp/workdir

[github-actions]

A friendly reminder that this issue had no activity for 30 days.