detect podman file-system corruption

[idleroamer]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind feature

The embedded environment should guarantee the sanity of podman file-system under all circumstances as there is no way to have end-users fix issues manually. Even with the latest podman 4.2, it is still possible to run into such corruptions with a sudden power loss or a forced termination.
I'd like to ask if there is any way to detect improper podman termination or podman kills. So that one can resort to drastic measures like podman system prune -a.
specifically such problem is still reproducible just by killing podman pull halfway.
WARN[0000] Failed to determine if an image is a manifest list: locating item named "manifest" for image with ID "2b7c1336a711be4ec1e5eb43b693770a8aa602d66257f1e11e1f941a41e257c1" (consider removing the image to resolve the issue): file does not exist, ignoring the error
ERRO[0000] Image 2b7c1336a711 exists in local storage but may be corrupted (remove the image to resolve the issue): locating item named "manifest" for image with ID "2b7c1336a711be4ec1e5eb43b693770a8aa602d66257f1e11e1f941a41e257c1" (consider removing the image to resolve the issue): file does not exist Error: 2b7c1336a711: image not known container running with code: 125
Some refs:
#8743
#8005
containers/storage#1136

Steps to reproduce the issue:
I can't reproduce the issue in a reliable manner, but running a script which pulls images and kills the podman in different points in time should reproduce the issue after a few iteration.

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@mheon @baude Thoughts?

[mheon]

This seems like more of a c/storage or libimage issue? I don't see why we'd do this at the Podman level, and not either Libimage or c/storage so Podman isn't the only thing that benefits from it? CC @nalind @vrothberg

[vrothberg]

Yes, it's more a libimage/storage issue. But we tackled such issues over the course of the years, so it would helpful to know which version of Podman @idleroamer is using.

[idleroamer]

Hi folks, thanks a lot for great job on podman.

The above error in original ticket description I could reproduce in podman 4.2 by killing the podman pull halfway.

[vrothberg]

Does a subsequent pull work?

[vrothberg]

Friendly ping @idleroamer

[idleroamer]

Sorry for late response but for some reason, I can't reproduce the issue anymore. Nevertheless I observed podman filesystem corruption when the system powers-off in the middle of container creation. I will try to come up with a reproducible scenario and more insights, till then I am fine to close this ticket.