system df: fix image-size calculations

[vrothberg]

Fix two bugs in system df:

1. The total size was calculated incorrectly as it was creating the sum of all image sizes but did not consider that a) the same image may be listed more than once (i.e., for each repo-tag pair), and that b) images share layers.

   The total size is now calculated directly in libimage by taking multi-layer use into account.

2. The reclaimable size was calculated incorrectly. This number indicates which data we can actually remove which means the total size minus what containers use (i.e., the "unique" size of the image in use by containers).

Fixes: #16135
Signed-off-by: Valentin Rothberg [email protected]

Does this PR introduce a user-facing change?

Fix a bug where `podman system df` reported wrong image sizes.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vrothberg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [vrothberg]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[vrothberg]

The failures seem to be triggered by containers/storage@main

[edsantiago]

Could you run your new tests against main? They pass for me, which means you're not actually testing your changes.

[vrothberg]

The failures seem to be triggered by containers/storage@main

For now I forced back the previous c/storage version via a replace. @mtrmac @rhatdan PTAL

[vrothberg]

Easy reproducer among others:

$ go get github.com/containers/storage@main
$ make vendor && make podman
$ ./test/apiv2/test-apiv2 images

[edsantiago]

Out of curiosity, do you know when this broke? Last night's treadmill job passed. Could the breakage have happened late last night? Or is the treadmill broken?

[vrothberg]

Out of curiosity, do you know when this broke? Last night's treadmill job passed. Could the breakage have happened late last night? Or is the treadmill broken?

No idea, sorry. c/storage had a lot of traffic recently.

[mheon]

Code LGTM

[edsantiago]

Tests are still broken, in the sense that they pass on main. You are not testing anything new. Can you please find a way to write tests that break on main and pass with this PR?

[vrothberg]

Tests are still broken, in the sense that they pass on main. You are not testing anything new. Can you please find a way to write tests that break on main and pass with this PR?

Thanks for checking, Ed! Will do 👍

Also need to update the apiv2 tests:

[+0165s] not ok 993 [45-system] GET libpod/system/df : output
[+0165s] #  expected: {"Images":[],"Containers":[],"Volumes":[]}
[+0165s] #    actual: {"ImagesSize":0,"Images":[],"Containers":[],"Volumes":[]}

[vrothberg]

@edsantiago can you take another look at the tests?

[edsantiago]

ARGH! Sorry; left tests running this morning, moved on to something else. New test looks great, and fails (as it should) on main. Thank you!
/lgtm