Prune prev images on build/run/{dev iteration start} #4792
Conversation
…string> conversions)
- better logging
Codecov Report
@@ Coverage Diff @@
## master #4792 +/- ##
==========================================
+ Coverage 71.73% 71.85% +0.11%
==========================================
Files 353 354 +1
Lines 12108 12214 +106
==========================================
+ Hits 8686 8776 +90
- Misses 2776 2786 +10
- Partials 646 652 +6
Continue to review full report at Codecov.
|
|
@ilya-zuyev I tried your PR locally, I see these messages at the end. This is not something you added but since we are now pruning aggressively, for a dev loop with many iterations, this number could be really high. Looks like there is also case where 2 threads are trying to clean up same image and one gets done faster. Ideally it would be great to only print out errors. |
|
@ilya-zuyev want to get this in before it becomes stale. |
Post-pruner tries to remove all images built during all iterations of dev loop. Some of them can be already pruned by pre-pruner. Also it's possible to have duplicated image ids in Builder.builtImages array, fix this case.
Done.
Actually, I had an issue there. When dev loop is complete, skaffold tries to prune all images built in all iterations of dev loop. Some of them can be already removed by the pre-pruner and this caused a warning. Fixed. |
|
@ilya-zuyev Can we move this skaffold/pkg/skaffold/docker/image.go Line 508 in d9cf47d skaffold/pkg/skaffold/docker/image.go Line 505 in d9cf47d to debug? |
@tejal29 Sure, but those lines were not changed in this PR and this output is made only in the old pruning scenarios (the new pruner uses |
I can see these lines printed on your branch. Is there a way to switch to new use pruner?
Do agree the behavior is not from newly added code like i mentioned before
|
both pruners are controlled by
Yeah, I agree. This PR reuses the existed |
|
got it. We never really thought about the output before :). Please feel free to move these to debug. |
Got it. Thanks! Will update |
|
Thanks @ilya-zuyev This looks great!! |
Prune local docker images at beginning of build/dev loop/run actions.
This PR removes previous versions of docker images from the local registry, when local docker builder is used.
Fixes #4693
The same flag rules are applied as for 'post' pruning. To activate the pruning the
--cache-artifacts=falsecommand line flag must be explicitly passed (its default value istrue) and--no-prunemust befalse(default)When the pruning is activated, and the local docker builder is used, we query all image versions with the artifact's image ref from the local registry, and try to remove all but the latest before we build. We keep the most recent version as If the build is going to fail, we need to keep the last runnable artifact. The pruning can be pretty slow, so it's started it in a separate go-routine
When the build is complete, we repeat the pruning (we may have two images in the registry, as we might skip on in pre-pruning, and need to keep only one). In
runanddevmodes we run it asynchronously again to avoid delays. Inbuildmode this pruning is the last step of the pipeline and we use sync run to wait for pruning to complete before we exit.This PR adds dependency to github.com/dustin/go-humanize to be able to show user numbers is the human readable format. (
3M bytesinstead of3145728 bytes)