mutate.Time should be lazy #1168
Labels
Comments
|
At one point I experimented with this. The API was gross due to streaming layers, but that was so long ago there might be a less naive way to do it: jonjohnsonjr@542d74b |
|
Yeah I like this Maybe we could expose some adapter that takes a |
|
For the layerTime, I ended up having a |
|
This issue is stale because it has been open for 90 days with no |
zx96
added a commit
to zx96/kaniko
that referenced
this issue
Jan 2, 2023
This change adds a new flag to zero timestamps in layer tarballs without making a fully reproducible image. My use case for this is maintaining a large image with build tooling. I have a multi-stage Dockerfile that generates an image containing several toolchains for cross-compilation, with each toolchain being prepared in a separate stage before being COPY'd into the final image. This is a very large image, and while it's incredibly convenient for development, making a change as simple as adding one new tool tends to invalidate caches and force the devs to download another 10+ GB image. If timestamps were removed from each layer, these images would be mostly unchanged with each minor update, greatly reducing disk space needed for keeping old versions around and time spent downloading updated images. I wanted to use Kaniko's --reproducible flag to help with this, but ran into issues with memory consumption (GoogleContainerTools#862) and build time (GoogleContainerTools#1960). Additionally, I didn't really care about reproducibility - I mainly cared about the layers having identical contents so Docker could skip pulling and storing redundant layers from a registry. This solution works around these problems by stripping out timestamps as the layer tarballs are built. It removes the need for a separate postprocessing step, and preserves image metadata so we can still see when the image itself was built. An alternative solution would be to use mutate.Time much like Kaniko currently uses mutate.Canonical to implement --reproducible, but that would not be a satisfactory solution for me until [issue 1168](google/go-containerregistry#1168) is addressed by go-containerregistry. Given my lack of Go experience, I don't feel comfortable tackling that myself, and this seems like a simple and useful workaround in the meantime. As a bonus, I believe that this change also fixes GoogleContainerTools#2005 (though that should really be addressed in go-containerregistry itself).
zx96
added a commit
to zx96/kaniko
that referenced
this issue
Apr 22, 2023
This change adds a new flag to zero timestamps in layer tarballs without making a fully reproducible image. My use case for this is maintaining a large image with build tooling. I have a multi-stage Dockerfile that generates an image containing several toolchains for cross-compilation, with each toolchain being prepared in a separate stage before being COPY'd into the final image. This is a very large image, and while it's incredibly convenient for development, making a change as simple as adding one new tool tends to invalidate caches and force the devs to download another 10+ GB image. If timestamps were removed from each layer, these images would be mostly unchanged with each minor update, greatly reducing disk space needed for keeping old versions around and time spent downloading updated images. I wanted to use Kaniko's --reproducible flag to help with this, but ran into issues with memory consumption (GoogleContainerTools#862) and build time (GoogleContainerTools#1960). Additionally, I didn't really care about reproducibility - I mainly cared about the layers having identical contents so Docker could skip pulling and storing redundant layers from a registry. This solution works around these problems by stripping out timestamps as the layer tarballs are built. It removes the need for a separate postprocessing step, and preserves image metadata so we can still see when the image itself was built. An alternative solution would be to use mutate.Time much like Kaniko currently uses mutate.Canonical to implement --reproducible, but that would not be a satisfactory solution for me until [issue 1168](google/go-containerregistry#1168) is addressed by go-containerregistry. Given my lack of Go experience, I don't feel comfortable tackling that myself, and this seems like a simple and useful workaround in the meantime. As a bonus, I believe that this change also fixes GoogleContainerTools#2005 (though that should really be addressed in go-containerregistry itself).
zx96
added a commit
to zx96/kaniko
that referenced
this issue
Apr 22, 2023
This change adds a new flag to zero timestamps in layer tarballs without making a fully reproducible image. My use case for this is maintaining a large image with build tooling. I have a multi-stage Dockerfile that generates an image containing several toolchains for cross-compilation, with each toolchain being prepared in a separate stage before being COPY'd into the final image. This is a very large image, and while it's incredibly convenient for development, making a change as simple as adding one new tool tends to invalidate caches and force the devs to download another 10+ GB image. If timestamps were removed from each layer, these images would be mostly unchanged with each minor update, greatly reducing disk space needed for keeping old versions around and time spent downloading updated images. I wanted to use Kaniko's --reproducible flag to help with this, but ran into issues with memory consumption (GoogleContainerTools#862) and build time (GoogleContainerTools#1960). Additionally, I didn't really care about reproducibility - I mainly cared about the layers having identical contents so Docker could skip pulling and storing redundant layers from a registry. This solution works around these problems by stripping out timestamps as the layer tarballs are built. It removes the need for a separate postprocessing step, and preserves image metadata so we can still see when the image itself was built. An alternative solution would be to use mutate.Time much like Kaniko currently uses mutate.Canonical to implement --reproducible, but that would not be a satisfactory solution for me until [issue 1168](google/go-containerregistry#1168) is addressed by go-containerregistry. Given my lack of Go experience, I don't feel comfortable tackling that myself, and this seems like a simple and useful workaround in the meantime.
4 tasks
|
I've improved this partially in GoogleContainerTools/kaniko#3347, but I wonder if I should submit the PR here? I see the value in using lazy logic here, that is likely a better method than what I've requested in kaniko. Would you be open to a PR implementing the lazy transformation much like @jonjohnsonjr had? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
pkg/v1/mutate.Timesets all timestamps in an image to the given time, typically to make the image more reproducible. For timestamps in layers, it callslayerTimewhich reads the layer contents and writes them to a bufferedtar.Writer. This means that if the layer is huge, we may run out of memory buffering its contents.We should consider making this a lazy transformation, so that layer mutations aren't made until they're read. We did a similar thing in
pkg/v1/cache, where layer contents aren't cached until they're read viaCompressedorUncompressed.The text was updated successfully, but these errors were encountered: