Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

imagebuildah: reorganize stage and per-stage logic #1473

Closed
wants to merge 2 commits into from

Conversation

nalind
Copy link
Member

@nalind nalind commented Apr 3, 2019

When we're not building with multiple layers, add empty layer items in the image's history so that the instructions from the Dockerfile don't get lost.

Move the preparation for the first container and the commit of the final image in a single-stage build into the per-stage Execute() function.

For multi-stage builds, only create a layer for ADD, COPY, RUN, and the final instruction in a stage, which changes the number of cache images that we create, so a number of tests needed to have their expectations updated.

It also affects the step counts that we output, so a test in the last hunk we change in bud.bats needed to be updated to expect the changed value. In the interests of not making this patch bigger, I expect to include the fix and change that back in a later PR.

Stop removing the final image for intermediate stages, since they're expected to be present as cached images, even if they don't end up as ancestors of the final image, except for single-stage builds, which we document as not populating the cache.

nalind added 2 commits April 2, 2019 14:29
When we're not building with multiple layers, add empty layer items in
the image's history so that the instructions from the Dockerfile don't
get lost.

Signed-off-by: Nalin Dahyabhai <[email protected]>
Move the preparation for the first container and the commit of the final
image in a single-stage build into the per-stage Execute() function.

For multi-stage builds, only create a layer for ADD, COPY, RUN, and the
final instruction in a stage, which changes the step counts that we
output.

Stop removing the final image for intermediate stages, since they're
expected to be present as cached images, even if they don't end up as
ancestors of the final image, except for single-stage builds, which we
document as not populating the cache.

Signed-off-by: Nalin Dahyabhai <[email protected]>
@nalind
Copy link
Member Author

nalind commented Apr 3, 2019

This should fix #1405.

Copy link
Member

@TomSweeneyRedHat TomSweeneyRedHat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this cure containers/podman#2823 which had this as a proposed fix #1466?

@nalind
Copy link
Member Author

nalind commented Apr 3, 2019

We shouldn't be leaving unintended containers after #1446, and leaving intermediate images around is consistent with docker build, so I'm not sure if we want to be making changes to our behavior there.

containers/podman#2823 might be the TODO that it added, which this should resolve.

@TomSweeneyRedHat
Copy link
Member

@nalind thanks. I just verified with Docker-ce and they are leaving an image `' behind for this, along with the images from the "FROM" clauses. My #1146 should not be removing those images I also verified that Docker also does not remove the containers, so perhaps that shouldn't be done either?

Now for the weird part, when I was looking at this earlier, there were no images with '' tagged but now there are. Has some other new code gotten in that might have fixed that or have I completely lost it?

@nalind
Copy link
Member Author

nalind commented Apr 3, 2019

Before this PR, we'd remove the image that was created for stages other than the last one (i.e., the intermediate stages) when we were finished, which I recently noticed differed from how docker build treats them, so this PR changes that.
Leaving them around allows the cache checking logic to find them, which is how this should resolve the part of #1456 where we create a new image with a new image ID, in cases when a suitable one should have been present in the cache.

@rhatdan
Copy link
Member

rhatdan commented Apr 3, 2019

LGTM
@vrothberg PTAL

@giuseppe
Copy link
Member

giuseppe commented Apr 4, 2019

LGTM

@rhatdan
Copy link
Member

rhatdan commented Apr 4, 2019

@rh-atomic-bot r+

@rh-atomic-bot
Copy link
Collaborator

📌 Commit 1892138 has been approved by rhatdan

@rh-atomic-bot
Copy link
Collaborator

⌛ Testing commit 1892138 with merge 8ceda28...

rh-atomic-bot pushed a commit that referenced this pull request Apr 4, 2019
Move the preparation for the first container and the commit of the final
image in a single-stage build into the per-stage Execute() function.

For multi-stage builds, only create a layer for ADD, COPY, RUN, and the
final instruction in a stage, which changes the step counts that we
output.

Stop removing the final image for intermediate stages, since they're
expected to be present as cached images, even if they don't end up as
ancestors of the final image, except for single-stage builds, which we
document as not populating the cache.

Signed-off-by: Nalin Dahyabhai <[email protected]>

Closes: #1473
Approved by: rhatdan
@rh-atomic-bot
Copy link
Collaborator

☀️ Test successful - status-papr, status-travis
Approved by: rhatdan
Pushing 8ceda28 to master...

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants