Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correctly detect changes in rootdir #1511

Closed
wants to merge 1 commit into from
Closed

Correctly detect changes in rootdir #1511

wants to merge 1 commit into from

Conversation

danstiner
Copy link
Contributor

@danstiner danstiner commented Apr 14, 2019

When using buildah bud --layers ... with copy of the root dir (eg COPY . /app), cached builds will skip the COPY layer even if files have in fact changed.

This is because resolveModifiedTime bails and says there are notifications if the rootdir is being checked. I don't see any reason we can't check the rootdir except for the custom string slicing to compute relative paths. I replaced that with a call to filepath.Rel() and things are working as expected.

I believe this may fix containers/podman#2544

I used the following script as a repro that changes the content of a file and re-builds with --layers.

#!/bin/sh
set -euo pipefail

BUILDAH=buildah

rm -r test || true
mkdir test

echo "A" > test/file
$BUILDAH bud -t layer-caching-experiment:A .

echo "B" > test/file
$BUILDAH bud --layers -t layer-caching-experiment:B-cached .
$BUILDAH bud -t layer-caching-experiment:B .

echo "Build with file content A"
podman run --rm layer-caching-experiment:A cat /app/test/file
echo "Build with file content B (cached)"
podman run --rm layer-caching-experiment:B-cached cat /app/test/file
echo "Build with file content B (no cache)"
podman run --rm layer-caching-experiment:B cat /app/test/file

Expected output shows a cached build picking up the changes to the file:

Build with file content A
A
Build with file content B (cached)
B
Build with file content B (no cache)
B

Actual output as of d43787b:

Build with file content A
A
Build with file content B (cached)
A
Build with file content B (no cache)
B

@rh-atomic-bot
Copy link
Collaborator

Can one of the admins verify this patch?
I understand the following commands:

  • bot, add author to whitelist
  • bot, test pull request
  • bot, test pull request once

@rhatdan
Copy link
Member

rhatdan commented Apr 14, 2019

bot, add author to whitelist

@rhatdan
Copy link
Member

rhatdan commented Apr 14, 2019

Thanks, @danstiner
Tests are failing
@TomSweeneyRedHat @umohnani8 @nalind PTAL

Fixes issue where `COPY . <dest>` would never detect changes by
using `filepath.Rel()` instead of custom string slicing.

Signed-off-by: Daniel Stiner <[email protected]>
@danstiner
Copy link
Contributor Author

Looks like I missed signing off the commit, did that and force pushed.

@TomSweeneyRedHat
Copy link
Member

The code LGTM, but would like a head nod from @nalind as I'm having problems testing this. I'm running into problems with Podman after I vendor this in, but I think it's self-induced and not related to this PR.

@rhatdan
Copy link
Member

rhatdan commented Apr 15, 2019

@danstiner Could you add the tests to make sure we don't regress this.

@danstiner
Copy link
Contributor Author

Happy to either as part of this PR or as a follow-up. May take a couple days depending how busy I am.

@nalind
Copy link
Member

nalind commented Apr 24, 2019

This approach looks correct to me. Nice catch!

@umohnani8
Copy link
Member

LGTM

@rhatdan
Copy link
Member

rhatdan commented Apr 24, 2019

@rh-atomic-bot r+
@danstiner I am going to merge, could you add a new PR with some tests, Thanks for this patch.

@rh-atomic-bot
Copy link
Collaborator

📌 Commit acee3b3 has been approved by rhatdan

@rh-atomic-bot
Copy link
Collaborator

⌛ Testing commit acee3b3 with merge 4bd9f81...

@rh-atomic-bot
Copy link
Collaborator

☀️ Test successful - status-papr, status-travis
Approved by: rhatdan
Pushing 4bd9f81 to master...

@danstiner danstiner deleted the layer-caching branch April 28, 2019 19:32
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 13, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

podman build appears to incorrectly reuse layers
6 participants