Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flakes: Do not ignore lock files of intermediate nodes #6326

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Flakes: Do not ignore lock files of intermediate nodes #6326

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d24aa76
Flakes: Add test for intermediate lock files
Mar 7, 2022
62eb056
Flakes: Add a second sub-flake to intermediate lock file test
Mar 28, 2022
7372f5c
Flakes: Do not ignore lock files of intermediate nodes
danme Mar 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions src/libexpr/flake/flake.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,7 @@ LockedFlake lockFlake(
std::shared_ptr<Node> node,
const InputPath & inputPathPrefix,
std::shared_ptr<const Node> oldNode,
std::shared_ptr<const Node> inputNode,
const InputPath & lockRootPath,
const Path & parentPath,
bool trustLock)>
Expand All @@ -359,6 +360,7 @@ LockedFlake lockFlake(
std::shared_ptr<Node> node,
const InputPath & inputPathPrefix,
std::shared_ptr<const Node> oldNode,
std::shared_ptr<const Node> inputNode,
const InputPath & lockRootPath,
const Path & parentPath,
bool trustLock)
Expand Down Expand Up @@ -423,6 +425,11 @@ LockedFlake lockFlake(
if (auto oldLock2 = get(oldNode->inputs, id))
if (auto oldLock3 = std::get_if<0>(&*oldLock2))
oldLock = *oldLock3;
// fallback to load input from this nodes lock file
if (!oldLock && inputNode && !lockFlags.inputUpdates.count(inputPath))
if (auto oldLock2 = get(inputNode->inputs, id))
if (auto oldLock3 = std::get_if<0>(&*oldLock2))
oldLock = *oldLock3;

if (oldLock
&& oldLock->originalRef == *input.ref
Expand Down Expand Up @@ -495,7 +502,7 @@ LockedFlake lockFlake(
mustRefetch
? getFlake(state, oldLock->lockedRef, false, flakeCache, inputPath).inputs
: fakeInputs,
childNode, inputPath, oldLock, lockRootPath, parentPath, !mustRefetch);
childNode, inputPath, oldLock, nullptr, lockRootPath, parentPath, !mustRefetch);

} else {
/* We need to create a new lock file entry. So fetch
Expand Down Expand Up @@ -543,7 +550,8 @@ LockedFlake lockFlake(
inputFlake.inputs, childNode, inputPath,
oldLock
? std::dynamic_pointer_cast<const Node>(oldLock)
: LockFile::read(
: nullptr,
LockFile::read(
inputFlake.sourceInfo->actualPath + "/" + inputFlake.lockedRef.subdir + "/flake.lock").root,
oldLock ? lockRootPath : inputPath, localPath, false);
}
Expand All @@ -568,7 +576,7 @@ LockedFlake lockFlake(

computeLocks(
flake.inputs, newLockFile.root, {},
lockFlags.recreateLockFile ? nullptr : oldLockFile.root, {}, parentPath, false);
lockFlags.recreateLockFile ? nullptr : oldLockFile.root, nullptr, {}, parentPath, false);

for (auto & i : lockFlags.inputOverrides)
if (!overridesUsed.count(i.first))
Expand Down
94 changes: 93 additions & 1 deletion tests/flakes.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,12 @@ flakeFollowsB=$TEST_ROOT/follows/flakeA/flakeB
flakeFollowsC=$TEST_ROOT/follows/flakeA/flakeB/flakeC
flakeFollowsD=$TEST_ROOT/follows/flakeA/flakeD
flakeFollowsE=$TEST_ROOT/follows/flakeA/flakeE
flakeHierMain=$TEST_ROOT/hier/flakeMain
flakeHierSub=$TEST_ROOT/hier/flakeSub
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This "subflake" terminology is confusing. To me, a subflake is a flake in a subdirectory of a repo, not an input of another flake.

flakeHierSubSub=$TEST_ROOT/hier/flakeSubSub
flakeHierOtherSub=$TEST_ROOT/hier/flakeOtherSub

for repo in $flake1Dir $flake2Dir $flake3Dir $flake7Dir $templatesDir $nonFlakeDir $flakeA $flakeB $flakeFollowsA; do
for repo in $flake1Dir $flake2Dir $flake3Dir $flake7Dir $templatesDir $nonFlakeDir $flakeA $flakeB $flakeFollowsA $flakeHierMain $flakeHierSub $flakeHierSubSub $flakeHierOtherSub; do
rm -rf $repo $repo.tmp
mkdir -p $repo
git -C $repo init
Expand Down Expand Up @@ -834,3 +838,91 @@ nix store delete $(nix store add-path $badFlakeDir)

[[ $(nix path-info $(nix store add-path $flake1Dir)) =~ flake1 ]]
[[ $(nix path-info path:$(nix store add-path $flake1Dir)) =~ simple ]]

# Test flake honors intermediate lock files

# 1. Creates and locks a "main" flake referencing a trivial empty
# "sub" flake via a specific commit hash;

cat > $flakeHierSub/flake.nix <<EOF
{
outputs = { self }: { };
}
EOF

git -C $flakeHierSub add flake.nix
git -C $flakeHierSub commit -m 'Trivial sub-flake'

hashSub="$(git -C $flakeHierSub rev-parse HEAD)"

cat > $flakeHierOtherSub/flake.nix <<EOF
{
outputs = { self }: { };
}
EOF

git -C $flakeHierOtherSub add flake.nix
git -C $flakeHierOtherSub commit -m 'Trivial sub-flake'

hashOtherSub="$(git -C $flakeHierSub rev-parse HEAD)"

cat > $flakeHierMain/flake.nix <<EOF
{
inputs.sub.url = "git+file://$flakeHierSub?rev=$hashSub";
inputs.otherSub.url = "git+file://$flakeHierOtherSub";
outputs = { self }: { };
}
EOF

git -C $flakeHierMain add flake.nix

nix flake update $flakeHierMain

git -C $flakeHierMain add flake.lock
git -C $flakeHierMain commit -m 'Reference sub-flakes'

# 2. Adds a "sub-sub" flake, references it *mutably* from the "sub"
# flake, and locks the "sub" flake via `nix flake update`;

cat > $flakeHierSubSub/flake.nix <<EOF
{
outputs = { self }: { };
}
EOF

git -C $flakeHierSubSub add flake.nix
git -C $flakeHierSubSub commit -m 'Trivial sub-sub-flake'

hashSubSub=$(git -C $flakeHierSubSub rev-parse HEAD)

sed -i $flakeHierSub/flake.nix \
-e '2i inputs.subSub.url = "git+file://'$flakeHierSubSub'";'

nix flake update $flakeHierSub

git -C $flakeHierSub add flake.lock
git -C $flakeHierSub commit -a -m 'Reference trivial sub-sub-flake mutably'

git -C $flakeHierOtherSub commit --allow-empty -m 'New commit in other-sub'

# 3. Adds a new commit to the "sub-sub" flake;

git -C $flakeHierSubSub commit --allow-empty -m 'New sub-sub commit'

# 4. Updates the "main" flake to reference the "sub" flake mutably, but
# does *not* lock it (yet).

sed -i $flakeHierMain/flake.nix \
-e 's/?rev=[a-f0-9]*//'

git -C $flakeHierMain commit -a -m 'Reference sub-flake mutably'

# Here, both `metadata` and `update` rewrite the lock file in "main",
# but the former fetches the last commit from "sub-sub," whereas the
# latter honors the contents of the lock file in "sub."

nix flake metadata $flakeHierMain

[[ $(jq -r .nodes.subSub.locked.rev $flakeHierMain/flake.lock) = $hashSubSub ]]

[[ $(jq -r .nodes.otherSub.locked.rev $flakeHierMain/flake.lock) = $hashOtherSub ]]