Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement path-bases (RFC 3529) 2/n: support for nested packages #14416

Closed
wants to merge 1 commit into from

Conversation

dpaoliello
Copy link
Contributor

@dpaoliello dpaoliello commented Aug 16, 2024

Cargo has the ability to discover "nested packages" when loading the dependencies of a package. For example, if a git dependency has a path dependency pointing to somewhere else in the repo, or a submodule, then Cargo needs to discover those dependencies as well.

This change makes the "nested packages" loading code aware of the "path bases" feature so that it can build the correct manifest path for any path dependencies that may be using a base.

RFC: rust-lang/rfcs#3529
Tracking Issue: #14355

@rustbot
Copy link
Collaborator

rustbot commented Aug 16, 2024

r? @weihanglo

rustbot has assigned @weihanglo.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added A-manifest Area: Cargo.toml issues S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 16, 2024
@dpaoliello dpaoliello mentioned this pull request Aug 16, 2024
10 tasks
@epage
Copy link
Contributor

epage commented Aug 17, 2024

Cargo has the ability to discover "nested packages" when loading the dependencies of a package. For example, if a git dependency has a path dependency pointing to somewhere else in the repo, or a submodule, then Cargo needs to discover those dependencies as well.

This change makes the "nested packages" loading code aware of the "path bases" feature so that it can build the correct manifest path for any path dependencies that may be using a base.

Note that this change also fixes a bug in the "nested package" loading code: specifically, that all nested packages were read using the GlobalContext of the "root" package, effectively ignoring any config.toml files of those nested packages.

I believe the existing behavior is what is "expected" and this PR is a regression in Cargo.

EDIT: To expand on this, configuration is environmental configuration and not project configuration. The latter is what Cargo.toml is for but we aren't supporting Cargo.toml yet. See also #14001, #2930

@dpaoliello
Copy link
Contributor Author

I believe the existing behavior is what is "expected" and this PR is a regression in Cargo.

EDIT: To expand on this, configuration is environmental configuration and not project configuration. The latter is what Cargo.toml is for but we aren't supporting Cargo.toml yet. See also #14001, #2930

Ah, ok, #2930 is what I was tripping over when writing my test: I'll adjust the test and revert the regression.

I'll also add a note to the docs to mention this: if you depend on a package that uses a path base, then that path base must be defined in the configuration loaded for the root package.

@epage
Copy link
Contributor

epage commented Aug 17, 2024

Interesting timing since I called this out in https://blog.rust-lang.org/inside-rust/2024/08/15/this-development-cycle-in-cargo-1.81.html#path-bases as a case that can't be supported

It can feel a bit strange for a Cargo.toml (project-specific configuration) to be dependent on .cargo/config.toml (environment-specific configuration), in this case to get the definition of base. In particular, to better support these roles we've been wanting to find how to support some .cargo/config.toml workflows in Cargo.toml (#12738). As a real-world consequence of this, a project using a base defined in .cargo/config.toml cannot be used as a git dependency or patch.

(it was as I was writing up that section that I connected the dots on this)

Comment on lines +440 to +446
write_config_at(
paths::home().join(".cargo/config.toml"),
r#"
[path-bases]
submodules = '../dep1/submods'
"#,
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo writing path-bases on behalf of git dependencies seems very questionable

  • Constructing the right absolute path is a mess
  • Constructing the right relative path is a mess

If you want to test RecursivePathSource, https://doc.rust-lang.org/cargo/reference/overriding-dependencies.html#paths-overrides is another user of that tyoe

Comment on lines +1008 to +1032
let workspace_root_cell: LazyCell<PathBuf> = LazyCell::new();

for (p, base) in nested.iter() {
let p = if let Some(base) = base {
let workspace_root = || {
workspace_root_cell
.try_borrow_with(|| {
find_workspace_root(&manifest_path, gctx)?
.ok_or_else(|| anyhow!("failed to find a workspace root"))
})
.map(|p| p.as_path())
};
// Pass in `None` for the `cargo-features` not to skip verification: when the
// package is loaded as a dependency, then it will be checked.
match lookup_path_base(base, gctx, &workspace_root, None) {
Ok(base) => Cow::Owned(base.join(p)),
Err(err) => {
errors.push(err);
continue;
}
}
} else {
Cow::Borrowed(p)
};
let path = paths::normalize_path(&path.join(p.as_path()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed? Aren't we operating on a normalized package and don't we resolve path-bases during normaization?

For either of us to verify that is one of the reasons I recommend splitting commits into

  • Test showing the existing behavior
  • Fix/feature with test updates to show how the behavior changed

@dpaoliello
Copy link
Contributor Author

Looks like this change isn't needed: #2930 really confused me.
Sorry for the noise!

@dpaoliello dpaoliello closed this Aug 19, 2024
@dpaoliello dpaoliello deleted the nested branch August 19, 2024 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-documenting-cargo-itself Area: Cargo's documentation A-manifest Area: Cargo.toml issues S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants