Skip to content

Commit

Permalink
Rewrote motiviation
Browse files Browse the repository at this point in the history
  • Loading branch information
dpaoliello committed Dec 11, 2023
1 parent cb65e59 commit 9615547
Showing 1 changed file with 81 additions and 36 deletions.
117 changes: 81 additions & 36 deletions text/3529-cargo-path-bases.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,51 +12,96 @@ turn enable base-relative `path` dependencies.
# Motivation
[motivation]: #motivation

While developing locally, users may wish to specify many `path`
dependencies that all live in the same local directory. If that local
directory is not a short distance from the `Cargo.toml`, this can get
unwieldy. They may end up with a `Cargo.toml` that contains
As a project grows in size, it becomes necessary to split it into smaller
sub-projects, architected into layers with well-defined boundaries.

One way to enforce these boundaries is to use different Git repos (aka
"multi-repo"). Cargo has good support for multi-repo projects using either `git`
dependencies, or developers can use private registries if they want to
explicitly publish code or need to preprocess their sub-projects (e.g.,
generating code) before they can be consumed.

If all of the code is kept in a single Git repo (aka "mono-repo"), then these
boundaries must be enforced a different way: either leveraging tooling during
the build to check layering, or requiring that sub-projects explicitly publish
and consume from some intermediate directory. Cargo has poor support for
mono-repos: the only viable mechanism is `path` dependencies, but these require
relative paths (which makes refactoring and moving sub-projects very difficult)
and don't work at all if the mono-repo requires publishing and consuming from an
intermediate directory (as this may very per host, or per target being built).

This RFC proposes a mechanism to specify `base` directories in `Config.toml`
files which can be used to prepend `path` dependencies. This allows mono-repos
to specify dependencies relative to their root directory, which allows the
consuming project to be moved freely (no relative paths to update) and a simple
find-and-replace to handle a producing project being moved. Additionally, a
host-specific or target-specific intermediate directory may be specified as a
`base`, allowing code to be consumed from there using `path` dependencies.

### Example

If we had a sub-project that depends on three others:

* `foo` which is in a different layer of the mono-repo.
* `bar_with_generated` that must be consumed from an intermediate directory
because it contains target-specific generated code.
* `baz` which is in the current layer.

We may have a `Cargo.toml` snippet that looks like this:

```toml
foo = { path = "/home/jon/dev/rust/foo" }
bar = { path = "/home/jon/dev/rust/bar" }
baz = { path = "/home/jon/dev/rust/ws/baz" }
[dependencies]
foo = { path = "../../../other_layer/foo" }
bar_with_generated = { path = "../../../../intermediates/x86_64/Debug/third_layer/bar_with_generated" }
baz = { path = "../baz" }
```

This is not only frustrating to type out, but also requires many changes
should any component of the path change. For example, if `foo`, `bar`,
and `ws/baz` were to move under a sub-directory of `libs`, all the paths
would have to be updated. If they are used in more than one local
project, each project would have to be updated.
This has many issues:

* Moving the current sub-project may require changing all of these relative
paths.
* `bar_with_generated` will only work if we're building x86_64 Debug.
* `bar_with_generated` assumes that the `intermediates` directory is a sibling
to our source directory, and not somewhere else completely (e.g., a different
drive for performance reasons).
* Moving `foo` or `baz` requires searching the code for each possible relative
path (e.g., `../../../other_layer/foo` and `../foo`) and may be error prone if
there is some other sub-project in directory with the same name.

As related issue arises in contexts where an external build system may
make certain dependencies available through vendoring. Such a build
system might place vendored packages under some complex path under a
build-root, like
Instead, if we could specify these `base` directories in a `Config.toml` (which
may be generated by an external build system which in turn invokes Cargo):

```toml
[base_path]
sources = "/home/user/dev/src"
intermediates = "/home/user/dev/intermediates/x86_64/Debug"
```
/home/user/workplace/feature-1/build/first-party-package/first-party-package-1.0/x86_64/dev/build/private/rust-vendored/

Then the `Cargo.toml` can use those `base` directories and avoid relative paths:

```toml
[dependencies]
foo = { path = "other_layer/foo", base = "sources" }
bar_with_generated = { path = "third_layer/bar_with_generated", base = "intermediates" }
baz = { path = "this_layer/baz", base = "sources" }
```

If a developer wishes to use such an auto-vendored dependency, a
contract must be established with the build system about exactly where
vendred dependencies will end up. And since that path may not be near
the project's `Cargo.toml`, the user's `Cargo.toml` may end up with
either an absolute path or a long relative path, both of which may not
work on other hosts, and thus cannot be checked in (or must be
overwritten in-place by the build system).

The proposed mechanism aims to simplify both of these use-cases by
introducing named "base" paths in the Cargo configuration
(`.cargo/config.toml`). Path dependencies can then be given relative to
those base path names, which can be set either by a local developer in
their user-wide configuration (`~/.cargo/config.toml`), or by an
external build system in a project-wide configuration file.

This effectively makes a "group" of path dependencies available at some
undisclosed location to `Cargo.toml`, which then only has to know the
layout to path dependencies _within_ that directory, and not the path
_to_ that directory.
Which resolves the issues we previously had:

* The current project can be moved without modifying the `Cargo.toml` at all.
* `bar_with_generated` works for all targets (assuming the `Config.toml` is
generated).
* The `intermediates` directory can be placed anywhere.
* Moving `foo` or `baz` only requires searching for the canonical form relative
to the `base` directory.

## Other uses

The ability to use `base` directories for `path` dependencies is convenient for
developers who are using a large number of `path` dependencies within the same
root directory. Instead of repeating the same path fragment many times in their
`Cargo.toml`, they can instead specify it once in a `Config.toml` as a `base`
directory, then use that `base` directory in each of their `path` dependencies.

# Guide-level explanation
[guide-level-explanation]: #guide-level-explanation
Expand Down

0 comments on commit 9615547

Please sign in to comment.