Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use a 'minor' version field (revision) in the lockfile #11500

Merged
merged 2 commits into from
Feb 14, 2025
Merged

Conversation

charliermarsh
Copy link
Member

Summary

This is an alternative to the approach we took in #11063 whereby we always included provides-extra and requires-dist, since we needed some way to differentiate between "no extras" and "lockfile was generated by a uv version that didn't include extras".

Instead, this PR adds a minor version (called a "revision") to the lockfile that we can use to indicate support for this feature. While lockfile version bumps are backwards-incompatible, older uv versions can read lockfiles with a later revision -- they just won't understand all the data.

In a future major version bump, we could simplify things and change the schema to use a (major, minor) format instead of these two separate fields. But this is the only way to do it that's backwards-compatible with existing uv versions.

@zanieb
Copy link
Member

zanieb commented Feb 14, 2025

I'm into it 👍 I'll let @Gankra give the approving review.

@charliermarsh
Copy link
Member Author

charliermarsh commented Feb 14, 2025

I think I made a mistake by suggesting that we make provides-extra non-optional (in order to power feature detection) -- I'm sorry about that. I don't think what I have here is perfect, but I do think it's less disruptive (e.g., we don't have to re-make these fields hidden-when-empty when we bump the version in the future) and closer to the principles we use throughout the format (favoring conciseness, etc.).

@charliermarsh charliermarsh marked this pull request as ready for review February 14, 2025 02:41
@charliermarsh charliermarsh changed the title Use a 'minor' version field in the lockfile Use a 'minor' version field (revision)in the lockfile Feb 14, 2025
@charliermarsh charliermarsh changed the title Use a 'minor' version field (revision)in the lockfile Use a 'minor' version field (revision) in the lockfile Feb 14, 2025
@charliermarsh charliermarsh added compatibility Compatibility with a specification or another tool resolver Related to the package resolver and removed compatibility Compatibility with a specification or another tool labels Feb 14, 2025
Copy link
Contributor

@Gankra Gankra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One potential issue but yeah this is just better! I was surprised you were happy with making it so much noisier, I wish I knew that adding the minor version would be this easy!

Comment on lines 262 to 266
// `provides-extra` was added in Revision 1.
if lock.revision() == 0 {
return Ok(());
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to check for the current lock.version() too, since 3.0 > 2.1 (unless you want 2.100 => 3.101 to be a thing..? that seems weird.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in 6571d2f

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be if (lock.version(), lock.revision()) < (1, 1) rather than hard-coding 1?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's actually impossible to get here since we reject lockfiles that don't match the major, but you're right that we should change this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree on both points

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -5627,7 +5599,7 @@ fn add_remove_script_lock() -> Result<()> {
wheels = [
{ url = "https://files.pythonhosted.org/packages/a2/73/a68704750a7679d0b6d3ad7aa8d4da8e14e151ae82e6fee774e6e0d05ec8/urllib3-2.2.1-py3-none-any.whl", hash = "sha256:450b20ec296a467077128bff42b73080516e71b56ff59a60a02bef2232c4fa9d", size = 121067 },
]
"#
"###
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lmao i see our copies of cargo-insta are fighting (I'm on cargo-insta 1.42.0)

@charliermarsh
Copy link
Member Author

One potential issue but yeah this is just better! I was surprised you were happy with making it so much noisier, I wish I knew that adding the minor version would be this easy!

Gah, yeah, I'm sorry that I went back and forth on it and made you do a bunch of extra work. The more I thought about it, the more it felt like the wrong tradeoff. I think the "hard" thing about adding the minor version was just figuring out what it should be called and what the schema should look like (since we need two version fields)... What we have here is okay, not perfect, but it's does feel like a better tradeoff.

Copy link
Member Author

@charliermarsh charliermarsh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Zanie!

@zanieb zanieb merged commit 29bdf1d into main Feb 14, 2025
73 checks passed
@zanieb zanieb deleted the charlie/rev branch February 14, 2025 16:17
zanieb added a commit that referenced this pull request Feb 14, 2025
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Feb 17, 2025
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [astral-sh/uv](https://github.com/astral-sh/uv) | minor | `0.5.31` -> `0.6.0` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>astral-sh/uv (astral-sh/uv)</summary>

### [`v0.6.0`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#060)

[Compare Source](astral-sh/uv@0.5.31...0.6.0)

There have been 31 releases and 1135 pull requests since [0.5.0](https://github.com/astral-sh/uv/releases/tag/0.5.0), our last release with breaking changes. As before, we've accumulated various changes that improve correctness and user experience, but could break some workflows. This release contains those changes; many have been marked as breaking out of an abundance of caution. We expect most users to be able to upgrade without making changes.

##### Breaking changes

-   **Create `main.py` instead of `hello.py` in `uv init`** ([#&#8203;10369](astral-sh/uv#10369))

    Previously, `uv init` created a `hello.py` sample file. Now, `uv init` will create `main.py` instead — which aligns with expectations from user feedback. The `--bare` option can be used to avoid creating the file altogether.

-   **Respect `UV_PYTHON` in `uv python install`** ([#&#8203;11487](astral-sh/uv#11487))

    Previously, `uv python install` did not read this environment variable; now it does. We believe this matches user expectations, however, this will take priority over `.python-version` files which could be considered breaking.

-   **Set `UV` to the uv executable path** ([#&#8203;11326](astral-sh/uv#11326))

    When uv spawns a subprocess, it will now have the `UV` environment variable set to the `uv` binary path. This change is breaking if you are setting the `UV` environment variable yourself, as we will overwrite its value.

    Additionally, this change requires marking the uv Rust entrypoint (`uv::main`) as `unsafe` to avoid unsoundness — this is only relevant if you are invoking uv using Rust. See the [Rust documentation](https://doc.rust-lang.org/std/env/fn.set_var.html#safety) for details about the safety of updating a process' environment.

-   **Error on non-existent extras, e.g., in `uv sync`** ([#&#8203;11426](astral-sh/uv#11426))

    Previously, uv would silently ignore non-existent extras requested on the command-line (e.g., via `uv sync --extra foo`). This is *generally* correct behavior when resolving requests for package extras, because an extra may be present on one compatible version of a package but not another. However, this flexibility doesn't need to apply to the local project and it's less surprising to error here.

-   **Error on missing dependency groups when `--frozen` is provided** ([#&#8203;11499](astral-sh/uv#11499))

    Previously, uv would not validate that the requested dependency groups were present in the lockfile when the `--frozen` flag was used. Now, an error will be raised if a requested dependency group is not present.

-   **Change `-p` to a `--python` alias in `uv pip compile`** ([#&#8203;11486](astral-sh/uv#11486))

    In `uv pip compile`, `-p` was an alias for `--python-version` while everywhere else in uv's interface it is an alias for `--python`. Additionally, `uv pip compile` did not respect the `UV_PYTHON` environment variable. Now, the semantics of this flag have been updated for parity with the rest of the CLI.

    However, `--python-version` is unique: if we cannot find an interpreter with the given version, we will not fail. Instead, we'll use an alternative interpreter and override its version tags with the requested version during package resolution. This behavior is retained here for backwards compatibility, `--python <version>` / `-p <version>` will not fail if the version cannot be found. However, if a specific interpreter is requested, e.g., with `--python <path>` or `--python pypy`, and cannot be found — uv will exit with an error.

    The breaking changes here are that `UV_PYTHON` is respected and `--python <version>` will no longer fail if the version cannot be found.

-   **Bump `alpine` default tag to 3.21 for derived Docker images** ([#&#8203;11157](astral-sh/uv#11157))

    Alpine 3.21 was released in Dec 2024 and is used in the official Alpine-based Python images. Our `uv:python3.x-alpine` images have been using 3.21 since uv v0.5.8. However, now the the `uv:alpine` image will use 3.21 instead of 3.20 and `uv:alpine3.20` will no longer be updated.

-   **Use files instead of junctions on Windows** ([#&#8203;11269](astral-sh/uv#11269))

    Previously, we used junctions for atomic replacement of cache entries on Windows. Now, we use a file with a pointer to the cache entry instead. This resolves various edge-case behaviors with junctions. These files are only intended to be consumed by uv and the cache version has been bumped. We do not think this change will affect workflows.

##### Stabilizations

-   **`uv publish` is no longer in preview** ([#&#8203;11032](astral-sh/uv#11032))

    This does not come with any behavior changes. You will no longer see an experimental warning when using `uv publish`. See the linked pull request for a report on the stabilization.

##### Enhancements

-   Support `--active` for PEP 723 script environments ([#&#8203;11433](astral-sh/uv#11433))
-   Add `revision` to the lockfile to allow backwards-compatible metadata changes ([#&#8203;11500](astral-sh/uv#11500))

##### Bug fixes

-   Avoid reading metadata from `.egg-info` files ([#&#8203;11395](astral-sh/uv#11395))
-   Include archive bucket version in archive pointers ([#&#8203;11306](astral-sh/uv#11306))
-   Omit lockfile version when additional fields are dynamic ([#&#8203;11468](astral-sh/uv#11468))
-   Respect executable name in `uvx --from tool@latest` ([#&#8203;11465](astral-sh/uv#11465))

##### Documentation

-   The `CHANGELOG.md` is now split into separate files for each "major" version to fix rendering ([#&#8203;11510](astral-sh/uv#11510))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xNzAuMSIsInVwZGF0ZWRJblZlciI6IjM5LjE3MC4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
resolver Related to the package resolver
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants