bootstrap using uv

[dsp]

NOTE This is the first iteration on the PR. There is probably a bunch to clean up. Nevertheless I am happy to get early feedback and iterate.

This PR aims to remove pip-tools with uv. We do this by:

1. Create a download url generator for uv downloads.
2. Rework our generated downloads structure to encapsulate python and uv downloads.
3. Implement bootstrapping using UV using the download urls. We will always fetch the only version we have in our downloads.

At the moment we only include one version of uv in the rye binary and ensure this one is used for bootstrapping (multiple versions can co-exists in $RYE_HOME/uv)

Open questions:

1. Should we fall-back to pip-tools if uv downloads fail?
2. How to best test this? Should we make it optionally behind a feature flag --use-uv for a while?

[dsp]

The PR works with behavior.use-uv=true. Still have to make it work for behavior.use-uv=false, since we don't bootstrap pip-tools themselves at the moment.

TODO:

• Ensure pip-tools are installed so that sync with pip-tools works when bootstrapping with uv.
• Uninstall uv when doing self uninstall
• Update docs

[dsp]

Okay the pull request as is works with pip-tools. There is an open question regarding pip-tools handling. At the moment pip-tools bootstraps into $RYE_HOME/pip-tools since we need it to bootstrap $RYE_HOME/self. However, since we now bootstrap with uv, we could just add pip-tools to SELF_REQUIREMENTS and use that during sync when behavior.use-uv is false. However, while i think that's the right thing to do, it's probably best if we do this in a separate PR. Let uv bootstrapping get released and after a while change pip tools. At least have it in another PR, since we squash commits during merging, we ight want to have it separately in case we need to revert it.

[dsp]

I ran some basic local tests, both with uv bootstrapping:

1. Use uv and create a new project and sync: Works ✅
2. Use pip-tools and create a new project and sync: Works ✅

[mitsuhiko]

However, since we now bootstrap with uv, we could just add pip-tools to SELF_REQUIREMENTS and use that during sync when behavior.use-uv is false

The issue is that pip-tools needs a fresh venv matching the target python version :( Cannot be installed into the global venv.

[dsp]

However, since we now bootstrap with uv, we could just add pip-tools to SELF_REQUIREMENTS and use that during sync when behavior.use-uv is false

The issue is that pip-tools needs a fresh venv matching the target python version :( Cannot be installed into the global venv.

Right. Let's keep it then how it is. There is a good chance I've missed a piece to the puzzle, but with my basic testing the PR is good for review.

[mitsuhiko]

The uv==0.1.9 line in the self requirements at this point probably should be removed. Otherwise you end up with two uvs for no good reason.

[mitsuhiko]

Is there much of a benefit of having multiple uv versions and not just to retain only the latest?

[dsp]

The main reason here is that i can check for the version with one simple stat call and not have to execv uv. This allows me in later changes to make use of ensure_exists to obtain a Uv on any code path. It also opens the door for pinning uv if we ever need to (I don't see why, but it feels more future proof). I am happy to just nuke the uv directory before I download a new version which will keep it clean for now. If you insist i can remove the version.

[mitsuhiko]

Other than the few nits I think this looks good. Also makes for a quicker bootstrap process. I think a good followup would be to make the virtualenv quiet as it now spits out some quite confusing output during installation:

Downloaded [email protected]
Using Python 3.12.1 interpreter at C:\Users\armin\.rye\py\[email protected]\install\python.exe
Creating virtualenv at: C:\Users\armin\.rye\self
Activate with: C:\Users\armin\.rye\self\Scripts\activate
Resolved 1 package in 2ms
Installed 1 package in 95ms
 + pip==23.3.2
Resolved 35 packages in 24ms

Since that env really only exists internally I don't think it makes sense to reveal where it is and how it's being activated.

[dsp]

The uv==0.1.9 line in the self requirements at this point probably should be removed. Otherwise you end up with two uvs for no good reason.

Okay that makes sense. This will add a few more changes as we rely on uv being in .rye/self/bin. I'll follow up with adjustment to path we use. A later changset will make use of Uv everywhere where we currently check for Config::current().use_uv() and then construct path to the binary. This should be more sane and allows us to have a canoncial, encapsulated way to construct the binary path. It will also allow us to ensure on a type level that in certain contexts we always operate on a venv.

[dsp]

Okay changes made: I keep uv/[version]/uv for now but will clean old directories after we downloaded and extracted the new version. This feels to me a bit more future proof and allows nicely to inspect which uv version we are on.

self-venv creation is now using CommandOutput::Quiet to surpress any call to uv during the process.

[mitsuhiko]

Looks good!

[God-damnit-all]

This is the commit that made it so rye tools install generates a rye-venv.json file, correct?

I really do not like the fact that it now means that a properly configured tool now has to point to an absolute path and it needs to be edited should RYE_HOME ever be relocated. I tried setting use-uv = false in the config under [behavior] in hopes of getting the old way it worked back, but no luck.

[bluss]

@ImportTaste what about all other non-relocatable things in a python virtualenv, is there a solution for them?

Let's say I inspect for example ~/.rye/self/bin/pip3 and find that its shebang has an absolute path as well to the python in the venv, this is typically how virtual environments work unfortunately(!). But since it's a long standing issue, I could see there could be tools to "relocate" them.

In my experience, python virtual environments need to be recreated and can't be relocated unfortunately.

[God-damnit-all]

@ImportTaste what about all other non-relocatable things in a python virtualenv, is there a solution for them?

Let's say I inspect for example ~/.rye/self/bin/pip3 and find that its shebang has an absolute path as well to the python in the venv, this is typically how virtual environments work unfortunately(!). But since it's a long standing issue, I could see there could be tools to "relocate" them.

In my experience, python virtual environments need to be recreated and can't be relocated unfortunately.

I wasn't aware of that. I guess nevermind then.