CI: Consider using uv's lock file instead of requirement files

[danielhollas]

Is your feature request related to a problem? Please describe

Right now to keep the CI stable we pin dependencies to exact versions in requirements files in requirements.txt. One of the issues with this that we need to have one file per Python version, and we need to have a special workflow to generate these files and keep them in sync with pyproject.toml, which makes adding / modifying dependencies cumbersome.

Describe the solution you'd like

uv manager offers support for multiplatform lock files, which I believe would be a much better solution.

I'd like to explore this as part of coding week.

Describe alternatives you've considered

The disadvantage here is that we'd require uv tool. But we already use it in CI so I don't think that's a huge problem.

We could also use other tools for this (PDM, Poetry) but uv is generally much faster and is garnering a lot of momentum in the Python ecosystem.

Additional context

There is currently a new PEP in preparation (PEP-751)for official lock file specification so we could wait until that is finished. But it is not clear how long that will take, see
https://discuss.python.org/t/pep-751-now-with-graphs/69721/103

[unkcpz]

Interesting! But it looks like a very new PEP and I guess it can takes long time for discussion.
The PEP looks a bit vague, I will have a look at discourse discussion.

Is the uv tool only need from us or also required by user to install it? If it become the mandatory to use uv to install the package I think it will become a limitation for aiida.
What I really think is python community need provide the only standard package and project manager, and keep on optimize it. It is too wild now 😭

[danielhollas]

No, the lockfile is strictly for development only.
Developers can still choose if they want install with pip (pip install -e .) or use uv and the lockfile. Although when updating dependencies, uv would have to be used. It's certainly up for discussion if we'd go with this. Happy to hear about people's workflows next week!

[unkcpz]

or use uv and the lockfile.

I think the goal was to also provide to users to install aiida-core with lockfile to stay away from unexpected issues from dependencies. Are we plan on that?
From my experience with Julia and Rust, I'd really enjoy to have a lockfile to reproduce the same environment.

[danielhollas]

That's a non-goal. :-) aiida-core is a library which is typically installed with other packages, so we can't really provide a lock file for users. (Not to mention that this is not supported by installers). The user is responsible for locking their own environment appropriately.

[danielhollas]

Note that rust packaging is quite different since rust, unlike python, allows for the same package to have different versions in a single build. But even in rust, when you use cargo in a project, cargo runs its own resolution to generate a lock file, it doesn't use the lock files from the dependencies.