Relative dependencies get lost after updating alr

[onox]

I just updated alr (from 625c906 to c63615d) and subsequently running alr build or alr update causes dependecies added with alr with --use to go "missing". The path and relative are removed from the lock file.

For example, running alr update in the orka crate in my orka repository shows:

warn: This workspace was created with a previous alr version. Internal data is going to be updated and, as a result, any existing pins will be unpinned and will need to be manually recreated.
                                                              
Changes to dependency solution:

   New solution is incomplete.
   +⚠ orka_egl        ^1.0.0 (new,missing)
   +⚠ orka_simd       ^1.0.0 (new,missing)
   +⚠ orka_transforms ^1.0.0 (new,missing)
   +⚠ orka_types      ^1.0.0 (new,missing)

Do you want to proceed?
[Y] Yes  [N] No  (default is No) no

I then need to (recursively) remove each dependency and then re-add with alr with --use.

This seems to involve quite a few steps to 'fix' each dependency. I'm not sure the lock files are the appropriate file for storing the relative paths. I should be able to just throw away a lock file and then run alr update to recreate it.

[mosteo]

Yes, that's to be expected with the current approach, in the sense that changes to the lockfile format are breaking in the master branch.

This file should stabilize so this ceases to be a problem, just like for the manifest. Of course that's not much consolation for you at this time. That said, some thoughts about we can do:

1. Stay the course, and make the pins more resilient to changes in format. (Now, any change to any part of how a solution is stored results in an invalidated lockfile. We could "rescue" the pins information so that it survives even if other parts of the lockfile change.) This way does not allow getting rid of the lockfile freely, as you mention (but why do that?)
2. Make the pins information explicitly part of the manifest. Then the lockfile would merely serve as the cache of the current solution. The advantage of this is that we only need to take care about compatibility for the manifest.

I see both ways working, and at this juncture it would be relatively easy to switch to 2., given that 1. carries its own complexity. Thoughts, @Fabien-Chouteau ?

[Fabien-Chouteau]

Not sure if I understand 1. completely, but it sounds risky. The "rescue" pins part can be a nightmare to support as the format evolve.

One issue with 2. is that we don't want alr to edit the manifest automatically (breaks the formatting).
Which means the alr pin would have to go. I don't mind it that munch, I see pinning as intermediate/advanced usage so it's ok not to have a command for that.

I had a look at Cargo's do: https://doc.rust-lang.org/cargo/reference/overriding-dependencies.html
As far as I understand, their "pinning" solution is called "Overriding Dependencies". It's a way in the manifest to override the already defined dependencies with: a path, a git repo, etc.

For us it could look like this:

[[depends-on]]
orka_types = "^1.0.0"
orka_simd = "^1.0.0"
orka_transforms = "^1.0.0"

[[pins]]
orka_types = {version = "=1.0.1"}
orka_simd = {path = "../orka_simd/"}
orka_transform = {git = "https://github.com/plop/plip", branch = "fix-plop"}

And I kinda like it.

[mosteo]

@Fabien-Chouteau , yes, exactly.

alr pin could remain to generate "frozen" manifests (we still may want this for reproducible builds). Down the road if there's nothing better to do (ha), we could modify the pins via command line just as we edit the manifest for alr with. But I agree that that is very low priority.