Revise should "follow" files that go from add->dev

[timholy]

Switching to dev in a session doesn't work---it still tracks the packages added add.

[quinnj]

Ran into this today; so +1 from me!

[timholy]

Yeah, it's pretty annoying. Thanks to some pointers from @KristofferC and @fredrikekre I think I know how to fix it: monitor the main environment Manifest.toml for changes, and when Revise gets notified about an update it should parse the file to see where all the packages point to now. It seems likely that doing this correctly will involve subtleties (e.g., what if the user activates a different environment or creates a new environment?) that will necessitate a dive into .toml file parsing and Pkg internals and presumably lots of conversations with people who know Pkg far better than me. Consequently, it hasn't become quite annoying enough to push me over the edge and prioritize it just yet 😅.

I consider this issue to be two-thirds bug and one-third feature request. For me, the main priority now is 100% about bugs, particularly exploiting the shiny new logging system to relentlessly exterminate every last problem in Revise's diff-detection-and-eval-changes core (within the scope of Revise's and Julia's capabilities). The logger was just finished this morning (#166) and the payoff is pretty good already: #167, #169. So keep those bug reports flowing.

[GunnarFarneback]

I'm interested in a somewhat similar use case. I have a package with multiple branches, and each branch is checked out in a separate directory. I want to be able to switch between these branches in a running Julia session, by activating the corresponding directory. Is this at all reasonable for Revise, and if it is, how much would it overlap with the changes needed to solve this issue?

(To complicate matters this is to be run in an embedded setting but on the other hand it would be perfectly fine to have to call some function explicitly to inform about the switch.)

[timholy]

Glad you mentioned this now. Seems like it should be possible to have a "remap to a different directory" function and add the rest as layers on top of it, but let's see what happens when I get to this. I don't plan on tackling this until JuliaLang/Pkg.jl#785 gets merged.

[KristofferC]

Could you elaborate on how the mode of the files of an added package influence Revise here? I am probably missing something obvious.

[timholy]

If Revise can be assured that there will be no changes to an added package, it significantly reduces the resources needed: fewer inotify watches needed and no pre-parsing needed even for packages that disable precompilation. (To compute the diff you need both the "old" and "new" versions of a package. For precompiled packages, we have a cache of the source-text in the *.ji files to be used as "old" and the source file itself for the new; for packages that disable precompilation, you only have the source file and thus have to defensively parse them as soon as you import them in case they change in the future.)

Reducing the resources is pretty tightly integrated with me just sitting down and figuring out the whole logic of detecting the add/dev status of a package and implementing the "following" transition; for reasons of efficiency I'd rather just figure all that stuff out at once rather than have to come back to it long after I've forgotten how all that code works.

[timholy]

@GunnarFarneback, you should be able to say dev /path/to/MyPkg_version1/ and then dev /path/to/MyPkg_version2/ and it should follow. If it doesn't work, please file an issue.