Attempt to fix BB julia_version issue #2942

[barche]

This basically skips an stdlib in deps_graph if it is no longer an stdlib in the current Julia version (and therefore doesn't exist). I'm not entirely sure I know what I'm doing :)

[KristofferC]

How do we get the dependencies for that stdlib then, to put in the manifest?

[barche]

How do we get the dependencies for that stdlib then, to put in the manifest?

Good point, I moved the check to the if condition, so now it should continue on the else branch for ex-stdlibs and attempt to get these from the registry. For the libjulia_jll example in issue #2942 the outcome is the same, since the dependency of libjulia_jll on LibOSXUnwind_jll is only there in older (and thus "yanked" as far as I understand) versions of libjulia_jll.

[barche]

I don't know if the test failures here are my doing...

[IanButterworth]

I believe the MacOS failure is a flaky test. The other is fixed on master. You just need to rebase/update.

Is there a test that could be added?

[fingolfin]

Regarding tests: the one package this affects right now is LibOSXUnwind_jll. So perhaps a test involving that could be added... perhaps trying to do "something" with libjulia_jll ? Sorry, I don't know enough about the test rig for Pkg.jl to make really sensible suggestions, just random ideas...

[barche]

Rebased and added test, which passes locally 🤞

[fingolfin]

@IanButterworth any chance to get this merged in a way that it gets into Julia 1.8? Or is that too late already?

[IanButterworth]

Sorry, I've tried to understand this, but I can't. I think the logic here might need comments in the code.
I requested @staticfloat's review who might be best placed to review?

It seems a reasonable thing to get into 1.8 IMO

[barche]

The change (line 431 in Operations.jl) checks if the dependency is still an stdlib in the current running version of Julia, since uuid_is_stdlib is true if the dependency was an stdlib in the targeted (older) version of Julia. If it is no longer and stdlib, it needs to be obtained from the repositories, which is done in the else branch. At least that is how I understand it ;)

[IanButterworth]

Sound like a great comment to add.

Also, might "download unavailable stdlibs during julia_version resolve" be a descriptive name for this PR?

[staticfloat]

I'm confused, as far as I can tell, this branch should already be false:

julia> using Pkg
       uuid = Base.UUID("a83860b7-747b-57cf-bf1f-3e79990d037f")
       stdlibs_for_julia_version = Pkg.Types.get_last_stdlibs(v"1.6.0")
       haskey(stdlibs_for_julia_version, uuid)
true

julia> stdlibs_for_julia_version = Pkg.Types.get_last_stdlibs(v"1.7.0")
       haskey(stdlibs_for_julia_version, uuid)
false

julia> Pkg.Types.is_unregistered_stdlib(uuid)
false

[barche]

But the error occurs when doing:

Pkg.add(Pkg.Types.Context(; julia_version = v"1.6"), [PackageSpec(; name = "libjulia_jll")])

So uuid_is_stdlib is true, and it tries to get the project file from an stdlib that is no longer there and crashes.

[IanButterworth]

Suggested change

	if (is_unregistered_stdlib(uuid) || uuid_is_stdlib) && haskey(stdlibs(), uuid)
	if (is_unregistered_stdlib(uuid) || uuid_is_stdlib) && haskey(stdlibs(), uuid)
	# if the dependency is still a stdlib in the current running version of Julia it doesn't need to be downloaded

@barche does this seem correct?

Although, I don't understand how the correct version is installed for the target julia version if it's different to the one provided by the host Julia

[barche]

Yes, I think this is a correct summary. It's true that there could be edge cases where this gets an incompatible version of an stdlib, but only in the case where a package is requested for a julia version that is different from the running version. I don't know how to deal with that and I'd propose to tackle that problem when and if it ever occurs.

[IanButterworth]

@staticfloat it'd be good to get your take on this part too.

Although, I don't understand how the correct version is installed for the target julia version if it's different to the one provided by the host Julia