Package groups

[c42f]

A recent discussion on discourse sparked the idea that some concept of "curated environments" / "package groups" might be useful. That thread introduced Numerics.jl which is a grab bag of somewhat unrelated packages which were useful to the author. As a trivial package which re-exports other packages, Numerics.jl doesn't doesn't contain any code and I think versioning it would be a bit of a nightmare! But the underlying idea makes sense if viewed as a curated environment or group of packages.

In particular, I could see it being useful for Julia beginners where they'd be able to take a standard curated environment and have all the packages relevant to their domain installed and ready to use. Perhaps as part of a base environment on top of which sub-environments could be built in the style of #1233

For handling versions, it might be possible to take a strict mixin-style approach: mixing in the curated environment would add the latest versions of packages in it using the usual version resolution declared in those packages. That is, the mixin could come without any version information in its own right, but just as a convenient grouping.

Several linux package managers have this concept as package groups

Related: pkg> env repl-mode discussion in #621

[PetrKryslUCSD]

I think the environment concept is orthogonal to which packages the user actually uses once she activated the environment. See also https://discourse.julialang.org/t/ann-numerics-jl-the-extended-standard-library/40694/41

[StefanKarpinski]

#1856

[KristofferC]

Who decides what goes into a group? For the Linux case I guess the maintainers of the distro set up these groups but for a case like General which is contributed to by users, I feel like it can easily lead to a lot of "my favorite package bundles" with a bunch of close copies of groups.

[StefanKarpinski]

That’s why in my proposal metadata sources are external (everyone can have their known if they want to), and users create policies that compute an allow/ask/deny for each package version based on some function of metadata sources.

[PetrKryslUCSD]

I was simply pointing out that at present users take the presence of a package in the registry to mean *something* in terms of quality. I think relegating the responsibility of checking the "quality" to the user, who is now being asked to inspect the curation metadata, will require a certain change of culture.

…

On Tue, Jun 9, 2020, 11:33 AM Stefan Karpinski ***@***.***> wrote: That’s why in my proposal metadata sources are external (everyone can have their known if they want to), and users create policies that compute an allow/ask/deny for each package version based on some function of metadata sources. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://urldefense.com/v3/__https://github.com/JuliaLang/Pkg.jl/issues/1854*issuecomment-641495973__;Iw!!Mih3wA!Rat_9Q0SC_UU06yELsa8GpM3am53ejGuaiWW3UqR2KMQh37gIcjrcVjQPs8URws$>, or unsubscribe <https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ACLGGWBDW2OQA5QKI5OBVN3RVZ56TANCNFSM4NZBX3YA__;!!Mih3wA!Rat_9Q0SC_UU06yELsa8GpM3am53ejGuaiWW3UqR2KMQh37gIcjrcVjQzsEkkAQ$> .

[StefanKarpinski]

The user is not being asked to inspect curation metadata, that would be done automatically.

[PetrKryslUCSD]

Okay, that makes more sense now. Now, who is in control of the curation metadata?

…

On Tue, Jun 9, 2020, 12:43 PM Stefan Karpinski ***@***.***> wrote: The user is not being asked to inspect curation metadata, that would be done automatically. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://urldefense.com/v3/__https://github.com/JuliaLang/Pkg.jl/issues/1854*issuecomment-641530593__;Iw!!Mih3wA!UIQ1uAGgC6dL4kJ3MGX7VxpH6TrhafO4ZX2jwtGgQBedWQLXtfYaM-dwqHM_q_Y$>, or unsubscribe <https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ACLGGWH542QMA3Z6PXTQ45DRV2GG7ANCNFSM4NZBX3YA__;!!Mih3wA!UIQ1uAGgC6dL4kJ3MGX7VxpH6TrhafO4ZX2jwtGgQBedWQLXtfYaM-dwV0xEr1k$> .

[StefanKarpinski]

I don't want to seem churlish here, but have you read #1856? Anyone who wants to produce curation metadata could do so an people would choose which curation metadata and trust policies they're interested in using. It would make sense to ship a simple but reasonably safe default policy with the default Julia install.

[PetrKryslUCSD]

Well, I do apologize if I missed it, but yes, I read #1856, and this was left open. Who will vet the providers of curation data? Unless one can trust the provider of this information, the whole scheme is suspect. Right now I trust Julia registry implicitly, but if I am given a choice and I have to make this decision on the fly (so to speak), doesn't the need to trust in arbitrary provider of curation data leave me open to attack? (And, yes, I realize that someone might register a malignant package right now, but the point is, if I have to trust too many sources based on limited information, I am in trouble.)

[c42f]

Who decides what goes into a group?

Anyone — a user could want their own locally named groups for convenience, or we could have a way to serve these (in which case #1856 comes into play).

I don't mean to open a can of worms here regarding how one trusts a given curated environment and the exact mechanism by which that trust is conveyed. That seems a somewhat separate issue from whether having a group abstraction is a good thing in the first place.

With that in mind, I'll rename this issue to "package groups" as one can have a local version of a group abstraction without any trust issues at all (unless you don't trust yourself :-) )