Ecosystem stability issues

[JJ]

Today, someone requested removing a module from the p6c ecosystem since it has been moved to CPAN. All's good and well, but I've realized there's no policy regarding total removal of some modules from the ecosystem. We have a small ecosystem now, but imagine when it's an order of magnitude bigger and some removes a module that's far down the river and it becomes unavailable. Of course, it's her privilege to do so as the owner, but then it might cause bigger issues, in Blin and other places, with all its dependencies.
There are possibly many solutions to this, but the one I can think of offhand is to simply maintain a mirror of all repositories that hold modules with a river score higher than something. Also, keep monitoring their existence (I don't know if Blin does this) so that they are not compromised or deleted.

[nxadm]

I am not proposing doing the same as Go does, but it may be a good example on how to deal with the problem. Summarize, a proxy checksums and caches all releases and never deletes them, even when the upstream repo disappears. This feature (enabled by default), combined with the "semver/minimal version selection" new module dependency model, assures that the Go ecosystem stays stable whatever happens upstream. More info can be found here: https://proxy.golang.org and https://github.com/golang/go/wiki/Modules#version-selection.

[AlexDaniel]

This happened once in 2017, when a dev removed several of their modules which other modules were depending on. I simply restored all removed modules by adding them to community modules. Luckily, these modules were not CPAN modules, meaning that anything that used zef to install them actually saved the whole git repository. So I didn't just restore last versions, I restored whole repos (minus tickets, wikis and other less relevant stuff).

I'd like to find a tool that can save github repos (not just source, but with bug trackers and wikis), and that can restore them when needed. Then we should keep a mirror of every module. Then we can be more strict about not having a LICENSE file. After that we can even add automation to restore removed modules automatically.

[JJ]

El dom., 15 sept. 2019 a las 19:14, Aleks-Daniel Jakimenko-Aleksejev (< [email protected]>) escribió:

This happened once in 2017, when a dev removed several of their modules which other modules were depending on. I simply restored all removed modules by adding them to community modules <https://github.com/perl6-community-modules/>. Luckily, these modules were *not* CPAN modules, meaning that anything that used zef to install them actually saved the whole git repository. So I didn't just restore last versions, I restored whole repos (minus tickets, wikis and other less relevant stuff). I'd like to find a tool that can save github repos (not just source, but with bug trackers and wikis), and that can restore them when needed. Then we should keep a mirror of every module. Then we can be more strict about not having a LICENSE file. After that we can even add automation to restore removed modules automatically.

We can probably program a GitHub action to do it for us. At least, for those in the ecosystem.

[vendethiel]

I think @chloekek has a system to download all github sources, and I know of https://github.com/nabijaczleweli/dumplingh to download gitjub stuff (thanks nab!), maybe that could be used

[tony-o]

This is being worked on as a separate project

[AlexDaniel]

@tony-o who's working on it and where? Any ETA?

[tony-o]

@AlexDaniel I'm working on it, I'm testing it with zef at the moment :-)

[AlexDaniel]

@tony-o ok, sounds awesome!

[lizmat]

Closing in light of #316