[Feature] Mirror the binaries from the Yarn website

[arcanis]

Describe the user story

Our binaries are built for every release and stored within the repository. As a result, the binary from any release can be accessed through the following url:

https://raw.githubusercontent.com/yarnpkg/berry/%40yarnpkg/cli/{}/packages/yarnpkg-cli/bin/yarn.js (replacing {} by the version number for the CLI one wish to obtain)

Then the yarn set version command downloads the files from this url. It works well enough, but it has a few downside:

• GitHub is sometimes down
• GitHub is blocked in some countries (China, Is there a mirror we chinese could use? #982)
• If we were to move repo someday, it would be a mess

Describe the solution you'd like

I'd like to mirror those URLs behind Cloudflare and expose them through yarnpkg.com (so that bin.yarnpkg.com/2.0.0/packages/yarnpkg-cli/bin/yarn.js would be a cached alias for the url aforementioned). @Daniel15 do you know if this is something we can do?

Describe the drawbacks of your solution

The old url will still exist, so there is no backward incompatibility.

[arcanis]

I've setup a Cloudflare worker that mirrors our repository with a 7-days TTL:
https://repo.yarnpkg.com/2.3.2/packages/yarnpkg-cli/bin/yarn.js

[Daniel15]

Interesting, I didn't think of using Cloudflare workers, but glad to know that it works. Is that free to use?

[ylemkimon]

Note to reliably serve in China, an ICP license from the Chinese government is required (Cloudflare, for instance). Another way would be to use CDN such as jsDelivr which has a multi-CDN network and the ICP license. jsDelivr seems to provide custom endpoints, too.

[arcanis]

Is that free to use?

Yep, up to a threshold. What's interesting is that it seems to be composable with the caching layer, so it's unlikely we'd hit the threshold! Only thing missing is to implement /latest, which will require to handle the cache invalidation (and to figure out a way to retrieve the latest).

[v1nc3n4]

Hi all,
Apart from choosing the right CDN/gateway/proxy Yarn would rely on to download distributions, what if the end-user could tweak this setting and, for instance, use the distribution server of his choice? A problem with the way distributions are downloaded is that the process is not customizable (yarn set version <version> is a kind of black box). Am I mistaking?
Regards

[ZeroAurora]

I'd recommend jsDelivr. (https://cdn.jsdelivr.net/gh/yarnpkg/berry@latest/packages/yarnpkg-cli/bin/yarn.js , "latest" can be any tag)
Besides, what should we do to replace the download URL?
(Please ignore my comment related to yarnUpgradePath (deleted).)

[luislezcair]

It will be great to have yarn2 on npm since as mentioned in #3001 many companies have their own internal indexes and also strict policies on what can be installed on production servers and installing from github (or other non-standard location) is nearly illegal

[arcanis]

New versions will use the Yarn website as source for the binaries (specifically, repo.yarnpkg.com).

[enyineer]

@arcanis I would second comments from @v1nc3n4 and @luislezcair

I have a need to mirror repo.yarnpkg.com to a corporate Mirror (in my case Nexus Repository) so that it can be installed. I haven't found any Config or Environment Variable to tell Yarn where to get it from. Is this hardcoded without any chance to configure? If so this would be a big reason to migrate away from Yarn.

For this to work

berry/packages/plugin-essentials/sources/commands/set/version.ts

Line 115 in 8504172

bundleRef = getRef(`https://repo.yarnpkg.com/{}/packages/yarnpkg-cli/bin/yarn.js`, await resolveTag(configuration, `stable`));

has to be configurable.

If it could be installed via npmjs would be even better as most Mirror-Repository Softwares have native support for such mirrors.

Proxies are not an option.

Thanks in advance.