Add custom headers to tarball fetcher

[Glavin001]

Summary

My team would like to download tarballs attached to GitLab Releases.
The projects are authenticated, therefore we need to add PRIVATE-TOKEN header to the request for the tarball. See https://docs.gitlab.com/ee/api/README.html#personal-access-tokens

I propose the following configuration addition:

~/.npmrc:

//custom.gitlab.host/:_header:PRIVATE-TOKEN=REDACTED

where:

• //custom.gitlab.host/ is the host which matches the tarball request.
• :_header:<HEADER-NAME> indicates a custom header should be added, with PRIVATE-TOKEN being the HEADER-NAME in this instance. Multiple custom headers could be added in this way.

Then we can run:

❯ yarn-local add --dev --verbose https://custom.gitlab.host/OWNER/REPO/uploads/REDACTED/PKG-NAME-v1.0.0.tgz

And the request will be authenticated.

Once I have approval on the above format then I will update the CHANGELOG.md 😃 .

Test plan

Once I have approval on the above format then I will add tests 😃 .

---

Update 1:

Now ~/.yarnrc looks like:

"//custom.gitlab.host/:_header:PRIVATE-TOKEN" "REDACTED"

after running

yarn config set "//custom.gitlab.host/:_header:PRIVATE-TOKEN" "REDACTED"

and ~/.npmrc is not used. Working great! 🎉

---

Update 2:
See #6756 (comment)

I created a release and uploaded a tarball generated from yarn pack to https://github.com/Glavin001/yarn/releases/tag/1.13.0-config-tarball-fetcher-headers-1

You can install this custom Yarn with:

npm install -g https://github.com/Glavin001/yarn/releases/download/1.13.0-config-tarball-fetcher-headers-1/yarn-v1.13.0-0.tgz

[Glavin001]

This is copied from

yarn/src/registries/npm-registry.js

Lines 77 to 88 in b8a565e

	type UrlParts = {
	host: string,
	path: string,
	};
	function urlParts(requestUrl: string): UrlParts {
	const normalizedUrl = normalizeUrl(requestUrl);
	const parsed = url.parse(normalizedUrl);
	const host = parsed.host || '';
	const path = parsed.path || '';
	return {host, path};
	}

I would like to put it in a more central location. I ask the Yarn team where would be the most appropriate destination? 😄

[arcanis]

Given that this affect the syntax of the .npmrc, is it also available on npm? Or would it be a Yarn-exclusive feature? In the later case I'm a bit concerned of risks of confusion 🤔

[Glavin001]

@arcanis : Good point and I agree. This feature is not available on npm - I reviewed https://docs.npmjs.com/misc/config#config-settings and hopefully did not miss a similar supported feature.

I'm currently retrieving the config values with

yarn/src/fetchers/tarball-fetcher.js

Lines 279 to 281 in ed76774

	requestHeaders(): {[string]: string} {
	const registry = this.config.registries[this.registry];
	const config = registry.config;

and it appeared to only load from .npmrc. I'm not yet sure how to load from .yarnrc config file.

Any advice on how I could load config from .yarnrc instead of .npmrc would be greatly appreciated. Thank you in advance!

[arcanis]

I think you should be able to access the yarnrc settings through this.config.registries.yarn (this.registry is typically "npm" - even though it goes through the Yarn domain name).

[Glavin001]

Thank you for the assistance, @arcanis !

I pushed a commit switching from npm to yarn config.

Now ~/.yarnrc looks like:

"//custom.gitlab.host/:_header:PRIVATE-TOKEN" "REDACTED"

after running

yarn config set "//custom.gitlab.host/:_header:PRIVATE-TOKEN" "REDACTED"

and ~/.npmrc is not used. Working great! 🎉

---

Once the above is all approved, I will add tests, push the changes, and update the Pull Request description to reflect these changes.

Thanks again for reviewing!

[Glavin001]

It looks like the Jest tests need a higher timeout:

Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.

They fail intermittently and without related code changes.

[arcanis]

Thanks! It'll be part of the next release 😃