Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add rfc about registry url in lock file #84

Merged
merged 4 commits into from
Nov 13, 2017
Merged

add rfc about registry url in lock file #84

Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
459e860
add rfc about registry url in lock file
Sep 21, 2017
02aad79
Update 0000-registry-url-in-lock-file.md
vernak2539 Sep 21, 2017
88beb71
update registry url in lock file 'detailed design'
vernak2539 Oct 9, 2017
9639772
update download from different registry rfc
vernak2539 Oct 25, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions accepted/0000-registry-url-in-lock-file.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
- Start Date: 2017-09-21
- RFC PR:
- Yarn Issue:

# Summary

The yarn.lock file includes information about the registry that it was generated against. It would
be nice if there was no registry information associated with the lock file

# Motivation

We currently have two internal registries/mirrors. We generate the lock file against one we have in
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add more details please.
Short examples how the yarn.lock files are different would be useful

our development environment. When our code is deployed to our build/testing environment, we
no longer have access to the original registry that was used to generate the lock file. This causes
our download to error as yarn cannot retrieve the packages.

The expected output is for yarn to recognise the registry URL supplied (via either cli flag or
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please provide the name of the flag

`.yarnrc` or `.npmrc`) and download from that if the registry used in the lock file is not available.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to do it in .npmrc, just mention .yarnrc.



# Detailed design

Let me know if this isn't enough, first time writing an RFC.

As a dev that generated their lock file against REGISTRY_A
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we need a more technical explanation with examples of yarn.lock and config parameter names and concrete URL examples.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bestander updated the PR. Let me know if more is needed

I would like to pass a cli flag designating REGISTRY_B
And I would like yarn to download my dependencies from REGISTRY_B

As a dev that generated their lock file against REGISTRY_A
I would like to designate an alternate registry URL in my `.yarnrc` or `.npmrc` (REGISTRY_B)
And I would like yarn to download my dependencies from the alternate URL (REGISTRY_B)

I feel like the simplest solution would be to add a "hash" or "commit-ish" field to entries.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a lot of text that does not add much value compared to the previous section.
Here we need to see concrete examples how Yarn will behave with the new feature.
I suggest something like this:

If yarn.lock file contains

abbrev@1:
   version "1.1.0"
   resolved "https://registry.yarnpkg.com/abbrev/-/abbrev-1.1.0.tgz#d0554c2256636e2f56e7c2e5ad183f859428d81f"

And .yarnrc has setting override-registry={"registry.yarn.pkg.com": "registry.npm.org"}
Then Yarn will fetch abbrev-1.1.0.tgz from https://registry.npm.org/abbrev/-/abbrev-1.1.0.tgz#d0554c2256636e2f56e7c2e5ad183f859428d81f instead of the one specified in yarn.lock.


```
abbrev@1:
version "1.1.0"
hash "d0554c2256636e2f56e7c2e5ad183f859428d81f"
resolved "https://registry.yarnpkg.com/abbrev/-/abbrev-1.1.0.tgz#d0554c2256636e2f56e7c2e5ad183f859428d81f"
```

This, with the version already provided, could then be used to generate a URL on the fly or at the beginning of operations if an alternate registry is specified in the scenarios above.

Or, the `resolved` field could change to not include the hostname/origin, which would then be inserted during operations. This would
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep this proposal simple and not have many possible solutions in one place, you can list them below in the alternatives section

probably have implications for backwards compatibility though.


# How We Teach This

This would largely be in the internal workings, and wouldn't require teaching. npm 5 does this
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We just need an entry in docs page, maybe here https://yarnpkg.com/en/docs/yarnrc.

A short blog post would be great too.

It is straight forward, so no need to mention anything else here.

already, so it would be inherent.

> If you generated your package lock against registry A, and you switch to registry B, npm will now try to install the packages from registry B, instead of A. If you want to use different registries for different packages, use scope-specific registries (npm config set @myscope:registry=https://myownregist.ry/packages/). Different registries for different unscoped packages are not supported anymore.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then let's just copy what npm5 does.
Make a proposal to match npm5 behavior with similar examples and configs

Copy link
Contributor Author

@vernak2539 vernak2539 Oct 25, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want a proposal to do just like npm, nothing would need to change, just yarn would have to take into account the registry in the rc file, and use the registry in that no matter how the lock file was generated. I think that's where I was going with the original one, but that would be a fundamental change in behaviour, probably a breaking change

I've updated the PR as well as my comment

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can I contribute something here? Very eager to see this go through - it is a dealbreaker at the moment for us since we (like @vernak2539 ) use different registries in different build environments.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@csvan, everyone's contribution is welcome.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,

our use case is sharing code between external (outside the company, seeing directly public npm repos) and internal developers (i.e. working inside a company, with proxy and an artifactory mirroring the public npm repos). moreover, there is a ci that also sees only internal repos.

actually, there are two issues with repos in yarn.lock:

  1. while reading (i.e. when installing npm modules)
  2. while writing (i.e. when new modules are added)

my remarks / desires (without regard whether they are fulfilled by the suggested solutions above):

for 1.: basically, what we need is to have a single yarn.lock that is valid and working in both environments, so it can reside in our git repo. even if we would be fine without them, yarn.lock must have embedded repo URLs for backward compatibility. hence we would like to have a single set of URLs, which should be: public URLs for public available npm packages and private (i.e. pointing to the artifactory repo) URLs for private packages (actually, the company has separate artifactory URLs for public and private npm packages). obviously, access to the private packages is only possible from inside the company, so this would break for external developers - which is OK.

for 2.: the desire to have a single set of URLs both valid inside and outside the company (obviously this is only possible for public packages) would mandate that public URLs are written into yarn.lock even when a dependency on a public npm package is added by an internal developer and thus served by the internal artifactory repo. note that this use case does typically not occur during a build process (e.g. in a ci/cd). of course, the package namager has no other means to decide whether a package is public than configuration and the final package URL (in our case, it would be possible to make this distinction, as private packages have a different internal URL, in other cases there might be a set of rules giving such a distinction - which in turn might be overshooting to support).

thus we would probably need means to override a certain public repo by a private one as a local configuration property (not going into the git repo - e.g. in .npmrc) and simultaneously using this override "backwards" when writing URLs into yarn.lock. note also that the override cannot only be only a host, as the URL path and/or the protocol under which npm modules are visible may differ between different repos.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any update on this rfc ?

http://blog.npmjs.org/post/161081169345/v500

# Drawbacks

I don't see any drawbacks in implementing this feature as it would keep track with npm

# Alternatives

Only workaround is to modify the yarn.lock file and replace all references to the registries with
the desired registry.

It seems that there are already many issues around this, so it seems there is a need.

# Unresolved questions

Optional, but suggested for first drafts. What parts of the design are still
TBD?