yarn.lock file changing due to non-deterministic generation. #1168
Comments
|
Sample files. |
|
This may be a MacOS vs. Linux thing, as the linux versions seem to be self consistent. |
|
Are you sure all the devs involve were using the same version for the "before" and "after" versions of the file @SPSpwetter ? |
|
Yes, it was a git pull in all cases. |
|
Confirming that this appears in 0.16.0 as well. |
|
Do the lockfiles still get modified if you use --pure-lockfile? AFAICS, if #570 was fixed, this might become much less of an issue - The lockfile would only get modified if there was a legitimate reason to modify the lockfile. It may still change order, but only when the file was changing for some other reason anyway. |
|
Testing... no they don't get modified. I'll have to think about that though. I can easily change the makefile the non-UI developers use to enforce that, but its kind of a "feature" that the lock file gets modified if the resolution is different somehow. If yarn changed to require --save-lockfile, I'd probably want a --warn-lockfile instead, unless that's what the behavior of --pure-lockfile does is not write a file, but warn if it wants to. Still, the versioning order should be deterministic across platforms. |
|
Closing this because #1712 resolves the issue. While the lock file generation should be the same across platforms, if it doesn't get rewritten without and install change, I'm good. |
|
It seems that the original issue is not solved. I just observed this after running @@ -2778,14 +2780,14 @@ hyphenate-style-name@^1.0.1:
version "1.0.2"
resolved "https://registry.yarnpkg.com/hyphenate-style-name/-/hyphenate-style-name-1.0.2.tgz#31160a36930adaf1fc04c6074f7eb41465d4ec4b"
-iconv-lite@^0.4.5, iconv-lite@~0.4.13, [email protected]:
- version "0.4.13"
- resolved "https://registry.yarnpkg.com/iconv-lite/-/iconv-lite-0.4.13.tgz#1f88aba4ab0b1508e8312acc39345f36e992e2f2"
-
[email protected]:
version "0.2.11"
resolved "https://registry.yarnpkg.com/iconv-lite/-/iconv-lite-0.2.11.tgz#1ce60a3a57864a292d1321ff4609ca4bb965adc8"
+[email protected], iconv-lite@^0.4.5, iconv-lite@~0.4.13:
+ version "0.4.13"
+ resolved "https://registry.yarnpkg.com/iconv-lite/-/iconv-lite-0.4.13.tgz#1f88aba4ab0b1508e8312acc39345f36e992e2f2"
+Original was commited on Mac OS, I'm on Linux. I understand that a big pain point was fixed (reordering without action) but the ordering issue itself is major. Yarn version: 0.18.1 |
|
@pior Were both the original and changed |
|
Hmmm... I just did some testing with 0.18.1, and got slightly only different results between linux and MacOS, but this seems to be much better than it used to be. @pior? |
|
@SPSpwetter could you post what the slight difference was? |
|
Ok, might just be different resolution: < domelementtype@1, domelementtype@~1.1.1:
---
> domelementtype@1:
> version "1.3.0"
> resolved "http://darkstar-devops.lab.skyportsystems.com:4873/domelementtype/-/domelementtype-1.3.0.tgz#b17aed82e8ab59e52dd9c19b1756e0fc187204c2"
>
> domelementtype@~1.1.1:
2223c2227
< [email protected], glob@^7.0.3, glob@^7.0.5:
---
> [email protected]:
2243a2248,2258
> glob@^7.0.3, glob@^7.0.5:
> version "7.1.1"
> resolved "http://darkstar-devops.lab.skyportsystems.com:4873/glob/-/glob-7.1.1.tgz#805211df04faaf1c63a3600306cdf5ade50b2ec8"
> dependencies:
> fs.realpath "^1.0.0"
> inflight "^1.0.4"
> inherits "2"
> minimatch "^3.0.2"
> once "^1.3.0"
> path-is-absolute "^1.0.0"
> |
|
|
|
@SPSpwetter Shouldn't this issue be re-opened? From your screenshot and diff, it looks like the Perhaps there is some relationship with the state of the local ./node_modules directory. Without a published spec for how |
|
We also encountered this issue recently. Simple ♥ yarn add async_fn@^0.1.7
yarn add v0.18.1
[1/4] 🔍 Resolving packages...
[2/4] 🚚 Fetching packages...
warning [email protected]: The engine "browser" appears to be invalid.
[3/4] 🔗 Linking dependencies...
warning Unmet peer dependency "[email protected] || 0.20.x".
[4/4] 📃 Building fresh packages...
success Saved lockfile.
success Saved 1 new dependency.
└─ [email protected]
✨ Done in 6.98s.
♥ git status
## master...origin/master
M package.json
M yarn.lock
♥ git diff --stat
package.json | 2 +-
yarn.lock | 1061 +++++++++++++++++++++++++++++++++++++++++++++++++-----------------------------------------------------------------------------------------------------------
2 files changed, 335 insertions(+), 728 deletions(-)We reproduced this issue both on yarn |
|
@conf Depending on the package you add and how many dependencies it has, I'd expect a lot of changes in yarn.lock. This bug is about yarn.lock changing without changing package.json. |
|
@rjmunro So you're saying this is normal behavior because |
|
@conf Possibly. I'm just a random user interested in yarn, not someone who makes it. I'm just saying how I feel it should work, based on my experience of yarn and similar tools in other languages. Was the old yarn.lock generated by the same version of yarn as the new one? If both of those are true, then there is still an issue. I think @markstos & @SPSpwetter's issue about different resolutions on different platforms is not the same as this one. This is about the same resolution being stored differently in yarn.lock. |
|
@rjmunro Thank you for your input. I'm also not 100% sure, I'll try to do some tests with different packages, and yarn versions. |
|
I'll confirm that the issue here is specific to getting different versions of yarn.lock across platforms with the same package.json, and using the same in-house yarn repository. |
|
The following issue happens to me as well, I'm using Linux, and other developers here are using Mac. -classnames@^2.2.0, classnames@^2.2.4, classnames@^2.2.5:
+classnames@^2.1.2, classnames@^2.2.0, classnames@^2.2.4, classnames@^2.2.5:All we did was yarn install to add new dependency that it is not that one specific. |
|
We're running into this issue immediately after migrating to yarn. In our case all devs are using OSX (possibly different minor versions) yet each dev gets different ordering of lock file dependencies every time we run Yes, presumably |
|
If there were a published spec for how yarn.lock is supposed to work, it would easier for other developers to dive into the code with some confidence about what's a bug expected behavior related to It would be best if someone involved with the Right now it's not clear if the spec itself has a flawed design or if the spec is sound and there's an implementation bug that someone can dive in and fix. |
|
I run into the same problem: |
|
This appears to still be not fixed... can we please reopen or get a pointer to a new bug that shows this is being addressed? The biggest problem is when non-deterministic lockfile regeneration leads to lockfile conflicts between developers. This has happened to our organization's dev team frequently we've had so much so that we've had to craft a process for dealing with these "lockfile conflicts". That process being merging the package.json file in the case of a lockfile conflict, discarding the lockfile changes, and then regenerating the lockfile based on the merged package.json. While this solution works and narrows the scope of the problem, it also overwrites other user's lockfile versions, so there is still an issue here that needs addressing imo. |
|
Hmmm... Wondering if this is a regression? I'm using 0.18.1 and I only get yarn.lock file changes when package.json changes, even across platforms. However, your process is correct, in that if your package.json file changes, you should discard the old one and generate a new one. |
|
It's hard to know what's a regression when there is no published spec for use of the |
|
Well, the docs make it pretty clear that you're supposed to checkin your yarn.lock file with your source, and know that running yarn again should give you the same set of node_modules and the same yarn.lock file. Our workflow here is:
Testing with latest. |
|
Hmmm... Ok, I'm getting a diff in the yarn.lock file, where I didn't used to: PierceWetter3:scweb pierce$ git diff yarn.lock
diff --git a/skycontrol/src/sc-web/yarn.lock b/skycontrol/src/sc-web/yarn.lock
index 3e169c3..a386fd7 100644
--- a/skycontrol/src/sc-web/yarn.lock
+++ b/skycontrol/src/sc-web/yarn.lock
@@ -5529,14 +5529,14 @@ semver-truncate@^1.0.0:
dependencies:
semver "^5.3.0"
-"semver@2 || 3 || 4 || 5", semver@^4.0.3, semver@~4.3.3:
- version "4.3.6"
- resolved "http://ourinternalnpmrespository:4873/semver/-/semver-4.3.6.tgz#300bc6e0e86374f7ba61068b5b1ecd57fc6532da"
-
-semver@^5.3.0:
+"semver@2 || 3 || 4 || 5", semver@^5.3.0:
version "5.3.0"
resolved "http://ourinternalnpmrespository:4873/semver/-/semver-5.3.0.tgz#9b2ce5d3de02d17c6012ad326aa6b4d0cf54f94f"
+semver@^4.0.3, semver@~4.3.3:
+ version "4.3.6"
+ resolved "http://ourinternalnpmrespository:4873/semver/-/semver-4.3.6.tgz#300bc6e0e86374f7ba61068b5b1ecd57fc6532da"
+
[email protected]:
version "0.15.1"
resolved "http://ourinternalnpmrespository:4873/send/-/send-0.15.1.tgz#8a02354c26e6f5cca700065f5f0cdeba90ec7b5f" |
|
Reverting to 0.18.1: So doesn't happen with old version. That means this is either a regression, or the yarn lock format changed? |
|
No idea why Versions: yarn 0.24.6, node v4.7.10.0. package.json used for testing: {
"name": "buildbot-nestedexample",
"engines": {
"node": ">=0.10.0",
"npm": ">=1.4.0"
},
"dependencies": {
"guanlecoja": "~0.8.3",
"gulp": "3.9.0"
}
} |
|
Thanks for reporting, we are tracking this issue in #3490 |
Bug: Yarn.lock file changing ordering of dependencies and resolutions.
What is the current behavior?
The yarn.lock file has changes between different developers systems, with small differences. Mostly they seem to be ordering differences.
If the current behavior is a bug, please provide the steps to reproduce.
See differences sample below. These are all transitive dependencies from the package.json root. Reviewing the differences, it seems that yarn isn't fully deterministic in ordering when generating the yarn.lock file, as the ultimate content is identical, but the pack ordering and the version ordering for a package shows differences.
What is the expected behavior?
Expectation: Developer can check in yarn.lock file. Other developers can run yarn and get an up-to-date node-modules folder. If a new yarn.lock file is produced, it is identical to the checked in version.
Please mention your node.js, yarn and operating system version.
node: 4.6.0 with npm 3.10.8
yarn: 0.15.1
OS: Mac OS Sierra, and Ubuntu 14.4
Sample Differences:
The text was updated successfully, but these errors were encountered: