Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

yarn install --production is broken, installs plenty of devDependencies #1379

Closed
Rush opened this issue Oct 23, 2016 · 28 comments
Closed

yarn install --production is broken, installs plenty of devDependencies #1379

Rush opened this issue Oct 23, 2016 · 28 comments
Labels

Comments

@Rush
Copy link

Rush commented Oct 23, 2016

I'm running yarn 0.16.1. Given the following package.json, yarn will install wrong dependencies:

{
  "name": "yarn-test",
  "version": "1.0.0",
  "private": true,
  "scripts": {
  },
  "dependencies": {
    "babel-plugin-transform-async-to-module-method": "6.8.0",
    "babel-preset-es2015-node6": "0.2.0",
    "babel-register": "6.5.0",
    "bluebird": "3.1.1",
    "body-parser": "~1.12.4",
    "bugsnag": "^1.6.5",
    "connect-gzip-static": "^1.0.0",
    "continuation-local-storage": "^3.1.4",
    "cookie-parser": "~1.3.5",
    "debug": "~2.2.0",
    "del": "^1.2.1",
    "express": "4.14.0",
    "filtrex": "0.5.4",
    "glob": "^5.0.10",
    "html-to-text": "^1.3.2",
    "http-proxy": "^1.11.2",
    "juice": "^1.4.2",
    "marked": "0.3.6",
    "md5": "^2.0.0",
    "memoizee": "^0.3.8",
    "messageformat": "0.3.1",
    "moment": "~2.10.6",
    "morgan": "~1.5.3",
    "ms": "^0.7.1",
    "multer": "~0.1.8",
    "node-watch": "^0.3.4",
    "pug": "2.0.0-beta6",
    "recursive-merge": "^1.2.0",
    "request-promise": "^1.0.0",
    "requirejs": "^2.1.17",
    "send": "^0.13.0",
    "serve-favicon": "~2.2.1",
    "wkhtmltopdf": "^0.1.5",
    "xcase": "^1.0.6"
  },
  "devDependencies": {
    "babel": "6.5.0",
    "babel-eslint": "5.0.0",
    "babel-plugin-transform-es2015-modules-commonjs": "6.5.0",
    "babel-plugin-transform-runtime": "6.15.0",
    "babel-preset-es2015": "6.5.0",
    "chai": "^2.3.0",
    "compressible": "^2.0.5",
    "eslint": "2.2.0",
    "fetch": "1.1.0",
    "gulp": "^3.9.0",
    "gulp-babel": "^6.1.1",
    "gulp-concat": "^2.6.0",
    "gulp-gzip": "^1.2.0",
    "gulp-if": "^1.2.5",
    "gulp-modify": "^0.1.1",
    "gulp-ng-annotate": "^1.1.0",
    "gulp-pug": "3.0.4",
    "gulp-rename": "^1.2.2",
    "gulp-rev": "^6.0.1",
    "gulp-sourcemaps": "^1.5.2",
    "gulp-uglify": "^1.3.0",
    "js-yaml": "^3.3.1",
    "jspm": "0.16.45",
    "mime": "^1.3.4",
    "minimist": "^1.2.0",
    "mocha": "^2.2.5",
    "ng-annotate": "^1.0.1",
    "run-sequence": "^1.1.3",
    "should": "^7.0.1"
  }
}

npm install --production yields 71M of node_modules
yarn install --production yields 340M of node_modules - clearly wrong

To reproduce:

mkdir -p test-yarn
cd test-yarn
wget http://x.rushbase.net/e089e40357ded1a4fff61ce528e5cc20297464b3/package.json
yarn install --production
# repeat again with npm install --production
@nrser
Copy link

nrser commented Oct 23, 2016

i'm seeing the same issue, also with yarn 0.16.1, which is breaking my production builds

@samccone
Copy link
Member

Thanks for the report!

@AlicanC
Copy link
Contributor

AlicanC commented Oct 23, 2016

If you actually hack yarn why to work with --production, it can't find a reason why a devDep should be installed.

I have tried to fix this, but I couldn't make sense of the code.

With --production, Install.p.init() does not immediately drop devDeps, but instead sets their visibility to ENVIRONMENT_IGNORE so your devDeps go through the whole installation process. During this process, something probably overwrites or just fails to detect ENVIRONMENT_IGNORE.

Why devDeps aren't completely dropped with --production? To still be able to generate a complete yarn.lock?

If that is the case, then isn't it pretty unnecessary to do all this work for production environments? Could we make --production require a yarn.lock and use it to skip the whole resolve step so --production is lightning fast?

@Rush
Copy link
Author

Rush commented Oct 23, 2016

Why devDeps aren't completely dropped with --production? To still be able to generate a complete yarn.lock?

If that is the case, then isn't it pretty unnecessary to do all this work for production environments? Could we make --production require a yarn.lock and use it to skip the whole resolve step so --production is lightning fast?

This would be a perfect solution in my workflow.

@sheerun
Copy link
Contributor

sheerun commented Oct 25, 2016

yarn install --production doesn't even work on yarn package itself. How to reproduce:

  1. Download latest yarn package from npm registry, unpack it
  2. run yarn install --production --ignore-scripts --pure-lockfile --ignore-engines

See following output:

yarn install v0.16.1
[1/4] 🔍  Resolving packages...
error "/Users/sheerun/Source/npm-packer/dist/scripts/eslint-rules" doesn't exist.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

While clearly eslint-rules is listed as devDependencies. So it even fails on validation.

@tanx
Copy link

tanx commented Oct 26, 2016

Perhaps a duplicate of #761 ?

@psylence303
Copy link

psylence303 commented Oct 28, 2016

Same happening for me, 'yarn install --production' tries to find a directory that is listed in 'devDependencies' and 'npm i --production' was not doing this.

@troydemonbreun
Copy link
Contributor

troydemonbreun commented Nov 3, 2016

Is it just me, or does the OP's issue seem to be that it is installing too much, not that it is expecting deps that were not installed (per #761)? @Rush would you confirm if #761 is a dupe for your issue?

@sheerun
Copy link
Contributor

sheerun commented Nov 4, 2016

@troydemonbreun It's correlated

@troydemonbreun
Copy link
Contributor

FYI, #1739 landed, which was a fix for #761 (don't have time at the moment to verify if it fixes OP's issue)

@leschekfm
Copy link

With the current yarn version (0.17.6) and the provided package.json I can still see a discrepancy between npm and yarn. So it seems that this issue wasn't fixed by the other issues

@sebmck
Copy link
Contributor

sebmck commented Dec 2, 2016

Fixed via #2116.

@sebmck sebmck closed this as completed Dec 2, 2016
@kyeotic
Copy link

kyeotic commented Feb 10, 2017

@kittens I am still seeing this behavior with the 0.20.0 release

@leschekfm
Copy link

I can see the issue still too. I renamed the node_modules folders after installation for comparison.

leschekfm@lesubunew:~/projects/temp$ npm --version
4.2.0
leschekfm@lesubunew:~/projects/temp$ yarn --version
0.20.0
leschekfm@lesubunew:~/projects/temp$ du -hs node_modules_yarn/
219M	node_modules_yarn/
leschekfm@lesubunew:~/projects/temp$ du -hs node_modules_npm/
70M	node_modules_npm/

@PatrickKing
Copy link

PatrickKing commented Feb 16, 2017

+1 for this, with Yarn 0.20.3

Edited to add: on at least one occasion, I have seen Yarn fail to install a production dependency with --prod as well.

@dwmkerr
Copy link

dwmkerr commented Feb 20, 2017

Same issue for me, causes a lot of problems with my build as dev dependencies contain things like custom linting rules which require SSH access to internal repos...

@dead-claudia
Copy link

dead-claudia commented Apr 18, 2017

Still seeing it myself. @kittens Could this be re-opened?

Edit: Yarn v0.23.2

@SimenB
Copy link
Contributor

SimenB commented Apr 18, 2017

@isiahmeadows there was a fix in 0.23.1 (#2921), are you on latest?

EDIT: You are 😭 What do you get with yarn --prod && yarn check --verify-tree --prod?

@dead-claudia
Copy link

@SimenB I edited my comment with my version (v0.23.2).

@dead-claudia
Copy link

In my case, I have 0 dependencies and several dev dependencies. So anything installing to node_modules is a bug.

@SimenB
Copy link
Contributor

SimenB commented Apr 18, 2017

Oh, fancy

@SimenB
Copy link
Contributor

SimenB commented Apr 18, 2017

Is this a case of doing yarn && yarn test && yarn --force not cleaning up, or a plain yarn --prod from non-existent node_modules installing lots of stuff?

@dead-claudia
Copy link

dead-claudia commented Apr 18, 2017

@SimenB Here's a 100% repro on macOS:

  1. Create this package.json in a clean directory:

    {
      "devDependencies": {
        "eslint": "*"
      }
    }
  2. Run yarn --prod, optionally with --pure-lockfile.

  3. There should be 3 entries installed in node_modules:

@dead-claudia
Copy link

You can also repro this with "electron-download": "*", in which [email protected] is installed because of jsprim and sshpk depended on by electron-download#nugget#request#http-signature.

@dead-claudia
Copy link

@SimenB

Is this a case of doing yarn && yarn test && yarn --force not cleaning up, or a plain yarn --prod from non-existent node_modules installing lots of stuff?

It's the latter case.

@Diokuz
Copy link

Diokuz commented Jun 13, 2017

Have same problems here: caniuse-db (and some other packages) installed from nowhere when yarn --production. But they are in devDeps.

Could it be reopened? Or there is another opened issue?

yarn --version
0.24.6

@miukki
Copy link

miukki commented Jul 6, 2017

how to install only production deps ? (ignore devDep) any solution for now ?

@mick352
Copy link

mick352 commented Feb 11, 2021

From stack overflow : you can use

yarn install --production --frozen-lockfile

it does what you really want.

@yarnpkg yarnpkg locked as resolved and limited conversation to collaborators Feb 11, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests