upgrading from 4.0.2 to 4.0.7 #2191

akoskm · 2019-07-18T13:24:12Z

Upgrading from 4.0.2 to 4.0.7 introduces several devDependencies that weren't required in 4.0.2:

 "@babel/plugin-proposal-class-properties": "7.5.0",
 "@babel/plugin-syntax-dynamic-import": "7.2.0",
 "@babel/plugin-transform-runtime": "7.5.0",
 "babel-loader": "8.0.6",
 "babel-plugin-dynamic-import-node": "2.3.0",
 "file-loader": "4.0.0",
 "webpack-cli": "3.3.6",
 "webpack-dev-server": "3.7.2"

I followed https://github.com/rails/webpacker#upgrading and https://github.com/rails/webpacker/blob/master/CHANGELOG.md to upgrade. I also copied the contents of https://github.com/rails/webpacker/blob/master/lib/install/config/babel.config.js over my local version of the same file.

I'm just curious if I'm doing something wrong that makes these new dependencies appear or this is ok.

Some of these dependencies like @babel/plugin-transform-runtime, @babel/plugin-proposal-class-properties were used in babel.config.js at 4.0.2 as well but webpacker worked just fine without include these packages in package.json.

The text was updated successfully, but these errors were encountered:

akoskm · 2019-09-09T06:46:38Z

Any idea why these new dependencies got added between these minor versions?

jakeNiemiec · 2019-09-09T15:48:07Z

Only webpack-dev-server should be in your devDeps. Can you directly link which document said to put (eg) file-loader into devDeps?

Can you additionally paste your entire package.json?

akoskm · 2019-09-09T16:11:40Z

Hi @jakeNiemiec, thanks for responding on this! After following the instructions here https://github.com/rails/webpacker#upgrading file-loader appeared with the other new devDependencies I listed. I didn't add anything to package.json manually, this is the diff after the update in package.json:

jakeNiemiec · 2019-09-09T16:15:57Z

The problem is that your devDeps are invalid to begin with. Can you paste your entire package.json?

jakeNiemiec · 2019-09-09T16:17:25Z

If you don't want to make it public, you can tag me in an unlisted https://gist.github.com and I can comment there.

akoskm · 2019-09-09T16:23:14Z

@jakeNiemiec I think I just did that. Mentioned you in the comment section of a secret gist.

jakeNiemiec · 2019-09-09T17:11:37Z

I'm still not sure why all those new dependencies appeared.

This is because your 4.0.2 package.json was invalid. See: https://github.com/rails/webpacker/blob/master/docs/yarn.md#add-an-npm-module-to-devdependencies

Did moving them to "dependencies" work?

akoskm · 2019-09-09T18:17:43Z

Did moving them to "dependencies" work?
There might be a misunderstanding. My application works after upgrading to webpacker 4.0.7. here's gem info from the project's folder:

$ gem info webpacker

*** LOCAL GEMS ***

webpacker (4.0.7, 4.0.2)
    Authors: David Heinemeier Hansson, Gaurav Tiwari
    Homepage: https://github.com/rails/webpacker
    License: MIT
    Installed at (4.0.7): /Users/akoskm/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0
                 (4.0.2): /Users/akoskm/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0

    Use webpack to manage app-like JavaScript modules in Rails

It's just the new packages that I don't know why are added.
Looking at yarn.lock some of these packages, like file-loader, "@babel/plugin-proposal-class-properties": "7.5.0", "@babel/plugin-syntax-dynamic-import": "7.2.0", "@babel/plugin-transform-runtime": "7.5.0", appear to be a dependency of @rails/[email protected].

akoskm · 2019-09-09T18:21:22Z

I'll start this upgrade process again but first I'll move the packages that should be in devDependencies there. Then proceed with these instructions: https://github.com/rails/webpacker#upgrading.

akoskm · 2019-09-09T18:46:42Z

@jakeNiemiec I just run the commands for upgrade and packages.json ended up with a substantially smaller diff than earlier, I don't know what caused this behavior.

I used the following to upgrade to 4.0.7:

bundle update webpacker
rails webpacker:binstubs
yarn upgrade @rails/[email protected]
yarn upgrade webpack-dev-server --latest

after that my packages.json is:

$ git diff package.json
diff --git a/package.json b/package.json
index e8cd552d..9f364621 100644
--- a/package.json
+++ b/package.json
@@ -13,7 +13,7 @@
     "@fortawesome/free-regular-svg-icons": "5.9.0",
     "@fortawesome/free-solid-svg-icons": "5.9.0",
     "@fortawesome/react-fontawesome": "0.1.4",
-    "@rails/webpacker": "4.0.2",
+    "@rails/webpacker": "4.0.7",
     "actioncable": "5.2.3",
     "babel-plugin-transform-react-remove-prop-types": "0.4.24",
     "bootstrap": "4.3.1",
@@ -62,7 +62,7 @@
     "sinon": "7.3.2",
     "sinon-chai": "3.3.0",
     "style-loader": "0.23.1",
-    "webpack-dev-server": "3.3.1"
+    "webpack-dev-server": "3.8.0"
   },
   "babel": {
     "presets": [

We can close this issue because I can't reproduce the behavior I initially reported.

akoskm · 2019-09-09T19:05:22Z

@jakeNiemiec Okay, I think I realized what was happening back in July when I first tried to do this.
So the upgrade commands produced this small diff in package.json back then as well.
I started running into problems when i tried to start the dev server:

$ ./bin/webpack-dev-server
The CLI moved into a separate package: webpack-cli
Please install 'webpack-cli' in addition to webpack itself to use the CLI
-> When using npm: npm i -D webpack-cli
-> When using yarn: yarn add -D webpack-cli
internal/modules/cjs/loader.js:638
    throw err;
    ^

Error: Cannot find module 'webpack-cli/bin/config-yargs'

so I installed webpack-cli (altough with the -D options, I didn't about this https://github.com/rails/webpacker/blob/master/docs/yarn.md#add-an-npm-module-to-devdependencies). Now I installed it without -D (I'm always using -E though).

After the dev server started up I was getting errors such:

ERROR in Entry module not found: Error: Can't resolve 'babel-loader' in '/Users/akoskm/Projects/example/webapp'

after installing babel-loader as well and giving a shot to ./bin/webpack-dev-server again I got:

ERROR in ./app/javascript/packs/components/sidebar/index.jsx
Module build failed (from ./node_modules/babel-loader/lib/index.js):
Error: Cannot find module '@babel/plugin-syntax-dynamic-import'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
    at Function.Module._load (internal/modules/cjs/loader.js:562:25)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at module.exports (/Users/akoskm/Projects/example/webapp/babel.config.js:47:7)

and I could go on but eventually when I fix all these errors I'll end up with the extra dependencies that I listed in my initial message.

Which returns me to my original questions, why my project works with an older webpacker version (4.0.2) with less dependencies and why it doesn't when I upgrade to a newer webpacker?

jakeNiemiec · 2019-09-09T19:46:15Z

Do you have a babel.config.js OR a .babelrc file in your project root?

jakeNiemiec · 2019-09-09T20:00:08Z

Please make the appropriate changes listed in the changelog: https://github.com/rails/webpacker/blob/master/CHANGELOG.md#407---2019-06-03

and

akoskm · 2019-09-09T20:20:11Z

Done.

@babel/polyfill isn't used in my project so no changes there.

Regarding the fixes for 4.0.7 I did the following:

copy/pasted the contents of the default babel.config.js from 4.0.7 tag
returned @babel/preset-react and babel-plugin-transform-react-remove-prop-types plugins that are added when you turn on react support.

When starting the dev server I'm receiving the same errors:

$ ./bin/webpack-dev-server
The CLI moved into a separate package: webpack-cli
Please install 'webpack-cli' in addition to webpack itself to use the CLI
-> When using npm: npm i -D webpack-cli
-> When using yarn: yarn add -D webpack-cli
internal/modules/cjs/loader.js:638
    throw err;
    ^

Error: Cannot find module 'webpack-cli/bin/config-yargs'

EDIT
and everything works the same after: if I add webpack-cli then it complains about babel-loader, then @babel/plugin-syntax-dynamic-import.

jakeNiemiec · 2019-09-09T20:55:24Z

How are you adding webpack-cli? Just yarn add webpack-cli?

akoskm · 2019-09-10T05:07:46Z

Yes, except I add an exact version: yarn add -E webpack-cli .

jakeNiemiec · 2019-09-10T14:58:28Z

Try this:

delete your yarn.lock file.
delete node_modules
yarn install

I see evidence that this may fix things for you. -E might have played a role in this, I would not use it in addition to your lock file.

akoskm · 2019-09-10T19:14:58Z

Unfortunately this makes no difference. After doing those steps running ./bin/webpack-dev-server still needs webpack-cli. After adding webpack-cli (without the -E) I need babel-loader, etc.

akoskm · 2019-09-10T19:37:03Z

Inspired by your comment that might the -E that this problem I updated every dependency in my package.json to ^version. Removed yarn.lock & node_modules and run yarn install.

yarn asked me to specify which version of babel-loader (8.0.6) and webpack-cli (3.3.8) I want to install.

I picked the latest stable for both. Running ./bin/webpack-dev-server doesn't require me to install anything and my app runs fine.

Thank you for your suggestions!

seanders · 2019-10-07T16:54:59Z

I just wanted to follow up on this thread. My application is upgrading from webpacker 4.0.0 to 4.0.7. The only consequential change that I can identify with respect to babel-loader is this:
https://github.com/rails/webpacker/compare/v4.0.0..v4.0.7#diff-b9cfc7f2cdf78a7f4b91a753d10865a2R24

Prior to this upgrade, this is where our babel-loader was location:
node_modules/babel-loader

After the upgrade to 4.0.7, I noticed that the location of the babel-loader seemed to move to this location:
node_modules/@rails/webpacker/node_modules/babel-loader

This change in location required us to explicitly add babel-loader as a dependency in our package.json (in addition to file-loader).

I'm guessing this change might have something to do with yarn's resolution algorithm.

Any thoughts on what explicitly might cause this change?

seanders · 2019-10-07T17:19:39Z

Seems like this is what I'm referring to: #1537

jakeNiemiec · 2019-10-07T17:30:56Z

Any thoughts on what explicitly might cause this change?

Please make note of the breaking changes.

I noticed that the location of the babel-loader seemed to move to this location:

It should be accessible from both of those locations. This #2191 (comment) may help.

seanders · 2019-10-07T17:50:38Z

I think that removing of the lock file and re-installing all the dependencies, incidentally moved the babel-loader package back to node_modules/babel-loader, which solved @akoskm 's issue.

According to this yarn contributor, babel-loader getting installed into the root node_modules(by virtue of being a dependency of @rails/webpacker) is a side-effect of yarn trying to minimize the size of the node_modules directory:

Relying on this is entirely undefined behavior, do not do it. Always list your dependencies explicitely. Note that this isn't unique to Yarn, the same is true for npm. And pnpm will break your application entirely.

See yarnpkg/yarn#5749 (comment)

If I'm understanding this correctly, the optimal fix is to explicitly list babel-loader as a direct dependency of any project using @rails/webpacker.

jakeNiemiec · 2019-10-07T18:20:55Z

@seanders You are entirely correct in your reasoning. I don't think that webpacker should be implicit about dependencies. Here is the list of those:

webpacker/package.json

Lines 14 to 53 in d393cc4

    
           "dependencies": { 
        
             "@babel/core": "^7.4.5", 
        
             "@babel/plugin-proposal-class-properties": "^7.4.4", 
        
             "@babel/plugin-proposal-object-rest-spread": "^7.4.4", 
        
             "@babel/plugin-syntax-dynamic-import": "^7.2.0", 
        
             "@babel/plugin-transform-destructuring": "^7.4.4", 
        
             "@babel/plugin-transform-regenerator": "^7.4.5", 
        
             "@babel/plugin-transform-runtime": "^7.4.4", 
        
             "@babel/preset-env": "^7.4.5", 
        
             "@babel/runtime": "^7.4.5", 
        
             "babel-loader": "^8.0.6", 
        
             "babel-plugin-dynamic-import-node": "^2.2.0", 
        
             "babel-plugin-macros": "^2.5.0", 
        
             "case-sensitive-paths-webpack-plugin": "^2.2.0", 
        
             "compression-webpack-plugin": "^2.0.0", 
        
             "core-js": "^3.1.3", 
        
             "css-loader": "^2.1.1", 
        
             "file-loader": "^3.0.1", 
        
             "flatted": "^2.0.0", 
        
             "glob": "^7.1.4", 
        
             "js-yaml": "^3.13.1", 
        
             "mini-css-extract-plugin": "^0.7.0", 
        
             "node-sass": "^4.12.0", 
        
             "optimize-css-assets-webpack-plugin": "^5.0.1", 
        
             "path-complete-extname": "^1.0.0", 
        
             "pnp-webpack-plugin": "^1.4.3", 
        
             "postcss-flexbugs-fixes": "^4.1.0", 
        
             "postcss-import": "^12.0.1", 
        
             "postcss-loader": "^3.0.0", 
        
             "postcss-preset-env": "^6.6.0", 
        
             "postcss-safe-parser": "^4.0.1", 
        
             "regenerator-runtime": "^0.13.2", 
        
             "sass-loader": "^7.1.0", 
        
             "style-loader": "^0.23.1", 
        
             "terser-webpack-plugin": "^1.3.0", 
        
             "webpack": "^4.32.2", 
        
             "webpack-assets-manifest": "^3.1.1", 
        
             "webpack-cli": "^3.3.2", 
        
             "webpack-sources": "^1.3.0" 
        
           },

@swrobel Is correct in that issue. The only problem is that I personally come from the webpack/node.js side of things, I am simply not aware of a good way to fix this on the rails side of things. Do we just add all those dependencies onto run "yarn add @rails/webpacker@next"? Should we create a rake task that updates your package.json?

seanders · 2019-10-08T20:38:43Z

Sounds like a rake take might be the right approach. Not sure of a better way to keep those dependencies ('@rails/webpacker', 'babel-loader', 'file-loader') in lockstep.

jrochkind · 2019-11-20T18:43:08Z

Sounds like a rake take might be the right approach. Not sure of a better way to keep those dependencies ('@rails/webpacker', 'babel-loader', 'file-loader') in lockstep.

Hmm. What about:

Mark them as "peer dependencies" in webpacker itself.
Have the existing webpacker:install rake task include them to the host app's package.json. (This rake task already exists, but doesn't add these).

I am not sure a rake task is needed to update them in an existing package.json. If you update your webpacker to something that needs different versions than the originally-generated ones in your local package.json, the peer dependencies in the (new version of) webpacker should give you a very comprehensible error message telling you what the issue is, leading you to go fix your local package.json.

Apologies if I'm missing the point or talking about something irrelevant; I come from ruby side, and am trying to figure out what the heck is going on.

elliotcm · 2019-12-24T14:53:12Z

If it helps anyone at all, this is the package.json I ended up with after resolving all of these issues, albeit with some extra React stuff in there:

{
  "name": "<app name>",
  "private": true,
  "dependencies": {
    "@babel/core": "^7.0.0-0",
    "@babel/plugin-proposal-class-properties": "^7.7.4",
    "@babel/plugin-syntax-dynamic-import": "^7.7.4",
    "@babel/plugin-transform-destructuring": "^7.7.4",
    "@babel/plugin-transform-runtime": "^7.7.6",
    "@babel/preset-env": "^7.7.7",
    "@babel/preset-react": "^7.7.4",
    "@rails/webpacker": "^4.2.0",
    "babel-loader": "^8.0.6",
    "prop-types": "^15.6.0",
    "react-dom": "^16.12.0",
    "react": "^16.12.0",
    "yarn": "^1.21.0"
  },
  "devDependencies": {
    "webpack": "^4.0.0",
    "webpack-cli": "^3.3.10",
    "webpack-dev-server": "^3.9.0"
  }
}

akoskm · 2020-01-29T10:20:06Z

@jakeNiemiec this seems to be a recurring problem when trying to upgrade to the latest version. Yesterday I wanted to update form 4.0.7 to 4.2.2, following the instructions here https://github.com/rails/webpacker#upgrading.

I experienced the same issue as I did during my previous upgrade. Ruby was complaining about babel-loader not being present, then about "@babel/plugin-proposal-class-properties", and basically all the package in babel.config.js. In the end, my package.json would probably look like the one @elliotcm posted.

This is not the way to go, because when you make a fresh install of webpacker you don't need any of those packages, right?

So I figured that removing yarn.lock and generating a new one resulted in a substantially different yarn.lock, compared to what I ended up with after following the upgrade instructions.

With the fresh yarn.lock I wasn't required to add all those modules to package.json that ruby was complaining about earlier.

I think you should update the upgrade instructions and mention that as last step we should remove yarn.lock and let yarn install generate a new one.

jrochkind · 2020-01-29T13:52:16Z

Removing yarn.lock might work as a workaround, but it's a problem if that's recommended/required, it should not be a required step to upgrade webpacker.

Compare to removing your Gemfile.lock. You are removing all record of what version of dependencies in your tree were working, and rebuilding them all fresh. Comparable to doing a bundle update. While sometimes it makes sense to do this, imagine if you were forced to do a complete bundle update every time you wanted to update, oh, sprockets, or bundler, or even rails. It would lead to much increased work and frustration.

If it's necessary for you to remove yarn.lock, that's a problem to be fixed, not something to be documented/recommended. There may also be another solution. But yarn/npm management confuses me; it's definitely not as smooth as bundler.

akoskm · 2020-01-29T14:49:34Z

Removing yarn.lock might work as a workaround, but it's a problem if that's recommended/required, it should not be a required step to upgrade webpacker.

@jrochkind you're right, this sounds more like a workaround than something that should be done as a step of the upgrade process.

Compare to removing your Gemfile.lock

I don't work with ruby at all so can't relate to this. Removing the yarn.lock and doing yarn install was trivial and it fixed all my problems.

If it's necessary for you to remove yarn.lock, that's a problem to be fixed, not something to be documented/recommended.

I agree. After running into this problem the second time, I'm pretty sure that this is one possible solution to these problems.

jakeNiemiec · 2020-01-31T18:43:40Z

This is not the way to go, because when you make a fresh install of webpacker you don't need any of those packages, right?

@akoskm You should have those in your package.json. As I said, Webpacker is relying on undocumented behavior (having access to dependencies of dependencies). More discussion on this over here: #2371 (comment)

The best solution is to just add those dependencies yourself with yarn add myPackage. This will save you from future breakage from the yarn/npm side as well.

akoskm · 2020-02-01T07:19:44Z

@jakeNiemiec sorry, I missed that comment and didn't realize that this is and undocumented behavior.

I work with other projects (babel/yarn/webpack, no webpacker) and you have to list "@babel/plugin-proposal-class-properties" in your dev dependencies, if you want to require it in babel.config.js. The way how dependencies are handled here is different from other JS projects, so thanks for clarifying this.

jakeNiemiec · 2020-02-03T18:22:52Z

@akoskm I'll start this upgrade process again but first I'll move the packages that should be in devDependencies there.

...you have to list "@babel/plugin-proposal-class-properties" in your dev dependencies, if you want to require it in babel.config.js. The way how dependencies are handled here is different from other JS projects...

This is a common misconception, even in other JS projects. Babel needs to be in your dependencies since a production environment needs it to compile your packs. From the docs:

webpacker/docs/yarn.md

Lines 21 to 23 in ed6c121

    
           Be careful not to add any build or app related JS modules in this fashion. Adding JS modules to `devDependencies` [will block them from being installed in **any** production environment](https://yarnpkg.com/lang/en/docs/cli/install/#toc-yarn-install-production-true-false). 
        
           Docs from JS modules may instruct you to use `--dev` or `devDependencies`, but this is generally under the assumption that you are using a `node.js` workflow.

Related issues #1880 (comment)
Longer discussion on this topic: #1212 (comment)

akoskm · 2020-02-03T19:20:48Z

This is a common misconception, even in other JS projects.

Sorry for the confusion, I was comparing webpacker to JS projects.

Babel needs to be in your dependencies since a production environment needs it to compile your packs.

Yup, I understand this. With webpacker, babel must be listed under dependencies, in other projects I list babel under devDependencies.

jakeNiemiec added the support Questions or unspecific problems label Sep 9, 2019

This comment has been minimized.

Sign in to view

jakeNiemiec closed this as completed Sep 9, 2019

This comment has been minimized.

Sign in to view

jakeNiemiec mentioned this issue Nov 20, 2019

Fresh install complains "[email protected]" has unmet peer dependency "webpack@^4.0.0"." #2371

Closed

jakeNiemiec added dependencies dependency, yarn or other package management and removed support Questions or unspecific problems labels Nov 20, 2019

itume mentioned this issue Nov 24, 2019

yarn add bulma itume/advent_calendar_2019_assets#2

Open

aried3r mentioned this issue Mar 24, 2020

Update dependencies #2508

Merged

lrhacker mentioned this issue Aug 18, 2021

Failed import of useReduceMotion with @rails/webpacker mantinedev/mantine#192

Closed

upgrading from 4.0.2 to 4.0.7 #2191

upgrading from 4.0.2 to 4.0.7 #2191

Comments

akoskm commented Jul 18, 2019 • edited Loading

akoskm commented Sep 9, 2019

jakeNiemiec commented Sep 9, 2019

akoskm commented Sep 9, 2019

jakeNiemiec commented Sep 9, 2019

jakeNiemiec commented Sep 9, 2019

akoskm commented Sep 9, 2019

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

jakeNiemiec commented Sep 9, 2019

akoskm commented Sep 9, 2019

akoskm commented Sep 9, 2019

akoskm commented Sep 9, 2019

akoskm commented Sep 9, 2019 • edited Loading

jakeNiemiec commented Sep 9, 2019

This comment has been minimized.

jakeNiemiec commented Sep 9, 2019

akoskm commented Sep 9, 2019 • edited Loading

jakeNiemiec commented Sep 9, 2019 • edited Loading

akoskm commented Sep 10, 2019 • edited Loading

jakeNiemiec commented Sep 10, 2019

akoskm commented Sep 10, 2019

akoskm commented Sep 10, 2019 • edited Loading

seanders commented Oct 7, 2019 • edited Loading

seanders commented Oct 7, 2019

jakeNiemiec commented Oct 7, 2019

seanders commented Oct 7, 2019 • edited Loading

jakeNiemiec commented Oct 7, 2019 • edited Loading

seanders commented Oct 8, 2019

jrochkind commented Nov 20, 2019

elliotcm commented Dec 24, 2019

akoskm commented Jan 29, 2020

jrochkind commented Jan 29, 2020

akoskm commented Jan 29, 2020

jakeNiemiec commented Jan 31, 2020

akoskm commented Feb 1, 2020 • edited Loading

jakeNiemiec commented Feb 3, 2020

akoskm commented Feb 3, 2020

akoskm commented Jul 18, 2019 •

edited

Loading

akoskm commented Sep 9, 2019 •

edited

Loading

akoskm commented Sep 9, 2019 •

edited

Loading

jakeNiemiec commented Sep 9, 2019 •

edited

Loading

akoskm commented Sep 10, 2019 •

edited

Loading

akoskm commented Sep 10, 2019 •

edited

Loading

seanders commented Oct 7, 2019 •

edited

Loading

seanders commented Oct 7, 2019 •

edited

Loading

jakeNiemiec commented Oct 7, 2019 •

edited

Loading

akoskm commented Feb 1, 2020 •

edited

Loading