Errors in Rails system tests when webpack-dev-server is running

[renchap]

I am using webpack-dev-server for development, so the packs in manifest.json are using it on http://localhost:8080/....

I am also using Rails system tests for browser-based testing. I am running NODE_ENV=test ./bin/webpack before running the system tests to generate all needed assets in public/packs, but manifest.json generated by the command is immediately overwritten by the running webpack-dev-server as the change is detected.

In my code, I am enabling some debug code using NODE_ENV == development, so when the system tests are ran I am seeing all the debug output as the assets for the tests are using the local webpack-dev-server and not the assets generated in public/packs.

@gauravtiwari do you have an idea on how to solve this? I thought about changing the manifest name to manifest-test.json in both webpack config and Webpacker config, but there it would be much easier if there was a Rails config option to set the manifest name (https://github.com/rails/webpacker/blob/master/lib/webpacker/configuration.rb#L19). Would this work for you?

[gauravtiwari]

@renchap I see, yes I think it would make sense to have another manifest just for testing, because we simply can't avoid overwriting if two executables are running at the same time with different config. I kinda of see now the use case of environment within paths.yml so, instead of direct declaration we have it scoped per environment.

Btw, why not use client side testing approach instead of Rails? I think it would be much faster and isolated.

[renchap]

I am also running JS tests using Jest, but system tests allows me to test complex workflows (file uploads, multiple client-side actions, ...) and check that the database have the correct content, files are correctly processed, ...

Environment-scoped paths.yml can definitely work. Maybe configure the manifest file name in it, and change Webpacker + Webpack config to get it from here?

[gauravtiwari]

Right I see, makes sense 👍 We had a discussion earlier in #153 about this, but that time this use case didn't occur to me.

default: &default
  config: config/webpack
  entry: bundles
  output: public
  manifest: manifest.json
  node_modules: node_modules
  source: frontend
  extensions:
    - .coffee
    - .js
    - .jsx
    - .ts
    - .vue
    - .sass
    - .css
    - .png
    - .svg
    - .gif
    - .jpeg

development:
  <<: *default

test:
  <<: *default
  manifest: manifest-test.json

production:
  <<: *default

Yepp, sounds good to me 👍

[gauravtiwari]

We probably have to scope this on both sides with environment when reading this file.

[samandmoore]

I also ran into some issues around this. I'd like to be able to run my automated test suite without having to run webpack-dev-server. I think that means that I'd like to be able to toggle enabled to false in development.server.yml using an ENV variable like ENV['CI'] or the approach you outlined above with environment-specific config blocks.

I think that the common way to use the ENV variable would be to pass the yaml through ERB, but I'm not sure how that would play out in JS-land. I'd expect that the environment-specific config blocks would be simpler.

[gauravtiwari]

Yes, I think for longer run having it scoped env wide would be easy and work well.

[gauravtiwari]

@samandmoore @renchap We now have scoped/env configs for webpack 🎉

[renchap]

Awesome, thanks @gauravtiwari!

[samandmoore]

Thanks so much for addressing this.

[justin808]

@gauravtiwari @samandmoore @renchap If one is not using the hot reload server for dev, then we'll still have this issue. The only way around this would be to have the webpack config do something to identify the "test files," like adding a suffix of -test to the test JS files.

CC: @robwise

[samandmoore]

@justin808 I was thinking that it would work fine because we can compile the files in the test env and read from the file system. But we could still use the dev server in development env. That just requires that the dev server files take precedence over the file system ones... which I'm not sure is the case. Hmm.