can't close webpack-dev-server --progress with Ctrl + C

[danburzo]

• Operating System: macOS High Sierra (10.13.6)
• Node Version: v10.9.0
• NPM Version: 6.2.0
• webpack Version: 4.16.5
• webpack-dev-server Version: 3.1.5

• This is a bug
• This is a modification request

Code

Running the dev server with: yarn webpack-dev-server --progress. I'm not including webpack configs since seems to be reproducing on a variety of projects, so I assume it's something that happens regardless of the config? (Happy to provide more info otherwise).

Expected Behavior

Ctrl + C stops the dev server.

Actual Behavior

Ctrl + C does not do anything, dev server continues to compile modules.

May be a regression of #860, #1360?

[michael-ciniawsky]

Could you try with v3.1.6 (#1432) ?

[danburzo]

Woo-hoo! Missed that there's 3.1.6 available, works like a charm with it.

Oops, spoke too soon. It still reproduces on 3.1.6...

[qkdreyer]

Still an issue with v3.1.9

[SebT]

Having the issue on Linux and Mac with webpack-dev-server v3.1.8.

[alexander-akait]

Please create minimum reproducible test repo with steps

[superhawk610]

@SebT I've also had this issue, specifically when using the -theme react preset from the ReasonReact starter. I'm working on getting reproducible steps - it doesn't seem to happen in a dry environment, so I imagine it's some conflict with another file watcher that occurs after some period of use.

Edit: Alright, I've created a branch that consistently reproduces this here. To reproduce:

1. Clone and install dependencies.

yarn

2. Install bucklescript globally.

yarn global add bs-platform

3. Start bucklescript watcher.

yarn start

4. Start dev server.

yarn server

5. Touch a Reason source file, eg:

touch src/components/Stateful/Stateful.re

At this point, the dev server gets stuck at Compiling..., with the logs looking something like

yarn server
yarn run v1.10.1
$ webpack-dev-server
ℹ ｢wds｣: Project is running at http://localhost:8000/
ℹ ｢wds｣: webpack output is served from /home/superhawk610/code/the-completionist/build/
ℹ ｢wds｣: Content not from webpack is served from /home/superhawk610/code/the-completionist/build/
ℹ ｢wds｣: 404s will fallback to /index.html
ℹ ｢wdm｣: Hash: d9fc1cbd98cbce78242b
Version: webpack 4.23.1
Time: 469ms
Built at: 10/31/2018 1:31:25 AM
     Asset       Size  Chunks             Chunk Names
  Index.js    1.2 MiB    main  [emitted]  main
index.html  257 bytes          [emitted]  
Entrypoint main = Index.js
[0] multi (webpack)-dev-server/client?http://localhost:8000 ./src/Index.bs.js 40 bytes {main} [built]
[./node_modules/ansi-html/index.js] 4.16 KiB {main} [built]
[./node_modules/ansi-regex/index.js] 135 bytes {main} [built]
[./node_modules/loglevel/lib/loglevel.js] 7.68 KiB {main} [built]
[./node_modules/reason-react/src/ReactDOMRe.js] 2.6 KiB {main} [built]
[./node_modules/reason-react/src/ReasonReact.js] 22.2 KiB {main} [built]
[./node_modules/strip-ansi/index.js] 161 bytes {main} [built]
[./node_modules/url/url.js] 22.8 KiB {main} [built]
[./node_modules/webpack-dev-server/client/index.js?http://localhost:8000] (webpack)-dev-server/client?http://localhost:8000 7.78 KiB {main} [built]
[./node_modules/webpack-dev-server/client/overlay.js] (webpack)-dev-server/client/overlay.js 3.58 KiB {main} [built]
[./node_modules/webpack-dev-server/client/socket.js] (webpack)-dev-server/client/socket.js 1.05 KiB {main} [built]
[./node_modules/webpack/hot sync ^\.\/log$] (webpack)/hot sync nonrecursive ^\.\/log$ 170 bytes {main} [built]
[./node_modules/webpack/hot/emitter.js] (webpack)/hot/emitter.js 75 bytes {main} [built]
[./src/Index.bs.js] 417 bytes {main} [built]
[./src/components/App/App.bs.js] 1.57 KiB {main} [built]
    + 32 hidden modules
Child html-webpack-plugin for "index.html":
     1 asset
    Entrypoint undefined = index.html
    [./node_modules/html-webpack-plugin/lib/loader.js!./src/index.html] 466 bytes {0} [built]
    [./node_modules/lodash/lodash.js] 527 KiB {0} [built]
    [./node_modules/webpack/buildin/global.js] (webpack)/buildin/global.js 489 bytes {0} [built]
    [./node_modules/webpack/buildin/module.js] (webpack)/buildin/module.js 497 bytes {0} [built]
ℹ ｢wdm｣: Compiled successfully.
ℹ ｢wdm｣: Compiling...

Ctrl+C exits the process but leaves the server running

$ ps -ef | grep webpack
superha+ 23562 23551  0 01:31 pts/2    00:00:00 /bin/sh -c webpack-dev-server
superha+ 23563 23562 95 01:31 pts/2    00:06:22 node /home/superhawk610/code/the-completionist/node_modules/.bin/webpack-dev-server
superha+ 24114 22506  0 01:38 pts/1    00:00:00 grep --color=auto webpack

[superhawk610]

The error (in this case, at least) appears to occur when the dev server gets stuck compiling. Ctrl+C at this point doesn't actually end any processes - all 3 processes listed above remain active after sending SIGINT to the yarn process.

FWIW, I'm using VSCode with the Reason/OCaml extension.

Edit: Alright, this latest commit consistently causes webpack to freeze at Compiling... and orphan the dev server process on SIGINT. It's preventing me from any further development 🙁

Edit 2: After trying to replicate on a second machine, this seems to be a borked install as everything is working fine.

[dcorb]

One developer in our team is having the same issue (the rest of the team not). So it's hard to offer a reproducible test.
webpack-dev-server 3.1.1, Mac Sierra

[alexander-akait]

If somebody can reproducible test please post how here, we can fix it asap

[el-dot]

The error (in this case, at least) appears to occur when the dev server gets stuck compiling. Ctrl+C at this point doesn't actually end any processes - all 3 processes listed above remain active after sending SIGINT to the yarn process.

The same happens for me on Linux (up-to-date ArchLinux). Also it drains 100% of one CPU core.

[rhys-vdw]

Can confirm I'm still experiencing this at webpack-dev-server 3.1.10, macOS 10.13.6

cc @nfm confirm whether you're still experiencing this one?

[nfm]

Yes, this is still happening for me if I try to Ctrl-C when actively compiling. I can successfully Ctrl-C when WDS is idle though.

[alexander-akait]

Please create minimum reproducible test repo

[danburzo]

@evilebottnawi here you go: https://github.com/danburzo/webpack-dev-server-repro
(instructions in Readme)

[alexander-akait]

@danburzo What os i need to reproduce?

[danburzo]

In my initial report, I was on macOS High Sierra (10.13.6) when this happened. I am currently on macOS Mojave (10.14.1) and it still reproduces.

[rhys-vdw]

OS:

-> sw_vers
ProductName:	Mac OS X
ProductVersion:	10.13.6
BuildVersion:	17G3025

Using the above repo:

Every time the progress bar is duplicated I am pressing ^C. Also I need to press it to bring the prompt back at the end.

[rhys-vdw]

(Note that @nfm is experiencing this on Ubuntu so it's not just macOS)

[alexander-akait]

@danburzo @rhys-vdw Very thanks for repo/screenshot and feedback. I will try to find Mac OS system on this week and fix problem. Anyway feel free to investigate why it is happens, code for killing process here

webpack-dev-server/bin/webpack-dev-server.js

Line 48 in 927a2b3

signals.forEach((signal) => {

. Maybe signal have another value?

[alexander-akait]

@danburzo Awesome, reproduce problem on linux (Ubuntu) too

[alexander-akait]

One interesting thing - it is happens when you try to require('webpack'), if you change this on console.log all works fine

[alexander-akait]

Problem found, we use server.close before run process.exit (close all connections, watchers and etc stuff). server.close use webpackDevMiddleware.close method, what use watching.close (when you in not lazy mode - using webpack-dev-server), but watching.close wait until all was compiled.

What we need to fix all problem with CRTL+C:

1. Implement Watching.kill() method in webpack
2. Add kill option to webpackDevServer.close (by default false)
3. Use webpackDevServer.close(cb, true) in Server.close.

If somebody have time to help with this issue PRs welcome

[alexander-akait]

I guess there is a technical reason to this which might talk to you, isn't it?

Yes, we need watch these files, because you can add them late... But they should be closed very quickly

[sebdvd]

The slowness is coming from this line https://github.com/webpack/watchpack/blob/master/lib/watchEventSource.js#L75 where the NodeJS FSWatcher is closed. In my case, I count 1219 executions of FSWatcher.close, if I comment the line: it's fast 😿

FYI, I'm on macos 11.5.2 with node 14.17.6 and I tried with node 16.9.0 too.

[alexander-akait]

Can you debug this too https://github.com/webpack/watchpack/blob/master/lib/watchEventSource.js#L188?

[sebdvd]

yes I passed here https://github.com/webpack/watchpack/blob/master/lib/watchEventSource.js#L194 then https://github.com/webpack/watchpack/blob/master/lib/watchEventSource.js#L75

So I can now reproduce easily the slowness on my laptop:

const fs = require('fs');
const watchers = [];

for (let n = 0; n < 1200; n += 1) {
  watchers.push(fs.watch(process.cwd()));
}

console.time('close');
for (let w of watchers) {
  w.close();
}
console.timeEnd('close');

These fs.FSWatcher instances are slow to close (~4.7s on my laptop)

[alexander-akait]

Big thanks, we will investigate this in near future, but maybe problem deeply, i.e. in Node.js or maybe there are technical reasons for this...

[laverdet]

I'm working around this now by adding this to my webpack.config.js:

const { watch } = require('fs');

// Ctrl + C is super duper slow because it has to close a bunch of fs.watch handles. Let's just skip
// all that.
// https://github.com/webpack/webpack-dev-server/issues/1479
process.once('SIGINT', () => {
    Object.getPrototypeOf(watch('.')).close = () => {};
    return true;
});

[alexander-akait]

We will fix it in near future, anyway not graceful closing can corrupt file caches

[raine]

Checked watchpack's this.fileWatchers upon closing and for me there are 75k files watched. Many of which are in node_modules.

Any idea what's the point of watching those files and what is adding them? Thanks.

[alexander-akait]

Due resolving logic, I am thinking long closing is a bug in Node.js, What is os?

[raine]

I'm on macOS Catalina.

I added **/node_modules to watchOptions.ignored. That removed around half of the watchers and closing is faster.

Now that I'm checking the watched files in this.fileWatchers, I'm seeing that many files are watched with multiple extensions, even if they don't exist.

  ...
  "app/scripts/client/index.tsx", // <-- Only this actually exists in file system
  "app/scripts/client/index",
  "app/scripts/client/index.web.js",
  "app/scripts/client/index.js",
  "app/scripts/client/index.json",
  "app/scripts/client/index.ts",

Is this normal behavior? Seems to me like there are useless watchers but I don't understand this library.

[alexander-akait]

Yes, because you can add index.ts and we need to watch these adding, for example you have index.ts, then you want to use index.tsx, if you add them we need rerun compilation, same for other

[niksy]

Seeing same issue in Webpack 5.53.0, macOS Big Sur, M1, Node 12.22.5.

[alexander-akait]

I have idea how to improve this, there are two approach:

• one CTRL+C - gracefully shutting down
• second CTRL + C - force process closing

So you can chose wait or close immediately

[raine]

How realistic case is it that force exit corrupts webpack cache or something like that?

[alexander-akait]

In stage when webpack try to write caches, it is based on many metrics, hard to say, but I think in this case webpack should skip broken cache and try to create new

[laverdet]

How realistic case is it that force exit corrupts webpack cache or something like that?

I'd eat my hat if leaving a dangling watch handle corrupts any file cache. My hack above just leaves the watch handles open, it doesn't touch anything else. The operating system is more than capable of cleaning our mess up.

[alexander-akait]

@laverdet yep hack above will be corrupt cache, it just fix Node.js problems with slow closing files.

Here two problems:

1. Bug on our side, we don't close compiler - feat: gracefully and force shutdown #3880, also we should ask developer to close gracefully or force
2. Bug on Node.js side with slow file closing

[laverdet]

The offer on eating my hat still stands. I just can't imagine a scenario in which a file would be corrupted by a dangling inotify handle. In fact, I'd go so far to say that any such corruption would warrant a CVE in the Linux kernel followed by a hushed patch by Google's top minds.

On the other hand calling process.exit will almost definitely cause partially written files to be flushed to the filesystem since the process terminates with open descriptors (of the mode == "w" kind).

I have a great deal of respect for your contributions to Webpack but I do think that leaving the watch handles open is a safer workaround in this case.

[alexander-akait]

There is another problems - developers middlewares, we can't know it is safe to close immediately or we need to do graceful shutdown, same for custom client/web socket server implementations, I think we need report problem with slow closing to Node.js. Anyway in the next release double pressing CTRL + C will close immediately process, so developer can choose what is better

[alexnix]

Hi, I am having an issue with might be related.

I start WebpackDevServer from a script, but when I Cmd+C to close it, the script stops immediately (at least the terminal looks like it did), but then not only additional output appears from webpack (the line with "gracefully shutdown") but also, if i try to input characters such as up/down arrows they appear as [OB^[OB.

I am wondering if i need to wait for webpack in anyway from within my script.

I tried but did not work.

process.on('SIGINT', () =>{
    devServer.stop().then(() => {
        process.exit(0);
    });
});

Maybe it is worth mentioning that if i send the SIGINT from htop or kill it does works as expected: it prints the "gracefully shutdown message" then waits for the file watches to close, and even prints a "done" afterwards.

In your opinion, is this related to the webpack - script interaction, or is it a terminal/zsh related issue (given that SIGINT is handled correct when coming from a source different than Cmd+C) ?

[laverdet]

@alexnix what you are experiencing is a common issue when terminating processes which use vterm escape sequences, for example colors in Webpack's output. You can fix your terminal by running reset or by just opening a new terminal.

[alexnix]

Thanks for the clarification.
So you are basically saying this issue is not related to my code, but rather some common issue with terminals ?

[laverdet]

Right, it's totally normal. Just remember the reset command for situations like this.

[sekoyo]

The only way it exits for me is if I run webpack directly from CLI (npx or package "scripts"). If I put it in a shell script it prints "Gracefully shutting down" but then never does, regardless of how many watchers there are. Also tried things like pkill -P $$ on EXIT but no avail (using 4.15.1 and webpack5).

Another CMD+C does exit, but sometimes the dev server is still running in the background and have to manually kill processes on the port, especially if there are a lot of watchers.

[jtbandes]

Adding watchOptions: { ignored: "**/node_modules/**" } made shutdown much faster for me (from several seconds to nearly instant).

I believe I figured out the reason that "first ^C graceful shutdown, second ^C force close" was not actually working — it's because the server was running fileWatcher.close() in a tight loop, which prevented the SIGINT handler from executing a second time. If these were closed asynchronously, the handler would be able to run again and call process.exit().