libuv update in 18.18.0 breaks webpack's thread-loader

[anomiex]

Version

v18.18.0

Platform

Linux abulia 6.5.0-1-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.5.3-1 (2023-09-13) x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

Starting in an empty directory,

1. Create the following files:
   • package.json:

     {
     	"dependencies": {
     		"thread-loader": "^4.0.2",
     		"webpack": "^5.88.2",
     		"webpack-cli": "^4.10.0"
     	}
     }

   • webpack.config.js:

     const path = require( 'path' );
     
     module.exports = {
         mode: 'production',
         entry: './index.js',
         output: {
             path: path.resolve( __dirname, 'dist' ),
             filename: 'foo.bundle.js',
         },
         module: {
             rules: [
                 {
                     test: /\.(js)$/,
                     exclude: /node_modules/,
                     use: [
                         {
                             loader: require.resolve( 'thread-loader' ),
                         },
                     ],
                 }
             ]
         },
     };
     

2. Run node -e 'console.log( "console.log( typeof {} );" ); for(let i=0; i<5000; i++){ console.log( "// we need a large file in order to trigger the bug" ); }' > index.js. Increase the "5000" if necessary, I don't know whether it depends on the system in some manner.
3. npm install
4. npm exec webpack

How often does it reproduce? Is there a required condition?

Always. No requirements I know of beyond what's described above and being on Linux.

What is the expected behavior? Why is that the expected behavior?

asset foo.bundle.js 22 bytes [emitted] [minimized] (name: main)
./index.js 254 KiB [built] [code generated]
webpack 5.88.2 compiled successfully in 376 ms

Because that's what happens when webpack runs successfully.

What do you see instead?

Process hangs before producing any output. CPU is not being used, processes seem to be waiting in epoll_pwait.

Additional information

It worked in 18.17.1. fb2b80f appears to be the first failing commit.

Setting UV_USE_IO_URING=0 works around the problem. Of course, that's undocumented and temporary so a real fix would be good.

I found this problem exists in 20.3.0 through 20.6.1 as well, but is not present in 20.7.0. A git bisect turned up 88ba79b as the commit that fixed it. Cherry-picking that commit onto 18.18.0 fixes it for me there too.

So my guess is that the new io_uring stuff started returning short writes when passed a lot of data, which WriteStream didn't handle correctly until 88ba79b fixed it.

See also webpack-contrib/thread-loader#191

[bnoordhuis]

The bug fix will be backported to v18.x automatically, it's just a matter of waiting until it shows up in a release.

[matthewp]

In Astro we had to pin to 18.17.1 after GitHub upgraded its CI to use 18.18. Not getting good stacktraces or failures, just seems to hang. Sounds similar to this issue.

[richardlau]

I found this problem exists in 20.3.0 through 20.6.1 as well, but is not present in 20.7.0. A git bisect turned up 88ba79b as the commit that fixed it. Cherry-picking that commit onto 18.18.0 fixes it for me there too.

I've cherry-picked 88ba79b (and a4928b0 which it depends on to land cleanly) onto v18.x-staging so it's ready for the next non-security Node.js 18 release. Tentatively we have a release planned for October, see nodejs/Release#737 for the plan, but bear in mind the plan is subject to change based on releaser availability.

FYI @nodejs/lts @nodejs/releasers

[dnalborczyk]

I'm wondering, since this appears to be a known bug (for a while?) in 20.x, and v18 is currently the only LTS line, shouldn't have the offending commit be either omitted from the v18.18.0 release, or the v18.18.0 release be post-poned while waiting for the fix, or a bugfix release being made now with v18.18.1, as opposed to be waiting for a 18.19.0 release sometime in October?

It's just a bit surprising, but I also don't have the entire context.

[simatec]

I think here the label would also have to be supplemented with Node20.
Node > 20.2.0 also has the bug

[simatec]

v18.18.1 fix the Problem...
In Node 20 the problem is still present

[anomiex]

I think here the label would also have to be supplemented with Node20. Node > 20.2.0 also has the bug

It was fixed in 20.7.0 in my testing.

[simatec]

In my testing is the same Bug in 20.7.0 and 20.8.0

[anomiex]

I just it tested again, 20.6.1 reproduce following the instructions here while 20.7.0 and 20.8.0 do not. Are you following the instructions on this bug, or are you doing something different?

[simatec]

For us, the error occurs in connection with writestream and CIFS mount

#50061

[bnoordhuis]

The fix for the issue reported by OP was released in v18.18.1 and v20.7.0. I think some of you may be talking about different-but-similar-looking bugs that are tracked elsewhere so I'll go ahead and close this report.

[bnoordhuis]

Oh, and @simatec, if you or anyone still experiences issues, then please open (or reopen) an issue but - and this is critical - include full steps to reproduce, without third-party code. That means no npm modules.

[simatec]

I think the problem still exists in Node 20.8.0. It has only been fixed in 18.18.1 so far. The last working version of Node 20 was 20.2.0

[simatec]

Oh, and @simatec, if you or anyone still experiences issues, then please open (or reopen) an issue but - and this is critical - include full steps to reproduce, without third-party code.

I had already done that, see issue in the link

#50061

[bnoordhuis]

Missing the critical bit though: steps to reproduce, see #50061 (comment).