worker: initial implementation #20876

addaleax · 2018-05-22T01:12:45Z

Hi everyone! 👋

This PR adds threading support for to Node.js. I realize that this is not exactly a small PR and is going to take a while to review, so: I appreciate comments, questions (any kind, as long as it’s somewhat related 😺), partial reviews and all other feedback from anybody, not just Node.js core collaborators.

The super-high-level description of the implementation here is that Workers can share and transfer memory, but not JS objects (they have to be cloned for transferring), and not yet handles like network sockets.

FAQ

See https://gist.github.com/benjamingr/3d5e86e2fb8ae4abe2ab98ffe4758665

Example usage

const { Worker, isMainThread, postMessage, workerData } = require('worker_threads');

if (isMainThread) {
  module.exports = async function parseJSAsync(script) {
    return new Promise((resolve, reject) => {
      const worker = new Worker(__filename, {
        workerData: script
      });
      worker.on('message', resolve);
      worker.on('error', reject);
      worker.on('exit', (code) => {
        if (code !== 0)
          reject(new Error(`Worker stopped with exit code ${code}`));
      });
    });
  };
} else {
  const { parse } = require('some-js-parsing-library');
  const script = workerData;
  postMessage(parse(script));
}

Feature set

The communication between threads largely builds on the MessageChannel Web API. Transferring ArrayBuffers and sharing memory through SharedArrayBuffers is supported.
Almost the entire Node.js core API is require()able or importable.

Some notable differences:

stdio streams may be captured by the parent thread.
Some functions, e.g. process.chdir() don’t exist in worker threads.
Native addons are not loadable from worker threads (yet).
No inspector support (yet).

(Keep in mind that PRs can change significantly based on reviews.)

Comparison with `child_process` and `cluster`

Workers are conceptually very similar to child_process and cluster.
Some of the key differences are:

Communication between Workers is different: Unlike child_process IPC, we don’t use JSON, but rather do the same thing that postMessage() does in browsers.
- This isn’t necessarily faster, although it can be and there might be more room for optimization. (Keep in mind how long JSON has been around and how much work has therefore been put into making it fast.)
- The serialized data doesn’t actually need to leave the process, so overall there’s less overhead in communication involved.
- Memory in the form of typed arrays can be transferred or shared between Workers and/or the main thread, which enables really fast communication for specific use cases.
- Handles, like network sockets, can not be transferred or shared (yet).
There are some limitations on the usable API within workers, since parts of it (e.g. process.chdir()) affect per-process state, loading native addons, etc.
Each workers have its own event loop, but some of the resources are shared between workers (e.g. the libuv thread pool for file system work)

Benchmarks

    $ ./node benchmark/cluster/echo.js
    cluster/echo.js n=100000 sendsPerBroadcast=1 payload="string" workers=1: 33,647.30473442063
    cluster/echo.js n=100000 sendsPerBroadcast=10 payload="string" workers=1: 12,927.907405288383
    cluster/echo.js n=100000 sendsPerBroadcast=1 payload="object" workers=1: 28,496.37373941151
    cluster/echo.js n=100000 sendsPerBroadcast=10 payload="object" workers=1: 8,975.53747186485
    $ ./node --experimental-worker benchmark/worker/echo.js
    worker/echo.js n=100000 sendsPerBroadcast=1 payload="string" workers=1: 88,044.32902365089
    worker/echo.js n=100000 sendsPerBroadcast=10 payload="string" workers=1: 39,873.33697018837
    worker/echo.js n=100000 sendsPerBroadcast=1 payload="object" workers=1: 64,451.29132425621
    worker/echo.js n=100000 sendsPerBroadcast=10 payload="object" workers=1: 22,325.635443739284

A caveat here is that startup performance for Workers using this model is still relatively slow (I don’t have exact numbers, but there’s definitely overhead).

Regarding semverness:

The only breaking change here is the introduction of a new top-level module. The name is currently worker, this is not under a scope as suggested in nodejs/TSC#389. It seems like the most natural name for this by far.

I’ve reached out to the owner of the worker module on npm, who declined to provide the name for this purpose – the package has 57 downloads/week, so whether we consider this semver-major because of that is probably a judgement call.

Alternatively, I’d suggest using workers – it’s not quite what we’re used to in core (e.g. child_process), but the corresponding npm package is essentially just a placeholder.

Acknowledgements

People I’d like to thank for their code, comments and reviews for this work in its original form, in no particular order:

… and finally @petkaantonov for a lot of inspiration and the ability to compare with previous work on this topic.

Individual commits

src: cleanup per-isolate state on platform on isolate unregister

Clean up once all references to an Isolate* are gone from the
NodePlatform, rather than waiting for the PerIsolatePlatformData
struct to be deleted since there may be cyclic references between
that struct and the individual tasks.

src: fix MallocedBuffer move assignment operator

src: break out of timers loop if `!can_call_into_js()`

Otherwise, this turns into an infinite loop.

src: simplify handle closing

Remove one extra closing state and use a smart pointer for
deleting HandleWraps.

worker: implement `MessagePort` and `MessageChannel`

Implement MessagePort and MessageChannel along the lines of
the DOM classes of the same names. MessagePorts initially
support transferring only ArrayBuffers.

worker: support MessagePort passing in messages

Support passing MessagePort instances through other MessagePorts,
as expected by the MessagePort spec.

worker: add `SharedArrayBuffer` sharing

Logic is added to the MessagePort mechanism that
attaches hidden objects to those instances when they are transferred
that track their lifetime and maintain a reference count, to make
sure that memory is freed at the appropriate times.

src: add Env::profiler_idle_notifier_started()

src: move DeleteFnPtr into util.h

This is more generally useful than just in a crypto context.

worker: initial implementation

Implement multi-threading support for most of the API.

test: add test against unsupported worker features

worker: restrict supported extensions

Only allow .js and .mjs extensions to provide future-proofing
for file type detection.

src: enable stdio for workers

Provide stdin, stdout and stderr options for the Worker
constructor, and make these available to the worker thread
under their usual names.

The default for stdin is an empty stream, the default for
stdout and stderr is redirecting to the parent thread’s
corresponding stdio streams.

benchmark: port cluster/echo to worker

worker: improve error (de)serialization

Rather than passing errors using some sort of string representation,
do a best effort for faithful serialization/deserialization of
uncaught exception objects.

test,tools: enable running tests under workers

Enable running tests inside workers by passing --worker
to tools/test.py. A number of tests are marked as skipped,
or have been slightly altered to fit the different environment.

Other work

I know that teams from Microsoft (/cc @fs-eire @helloshuangzi) and Alibaba (/cc @aaronleeatali) have been working on forms of multithreading that have higher degrees of interaction between threads, such as sharing code and JS objects. I’d love if you could take a look at this PR and see how well it aligns with your own work, and what conflicts there might be. (From what I’ve seen of the other code, I’m actually quite optimistic that this PR is just going to help everybody.)

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

devsnek · 2018-05-22T01:21:57Z

i'm so heckin' happy to see this 🎉 🎉 🎉 🎉

regarding the whole name thing, why not use @nodejs/workers? we're pratically being shoved into it due to the owner of workers not wanting to give that name up, and we've been looking for excuses to namespace builtin modules anyway.

devsnek · 2018-05-22T01:23:09Z

doc/api/process.md

@@ -1204,6 +1213,8 @@ console.log(process.getgroups());         // [ 27, 30, 46, 1000 ]
 This function is only available on POSIX platforms (i.e. not Windows or
 Android).

+This feature is not available in [`Worker`][] threads.


maybe we should set up some sort of doc macro for this so that we can keep them all in sync if we decide to change the formatting of it.

Do we know how to do that?

devsnek · 2018-05-22T01:26:29Z

lib/worker.js

+
+if (!process.binding('config').experimentalWorker) {
+  // TODO(addaleax): Is this the right way to do this?
+  // eslint-disable-next-line no-restricted-syntax


i would try and gate it in the nativemodule loader instead

tools/test.py

@@ -1375,6 +1375,8 @@ def BuildOptions():
      help="Expect test cases to fail", default=False, action="store_true")
  result.add_option("--valgrind", help="Run tests through valgrind",
      default=False, action="store_true")
+  result.add_option("--worker", help="Run parallel tests inside a worker context",


Trott · 2018-05-22T04:06:11Z

doc/api/errors.md

+<a id="ERR_CLOSED_MESSAGE_PORT"></a>
+### ERR_CLOSED_MESSAGE_PORT
+
+Used when there was an attempt to use a `MessagePort` instance in a closed


Nit: Remove Used when.

Trott · 2018-05-22T04:06:36Z

doc/api/errors.md

+<a id="ERR_MISSING_PLATFORM_FOR_WORKER"></a>
+### ERR_MISSING_PLATFORM_FOR_WORKER
+
+The V8 platform used by this instance of Node does not support creating Workers.


Nit: Node -> Node.js

Trott · 2018-05-22T04:07:34Z

doc/api/errors.md

+<a id="ERR_WORKER_DOMAIN"></a>
+### ERR_WORKER_DOMAIN
+
+Used when trying to access the `domain` module inside of a worker thread.


Remove Used when and reword:

The `domain` module cannot be used inside of a worker thread.

This was one of two outdated error entries anyway. :) It might be cool if our linter could check that entries in this markdown file are also present in lib/internal/errors.h or src/node_errors.h…

Trott · 2018-05-22T04:07:48Z

doc/api/errors.md

+<a id="ERR_WORKER_NEED_ABSOLUTE_PATH"></a>
+### ERR_WORKER_NEED_ABSOLUTE_PATH
+
+Used when the path for the main script of a worker is not an absolute path.


Remove Used when

Trott · 2018-05-22T04:08:12Z

doc/api/errors.md

+<a id="ERR_WORKER_OUT_OF_MEMORY"></a>
+### ERR_WORKER_OUT_OF_MEMORY
+
+Used when a worker hits its memory limit.


A worker has exceeded its memory limit.

Trott · 2018-05-22T04:08:24Z

doc/api/errors.md

+Used when a worker hits its memory limit.
+
+<a id="ERR_WORKER_UNAVAILABLE_FEATURE"></a>
+### ERR_WORKER_UNAVAILABLE_FEATURE


Missing description?

Trott · 2018-05-22T04:09:24Z

doc/api/errors.md

+<a id="ERR_WORKER_UNSERIALIZABLE_ERROR"></a>
+### ERR_WORKER_UNSERIALIZABLE_ERROR
+
+Used when all attempts at serializing an uncaught exception from a worker fail.


All attempts to serialize an uncaught exception from a worker failed.

...or perhaps...

All attempts to serialize an uncaught worker exception failed.

Trott · 2018-05-22T04:10:22Z

doc/api/errors.md

+<a id="ERR_WORKER_UNSUPPORTED_EXTENSION"></a>
+### ERR_WORKER_UNSUPPORTED_EXTENSION
+
+Used when the pathname used for the main script of a worker has an


Remove Used when

Trott · 2018-05-22T04:12:01Z

doc/api/process.md

@@ -918,6 +922,8 @@ console.log(process.env.test);
 // => 1
 ```

+*Note*: `process.env` is read-only in [`Worker`][] threads.


My opinion only but I prefer we leave out *Note*: in nearly all cases.

I believe *Note*: prefixes were eradicated earlier from the docs.

Trott · 2018-05-22T04:12:08Z

doc/api/process.md

@@ -1030,6 +1036,9 @@ If it is necessary to terminate the Node.js process due to an error condition,
 throwing an *uncaught* error and allowing the process to terminate accordingly
 is safer than calling `process.exit()`.

+*Note*: in [`Worker`][] threads, this function stops the current thread rather


My opinion only but I prefer we leave out *Note*: in nearly all cases.

Trott · 2018-05-22T04:17:52Z

doc/api/worker.md

+added: REPLACEME
+-->
+
+Opposite of `unref`, calling `ref` on a previously `unref`d worker will *not*


Nit: Add parentheses, so unfref() and ref(). Also elsewhere in this doc.

Nit: Change comma to period on this line.

Nit: We use `unref`ed rather than `unref'd` in existing documentation so use that here and in line 97, 105, and anywhere else for consistency.

Trott · 2018-05-22T04:18:14Z

doc/api/worker.md

+
+Opposite of `unref`, calling `ref` on a previously `unref`d worker will *not*
+let the program exit if it's the only active handle left (the default behavior).
+If the worker is `ref`d calling `ref` again will have no effect.


Add comma before "calling".

Trott · 2018-05-22T04:19:54Z

doc/api/worker.md

+* Returns: {undefined}
+
+Disables further sending of messages on either side of the connection.
+This this method can be called once you know that no further communication


Typo: "This this"

Trott · 2018-05-22T04:20:30Z

doc/api/worker.md

+behind this API, see the [serialization API of the `v8` module][v8.serdes].
+
+*Note*: Because the object cloning uses the structured clone algorithm,
+non-enumberable properties, property accessors, and object prototypes are


Typo: non-enumberable -> non-enumerable

Trott · 2018-05-22T04:20:59Z

doc/api/worker.md

+- [`process.chdir()`][] as well as `process` methods that set group or user ids
+  are not available.
+- [`process.env`][] is a read-only reference to the environment variables.
+- [`process.title`][] can not be modified.


can not -> cannot

Trott · 2018-05-22T04:21:17Z

doc/api/worker.md

+child thread.
+
+To create custom messaging channels (which is encouraged over using the default
+global channel because it facilitates seperation of concerns), users can create


Typo: seperation -> separation

Trott · 2018-05-22T04:21:42Z

doc/api/worker.md

+  * data {any} Any JavaScript value that will be cloned and made
+    available as [`require('worker').workerData`][]. The cloning will occur as
+    described in the [HTML structured clone algorithm][], and an error will be
+    thrown if the object can not be cloned (e.g. because it contains


can not -> cannot

Trott · 2018-05-22T04:21:58Z

doc/api/worker.md

+-->
+
+The `'error'` event is emitted if the worker thread throws an uncaught
+expection. In that case, the worker will be terminated.


Typo: expection -> exception

idibidiart · 2018-06-28T16:36:23Z

As the author noted, this is work in progress and still in experimental stage ... you can certainly have more threads than much heavier nodejs processes/instances on any given machine... threads give you shared memory with its pros and cons... without threads, nodejs is designed for I/O bound work like fetching something from db... with threads it can do cpu bound work (potentially in parallel if you design it that way) outside the main thread so node can remain responsive to I/O, while processing in background... but even threads are limited by the number of cores and memory available ...

ChALkeR · 2018-07-23T15:56:14Z

@addaleax This landed without proper error documentation.
ERR_TRANSFERRING_EXTERNALIZED_SHAREDARRAYBUFFER is undocumented, but is implemented and used.

benjamingr · 2018-07-23T16:06:49Z

@ChALkeR good catch #21947

iravishah · 2018-09-24T13:29:35Z

When can i expect LTS of thread? Eagerly waiting...

jasnell · 2018-09-24T13:39:11Z

Workers are likely to be experimental for a while. They'll be in 10.x after it goes LTS but still as experimental.

iravishah · 2018-09-24T16:26:39Z

Thanks 👍

Clean up once all references to an `Isolate*` are gone from the `NodePlatform`, rather than waiting for the `PerIsolatePlatformData` struct to be deleted since there may be cyclic references between that struct and the individual tasks. PR-URL: nodejs#20876 Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Shingo Inoue <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Tiancheng "Timothy" Gu <[email protected]> Reviewed-By: John-David Dalton <[email protected]> Reviewed-By: Gus Caplan <[email protected]>

Currently the following compiler warnings are generated: In file included from ../src/node_platform.cc:1: ../src/node_platform.h:83:16: warning: private field 'isolate_' is not used [-Wunused-private-field] v8::Isolate* isolate_; ^ 1 warning generated. This commit removes these unused private member. PR-URL: nodejs#20876 Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Shingo Inoue <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Tiancheng "Timothy" Gu <[email protected]> Reviewed-By: John-David Dalton <[email protected]> Reviewed-By: Gus Caplan <[email protected]>

The current documentation is incorrect in that it says "A single instance of Node.js runs in a single thread," which is not true due to the addition of worker threads. This patch removes the incorrect statement and instead suggests that applications consider using worker threads when process isolation is not needed. Refs: nodejs#20876

The current documentation is incorrect in that it says "A single instance of Node.js runs in a single thread," which is not true due to the addition of worker threads. This patch removes the incorrect statement and instead suggests that applications consider using worker threads when process isolation is not needed. Refs: #20876 PR-URL: #41616 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Mestery <[email protected]>

The current documentation is incorrect in that it says "A single instance of Node.js runs in a single thread," which is not true due to the addition of worker threads. This patch removes the incorrect statement and instead suggests that applications consider using worker threads when process isolation is not needed. Refs: nodejs#20876 PR-URL: nodejs#41616 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Mestery <[email protected]>

The current documentation is incorrect in that it says "A single instance of Node.js runs in a single thread," which is not true due to the addition of worker threads. This patch removes the incorrect statement and instead suggests that applications consider using worker threads when process isolation is not needed. Refs: #20876 PR-URL: #41616 Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Mestery <[email protected]>

nodejs-github-bot added the lib / src Issues and PRs related to general changes in the lib or src directory. label May 22, 2018

This comment has been minimized.

Sign in to view

devsnek reviewed May 22, 2018

View reviewed changes

Trott reviewed May 22, 2018

View reviewed changes

larshp mentioned this pull request Jun 27, 2018

multithreading: node, threads vs cluster abaplint/abaplint#216

Open

addaleax mentioned this pull request Jun 28, 2018

allow hiding of comments (Moderation Policy udpate) nodejs/admin#172

Merged

7rulnik mentioned this pull request Jul 1, 2018

WIP: Native workers webpack-contrib/thread-loader#29

Closed

GretaCB mentioned this pull request Jul 5, 2018

[FUTURE] Write a comparison benchmark using worker_threads module mapbox/node-cpp-skel#143

Open

MylesBorins added dont-land-on-v6.x labels Aug 10, 2018

davisjam mentioned this pull request Sep 3, 2018

WIP: Pluggable thread pool #22631

Closed

4 tasks

justrhysism mentioned this pull request Nov 22, 2018

Compile to WebAssembly sass/node-sass#2011

Open

aschaeffer mentioned this pull request Oct 6, 2019

Evaluate if embedding Node.js as a static library is easier than embedding V8 inexorgame-obsolete/entity-system-inactive#129

Open

tniessen mentioned this pull request Jan 20, 2022

doc: suggest worker threads in cluster docs #41616

Merged

skyclouds2001 mentioned this pull request Nov 9, 2024

Update nodejs data for Worker mdn/browser-compat-data#25012

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

worker: initial implementation #20876

worker: initial implementation #20876

addaleax commented May 22, 2018 •

edited

Loading

This comment has been minimized.

devsnek commented May 22, 2018

devsnek May 22, 2018

addaleax May 22, 2018

This comment was marked as resolved.

This comment was marked as resolved.

devsnek May 22, 2018

This comment was marked as resolved.

Trott May 22, 2018

Trott May 22, 2018

Trott May 22, 2018

addaleax May 22, 2018

Trott May 22, 2018

Trott May 22, 2018

Trott May 22, 2018

Trott May 22, 2018

Trott May 22, 2018

Trott May 22, 2018

ChALkeR May 22, 2018

Trott May 22, 2018

Trott May 22, 2018 •

edited

Loading

Trott May 22, 2018

Trott May 22, 2018

Trott May 22, 2018

Trott May 22, 2018

Trott May 22, 2018

Trott May 22, 2018

Trott May 22, 2018

idibidiart commented Jun 28, 2018 •

edited

Loading

ChALkeR commented Jul 23, 2018

benjamingr commented Jul 23, 2018 •

edited

Loading

iravishah commented Sep 24, 2018

jasnell commented Sep 24, 2018

iravishah commented Sep 24, 2018

worker: initial implementation #20876

worker: initial implementation #20876

Conversation

addaleax commented May 22, 2018 • edited Loading

FAQ

Example usage

Feature set

Comparison with child_process and cluster

Benchmarks

Regarding semverness:

Acknowledgements

Individual commits

src: cleanup per-isolate state on platform on isolate unregister

src: fix MallocedBuffer move assignment operator

src: break out of timers loop if !can_call_into_js()

src: simplify handle closing

worker: implement MessagePort and MessageChannel

worker: support MessagePort passing in messages

worker: add SharedArrayBuffer sharing

src: add Env::profiler_idle_notifier_started()

src: move DeleteFnPtr into util.h

worker: initial implementation

test: add test against unsupported worker features

worker: restrict supported extensions

src: enable stdio for workers

benchmark: port cluster/echo to worker

worker: improve error (de)serialization

test,tools: enable running tests under workers

Other work

Checklist

This comment has been minimized.

devsnek commented May 22, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as resolved.

This comment was marked as resolved.

Choose a reason for hiding this comment

This comment was marked as resolved.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Trott May 22, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

idibidiart commented Jun 28, 2018 • edited Loading

ChALkeR commented Jul 23, 2018

benjamingr commented Jul 23, 2018 • edited Loading

iravishah commented Sep 24, 2018

jasnell commented Sep 24, 2018

iravishah commented Sep 24, 2018

addaleax commented May 22, 2018 •

edited

Loading

Comparison with `child_process` and `cluster`

src: break out of timers loop if `!can_call_into_js()`

worker: implement `MessagePort` and `MessageChannel`

worker: add `SharedArrayBuffer` sharing

Trott May 22, 2018 •

edited

Loading

idibidiart commented Jun 28, 2018 •

edited

Loading

benjamingr commented Jul 23, 2018 •

edited

Loading