Skip to content
This repository has been archived by the owner on Apr 22, 2023. It is now read-only.

Cluster 2.0 #2038

Closed
wants to merge 155 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
155 commits
Select commit Hold shift + click to select a range
256ec65
send tcp self to cluster._getServer
AndreasMadsen Nov 7, 2011
0afde5b
renamed function from _startWorker to _setupWorker
AndreasMadsen Nov 7, 2011
50820e7
added testcase for cluster events
AndreasMadsen Nov 7, 2011
77e3f99
eachWorker: made public; startMaster: made public, renamed to setupMa…
AndreasMadsen Nov 7, 2011
c97c232
Documented cluster events
AndreasMadsen Nov 7, 2011
8405853
Documented .fork and .autoFork
AndreasMadsen Nov 7, 2011
c96a8fb
Documented cluster.worker.send and cluster.worker.on('message');
AndreasMadsen Nov 7, 2011
e3d4d9c
Ups, removed event title
AndreasMadsen Nov 7, 2011
5fea68b
Documented worker object
AndreasMadsen Nov 7, 2011
5a2b365
Documented setupMaster
AndreasMadsen Nov 7, 2011
492a261
Now fit the documentation pattern
AndreasMadsen Nov 7, 2011
a37e413
Created a worker class
AndreasMadsen Nov 8, 2011
1681943
rearrange log
AndreasMadsen Nov 8, 2011
5c1d481
sync API with documentation
AndreasMadsen Nov 8, 2011
f0ba47e
allow env object to be passed to cluster.fork()
AndreasMadsen Nov 8, 2011
4f9c938
fit test after removing chain
AndreasMadsen Nov 8, 2011
c2a074e
Dramatically simplifyed code, and made Worker class usefull in both m…
AndreasMadsen Nov 8, 2011
d79ccef
fixed bug, and set state to dead when worker die
AndreasMadsen Nov 8, 2011
29647e8
updated documentation
AndreasMadsen Nov 14, 2011
25f5793
added disconnect method and event
AndreasMadsen Nov 14, 2011
1f1153f
testcase for disconnect method, could be better
AndreasMadsen Nov 14, 2011
6192fb4
ups, reactivated uncaughtException
AndreasMadsen Nov 14, 2011
b766a21
jslint
AndreasMadsen Nov 14, 2011
026fe30
jslint
AndreasMadsen Nov 14, 2011
cb317ff
Cluster should be silent
AndreasMadsen Nov 14, 2011
04644f5
made cluster silent
AndreasMadsen Nov 14, 2011
a0ec887
workers can be killed and disconnected
AndreasMadsen Nov 14, 2011
bb766cf
master support SIGINT, SIGTERM, SIGQUIT
AndreasMadsen Nov 16, 2011
e4ea5cd
no longer need uncaught exception handling
AndreasMadsen Nov 16, 2011
1d2cf17
made isMaster and isWorker protected
AndreasMadsen Nov 16, 2011
df583e2
updated to ES5
AndreasMadsen Nov 16, 2011
666f3c7
renamed workerID til uniqueID; introduced workerID as a cpuID when us…
AndreasMadsen Nov 19, 2011
5f45e56
bug fix
AndreasMadsen Nov 19, 2011
9a3cf97
jslint
AndreasMadsen Nov 19, 2011
20e88ff
detect citical worker error loop
AndreasMadsen Nov 19, 2011
a6b87e1
allow user to run .autoFork multiply times
AndreasMadsen Nov 19, 2011
b63c420
updated documentation
AndreasMadsen Nov 19, 2011
3375ab3
Made the Worker constructor public
AndreasMadsen Nov 21, 2011
2ac25c0
removed old testcases
AndreasMadsen Nov 21, 2011
d10ad89
added online and listening event on worker object
AndreasMadsen Nov 22, 2011
c1532c9
updated documentation
AndreasMadsen Nov 22, 2011
29bf1c7
fixed callback related bug
AndreasMadsen Nov 22, 2011
4e60e29
comment fix
AndreasMadsen Nov 22, 2011
03eb277
rewriten IPC system to support no-object messages
AndreasMadsen Nov 22, 2011
05321a2
fixed bugs after IPC rewrite
AndreasMadsen Nov 22, 2011
7db43b0
bug fix - detected by auto-fork testcase
AndreasMadsen Nov 24, 2011
bef7153
fixed quick kill bug
AndreasMadsen Nov 25, 2011
ccf1077
jslint
AndreasMadsen Nov 25, 2011
b9a9c5a
fixed assert error - detected by throw testcase
AndreasMadsen Nov 26, 2011
d2ee4c4
added silent worker option
AndreasMadsen Nov 26, 2011
d67b570
documented the silent option
AndreasMadsen Nov 26, 2011
387b1f5
updated testcases
AndreasMadsen Nov 26, 2011
873877b
merged 'v0.6' intro AndreasMadsen/master
AndreasMadsen Nov 26, 2011
9f08551
removed testcase from v0.6 merge
AndreasMadsen Nov 26, 2011
e453d17
comment fix
AndreasMadsen Nov 27, 2011
b4dc4f5
added simple process watch module
AndreasMadsen Nov 27, 2011
3051449
added simple progress tracker module
AndreasMadsen Nov 27, 2011
c7b6a18
upated testcases
AndreasMadsen Nov 27, 2011
70f1c97
isolated exit from disconnect
AndreasMadsen Nov 27, 2011
b70b968
updated documentation
AndreasMadsen Nov 27, 2011
af8c582
updated and added testcases
AndreasMadsen Dec 2, 2011
989bb2f
updated and added testcases
AndreasMadsen Dec 2, 2011
12fadf2
updated and added testcases
AndreasMadsen Dec 2, 2011
8091540
jslint
AndreasMadsen Dec 2, 2011
26ae1a9
fixed documentation format
AndreasMadsen Dec 3, 2011
77133bd
fixed documentation format
AndreasMadsen Dec 3, 2011
920f9a1
fix documentation spell issues
AndreasMadsen Dec 3, 2011
f808a3a
added zero-downtime restart method
AndreasMadsen Dec 7, 2011
e86bae1
updated documentation
AndreasMadsen Dec 7, 2011
c983388
updated testcases
AndreasMadsen Dec 7, 2011
ffd1b18
updated testcases
AndreasMadsen Dec 8, 2011
fbcc88e
worker.restart do now work in workers too
AndreasMadsen Dec 8, 2011
0ae9412
destroy, disconnect, and restart can only be called from master now
AndreasMadsen Dec 8, 2011
de12e9d
updated testcases
AndreasMadsen Dec 8, 2011
0505d12
renamed exit to death and kill to destroy
AndreasMadsen Dec 16, 2011
123f069
updated documentation
AndreasMadsen Dec 16, 2011
18616a4
changed cluster.workers from array to object
AndreasMadsen Dec 16, 2011
29c147f
changed assert message
AndreasMadsen Dec 16, 2011
60c3915
added setup event and settings object
AndreasMadsen Dec 16, 2011
6cec470
updated documentation
AndreasMadsen Dec 16, 2011
cd1a6e1
bug fix
AndreasMadsen Dec 16, 2011
b2b407f
updated testcase setup-master
AndreasMadsen Dec 16, 2011
e5902f7
better channel closeing
AndreasMadsen Dec 17, 2011
441cb17
updated testcases
AndreasMadsen Dec 17, 2011
52cb324
jslinting
AndreasMadsen Dec 17, 2011
9bb2142
merged 'master' intro AndreasMadsen/master
AndreasMadsen Dec 19, 2011
5fd6d6a
rewirte processWatch
AndreasMadsen Dec 19, 2011
cf7ccba
renamed autoFork testcase
AndreasMadsen Dec 19, 2011
a642260
renamed fork restart testcase
AndreasMadsen Dec 19, 2011
9d73a03
renamed autoFork restart testcase
AndreasMadsen Dec 19, 2011
769f629
cleanup
AndreasMadsen Dec 19, 2011
3454ab9
improved error handling
AndreasMadsen Dec 19, 2011
c0f0ee8
rewrite testcase
AndreasMadsen Dec 19, 2011
4345d5b
improved quick deadth handling
AndreasMadsen Dec 19, 2011
9b8a2a8
cleanup
AndreasMadsen Dec 19, 2011
218ff06
renamed fork-args to fork-env
AndreasMadsen Dec 19, 2011
b12a9cd
env parse now match step 3
AndreasMadsen Dec 19, 2011
108a386
updated testcase
AndreasMadsen Dec 19, 2011
286b76d
remove console.log
AndreasMadsen Dec 19, 2011
3e9f3b9
hotfix extendObject
AndreasMadsen Dec 19, 2011
11a656f
simplify testcase
AndreasMadsen Dec 20, 2011
e8ba702
test against UNIQUE_ID not WORKER_ID
AndreasMadsen Dec 20, 2011
2585160
merged master intro AndreasMadsen/master
AndreasMadsen Dec 20, 2011
6a03c7a
updated documentation
AndreasMadsen Dec 20, 2011
ce8a35f
updated documentation
AndreasMadsen Dec 20, 2011
7494c50
jslinting
AndreasMadsen Dec 20, 2011
7056c1a
sync with step 4
AndreasMadsen Dec 26, 2011
904d8a3
bug fix
AndreasMadsen Dec 26, 2011
29d70bd
ignore non-object values in extendObject
AndreasMadsen Dec 26, 2011
7b2d931
jslint
AndreasMadsen Dec 26, 2011
72d3a26
fix eachWorker renameing
AndreasMadsen Dec 26, 2011
36f8343
fix cluster constructor name
AndreasMadsen Dec 26, 2011
c20c683
fix comment
AndreasMadsen Dec 26, 2011
e2de1e7
fixed testcase
AndreasMadsen Dec 26, 2011
03f4e68
fix eachWorker exameple
AndreasMadsen Dec 26, 2011
50028af
moved forkMode test
AndreasMadsen Dec 27, 2011
7c0b45b
removed testcaes after unprotecting consts
AndreasMadsen Dec 27, 2011
db6005f
dosn't sleep when an error is found
AndreasMadsen Dec 27, 2011
ce3ee69
bug fix : detected by worker-restart
AndreasMadsen Dec 27, 2011
276e115
updated testcase
AndreasMadsen Dec 27, 2011
ebcd266
removed callback support from public API
AndreasMadsen Dec 28, 2011
19fd469
updated testcase after removing callback support
AndreasMadsen Dec 28, 2011
b8bfaa6
simplified testcase
AndreasMadsen Dec 28, 2011
4b54543
updated documentation
AndreasMadsen Dec 28, 2011
481a55e
added comment spaceing
AndreasMadsen Dec 28, 2011
216c29f
test states
AndreasMadsen Dec 28, 2011
f043179
bug fix - detected by restart-autoFork
AndreasMadsen Dec 28, 2011
d87690b
remove console.log
AndreasMadsen Dec 28, 2011
0ea6a5d
don't use fast shoutdown here
AndreasMadsen Dec 28, 2011
bf90992
cleanup
AndreasMadsen Dec 28, 2011
cc09ac1
rewrote testcase
AndreasMadsen Dec 28, 2011
04fa43c
rewrote IPC to not use a wrapper object
AndreasMadsen Dec 28, 2011
902f91b
filter out internal messages in core
AndreasMadsen Dec 29, 2011
874af15
update PIC to use inernalMessage event
AndreasMadsen Dec 29, 2011
b407ba5
tell users to not use a _NODE prefix
AndreasMadsen Dec 29, 2011
e6423a7
internal message testcase
AndreasMadsen Dec 29, 2011
f34c748
documented new IPC architecture
AndreasMadsen Dec 29, 2011
48a24ea
relay also handle in message event
AndreasMadsen Dec 29, 2011
a2a1d9a
switch back to inMessage
AndreasMadsen Dec 29, 2011
7b61004
cluster.eachWorker do not exist
AndreasMadsen Dec 29, 2011
ec18b6f
Merge remote-tracking branch 'upstream/master'
AndreasMadsen Dec 30, 2011
aca3847
fix spaceing
AndreasMadsen Dec 31, 2011
8a257c3
update worker.send doc
AndreasMadsen Dec 31, 2011
9a81b59
fix spaceing
AndreasMadsen Dec 31, 2011
7cbf805
sync code with step 4
AndreasMadsen Dec 31, 2011
57d16ff
sync testcases with step 4
AndreasMadsen Dec 31, 2011
27137d4
merged 'master' intro AndreasMadsen/master
AndreasMadsen Jan 5, 2012
b38bdc8
sync with step 4
AndreasMadsen Jan 5, 2012
f57f1e4
debugging
AndreasMadsen Jan 5, 2012
f4d11fb
refactor much of the cluster.disconnect handling to fix rare issues, …
AndreasMadsen Jan 5, 2012
29ccb75
remove outdated testcase
AndreasMadsen Jan 5, 2012
eebab94
merged upstream/master intro this branch
AndreasMadsen Jan 24, 2012
987f07c
Merge remote-tracking branch 'upstream/master'
AndreasMadsen Jan 31, 2012
6d8231d
cleanup after adding propper disconnect support in child_process
AndreasMadsen Jan 31, 2012
1e4b165
Merge remote-tracking branch 'upstream/master'
AndreasMadsen Feb 1, 2012
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
182 changes: 153 additions & 29 deletions doc/api/cluster.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,11 @@ all share server ports.

var cluster = require('cluster');
var http = require('http');
var numCPUs = require('os').cpus().length;

if (cluster.isMaster) {
// Fork workers.
for (var i = 0; i < numCPUs; i++) {
cluster.fork();
}
// Spawn workers
// By default the number of workers is the number of cores in your CPU.
cluster.autoFork();

cluster.on('death', function(worker) {
console.log('worker ' + worker.pid + ' died');
Expand Down Expand Up @@ -101,46 +99,101 @@ This can be used to restart the worker by calling `fork()` again.
cluster.fork();
});

### Event 'setup'
However the workers will automatically respawn when using the `autoFork()` method.

When the `.setupMaster()` function has been executed this event emits. If `.setupMaster()`
was not executed before `fork()` or `.autoFork()`, they will execute the function with no
arguments.
### Event: 'disconnect'

### cluster.setupMaster([options])
When using the `disconnect()` in the master or in a worker, the `disconnect` event
will emit when a connection is disconnected.

The `setupMaster` is used to change the default 'fork' behavior. It takes one option
object argument.
cluster.on('disconnect', function (worker) {
console.log("We can no longer recive messages from worker");
});

Example:
If the worker was killed using`.destroy()` this event will not emit.

var cluster = require("cluster");
cluster.setupMaster({
exec : "worker.js",
args : ["--use", "https"],
silent : true
});
cluster.autoFork();
### Event: 'criticalError'

The options argument can contain 3 different properties.
When using the `autoFork()`. Worker will automatically be respawed when they die.
However if there is a permanent error in a worker, lets say a syntax error, the worker
will die in a second and be respawend again. To prevent an infinity respawn loop, the module
log the previously 5 deaths. If all 5 workers was alive in less than a second it won't
respawn any more workes, and the `cluster.disconnect()` function will be executed.

- `exec` are the file path to the worker file, by default this is the same file as the master.
- `args` are a array of arguments send along with the worker, by default this is `process.argv.slice(2)`.
- `silent`, if this option is true the output of a worker won't propagate to the master, by default this is false.
In this case the criticalError event will be emitted. The event can be used to notify
the admin about an critical error.

### cluster.settings
cluster.on('criticalError', function () {
// Send admin an email
});

All settings set by the `.setupMaster` is stored in this settings object.
This object is not supposed to be change or set manually, by you.
### Event 'setup'

All propertys are `undefined` if they are not yet set.
When the `.setupMaster()` function has been executed this event emits. If `.setupMaster()`
was not executed before `fork()` or `.autoFork()`, they will execute the function with no
arguments.

### cluster.fork([env])

Spawn a new worker process. This can only be called from the master process.
The function takes an optional `env` object. The properties in this object
will be added to the process environment in the worker.

When using `.fork()` you can not use the `.autoFork()` method. If you call
`.autoFork()` it will throw an error.

### cluster.autoFork()

When `autoFork()` is called a number of worker is forked. How many workers
you will end up with, are determined by `workers` property set in `setupMaster`.
If no property it will default to the number CPU cores you have. You can find
the number by using `require('os').cups().length`.

When using `.autoFork()` you can not use the `.fork()` method. If you do so
`.fork()` it will throw an error.

The method does also spawn a new worker when on is dead. Unless they commit suicide
this is known by checking the `worker.suicide` boolean. If you want to respawn
workers there commit suicide you simply run `.autoFork()` manually.

### cluster.destroy([callback])

When calling this method all workers will commit a non-graceful suicide.
This is useful when you want to shutdown the master quickly. It takes an optional
callback argument there will be called when finished.

This method is automatically used just before the mater dies. This can happen by
calling `process.exit()`, the master gets a `SIGINT` or a `SIGTERM` signal, or by
an uncatched error.

### cluster.disconnect([callback])

When calling this method all workers will commit a graceful suicide. It takes an optional
callback argument there will be called when finished.

This method is automaticly used when the master gets a `SIGQUIT` signal.

### cluster.restart([callback])

When updateing your workers you don't want to restart the cluster by destroying all client socket.
By using the this method the cluster will restart workers gracefully when nobody use them, and have
minimum one worker online.

This example restart the cluster each time the worker file changes:

var cluster = require('cluster');
var fs = require('fs');
cluster.setupMaster({
exec: 'worker.js'
});
fs.watchFile('worker.js', {persistent: false}, function (curr, prev) {
if (curr.ctime.getTime() === prev.ctime.getTime()) {
cluster.restart(function () {
console.log('all workers restarted');
});
}
});

### cluster.workers

In the cluster all living worker objects are stored in this object by there
Expand All @@ -163,6 +216,38 @@ the worker's uniqueID is the easiest way to find the worker.
var worker = cluster.workers[uniqueID];
});

### cluster.setupMaster([options])

The `setupMaster` is used to change the default 'fork' behavior. It takes one option
object argument.

Example:

var cluster = require("cluster");
cluster.setupMaster({
exec : "worker.js",
args : ["--use", "https"],
workers : 2,
silent : true
});
cluster.autoFork();

The options argument can contain 3 different properties.

- `exec` are the file path to the worker file, by default this is the same file as the master.
- `args` are a array of arguments send along with the worker, by default this is `process.argv.slice(2)`.
- `workers` are the number of worker there will be created when using `autoFork()`
- `silent`, if this option is true the output of a worker won't propagate to the master, by default this is false.

### cluster.settings

All settings set by the `.setupMaster` is stored in this settings object.
This object is not supposed to be change or set manually, by you.
There is also an `autoFork` property there is set to `auto` if you used the
`autoFork` method, and manual if the `fork` method was used.

All propertys are `undefined` if they are not yet set.

## Worker

This object contains all public information and method about a worker.
Expand All @@ -173,6 +258,13 @@ it can be obtained using `cluster.worker`.

Each new worker is given its own unique id, this id is stored in the `uniqueID`.


### Worker.workerID

When using the `autoFork` method, workers are also given a workerID. This id start from 0
and incress to the value of the `workers` property set by `setupMaster`. When a worker
restart this id is resused unlike the uniqueID.

### Worker.process

All workers are created using `child_process.fork()`, the returned object from this
Expand Down Expand Up @@ -211,10 +303,34 @@ To know the difference between suicide and accidentally death a suicide boolean
// destroy worker
worker.destroy();

This method is automaticly used when the worker gets a `SIGINT` or `SIGTERM` signal.

### Worker.suicide

This property is a boolean. It is set when a worker dies, until then it is `undefined`.
It is true if the worker was killed using the `.destroy()` method, and false otherwise.
It is true if the worker was killed using the `.destroy()` or `.disconnect()` method, and false otherwise.

### Worker.startup

This property is the timestamp set when the worker was forked. It is basically set by:

worker.statup = Date.now();

### Worker.disconnect();

When running the function the worker will make a graceful shutdown. This involves closing
all TCP sockets, and stopping the IPC channel between master and worker. Using the `.send()` method
will throw an error.

Calling this method will emit first a `disconnect` event, followed by a `death` event. It will also set
`suicide` to true.

This method is automatically used when the worker gets a `SIGQUIT` signal.

### Worker.restart([callback])

This method will restart the worker, but without change in the `workerID` or the custom environment.
The callback will be called when the new worker is listening for new connections.

### Event: message

Expand Down Expand Up @@ -287,3 +403,11 @@ on the specified worker.
cluster.fork().on('death', function (worker) {
// Worker has died
};

### Event: disconnect

Same as the `cluster.on('disconnect')` event, but emits only when the state change on the specified worker.

cluster.fork().on('disconnect', function (worker) {
// Worker has been disconnected
};
Loading