process.send is sync

[AndreasMadsen]

testcase: https://gist.github.com/1663885

When using child_process.fork a process.send method exist in the child, allowing one to send objects and strings to the parent. But the method is sync!. Note that the parent get new messages using child.on('message') and this is fully async.

I think this is in violation with the node.JS never block policy.

[AndreasMadsen]

This is still an issue in a64acd8 (node 0.7.9-pre)

[AndreasMadsen]

This is still an issue in a1b2741 (node 0.8.0-pre)

[piscisaureus]

Yes, It's a known issue.

However I am not sure that making it async is a good idea. On Windows it is actually async at the moment and that causes another problem: if there are many workers sending messages to the master and the master can't keep up, these messages are buffered in the workers. This will make the workers consume a lot of memory, or even crash. Normally (with streams) you would use pause/resume and the drain event, but the cluster message passing api does not support that.

[AndreasMadsen]

The main case is that there is no good way to send a 1MB state (or smaller) object from and to workers. IPC is not good because it will block the sender, UDP can handle big messages and TCP is shared between in cluster and will require some hacking to go around.

Another case is heavy computation, for instance I have an application where I use two 30 dimensional matrixes. Sending that type of message is about (30*30*10 + 30*30*10) B == 18KB and can block the sender usually an REST HTTP server for many ms.

I do see the problem you are mentioning, however I do not understand why the issue is in cluster. The message passing API exist in child_process.js and is just as big an issue when using cp.fork() only. (The matix calculator is an example of this).

[mmalecki]

@bnoordhuis This is the issue I mentioned on IRC back in a while. Simplest code to demonstrate is:

if (!process.send) {
  var fork = require('child_process').fork;
  var child = fork('child.js');
  child.on('message', function (msg) {
    console.log('msg', msg.length);
  });
}
else {
  var crypto = require('crypto');
  var bytes = crypto.randomBytes(1024 * 1024);
  console.time('send');
  process.send(bytes);
  console.timeEnd('send');
}

[rlidwka]

IPC is not good because it will block the sender, UDP can handle big messages and TCP is shared between in cluster and will require some hacking to go around.

Unix sockets maybe?

net.connect({path: '/tmp/worker.sock'}) and so on

[mmalecki]

@bnoordhuis that's the practical solution, but don't you think that this is a bug anyway?
I sent a pull request for v0.10 docs, since we probably shouldn't introduce a change like that in stable. See #6501.

[indutny]

@mmalecki Essentially that's because all ttys are blocking in node. However there're some caveats if making it async, especially in situations where child process is reading incoming data slower than master it writes.

In such case all data will queue up in master and won't be released until child will read it all. I have some ideas about this, though

[jasnell]

@indutny ... do you happen to know the status of this one?

[jasnell]

This does not actually appear to be a bug. There are fairly straightforward workarounds and the docs are clear that send is sync and that sending large amounts of data using send is not recommended. Closing. Can reopen if new information is received.