Error "handshake authorized" in Cluster

[jamlfy]

Well, I was working to make running Socket.io in cluster. But unfortunately, I dicuenta that there is a big mistake in authentication system, the same Socket.io.

In my master.js

if( options.cluster && cluster.isMaster) {
     var numCPUs = os.cpus().length;
     for (var i = 0; i < numCPUs; i++)
         cluster.fork();
} else {
     var socket = require( path.join( 'api', 'socket') );
     var server = require('http').createServer( app );
     socket( io.listen(server) );
     server.listen( 3005 );
};

api/socket.js

module.exports = function( io  ) {
    var SessionStore = auth.session( require('express') );
    io.configure("devolper", function(){ 
        io.enable('browser client gzip');
        io.set("authorization", function(data, accept){
           if (!data.headers.cookie){
            console.log( "[ Error] No Cookes " + data.headers );
            return accept(null, false);
           } else {
                      // And more... code!! :P
                      return accept(null, true);
                   } });

In theory all was well, however the problem is with socket.io authorizations. Here I leave the record!

info: socket.io started
info: socket.io started
info: socket.io started
info: socket.io started
[ www ] 200 GET 269 ms      /
debug: authorized
info: handshake authorized BEOzKHb726SDxdPEib5L
debug: setting request GET /socket.io/1/websocket/BEOzKHb726SDxdPEib5L
debug: set heartbeat interval for client BEOzKHb726SDxdPEib5L
debug: websocket writing 7:::1+0
warn: client not handshaken client should reconnect
info: transport end (error)
debug: set close timeout for client BEOzKHb726SDxdPEib5L
debug: cleared close timeout for client BEOzKHb726SDxdPEib5L
debug: cleared heartbeat interval for client BEOzKHb726SDxdPEib5L
debug: discarding transport
debug: authorized
info: handshake authorized DyMapyEIbuVWTqCfib7D
debug: setting request GET /socket.io/1/websocket/DyMapyEIbuVWTqCfib7D
debug: set heartbeat interval for client DyMapyEIbuVWTqCfib7D
debug: websocket writing 7:::1+0
warn: client not handshaken client should reconnect
info: transport end (error)
debug: set close timeout for client DyMapyEIbuVWTqCfib7D
debug: cleared close timeout for client DyMapyEIbuVWTqCfib7D
debug: cleared heartbeat interval for client DyMapyEIbuVWTqCfib7D
debug: discarding transport
debug: authorized
info: handshake authorized Yn8XxsVHU0840B6dib5M
debug: setting request GET /socket.io/1/websocket/Yn8XxsVHU0840B6dib5M
debug: set heartbeat interval for client Yn8XxsVHU0840B6dib5M
debug: websocket writing 7:::1+0
warn: client not handshaken client should reconnect
info: transport end (error)
debug: set close timeout for client Yn8XxsVHU0840B6dib5M
debug: cleared close timeout for client Yn8XxsVHU0840B6dib5M
debug: cleared heartbeat interval for client Yn8XxsVHU0840B6dib5M
debug: discarding transport
debug: authorized
info: handshake authorized bNV2APuF1EIgiJqsib7E
debug: setting request GET /socket.io/1/websocket/bNV2APuF1EIgiJqsib7E
debug: set heartbeat interval for client bNV2APuF1EIgiJqsib7E
debug: websocket writing 7:::1+0
warn: client not handshaken client should reconnect
info: transport end (error)
debug: set close timeout for client bNV2APuF1EIgiJqsib7E
debug: cleared close timeout for client bNV2APuF1EIgiJqsib7E
debug: cleared heartbeat interval for client bNV2APuF1EIgiJqsib7E
debug: discarding transport
debug: authorized
info: handshake authorized 67txvyk6fFPQ5At2ib5N
debug: setting request GET /socket.io/1/websocket/67txvyk6fFPQ5At2ib5N
debug: set heartbeat interval for client 67txvyk6fFPQ5At2ib5N
debug: websocket writing 7:::1+0
warn: client not handshaken client should reconnect
info: transport end (error)
debug: set close timeout for client 67txvyk6fFPQ5At2ib5N
debug: cleared close timeout for client 67txvyk6fFPQ5At2ib5N
debug: cleared heartbeat interval for client 67txvyk6fFPQ5At2ib5N
debug: discarding transport
debug: authorized
info: handshake authorized W8DInTQdNzJUCV4Xib7F
debug: setting request GET /socket.io/1/websocket/W8DInTQdNzJUCV4Xib7F
debug: set heartbeat interval for client W8DInTQdNzJUCV4Xib7F
debug: client authorized for 
debug: websocket writing 1::
debug: got disconnection packet
info: transport end by forced client disconnection
debug: websocket writing 0::
info: transport end (booted)
debug: set close timeout for client W8DInTQdNzJUCV4Xib7F
debug: cleared close timeout for client W8DInTQdNzJUCV4Xib7F
debug: cleared heartbeat interval for client W8DInTQdNzJUCV4Xib7F
debug: discarding transport
debug: got disconnection packet
debug: got disconnection packet
debug: got disconnection packet
debug: got disconnection packet
[ Worker ] 31811 [ died ]

If we realize packets of authorization are different for each worker. Thus each worker is receiving a different session id. when it should be the same. This generates an error, and ends up killing the worker :S

Sorry to put a record as long ... but is that so it looks better (I shortened, so that you see the error)

[reatailret]

I think you should use RedisStore.

[bajtos]

You could use strong-cluster-socket.io-store instead of RedisStore too.

However, there is a major design problem in socket.io that causes race condition in authorization handshake. Therefore socket.io 0.9.x will never work reliably in a cluster. See #952 for details.

[3rd-Eden]

strong-cluster-socket.io-store uses the same broken interface as RedisStore it wouldn't resolve any of the issues that are illustrated here as the fundamental flaws are within the store handling of socket.io

[bajtos]

strong-cluster-socket.io-store uses the same broken interface as RedisStore it wouldn't resolve any of the issues that are illustrated here as the fundamental flaws are within the store handling of socket.io

Exactly. We have an updated README at github with a warning about this issue, unfortunately there was nothing new to publish to npmjs.org to update the copy of README stored there.

[jamlfy]

I was thinking about this problem, I think the solution is going back to structure, store. Because the problem is that I would think that none socket.io engines are communicating with each other.

But for communicating processes, one must construct a database or a method contrurir Express.js like. The only thing I can think of...

[filipedeschamps]

Guys, I implemented socket.io with redis + clusters, and I'm not receiving erros, including on authentication proccess. I'm using [email protected] with it's own redis lib.

Am I doing something wrong?

[toblerpwn]

@filipedeschamps - I am jealous. ;)

Out of curiosity, when did you pull down socket.io? And how did you get it? (If Github, which branch?) I noticed a commit on the 0.9 branch that addresses a memory leak ~24 hours ago (first commit in MONTHS), but I don't think it's been 'versioned' or made it to NPM yet. I wonder if it's somehow related.

Either way: also do you mind posting your (sanitized) source as a gist or something? I don't think many (if any?) have gotten this combination working out-of-the-box.

[jamlfy]

It is still the problem, the version is 0.9.16. Perfectly can use code and review.

[ranadeep47]

load balancing with sticky sessions and using different ports for each worker (usually the the same port is shared forall )seems to work
#939 (comment)

[toblerpwn]

@alejonext - Just to confirm, are you saying this works correctly for you now? But it was failing 5 months ago? (Also 9 days ago?)

@aardee47 Hate to ask a noob question, but my colleague and I are both iOS guys, learned Javascript/node.js/socket.io very recently so we could build out a game, still spinning up on the finer points of lower-level traffic handling.

Is there a good resource (or gist, or framework) that you would recommend to help us go down this port-handling route that you're suggesting? Would help me (A LOT) and future readers of this issue, I'm sure.

[ranadeep47]

@toblerpwn I havent personally tried it . But, there is an unresolved issue in socket.io , race condition between HTTP-Upgrade and client sending a request to a oblivious worker (#939). So using cluster with socket.io is unreliable .

Im not sure , but i guess if you can assign different port to each worker ( http://nodejs.org/api/cluster.html read the 'how it works' edge case 3 in that section ) and load balance with sticky sessions it might work

To load balance and to make client socket always connect to a specific worker you need to use cookie/IP based sticky session LB . seems like haproxy is really good at this .

http://blog.davidmisshula.com/blog/2013/02/04/configure-haproxy-to-scale-multiple-nodes-with-stickiness-and-ssl/

Hope this helps

[jamlfy]

@toblerpwn This error is found.

No, if the best solution for this error is construct alternative modules to solve. But what is certain is that it touches fix.

[ranadeep47]

@toblerpwn After a long hectic search for workarounds for this socket.io cluster issues and sticky sessions etc.. i ve finally decided to seperate socket.io logic from my core app . Im doing it this way , i have my main app in app.js and socket.io only app in socketApp.js , Both are connected to a db to have access to the sessions . Now every time you need to emit or listen to events on a connected socket which is stored as a map(like connectedClients = {}) in socketApp.js you have to somehow use some IPC . I am using https://github.com/visionmedia/axon req/rep to do this . Its doing great so far , though i dont know if this setup has any problems in production