Reapply changes for new message rpc

[tortmayr]

What it does

Reapplies the changes for the binary message rpc, originally introduced with #11011 and fixes the critical issues which lead to reverting the original change.

In particular the following issues are addressed:

• No Yellow Status Bar on Disconnect #11199 Ensure that the new message rpc properly handles backend disconnects/reconnects.
• New IPC does not handle debugging gracefully #11203 Ensure that debugging is handled gracefully (implicitly fixed with the fix for No Yellow Status Bar on Disconnect #11199)

These bugs were caused by a combination of underlying issues:

• Trying to send RPC calls on a already disposed/disconnected websocket channel resulted in an endless loop due to using console.warn. Using console logging in the browser indirectly also triggers an RPC call to the LoggerServer. Naturally this call also fails and due to the console logging triggers another rpc call => we end up in and endless loop. Fixed by removing the console.warn statement in websocket-channel.ts.
• On socket connection loss (i.e. backend disconnect) channels and RpcProtocol instances were only partly disposed. This means once the backend reconnected did end up with multiple RpcProtocl instances for the same proxy which lead to weird issues e.g. Sending a request from one instance but the reply for the request is then handled by the other instance.
  This is now fixed by ensuring that all components involved in the rpc communication (websocket main channel, multiplexer, multiplexed service channels, RpcProtocol instances) are fully disposed on disconnect and re-instantiated on reconnect.
• On reconnect the connection-service was informed too early i.e. before all previously open channels have been restored. As a consequence some RPC calls (e.g. ping requests) were triggered before the responsible service channel was available.
  Fixed in WebSocketConnectionProvider by ensuring that the onSocketDidOpen event is fired after all previously openend channels have been restored.
• The onIncomingMessgeActivity event of the WebSocketConnectionProvider was never fired.

How to test

To test the fix for #11199:

• Start the browser application and connect in a browser.
• Kill the backend.
• Application status should change color to indicate the lost connection.
• Restart backend
• Application should reconnect and work as before the connection loss

To test the fix for #11203

• In the browser application (may also apply to Electron)
• Set a break point in the developer tools and hit it.
• Wait a few minutes.
• Remove your breakpoint and hit continue.
• Debugging process should continue without any issues and application should be responsive. No errors about failures to send messages should be logged.

Review checklist

• As an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

• As a reviewer, I agree to behave in accordance with the review guidelines

[colin-grant-work]

Both of the problems I had previously identified have been addressed, and I believe we should merge this to give us as much time as possible to identify any other issues before the next release.

I did run into one odd log from the backend. Using Theia as a workspace and navigating around a few files with go-to-definition, I saw this log:

That appears to come from some of the nls machinery (@msujew) that is a common dependency of the built-in extensions, but I haven't seen it before, so it may be worth checking on.

function createScopedLocalizeFunction(messages) {
    return function (key, message) {
        var args = [];
        for (var _i = 2; _i < arguments.length; _i++) {
            args[_i - 2] = arguments[_i];
        }
        if (isNumber(key)) {
            if (key >= messages.length) {
                console.error("Broken localize call found. Index out of bounds. Stacktrace is\n: " + new Error('').stack);
                return;
            }
            return common_1.format(messages[key], args);
        }
        else {
            if (isString(message)) {
                console.warn("Message " + message + " didn't get externalized correctly.");
                return common_1.format(message, args);
            }
            else {
                console.error("Broken localize call found. Stacktrace is\n: " + new Error('').stack);
            }
        }
    };
}

[colin-grant-work]

I did run into one odd log from the backend. Using Theia as a workspace and navigating around a few files with go-to-definition, I saw this log:

I saw this later when working on master as well, so it likely isn't related to this PR at all.

[tortmayr]

@colin-grant-work Thanks for the fast review. I agree that we should merge this as fast as possible to give us enough time to react to potential bugs and issues.

[tsmaeder]

@tortmayr if you want an in-depth code review, of the latest changes, it might take until end of week. I'm fine with merging, though: I'm confident we can find/fix outstanding issues before the next release.

[msujew]

@tortmayr I agree with Thomas here. I'm fine with merging the changes, but won't look any deeper into them for now. On a surface level everything seems to be alright 👍

@colin-grant-work I'll take a look at the nls behavior (hopefully sometime this week).

[tortmayr]

@tsmaeder @msujew
Yes, sounds good to me. I don't think that a in-depth review of the bugfix changes is required at this point.