Streaming for large responses

[odisseus]

When working with very large projects (on the order of 10 thousand targets), server responses that contain target information tend to be very large too. Such responses can be very costly to prepare and send at once, in terms of both time and memory. The resulting JSON message can even exceed the size limit for some clients.

However, in many cases such response consists of a sequence of data objects that don't depend on each other. An obvious example of such response is workspace/buildTargets. The server could theoretically emit these objects in portions of smaller size, so that the client may process them at its own pace.

In order to support large projects, I think the protocol must support streaming the data objects in those responses where this is applicable. For what it's worth, there are implementations of JSON-RPC 2.0 that allow streaming.

As a workaround, we could split the response into chunks and send them asynchronously as notifications. This would work in a manner similar to buildTarget/compile.