Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: rpc response message chunking #3336

Merged
merged 2 commits into from
Sep 13, 2021

Conversation

sdbondi
Copy link
Member

@sdbondi sdbondi commented Sep 12, 2021

Description

Adds server-side "chunking" protocol for large RPC responses.

Given a payload exceeding some threshold, comms RPC will split message payloads into a number (maximum 16)
RPC messages and stream them to the client. The client will recombine the message payloads before emitting it to
the caller.

BREAKING CHANGE: Backward compatible (client-side), an upgraded node may request from an older node and process
responses. Since an older node cannot process chunks, if a response contains any, the response will be invalid to the older node. Otherwise, if less than the payload threshold, the response is exactly as before.

Motivation and Context

Large streamed message payloads (those seen in block sync) are often ~ 1.2 MiB over the wire.
The Bandwidth Delay product is often low/very low for connections over tor resulting in large message frames will take many seconds to be completely received.

This PR breaks message payloads into smaller chunks with less time between receiving a full frame on the client side.

The algorithm for chunking is as follows:

THRESHOLD = 256kb 
SIZE_LIMIT = 384Kb
MAX_CHUNKS = 16

a. payload <= THRESHOLD: emit a single frame
b. payload > THRESHOLD && payload <= SIZE_LIMIT : emit a single frame containing the full payload
c. payload > SIZE_LIMIT : emit a one or more frames containing the next SIZE_THRESHOLD bytes, with the final frame either as per (a) or (b).

This is done to prevent an unnecessary/very small trailing chunk when a payload is only a handful of bytes over the threshold
e.g 256Kb (THRESHOLD) + 1 byte results in one chunk not two. 384Kb (SIZE_LIMIT) + 1 kb results
in 2 frames of 256Kb and 129Kb respectively. meaning each chunk has a maximum size of SIZE_LIMIT and the
trailing/last chunk has a minimum size of SIZE_LIMIT - THRESHOLD (128kb)

How Has This Been Tested?

Some new unit tests, existing large message tests, manual archival sync using force sync

@sdbondi sdbondi force-pushed the comms-rpc-chunked branch 2 times, most recently from b719fca to 8fa131f Compare September 12, 2021 13:20
@sdbondi sdbondi force-pushed the comms-rpc-chunked branch 5 times, most recently from 312aa8d to 234a460 Compare September 12, 2021 17:26
Adds "chunking" protocol for large RPC responses.
Copy link
Collaborator

@stringhandler stringhandler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally and works very well

@aviator-app aviator-app bot merged commit 496ff14 into tari-project:development Sep 13, 2021
@sdbondi sdbondi deleted the comms-rpc-chunked branch September 13, 2021 13:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-comms Area - Comms
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants