Migrate x/sync to p2p
#3106
Migrate x/sync to p2p
#3106
Conversation
x/sync/manager.go
Outdated
| func (m *Manager) doWork(ctx context.Context, work *workItem) { | ||
| // Backoff for failed requests accounting for time this job has already | ||
| // spent waiting in the unprocessed queue | ||
| backoff := calculateBackoff(work.attempt) |
There was a problem hiding this comment.
Not optimal since this worker goroutine is going to be waiting until this timeout passes and ideally we would prioritize requests that are always executable, but felt like something more complex would make the diff hard to review since the scope of this PR is just trying to get rid of the blocking networking calls
| return client.GetRangeProof(ctx, request) | ||
| } | ||
|
|
||
| func TestGetRangeProof(t *testing.T) { |
There was a problem hiding this comment.
All client-side error handling cases were moved into Test_Sync_Result_Correct_Root and server-side cases were moved into network_server_test
joshua-kim
requested review from
danlaine,
darioush,
dboehm-avalabs and
StephenButtolph
as code owners
| // tracking of peers & bandwidth usage | ||
| peers *p2p.PeerTracker |
There was a problem hiding this comment.
Is it possible to configure the p2p client to perform a similar type of peer sampling? It's a critical performance improvement to select peers based on historical bandwidth.
There was a problem hiding this comment.
Yeah we'll need to merge in some form of #3007
|
This PR has become stale because it has been open for 30 days with no activity. Adding the |
joshua-kim
force-pushed
the
p2p-sync
branch
2 times, most recently
from
2e59a5d
to
e7648e5
Compare
Signed-off-by: Joshua Kim <[email protected]>
Signed-off-by: Joshua Kim <[email protected]>
Co-authored-by: Stephen Buttolph <[email protected]> Signed-off-by: Joshua Kim <[email protected]>
Signed-off-by: Joshua Kim <[email protected]>
Signed-off-by: Joshua Kim <[email protected]>
| request: &pb.SyncGetRangeProofRequest{ | ||
| RootHash: smallTrieRoot[:], | ||
| KeyLimit: 2 * defaultRequestKeyLimit, | ||
| BytesLimit: defaultRequestByteSizeLimit, | ||
| }, | ||
| expectedResponseLen: defaultRequestKeyLimit, | ||
| }, | ||
| "bytes limit too large": { | ||
| { | ||
| name: "response bounded by byte limit", |
There was a problem hiding this comment.
I think this is meant to test the case where the specified byte limit in the request is larger than the server setting. (did we accidentally drop the "response bounded by byte limit" test?)
There was a problem hiding this comment.
We're not removing a test here, not sure what you're referring to? it's the same test (prev: "bytes limit too large") but we renamed it because the previous name makes it sound like it's supposed to error when the expected behavior is to truncate/cap the response from the server.
There was a problem hiding this comment.
We could have a case where the bytes limit requested by the client is valid (less than the limit enforced by the server) and is the limiting factor in the response size.
Signed-off-by: Joshua Kim <[email protected]>
Co-authored-by: Darioush Jalali <[email protected]> Signed-off-by: Joshua Kim <[email protected]>
Signed-off-by: Joshua Kim <[email protected]>
Signed-off-by: Joshua Kim <[email protected]>
Signed-off-by: Joshua Kim <[email protected]>
Signed-off-by: Joshua Kim <[email protected]>
Signed-off-by: Joshua Kim <[email protected]>
Signed-off-by: Joshua Kim <[email protected]>
Co-authored-by: Darioush Jalali <[email protected]> Signed-off-by: Joshua Kim <[email protected]>
Why this should be merged
Eliminates our last usage of the blocking clients
How this works
Replaces usage of blocking client with
p2pasync clientsHow this was tested
Added unit tests