Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bump to new gossipsub #3858

Merged
merged 36 commits into from
Apr 5, 2022
Merged

Bump to new gossipsub #3858

merged 36 commits into from
Apr 5, 2022

Conversation

dapplion
Copy link
Contributor

Motivation

Adds ChainSafe/js-libp2p-gossipsub#206 to Lodestar

Description

TBD

@github-actions
Copy link
Contributor

github-actions bot commented Mar 16, 2022 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: a08a044 Previous: 111b194 Ratio
BeaconState.hashTreeRoot - No change 372.00 ns/op 605.00 ns/op 0.61
BeaconState.hashTreeRoot - 1 full validator 49.008 us/op 68.416 us/op 0.72
BeaconState.hashTreeRoot - 32 full validator 477.89 us/op 656.07 us/op 0.73
BeaconState.hashTreeRoot - 512 full validator 5.3623 ms/op 6.8187 ms/op 0.79
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 60.514 us/op 85.208 us/op 0.71
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 844.40 us/op 1.3175 ms/op 0.64
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 11.786 ms/op 16.504 ms/op 0.71
BeaconState.hashTreeRoot - 1 balances 47.086 us/op 68.486 us/op 0.69
BeaconState.hashTreeRoot - 32 balances 411.90 us/op 619.54 us/op 0.66
BeaconState.hashTreeRoot - 512 balances 4.1126 ms/op 5.8070 ms/op 0.71
BeaconState.hashTreeRoot - 250000 balances 89.363 ms/op 126.17 ms/op 0.71
processSlot - 1 slots 9.4490 us/op 14.201 us/op 0.67
processSlot - 32 slots 1.3982 ms/op 2.1371 ms/op 0.65
getCommitteeAssignments - req 1 vs - 250000 vc 4.6244 ms/op 5.4124 ms/op 0.85
getCommitteeAssignments - req 100 vs - 250000 vc 6.3877 ms/op 7.4952 ms/op 0.85
getCommitteeAssignments - req 1000 vs - 250000 vc 6.8011 ms/op 7.9069 ms/op 0.86
computeProposers - vc 250000 15.037 ms/op 20.056 ms/op 0.75
computeEpochShuffling - vc 250000 141.07 ms/op 154.47 ms/op 0.91
getNextSyncCommittee - vc 250000 249.16 ms/op 328.90 ms/op 0.76
altair processAttestation - 250000 vs - 7PWei normalcase 3.7955 ms/op 4.7117 ms/op 0.81
altair processAttestation - 250000 vs - 7PWei worstcase 5.4757 ms/op 7.0757 ms/op 0.77
altair processAttestation - setStatus - 1/6 committees join 171.77 us/op 228.86 us/op 0.75
altair processAttestation - setStatus - 1/3 committees join 331.25 us/op 440.90 us/op 0.75
altair processAttestation - setStatus - 1/2 committees join 460.19 us/op 626.93 us/op 0.73
altair processAttestation - setStatus - 2/3 committees join 593.33 us/op 811.70 us/op 0.73
altair processAttestation - setStatus - 4/5 committees join 827.83 us/op 1.1739 ms/op 0.71
altair processAttestation - setStatus - 100% committees join 979.85 us/op 1.2955 ms/op 0.76
altair processBlock - 250000 vs - 7PWei normalcase 24.656 ms/op 28.367 ms/op 0.87
altair processBlock - 250000 vs - 7PWei normalcase hashState 37.479 ms/op 40.647 ms/op 0.92
altair processBlock - 250000 vs - 7PWei worstcase 69.910 ms/op 90.065 ms/op 0.78
altair processBlock - 250000 vs - 7PWei worstcase hashState 90.119 ms/op 133.85 ms/op 0.67
altair processEth1Data - 250000 vs - 7PWei normalcase 843.94 us/op 969.72 us/op 0.87
altair processEpoch - mainnet_e81889 499.82 ms/op 539.20 ms/op 0.93
mainnet_e81889 - altair beforeProcessEpoch 140.25 ms/op 161.04 ms/op 0.87
mainnet_e81889 - altair processJustificationAndFinalization 25.548 us/op 74.271 us/op 0.34
mainnet_e81889 - altair processInactivityUpdates 10.503 ms/op 11.091 ms/op 0.95
mainnet_e81889 - altair processRewardsAndPenalties 77.986 ms/op 150.86 ms/op 0.52
mainnet_e81889 - altair processRegistryUpdates 4.7190 us/op 16.540 us/op 0.29
mainnet_e81889 - altair processSlashings 1.3120 us/op 6.2680 us/op 0.21
mainnet_e81889 - altair processEth1DataReset 1.7750 us/op 6.1730 us/op 0.29
mainnet_e81889 - altair processEffectiveBalanceUpdates 6.4312 ms/op 8.2484 ms/op 0.78
mainnet_e81889 - altair processSlashingsReset 8.9250 us/op 23.326 us/op 0.38
mainnet_e81889 - altair processRandaoMixesReset 11.064 us/op 26.736 us/op 0.41
mainnet_e81889 - altair processHistoricalRootsUpdate 1.5750 us/op 8.8150 us/op 0.18
mainnet_e81889 - altair processParticipationFlagUpdates 6.3950 us/op 16.245 us/op 0.39
mainnet_e81889 - altair processSyncCommitteeUpdates 2.9880 us/op 6.5180 us/op 0.46
mainnet_e81889 - altair afterProcessEpoch 185.28 ms/op 193.20 ms/op 0.96
altair processInactivityUpdates - 250000 normalcase 30.520 ms/op 62.372 ms/op 0.49
altair processInactivityUpdates - 250000 worstcase 26.580 ms/op 41.615 ms/op 0.64
altair processParticipationFlagUpdates - 250000 anycase 6.0800 us/op 12.839 us/op 0.47
altair processRewardsAndPenalties - 250000 normalcase 106.18 ms/op 92.538 ms/op 1.15
altair processRewardsAndPenalties - 250000 worstcase 113.66 ms/op 121.18 ms/op 0.94
altair processSyncCommitteeUpdates - 250000 270.21 ms/op 388.91 ms/op 0.69
Tree 40 250000 create 633.61 ms/op 861.50 ms/op 0.74
Tree 40 250000 get(125000) 250.82 ns/op 330.80 ns/op 0.76
Tree 40 250000 set(125000) 2.0328 us/op 2.7739 us/op 0.73
Tree 40 250000 toArray() 29.941 ms/op 36.049 ms/op 0.83
Tree 40 250000 iterate all - toArray() + loop 30.219 ms/op 37.054 ms/op 0.82
Tree 40 250000 iterate all - get(i) 99.072 ms/op 133.49 ms/op 0.74
MutableVector 250000 create 14.185 ms/op 16.827 ms/op 0.84
MutableVector 250000 get(125000) 11.583 ns/op 14.276 ns/op 0.81
MutableVector 250000 set(125000) 539.46 ns/op 682.97 ns/op 0.79
MutableVector 250000 toArray() 6.0143 ms/op 6.0574 ms/op 0.99
MutableVector 250000 iterate all - toArray() + loop 5.8467 ms/op 6.4790 ms/op 0.90
MutableVector 250000 iterate all - get(i) 2.8945 ms/op 3.3819 ms/op 0.86
Array 250000 create 5.0208 ms/op 5.6761 ms/op 0.88
Array 250000 clone - spread 2.5277 ms/op 2.7477 ms/op 0.92
Array 250000 get(125000) 1.1740 ns/op 1.3130 ns/op 0.89
Array 250000 set(125000) 1.1940 ns/op 1.2790 ns/op 0.93
Array 250000 iterate all - loop 167.84 us/op 145.59 us/op 1.15
effectiveBalanceIncrements clone Uint8Array 300000 75.733 us/op 406.98 us/op 0.19
effectiveBalanceIncrements clone MutableVector 300000 543.00 ns/op 623.00 ns/op 0.87
effectiveBalanceIncrements rw all Uint8Array 300000 301.87 us/op 195.22 us/op 1.55
effectiveBalanceIncrements rw all MutableVector 300000 157.11 ms/op 193.42 ms/op 0.81
aggregationBits - 2048 els - zipIndexesInBitList 26.355 us/op 26.614 us/op 0.99
regular array get 100000 times 67.428 us/op 59.444 us/op 1.13
wrappedArray get 100000 times 67.424 us/op 59.569 us/op 1.13
arrayWithProxy get 100000 times 52.935 ms/op 40.293 ms/op 1.31
ssz.Root.equals 374.00 ns/op 541.00 ns/op 0.69
byteArrayEquals 361.00 ns/op 505.00 ns/op 0.71
phase0 processBlock - 250000 vs - 7PWei normalcase 3.0307 ms/op 4.3106 ms/op 0.70
phase0 processBlock - 250000 vs - 7PWei worstcase 40.343 ms/op 55.006 ms/op 0.73
phase0 afterProcessEpoch - 250000 vs - 7PWei 154.90 ms/op 180.28 ms/op 0.86
phase0 beforeProcessEpoch - 250000 vs - 7PWei 71.747 ms/op 106.90 ms/op 0.67
phase0 processEpoch - mainnet_e58758 529.84 ms/op 611.35 ms/op 0.87
mainnet_e58758 - phase0 beforeProcessEpoch 249.58 ms/op 256.94 ms/op 0.97
mainnet_e58758 - phase0 processJustificationAndFinalization 24.023 us/op 73.346 us/op 0.33
mainnet_e58758 - phase0 processRewardsAndPenalties 67.665 ms/op 96.084 ms/op 0.70
mainnet_e58758 - phase0 processRegistryUpdates 10.451 us/op 35.641 us/op 0.29
mainnet_e58758 - phase0 processSlashings 2.0180 us/op 6.8830 us/op 0.29
mainnet_e58758 - phase0 processEth1DataReset 2.0360 us/op 7.0350 us/op 0.29
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 5.0071 ms/op 7.0136 ms/op 0.71
mainnet_e58758 - phase0 processSlashingsReset 9.4580 us/op 24.022 us/op 0.39
mainnet_e58758 - phase0 processRandaoMixesReset 10.207 us/op 27.615 us/op 0.37
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.2400 us/op 7.7550 us/op 0.16
mainnet_e58758 - phase0 processParticipationRecordUpdates 7.3250 us/op 25.950 us/op 0.28
mainnet_e58758 - phase0 afterProcessEpoch 142.80 ms/op 160.36 ms/op 0.89
phase0 processEffectiveBalanceUpdates - 250000 normalcase 5.5685 ms/op 7.6590 ms/op 0.73
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 6.0313 ms/op 7.3500 ms/op 0.82
phase0 processRegistryUpdates - 250000 normalcase 8.9430 us/op 31.372 us/op 0.29
phase0 processRegistryUpdates - 250000 badcase_full_deposits 366.30 us/op 502.29 us/op 0.73
phase0 processRegistryUpdates - 250000 worstcase 0.5 196.57 ms/op 285.73 ms/op 0.69
phase0 getAttestationDeltas - 250000 normalcase 12.581 ms/op 16.054 ms/op 0.78
phase0 getAttestationDeltas - 250000 worstcase 12.505 ms/op 15.998 ms/op 0.78
phase0 processSlashings - 250000 worstcase 4.9944 ms/op 6.8582 ms/op 0.73
shuffle list - 16384 els 9.4492 ms/op 10.654 ms/op 0.89
shuffle list - 250000 els 137.19 ms/op 152.74 ms/op 0.90
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 327.62 us/op 485.46 us/op 0.67
pass gossip attestations to forkchoice per slot 3.6486 ms/op 4.1392 ms/op 0.88
computeDeltas 3.3412 ms/op 3.7784 ms/op 0.88
computeProposerBoostScoreFromBalances 443.92 us/op 482.71 us/op 0.92
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.9841 ms/op 2.7597 ms/op 0.72
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 75.600 us/op 87.625 us/op 0.86
BLS verify - blst-native 1.6417 ms/op 2.3000 ms/op 0.71
BLS verifyMultipleSignatures 3 - blst-native 3.3494 ms/op 4.5992 ms/op 0.73
BLS verifyMultipleSignatures 8 - blst-native 7.2297 ms/op 9.8084 ms/op 0.74
BLS verifyMultipleSignatures 32 - blst-native 26.176 ms/op 35.834 ms/op 0.73
BLS aggregatePubkeys 32 - blst-native 34.679 us/op 46.519 us/op 0.75
BLS aggregatePubkeys 128 - blst-native 135.55 us/op 189.44 us/op 0.72
getAttestationsForBlock 50.632 ms/op 61.301 ms/op 0.83
CheckpointStateCache - add get delete 9.3140 us/op 13.270 us/op 0.70
validate gossip signedAggregateAndProof - struct 3.7704 ms/op 5.0898 ms/op 0.74
validate gossip attestation - struct 1.7797 ms/op 2.5761 ms/op 0.69
pickEth1Vote - no votes 2.0125 ms/op 2.8378 ms/op 0.71
pickEth1Vote - max votes 21.349 ms/op 21.697 ms/op 0.98
pickEth1Vote - Eth1Data hashTreeRoot value x2048 10.994 ms/op 15.271 ms/op 0.72
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 20.352 ms/op 25.023 ms/op 0.81
pickEth1Vote - Eth1Data fastSerialize value x2048 1.5181 ms/op 2.0374 ms/op 0.75
pickEth1Vote - Eth1Data fastSerialize tree x2048 14.550 ms/op 15.000 ms/op 0.97
bytes32 toHexString 932.00 ns/op 1.2960 us/op 0.72
bytes32 Buffer.toString(hex) 630.00 ns/op 829.00 ns/op 0.76
bytes32 Buffer.toString(hex) from Uint8Array 834.00 ns/op 1.1230 us/op 0.74
bytes32 Buffer.toString(hex) + 0x 632.00 ns/op 837.00 ns/op 0.76
Object access 1 prop 0.30500 ns/op 0.44500 ns/op 0.69
Map access 1 prop 0.25600 ns/op 0.37100 ns/op 0.69
Object get x1000 15.553 ns/op 16.851 ns/op 0.92
Map get x1000 0.89800 ns/op 1.0530 ns/op 0.85
Object set x1000 100.86 ns/op 105.26 ns/op 0.96
Map set x1000 62.618 ns/op 78.255 ns/op 0.80
Return object 10000 times 0.32980 ns/op 0.41980 ns/op 0.79
Throw Error 10000 times 5.1972 us/op 7.2353 us/op 0.72
enrSubnets - fastDeserialize 64 bits 893.00 ns/op 1.3060 us/op 0.68
enrSubnets - ssz BitVector 64 bits 544.00 ns/op 787.00 ns/op 0.69
enrSubnets - fastDeserialize 4 bits 369.00 ns/op 510.00 ns/op 0.72
enrSubnets - ssz BitVector 4 bits 551.00 ns/op 723.00 ns/op 0.76
RateTracker 1000000 limit, 1 obj count per request 168.21 ns/op 204.56 ns/op 0.82
RateTracker 1000000 limit, 2 obj count per request 125.87 ns/op 146.58 ns/op 0.86
RateTracker 1000000 limit, 4 obj count per request 105.20 ns/op 117.44 ns/op 0.90
RateTracker 1000000 limit, 8 obj count per request 93.642 ns/op 106.27 ns/op 0.88
RateTracker with prune 3.9380 us/op 4.7280 us/op 0.83
array of 16000 items push then shift 2.7331 us/op 5.4408 us/op 0.50
LinkedList of 16000 items push then shift 17.115 ns/op 18.441 ns/op 0.93
array of 16000 items push then pop 193.43 ns/op 251.19 ns/op 0.77
LinkedList of 16000 items push then pop 15.113 ns/op 17.888 ns/op 0.84
array of 24000 items push then shift 4.0303 us/op 7.9181 us/op 0.51
LinkedList of 24000 items push then shift 21.721 ns/op 18.401 ns/op 1.18
array of 24000 items push then pop 191.16 ns/op 232.07 ns/op 0.82
LinkedList of 24000 items push then pop 18.683 ns/op 18.204 ns/op 1.03

by benchmarkbot/action

@codecov
Copy link

codecov bot commented Mar 16, 2022 •
edited
Loading

Codecov Report

Merging #3858 (378c2f4) into master (111b194) will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3858      +/-   ##
==========================================
+ Coverage   36.23%   36.26%   +0.02%     
==========================================
  Files         327      326       -1     
  Lines        9095     9074      -21     
  Branches     1430     1415      -15     
==========================================
- Hits         3296     3291       -5     
+ Misses       5656     5640      -16     
  Partials      143      143

dapplion and others added 22 commits March 16, 2022 11:43
@dapplion
Cleanup test nodes on success and error
0983af7
@dapplion
Add allowPublishToZeroPeers option
1855592
@dapplion
Add allowPublishToZeroPeers to dev options
52938fe
@dapplion
Add scripts to run dev servers
22717fb
@dapplion
Bump gossipsub
73a14c9
@dapplion
Bump gossipsub
b91f867
@dapplion
Expose gossip peer score stats
35b5195
@dapplion
bump gossipsub
7ccfd00
@dapplion
Dump gossipsub score params
da9a152
@dapplion
Bump lodestar
adce232
@twoeths
Add behaviourPenaltyThreshold to gossip scoring params
74130f2
@twoeths
Switch to latest gossipsub update
8fe6fd5
@twoeths
Fix cli unit test
be979ab
@twoeths
Fix unknownBlockSync e2e test
06fd45c
@twoeths
Update latest gossipsub
b9985fc
@twoeths
Fix e2e - network addr in use
9cf8127
@twoeths
Use latest gossipsub 3efae56c9d39ff3a047a7e6c65fa32e5756bebde
c926b6d
@twoeths
Use latest gossipsub 039af54c7bf47396c1be414d9fa33aa2e8cc2697
a1228ca
@twoeths
Increase meshMessageDeliveriesWindow and gossipsubIWantFollowupMs
e56691d
@twoeths
Set gossipsubIWantFollowupMs and meshMessageDeliveriesWindow 12s
cdd8e71
@twoeths
Reset gossipsub metric behaviourPenalty per scrape
dd76614
@twoeths
Compute score using lodestar score and gossipsub score (#3875)
d7cdd4a 
* Refactor PeerRpcScoreStore: add PeerScore class

* Aggregate lodestarScore, gossipsubScore to compute final score

* updateGossipsubScores util and unit test

* Populate PeerScore on updateGossipsubScore

* Fix peerManager e2e test

* Fix test/sim/multiNodeSingleThread.test.ts
@twoeths twoeths dismissed a stale review via d7cdd4a March 29, 2022 09:22
@twoeths
Copy link
Contributor

twoeths commented Mar 31, 2022

The test in hetzner-test0 has been good:

  • Most gossip peer scores are positive

Screen Shot 2022-03-31 at 10 47 35

  • It helps us get a very stable mesh peers

Screen Shot 2022-03-31 at 10 48 42

  • Validator monitor metrics are almost the same: balance delta, missed attestations ratio

  • Gossip validation time is slightly increased but it does not affect the gossip score

  • Gossip block processed time is slightly increased, however "till received" is slightly better so overall "till processed" is the same

Some down side monitored on contabo nodes: the better gossip scores we have, the more mesh peers we have and the more gossip messages we receive and it causes performance issue (since messages have to get thru a lot of middle libraries before it reaches lodestar). So contabo can run stably only if:

  • We tweak gossipsub Dhi, D, Dlo to be 4, 3, 2 (default is 12, 8 , 6) (as in contabo-20)
  • Or we reduce connected validators in order to join less subnet topics, if connected validators is 20 it works fine, but with 30 validators it does not (as in contabo-19)

@dapplion
Copy link
Contributor Author

dapplion commented Apr 3, 2022

and it causes performance issue (since messages have to get thru a lot of middle libraries before it reaches lodestar)

Can you post a CPU profile and detail where are the bottlenecks exactly?

dapplion
dapplion commented Apr 4, 2022
View reviewed changes
packages/lodestar/src/network/gossip/gossipsub.ts
@@ -113,6 +113,7 @@ export class Eth2Gossipsub extends Gossipsub {
dataTransform: new DataTransformSnappy(gossipTopicCache),
metricsRegister: modules.metrics ? ((modules.metrics.register as unknown) as MetricsRegister) : null,
metricsTopicStrToLabel: modules.metrics ? getMetricsTopicStrToLabel(modules.config) : undefined,
asyncValidation: true,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh lol

@philknows philknows mentioned this pull request Apr 5, 2022
Closed
@twoeths
Copy link
Contributor

twoeths commented Apr 5, 2022

and it causes performance issue (since messages have to get thru a lot of middle libraries before it reaches lodestar)

Can you post a CPU profile and detail where are the bottlenecks exactly?

with latest update, our contabo node can handle default gossipsub params (D params of 6, 8, 12) and 50 validators

@twoeths twoeths marked this pull request as ready for review April 5, 2022 10:21
@twoeths twoeths merged commit d61d898 into master Apr 5, 2022
@twoeths twoeths deleted the dapplion/gossipsub-bump branch April 5, 2022 10:32
@nflaig nflaig mentioned this pull request Jul 4, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@twoeths twoeths twoeths approved these changes

@wemeetagain wemeetagain Awaiting requested review from wemeetagain

@g11tech g11tech Awaiting requested review from g11tech

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@dapplion @twoeths @philknows @wemeetagain