Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: make sure shuffling is calculated when querying next epoch proposers #7156

Merged
merged 4 commits into from
Oct 14, 2024
Merged

fix: make sure shuffling is calculated when querying next epoch proposers #7156

merged 4 commits into from
Oct 14, 2024

Conversation

matthewkeil
Copy link
Member

@matthewkeil matthewkeil commented Oct 12, 2024 •
edited
Loading

Description

Make pulling of next proposers async to await for shuffling calculation

Motivation
Error was found on 1.23 release after #7120 was merged with unstable due to change made in #7130

Notes
@twoeths I made getBeaconProposersNextEpoch async to resolve this but curious if you would prefer and async helper function instead. This is the only place where getBeaconProposersNextEpoch gets called so will be an easy switch if you want to keep everything on EpochCache sync. Just let me know and will fix it quickly. Currently the proposers are cached on the epochCtx so i could just pass that through the free function to maintain the caching or we can just recompute on each API call if there will not be many. Just let me know which way you prefer to go but I built it like this because it seemed like the best way forward.

@matthewkeil matthewkeil requested a review from a team as a code owner October 12, 2024 08:40
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Oct 12, 2024 •
edited
Loading

Performance Report

✔️ no performance regression detected

🚀🚀 Significant benchmark improvement detected

Benchmark suite Current: 5ee87e0 Previous: d37bdb0 Ratio
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4196 ms/op 9.6080 ms/op 0.15
Full benchmark results
Benchmark suite Current: 5ee87e0 Previous: d37bdb0 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.8350 ms/op 2.7476 ms/op 0.67
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 67.257 us/op 80.185 us/op 0.84
BLS verify - blst 920.46 us/op 1.0552 ms/op 0.87
BLS verifyMultipleSignatures 3 - blst 1.4433 ms/op 1.3518 ms/op 1.07
BLS verifyMultipleSignatures 8 - blst 2.0870 ms/op 1.9605 ms/op 1.06
BLS verifyMultipleSignatures 32 - blst 4.5659 ms/op 6.2975 ms/op 0.73
BLS verifyMultipleSignatures 64 - blst 9.0280 ms/op 11.041 ms/op 0.82
BLS verifyMultipleSignatures 128 - blst 16.715 ms/op 18.654 ms/op 0.90
BLS deserializing 10000 signatures 635.71 ms/op 714.52 ms/op 0.89
BLS deserializing 100000 signatures 6.3030 s/op 6.9907 s/op 0.90
BLS verifyMultipleSignatures - same message - 3 - blst 935.84 us/op 1.0918 ms/op 0.86
BLS verifyMultipleSignatures - same message - 8 - blst 1.0395 ms/op 1.0658 ms/op 0.98
BLS verifyMultipleSignatures - same message - 32 - blst 1.6843 ms/op 1.8954 ms/op 0.89
BLS verifyMultipleSignatures - same message - 64 - blst 2.4432 ms/op 2.7597 ms/op 0.89
BLS verifyMultipleSignatures - same message - 128 - blst 4.1381 ms/op 4.2411 ms/op 0.98
BLS aggregatePubkeys 32 - blst 18.548 us/op 19.168 us/op 0.97
BLS aggregatePubkeys 128 - blst 64.261 us/op 68.010 us/op 0.94
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 63.766 ms/op 62.542 ms/op 1.02
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 66.240 ms/op 56.029 ms/op 1.18
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 43.462 ms/op 50.636 ms/op 0.86
getSlashingsAndExits - default max 82.685 us/op 106.45 us/op 0.78
getSlashingsAndExits - 2k 379.32 us/op 323.09 us/op 1.17
proposeBlockBody type=full, size=empty 7.4090 ms/op 6.7640 ms/op 1.10
isKnown best case - 1 super set check 555.00 ns/op 323.00 ns/op 1.72
isKnown normal case - 2 super set checks 545.00 ns/op 316.00 ns/op 1.72
isKnown worse case - 16 super set checks 605.00 ns/op 295.00 ns/op 2.05
InMemoryCheckpointStateCache - add get delete 5.3010 us/op 3.1890 us/op 1.66
updateUnfinalizedPubkeys - updating 10 pubkeys 1.3722 ms/op 1.2347 ms/op 1.11
updateUnfinalizedPubkeys - updating 100 pubkeys 4.9222 ms/op 4.5262 ms/op 1.09
updateUnfinalizedPubkeys - updating 1000 pubkeys 63.720 ms/op 59.492 ms/op 1.07
validate api signedAggregateAndProof - struct 1.7060 ms/op 1.6519 ms/op 1.03
validate gossip signedAggregateAndProof - struct 1.8492 ms/op 1.5385 ms/op 1.20
batch validate gossip attestation - vc 640000 - chunk 32 148.77 us/op 139.66 us/op 1.07
batch validate gossip attestation - vc 640000 - chunk 64 131.99 us/op 120.66 us/op 1.09
batch validate gossip attestation - vc 640000 - chunk 128 125.43 us/op 117.95 us/op 1.06
batch validate gossip attestation - vc 640000 - chunk 256 125.90 us/op 113.67 us/op 1.11
pickEth1Vote - no votes 1.3422 ms/op 1.1652 ms/op 1.15
pickEth1Vote - max votes 9.3456 ms/op 7.8237 ms/op 1.19
pickEth1Vote - Eth1Data hashTreeRoot value x2048 24.020 ms/op 16.307 ms/op 1.47
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 30.440 ms/op 23.186 ms/op 1.31
pickEth1Vote - Eth1Data fastSerialize value x2048 426.09 us/op 496.62 us/op 0.86
pickEth1Vote - Eth1Data fastSerialize tree x2048 3.6604 ms/op 4.2911 ms/op 0.85
bytes32 toHexString 913.00 ns/op 546.00 ns/op 1.67
bytes32 Buffer.toString(hex) 511.00 ns/op 275.00 ns/op 1.86
bytes32 Buffer.toString(hex) from Uint8Array 726.00 ns/op 448.00 ns/op 1.62
bytes32 Buffer.toString(hex) + 0x 538.00 ns/op 272.00 ns/op 1.98
Object access 1 prop 0.40000 ns/op 0.15100 ns/op 2.65
Map access 1 prop 0.35500 ns/op 0.14100 ns/op 2.52
Object get x1000 5.3800 ns/op 5.9910 ns/op 0.90
Map get x1000 6.1550 ns/op 6.4530 ns/op 0.95
Object set x1000 52.716 ns/op 35.462 ns/op 1.49
Map set x1000 35.569 ns/op 25.028 ns/op 1.42
Return object 10000 times 0.31270 ns/op 0.30060 ns/op 1.04
Throw Error 10000 times 3.0027 us/op 3.4073 us/op 0.88
toHex 144.16 ns/op 171.69 ns/op 0.84
Buffer.from 136.63 ns/op 156.86 ns/op 0.87
shared Buffer 80.956 ns/op 96.247 ns/op 0.84
fastMsgIdFn sha256 / 200 bytes 2.4360 us/op 2.4180 us/op 1.01
fastMsgIdFn h32 xxhash / 200 bytes 503.00 ns/op 293.00 ns/op 1.72
fastMsgIdFn h64 xxhash / 200 bytes 508.00 ns/op 275.00 ns/op 1.85
fastMsgIdFn sha256 / 1000 bytes 7.0660 us/op 7.3230 us/op 0.96
fastMsgIdFn h32 xxhash / 1000 bytes 686.00 ns/op 416.00 ns/op 1.65
fastMsgIdFn h64 xxhash / 1000 bytes 609.00 ns/op 356.00 ns/op 1.71
fastMsgIdFn sha256 / 10000 bytes 51.600 us/op 64.942 us/op 0.79
fastMsgIdFn h32 xxhash / 10000 bytes 2.2050 us/op 1.9460 us/op 1.13
fastMsgIdFn h64 xxhash / 10000 bytes 1.4730 us/op 1.2580 us/op 1.17
send data - 1000 256B messages 15.774 ms/op 14.210 ms/op 1.11
send data - 1000 512B messages 18.830 ms/op 18.034 ms/op 1.04
send data - 1000 1024B messages 31.105 ms/op 29.206 ms/op 1.07
send data - 1000 1200B messages 28.770 ms/op 28.542 ms/op 1.01
send data - 1000 2048B messages 31.310 ms/op 35.521 ms/op 0.88
send data - 1000 4096B messages 28.472 ms/op 33.970 ms/op 0.84
send data - 1000 16384B messages 65.592 ms/op 71.594 ms/op 0.92
send data - 1000 65536B messages 284.26 ms/op 246.28 ms/op 1.15
enrSubnets - fastDeserialize 64 bits 1.1420 us/op 1.1390 us/op 1.00
enrSubnets - ssz BitVector 64 bits 555.00 ns/op 378.00 ns/op 1.47
enrSubnets - fastDeserialize 4 bits 388.00 ns/op 194.00 ns/op 2.00
enrSubnets - ssz BitVector 4 bits 642.00 ns/op 413.00 ns/op 1.55
prioritizePeers score -10:0 att 32-0.1 sync 2-0 145.39 us/op 195.59 us/op 0.74
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 167.08 us/op 173.41 us/op 0.96
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 317.58 us/op 347.48 us/op 0.91
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 457.56 us/op 514.41 us/op 0.89
prioritizePeers score 0:0 att 64-1 sync 4-1 548.36 us/op 934.25 us/op 0.59
array of 16000 items push then shift 1.3774 us/op 1.6587 us/op 0.83
LinkedList of 16000 items push then shift 6.5330 ns/op 7.5550 ns/op 0.86
array of 16000 items push then pop 111.38 ns/op 145.47 ns/op 0.77
LinkedList of 16000 items push then pop 6.4980 ns/op 7.4430 ns/op 0.87
array of 24000 items push then shift 2.2411 us/op 2.4685 us/op 0.91
LinkedList of 24000 items push then shift 6.7460 ns/op 7.7110 ns/op 0.87
array of 24000 items push then pop 110.25 ns/op 145.96 ns/op 0.76
LinkedList of 24000 items push then pop 6.4580 ns/op 7.7940 ns/op 0.83
intersect bitArray bitLen 8 5.6230 ns/op 6.6380 ns/op 0.85
intersect array and set length 8 43.976 ns/op 47.909 ns/op 0.92
intersect bitArray bitLen 128 26.679 ns/op 29.394 ns/op 0.91
intersect array and set length 128 585.37 ns/op 687.80 ns/op 0.85
bitArray.getTrueBitIndexes() bitLen 128 2.3580 us/op 2.0120 us/op 1.17
bitArray.getTrueBitIndexes() bitLen 248 3.9630 us/op 3.8600 us/op 1.03
bitArray.getTrueBitIndexes() bitLen 512 9.3440 us/op 8.1200 us/op 1.15
Buffer.concat 32 items 1.0860 us/op 932.00 ns/op 1.17
Uint8Array.set 32 items 1.6810 us/op 1.5410 us/op 1.09
Buffer.copy 1.9880 us/op 1.7400 us/op 1.14
Uint8Array.set - with subarray 2.9750 us/op 2.7380 us/op 1.09
Uint8Array.set - without subarray 1.6810 us/op 1.7480 us/op 0.96
getUint32 - dataview 435.00 ns/op 251.00 ns/op 1.73
getUint32 - manual 422.00 ns/op 162.00 ns/op 2.60
Set add up to 64 items then delete first 1.8816 us/op 2.2052 us/op 0.85
OrderedSet add up to 64 items then delete first 2.7695 us/op 3.2819 us/op 0.84
Set add up to 64 items then delete last 2.0826 us/op 2.5264 us/op 0.82
OrderedSet add up to 64 items then delete last 3.2650 us/op 3.7398 us/op 0.87
Set add up to 64 items then delete middle 2.0097 us/op 2.5254 us/op 0.80
OrderedSet add up to 64 items then delete middle 4.4743 us/op 5.2466 us/op 0.85
Set add up to 128 items then delete first 3.8241 us/op 5.0441 us/op 0.76
OrderedSet add up to 128 items then delete first 5.8530 us/op 7.8456 us/op 0.75
Set add up to 128 items then delete last 3.8186 us/op 4.9242 us/op 0.78
OrderedSet add up to 128 items then delete last 5.9646 us/op 7.3923 us/op 0.81
Set add up to 128 items then delete middle 3.8731 us/op 4.8184 us/op 0.80
OrderedSet add up to 128 items then delete middle 12.173 us/op 13.760 us/op 0.88
Set add up to 256 items then delete first 7.9531 us/op 10.572 us/op 0.75
OrderedSet add up to 256 items then delete first 11.897 us/op 15.613 us/op 0.76
Set add up to 256 items then delete last 7.8737 us/op 9.5145 us/op 0.83
OrderedSet add up to 256 items then delete last 12.317 us/op 14.024 us/op 0.88
Set add up to 256 items then delete middle 7.7734 us/op 9.3054 us/op 0.84
OrderedSet add up to 256 items then delete middle 34.462 us/op 40.017 us/op 0.86
transfer serialized Status (84 B) 1.3420 us/op 1.2670 us/op 1.06
copy serialized Status (84 B) 1.2010 us/op 1.0680 us/op 1.12
transfer serialized SignedVoluntaryExit (112 B) 1.5400 us/op 1.3450 us/op 1.14
copy serialized SignedVoluntaryExit (112 B) 1.2420 us/op 1.1390 us/op 1.09
transfer serialized ProposerSlashing (416 B) 2.0320 us/op 1.5160 us/op 1.34
copy serialized ProposerSlashing (416 B) 1.8120 us/op 1.9120 us/op 0.95
transfer serialized Attestation (485 B) 1.5520 us/op 1.8730 us/op 0.83
copy serialized Attestation (485 B) 1.5810 us/op 1.6680 us/op 0.95
transfer serialized AttesterSlashing (33232 B) 2.2670 us/op 2.5780 us/op 0.88
copy serialized AttesterSlashing (33232 B) 6.3340 us/op 4.9710 us/op 1.27
transfer serialized Small SignedBeaconBlock (128000 B) 1.8020 us/op 2.0860 us/op 0.86
copy serialized Small SignedBeaconBlock (128000 B) 11.222 us/op 15.635 us/op 0.72
transfer serialized Avg SignedBeaconBlock (200000 B) 2.5990 us/op 2.8540 us/op 0.91
copy serialized Avg SignedBeaconBlock (200000 B) 12.968 us/op 20.508 us/op 0.63
transfer serialized BlobsSidecar (524380 B) 2.4340 us/op 2.1800 us/op 1.12
copy serialized BlobsSidecar (524380 B) 128.84 us/op 93.620 us/op 1.38
transfer serialized Big SignedBeaconBlock (1000000 B) 3.1330 us/op 2.4690 us/op 1.27
copy serialized Big SignedBeaconBlock (1000000 B) 142.94 us/op 154.68 us/op 0.92
pass gossip attestations to forkchoice per slot 2.3677 ms/op 2.7914 ms/op 0.85
forkChoice updateHead vc 100000 bc 64 eq 0 421.43 us/op 611.82 us/op 0.69
forkChoice updateHead vc 600000 bc 64 eq 0 2.4660 ms/op 3.5536 ms/op 0.69
forkChoice updateHead vc 1000000 bc 64 eq 0 3.7399 ms/op 4.7344 ms/op 0.79
forkChoice updateHead vc 600000 bc 320 eq 0 2.0969 ms/op 2.7598 ms/op 0.76
forkChoice updateHead vc 600000 bc 1200 eq 0 2.1552 ms/op 2.8303 ms/op 0.76
forkChoice updateHead vc 600000 bc 7200 eq 0 2.8529 ms/op 3.1648 ms/op 0.90
forkChoice updateHead vc 600000 bc 64 eq 1000 8.8981 ms/op 10.061 ms/op 0.88
forkChoice updateHead vc 600000 bc 64 eq 10000 8.8671 ms/op 10.509 ms/op 0.84
forkChoice updateHead vc 600000 bc 64 eq 300000 11.341 ms/op 13.348 ms/op 0.85
computeDeltas 500000 validators 300 proto nodes 3.1938 ms/op 3.9390 ms/op 0.81
computeDeltas 500000 validators 1200 proto nodes 3.2813 ms/op 4.0326 ms/op 0.81
computeDeltas 500000 validators 7200 proto nodes 3.3107 ms/op 3.9571 ms/op 0.84
computeDeltas 750000 validators 300 proto nodes 4.8996 ms/op 6.0330 ms/op 0.81
computeDeltas 750000 validators 1200 proto nodes 5.0144 ms/op 6.1541 ms/op 0.81
computeDeltas 750000 validators 7200 proto nodes 4.9927 ms/op 6.2275 ms/op 0.80
computeDeltas 1400000 validators 300 proto nodes 9.4670 ms/op 11.587 ms/op 0.82
computeDeltas 1400000 validators 1200 proto nodes 9.3822 ms/op 11.649 ms/op 0.81
computeDeltas 1400000 validators 7200 proto nodes 9.3862 ms/op 11.614 ms/op 0.81
computeDeltas 2100000 validators 300 proto nodes 14.013 ms/op 17.616 ms/op 0.80
computeDeltas 2100000 validators 1200 proto nodes 14.042 ms/op 17.409 ms/op 0.81
computeDeltas 2100000 validators 7200 proto nodes 14.356 ms/op 17.989 ms/op 0.80
altair processAttestation - 250000 vs - 7PWei normalcase 1.4831 ms/op 1.6642 ms/op 0.89
altair processAttestation - 250000 vs - 7PWei worstcase 2.1286 ms/op 2.5261 ms/op 0.84
altair processAttestation - setStatus - 1/6 committees join 65.938 us/op 93.740 us/op 0.70
altair processAttestation - setStatus - 1/3 committees join 126.13 us/op 174.90 us/op 0.72
altair processAttestation - setStatus - 1/2 committees join 200.22 us/op 249.59 us/op 0.80
altair processAttestation - setStatus - 2/3 committees join 261.81 us/op 328.97 us/op 0.80
altair processAttestation - setStatus - 4/5 committees join 372.23 us/op 483.20 us/op 0.77
altair processAttestation - setStatus - 100% committees join 460.30 us/op 575.41 us/op 0.80
altair processBlock - 250000 vs - 7PWei normalcase 3.6226 ms/op 5.7144 ms/op 0.63
altair processBlock - 250000 vs - 7PWei normalcase hashState 25.609 ms/op 25.768 ms/op 0.99
altair processBlock - 250000 vs - 7PWei worstcase 32.507 ms/op 36.829 ms/op 0.88
altair processBlock - 250000 vs - 7PWei worstcase hashState 65.478 ms/op 73.998 ms/op 0.88
phase0 processBlock - 250000 vs - 7PWei normalcase 1.4471 ms/op 2.4303 ms/op 0.60
phase0 processBlock - 250000 vs - 7PWei worstcase 21.344 ms/op 25.598 ms/op 0.83
altair processEth1Data - 250000 vs - 7PWei normalcase 244.82 us/op 337.42 us/op 0.73
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 4.9530 us/op 5.8120 us/op 0.85
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 28.066 us/op 36.373 us/op 0.77
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 7.8090 us/op 10.710 us/op 0.73
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 5.6870 us/op 7.1860 us/op 0.79
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 120.10 us/op 172.08 us/op 0.70
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 852.43 us/op 1.6278 ms/op 0.52
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.2384 ms/op 1.6714 ms/op 0.74
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.1907 ms/op 1.4420 ms/op 0.83
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.8595 ms/op 3.5320 ms/op 0.81
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.1999 ms/op 2.1378 ms/op 0.56
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 2.8747 ms/op 3.7022 ms/op 0.78
Tree 40 250000 create 177.71 ms/op 222.91 ms/op 0.80
Tree 40 250000 get(125000) 107.55 ns/op 155.08 ns/op 0.69
Tree 40 250000 set(125000) 529.02 ns/op 683.69 ns/op 0.77
Tree 40 250000 toArray() 13.579 ms/op 20.245 ms/op 0.67
Tree 40 250000 iterate all - toArray() + loop 13.277 ms/op 19.674 ms/op 0.67
Tree 40 250000 iterate all - get(i) 41.874 ms/op 55.397 ms/op 0.76
Array 250000 create 2.4877 ms/op 2.8607 ms/op 0.87
Array 250000 clone - spread 1.2944 ms/op 1.4130 ms/op 0.92
Array 250000 get(125000) 0.58700 ns/op 0.42800 ns/op 1.37
Array 250000 set(125000) 0.59100 ns/op 0.44500 ns/op 1.33
Array 250000 iterate all - loop 75.010 us/op 84.961 us/op 0.88
phase0 afterProcessEpoch - 250000 vs - 7PWei 40.398 ms/op 50.721 ms/op 0.80
Array.fill - length 1000000 2.4849 ms/op 3.5051 ms/op 0.71
Array push - length 1000000 9.3909 ms/op 17.706 ms/op 0.53
Array.get 0.26334 ns/op 0.27539 ns/op 0.96
Uint8Array.get 0.34495 ns/op 0.43945 ns/op 0.78
phase0 beforeProcessEpoch - 250000 vs - 7PWei 12.083 ms/op 18.093 ms/op 0.67
altair processEpoch - mainnet_e81889 280.98 ms/op 278.19 ms/op 1.01
mainnet_e81889 - altair beforeProcessEpoch 16.059 ms/op 20.030 ms/op 0.80
mainnet_e81889 - altair processJustificationAndFinalization 8.1790 us/op 14.213 us/op 0.58
mainnet_e81889 - altair processInactivityUpdates 4.2166 ms/op 5.7311 ms/op 0.74
mainnet_e81889 - altair processRewardsAndPenalties 56.105 ms/op 55.869 ms/op 1.00
mainnet_e81889 - altair processRegistryUpdates 1.9050 us/op 2.2930 us/op 0.83
mainnet_e81889 - altair processSlashings 718.00 ns/op 449.00 ns/op 1.60
mainnet_e81889 - altair processEth1DataReset 702.00 ns/op 773.00 ns/op 0.91
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.0166 ms/op 3.2627 ms/op 0.31
mainnet_e81889 - altair processSlashingsReset 2.5990 us/op 10.908 us/op 0.24
mainnet_e81889 - altair processRandaoMixesReset 4.2250 us/op 7.4220 us/op 0.57
mainnet_e81889 - altair processHistoricalRootsUpdate 824.00 ns/op 1.7530 us/op 0.47
mainnet_e81889 - altair processParticipationFlagUpdates 1.6810 us/op 8.6660 us/op 0.19
mainnet_e81889 - altair processSyncCommitteeUpdates 722.00 ns/op 1.6530 us/op 0.44
mainnet_e81889 - altair afterProcessEpoch 42.127 ms/op 58.071 ms/op 0.73
capella processEpoch - mainnet_e217614 997.40 ms/op 1.5775 s/op 0.63
mainnet_e217614 - capella beforeProcessEpoch 73.350 ms/op 151.79 ms/op 0.48
mainnet_e217614 - capella processJustificationAndFinalization 15.188 us/op 43.174 us/op 0.35
mainnet_e217614 - capella processInactivityUpdates 15.180 ms/op 34.807 ms/op 0.44
mainnet_e217614 - capella processRewardsAndPenalties 254.79 ms/op 354.85 ms/op 0.72
mainnet_e217614 - capella processRegistryUpdates 11.739 us/op 31.060 us/op 0.38
mainnet_e217614 - capella processSlashings 453.00 ns/op 668.00 ns/op 0.68
mainnet_e217614 - capella processEth1DataReset 674.00 ns/op 575.00 ns/op 1.17
mainnet_e217614 - capella processEffectiveBalanceUpdates 7.8190 ms/op 12.854 ms/op 0.61
mainnet_e217614 - capella processSlashingsReset 2.5420 us/op 3.5150 us/op 0.72
mainnet_e217614 - capella processRandaoMixesReset 5.7930 us/op 3.6510 us/op 1.59
mainnet_e217614 - capella processHistoricalRootsUpdate 1.0040 us/op 2.0300 us/op 0.49
mainnet_e217614 - capella processParticipationFlagUpdates 2.0410 us/op 7.9960 us/op 0.26
mainnet_e217614 - capella afterProcessEpoch 103.24 ms/op 142.00 ms/op 0.73
phase0 processEpoch - mainnet_e58758 320.08 ms/op 343.46 ms/op 0.93
mainnet_e58758 - phase0 beforeProcessEpoch 81.519 ms/op 102.09 ms/op 0.80
mainnet_e58758 - phase0 processJustificationAndFinalization 9.8580 us/op 37.066 us/op 0.27
mainnet_e58758 - phase0 processRewardsAndPenalties 24.397 ms/op 48.583 ms/op 0.50
mainnet_e58758 - phase0 processRegistryUpdates 9.8670 us/op 16.297 us/op 0.61
mainnet_e58758 - phase0 processSlashings 828.00 ns/op 789.00 ns/op 1.05
mainnet_e58758 - phase0 processEth1DataReset 786.00 ns/op 1.5690 us/op 0.50
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 909.31 us/op 1.9013 ms/op 0.48
mainnet_e58758 - phase0 processSlashingsReset 4.3320 us/op 3.4600 us/op 1.25
mainnet_e58758 - phase0 processRandaoMixesReset 4.6230 us/op 5.2280 us/op 0.88
mainnet_e58758 - phase0 processHistoricalRootsUpdate 701.00 ns/op 845.00 ns/op 0.83
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.1240 us/op 7.9580 us/op 0.39
mainnet_e58758 - phase0 afterProcessEpoch 37.099 ms/op 49.276 ms/op 0.75
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.3569 ms/op 2.4720 ms/op 0.55
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4196 ms/op 9.6080 ms/op 0.15
altair processInactivityUpdates - 250000 normalcase 15.845 ms/op 33.898 ms/op 0.47
altair processInactivityUpdates - 250000 worstcase 17.743 ms/op 29.788 ms/op 0.60
phase0 processRegistryUpdates - 250000 normalcase 9.1170 us/op 17.230 us/op 0.53
phase0 processRegistryUpdates - 250000 badcase_full_deposits 316.56 us/op 425.31 us/op 0.74
phase0 processRegistryUpdates - 250000 worstcase 0.5 116.77 ms/op 131.11 ms/op 0.89
altair processRewardsAndPenalties - 250000 normalcase 39.905 ms/op 51.527 ms/op 0.77
altair processRewardsAndPenalties - 250000 worstcase 43.552 ms/op 61.038 ms/op 0.71
phase0 getAttestationDeltas - 250000 normalcase 7.5517 ms/op 7.6217 ms/op 0.99
phase0 getAttestationDeltas - 250000 worstcase 6.3629 ms/op 7.3581 ms/op 0.86
phase0 processSlashings - 250000 worstcase 97.337 us/op 126.23 us/op 0.77
altair processSyncCommitteeUpdates - 250000 101.41 ms/op 127.70 ms/op 0.79
BeaconState.hashTreeRoot - No change 531.00 ns/op 236.00 ns/op 2.25
BeaconState.hashTreeRoot - 1 full validator 178.57 us/op 122.71 us/op 1.46
BeaconState.hashTreeRoot - 32 full validator 1.3318 ms/op 1.0481 ms/op 1.27
BeaconState.hashTreeRoot - 512 full validator 14.025 ms/op 15.550 ms/op 0.90
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 173.78 us/op 185.88 us/op 0.93
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.4804 ms/op 2.7515 ms/op 0.90
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 29.155 ms/op 41.677 ms/op 0.70
BeaconState.hashTreeRoot - 1 balances 92.116 us/op 146.63 us/op 0.63
BeaconState.hashTreeRoot - 32 balances 1.0078 ms/op 1.2319 ms/op 0.82
BeaconState.hashTreeRoot - 512 balances 8.2874 ms/op 12.863 ms/op 0.64
BeaconState.hashTreeRoot - 250000 balances 164.49 ms/op 170.05 ms/op 0.97
aggregationBits - 2048 els - zipIndexesInBitList 29.630 us/op 38.825 us/op 0.76
byteArrayEquals 32 50.239 ns/op 58.570 ns/op 0.86
Buffer.compare 32 16.013 ns/op 18.470 ns/op 0.87
byteArrayEquals 1024 1.3155 us/op 1.6357 us/op 0.80
Buffer.compare 1024 23.615 ns/op 26.619 ns/op 0.89
byteArrayEquals 16384 20.801 us/op 27.011 us/op 0.77
Buffer.compare 16384 208.32 ns/op 229.47 ns/op 0.91
byteArrayEquals 123687377 151.43 ms/op 195.33 ms/op 0.78
Buffer.compare 123687377 6.5670 ms/op 7.0265 ms/op 0.93
byteArrayEquals 32 - diff last byte 47.464 ns/op 54.081 ns/op 0.88
Buffer.compare 32 - diff last byte 15.671 ns/op 18.651 ns/op 0.84
byteArrayEquals 1024 - diff last byte 1.2603 us/op 1.6322 us/op 0.77
Buffer.compare 1024 - diff last byte 24.371 ns/op 27.193 ns/op 0.90
byteArrayEquals 16384 - diff last byte 20.250 us/op 25.724 us/op 0.79
Buffer.compare 16384 - diff last byte 199.89 ns/op 209.10 ns/op 0.96
byteArrayEquals 123687377 - diff last byte 153.94 ms/op 191.05 ms/op 0.81
Buffer.compare 123687377 - diff last byte 5.9880 ms/op 6.4105 ms/op 0.93
byteArrayEquals 32 - random bytes 4.9200 ns/op 5.1820 ns/op 0.95
Buffer.compare 32 - random bytes 15.652 ns/op 16.991 ns/op 0.92
byteArrayEquals 1024 - random bytes 4.8810 ns/op 5.1550 ns/op 0.95
Buffer.compare 1024 - random bytes 15.609 ns/op 17.005 ns/op 0.92
byteArrayEquals 16384 - random bytes 4.9140 ns/op 5.1230 ns/op 0.96
Buffer.compare 16384 - random bytes 15.361 ns/op 17.005 ns/op 0.90
byteArrayEquals 123687377 - random bytes 7.7800 ns/op 6.4900 ns/op 1.20
Buffer.compare 123687377 - random bytes 18.330 ns/op 18.330 ns/op 1.00
regular array get 100000 times 31.049 us/op 34.921 us/op 0.89
wrappedArray get 100000 times 30.864 us/op 33.041 us/op 0.93
arrayWithProxy get 100000 times 11.665 ms/op 13.143 ms/op 0.89
ssz.Root.equals 44.930 ns/op 46.250 ns/op 0.97
byteArrayEquals 44.492 ns/op 45.401 ns/op 0.98
Buffer.compare 9.3520 ns/op 10.449 ns/op 0.90
processSlot - 1 slots 14.628 us/op 18.034 us/op 0.81
processSlot - 32 slots 2.4012 ms/op 3.0447 ms/op 0.79
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 39.155 ms/op 41.504 ms/op 0.94
getCommitteeAssignments - req 1 vs - 250000 vc 1.7849 ms/op 2.0974 ms/op 0.85
getCommitteeAssignments - req 100 vs - 250000 vc 3.5472 ms/op 4.0071 ms/op 0.89
getCommitteeAssignments - req 1000 vs - 250000 vc 3.7467 ms/op 4.2799 ms/op 0.88
findModifiedValidators - 10000 modified validators 258.95 ms/op 238.01 ms/op 1.09
findModifiedValidators - 1000 modified validators 161.87 ms/op 167.17 ms/op 0.97
findModifiedValidators - 100 modified validators 166.91 ms/op 164.46 ms/op 1.01
findModifiedValidators - 10 modified validators 139.80 ms/op 165.15 ms/op 0.85
findModifiedValidators - 1 modified validators 154.60 ms/op 175.33 ms/op 0.88
findModifiedValidators - no difference 158.78 ms/op 162.60 ms/op 0.98
compare ViewDUs 3.0294 s/op 3.1299 s/op 0.97
compare each validator Uint8Array 1.5261 s/op 1.6158 s/op 0.94
compare ViewDU to Uint8Array 1.1729 s/op 957.35 ms/op 1.23
migrate state 1000000 validators, 24 modified, 0 new 787.00 ms/op 758.24 ms/op 1.04
migrate state 1000000 validators, 1700 modified, 1000 new 1.0020 s/op 1.0478 s/op 0.96
migrate state 1000000 validators, 3400 modified, 2000 new 1.1550 s/op 1.2439 s/op 0.93
migrate state 1500000 validators, 24 modified, 0 new 692.79 ms/op 779.86 ms/op 0.89
migrate state 1500000 validators, 1700 modified, 1000 new 907.01 ms/op 925.59 ms/op 0.98
migrate state 1500000 validators, 3400 modified, 2000 new 1.1816 s/op 1.2526 s/op 0.94
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.8100 ns/op 4.6100 ns/op 1.26
state getBlockRootAtSlot - 250000 vs - 7PWei 800.44 ns/op 763.16 ns/op 1.05
computeProposers - vc 250000 6.0089 ms/op 7.9138 ms/op 0.76
computeEpochShuffling - vc 250000 33.652 ms/op 41.797 ms/op 0.81
getNextSyncCommittee - vc 250000 100.81 ms/op 127.69 ms/op 0.79
computeSigningRoot for AttestationData 24.701 us/op 22.675 us/op 1.09
hash AttestationData serialized data then Buffer.toString(base64) 1.0933 us/op 1.5383 us/op 0.71
toHexString serialized data 726.27 ns/op 880.39 ns/op 0.82
Buffer.toString(base64) 134.51 ns/op 173.42 ns/op 0.78
nodejs block root to RootHex using toHex 123.28 ns/op 146.68 ns/op 0.84
nodejs block root to RootHex using toRootHex 69.488 ns/op 87.576 ns/op 0.79
browser block root to RootHex using the deprecated toHexString 192.71 ns/op 225.20 ns/op 0.86
browser block root to RootHex using toHex 155.40 ns/op 180.01 ns/op 0.86
browser block root to RootHex using toRootHex 140.72 ns/op 157.34 ns/op 0.89

by benchmarkbot/action

@nflaig
Copy link
Member

nflaig commented Oct 12, 2024

due to change made in #7130

Didn't change anything in regard to getBeaconProposersNextEpoch, making it async seems fine to me

@nflaig nflaig changed the title feat: make getBeaconProposersNextEpoch async to await for shuffling c… fix: make getBeaconProposersNextEpoch async to await for shuffling Oct 12, 2024
nflaig
nflaig reviewed Oct 12, 2024
View reviewed changes
Copy link
Member

@nflaig nflaig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unit tests are failing https://github.com/ChainSafe/lodestar/actions/runs/11304340795/job/31442660535?pr=7156#step:5:3296

@matthewkeil
Copy link
Member Author

unit tests are failing https://github.com/ChainSafe/lodestar/actions/runs/11304340795/job/31442660535?pr=7156#step:5:3296

Pushed it right before I left for dinner. Will check when I get home and fix

@matthewkeil
Copy link
Member Author

due to change made in #7130

Didn't change anything in regard to getBeaconProposersNextEpoch, making it async seems fine to me

@twoeths was pretty specific to keep EpochCache all sync. But you are right, I think it's the right fix

@nflaig
Copy link
Member

nflaig commented Oct 12, 2024

due to change made in #7130

Didn't change anything in regard to getBeaconProposersNextEpoch, making it async seems fine to me

@twoeths was pretty specific to keep EpochCache all sync. But you are right, I think it's the right fix

Makes sense in general, if we can come up with an alternative

@codecov Codecov
Copy link

codecov bot commented Oct 12, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Project coverage is 49.17%. Comparing base (d37bdb0) to head (9cb0c6f).
Report is 1 commits behind head on unstable.

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #7156      +/-   ##
============================================
- Coverage     49.18%   49.17%   -0.01%     
============================================
  Files           598      598              
  Lines         39801    39801              
  Branches       2082     2081       -1     
============================================
- Hits          19576    19574       -2     
- Misses        20184    20186       +2     
  Partials         41       41

nflaig
nflaig reviewed Oct 12, 2024
View reviewed changes
Copy link
Member

@nflaig nflaig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested the branch on mainnet, issue seems resolved, leaving it up to @twoeths to approve

@matthewkeil
Copy link
Member Author

Tested the branch on mainnet, issue seems resolved, leaving it up to @twoeths to approve

yep.. its deployed on feat2

twoeths
twoeths approved these changes Oct 14, 2024
View reviewed changes
Copy link
Contributor

@twoeths twoeths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, this looks simpler and thanks for keeping state-transition sync @matthewkeil

nflaig
nflaig approved these changes Oct 14, 2024
View reviewed changes
Copy link
Member

@nflaig nflaig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nflaig nflaig enabled auto-merge (squash) October 14, 2024 12:15
@nflaig nflaig disabled auto-merge October 14, 2024 12:15
@nflaig nflaig changed the title fix: make getBeaconProposersNextEpoch async to await for shuffling fix: make sure shuffling is calculated when querying next epoch proposers Oct 14, 2024
@nflaig nflaig enabled auto-merge (squash) October 14, 2024 12:20
@nflaig nflaig merged commit 2a7f7e6 into unstable Oct 14, 2024
22 checks passed
@nflaig nflaig deleted the mkeil/async-get-next-proposers branch October 14, 2024 12:27
philknows pushed a commit that referenced this pull request Oct 18, 2024
@matthewkeil @philknows
fix: make sure shuffling is calculated when querying next epoch propo…
5ca8a96 
…sers (#7156)

* feat: make getBeaconProposersNextEpoch async to await for shuffling calculation

* fix: build error in getBeaconProposersNextEpoch

* fix: failed unit test

* refactor: use tuyen suggestion to await for shuffling keeping state-transition sync
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.23.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nflaig nflaig nflaig approved these changes

@twoeths twoeths twoeths approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@matthewkeil @nflaig @wemeetagain @twoeths