Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update kzg to the latest decoupled and bigendian version #5659

Merged
merged 2 commits into from
Jun 17, 2023

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Jun 16, 2023

  1. updates kzg to latest decoupled and bigendian version
  2. adds a cli flag to provide customized trusted setup txt file
  3. updates trusted setup txt to latest devnet-6 version fron c-kzg main repo trusted setup
  4. remove the bin=> txt conversion step to directly use the trusted setup txt format

@g11tech g11tech requested a review from a team as a code owner June 16, 2023 13:37
@github-actions
Copy link
Contributor

github-actions bot commented Jun 16, 2023

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 48f2106 Previous: 83d1b7c Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 476.13 us/op 914.05 us/op 0.52
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 45.830 us/op 50.459 us/op 0.91
BLS verify - blst-native 1.2165 ms/op 1.2816 ms/op 0.95
BLS verifyMultipleSignatures 3 - blst-native 2.4742 ms/op 2.5326 ms/op 0.98
BLS verifyMultipleSignatures 8 - blst-native 5.3081 ms/op 5.6352 ms/op 0.94
BLS verifyMultipleSignatures 32 - blst-native 19.205 ms/op 19.873 ms/op 0.97
BLS aggregatePubkeys 32 - blst-native 25.635 us/op 26.977 us/op 0.95
BLS aggregatePubkeys 128 - blst-native 100.65 us/op 108.06 us/op 0.93
getAttestationsForBlock 52.036 ms/op 57.822 ms/op 0.90
isKnown best case - 1 super set check 252.00 ns/op 259.00 ns/op 0.97
isKnown normal case - 2 super set checks 245.00 ns/op 251.00 ns/op 0.98
isKnown worse case - 16 super set checks 250.00 ns/op 252.00 ns/op 0.99
CheckpointStateCache - add get delete 4.9120 us/op 5.8460 us/op 0.84
validate gossip signedAggregateAndProof - struct 2.7525 ms/op 2.8861 ms/op 0.95
validate gossip attestation - struct 1.3022 ms/op 1.3797 ms/op 0.94
pickEth1Vote - no votes 1.1951 ms/op 1.4051 ms/op 0.85
pickEth1Vote - max votes 9.2105 ms/op 10.237 ms/op 0.90
pickEth1Vote - Eth1Data hashTreeRoot value x2048 8.7805 ms/op 9.6611 ms/op 0.91
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 14.910 ms/op 17.388 ms/op 0.86
pickEth1Vote - Eth1Data fastSerialize value x2048 699.13 us/op 799.56 us/op 0.87
pickEth1Vote - Eth1Data fastSerialize tree x2048 4.9468 ms/op 7.4531 ms/op 0.66
bytes32 toHexString 531.00 ns/op 717.00 ns/op 0.74
bytes32 Buffer.toString(hex) 395.00 ns/op 433.00 ns/op 0.91
bytes32 Buffer.toString(hex) from Uint8Array 583.00 ns/op 615.00 ns/op 0.95
bytes32 Buffer.toString(hex) + 0x 372.00 ns/op 437.00 ns/op 0.85
Object access 1 prop 0.16700 ns/op 0.19200 ns/op 0.87
Map access 1 prop 0.15300 ns/op 0.17200 ns/op 0.89
Object get x1000 6.6700 ns/op 7.3020 ns/op 0.91
Map get x1000 0.62600 ns/op 0.70000 ns/op 0.89
Object set x1000 55.507 ns/op 64.394 ns/op 0.86
Map set x1000 44.659 ns/op 55.124 ns/op 0.81
Return object 10000 times 0.24060 ns/op 0.25340 ns/op 0.95
Throw Error 10000 times 4.3586 us/op 4.3702 us/op 1.00
fastMsgIdFn sha256 / 200 bytes 3.5230 us/op 3.5910 us/op 0.98
fastMsgIdFn h32 xxhash / 200 bytes 308.00 ns/op 302.00 ns/op 1.02
fastMsgIdFn h64 xxhash / 200 bytes 436.00 ns/op 441.00 ns/op 0.99
fastMsgIdFn sha256 / 1000 bytes 11.631 us/op 11.925 us/op 0.98
fastMsgIdFn h32 xxhash / 1000 bytes 428.00 ns/op 432.00 ns/op 0.99
fastMsgIdFn h64 xxhash / 1000 bytes 495.00 ns/op 514.00 ns/op 0.96
fastMsgIdFn sha256 / 10000 bytes 103.82 us/op 107.83 us/op 0.96
fastMsgIdFn h32 xxhash / 10000 bytes 2.0650 us/op 2.0160 us/op 1.02
fastMsgIdFn h64 xxhash / 10000 bytes 1.4740 us/op 1.4460 us/op 1.02
enrSubnets - fastDeserialize 64 bits 1.2930 us/op 1.3750 us/op 0.94
enrSubnets - ssz BitVector 64 bits 491.00 ns/op 519.00 ns/op 0.95
enrSubnets - fastDeserialize 4 bits 192.00 ns/op 178.00 ns/op 1.08
enrSubnets - ssz BitVector 4 bits 593.00 ns/op 512.00 ns/op 1.16
prioritizePeers score -10:0 att 32-0.1 sync 2-0 110.92 us/op 113.84 us/op 0.97
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 152.32 us/op 151.63 us/op 1.00
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 190.47 us/op 183.30 us/op 1.04
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 368.42 us/op 362.29 us/op 1.02
prioritizePeers score 0:0 att 64-1 sync 4-1 433.89 us/op 419.50 us/op 1.03
array of 16000 items push then shift 1.6693 us/op 1.7271 us/op 0.97
LinkedList of 16000 items push then shift 8.9370 ns/op 9.1040 ns/op 0.98
array of 16000 items push then pop 116.22 ns/op 111.69 ns/op 1.04
LinkedList of 16000 items push then pop 8.8930 ns/op 8.9120 ns/op 1.00
array of 24000 items push then shift 2.4131 us/op 2.3816 us/op 1.01
LinkedList of 24000 items push then shift 9.1080 ns/op 9.3030 ns/op 0.98
array of 24000 items push then pop 77.014 ns/op 84.662 ns/op 0.91
LinkedList of 24000 items push then pop 8.4410 ns/op 8.8480 ns/op 0.95
intersect bitArray bitLen 8 13.636 ns/op 13.704 ns/op 1.00
intersect array and set length 8 79.783 ns/op 88.226 ns/op 0.90
intersect bitArray bitLen 128 43.345 ns/op 46.095 ns/op 0.94
intersect array and set length 128 1.1454 us/op 1.2935 us/op 0.89
Buffer.concat 32 items 3.0840 us/op 3.1890 us/op 0.97
Uint8Array.set 32 items 2.8380 us/op 2.2910 us/op 1.24
transfer serialized Status (84 B) 2.0650 us/op 2.1160 us/op 0.98
copy serialized Status (84 B) 1.7490 us/op 1.7690 us/op 0.99
transfer serialized SignedVoluntaryExit (112 B) 2.1180 us/op 2.2540 us/op 0.94
copy serialized SignedVoluntaryExit (112 B) 1.7020 us/op 1.8910 us/op 0.90
transfer serialized ProposerSlashing (416 B) 2.2730 us/op 2.5930 us/op 0.88
copy serialized ProposerSlashing (416 B) 2.5110 us/op 2.8160 us/op 0.89
transfer serialized Attestation (485 B) 2.4400 us/op 2.6080 us/op 0.94
copy serialized Attestation (485 B) 2.0750 us/op 2.3330 us/op 0.89
transfer serialized AttesterSlashing (33232 B) 2.6210 us/op 2.5050 us/op 1.05
copy serialized AttesterSlashing (33232 B) 5.0230 us/op 6.8110 us/op 0.74
transfer serialized Small SignedBeaconBlock (128000 B) 3.1180 us/op 3.3860 us/op 0.92
copy serialized Small SignedBeaconBlock (128000 B) 21.897 us/op 21.147 us/op 1.04
transfer serialized Avg SignedBeaconBlock (200000 B) 3.3430 us/op 3.5600 us/op 0.94
copy serialized Avg SignedBeaconBlock (200000 B) 19.323 us/op 26.712 us/op 0.72
transfer serialized BlobsSidecar (524380 B) 3.0150 us/op 3.5460 us/op 0.85
copy serialized BlobsSidecar (524380 B) 88.970 us/op 217.95 us/op 0.41
transfer serialized Big SignedBeaconBlock (1000000 B) 3.2860 us/op 3.9670 us/op 0.83
copy serialized Big SignedBeaconBlock (1000000 B) 273.47 us/op 392.15 us/op 0.70
pass gossip attestations to forkchoice per slot 2.7911 ms/op 2.8193 ms/op 0.99
forkChoice updateHead vc 100000 bc 64 eq 0 2.1031 ms/op 2.1868 ms/op 0.96
forkChoice updateHead vc 600000 bc 64 eq 0 11.737 ms/op 12.589 ms/op 0.93
forkChoice updateHead vc 1000000 bc 64 eq 0 22.744 ms/op 26.057 ms/op 0.87
forkChoice updateHead vc 600000 bc 320 eq 0 17.098 ms/op 21.889 ms/op 0.78
forkChoice updateHead vc 600000 bc 1200 eq 0 81.326 ms/op 93.042 ms/op 0.87
forkChoice updateHead vc 600000 bc 64 eq 1000 20.964 ms/op 21.872 ms/op 0.96
forkChoice updateHead vc 600000 bc 64 eq 10000 23.042 ms/op 27.598 ms/op 0.83
forkChoice updateHead vc 600000 bc 64 eq 300000 52.710 ms/op 42.576 ms/op 1.24
computeDeltas 3.1187 ms/op 3.4384 ms/op 0.91
computeProposerBoostScoreFromBalances 1.8198 ms/op 1.9258 ms/op 0.94
altair processAttestation - 250000 vs - 7PWei normalcase 2.5506 ms/op 2.9319 ms/op 0.87
altair processAttestation - 250000 vs - 7PWei worstcase 3.6338 ms/op 3.9891 ms/op 0.91
altair processAttestation - setStatus - 1/6 committees join 140.47 us/op 157.94 us/op 0.89
altair processAttestation - setStatus - 1/3 committees join 289.63 us/op 300.95 us/op 0.96
altair processAttestation - setStatus - 1/2 committees join 376.22 us/op 388.69 us/op 0.97
altair processAttestation - setStatus - 2/3 committees join 479.68 us/op 481.85 us/op 1.00
altair processAttestation - setStatus - 4/5 committees join 658.76 us/op 709.24 us/op 0.93
altair processAttestation - setStatus - 100% committees join 786.60 us/op 795.50 us/op 0.99
altair processBlock - 250000 vs - 7PWei normalcase 19.311 ms/op 18.093 ms/op 1.07
altair processBlock - 250000 vs - 7PWei normalcase hashState 28.578 ms/op 27.644 ms/op 1.03
altair processBlock - 250000 vs - 7PWei worstcase 49.359 ms/op 53.011 ms/op 0.93
altair processBlock - 250000 vs - 7PWei worstcase hashState 70.203 ms/op 72.430 ms/op 0.97
phase0 processBlock - 250000 vs - 7PWei normalcase 2.4041 ms/op 2.5670 ms/op 0.94
phase0 processBlock - 250000 vs - 7PWei worstcase 29.970 ms/op 32.790 ms/op 0.91
altair processEth1Data - 250000 vs - 7PWei normalcase 579.97 us/op 607.81 us/op 0.95
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 7.9760 us/op 11.385 us/op 0.70
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 27.364 us/op 33.869 us/op 0.81
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 9.9540 us/op 12.476 us/op 0.80
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 7.7430 us/op 9.2200 us/op 0.84
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 106.79 us/op 101.35 us/op 1.05
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 657.00 us/op 685.62 us/op 0.96
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 921.02 us/op 924.48 us/op 1.00
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 921.79 us/op 918.48 us/op 1.00
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.4142 ms/op 2.4453 ms/op 0.99
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.6216 ms/op 1.8150 ms/op 0.89
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.9926 ms/op 4.3968 ms/op 0.91
Tree 40 250000 create 337.59 ms/op 373.35 ms/op 0.90
Tree 40 250000 get(125000) 194.03 ns/op 200.11 ns/op 0.97
Tree 40 250000 set(125000) 1.0359 us/op 1.0376 us/op 1.00
Tree 40 250000 toArray() 22.277 ms/op 21.467 ms/op 1.04
Tree 40 250000 iterate all - toArray() + loop 22.168 ms/op 21.697 ms/op 1.02
Tree 40 250000 iterate all - get(i) 72.786 ms/op 76.931 ms/op 0.95
MutableVector 250000 create 12.450 ms/op 11.274 ms/op 1.10
MutableVector 250000 get(125000) 6.2790 ns/op 6.9270 ns/op 0.91
MutableVector 250000 set(125000) 296.31 ns/op 304.17 ns/op 0.97
MutableVector 250000 toArray() 2.9958 ms/op 4.0918 ms/op 0.73
MutableVector 250000 iterate all - toArray() + loop 3.0844 ms/op 3.9803 ms/op 0.77
MutableVector 250000 iterate all - get(i) 1.5310 ms/op 1.5572 ms/op 0.98
Array 250000 create 2.7600 ms/op 3.6499 ms/op 0.76
Array 250000 clone - spread 1.1688 ms/op 1.4693 ms/op 0.80
Array 250000 get(125000) 0.57700 ns/op 0.70900 ns/op 0.81
Array 250000 set(125000) 0.62400 ns/op 0.82900 ns/op 0.75
Array 250000 iterate all - loop 104.66 us/op 92.425 us/op 1.13
effectiveBalanceIncrements clone Uint8Array 300000 34.507 us/op 49.952 us/op 0.69
effectiveBalanceIncrements clone MutableVector 300000 360.00 ns/op 453.00 ns/op 0.79
effectiveBalanceIncrements rw all Uint8Array 300000 169.06 us/op 178.14 us/op 0.95
effectiveBalanceIncrements rw all MutableVector 300000 82.508 ms/op 102.80 ms/op 0.80
phase0 afterProcessEpoch - 250000 vs - 7PWei 116.43 ms/op 116.39 ms/op 1.00
phase0 beforeProcessEpoch - 250000 vs - 7PWei 44.501 ms/op 35.915 ms/op 1.24
altair processEpoch - mainnet_e81889 308.90 ms/op 324.82 ms/op 0.95
mainnet_e81889 - altair beforeProcessEpoch 51.854 ms/op 66.419 ms/op 0.78
mainnet_e81889 - altair processJustificationAndFinalization 18.555 us/op 16.404 us/op 1.13
mainnet_e81889 - altair processInactivityUpdates 8.5200 ms/op 6.3715 ms/op 1.34
mainnet_e81889 - altair processRewardsAndPenalties 49.836 ms/op 49.900 ms/op 1.00
mainnet_e81889 - altair processRegistryUpdates 3.1390 us/op 2.3200 us/op 1.35
mainnet_e81889 - altair processSlashings 581.00 ns/op 444.00 ns/op 1.31
mainnet_e81889 - altair processEth1DataReset 970.00 ns/op 474.00 ns/op 2.05
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.3052 ms/op 1.2518 ms/op 1.04
mainnet_e81889 - altair processSlashingsReset 5.4230 us/op 4.0490 us/op 1.34
mainnet_e81889 - altair processRandaoMixesReset 5.0630 us/op 4.3200 us/op 1.17
mainnet_e81889 - altair processHistoricalRootsUpdate 789.00 ns/op 909.00 ns/op 0.87
mainnet_e81889 - altair processParticipationFlagUpdates 3.6540 us/op 4.5330 us/op 0.81
mainnet_e81889 - altair processSyncCommitteeUpdates 1.2830 us/op 597.00 ns/op 2.15
mainnet_e81889 - altair afterProcessEpoch 135.35 ms/op 127.18 ms/op 1.06
phase0 processEpoch - mainnet_e58758 385.48 ms/op 361.63 ms/op 1.07
mainnet_e58758 - phase0 beforeProcessEpoch 144.79 ms/op 137.01 ms/op 1.06
mainnet_e58758 - phase0 processJustificationAndFinalization 20.241 us/op 16.745 us/op 1.21
mainnet_e58758 - phase0 processRewardsAndPenalties 62.128 ms/op 65.661 ms/op 0.95
mainnet_e58758 - phase0 processRegistryUpdates 8.2670 us/op 7.8080 us/op 1.06
mainnet_e58758 - phase0 processSlashings 507.00 ns/op 513.00 ns/op 0.99
mainnet_e58758 - phase0 processEth1DataReset 520.00 ns/op 510.00 ns/op 1.02
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0189 ms/op 1.0401 ms/op 0.98
mainnet_e58758 - phase0 processSlashingsReset 3.9740 us/op 4.7210 us/op 0.84
mainnet_e58758 - phase0 processRandaoMixesReset 4.5710 us/op 4.1150 us/op 1.11
mainnet_e58758 - phase0 processHistoricalRootsUpdate 770.00 ns/op 672.00 ns/op 1.15
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.1260 us/op 4.5090 us/op 0.92
mainnet_e58758 - phase0 afterProcessEpoch 100.39 ms/op 97.223 ms/op 1.03
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2748 ms/op 1.2113 ms/op 1.05
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.5333 ms/op 1.4762 ms/op 1.04
altair processInactivityUpdates - 250000 normalcase 25.568 ms/op 25.403 ms/op 1.01
altair processInactivityUpdates - 250000 worstcase 27.206 ms/op 26.045 ms/op 1.04
phase0 processRegistryUpdates - 250000 normalcase 8.5270 us/op 7.7290 us/op 1.10
phase0 processRegistryUpdates - 250000 badcase_full_deposits 278.00 us/op 261.89 us/op 1.06
phase0 processRegistryUpdates - 250000 worstcase 0.5 133.22 ms/op 116.04 ms/op 1.15
altair processRewardsAndPenalties - 250000 normalcase 62.744 ms/op 69.356 ms/op 0.90
altair processRewardsAndPenalties - 250000 worstcase 64.291 ms/op 65.977 ms/op 0.97
phase0 getAttestationDeltas - 250000 normalcase 7.0007 ms/op 6.8377 ms/op 1.02
phase0 getAttestationDeltas - 250000 worstcase 6.9177 ms/op 6.6218 ms/op 1.04
phase0 processSlashings - 250000 worstcase 3.4242 ms/op 3.6077 ms/op 0.95
altair processSyncCommitteeUpdates - 250000 186.65 ms/op 176.74 ms/op 1.06
BeaconState.hashTreeRoot - No change 282.00 ns/op 270.00 ns/op 1.04
BeaconState.hashTreeRoot - 1 full validator 56.385 us/op 51.148 us/op 1.10
BeaconState.hashTreeRoot - 32 full validator 536.63 us/op 523.68 us/op 1.02
BeaconState.hashTreeRoot - 512 full validator 5.3215 ms/op 5.5491 ms/op 0.96
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 68.210 us/op 63.557 us/op 1.07
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 930.57 us/op 867.50 us/op 1.07
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 11.733 ms/op 11.535 ms/op 1.02
BeaconState.hashTreeRoot - 1 balances 50.685 us/op 48.523 us/op 1.04
BeaconState.hashTreeRoot - 32 balances 493.56 us/op 447.09 us/op 1.10
BeaconState.hashTreeRoot - 512 balances 4.6188 ms/op 4.9143 ms/op 0.94
BeaconState.hashTreeRoot - 250000 balances 74.037 ms/op 78.221 ms/op 0.95
aggregationBits - 2048 els - zipIndexesInBitList 15.737 us/op 17.410 us/op 0.90
regular array get 100000 times 33.283 us/op 43.407 us/op 0.77
wrappedArray get 100000 times 33.181 us/op 44.505 us/op 0.75
arrayWithProxy get 100000 times 17.343 ms/op 15.390 ms/op 1.13
ssz.Root.equals 542.00 ns/op 585.00 ns/op 0.93
byteArrayEquals 576.00 ns/op 539.00 ns/op 1.07
shuffle list - 16384 els 7.1299 ms/op 6.9490 ms/op 1.03
shuffle list - 250000 els 103.07 ms/op 101.92 ms/op 1.01
processSlot - 1 slots 9.3810 us/op 9.5860 us/op 0.98
processSlot - 32 slots 1.4414 ms/op 1.3918 ms/op 1.04
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 36.387 ms/op 35.574 ms/op 1.02
getCommitteeAssignments - req 1 vs - 250000 vc 2.8949 ms/op 3.2165 ms/op 0.90
getCommitteeAssignments - req 100 vs - 250000 vc 4.1055 ms/op 4.2213 ms/op 0.97
getCommitteeAssignments - req 1000 vs - 250000 vc 4.4565 ms/op 4.5579 ms/op 0.98
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.9300 ns/op 4.9800 ns/op 0.99
state getBlockRootAtSlot - 250000 vs - 7PWei 603.05 ns/op 642.81 ns/op 0.94
computeProposers - vc 250000 10.629 ms/op 10.414 ms/op 1.02
computeEpochShuffling - vc 250000 102.57 ms/op 106.46 ms/op 0.96
getNextSyncCommittee - vc 250000 172.39 ms/op 195.75 ms/op 0.88
computeSigningRoot for AttestationData 12.796 us/op 14.112 us/op 0.91
hash AttestationData serialized data then Buffer.toString(base64) 2.4400 us/op 2.5590 us/op 0.95
toHexString serialized data 1.0604 us/op 1.0972 us/op 0.97
Buffer.toString(base64) 320.05 ns/op 353.51 ns/op 0.91

by benchmarkbot/action

@g11tech g11tech enabled auto-merge (squash) June 17, 2023 06:43
@g11tech g11tech merged commit 4313726 into unstable Jun 17, 2023
@g11tech g11tech deleted the update-kzg branch June 17, 2023 07:33
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.9.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants