ci: add support for the CI env for e2e tests

[nazarhussain]

Motivation

Reduce dependencies of external resources during the e2e tests.

Description

• Removed dependency of e2e tests to external sepolia nodes
• Created a script to spinup the environment for the e2e tests
• Take backup of the logs just in case need to debug

Steps to test or reproduce

Run e2e tests.

[github-actions]

Performance Report

✔️ no performance regression detected

Full benchmark results

Benchmark suite	Current: 6ea3064	Previous: 8490369	Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc	1.0134 ms/op	497.32 us/op	2.04
getPubkeys - validatorsArr - req 1000 vs - 250000 vc	97.279 us/op	45.225 us/op	2.15
BLS verify - blst-native	1.7069 ms/op	1.1990 ms/op	1.42
BLS verifyMultipleSignatures 3 - blst-native	3.1899 ms/op	2.4431 ms/op	1.31
BLS verifyMultipleSignatures 8 - blst-native	6.7082 ms/op	5.2412 ms/op	1.28
BLS verifyMultipleSignatures 32 - blst-native	23.791 ms/op	19.026 ms/op	1.25
BLS aggregatePubkeys 32 - blst-native	34.207 us/op	25.494 us/op	1.34
BLS aggregatePubkeys 128 - blst-native	118.44 us/op	99.038 us/op	1.20
getAttestationsForBlock	121.25 ms/op	60.243 ms/op	2.01
isKnown best case - 1 super set check	571.00 ns/op	263.00 ns/op	2.17
isKnown normal case - 2 super set checks	459.00 ns/op	263.00 ns/op	1.75
isKnown worse case - 16 super set checks	491.00 ns/op	260.00 ns/op	1.89
CheckpointStateCache - add get delete	9.6160 us/op	5.7350 us/op	1.68
validate gossip signedAggregateAndProof - struct	4.2296 ms/op	2.7782 ms/op	1.52
validate gossip attestation - struct	1.8033 ms/op	1.3178 ms/op	1.37
pickEth1Vote - no votes	2.3621 ms/op	1.3828 ms/op	1.71
pickEth1Vote - max votes	22.692 ms/op	10.783 ms/op	2.10
pickEth1Vote - Eth1Data hashTreeRoot value x2048	15.774 ms/op	9.8935 ms/op	1.59
pickEth1Vote - Eth1Data hashTreeRoot tree x2048	24.717 ms/op	16.273 ms/op	1.52
pickEth1Vote - Eth1Data fastSerialize value x2048	1.1301 ms/op	741.19 us/op	1.52
pickEth1Vote - Eth1Data fastSerialize tree x2048	8.5738 ms/op	5.7572 ms/op	1.49
bytes32 toHexString	1.1440 us/op	703.00 ns/op	1.63
bytes32 Buffer.toString(hex)	736.00 ns/op	429.00 ns/op	1.72
bytes32 Buffer.toString(hex) from Uint8Array	933.00 ns/op	613.00 ns/op	1.52
bytes32 Buffer.toString(hex) + 0x	665.00 ns/op	427.00 ns/op	1.56
Object access 1 prop	0.27000 ns/op	0.20400 ns/op	1.32
Map access 1 prop	0.24100 ns/op	0.16600 ns/op	1.45
Object get x1000	13.673 ns/op	6.7580 ns/op	2.02
Map get x1000	1.5160 ns/op	0.61500 ns/op	2.47
Object set x1000	95.245 ns/op	68.331 ns/op	1.39
Map set x1000	66.219 ns/op	54.340 ns/op	1.22
Return object 10000 times	0.41240 ns/op	0.24640 ns/op	1.67
Throw Error 10000 times	5.8381 us/op	4.3296 us/op	1.35
fastMsgIdFn sha256 / 200 bytes	4.3040 us/op	3.5840 us/op	1.20
fastMsgIdFn h32 xxhash / 200 bytes	610.00 ns/op	328.00 ns/op	1.86
fastMsgIdFn h64 xxhash / 200 bytes	666.00 ns/op	479.00 ns/op	1.39
fastMsgIdFn sha256 / 1000 bytes	13.885 us/op	11.847 us/op	1.17
fastMsgIdFn h32 xxhash / 1000 bytes	610.00 ns/op	465.00 ns/op	1.31
fastMsgIdFn h64 xxhash / 1000 bytes	710.00 ns/op	570.00 ns/op	1.25
fastMsgIdFn sha256 / 10000 bytes	121.00 us/op	104.38 us/op	1.16
fastMsgIdFn h32 xxhash / 10000 bytes	2.4580 us/op	1.9990 us/op	1.23
fastMsgIdFn h64 xxhash / 10000 bytes	1.9750 us/op	1.4680 us/op	1.35
enrSubnets - fastDeserialize 64 bits	2.4060 us/op	1.7630 us/op	1.36
enrSubnets - ssz BitVector 64 bits	861.00 ns/op	640.00 ns/op	1.35
enrSubnets - fastDeserialize 4 bits	322.00 ns/op	215.00 ns/op	1.50
enrSubnets - ssz BitVector 4 bits	1.0530 us/op	641.00 ns/op	1.64
prioritizePeers score -10:0 att 32-0.1 sync 2-0	183.31 us/op	116.91 us/op	1.57
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25	255.81 us/op	164.50 us/op	1.56
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5	301.46 us/op	197.32 us/op	1.53
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75	536.76 us/op	375.24 us/op	1.43
prioritizePeers score 0:0 att 64-1 sync 4-1	556.43 us/op	454.58 us/op	1.22
array of 16000 items push then shift	2.1042 us/op	1.6856 us/op	1.25
LinkedList of 16000 items push then shift	12.475 ns/op	9.5600 ns/op	1.30
array of 16000 items push then pop	128.31 ns/op	120.49 ns/op	1.06
LinkedList of 16000 items push then pop	12.000 ns/op	9.1090 ns/op	1.32
array of 24000 items push then shift	2.8519 us/op	2.3736 us/op	1.20
LinkedList of 24000 items push then shift	11.519 ns/op	9.5760 ns/op	1.20
array of 24000 items push then pop	101.94 ns/op	94.468 ns/op	1.08
LinkedList of 24000 items push then pop	9.2370 ns/op	9.0590 ns/op	1.02
intersect bitArray bitLen 8	14.472 ns/op	13.504 ns/op	1.07
intersect array and set length 8	82.317 ns/op	105.85 ns/op	0.78
intersect bitArray bitLen 128	48.387 ns/op	45.217 ns/op	1.07
intersect array and set length 128	1.2590 us/op	1.3050 us/op	0.96
Buffer.concat 32 items	2.8310 us/op	3.0030 us/op	0.94
Uint8Array.set 32 items	2.4180 us/op	2.4150 us/op	1.00
transfer serialized Status (84 B)	2.2650 us/op	2.2580 us/op	1.00
copy serialized Status (84 B)	1.8010 us/op	1.9210 us/op	0.94
transfer serialized SignedVoluntaryExit (112 B)	2.2470 us/op	2.4960 us/op	0.90
copy serialized SignedVoluntaryExit (112 B)	1.8680 us/op	1.9640 us/op	0.95
transfer serialized ProposerSlashing (416 B)	2.6450 us/op	2.9260 us/op	0.90
copy serialized ProposerSlashing (416 B)	2.5190 us/op	3.2060 us/op	0.79
transfer serialized Attestation (485 B)	2.6420 us/op	3.3390 us/op	0.79
copy serialized Attestation (485 B)	2.4100 us/op	3.1090 us/op	0.78
transfer serialized AttesterSlashing (33232 B)	2.6540 us/op	2.8370 us/op	0.94
copy serialized AttesterSlashing (33232 B)	8.0950 us/op	8.9810 us/op	0.90
transfer serialized Small SignedBeaconBlock (128000 B)	2.9190 us/op	3.6780 us/op	0.79
copy serialized Small SignedBeaconBlock (128000 B)	30.078 us/op	32.235 us/op	0.93
transfer serialized Avg SignedBeaconBlock (200000 B)	3.4780 us/op	3.6390 us/op	0.96
copy serialized Avg SignedBeaconBlock (200000 B)	74.559 us/op	32.741 us/op	2.28
transfer serialized BlobsSidecar (524380 B)	3.2970 us/op	3.9790 us/op	0.83
copy serialized BlobsSidecar (524380 B)	203.58 us/op	213.59 us/op	0.95
transfer serialized Big SignedBeaconBlock (1000000 B)	3.5990 us/op	3.9490 us/op	0.91
copy serialized Big SignedBeaconBlock (1000000 B)	371.38 us/op	366.68 us/op	1.01
pass gossip attestations to forkchoice per slot	2.5372 ms/op	2.4138 ms/op	1.05
computeDeltas	3.2818 ms/op	3.1568 ms/op	1.04
computeProposerBoostScoreFromBalances	1.8971 ms/op	1.7702 ms/op	1.07
altair processAttestation - 250000 vs - 7PWei normalcase	3.4450 ms/op	2.3706 ms/op	1.45
altair processAttestation - 250000 vs - 7PWei worstcase	4.6646 ms/op	3.3652 ms/op	1.39
altair processAttestation - setStatus - 1/6 committees join	150.96 us/op	140.85 us/op	1.07
altair processAttestation - setStatus - 1/3 committees join	298.76 us/op	284.81 us/op	1.05
altair processAttestation - setStatus - 1/2 committees join	395.23 us/op	369.74 us/op	1.07
altair processAttestation - setStatus - 2/3 committees join	506.24 us/op	458.18 us/op	1.10
altair processAttestation - setStatus - 4/5 committees join	707.38 us/op	661.12 us/op	1.07
altair processAttestation - setStatus - 100% committees join	811.59 us/op	745.14 us/op	1.09
altair processBlock - 250000 vs - 7PWei normalcase	18.736 ms/op	14.458 ms/op	1.30
altair processBlock - 250000 vs - 7PWei normalcase hashState	26.867 ms/op	25.233 ms/op	1.06
altair processBlock - 250000 vs - 7PWei worstcase	55.078 ms/op	46.349 ms/op	1.19
altair processBlock - 250000 vs - 7PWei worstcase hashState	73.359 ms/op	68.550 ms/op	1.07
phase0 processBlock - 250000 vs - 7PWei normalcase	2.4109 ms/op	2.3659 ms/op	1.02
phase0 processBlock - 250000 vs - 7PWei worstcase	32.017 ms/op	29.189 ms/op	1.10
altair processEth1Data - 250000 vs - 7PWei normalcase	660.85 us/op	487.47 us/op	1.36
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15	9.6670 us/op	7.7250 us/op	1.25
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219	33.397 us/op	22.694 us/op	1.47
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42	13.609 us/op	9.0060 us/op	1.51
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18	9.3990 us/op	6.9600 us/op	1.35
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020	118.07 us/op	90.176 us/op	1.31
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777	678.77 us/op	626.61 us/op	1.08
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384	921.85 us/op	896.49 us/op	1.03
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384	926.83 us/op	868.37 us/op	1.07
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384	2.4785 ms/op	2.1902 ms/op	1.13
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384	1.7065 ms/op	1.4941 ms/op	1.14
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384	4.5599 ms/op	4.1218 ms/op	1.11
Tree 40 250000 create	436.05 ms/op	309.05 ms/op	1.41
Tree 40 250000 get(125000)	199.45 ns/op	176.44 ns/op	1.13
Tree 40 250000 set(125000)	1.1995 us/op	892.00 ns/op	1.34
Tree 40 250000 toArray()	22.902 ms/op	17.256 ms/op	1.33
Tree 40 250000 iterate all - toArray() + loop	22.899 ms/op	17.051 ms/op	1.34
Tree 40 250000 iterate all - get(i)	76.263 ms/op	65.847 ms/op	1.16
MutableVector 250000 create	11.399 ms/op	10.563 ms/op	1.08
MutableVector 250000 get(125000)	6.7780 ns/op	6.2540 ns/op	1.08
MutableVector 250000 set(125000)	320.58 ns/op	278.05 ns/op	1.15
MutableVector 250000 toArray()	3.8641 ms/op	3.0332 ms/op	1.27
MutableVector 250000 iterate all - toArray() + loop	3.7694 ms/op	2.8428 ms/op	1.33
MutableVector 250000 iterate all - get(i)	1.5885 ms/op	1.5162 ms/op	1.05
Array 250000 create	3.6682 ms/op	2.5604 ms/op	1.43
Array 250000 clone - spread	1.4212 ms/op	1.1122 ms/op	1.28
Array 250000 get(125000)	0.70100 ns/op	0.56100 ns/op	1.25
Array 250000 set(125000)	0.80300 ns/op	0.63000 ns/op	1.27
Array 250000 iterate all - loop	111.17 us/op	110.25 us/op	1.01
effectiveBalanceIncrements clone Uint8Array 300000	52.200 us/op	27.056 us/op	1.93
effectiveBalanceIncrements clone MutableVector 300000	449.00 ns/op	345.00 ns/op	1.30
effectiveBalanceIncrements rw all Uint8Array 300000	174.73 us/op	168.05 us/op	1.04
effectiveBalanceIncrements rw all MutableVector 300000	97.819 ms/op	81.407 ms/op	1.20
phase0 afterProcessEpoch - 250000 vs - 7PWei	126.18 ms/op	114.35 ms/op	1.10
phase0 beforeProcessEpoch - 250000 vs - 7PWei	35.182 ms/op	41.838 ms/op	0.84
altair processEpoch - mainnet_e81889	343.20 ms/op	305.75 ms/op	1.12
mainnet_e81889 - altair beforeProcessEpoch	51.562 ms/op	69.173 ms/op	0.75
mainnet_e81889 - altair processJustificationAndFinalization	19.096 us/op	18.213 us/op	1.05
mainnet_e81889 - altair processInactivityUpdates	6.3225 ms/op	6.3750 ms/op	0.99
mainnet_e81889 - altair processRewardsAndPenalties	64.428 ms/op	52.945 ms/op	1.22
mainnet_e81889 - altair processRegistryUpdates	3.2200 us/op	3.6140 us/op	0.89
mainnet_e81889 - altair processSlashings	529.00 ns/op	498.00 ns/op	1.06
mainnet_e81889 - altair processEth1DataReset	558.00 ns/op	548.00 ns/op	1.02
mainnet_e81889 - altair processEffectiveBalanceUpdates	1.2774 ms/op	1.2494 ms/op	1.02
mainnet_e81889 - altair processSlashingsReset	6.9960 us/op	4.7570 us/op	1.47
mainnet_e81889 - altair processRandaoMixesReset	7.4340 us/op	4.5270 us/op	1.64
mainnet_e81889 - altair processHistoricalRootsUpdate	854.00 ns/op	613.00 ns/op	1.39
mainnet_e81889 - altair processParticipationFlagUpdates	2.4840 us/op	2.4380 us/op	1.02
mainnet_e81889 - altair processSyncCommitteeUpdates	509.00 ns/op	880.00 ns/op	0.58
mainnet_e81889 - altair afterProcessEpoch	130.90 ms/op	126.56 ms/op	1.03
phase0 processEpoch - mainnet_e58758	374.59 ms/op	357.97 ms/op	1.05
mainnet_e58758 - phase0 beforeProcessEpoch	144.23 ms/op	140.42 ms/op	1.03
mainnet_e58758 - phase0 processJustificationAndFinalization	21.105 us/op	17.104 us/op	1.23
mainnet_e58758 - phase0 processRewardsAndPenalties	67.052 ms/op	65.821 ms/op	1.02
mainnet_e58758 - phase0 processRegistryUpdates	12.404 us/op	8.0200 us/op	1.55
mainnet_e58758 - phase0 processSlashings	597.00 ns/op	489.00 ns/op	1.22
mainnet_e58758 - phase0 processEth1DataReset	527.00 ns/op	494.00 ns/op	1.07
mainnet_e58758 - phase0 processEffectiveBalanceUpdates	1.0587 ms/op	1.0617 ms/op	1.00
mainnet_e58758 - phase0 processSlashingsReset	4.5970 us/op	3.3670 us/op	1.37
mainnet_e58758 - phase0 processRandaoMixesReset	7.5810 us/op	4.4460 us/op	1.71
mainnet_e58758 - phase0 processHistoricalRootsUpdate	920.00 ns/op	588.00 ns/op	1.56
mainnet_e58758 - phase0 processParticipationRecordUpdates	5.1650 us/op	3.8130 us/op	1.35
mainnet_e58758 - phase0 afterProcessEpoch	104.39 ms/op	97.858 ms/op	1.07
phase0 processEffectiveBalanceUpdates - 250000 normalcase	1.3096 ms/op	1.2303 ms/op	1.06
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5	1.5752 ms/op	1.5625 ms/op	1.01
altair processInactivityUpdates - 250000 normalcase	27.363 ms/op	28.439 ms/op	0.96
altair processInactivityUpdates - 250000 worstcase	28.601 ms/op	29.266 ms/op	0.98
phase0 processRegistryUpdates - 250000 normalcase	14.337 us/op	6.8990 us/op	2.08
phase0 processRegistryUpdates - 250000 badcase_full_deposits	389.93 us/op	268.72 us/op	1.45
phase0 processRegistryUpdates - 250000 worstcase 0.5	155.22 ms/op	130.13 ms/op	1.19
altair processRewardsAndPenalties - 250000 normalcase	77.077 ms/op	68.495 ms/op	1.13
altair processRewardsAndPenalties - 250000 worstcase	79.975 ms/op	72.642 ms/op	1.10
phase0 getAttestationDeltas - 250000 normalcase	10.094 ms/op	6.8962 ms/op	1.46
phase0 getAttestationDeltas - 250000 worstcase	9.9854 ms/op	6.8561 ms/op	1.46
phase0 processSlashings - 250000 worstcase	4.2208 ms/op	3.5053 ms/op	1.20
altair processSyncCommitteeUpdates - 250000	195.27 ms/op	180.48 ms/op	1.08
BeaconState.hashTreeRoot - No change	371.00 ns/op	261.00 ns/op	1.42
BeaconState.hashTreeRoot - 1 full validator	54.872 us/op	52.156 us/op	1.05
BeaconState.hashTreeRoot - 32 full validator	554.50 us/op	474.31 us/op	1.17
BeaconState.hashTreeRoot - 512 full validator	6.3941 ms/op	5.7647 ms/op	1.11
BeaconState.hashTreeRoot - 1 validator.effectiveBalance	73.229 us/op	61.876 us/op	1.18
BeaconState.hashTreeRoot - 32 validator.effectiveBalance	956.79 us/op	899.15 us/op	1.06
BeaconState.hashTreeRoot - 512 validator.effectiveBalance	14.136 ms/op	11.172 ms/op	1.27
BeaconState.hashTreeRoot - 1 balances	51.301 us/op	50.700 us/op	1.01
BeaconState.hashTreeRoot - 32 balances	489.42 us/op	463.54 us/op	1.06
BeaconState.hashTreeRoot - 512 balances	4.7485 ms/op	4.1975 ms/op	1.13
BeaconState.hashTreeRoot - 250000 balances	75.656 ms/op	70.520 ms/op	1.07
aggregationBits - 2048 els - zipIndexesInBitList	19.117 us/op	16.839 us/op	1.14
regular array get 100000 times	42.843 us/op	32.601 us/op	1.31
wrappedArray get 100000 times	46.795 us/op	32.552 us/op	1.44
arrayWithProxy get 100000 times	17.087 ms/op	16.305 ms/op	1.05
ssz.Root.equals	607.00 ns/op	569.00 ns/op	1.07
byteArrayEquals	603.00 ns/op	546.00 ns/op	1.10
shuffle list - 16384 els	7.3415 ms/op	7.0448 ms/op	1.04
shuffle list - 250000 els	106.78 ms/op	99.512 ms/op	1.07
processSlot - 1 slots	9.8970 us/op	8.8070 us/op	1.12
processSlot - 32 slots	1.5115 ms/op	1.3720 ms/op	1.10
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei	37.303 ms/op	36.527 ms/op	1.02
getCommitteeAssignments - req 1 vs - 250000 vc	3.0515 ms/op	2.8883 ms/op	1.06
getCommitteeAssignments - req 100 vs - 250000 vc	4.3373 ms/op	4.0972 ms/op	1.06
getCommitteeAssignments - req 1000 vs - 250000 vc	4.6941 ms/op	4.3645 ms/op	1.08
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei	5.3200 ns/op	4.7900 ns/op	1.11
state getBlockRootAtSlot - 250000 vs - 7PWei	1.0112 us/op	797.70 ns/op	1.27
computeProposers - vc 250000	11.804 ms/op	10.567 ms/op	1.12
computeEpochShuffling - vc 250000	109.23 ms/op	103.35 ms/op	1.06
getNextSyncCommittee - vc 250000	187.41 ms/op	172.99 ms/op	1.08
computeSigningRoot for AttestationData	14.632 us/op	12.757 us/op	1.15
hash AttestationData serialized data then Buffer.toString(base64)	2.5945 us/op	2.4535 us/op	1.06
toHexString serialized data	1.3159 us/op	1.0696 us/op	1.23
Buffer.toString(base64)	376.76 ns/op	319.15 ns/op	1.18

by benchmarkbot/action

[dapplion]

Does this approach leave a network alive for the entire duration of the tests? Which is going through forks? If yes, this means test outcomes of different e2e tests influence each other since they share global state, is that correct?

[nazarhussain]

Yes the network will be alive through out execution of all e2e tests. It's like connecting to external node, but with this the node is configured and running locally.

Every e2e tests should scaffold their specific network condition on this live network. e.g. If one test need to a contract, can deploy it. Or one network need a specific account, can transfer funds from genesis account. This way execution of tests would not effect each other.

[wemeetagain]

Approving/merging to unblock other PRs but lets make sure the dicussion here doesn't get lost

[wemeetagain]

🎉 This PR is included in v1.9.0 🎉