Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pevm rebase v054 #10

Open
wants to merge 105 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
efc152e
Feat: Parallel Transaction Execution Implementation
sunny2022da Jul 29, 2024
5e7f2e7
Fix: fix incorrectly set origin storage.
DavidZangNR Jul 3, 2024
3e2f523
Feat: TxDAG: support TxDAG
galaio Jul 5, 2024
d3c6705
fix several UT with racing issues (#5)
DavidZangNR Jul 16, 2024
5c81e49
TxDAG: support PEVM static dispatch; (#6)
galaio Jul 18, 2024
0d667e6
fix UT test and contention issue (#7)
DavidZangNR Jul 18, 2024
9fc4328
Fix: dead lock issue
sunny2022da Jul 18, 2024
8ea4d1b
Fix: avoid update the stateObjects at conflict check phase
sunny2022da Jul 18, 2024
74f8119
TxDAG: support TxDAG transfer, it can be used in QA performance testi…
galaio Jul 24, 2024
756e93e
txdag: support write & read TxDAG from file; (#9)
galaio Jul 24, 2024
74cc070
FIX: redundancy execution and incorrect merge of dirty object (#12)
DavidZangNR Jul 29, 2024
7c5bf71
pevm: support delay gas fee calculation & Uts; (#11)
galaio Jul 29, 2024
a4cf50e
FIX: issue in fixUpOriginAndResetPendingStorage (#14)
DavidZangNR Jul 29, 2024
c34a671
Fix: racying issue in fixUpOriginAndResetPendingStorage
sunny2022da Jul 30, 2024
60fbe9b
txdag: remove legacy TxDAG transfer logic; (#16)
galaio Jul 30, 2024
ad37a95
txdag: opt txdag logic & clean todos; (#17)
galaio Jul 31, 2024
1ec90fb
refine lock and fix racying issue (#18)
DavidZangNR Aug 2, 2024
4217365
txdag: opt TxDAG rwset collecting & generating; (#19)
galaio Aug 2, 2024
ba16a56
contention issue fix (#21)
DavidZangNR Aug 5, 2024
60b93b3
txdag: support multi flags, and supported in pevm; (#22)
galaio Aug 6, 2024
9c1c7dd
pevm: opt slot trigger mechanism; (#24)
galaio Aug 6, 2024
f98be3a
fix addBalance for delayGasFee (#25)
DavidZangNR Aug 7, 2024
a81f34d
txdag: opt read txdag file and validation logic; (#26)
galaio Aug 8, 2024
f0744f3
pevm: opt read large txdag logic and add conflict metrics; (#29)
galaio Aug 13, 2024
0de9577
feat: avoid parallel process for block with few txs
sunny2022da Aug 7, 2024
0fac12d
feat: avoid parallel process for block with few txs
sunny2022da Aug 7, 2024
dea715a
fix: avoid rewrite readsCache in slotDB
sunny2022da Aug 8, 2024
37a617a
Fix: incorrect GetState of obsoleted data caused by createObject
sunny2022da Aug 8, 2024
0d683a7
fix: DAG disable access unconfirmedDB
sunny2022da Aug 11, 2024
9babd88
do not check special addr 0x1 for destruct
sunny2022da Aug 11, 2024
73172b3
fix log issue
sunny2022da Aug 12, 2024
3b6b406
fix: false report of conflict
sunny2022da Aug 13, 2024
1654d3f
txDAG transfer (#28)
andyzhang2023 Aug 13, 2024
0e8f822
txdag: using pending writes to accelerate txdag generation, add more …
galaio Aug 13, 2024
29ed6b4
feat: code cleanup (#23)
DavidZangNR Aug 13, 2024
2a1169b
recover test case
sunny2022da Aug 14, 2024
bc852c7
fix ut of txDAG (#32)
andyzhang2023 Aug 14, 2024
7af9f3f
txdag: support reset txdag reader when SetHead; (#31)
galaio Aug 14, 2024
48ac372
txdag: fix system tx finalise issue; (#33)
galaio Aug 14, 2024
36fc3cb
fix: refine the log level of PEVM (#34)
DavidZangNR Aug 14, 2024
2d45d90
mvstates: fix async dep gen deadlock issue & opt mining txdag generat…
galaio Aug 15, 2024
3db0ca9
fix parallel Num (#36)
DavidZangNR Aug 15, 2024
8b839d2
fix: remove unnecessary locks for stateobjects (#37)
DavidZangNR Aug 15, 2024
4a6ec28
mvstates: opt async dep generation; (#38)
galaio Aug 16, 2024
058ff1b
mvstates: fix oom issue when mining is enabled; (#40)
galaio Aug 16, 2024
446db58
reduce overhead of slotDB initialize (#39)
DavidZangNR Aug 19, 2024
3c0058c
mvstates: fix oom issue when mining is enabled;
galaio Aug 16, 2024
ad5f2ef
Fix: contention issue of Trie for PEVM (#41)
DavidZangNR Aug 20, 2024
82d12d1
worker: fix TxDAG generation issues when mining block; (#43)
galaio Aug 20, 2024
5ec1321
pevm-opt: Enable parallel kv conflict check
sunny2022da Aug 21, 2024
d9bbda1
pevm-opt: Add conflict check cache
sunny2022da Aug 21, 2024
54dd397
PEVM-fix: avoid checkout old tx in stage2 conflict check
sunny2022da Aug 22, 2024
2e40b22
pevm: add a side slot to trigger next tx advance;
galaio Aug 23, 2024
c85e980
pevm: fix some bad check & support to fallback to serial processor;
galaio Aug 27, 2024
d1acaf7
pevm: fix some bad check & support to fallback to serial processor;
galaio Aug 27, 2024
3a92a2a
async the merge phase
sunny2022da Aug 28, 2024
4c410a0
disable parallel if parallel.num is low
sunny2022da Aug 30, 2024
f4ab638
do CompareAndSwap only necessary
sunny2022da Aug 30, 2024
7387419
remove uncessary memory overhead and reuse SyncPool
sunny2022da Sep 2, 2024
8e01b25
fix putSyncPool and GC issue
sunny2022da Sep 3, 2024
37a3222
parallelDBManager global
sunny2022da Sep 4, 2024
497d385
pevm-opt: lock free localstateObjects (#167)
sunny2022da Sep 9, 2024
8602936
PEVM-fix: assesslist append and optimize mergeSlotDB (#168)
sunny2022da Sep 10, 2024
0a7a7b8
PEVM-opt: parallel Txs Prepare (#176)
sunny2022da Sep 12, 2024
bfa857b
fix issue after rebase
sunny2022da Sep 25, 2024
a22f951
fix code hash issue after rebase
sunny2022da Sep 26, 2024
2ffe544
fix issue in createObject after rebase
sunny2022da Sep 26, 2024
67fd679
fix: test case issue after rebase (#190)
sunny2022da Sep 29, 2024
cbb0317
fix getcode issue after rebase
sunny2022da Sep 29, 2024
dd5d24d
fix trie prefetch issue after rebase
sunny2022da Oct 8, 2024
eea8553
PEVM-fix: check in ValidatePlainTxDAG (#195)
sunny2022da Oct 11, 2024
4cee5f2
Pevm rebased to v0.5.5 (#192)
andyzhang2023 Oct 11, 2024
7601e06
Fix/pevm v0.5.0 invalid gas used (#199)
andyzhang2023 Oct 18, 2024
0a5e375
rename pevm option (#200)
andyzhang2023 Oct 18, 2024
6ffd3e1
pevm: unordered merge mode flag (#202)
welkin22 Oct 22, 2024
3c2bedf
pevm: remove the ParallelLegacy (#201)
sunny2022da Oct 22, 2024
d00d951
If TxDAG is nil, then use the serial processor to handle it.
welkin22 Oct 22, 2024
9729844
fix: use ParallelTxNum config and uber automaxprocs (#204)
welkin22 Oct 23, 2024
2ac990c
fix: Transient Storage should set its prev value but not delete it wh…
Oct 23, 2024
7cf3d35
bugfix: it panic when accessing storage from a reverted object which …
Oct 25, 2024
3586be6
bugfix: 'invalid mercle root' after reverting an object which had bee…
Oct 25, 2024
5ac0770
UncommittedDB of PEVM shutdown when an invalid snapshot id found (jus…
Oct 25, 2024
4df7b8d
fix: do initParallelRunner() only once no matter how many chains are …
Oct 25, 2024
bc5d72d
fixut: init the runner before running the cases of PevmProcessor, oth…
Oct 25, 2024
0dea5ce
fixut: err should be returned by a seprated method ConflictsToMaindb(…
Oct 25, 2024
b3c941c
increase the conflict counter only if the state of uncommitted db con…
andyzhang2023 Oct 25, 2024
7e7e807
fix ut case of TestBlockchainWithTxDAGDebug (#213)
andyzhang2023 Nov 1, 2024
db297bb
pevm: refactor txdag reader (#212)
welkin22 Nov 1, 2024
c43afd3
use currentBlock for init
welkin22 Oct 31, 2024
a759fdf
doc: add release note for TxDAG-PEVM alpha release (#215)
sunny2022da Nov 4, 2024
ca71a5c
PEVM-fix: clear useless legacy pevm code (#216)
sunny2022da Nov 6, 2024
8fa2772
fix unit test
welkin22 Nov 6, 2024
b230aab
use patch to fix ut instead of changing code
welkin22 Nov 6, 2024
5768dd5
PEVM-fix: Disable prefetch when PEVM enabled (#218)
sunny2022da Nov 12, 2024
617cc39
feat: disable parallel when txs count is low (#226)
sunny2022da Nov 26, 2024
38f0846
feat: PEVM fallback to serial process (#225)
sunny2022da Nov 26, 2024
9e730b3
fix: Add test for PEVM fall back (#231)
sunny2022da Dec 3, 2024
16bbb44
Revert "use patch to fix ut instead of changing code" (#235)
sunny2022da Dec 12, 2024
62093b3
pevm: support parallel merge mode
welkin22 Nov 12, 2024
e9f99d6
pevm-fix: fix journal record in add/sub balance with 0 (#249)
sunny2022da Jan 9, 2025
f8d6a95
PEVM-fix: try load TxDAG from block if fail from file (#253)
sunny2022da Jan 13, 2025
3fd6e45
pevm: fallback to sequencial processor when the TxDAG is too deep (#251)
andyzhang2023 Jan 13, 2025
ffe9dc0
Merge branch 'main' into PEVM-rebase-v054
sunny2022da Jan 14, 2025
921afc5
add metrics for parallel processor (#254)
andyzhang2023 Jan 15, 2025
48aaff6
Fix pevm 20per performance issue (#256)
andyzhang2023 Jan 16, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: PEVM fallback to serial process (bnb-chain#225)
sunny2022da committed Dec 11, 2024

Verified

This commit was signed with the committer’s verified signature.
inteon Tim Ramlot
commit 38f08460ab2ea744e1368bbe7e9dde7a4b9890c7
75 changes: 72 additions & 3 deletions core/blockchain.go
Original file line number Diff line number Diff line change
@@ -1896,6 +1896,11 @@ func (bc *BlockChain) insertChain(chain types.Blocks, setHead bool) (int, error)
defer func() {
DebugInnerExecutionDuration = 0
}()

if bc.serialProcessor == nil {
bc.serialProcessor = bc.processor
}

for ; block != nil && err == nil || errors.Is(err, ErrKnownBlock); block, err = it.next() {
DebugInnerExecutionDuration = 0
// If the chain is terminating, stop processing blocks
@@ -1961,6 +1966,7 @@ func (bc *BlockChain) insertChain(chain types.Blocks, setHead bool) (int, error)
err error
)

blockProcessedInParallel := false
// skip block process if we already have the state, receipts and logs from mining work
if !(receiptExist && logExist && stateExist) {
// Retrieve the parent block and it's state to execute on top
@@ -2010,8 +2016,26 @@ func (bc *BlockChain) insertChain(chain types.Blocks, setHead bool) (int, error)

if useSerialProcessor {
receipts, logs, usedGas, err = bc.serialProcessor.Process(block, statedb, bc.vmConfig)
blockProcessedInParallel = false
} else {
receipts, logs, usedGas, err = bc.processor.Process(block, statedb, bc.vmConfig)
blockProcessedInParallel = true
if err != nil {
// parallel processing fail , fallback to serial with new statDB.
log.Warn("ParallelEVM fallback to serial process", "error", err.Error())
execErr := err
statedb, err = bc.reGenerateStateForFallBack(parent.Root, block.Root(), statedb)
if err != nil {
// Can not get new statedb for serial run, report the process error.
bc.reportBlock(block, receipts, execErr)
followupInterrupt.Store(true)
return it.index, err
}
statedb.StartPrefetcher("chain")
activeState = statedb
receipts, logs, usedGas, err = bc.serialProcessor.Process(block, statedb, bc.vmConfig)
blockProcessedInParallel = false
}
}
if err != nil {
bc.reportBlock(block, receipts, err)
@@ -2023,9 +2047,43 @@ func (bc *BlockChain) insertChain(chain types.Blocks, setHead bool) (int, error)

vstart := time.Now()
if err := bc.validator.ValidateState(block, statedb, receipts, usedGas); err != nil {
bc.reportBlock(block, receipts, err)
followupInterrupt.Store(true)
return it.index, err
if blockProcessedInParallel {
// invalid parallel execution, try serial
log.Warn("ParallelEVM fallback to serial process after ValidateState", "error", err.Error())
parent := it.previous()
if parent == nil {
parent = bc.GetHeader(block.ParentHash(), block.NumberU64()-1)
}

validateErr := err
statedb, err = bc.reGenerateStateForFallBack(parent.Root, block.Root(), statedb)
if err != nil {
// can not get new statedb for serial run, report the validate error.
bc.reportBlock(block, receipts, validateErr)
followupInterrupt.Store(true)
return it.index, err
}
statedb.StartPrefetcher("chain")
activeState = statedb
receipts, logs, usedGas, err = bc.serialProcessor.Process(block, statedb, bc.vmConfig)
if err != nil {
// serial process with process error.
bc.reportBlock(block, receipts, err)
followupInterrupt.Store(true)
return it.index, err
}
blockProcessedInParallel = false
if err := bc.validator.ValidateState(block, statedb, receipts, usedGas); err != nil {
// serial process with validation error.
bc.reportBlock(block, receipts, err)
followupInterrupt.Store(true)
return it.index, err
}
} else {
bc.reportBlock(block, receipts, err)
followupInterrupt.Store(true)
return it.index, err
}
}

vtime := time.Since(vstart)
@@ -2921,6 +2979,17 @@ func (bc *BlockChain) SetupTxDAGGeneration(output string, readFile bool) {

}

func (bc *BlockChain) reGenerateStateForFallBack(parentRoot common.Hash, blockRoot common.Hash, oldDB *state.StateDB) (*state.StateDB, error) {
oldDB.StopPrefetcher()
statedb, err := state.New(parentRoot, bc.stateCache, bc.snaps)
if err != nil {
return nil, err
}

statedb.SetExpectedStateRoot(blockRoot)
return statedb, nil
}

type TxDAGOutputItem struct {
blockNumber uint64
txDAG types.TxDAG