Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change the complexity scaling of reschedule recovered search #677

Merged
merged 3 commits into from
Sep 23, 2022

Conversation

Kubuxu
Copy link
Contributor

@Kubuxu Kubuxu commented Sep 19, 2022

The reschedule recovered while searching, for each expiration set, was
performing a loop Recovering a number of times to look up in a set of
Expiring sectors. This leads to scaling O(recovering*log(expiring)).

Following PR changes it such that the work performed is
O(expiring*log(recovering)). This should improve the gas usage as on
expectation there are 3-5 sectors expiring in any given queue entry
until we find the right queue.
Resolves #545

The reschedule recovered wile searching, for each expiration set, was
performing a loop Recovering number of times to look up in a set of
Expiring sectors. This leads to scaling `O(recovering*log(expiring))`.

Following PR changes it such that the work performed is
`O(expiring*log(recovering))`. This should improve the gas usage as on
expectation there is 3-5 sectors expiring in any given queue.

Signed-off-by: Jakub Sztandera <[email protected]>
@Kubuxu Kubuxu marked this pull request as ready for review September 20, 2022 01:33
@marco-storswift
Copy link

@Kubuxu I used the #677 and#849 the test the benchmark message
lotus state replay bafy2bzacedz5jt3rllzlxgbpzfqujj52g5fu3hb3soosa6wodj6tujzrvyltq
got the resutl is 2022-09-20T10:58:19.829 ERROR fvm::kernel::default > AL: xxxx p4; gas: 1421081321.500,
the origined gasused is 1549095894.
with filecoin-project/ref-fvm#849 the gas change to 1529885689.500
with storswift optimized the commit 793b986 the result is 1447431141.500.

@Kubuxu
Copy link
Contributor Author

Kubuxu commented Sep 20, 2022

I hoped for a larger change, I will investigate what more can be done.

@marco-storswift
Copy link

@Kubuxu this is the total log info
reschedule recovered gasused between the rr-0 and rr-2
2022-09-20T11:27:08.393 ERROR fvm::kernel::default > AL: xxxx rr-0; gas: 493371139.000
2022-09-20T11:27:08.393 ERROR fvm::kernel::default > AL: xxxx rr-0x-0; gas: 494918454.000
2022-09-20T11:27:08.426 ERROR fvm::kernel::default > AL: xxxx rr-2; gas: 945218902.000
all log info
2022-09-20T11:27:08.343 ERROR fvm::kernel::default > AL: xxxx p1-1 f08453079; gas: 1385178.000
2022-09-20T11:27:08.343 ERROR fvm::kernel::default > AL: xxxx p1-2; gas: 1404926.000
2022-09-20T11:27:08.343 ERROR fvm::kernel::default > AL: xxxx p1-3; gas: 1423054.000
2022-09-20T11:27:08.343 ERROR fvm::kernel::default > AL: xxxx p1-4; gas: 1440150.000
2022-09-20T11:27:08.343 ERROR fvm::kernel::default > AL: xxxx p1-5; gas: 1457238.000
2022-09-20T11:27:08.343 ERROR fvm::kernel::default > AL: xxxx p1-6; gas: 1474462.000
2022-09-20T11:27:08.343 ERROR fvm::kernel::default > AL: xxxx p1-7; gas: 1491634.000
2022-09-20T11:27:08.343 ERROR fvm::kernel::default > AL: xxxx p1-8; gas: 1510846.000
2022-09-20T11:27:08.344 ERROR fvm::kernel::default > AL: xxxx p1-9; gas: 1569158.000
2022-09-20T11:27:08.344 ERROR fvm::kernel::default > AL: xxxx p1-10; gas: 1865918.000
2022-09-20T11:27:08.344 ERROR fvm::kernel::default > AL: xxxx p1-11; gas: 3111460.000
2022-09-20T11:27:08.344 ERROR fvm::kernel::default > AL: xxxx p1-12; gas: 3487325.000
2022-09-20T11:27:08.344 ERROR fvm::kernel::default > AL: xxxx rps-0; gas: 3504769.000
2022-09-20T11:27:08.344 ERROR fvm::kernel::default > AL: xxxx rps-1; gas: 3531545.000
2022-09-20T11:27:08.344 ERROR fvm::kernel::default > AL: xxxx rps-2; gas: 3557377.000
2022-09-20T11:27:08.344 ERROR fvm::kernel::default > AL: xxxx rps-3; gas: 5025539.000
2022-09-20T11:27:08.344 ERROR fvm::kernel::default > AL: xxxx rps-3-1; gas: 5157707.000
2022-09-20T11:27:08.344 ERROR fvm::kernel::default > AL: xxxx rps-3-2; gas: 5178591.000
2022-09-20T11:27:08.344 ERROR fvm::kernel::default > AL: xxxx rps-3-3; gas: 5195715.000
2022-09-20T11:27:08.344 ERROR fvm::kernel::default > AL: xxxx prf-0; gas: 5212867.000
2022-09-20T11:27:08.393 ERROR fvm::kernel::default > AL: xxxx prf-1; gas: 489822107.000
2022-09-20T11:27:08.393 ERROR fvm::kernel::default > AL: xxxx prf-2; gas: 490070411.000
2022-09-20T11:27:08.393 ERROR fvm::kernel::default > AL: xxxx rr-0; gas: 493371139.000
2022-09-20T11:27:08.393 ERROR fvm::kernel::default > AL: xxxx rr-0x-0; gas: 494918454.000
2022-09-20T11:27:08.426 ERROR fvm::kernel::default > AL: xxxx rr-2; gas: 945218902.000
2022-09-20T11:27:08.427 ERROR fvm::kernel::default > AL: xxxx prf-3; gas: 967815114.000
2022-09-20T11:27:08.428 ERROR fvm::kernel::default > AL: xxxx prf-4; gas: 1001640784.000
2022-09-20T11:27:08.428 ERROR fvm::kernel::default > AL: xxxx prf-5; gas: 1003274796.000
2022-09-20T11:27:08.428 ERROR fvm::kernel::default > AL: xxxx prf-6; gas: 1003308356.000
2022-09-20T11:27:08.428 ERROR fvm::kernel::default > AL: xxxx rps-3-4; gas: 1003957548.000
2022-09-20T11:27:08.428 ERROR fvm::kernel::default > AL: xxxx rps-3-5; gas: 1003979428.000
2022-09-20T11:27:08.428 ERROR fvm::kernel::default > AL: xxxx rps-3-6; gas: 1004040344.000
2022-09-20T11:27:08.428 ERROR fvm::kernel::default > AL: xxxx rps-3-7; gas: 1004084716.000
2022-09-20T11:27:08.428 ERROR fvm::kernel::default > AL: xxxx rps-3-8; gas: 1004118864.000
2022-09-20T11:27:08.428 ERROR fvm::kernel::default > AL: xxxx rps-3-9; gas: 1004138376.000
2022-09-20T11:27:08.428 ERROR fvm::kernel::default > AL: xxxx rps-4; gas: 1004162452.000
2022-09-20T11:27:08.428 ERROR fvm::kernel::default > AL: xxxx rps-5; gas: 1004179776.000
2022-09-20T11:27:08.429 ERROR fvm::kernel::default > AL: xxxx rps-6; gas: 1006204842.500
2022-09-20T11:27:08.429 ERROR fvm::kernel::default > AL: xxxx rps-7; gas: 1006265770.500
2022-09-20T11:27:08.429 ERROR fvm::kernel::default > AL: xxxx p1-13; gas: 1006315294.500
2022-09-20T11:27:08.429 ERROR fvm::kernel::default > AL: xxxx p1-14; gas: 1006332498.500
2022-09-20T11:27:08.431 ERROR fvm::kernel::default > AL: xxxx p1-15.2-1; gas: 1044817546.500
2022-09-20T11:27:10.625 ERROR fvm::kernel::default > AL: xxxx p1-15.2-2; gas: 1336287215.500
2022-09-20T11:27:10.628 ERROR fvm::kernel::default > AL: xxxx p1-16; gas: 1358432716.000
2022-09-20T11:27:10.628 ERROR fvm::kernel::default > AL: xxxx p1-17; gas: 1362751138.500
2022-09-20T11:27:10.631 ERROR fvm::kernel::default > AL: xxxx p2; gas: 1405061691.000
2022-09-20T11:27:10.632 ERROR fvm::kernel::default > AL: xxxx p3; gas: 1420623883.500
2022-09-20T11:27:10.632 ERROR fvm::kernel::default > AL: xxxx p4; gas: 1421081321.500

@Kubuxu
Copy link
Contributor Author

Kubuxu commented Sep 22, 2022

This patch optimizes gas usage of bafy2bzacebi3rqchvhdcswdihsmjw4tho7frrjavpqk6lp5bx4usgma4tmuey by 62%.

@Kubuxu
Copy link
Contributor Author

Kubuxu commented Sep 22, 2022

Trace:

With rr optim:
AL: SubmitWindowedPoSt CALLED; gas: 644316.000
AL: swp-0; gas: 716188.000
AL: swp-1; gas: 736704.000
AL: swp-1.001; gas: 753488.000
AL: swp-1.01; gas: 1228829.000
AL: swp-1.06; gas: 1514517.000
AL: swp-1.07; gas: 1532045.000
AL: swp-1.08; gas: 1550217.000
AL: swp-1.09; gas: 1567437.000
AL: swp-1.1; gas: 1584725.000
AL: swp-1.2; gas: 1601961.000
AL: swp-1.3; gas: 1619397.000
AL: swp-1.4; gas: 1637277.000
AL: swp-1.5; gas: 1654449.000
AL: swp-1.6; gas: 1671625.000
AL: swp-1.7; gas: 1688765.000
AL: swp-2; gas: 1705981.000
AL: swp-3; gas: 3925523.000
AL: record_proven_sectors CALLED; gas: 3942627.000
AL: post ITERATE; gas: 4930285.000
AL: parition.recover_faults CALLED; gas: 5033745.000
AL: RECOVER faults; gas: 181922594.000
AL: GAS TRACE: rr-0; gas: 181940186.000
AL: GAS TRACE: rr-4; gas: 319861923.000
AL: GAS TRACE: rr-5; gas: 437466087.000
AL: swp-4; gas: 700957079.500
AL: swp-5; gas: 702271771.500
AL: swp-6-verify; gas: 1363356188.500
AL: swp-7; gas: 1395252674.000
AL: swp-8; gas: 1399642776.500
AL: swp-9; gas: 1456092232.000
AL: swp-10; gas: 1456596941.000

Without rr optim:
AL: SubmitWindowedPoSt CALLED; gas: 644316.000
AL: swp-0; gas: 716188.000
AL: swp-1; gas: 736704.000
AL: swp-1.001; gas: 753488.000
AL: swp-1.01; gas: 1228829.000
AL: swp-1.06; gas: 1514517.000
AL: swp-1.07; gas: 1532045.000
AL: swp-1.08; gas: 1550217.000
AL: swp-1.09; gas: 1567437.000
AL: swp-1.1; gas: 1584725.000
AL: swp-1.2; gas: 1601961.000
AL: swp-1.3; gas: 1619397.000
AL: swp-1.4; gas: 1637277.000
AL: swp-1.5; gas: 1654449.000
AL: swp-1.6; gas: 1671625.000
AL: swp-1.7; gas: 1688765.000
AL: swp-2; gas: 1705981.000
AL: swp-3; gas: 3925523.000
AL: record_proven_sectors CALLED; gas: 3942627.000
AL: post ITERATE; gas: 4930285.000
AL: parition.recover_faults CALLED; gas: 5033745.000
AL: RECOVER faults; gas: 181922594.000
AL: GAS TRACE: rr-0; gas: 181940198.000
AL: GAS TRACE: rr-4; gas: 2756118043.000
AL: GAS TRACE: rr-5; gas: 2874211911.000
AL: swp-4; gas: 3141210439.500
AL: swp-5; gas: 3142525131.500
AL: swp-6-verify; gas: 3803597052.500
AL: swp-7; gas: 3835574950.000
AL: swp-8; gas: 3839965052.500
AL: swp-9; gas: 3896303992.000
AL: swp-10; gas: 3896808701.000

@codecov-commenter
Copy link

Codecov Report

Merging #677 (83e2d5f) into master (2501329) will decrease coverage by 0.09%.
The diff coverage is 100.00%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #677      +/-   ##
==========================================
- Coverage   84.89%   84.80%   -0.10%     
==========================================
  Files          89       89              
  Lines       18131    18126       -5     
==========================================
- Hits        15393    15371      -22     
- Misses       2738     2755      +17     
Impacted Files Coverage Δ
actors/miner/src/expiration_queue.rs 88.71% <100.00%> (-0.09%) ⬇️
actors/miner/src/deadline_info.rs 79.77% <0.00%> (-16.86%) ⬇️
actors/init/src/lib.rs 96.77% <0.00%> (-1.62%) ⬇️
actors/verifreg/src/lib.rs 75.60% <0.00%> (-0.18%) ⬇️
test_vm/src/lib.rs 88.57% <0.00%> (-0.12%) ⬇️
actors/multisig/src/lib.rs 95.23% <0.00%> (+0.22%) ⬆️

@Kubuxu
Copy link
Contributor Author

Kubuxu commented Sep 23, 2022

Notes from my exploration: using HashMap decreases the performance, the unchecked math tweak is an insignificant optimization in this case, add_active_sectors optimization gives some minimal results I might pull it in as separate PR.

@Kubuxu Kubuxu merged commit d7eacef into master Sep 23, 2022
@Kubuxu Kubuxu deleted the optim/resechedule-recoveries branch September 23, 2022 14:54
shamb0 pushed a commit to shamb0/builtin-actors that referenced this pull request Jan 31, 2023
…n-project#677)

The reschedule recovered while searching, for each expiration set, was
performing a loop Recovering number of times to look up in a set of
Expiring sectors. This leads to scaling `O(recovering*log(expiring))`.

Following PR changes it such that the work performed is
`O(expiring*log(recovering))`. This should improve the gas usage as on
expectation there are 3-5 sectors expiring in any given queue.

Signed-off-by: Jakub Sztandera <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Optimize reschedule_recovered
4 participants