Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

brian/exec window forge #15506

Draft
wants to merge 3 commits into
base: brian/exec-window-exec-optimize-3
Choose a base branch
from

Conversation

bchocho
Copy link
Contributor

@bchocho bchocho commented Dec 5, 2024

  • [execution-pool] Add window and block retrieval to block_tree, block_store and persistent_liveness_storage.
  • [execution-pool] Remove round manager test assertion
  • [execution-pool] Add consensus config v4
  • [execution-pool] Update consensus.yaml for target_block_and_round
  • Execute the block window. Doesn't have any optimizations.
  • [NOT FOR MAIN] turn off consensus observer
  • Filter committed transactions. Up until round-1 in prepare phase, and round-1 in execution phase. This requires using blocking txn provider to quickly provide shuffled txns in the execution phase.
  • incremental update of duplicate counts
  • Remove QS backpressure
  • reinstate max_txns_to_execute
  • filter expired txns
  • execution backpressure with block gas limit
  • fix waiting for window prepare vs execute
  • New forge test

Description

How Has This Been Tested?

Key Areas to Review

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Move Compiler
  • Other (specify)

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

Copy link

trunk-io bot commented Dec 5, 2024

⏱️ 6h 51m total CI duration on this PR
Slowest 15 Jobs Cumulative Duration Recent Runs
test-target-determinator 1h 27m 🟩🟩🟩🟩 (+15 more)
forge-e2e-test / forge 41m 🟥🟥🟥
rust-cargo-deny 26m 🟩🟩🟩🟩🟩 (+10 more)
forge-e2e-test / forge 20m 🟥
check-dynamic-deps 19m 🟩🟩🟩🟩🟩 (+11 more)
forge-e2e-test / forge 15m 🟥
rust-move-tests 13m 🟩
rust-move-tests 13m 🟩
rust-move-tests 13m 🟩
rust-move-tests 13m 🟩
rust-move-tests 12m 🟩
rust-move-tests 12m 🟩
rust-move-tests 12m 🟩
rust-move-tests 12m 🟩
rust-move-tests 12m 🟩

settingsfeedbackdocs ⋅ learn more about trunk.io

@bchocho bchocho added the CICD:run-forge-e2e-perf Run the e2e perf forge only label Dec 5, 2024

This comment has been minimized.

This comment has been minimized.

@bchocho bchocho force-pushed the brian/exec-window-forge branch from 7084600 to e05d73a Compare December 5, 2024 22:29

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@bchocho bchocho changed the base branch from main to brian/exec-window-exec-optimize-3 December 9, 2024 18:02

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

@bchocho bchocho force-pushed the brian/exec-window-forge branch from 14f4587 to f4ad114 Compare December 10, 2024 19:22

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

adjust configs

increase min gas limit

adjust more, logs

prototype integration in gas meter

ajdust min

adjust min 2000

adjust to be way more conservative

spread more

use average instead of median

mempool backlog: 50 -> 20

Add more txn types

adjust

adjust more

no agg

adjust to smaller

more sequential workloads

adjust
@bchocho bchocho force-pushed the brian/exec-window-forge branch from 8f128e9 to 7888c3c Compare January 7, 2025 00:24

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

github-actions bot commented Jan 8, 2025

✅ Forge suite realistic_env_max_load success on 739ecfe99efada57afb656425ee97a7224083c2e

Workload                                      | submitted/s  | committed/s  | expired/s    | rejected/s   | chain txn/s  | latency      | p50 lat      | p90 lat      | p99 lat      | mempool->block | prop->order  | order->commit | actual dur   | idx_fn       | idx_cache    | idx_data    
0: ResourceGroupsGlobalWriteTag1KB[B:5.0k]    | 1930.72      | 1930.72      | 0.00         | 0.00         | 2989         | 2.043        | 2.000        | 2.300        | 2.700        | 0.648          | 0.310        | 0.240         | 51           | 0            | 0.000        | 0.000       
1: TokenV1FTMintAndTransfer[B:4.0k]           | 1236.35      | 1236.35      | 0.00         | 0.00         | 2287         | 2.449        | 2.400        | 2.700        | 2.900        | 0.697          | 0.325        | 0.633         | 51           | 0            | 0.000        | 0.000       
2: LiquidityPoolSwap[B:2.0k]                  | 849.68       | 849.68       | 0.00         | 0.00         | 1866         | 1.908        | 1.900        | 2.200        | 2.500        | 0.445          | 0.279        | 0.402         | 51           | 0            | 0.000        | 0.000       
3: NoOpFeePayer[B:5.0k]                       | 2180.79      | 2180.79      | 0.00         | 0.00         | 3212         | 2.085        | 2.000        | 2.200        | 2.400        | 0.359          | 0.279        | 0.870         | 51           | 0            | 0.000        | 0.000       
4: ModifyGlobalResource[B:4.0k]               | 2145.75      | 2145.75      | 0.00         | 0.00         | 3169         | 1.686        | 1.700        | 1.800        | 1.900        | 0.381          | 0.279        | 0.409         | 51           | 0            | 0.000        | 0.000       
5: SmartTablePicture1MWith256Change[B:0.0k]   | 7.00         | 7.00         | 0.00         | 0.00         | 1029         | 3.293        | 3.100        | 3.500        | 4.000        | 0.485          | 0.411        | 1.703         | 51           | 0            | 0.000        | 0.000       
background traffic             | submitted/s  | committed/s  | expired/s    | rejected/s   | latency      | p50 lat      | p90 lat      | p99 lat     
background with traffic 0      | 996.28       | 996.28       | 0.00         | 0.00         | 1.720        | 1.800        | 1.800        | 2.100       
background with traffic 1      | 1009.84      | 1009.84      | 0.00         | 0.00         | 1.691        | 1.500        | 2.100        | 2.400       
background with traffic 2      | 991.55       | 991.55       | 0.00         | 0.00         | 1.535        | 1.500        | 1.800        | 1.800       
background with traffic 3      | 995.00       | 995.00       | 0.00         | 0.00         | 2.005        | 2.100        | 2.100        | 2.400       
background with traffic 4      | 996.57       | 996.57       | 0.00         | 0.00         | 1.593        | 1.500        | 1.800        | 1.800       
background with traffic 5      | 999.50       | 999.50       | 0.00         | 0.00         | 2.944        | 3.000        | 3.900        | 6.300       
Test Ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CICD:run-forge-e2e-perf Run the e2e perf forge only
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant