Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flush buffered state in executor-benchmark #15686

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

msmouse
Copy link
Contributor

@msmouse msmouse commented Jan 7, 2025

Description

expect more reasonable and predicatable numbers.

gonna run calibration and see how it affects numbers

How Has This Been Tested?

Key Areas to Review

Type of Change

  • New feature

Which Components or Systems Does This Change Impact?

  • Other (specify) -- benchmarking

expect more reasonable and predicatable numbers.
Copy link

trunk-io bot commented Jan 7, 2025

⏱️ 2h 24m total CI duration on this PR
Slowest 15 Jobs Cumulative Duration Recent Runs
execution-performance / single-node-performance 1h 40m 🟩🟩
rust-move-tests 13m 🟩
execution-performance / test-target-determinator 10m 🟩🟩
check-dynamic-deps 6m 🟩🟩🟩
rust-doc-tests 5m 🟩
test-target-determinator 4m 🟩
rust-cargo-deny 2m 🟩
fetch-last-released-docker-image-tag 1m 🟩
semgrep/ci 1m 🟩🟩🟩
general-lints 26s 🟩
file_change_determinator 26s 🟩🟩
file_change_determinator 10s 🟩
permission-check 8s 🟩🟩🟩
permission-check 7s 🟩🟩🟩
permission-check 5s 🟩🟩

🚨 1 job on the last run was significantly faster/slower than expected

Job Duration vs 7d avg Delta
check-dynamic-deps 3m 1m +115%

settingsfeedbackdocs ⋅ learn more about trunk.io

@msmouse msmouse added CICD:run-execution-performance-full-test Run execution performance test (full version) CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR labels Jan 7, 2025

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

github-actions bot commented Jan 7, 2025

✅ Forge suite compat success on 6593fb81261f25490ffddc2252a861c994234c2a ==> 75074390c7ce8e46dac0962e01416e8041228165

Compatibility test results for 6593fb81261f25490ffddc2252a861c994234c2a ==> 75074390c7ce8e46dac0962e01416e8041228165 (PR)
1. Check liveness of validators at old version: 6593fb81261f25490ffddc2252a861c994234c2a
compatibility::simple-validator-upgrade::liveness-check : committed: 17749.18 txn/s, latency: 1927.85 ms, (p50: 1500 ms, p70: 1600, p90: 2100 ms, p99: 17400 ms), latency samples: 611500
2. Upgrading first Validator to new version: 75074390c7ce8e46dac0962e01416e8041228165
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 8602.35 txn/s, latency: 3447.96 ms, (p50: 3800 ms, p70: 4100, p90: 4600 ms, p99: 4900 ms), latency samples: 155880
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 8586.33 txn/s, latency: 3890.40 ms, (p50: 4200 ms, p70: 4300, p90: 4500 ms, p99: 4800 ms), latency samples: 285260
3. Upgrading rest of first batch to new version: 75074390c7ce8e46dac0962e01416e8041228165
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 8427.85 txn/s, latency: 3575.34 ms, (p50: 4100 ms, p70: 4200, p90: 4400 ms, p99: 4400 ms), latency samples: 154020
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 8691.80 txn/s, latency: 3831.49 ms, (p50: 4100 ms, p70: 4300, p90: 4400 ms, p99: 4800 ms), latency samples: 290200
4. upgrading second batch to new version: 75074390c7ce8e46dac0962e01416e8041228165
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 14624.93 txn/s, latency: 1968.06 ms, (p50: 2200 ms, p70: 2300, p90: 2300 ms, p99: 2400 ms), latency samples: 249700
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 14741.18 txn/s, latency: 2195.92 ms, (p50: 2300 ms, p70: 2300, p90: 2400 ms, p99: 2500 ms), latency samples: 473040
5. check swarm health
Compatibility test for 6593fb81261f25490ffddc2252a861c994234c2a ==> 75074390c7ce8e46dac0962e01416e8041228165 passed
Test Ok

Copy link
Contributor

github-actions bot commented Jan 7, 2025

✅ Forge suite realistic_env_max_load success on 75074390c7ce8e46dac0962e01416e8041228165

two traffics test: inner traffic : committed: 15244.96 txn/s, latency: 2608.67 ms, (p50: 2700 ms, p70: 2700, p90: 2700 ms, p99: 3300 ms), latency samples: 5796560
two traffics test : committed: 99.97 txn/s, latency: 1335.30 ms, (p50: 1300 ms, p70: 1400, p90: 1500 ms, p99: 2600 ms), latency samples: 1720
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 1.545, avg: 1.484", "ConsensusProposalToOrdered: max: 0.287, avg: 0.281", "ConsensusOrderedToCommit: max: 0.289, avg: 0.284", "ConsensusProposalToCommit: max: 0.570, avg: 0.566"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.54s no progress at version 28270 (avg 0.19s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.54s no progress at version 2836004 (avg 0.54s) [limit 16].
Test Ok

@msmouse
Copy link
Contributor Author

msmouse commented Jan 8, 2025

Okay, to my surprise this didn't change any number much, but still a good peace of mind to have I think..

@igor-aptos @grao1991 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR CICD:run-execution-performance-full-test Run execution performance test (full version)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant