Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

consumer bench same number of txs per iteration #34752

Merged
merged 1 commit into from
Jan 12, 2024

Conversation

apfitzge
Copy link
Contributor

@apfitzge apfitzge commented Jan 11, 2024

Problem

Comment #34717 (comment) for similar benchmark of the block-validation execution path. Bench results of different batch sizes are directly comparable if we always execute the same number of txs per bench iteration.

Summary of Changes

Run same number of txs per bench iteration in consumer benchmarks.

Sample result

before

running 6 tests
test bench_process_and_record_transactions_full_batch                        ... bench:     940,906 ns/iter (+/- 150,107)
test bench_process_and_record_transactions_full_batch_disable_tx_cost_update ... bench:     954,505 ns/iter (+/- 148,716)
test bench_process_and_record_transactions_half_batch                        ... bench:     486,423 ns/iter (+/- 95,842)
test bench_process_and_record_transactions_half_batch_disable_tx_cost_update ... bench:     495,185 ns/iter (+/- 117,344)
test bench_process_and_record_transactions_unbatched                         ... bench:      38,111 ns/iter (+/- 5,206)
test bench_process_and_record_transactions_unbatched_disable_tx_cost_update  ... bench:      39,969 ns/iter (+/- 11,277)

after

running 6 tests
test bench_process_and_record_transactions_full_batch                        ... bench:     906,828 ns/iter (+/- 125,829)
test bench_process_and_record_transactions_full_batch_disable_tx_cost_update ... bench:     938,470 ns/iter (+/- 181,095)
test bench_process_and_record_transactions_half_batch                        ... bench:     961,736 ns/iter (+/- 209,425)
test bench_process_and_record_transactions_half_batch_disable_tx_cost_update ... bench:     968,375 ns/iter (+/- 162,750)
test bench_process_and_record_transactions_unbatched                         ... bench:   2,492,309 ns/iter (+/- 718,294)
test bench_process_and_record_transactions_unbatched_disable_tx_cost_update  ... bench:   2,528,124 ns/iter (+/- 566,048)

Fixes #

Copy link

codecov bot commented Jan 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (5cb30cf) 81.8% compared to head (07568e8) 81.8%.
Report is 8 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff            @@
##           master   #34752     +/-   ##
=========================================
- Coverage    81.8%    81.8%   -0.1%     
=========================================
  Files         823      823             
  Lines      222725   222725             
=========================================
- Hits       182332   182307     -25     
- Misses      40393    40418     +25     

Copy link
Member

@ryoqun ryoqun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@apfitzge apfitzge merged commit 98a2873 into solana-labs:master Jan 12, 2024
35 checks passed
@apfitzge apfitzge deleted the consumer_bench_same_tx branch January 12, 2024 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants