Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Measure block building times, history processing times, and db sizes #2733

Merged
merged 7 commits into from
Oct 11, 2023

Conversation

spalladino
Copy link
Collaborator

@spalladino spalladino commented Oct 6, 2023

  • Measures total block building time, as well as the time to process public txs and the time to run the rollup circuits.
  • Changes the elapsed methods so they return a duration and not a live timer, otherwise the timer would keep running after the nested function passed onto elapsed has finished.
  • Introduces a new benchmark test for building a chain with multiple blocks, and at predetermined checkpoints measures the total time for a node and a pxe to sync it.
  • Measures db sizes for both aztec node and pxe in the history processing benchmark.

@spalladino spalladino changed the title chore: Measure block building time chore: Measure block building and history processing times Oct 6, 2023
@spalladino spalladino force-pushed the palla/measure-block-build-time branch from ce71ff9 to abd042e Compare October 6, 2023 22:27
@spalladino spalladino marked this pull request as draft October 7, 2023 01:01
@spalladino spalladino changed the title chore: Measure block building and history processing times chore: Measure block building times, history processing times, and db sizes Oct 7, 2023
@spalladino spalladino marked this pull request as ready for review October 7, 2023 17:44
@spalladino spalladino force-pushed the palla/measure-block-build-time branch from a41dc8d to 48b4d4f Compare October 9, 2023 18:59
@spalladino
Copy link
Collaborator Author

Testing comments

@AztecBot
Copy link
Collaborator

Testing

@AztecBot
Copy link
Collaborator

AztecBot commented Oct 10, 2023

(Manually edited comment for testing)

Benchmark results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

Values are compared against data from master at commit 37d9f9cc and shown if the difference exceeds 1%.

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric 8 txs 32 txs 128 txs
l1_rollup_calldata_size_in_bytes 45,444 179,588 716,132
l1_rollup_calldata_gas 222,984 867,956 3,449,696
l1_rollup_execution_gas 842,071 3,595,064 22,205,065
l2_block_processing_time_in_ms 1,060 3,981 15,688
note_successful_decrypting_time_in_ms 332 1,019 3,780
note_trial_decrypting_time_in_ms 34.0 108 138
l2_block_building_time_in_ms 9,114 36,117 152,315
l2_block_rollup_simulation_time_in_ms 6,771 26,781 107,164
l2_block_public_tx_process_time_in_ms 2,300 9,209 44,431

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric 10 blocks 20 blocks 30 blocks
node_history_sync_time_in_ms 30,823 75,516 136,231
note_history_successful_decrypting_time_in_ms 4,653 12,961 20,148
note_history_trial_decrypting_time_in_ms 147 208 254
node_database_size_in_bytes 1,194,179 1,900,681 2,754,125
pxe_database_size_in_bytes 54,187 108,338 162,578

Circuits stats

Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit circuit_simulation_time_in_ms circuit_input_size_in_bytes circuit_output_size_in_bytes
private-kernel-init 54.9 56,577 14,745
private-kernel-ordering 30.2 20,137 8,089
base-rollup ⚠️ 871 (-53%) 631,604 810
root-rollup 37.9 ⚠️ 4,072 (+5556%) 1,097
private-kernel-inner 51.9 72,288 14,745
public-kernel-private-input 51.7 37,359 (-3%) 14,745
public-kernel-non-first-iteration 31.6 37,401 14,745
merge-rollup 1.00 2,592 873

@PhilWindle
Copy link
Collaborator

Looks great.

Some of these benchmarks are rather concerning. Especially as we aren't doing any proving yet!

l1_rollup_execution_gas is mental. Firstly the fact that it's 100K+ gas per tx but that it gets worse as the rollups get bigger.

Really all of the benchmarks at 128 txs. 22 million gas! 2.5 minutes to build the block with no proofs etc.

@PhilWindle PhilWindle merged commit 0cc553a into master Oct 11, 2023
@PhilWindle PhilWindle deleted the palla/measure-block-build-time branch October 11, 2023 09:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants