Tests: Batching Benchmarks

[axelKingsley]

This Benchmark measures the time it takes to add a singular batch to either a Singular Batch, or a Span Batch. In its present configuration it measures only the FINAL batch added, in order to show that as a batch grows, its performance changes over time.

It has a collection of test targets for number-of-batches and tx-per-batch, which it then multiplies by the two Batch Types, and then multiplies again by each Compressor type.

We are investigating the root cause of Quadratic Batcher Performance when using span batches, so this helps to demonstrate the loss in performance, and will help us to confirm the improvement as we fix it.

Sample Output

Span Batches are low performance to the extent that I am still waiting for results from this latest version of the test, but here is an example of Singular vs Span with the NonCompressor:

BenchmarkChannelOut/BatchType=Singular,_txPerBatch=1,_BatchCount=10,_Compressor=NonCompressor-12            	       1	      2791 ns/op	     408 B/op	       4 allocs/op
BenchmarkChannelOut/BatchType=Singular,_txPerBatch=1,_BatchCount=100,_Compressor=NonCompressor-12           	       1	      2875 ns/op	     984 B/op	       4 allocs/op
BenchmarkChannelOut/BatchType=Singular,_txPerBatch=1,_BatchCount=1000,_Compressor=NonCompressor-12          	       1	      2375 ns/op	     440 B/op	       4 allocs/op
BenchmarkChannelOut/BatchType=Singular,_txPerBatch=1,_BatchCount=10000,_Compressor=NonCompressor-12         	       1	      9167 ns/op	     328 B/op	       4 allocs/op
-vs-
BenchmarkChannelOut/BatchType=Span,_txPerBatch=1,_BatchCount=10,_Compressor=NonCompressor-12                	       1	     30667 ns/op	   35912 B/op	     392 allocs/op
BenchmarkChannelOut/BatchType=Span,_txPerBatch=1,_BatchCount=100,_Compressor=NonCompressor-12               	       1	    175542 ns/op	  456648 B/op	    3277 allocs/op
BenchmarkChannelOut/BatchType=Span,_txPerBatch=1,_BatchCount=1000,_Compressor=NonCompressor-12              	       1	   1614501 ns/op	 3107656 B/op	   31859 allocs/op
BenchmarkChannelOut/BatchType=Span,_txPerBatch=1,_BatchCount=10000,_Compressor=NonCompressor-12             	       1	  18388584 ns/op	51128768 B/op	  316991 allocs/op

And the same for Ratio Compressor:

BenchmarkChannelOut/BatchType=Singular,_txPerBatch=1,_BatchCount=10,_Compressor=RatioCompressor-12          	       1	      8167 ns/op	     408 B/op	       4 allocs/op
BenchmarkChannelOut/BatchType=Singular,_txPerBatch=1,_BatchCount=100,_Compressor=RatioCompressor-12         	       1	     27251 ns/op	     984 B/op	       4 allocs/op
BenchmarkChannelOut/BatchType=Singular,_txPerBatch=1,_BatchCount=1000,_Compressor=RatioCompressor-12        	       1	     26459 ns/op	     440 B/op	       4 allocs/op
BenchmarkChannelOut/BatchType=Singular,_txPerBatch=1,_BatchCount=10000,_Compressor=RatioCompressor-12       	       1	     22124 ns/op	     328 B/op	       4 allocs/op
-vs-
BenchmarkChannelOut/BatchType=Span,_txPerBatch=1,_BatchCount=10,_Compressor=RatioCompressor-12              	       1	    136708 ns/op	   35912 B/op	     392 allocs/op
BenchmarkChannelOut/BatchType=Span,_txPerBatch=1,_BatchCount=100,_Compressor=RatioCompressor-12             	       1	   1700959 ns/op	  325576 B/op	    3276 allocs/op
BenchmarkChannelOut/BatchType=Span,_txPerBatch=1,_BatchCount=1000,_Compressor=RatioCompressor-12            	       1	   9804126 ns/op	 3107656 B/op	   31859 allocs/op
BenchmarkChannelOut/BatchType=Span,_txPerBatch=1,_BatchCount=10000,_Compressor=RatioCompressor-12           	       1	  96453916 ns/op	32441608 B/op	  316943 allocs/op

[coderabbitai]

Walkthrough

Walkthrough

The recent update streamlines the batch handling process in Ethereum Optimism by enhancing testing capabilities through benchmarking for batch building operations. This update aims to optimize transaction processing efficiency and system reliability by evaluating performance under various compressors.

Changes

File	Change Summary
op-node/benchmarks/batchbuilding_test.go	Introduced benchmarking functionality for batch building, covering diverse batch types and compressors.
op-node/rollup/derive/batch_test.go	Refactored or replaced logic for generating singular batches, removing the RandomSingularBatch function.
op-node/rollup/derive/batch_test_utils.go	Added RandomSingularBatch function to generate random singular batches for testing in an Optimism rollup environment.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

• X
• Mastodon
• Reddit
• LinkedIn

---

Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>.
  • Generate unit-tests for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit tests for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai generate interesting stats about this repository and render them as a table.
  • @coderabbitai show all the console.log statements in this repository.
  • @coderabbitai read src/utils.ts and generate unit tests.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• The JSON schema for the configuration file is available here.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

CodeRabbit Discord Community

Join our Discord Community to get help, request features, and share feedback.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 0.60%. Comparing base (9055278) to head (43d5248).
Report is 33 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop   #9927       +/-   ##
===========================================
- Coverage    28.33%   0.60%   -27.74%     
===========================================
  Files          165      92       -73     
  Lines         7270    2477     -4793     
  Branches      1330     575      -755     
===========================================
- Hits          2060      15     -2045     
+ Misses        5104    2462     -2642     
+ Partials       106       0      -106     

Flag	Coverage Δ
cannon-go-tests	?
chain-mon-tests	?
common-ts-tests	?
contracts-bedrock-tests	0.60% <ø> (ø)
contracts-ts-tests	?
core-utils-tests	?
sdk-next-tests	?
sdk-tests	?

Flags with carried forward coverage won't be shown. Click here to find out more.

see 73 files with indirect coverage changes

[trianglesphere]

Bench looks good to me & runs locally. Just some nits on dad code.