chore: use logs for benchmarking

[TomAFrench]

Description

Problem*

Paritial work towards #6844

Summary*

This PR shows an example of how we can pull our benchmarking data from the log files which allows us to decouple gathering and processing these values.

Additional Context

Documentation*

Check one:

• No documentation needed.
• Documentation included in this PR.
• [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

• I have tested the changes locally.
• I have formatted the changes with Prettier and/or cargo fmt on default settings.

[github-actions]

Peak Memory Sample

Program	Peak Memory
keccak256	78.48M
workspace	123.81M
regression_4709	422.91M
ram_blowup_regression	1.58G
rollup-base-public	10.47G
rollup-base-private	6.57G
private-kernel-tail	201.81M
private-kernel-reset	717.07M
private-kernel-inner	291.88M
parity-root	172.14M

[github-actions]

Execution Report

Program	Execution Time
sha256_regression	0.100s
regression_4709	0.001s
ram_blowup_regression	0.585s
rollup-root	0.128s
rollup-block-merge	0.126s
rollup-base-public	6.610s
rollup-base-private	5.740s
private-kernel-tail	0.023s
private-kernel-reset	0.387s
private-kernel-inner	0.116s

[github-actions]

Compilation Report

Program	Compilation Time
sha256_regression	1.326s
regression_4709	0.774s
ram_blowup_regression	15.100s
rollup-root	3.970s
rollup-block-merge	3.800s
rollup-base-public	121.000s
rollup-base-private	97.300s
private-kernel-tail	1.030s
private-kernel-reset	7.330s
private-kernel-inner	2.048s
noir-contracts	87.300s

[TomAFrench]

The time difference for compilation times basically boils down to reflecting that we're only measuring actual compilation now. The most significant omission is that we no longer track the time taken by the ACIR transformation passes which can be significant (I think worth tracking separately).

[TomAFrench]

cc @vezenovm

I've got this to a POC but it seems like we've got pretty significant time spent outside of the main compilation flow (see above).

• For now we probably want to measure a span over start_cli rather than purely compilation due to the above discrepancy.
  • It would be good to get more granular data in future (we're spending ~50% of our time in acvm::compiler::compile in some cases!) although this would require more changes to noir-bench-diff.
• Measuring the compilation flow span means this method breaks on workspaces where we're compiling more than one program (noir-contracts is currently measuring the average contract function compilation time rather than the full workspace time.)
  • This is solved by measuring start_cli span for now.

Could you pick up this PR and apply it to the execution benchmarks as well to get it over the line? In the benchmarks case we can measure just the execution time rather than start_cli.

[vezenovm]

This is currently blocked by AztecProtocol/aztec-packages#11011

[github-actions]

Compilation Memory Report

Program	Peak Memory
keccak256	78.48M
workspace	123.82M
regression_4709	422.91M
ram_blowup_regression	1.58G
rollup-base-public	10.47G
rollup-base-private	6.57G
private-kernel-tail	201.79M
private-kernel-reset	717.06M
private-kernel-inner	291.86M

[github-actions]

Execution Memory Report

Program	Peak Memory
keccak256	74.61M
workspace	123.78M
regression_4709	315.92M
ram_blowup_regression	512.47M
rollup-base-public	3.05G
rollup-base-private	2.97G
private-kernel-tail	181.16M
private-kernel-reset	255.25M
private-kernel-inner	214.35M

[TomAFrench]

LGTM