feat: protogalaxy perturbator!

[maramihali]

This PR implements the power perturbator polynomial (F(X) in the paper), following the tree technique in the paper, allowing computation of the polynomial in coefficient form in O(N) time where N is the size of the instance. Resolves #693 and #687

The relation-related functionality from sumcheck that is now used in both sumcheck and PG has been moved in a shared RelationUtils file

Checklist:

Remove the checklist to signal you've completed it. Enable auto-merge if the PR is ready to merge.

• If the pull request requires a cryptography review (e.g. cryptographic algorithm implementations) I have added the 'crypto' tag.
• I have reviewed my diff in github, line by line and removed unexpected formatting changes, testing logs, or commented-out code.
• Every change is related to the PR description.
• I have linked this pull request to relevant issues (if any exist).

[maramihali]

I started with having two different structures for ClaimedEvaluations and RowEvaluations but they have exactly the same form so it seemed wasteful. Wdyt of the name?

[codygunton]

In #2619 I renamed this to just AllValues since it's now being used more generally, in the check circuit functions.

[codygunton]

We can give a more informative alias within the sumcheck class.

[ludamad]

ProverPolynomialEvaluations might feel a bit more natural, but I'm not sure I have full context

[maramihali]

These are taken from sumcheck functionality atm, I added an issue to make a testing utility file with functionality shared amongst tests in the codebase. In my mind, I'll do that when writing the final comprehensive tests. I noticed you started creating it in the combiner work.

[codygunton]

Big steps forward here!

[codygunton]

In #2619 I renamed this to just AllValues since it's now being used more generally, in the check circuit functions.

[codygunton]

We can give a more informative alias within the sumcheck class.

[codygunton]

I checked this example with this script

import numpy as np
from random import randint

maras_coeffs = [648, 936, 432, 64]

b = np.array([11, 8, 5])
d = np.array([8, 4, 2])
iis = [bin(i)[2:].zfill(3) for i in range(8)]
iis = [np.array([bool(int(n)) for n in s]) for s in iis]
bs = [b[idx] for idx in iis]
ds = [d[idx] for idx in iis]

def run_test():
    x = randint(0, 1<<256)
    prods = [np.prod((beta + x * delta)) for beta, delta in zip(bs, ds)]
    my_value = sum(prods)
    
    maras_value = sum(x**i * maras_coeffs[i] for i in range(len(maras_coeffs)))
    assert(maras_value == my_value)

run_test()
print("Test succeeded!")

[maramihali]

sometimes I enjoy doing arithmetic by hand, but I can add a script that double checks if you think it's necessary

[codygunton]

No I was just documenting how I checked your work in case it's useful to have the code at some point in the future.

[codygunton]

Can you add a description?

[maramihali]

it was in the source file, moved to header file

[codygunton]

Do you wanted to keep this commented-out stuff?

[maramihali]

nope, that's work that should go in the prover, removed

[AztecBot]

Benchmark results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Values are compared against data from master at commit d7b7fb1d and shown if the difference exceeds 1%.

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric	8 txs	32 txs	128 txs
l1_rollup_calldata_size_in_bytes	45,444	179,588	716,132
l1_rollup_calldata_gas	222,792	868,232	3,449,432
l1_rollup_execution_gas	841,879	3,595,340	22,204,801
l2_block_processing_time_in_ms	1,027 (+1%)	3,880	15,181 (-1%)
note_successful_decrypting_time_in_ms	334 (+2%)	1,011 (-1%)	3,868 (-3%)
note_trial_decrypting_time_in_ms	⚠️ 37.0 (+12%)	107 (-1%)	137 (-1%)
l2_block_building_time_in_ms	8,926	35,848	157,593 (+1%)
l2_block_rollup_simulation_time_in_ms	6,604 (+1%)	26,470	105,184
l2_block_public_tx_process_time_in_ms	2,283 (-1%)	9,255 (+1%)	51,787 (+1%)

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric	5 blocks	10 blocks
node_history_sync_time_in_ms	13,900	26,404 (-1%)
note_history_successful_decrypting_time_in_ms	3,026 (-1%)	5,916
note_history_trial_decrypting_time_in_ms	128 (+1%)	⚠️ 227 (+45%)
node_database_size_in_bytes	1,662,476	1,201,580
pxe_database_size_in_bytes	27,188	54,187

Circuits stats

Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit	circuit_simulation_time_in_ms	circuit_input_size_in_bytes	circuit_output_size_in_bytes
private-kernel-init	52.4	61,697	18,841
private-kernel-ordering	26.0 (-1%)	24,233	8,089
base-rollup	941	639,797	811
root-rollup	42.1	4,072	1,097
private-kernel-inner	47.0	81,504	18,841
public-kernel-private-input	52.5	41,455	18,841
public-kernel-non-first-iteration	34.4	41,497	18,841
merge-rollup	0.920 (-9%)	2,592	873

Miscellaneous

Transaction sizes based on how many contracts are deployed in the tx.

Metric	0 deployed contracts	1 deployed contracts
tx_size_in_bytes	8,723	27,508