feat: update honk recursion constraint #6545

lucasxia01 · 2024-05-20T21:09:35Z

Updates recursion constraint for honk so that it calls the ultra honk recursive verifier.
Removes key hash from the constraint. Also recursively aggregates the aggregation objects.

Ignores the case where we don't have the witness values and want to use dummy witnesses.

Please read contributing guidelines and remove this line.

AztecBot · 2024-05-20T23:01:29Z

Benchmark results

No metrics with a significant change found.

Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Proof generation

Each column represents the number of threads used in proof generation.

Metric	1 threads	4 threads	16 threads	32 threads	64 threads
proof_construction_time_sha256	5,780 (-1%)	1,551 (-1%)	712 (-1%)	784	781 (+1%)

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric	8 txs	32 txs	64 txs
l1_rollup_calldata_size_in_bytes	1,412	1,412	1,412
l1_rollup_calldata_gas	9,464	9,464	9,464
l1_rollup_execution_gas	616,105	616,105	616,105
l2_block_processing_time_in_ms	1,278 (-1%)	4,787 (-1%)	9,489 (-2%)
l2_block_building_time_in_ms	44,826	177,900	355,421
l2_block_rollup_simulation_time_in_ms	44,657	177,272	354,082
l2_block_public_tx_process_time_in_ms	24,053	100,711	205,680

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric	3 blocks	5 blocks
node_history_sync_time_in_ms	9,488	14,444
node_database_size_in_bytes	14,499,920	21,348,432
pxe_database_size_in_bytes	18,071	29,868

Circuits stats

Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit	simulation_time_in_ms	witness_generation_time_in_ms	proving_time_in_ms	input_size_in_bytes	output_size_in_bytes	proof_size_in_bytes	num_public_inputs	size_in_gates
private-kernel-init	161 (+1%)	3,719 (-1%)	22,149 (+1%)	20,630	64,614	89,536	2,731	1,048,576
private-kernel-inner	621	4,284 (-1%)	46,309	92,318	64,614	89,536	2,731	2,097,152
private-kernel-tail	567	2,922 (-1%)	35,917	96,541	77,498	10,656	266	2,097,152
base-parity	6.56	1,233 (+8%)	2,915	128	64.0	2,208	2.00	131,072
root-parity	49.8 (-1%)	48.7	44,848 (+2%)	27,084	64.0	2,720	18.0	2,097,152
base-rollup	725 (-3%)	2,291 (+1%)	74,826 (+1%)	119,610	756	3,648	47.0	4,194,304
root-rollup	112	65.9 (+1%)	19,922 (+1%)	25,297	620	3,456	41.0	1,048,576
public-kernel-app-logic	521	2,986	47,067 (-2%)	104,941	86,302	114,784	3,520	2,097,152
public-kernel-tail	1,080	23,171 (+1%)	154,813	395,386	7,522	10,656	266	8,388,608
private-kernel-reset-small	591	2,108 (-3%)	45,501 (+1%)	120,733	64,614	89,536	2,731	2,097,152
merge-rollup	29.0	N/A	N/A	16,534	756	N/A	N/A	N/A
public-kernel-setup	626	N/A	N/A	104,941	86,302	N/A	N/A	N/A
public-kernel-teardown	529	N/A	N/A	104,941	86,302	N/A	N/A	N/A
private-kernel-tail-to-public	N/A	8,835	92,821	N/A	N/A	114,784	3,520	4,194,304

Stats on running time collected for app circuits

Function	input_size_in_bytes	output_size_in_bytes	witness_generation_time_in_ms	proof_size_in_bytes	proving_time_in_ms	size_in_gates	num_public_inputs
ContractClassRegisterer:register	1,344	9,944	472 (+1%)	N/A	N/A	N/A	N/A
ContractInstanceDeployer:deploy	1,408	9,944	42.3 (+1%)	N/A	N/A	N/A	N/A
MultiCallEntrypoint:entrypoint	1,920	9,944	1,446	N/A	N/A	N/A	N/A
SchnorrAccount:constructor	1,312	9,944	1,005 (+1%)	N/A	N/A	N/A	N/A
SchnorrAccount:entrypoint	2,304	9,944	2,086	16,768	51,316 (+1%)	2,097,152	457
Token:privately_mint_private_note	1,280	9,944	1,146	N/A	N/A	N/A	N/A
Token:transfer	1,376	9,944	4,071	16,768	53,630 (+1%)	2,097,152	457
Benchmarking:create_note	1,312	9,944	951	N/A	N/A	N/A	N/A
FPC:fee_entrypoint_public	1,344	9,944	221 (+1%)	N/A	N/A	N/A	N/A
SchnorrAccount:spend_private_authwit	1,280	9,944	77.5	N/A	N/A	N/A	N/A
Token:unshield	1,376	9,944	3,253	N/A	N/A	N/A	N/A
FPC:fee_entrypoint_private	1,376	9,944	4,027	N/A	N/A	N/A	N/A

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric	1 leaves	16 leaves	64 leaves	128 leaves	512 leaves	1024 leaves	2048 leaves	4096 leaves	32 leaves
batch_insert_into_append_only_tree_16_depth_ms	10.4 (-2%)	17.0	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_count	16.7	31.8	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_ms	0.606 (-2%)	0.521	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_32_depth_ms	N/A	N/A	48.5	75.7 (-1%)	247	474	927 (-1%)	1,829 (-1%)	N/A
batch_insert_into_append_only_tree_32_depth_hash_count	N/A	N/A	95.9	159	543	1,055	2,079	4,127	N/A
batch_insert_into_append_only_tree_32_depth_hash_ms	N/A	N/A	0.496	0.467 (-1%)	0.448	0.443	0.439 (-1%)	0.438 (-1%)	N/A
batch_insert_into_indexed_tree_20_depth_ms	N/A	N/A	58.5	113 (+1%)	355	699	1,385	2,756 (-1%)	N/A
batch_insert_into_indexed_tree_20_depth_hash_count	N/A	N/A	106	208	692	1,363	2,707	5,395	N/A
batch_insert_into_indexed_tree_20_depth_hash_ms	N/A	N/A	0.507	0.505	0.481	0.480	0.479	0.478 (-1%)	N/A
batch_insert_into_indexed_tree_40_depth_ms	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	62.4
batch_insert_into_indexed_tree_40_depth_hash_count	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	107
batch_insert_into_indexed_tree_40_depth_hash_ms	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	0.553

Miscellaneous

Transaction sizes based on how many contract classes are registered in the tx.

Metric	0 registered classes	1 registered classes
tx_size_in_bytes	83,794	665,117

Transaction size based on fee payment method

| Metric | |
| - | |

Co-authored-by: Lucas Xia <[email protected]>

…have same x value?)

…cursion separator challenge)

indexing and other related fixes

lucasxia01 · 2024-05-23T20:34:30Z

barretenberg/cpp/src/barretenberg/dsl/acir_format/acir_format.cpp

            std::array<uint32_t, HonkRecursionConstraint::AGGREGATION_OBJECT_SIZE> nested_aggregation_object = {};
-            // If the proof has public inputs attached to it, we should handle setting the nested aggregation object


not sure how to replace this check that the aggregation object is there. Maybe impossible given proof sizes are variable.

I think it is not possible given the proof sizes are variable. If we add some way to attach constant metadata to the opcode we should be able to determine this as well as other information about the proof (like which verifier we want to use)

…ocol/aztec-packages into lx/honk-recursion-constraint-2

barretenberg/cpp/src/barretenberg/dsl/acir_format/acir_format.cpp

lucasxia01 · 2024-05-23T21:12:33Z

barretenberg/cpp/src/barretenberg/ultra_honk/ultra_verifier.hpp

@@ -26,7 +26,7 @@ template <typename Flavor> class UltraVerifier_ {
    bool verify_proof(const HonkProof& proof);

    std::shared_ptr<VerificationKey> key;
-    std::shared_ptr<Transcript> transcript;
+    std::shared_ptr<Transcript> transcript; // this seems useless


I'll leave a todo here I guess

lucasxia01 · 2024-05-23T21:12:59Z

barretenberg/cpp/src/barretenberg/stdlib/honk_recursion/verifier/ultra_recursive_verifier.hpp


    // TODO(luke): Eventually this will return something like aggregation_state but I'm simplifying for now until we
    // determine the exact interface. Simply returns the two pairing points.
    PairingPoints verify_proof(const HonkProof& proof);
+    PairingPoints verify_proof(const StdlibProof<Builder>& proof);


just a new function that allows you directly pass in a stdlib proof

barretenberg/cpp/src/barretenberg/dsl/acir_format/acir_format.cpp

vezenovm

I like the use of one aggregation object in the constraint, simplifies the mental model. Mostly nits

barretenberg/cpp/src/barretenberg/dsl/acir_format/acir_format.cpp

barretenberg/cpp/src/barretenberg/dsl/acir_format/honk_recursion_constraint.cpp

barretenberg/cpp/src/barretenberg/dsl/acir_format/honk_recursion_constraint.test.cpp

barretenberg/cpp/src/barretenberg/dsl/acir_format/acir_format.cpp

barretenberg/cpp/src/barretenberg/dsl/acir_format/honk_recursion_constraint.hpp

…ursionConstraint

…onstraint-2

vezenovm

LGTM. Just noting that we will need to test this against prove THEN verify rather than just prove AND verify to test we pass in the case of invalid witness assignments

vezenovm · 2024-05-28T19:57:24Z

barretenberg/cpp/src/barretenberg/dsl/acir_format/acir_format.cpp

            std::array<uint32_t, HonkRecursionConstraint::AGGREGATION_OBJECT_SIZE> nested_aggregation_object = {};
-            // If the proof has public inputs attached to it, we should handle setting the nested aggregation object


I think it is not possible given the proof sizes are variable. If we add some way to attach constant metadata to the opcode we should be able to determine this as well as other information about the proof (like which verifier we want to use)

* master: feat: prepare circuit output for validation (#6678) chore: stop building/publishing `acvm_backend.wasm` (#6584) chore: add bench programs (#6566) chore: make public data update requests, note hashes, and unencrypted logs readonly in TS (#6658) git subrepo push --branch=master noir-projects/aztec-nr git_subrepo.sh: Fix parent in .gitrepo file. [skip ci] chore: replace relative paths to noir-protocol-circuits git subrepo push --branch=master barretenberg feat: update honk recursion constraint (#6545) feat: Add code-workspace and update build dirs (#6723) feat: Sync from noir (#6717) feat: folding acir programs (#6685) feat: sumcheck part of ECCVM recursive verifier instantiated as an UltraCircuit (#6413)

update honk recursion

4d27982

lucasxia01 marked this pull request as draft May 20, 2024 21:09

lucasxia01 self-assigned this May 20, 2024

removing key hash, fixing deserialization

094f6a3

lucasxia01 added 5 commits May 22, 2024 20:49

use honk in the honk rec constraint tests

c238cd7

Co-authored-by: Lucas Xia <[email protected]>

aggregation, but failed (because biggroup elements being added can't …

f311084

…have same x value?)

aggregating input object with current object works (using 2 as the re…

be711cd

…cursion separator challenge)

hacking harder than I've ever hacked

d97b663

indexing and other related fixes

Merge branch 'master' into lx/honk-recursion-constraint-2

52e3fc0

lucasxia01 commented May 23, 2024

View reviewed changes

lucasxia01 added 2 commits May 23, 2024 20:57

fixed some of the comments, removed debug

f329435

Merge branch 'lx/honk-recursion-constraint-2' of github.com:AztecProt…

7b04aeb

…ocol/aztec-packages into lx/honk-recursion-constraint-2

lucasxia01 marked this pull request as ready for review May 23, 2024 21:10

lucasxia01 commented May 23, 2024

View reviewed changes

barretenberg/cpp/src/barretenberg/dsl/acir_format/acir_format.cpp Outdated Show resolved Hide resolved

lucasxia01 commented May 23, 2024

View reviewed changes

barretenberg/cpp/src/barretenberg/dsl/acir_format/acir_format.cpp Outdated Show resolved Hide resolved

small cleanup

b26555b

lucasxia01 requested a review from vezenovm May 23, 2024 22:34

vezenovm reviewed May 24, 2024

View reviewed changes

lucasxia01 added 2 commits May 24, 2024 18:32

changes to comments, making inner_public_input_offset part of HonkRec…

a8ce1da

…ursionConstraint

address comments

aa9bfcc

lucasxia01 requested a review from vezenovm May 24, 2024 18:55

lucasxia01 added 2 commits May 24, 2024 14:55

Merge branch 'master' into lx/honk-recursion-constraint-2

f1299f3

Merge remote-tracking branch 'origin/master' into lx/honk-recursion-c…

318eb49

…onstraint-2

vezenovm approved these changes May 28, 2024

View reviewed changes

Merge branch 'master' into lx/honk-recursion-constraint-2

06c2a0a

lucasxia01 enabled auto-merge (squash) May 28, 2024 20:50

lucasxia01 mentioned this pull request May 28, 2024

feat: Update recursion constraint for Honk #6315

Closed

lucasxia01 merged commit 6f86352 into master May 28, 2024
85 checks passed

lucasxia01 deleted the lx/honk-recursion-constraint-2 branch May 28, 2024 22:04

AztecBot mentioned this pull request May 28, 2024

chore(master): Release 0.42.0 #6572

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: update honk recursion constraint #6545

feat: update honk recursion constraint #6545

lucasxia01 commented May 20, 2024 •

edited

Loading

AztecBot commented May 20, 2024 •

edited

Loading

Proof generation

L2 block published to L1

L2 chain processing

Circuits stats

Tree insertion stats

Miscellaneous

lucasxia01 May 23, 2024

vezenovm May 28, 2024

lucasxia01 May 23, 2024

lucasxia01 May 23, 2024

vezenovm left a comment

vezenovm left a comment

vezenovm May 28, 2024

		std::array<uint32_t, HonkRecursionConstraint::AGGREGATION_OBJECT_SIZE> nested_aggregation_object = {};
		// If the proof has public inputs attached to it, we should handle setting the nested aggregation object

feat: update honk recursion constraint #6545

feat: update honk recursion constraint #6545

Conversation

lucasxia01 commented May 20, 2024 • edited Loading

AztecBot commented May 20, 2024 • edited Loading

Benchmark results

Proof generation

L2 block published to L1

L2 chain processing

Circuits stats

Tree insertion stats

Miscellaneous

lucasxia01 May 23, 2024

Choose a reason for hiding this comment

vezenovm May 28, 2024

Choose a reason for hiding this comment

lucasxia01 May 23, 2024

Choose a reason for hiding this comment

lucasxia01 May 23, 2024

Choose a reason for hiding this comment

vezenovm left a comment

Choose a reason for hiding this comment

vezenovm left a comment

Choose a reason for hiding this comment

vezenovm May 28, 2024

Choose a reason for hiding this comment

lucasxia01 commented May 20, 2024 •

edited

Loading

AztecBot commented May 20, 2024 •

edited

Loading