feat: Protogalaxy Combiner #2436

codygunton · 2023-09-20T16:26:54Z

This work introduces the Protogalaxy combiner polynomial ($G$ in the paper). I reuse structures from our sumcheck code, and also try to reflect the sumcheck code's organization, as much as possible. The main result is a test file that tests the combiner construction in three special cases, touching on 2- and 4-instance folding.

Major points

Generalize tuple of tuples containers to Protogalaxy, where they need to handle a varying number of instances.
Introduce compute_combiner function.

Minor points

Move extend and evaluate out of BarycentricData and into Univariate for convenience.
Move shared sumcheck functions (e.g. for operating on tuples of tuples) into a utilities class.
Add get_random method to RelationParameters.

…oved

maramihali · 2023-10-17T11:06:49Z

barretenberg/cpp/src/barretenberg/honk/flavor/ecc_vm.hpp

-    using TupleOfTuplesOfUnivariates = decltype(create_relation_univariates_container<FF, Relations>());
-    using TupleOfArraysOfValues = decltype(create_relation_values_container<FF, Relations>());
+    using SumcheckTupleOfTuplesOfUnivariates = decltype(create_sumcheck_tuple_of_tuples_of_univariates<Relations>());
+    using SumcheckTupleOfArraysOfValues = decltype(create_sumcheck_tuple_of_arrays_of_values<Relations>());



these are much clear names to me, also thanks for the comments :)

maramihali · 2023-10-17T11:16:52Z

barretenberg/cpp/src/barretenberg/honk/flavor/ecc_vm.hpp

@@ -712,12 +712,16 @@ template <typename CycleGroup_T, typename Curve_T, typename PCS_T> class ECCVMBa
    };

    /**
-     * @brief A container for univariates produced during the hot loop in sumcheck.
+     * @brief A container for univariates used during Protogalaxy folding and sumcheck.


we should probably have no mention of ProtoGalaxy in here since we are not going to fold ECC VM

maramihali · 2023-10-17T11:24:54Z

barretenberg/cpp/src/barretenberg/honk/flavor/ultra.hpp

@@ -54,15 +54,18 @@ class Ultra {
                                 proof_system::AuxiliaryRelation<FF>>;

    static constexpr size_t MAX_RELATION_LENGTH = get_max_relation_length<Relations>();
+    static_assert(MAX_RELATION_LENGTH == 6);


why is there a static_assert only here?

Oh this was just me checking what I thought was true. Will remove.

maramihali · 2023-10-17T11:25:55Z

barretenberg/cpp/src/barretenberg/honk/flavor/ultra.hpp

@@ -315,12 +318,16 @@ class Ultra {
    };

    /**
-     * @brief A container for univariates produced during the hot loop in sumcheck.
+     * @brief A container for univariates used during Protogalaxy folding and sumcheck.


maybe one sentence explainer of what these univariates are? here and in the other flavors

maramihali · 2023-10-17T11:27:16Z

barretenberg/cpp/src/barretenberg/honk/instance/instances.hpp

@@ -5,22 +5,54 @@ namespace proof_system::honk {

 template <typename Flavor_, size_t NUM_> struct ProverInstances_ {
    using Flavor = Flavor_;
+    using FF = typename Flavor_::FF;


is this preferred to Flavor::FF?

No there's no difference

maramihali · 2023-10-17T12:00:22Z

barretenberg/cpp/src/barretenberg/proof_system/flavor/flavor.hpp

- * Relation contributes a tuple with num-identities many Univariates and there are num-relations many tuples in the
- * outer tuple.
+ * @brief Recursive utility function to construct a container for the subrelation accumulators of Protogalaxy folding.
+ * @details Each relation contributes a tuple with num-identities many Univariates and there are num-relations many


can you pleasea define what num-identities is here?

I didn't write this but I added a clarification

maramihali · 2023-10-17T12:01:07Z

barretenberg/cpp/src/barretenberg/proof_system/flavor/flavor.hpp

+ * @brief Recursive utility function to construct a container for the subrelation accumulators of Protogalaxy folding.
+ * @details Each relation contributes a tuple with num-identities many Univariates and there are num-relations many
+ * tuples in the outer tuple. The lengths of the univariaties are determined by subrelation lengths and the number of
+ * instances to be folded.


worth giving the mathematical formula here to make it more clear

IMO more detail than we should give here. Very easy to find the function that computes this.

maramihali · 2023-10-17T13:37:12Z

barretenberg/cpp/src/barretenberg/honk/utils/testing.hpp

+template <typename Flavor>
+std::pair<std::array<barretenberg::Polynomial<typename Flavor::FF>, Flavor::NUM_ALL_ENTITIES>,
+          typename Flavor::ProverPolynomials>
+get_sequential_prover_polynomials(const size_t log_circuit_size, const size_t starting_value)


can you briefly document this?

maramihali · 2023-10-17T13:39:57Z

barretenberg/cpp/src/barretenberg/honk/sumcheck/sumcheck_round.hpp


  public:
    using FF = typename Flavor::FF;
-    template <size_t univariate_length>
-    using ExtendedEdges = typename Flavor::template ExtendedEdges<univariate_length>;
+    using ExtendedEdges = typename Flavor::ExtendedEdges;


formatting weirdness?

No, MAX_RELATION_LENGTH is computed at compile time. This didn't need to be a template.

maramihali · 2023-10-17T14:03:29Z

barretenberg/cpp/src/barretenberg/honk/proof_system/combiner.test.cpp

+using FF = typename Flavor::FF;
+using RelationParameters = proof_system::RelationParameters<FF>;
+
+TEST(Protogalaxy, Combiner2Instances)


CombinerOnTwoInstances maybe? ditto on the other test

…oved

maramihali

great work :)

🤖 I have created a release *beep* *boop* --- <details><summary>aztec-packages: 0.10.0</summary> ## [0.10.0](aztec-packages-v0.9.0...aztec-packages-v0.10.0) (2023-10-24) ### ⚠ BREAKING CHANGES * Emitting encrypted log by default ([#2926](#2926)) ### Features * Added register-account command to cli ([#2980](#2980)) ([0977a90](0977a90)) * **docs:** Fix portals tutorial formatting ([#2929](#2929)) ([ab19b67](ab19b67)) * Emitting encrypted log by default ([#2926](#2926)) ([1ea2d4f](1ea2d4f)), closes [#2912](#2912) * Goblin translator non-native field relation (Goblin Translator part 6) ([#2871](#2871)) ([c4d8d96](c4d8d96)) * Honk profiling by pass, tsan preset ([#2982](#2982)) ([a1592fd](a1592fd)) * Incorporate docs feedback and add "intermediate" level intros to some pages ([#2598](#2598)) ([78f9f52](78f9f52)) * Nuking `Pokeable` contract ([#2939](#2939)) ([583d6fb](583d6fb)) * Protogalaxy Combiner ([#2436](#2436)) ([a60c70d](a60c70d)) * Protogalaxy perturbator! ([#2624](#2624)) ([509dee6](509dee6)) * Refactor pedersen hash standard ([#2592](#2592)) ([3085676](3085676)) * Widget benchmarking ([#2897](#2897)) ([0e927e9](0e927e9)) ### Bug Fixes * Add @jest/types to box deps ([#2903](#2903)) ([db3fa62](db3fa62)) * Add lint rule for focused tests ([#2901](#2901)) ([fd1a1a8](fd1a1a8)) * Avoid tsc OOM by unignoring an old contract artifact ([#2932](#2932)) ([7310600](7310600)) * Bad it.only in tests ([#2900](#2900)) ([a1f3af1](a1f3af1)) * Boxes boostrap dont use ts-node directly and add .prettierignore ([#2890](#2890)) ([a3b1804](a3b1804)) * Confusing "Unknown complete address" error ([#2967](#2967)) ([3a8f54a](3a8f54a)) * Force jest to quit, otherwise CI can rack up to 3hrs of credits per job. ([#2899](#2899)) ([ba2f671](ba2f671)) * Honk sumcheck performance ([#2925](#2925)) ([5fbfe6e](5fbfe6e)) * Pending commitments contract using the wrong number of arguments ([#2959](#2959)) ([655c322](655c322)) * Prettierignore in boxes ([#2902](#2902)) ([8f7a200](8f7a200)) * Randomness in `AddressNote` ([#2965](#2965)) ([4dc49a9](4dc49a9)) * Yarn lock ([#2923](#2923)) ([7042bc6](7042bc6)) ### Miscellaneous * `Private Data Tree` --> `Note Hash Tree` ([#2945](#2945)) ([abaec9c](abaec9c)), closes [#2906](#2906) * Apply hash abstraction over aztec-nr ([#2958](#2958)) ([52f01ae](52f01ae)) * **docs:** Add Singleton and ImmutableSingleton `view_note` methods ([#2934](#2934)) ([c1497f8](c1497f8)) * Fix box frontend styling ([#2919](#2919)) ([7e9e8cc](7e9e8cc)) * Less noisy benchmark reports ([#2916](#2916)) ([0df166c](0df166c)) * Remove unused nix files ([#2933](#2933)) ([3174f84](3174f84)) * Run all e2e tests against sandbox ([#2891](#2891)) ([6c4e26c](6c4e26c)) * Token box copies noir source files from noir-contracts on bootstrap ([#2940](#2940)) ([a467b96](a467b96)) ### Documentation * Fix: update cheat codes to connect to ethRpcUrl ([#2922](#2922)) ([4ffe9be](4ffe9be)) </details> <details><summary>barretenberg.js: 0.10.0</summary> ## [0.10.0](barretenberg.js-v0.9.0...barretenberg.js-v0.10.0) (2023-10-24) ### Features * Refactor pedersen hash standard ([#2592](#2592)) ([3085676](3085676)) </details> <details><summary>barretenberg: 0.10.0</summary> ## [0.10.0](barretenberg-v0.9.0...barretenberg-v0.10.0) (2023-10-24) ### Features * Goblin translator non-native field relation (Goblin Translator part 6) ([#2871](#2871)) ([c4d8d96](c4d8d96)) * Honk profiling by pass, tsan preset ([#2982](#2982)) ([a1592fd](a1592fd)) * Protogalaxy Combiner ([#2436](#2436)) ([a60c70d](a60c70d)) * Protogalaxy perturbator! ([#2624](#2624)) ([509dee6](509dee6)) * Refactor pedersen hash standard ([#2592](#2592)) ([3085676](3085676)) * Widget benchmarking ([#2897](#2897)) ([0e927e9](0e927e9)) ### Bug Fixes * Honk sumcheck performance ([#2925](#2925)) ([5fbfe6e](5fbfe6e)) ### Miscellaneous * Remove unused nix files ([#2933](#2933)) ([3174f84](3174f84)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

🤖 I have created a release *beep* *boop* --- <details><summary>aztec-packages: 0.10.0</summary> ## [0.10.0](AztecProtocol/aztec-packages@aztec-packages-v0.9.0...aztec-packages-v0.10.0) (2023-10-24) ### ⚠ BREAKING CHANGES * Emitting encrypted log by default ([#2926](AztecProtocol/aztec-packages#2926)) ### Features * Added register-account command to cli ([#2980](AztecProtocol/aztec-packages#2980)) ([0977a90](AztecProtocol/aztec-packages@0977a90)) * **docs:** Fix portals tutorial formatting ([#2929](AztecProtocol/aztec-packages#2929)) ([ab19b67](AztecProtocol/aztec-packages@ab19b67)) * Emitting encrypted log by default ([#2926](AztecProtocol/aztec-packages#2926)) ([1ea2d4f](AztecProtocol/aztec-packages@1ea2d4f)), closes [#2912](AztecProtocol/aztec-packages#2912) * Goblin translator non-native field relation (Goblin Translator part 6) ([#2871](AztecProtocol/aztec-packages#2871)) ([c4d8d96](AztecProtocol/aztec-packages@c4d8d96)) * Honk profiling by pass, tsan preset ([#2982](AztecProtocol/aztec-packages#2982)) ([a1592fd](AztecProtocol/aztec-packages@a1592fd)) * Incorporate docs feedback and add "intermediate" level intros to some pages ([#2598](AztecProtocol/aztec-packages#2598)) ([78f9f52](AztecProtocol/aztec-packages@78f9f52)) * Nuking `Pokeable` contract ([#2939](AztecProtocol/aztec-packages#2939)) ([583d6fb](AztecProtocol/aztec-packages@583d6fb)) * Protogalaxy Combiner ([#2436](AztecProtocol/aztec-packages#2436)) ([a60c70d](AztecProtocol/aztec-packages@a60c70d)) * Protogalaxy perturbator! ([#2624](AztecProtocol/aztec-packages#2624)) ([509dee6](AztecProtocol/aztec-packages@509dee6)) * Refactor pedersen hash standard ([#2592](AztecProtocol/aztec-packages#2592)) ([3085676](AztecProtocol/aztec-packages@3085676)) * Widget benchmarking ([#2897](AztecProtocol/aztec-packages#2897)) ([0e927e9](AztecProtocol/aztec-packages@0e927e9)) ### Bug Fixes * Add @jest/types to box deps ([#2903](AztecProtocol/aztec-packages#2903)) ([db3fa62](AztecProtocol/aztec-packages@db3fa62)) * Add lint rule for focused tests ([#2901](AztecProtocol/aztec-packages#2901)) ([fd1a1a8](AztecProtocol/aztec-packages@fd1a1a8)) * Avoid tsc OOM by unignoring an old contract artifact ([#2932](AztecProtocol/aztec-packages#2932)) ([7310600](AztecProtocol/aztec-packages@7310600)) * Bad it.only in tests ([#2900](AztecProtocol/aztec-packages#2900)) ([a1f3af1](AztecProtocol/aztec-packages@a1f3af1)) * Boxes boostrap dont use ts-node directly and add .prettierignore ([#2890](AztecProtocol/aztec-packages#2890)) ([a3b1804](AztecProtocol/aztec-packages@a3b1804)) * Confusing "Unknown complete address" error ([#2967](AztecProtocol/aztec-packages#2967)) ([3a8f54a](AztecProtocol/aztec-packages@3a8f54a)) * Force jest to quit, otherwise CI can rack up to 3hrs of credits per job. ([#2899](AztecProtocol/aztec-packages#2899)) ([ba2f671](AztecProtocol/aztec-packages@ba2f671)) * Honk sumcheck performance ([#2925](AztecProtocol/aztec-packages#2925)) ([5fbfe6e](AztecProtocol/aztec-packages@5fbfe6e)) * Pending commitments contract using the wrong number of arguments ([#2959](AztecProtocol/aztec-packages#2959)) ([655c322](AztecProtocol/aztec-packages@655c322)) * Prettierignore in boxes ([#2902](AztecProtocol/aztec-packages#2902)) ([8f7a200](AztecProtocol/aztec-packages@8f7a200)) * Randomness in `AddressNote` ([#2965](AztecProtocol/aztec-packages#2965)) ([4dc49a9](AztecProtocol/aztec-packages@4dc49a9)) * Yarn lock ([#2923](AztecProtocol/aztec-packages#2923)) ([7042bc6](AztecProtocol/aztec-packages@7042bc6)) ### Miscellaneous * `Private Data Tree` --> `Note Hash Tree` ([#2945](AztecProtocol/aztec-packages#2945)) ([abaec9c](AztecProtocol/aztec-packages@abaec9c)), closes [#2906](AztecProtocol/aztec-packages#2906) * Apply hash abstraction over aztec-nr ([#2958](AztecProtocol/aztec-packages#2958)) ([52f01ae](AztecProtocol/aztec-packages@52f01ae)) * **docs:** Add Singleton and ImmutableSingleton `view_note` methods ([#2934](AztecProtocol/aztec-packages#2934)) ([c1497f8](AztecProtocol/aztec-packages@c1497f8)) * Fix box frontend styling ([#2919](AztecProtocol/aztec-packages#2919)) ([7e9e8cc](AztecProtocol/aztec-packages@7e9e8cc)) * Less noisy benchmark reports ([#2916](AztecProtocol/aztec-packages#2916)) ([0df166c](AztecProtocol/aztec-packages@0df166c)) * Remove unused nix files ([#2933](AztecProtocol/aztec-packages#2933)) ([3174f84](AztecProtocol/aztec-packages@3174f84)) * Run all e2e tests against sandbox ([#2891](AztecProtocol/aztec-packages#2891)) ([6c4e26c](AztecProtocol/aztec-packages@6c4e26c)) * Token box copies noir source files from noir-contracts on bootstrap ([#2940](AztecProtocol/aztec-packages#2940)) ([a467b96](AztecProtocol/aztec-packages@a467b96)) ### Documentation * Fix: update cheat codes to connect to ethRpcUrl ([#2922](AztecProtocol/aztec-packages#2922)) ([4ffe9be](AztecProtocol/aztec-packages@4ffe9be)) </details> <details><summary>barretenberg.js: 0.10.0</summary> ## [0.10.0](AztecProtocol/aztec-packages@barretenberg.js-v0.9.0...barretenberg.js-v0.10.0) (2023-10-24) ### Features * Refactor pedersen hash standard ([#2592](AztecProtocol/aztec-packages#2592)) ([3085676](AztecProtocol/aztec-packages@3085676)) </details> <details><summary>barretenberg: 0.10.0</summary> ## [0.10.0](AztecProtocol/aztec-packages@barretenberg-v0.9.0...barretenberg-v0.10.0) (2023-10-24) ### Features * Goblin translator non-native field relation (Goblin Translator part 6) ([#2871](AztecProtocol/aztec-packages#2871)) ([c4d8d96](AztecProtocol/aztec-packages@c4d8d96)) * Honk profiling by pass, tsan preset ([#2982](AztecProtocol/aztec-packages#2982)) ([a1592fd](AztecProtocol/aztec-packages@a1592fd)) * Protogalaxy Combiner ([#2436](AztecProtocol/aztec-packages#2436)) ([a60c70d](AztecProtocol/aztec-packages@a60c70d)) * Protogalaxy perturbator! ([#2624](AztecProtocol/aztec-packages#2624)) ([509dee6](AztecProtocol/aztec-packages@509dee6)) * Refactor pedersen hash standard ([#2592](AztecProtocol/aztec-packages#2592)) ([3085676](AztecProtocol/aztec-packages@3085676)) * Widget benchmarking ([#2897](AztecProtocol/aztec-packages#2897)) ([0e927e9](AztecProtocol/aztec-packages@0e927e9)) ### Bug Fixes * Honk sumcheck performance ([#2925](AztecProtocol/aztec-packages#2925)) ([5fbfe6e](AztecProtocol/aztec-packages@5fbfe6e)) ### Miscellaneous * Remove unused nix files ([#2933](AztecProtocol/aztec-packages#2933)) ([3174f84](AztecProtocol/aztec-packages@3174f84)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

codygunton added 26 commits September 8, 2023 19:42

Note mistake in ultra grumpkin flavor.

37599b1

Move some sumcheck utils to a separate class.

b37d9a1

Move batching functions to Utils

ae8455b

WIP combiner

6cb2c6a

Fix GCC compilation.

6ef8b7f

Move extend and evaluate into Univariate.

3613f74

Fix compilation.

1ac5217

Remove TODO?

68b3c2c

Note unused template arg.

c7296a6

Note out of date documentation

33dc1ce

Use relations statically.

8e7d8c0

Touch up relations test.

0a5496e

Remove unused template params

613e0a5

Merge branch 'master' into cg/pg-combiner

5e3a545

Prep_ more informative type names

265b54c

Rename ExtendedEdges template.

4f0d00e

Combiner builds!

e74d800

Remove declared but undefined funtion.

88355da

WIP debugging (for Mara).

a9a3f2b

Fix indexing

ca9dfd5

Combiner works in two examples.

05db33a

Merge commit '404ec34d3^' into cg/pg-combiner

84b3e59

first suggested commit

f2eb5fc

removed build system

f538613

Merge commit '404ec34d3' into cg/pg-combiner

68c8a91

Merge branch 'master' into cg/pg-combiner

13b034a

codygunton mentioned this pull request Sep 20, 2023

feat: ProtoGalaxy Combiner #2152

Closed

4 tasks

codygunton changed the title ~~Cg/pg combiner bb moved~~ feat: Protogalaxy Combiner Sep 20, 2023

codygunton changed the base branch from master to cg/pg-combiner September 20, 2023 16:31

codygunton changed the base branch from cg/pg-combiner to master September 20, 2023 16:32

codygunton added 17 commits October 12, 2023 18:50

Handle in another PR.

5c7f244

Resolve TODO with static_assert

8c8abab

Attach issue to potential bug.

85914c4

Make ExtendedEdges alias in each flavor.

cd3b6f4

Merge remote-tracking branch 'origin/master' into cg/pg-combiner-bb-m…

fdf65a0

…oved

Rename tuple creation functions

58c82fc

Rename sumcheck-specific type.

bcca285

More type massage.

9ab5534

Add Sumcheck to type names

80ee994

Name to Protogalaxy tuple

324828b

Use the PG type...

16ac9a1

Settable numbers of instances.

fff7f52

Better setting

f6bae20

Test 1+3 instances.

a1211dd

Document instance function and add TODO.

f682b18

Move into Mara's classes.

6e8c34c

Improve comments.

6c4ced5

codygunton requested a review from maramihali October 16, 2023 13:40

Merge remote-tracking branch 'origin/master' into cg/pg-combiner-bb-m…

1dbc832

…oved

codygunton marked this pull request as ready for review October 16, 2023 13:42

maramihali requested changes Oct 17, 2023

View reviewed changes

codygunton added 2 commits October 17, 2023 19:07

Respond to review

1de58f7

Merge remote-tracking branch 'origin/master' into cg/pg-combiner-bb-m…

4e6a7f6

…oved

maramihali approved these changes Oct 17, 2023

View reviewed changes

codygunton merged commit a60c70d into master Oct 17, 2023
3 checks passed

codygunton deleted the cg/pg-combiner-bb-moved branch October 17, 2023 21:29

AztecBot mentioned this pull request Oct 17, 2023

chore(master): Release 0.10.0 #2893

Merged

codygunton mentioned this pull request Oct 17, 2023

Implement computation of G = folding combiner polynomial AztecProtocol/barretenberg#688

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Protogalaxy Combiner #2436

feat: Protogalaxy Combiner #2436

codygunton commented Sep 20, 2023 •

edited

Loading

maramihali Oct 17, 2023

maramihali Oct 17, 2023

codygunton Oct 17, 2023

maramihali Oct 17, 2023

codygunton Oct 17, 2023

maramihali Oct 17, 2023

codygunton Oct 17, 2023

maramihali Oct 17, 2023

codygunton Oct 17, 2023

maramihali Oct 17, 2023

codygunton Oct 17, 2023 •

edited

Loading

maramihali Oct 17, 2023

codygunton Oct 17, 2023 •

edited

Loading

maramihali Oct 17, 2023

codygunton Oct 17, 2023

maramihali Oct 17, 2023

codygunton Oct 17, 2023 •

edited

Loading

maramihali Oct 17, 2023

codygunton Oct 17, 2023

maramihali left a comment

feat: Protogalaxy Combiner #2436

feat: Protogalaxy Combiner #2436

Conversation

codygunton commented Sep 20, 2023 • edited Loading

Major points

Minor points

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codygunton Oct 17, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codygunton Oct 17, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codygunton Oct 17, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maramihali left a comment

Choose a reason for hiding this comment

codygunton commented Sep 20, 2023 •

edited

Loading

codygunton Oct 17, 2023 •

edited

Loading

codygunton Oct 17, 2023 •

edited

Loading

codygunton Oct 17, 2023 •

edited

Loading