Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Protogalaxy Combiner #2436

Merged
merged 66 commits into from
Oct 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
37599b1
Note mistake in ultra grumpkin flavor.
codygunton Sep 6, 2023
b37d9a1
Move some sumcheck utils to a separate class.
codygunton Sep 7, 2023
ae8455b
Move batching functions to Utils
codygunton Sep 8, 2023
6cb2c6a
WIP combiner
codygunton Sep 8, 2023
6ef8b7f
Fix GCC compilation.
codygunton Sep 8, 2023
3613f74
Move extend and evaluate into Univariate.
codygunton Sep 9, 2023
1ac5217
Fix compilation.
codygunton Sep 9, 2023
68b3c2c
Remove TODO?
codygunton Sep 10, 2023
c7296a6
Note unused template arg.
codygunton Sep 10, 2023
33dc1ce
Note out of date documentation
codygunton Sep 10, 2023
8e7d8c0
Use relations statically.
codygunton Sep 10, 2023
0a5496e
Touch up relations test.
codygunton Sep 11, 2023
613e0a5
Remove unused template params
codygunton Sep 11, 2023
5e3a545
Merge branch 'master' into cg/pg-combiner
codygunton Sep 11, 2023
265b54c
Prep_ more informative type names
codygunton Sep 12, 2023
4f0d00e
Rename ExtendedEdges template.
codygunton Sep 12, 2023
e74d800
Combiner builds!
codygunton Sep 12, 2023
88355da
Remove declared but undefined funtion.
codygunton Sep 13, 2023
a9a3f2b
WIP debugging (for Mara).
codygunton Sep 18, 2023
ca9dfd5
Fix indexing
codygunton Sep 18, 2023
05db33a
Combiner works in two examples.
codygunton Sep 19, 2023
84b3e59
Merge commit '404ec34d3^' into cg/pg-combiner
codygunton Sep 20, 2023
f2eb5fc
first suggested commit
codygunton Sep 20, 2023
f538613
removed build system
codygunton Sep 20, 2023
68c8a91
Merge commit '404ec34d3' into cg/pg-combiner
codygunton Sep 20, 2023
13b034a
Merge branch 'master' into cg/pg-combiner
codygunton Sep 20, 2023
9f2b1c7
Fix ECCVM flavor
codygunton Sep 20, 2023
cdb65c4
Reinstate dockerfile
codygunton Sep 20, 2023
eea3b05
Remove files added in merge.
codygunton Sep 20, 2023
a1c827a
clan16 in barretenberg-x86_64-linux-clang-assert
codygunton Sep 20, 2023
916d1bf
Revert "clan16 in barretenberg-x86_64-linux-clang-assert"
codygunton Sep 20, 2023
f48e0e5
Delete file added in merge
codygunton Sep 20, 2023
97c093a
Merge branch 'master' into cg/pg-combiner-bb-moved
codygunton Sep 23, 2023
24f48ad
It compiles
codygunton Sep 25, 2023
6501937
Fix test for Ultra flavor.
codygunton Sep 25, 2023
40051c8
Merge remote-tracking branch 'origin/master' into cg/pg-combiner-bb-m…
codygunton Sep 25, 2023
f0101bd
Manually apply formatter.
codygunton Sep 25, 2023
4b73492
Merge remote-tracking branch 'origin/master' into cg/pg-combiner-bb-m…
codygunton Sep 25, 2023
fea7da8
Merge remote-tracking branch 'origin/master' into cg/pg-combiner-bb-m…
codygunton Sep 27, 2023
3eec73b
Fix second combiner example.
codygunton Sep 27, 2023
ab88d34
Fix formatting.
codygunton Sep 27, 2023
8ecf064
Merge branch 'master' into cg/pg-combiner-bb-moved
codygunton Oct 11, 2023
c02f71d
Unstaged changes.
codygunton Oct 11, 2023
8d080b9
Merge remote-tracking branch 'origin/master' into cg/pg-combiner-bb-m…
codygunton Oct 11, 2023
4cc66ea
Merge remote-tracking branch 'origin/master' into cg/pg-combiner-bb-m…
codygunton Oct 12, 2023
2f68fe6
Merge remote-tracking branch 'origin/master' into cg/pg-combiner-bb-m…
codygunton Oct 12, 2023
5c7f244
Handle in another PR.
codygunton Oct 12, 2023
8c8abab
Resolve TODO with static_assert
codygunton Oct 12, 2023
85914c4
Attach issue to potential bug.
codygunton Oct 12, 2023
cd3b6f4
Make ExtendedEdges alias in each flavor.
codygunton Oct 12, 2023
fdf65a0
Merge remote-tracking branch 'origin/master' into cg/pg-combiner-bb-m…
codygunton Oct 12, 2023
58c82fc
Rename tuple creation functions
codygunton Oct 12, 2023
bcca285
Rename sumcheck-specific type.
codygunton Oct 12, 2023
9ab5534
More type massage.
codygunton Oct 12, 2023
80ee994
Add Sumcheck to type names
codygunton Oct 16, 2023
324828b
Name to Protogalaxy tuple
codygunton Oct 16, 2023
16ac9a1
Use the PG type...
codygunton Oct 16, 2023
fff7f52
Settable numbers of instances.
codygunton Oct 16, 2023
f6bae20
Better setting
codygunton Oct 16, 2023
a1211dd
Test 1+3 instances.
codygunton Oct 16, 2023
f682b18
Document instance function and add TODO.
codygunton Oct 16, 2023
6e8c34c
Move into Mara's classes.
codygunton Oct 16, 2023
6c4ced5
Improve comments.
codygunton Oct 16, 2023
1dbc832
Merge remote-tracking branch 'origin/master' into cg/pg-combiner-bb-m…
codygunton Oct 16, 2023
1de58f7
Respond to review
codygunton Oct 17, 2023
4e6a7f6
Merge remote-tracking branch 'origin/master' into cg/pg-combiner-bb-m…
codygunton Oct 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#include "barretenberg/polynomials/barycentric.hpp"
#include "barretenberg/ecc/curves/bn254/fr.hpp"
#include "barretenberg/polynomials/univariate.hpp"
#include <benchmark/benchmark.h>

using namespace benchmark;
Expand All @@ -17,9 +17,8 @@ namespace proof_system::benchmark {
void extend_2_to_6(State& state) noexcept
{
auto univariate = Univariate<FF, 2>::get_random();
BarycentricData<FF, 2, 6> barycentric_2_to_6;
for (auto _ : state) {
DoNotOptimize(barycentric_2_to_6.extend(univariate));
DoNotOptimize(univariate.extend_to<6>());
}
}
BENCHMARK(extend_2_to_6);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,24 +19,15 @@ using FF = barretenberg::fr;

template <typename Flavor, typename Relation> void execute_relation(::benchmark::State& state)
{
// Generate beta and gamma
auto beta = FF::random_element();
auto gamma = FF::random_element();
auto public_input_delta = FF::random_element();

RelationParameters<FF> params{
.beta = beta,
.gamma = gamma,
.public_input_delta = public_input_delta,
};

using AllValues = typename Flavor::AllValues;
using ArrayOfValuesOverSubrelations = typename Relation::ArrayOfValuesOverSubrelations;
using SumcheckArrayOfValuesOverSubrelations = typename Relation::SumcheckArrayOfValuesOverSubrelations;

auto params = proof_system::RelationParameters<FF>::get_random();

// Extract an array containing all the polynomial evaluations at a given row i
AllValues new_value;
// Define the appropriate ArrayOfValuesOverSubrelations type for this relation and initialize to zero
ArrayOfValuesOverSubrelations accumulator;
// Define the appropriate SumcheckArrayOfValuesOverSubrelations type for this relation and initialize to zero
SumcheckArrayOfValuesOverSubrelations accumulator;
// Evaluate each constraint in the relation and check that each is satisfied

for (auto _ : state) {
Expand Down
15 changes: 9 additions & 6 deletions barretenberg/cpp/src/barretenberg/honk/flavor/ecc_vm.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,8 @@ template <typename CycleGroup_T, typename Curve_T, typename PCS_T> class ECCVMBa
// static_assert(instantiate_barycentric_utils<FF, MAX_RANDOM_RELATION_LENGTH>());

// define the containers for storing the contributions from each relation in Sumcheck
using TupleOfTuplesOfUnivariates = decltype(create_relation_univariates_container<FF, Relations>());
using TupleOfArraysOfValues = decltype(create_relation_values_container<FF, Relations>());
using SumcheckTupleOfTuplesOfUnivariates = decltype(create_sumcheck_tuple_of_tuples_of_univariates<Relations>());
using TupleOfArraysOfValues = decltype(create_sumcheck_tuple_of_arrays_of_values<Relations>());

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are much clear names to me, also thanks for the comments :)

private:
/**
Expand Down Expand Up @@ -712,13 +712,16 @@ template <typename CycleGroup_T, typename Curve_T, typename PCS_T> class ECCVMBa
}
};

/**
* @brief A container for univariates used during sumcheck.
*/
template <size_t LENGTH>
using ProverUnivariates = AllEntities<barretenberg::Univariate<FF, LENGTH>, barretenberg::Univariate<FF, LENGTH>>;

/**
* @brief A container for univariates produced during the hot loop in sumcheck.
* @todo TODO(#390): Simplify this by moving MAX_RELATION_LENGTH?
*/
template <size_t MAX_RELATION_LENGTH>
using ExtendedEdges = AllEntities<barretenberg::Univariate<FF, MAX_RELATION_LENGTH>,
barretenberg::Univariate<FF, MAX_RELATION_LENGTH>>;
using ExtendedEdges = ProverUnivariates<MAX_RELATION_LENGTH>;

/**
* @brief A container for the prover polynomials handles; only stores spans.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ TEST(Flavor, Getters)

Flavor::VerificationKey verification_key;
Flavor::ProverPolynomials prover_polynomials;
Flavor::ExtendedEdges<Flavor::NUM_ALL_ENTITIES> edges;
Flavor::AllValues evals;
Flavor::CommitmentLabels commitment_labels;

Expand Down
20 changes: 13 additions & 7 deletions barretenberg/cpp/src/barretenberg/honk/flavor/goblin_ultra.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,11 @@ class GoblinUltra {
static constexpr size_t MAX_RANDOM_RELATION_LENGTH = MAX_RELATION_LENGTH + 1;
static constexpr size_t NUM_RELATIONS = std::tuple_size<Relations>::value;

// define the container for storing the univariate contribution from each relation in Sumcheck
using TupleOfTuplesOfUnivariates = decltype(create_relation_univariates_container<FF, Relations>());
using TupleOfArraysOfValues = decltype(create_relation_values_container<FF, Relations>());
template <size_t NUM_INSTANCES>
using ProtogalaxyTupleOfTuplesOfUnivariates =
decltype(create_protogalaxy_tuple_of_tuples_of_univariates<Relations, NUM_INSTANCES>());
using SumcheckTupleOfTuplesOfUnivariates = decltype(create_sumcheck_tuple_of_tuples_of_univariates<Relations>());
using TupleOfArraysOfValues = decltype(create_sumcheck_tuple_of_arrays_of_values<Relations>());

// Whether or not the first row of the execution trace is reserved for 0s to enable shifts
static constexpr bool has_zero_row = true;
Expand Down Expand Up @@ -318,13 +320,17 @@ class GoblinUltra {
}
};

/**
* @brief A container for univariates used during Protogalaxy folding and sumcheck.
* @details During folding and sumcheck, the prover evaluates the relations on these univariates.
*/
template <size_t LENGTH>
using ProverUnivariates = AllEntities<barretenberg::Univariate<FF, LENGTH>, barretenberg::Univariate<FF, LENGTH>>;

/**
* @brief A container for univariates produced during the hot loop in sumcheck.
* @todo TODO(#390): Simplify this by moving MAX_RELATION_LENGTH?
*/
template <size_t MAX_RELATION_LENGTH>
using ExtendedEdges = AllEntities<barretenberg::Univariate<FF, MAX_RELATION_LENGTH>,
barretenberg::Univariate<FF, MAX_RELATION_LENGTH>>;
using ExtendedEdges = ProverUnivariates<MAX_RELATION_LENGTH>;

/**
* @brief A field element for each entity of the flavor. These entities represent the prover polynomials evaluated
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,8 @@ template <typename BuilderType> class GoblinUltraRecursive_ {
static constexpr size_t NUM_RELATIONS = std::tuple_size<Relations>::value;

// define the container for storing the univariate contribution from each relation in Sumcheck
using TupleOfTuplesOfUnivariates = decltype(create_relation_univariates_container<FF, Relations>());
using TupleOfArraysOfValues = decltype(create_relation_values_container<FF, Relations>());
using SumcheckTupleOfTuplesOfUnivariates = decltype(create_sumcheck_tuple_of_tuples_of_univariates<Relations>());
using TupleOfArraysOfValues = decltype(create_sumcheck_tuple_of_arrays_of_values<Relations>());

private:
template <typename DataType, typename HandleType>
Expand Down
20 changes: 13 additions & 7 deletions barretenberg/cpp/src/barretenberg/honk/flavor/ultra.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,11 @@ class Ultra {
static constexpr size_t MAX_RANDOM_RELATION_LENGTH = MAX_RELATION_LENGTH + 1;
static constexpr size_t NUM_RELATIONS = std::tuple_size<Relations>::value;

// define the container for storing the univariate contribution from each relation in Sumcheck
using TupleOfTuplesOfUnivariates = decltype(create_relation_univariates_container<FF, Relations>());
using TupleOfArraysOfValues = decltype(create_relation_values_container<FF, Relations>());
template <size_t NUM_INSTANCES>
using ProtogalaxyTupleOfTuplesOfUnivariates =
decltype(create_protogalaxy_tuple_of_tuples_of_univariates<Relations, NUM_INSTANCES>());
using SumcheckTupleOfTuplesOfUnivariates = decltype(create_sumcheck_tuple_of_tuples_of_univariates<Relations>());
using TupleOfArraysOfValues = decltype(create_sumcheck_tuple_of_arrays_of_values<Relations>());

// Whether or not the first row of the execution trace is reserved for 0s to enable shifts
static constexpr bool has_zero_row = true;
Expand Down Expand Up @@ -315,13 +317,17 @@ class Ultra {
}
};

/**
* @brief A container for univariates used during Protogalaxy folding and sumcheck.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe one sentence explainer of what these univariates are? here and in the other flavors

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

* @details During folding and sumcheck, the prover evaluates the relations on these univariates.
*/
template <size_t LENGTH>
using ProverUnivariates = AllEntities<barretenberg::Univariate<FF, LENGTH>, barretenberg::Univariate<FF, LENGTH>>;

/**
* @brief A container for univariates produced during the hot loop in sumcheck.
* @todo TODO(#390): Simplify this by moving MAX_RELATION_LENGTH?
*/
template <size_t MAX_RELATION_LENGTH>
using ExtendedEdges = AllEntities<barretenberg::Univariate<FF, MAX_RELATION_LENGTH>,
barretenberg::Univariate<FF, MAX_RELATION_LENGTH>>;
using ExtendedEdges = ProverUnivariates<MAX_RELATION_LENGTH>;

/**
* @brief A container for commitment labels.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ namespace proof_system::honk::flavor {
* @details This flavor can be used to instantiate a recursive Ultra Honk verifier for a proof created using the
* conventional Ultra flavor. It is similar in structure to its native counterpart with two main differences: 1) the
* curve types are stdlib types (e.g. field_t instead of field) and 2) it does not specify any Prover related types
* (e.g. Polynomial, ExtendedEdges, etc.) since we do not emulate prover computation in circuits, i.e. it only makes
* (e.g. Polynomial, ProverUnivariates, etc.) since we do not emulate prover computation in circuits, i.e. it only makes
* sense to instantiate a Verifier with this flavor.
*
* @note Unlike conventional flavors, "recursive" flavors are templated by a builder (much like native vs stdlib types).
Expand Down Expand Up @@ -84,8 +84,8 @@ template <typename BuilderType> class UltraRecursive_ {
static constexpr size_t NUM_RELATIONS = std::tuple_size<Relations>::value;

// define the container for storing the univariate contribution from each relation in Sumcheck
using TupleOfTuplesOfUnivariates = decltype(create_relation_univariates_container<FF, Relations>());
using TupleOfArraysOfValues = decltype(create_relation_values_container<FF, Relations>());
using SumcheckTupleOfTuplesOfUnivariates = decltype(create_sumcheck_tuple_of_tuples_of_univariates<Relations>());
using TupleOfArraysOfValues = decltype(create_sumcheck_tuple_of_arrays_of_values<Relations>());

private:
template <typename DataType, typename HandleType>
Expand Down
32 changes: 32 additions & 0 deletions barretenberg/cpp/src/barretenberg/honk/instance/instances.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ namespace proof_system::honk {

template <typename Flavor_, size_t NUM_> struct ProverInstances_ {
using Flavor = Flavor_;
using FF = typename Flavor::FF;
using Instance = ProverInstance_<Flavor>;
using ArrayType = std::array<std::shared_ptr<Instance>, NUM_>;

Expand All @@ -14,13 +15,44 @@ template <typename Flavor_, size_t NUM_> struct ProverInstances_ {
std::shared_ptr<Instance> const& operator[](size_t idx) const { return _data[idx]; }
typename ArrayType::iterator begin() { return _data.begin(); };
typename ArrayType::iterator end() { return _data.end(); };
ProverInstances_() = default;
ProverInstances_(std::vector<std::shared_ptr<Instance>> data)
{
ASSERT(data.size() == NUM);
for (size_t idx = 0; idx < data.size(); idx++) {
_data[idx] = std::move(data[idx]);
}
};

/**
* @brief For a prover polynomial label and a fixed row index, construct a uninvariate from the corresponding value
* from each instance.
*
* @example if the prover polynomia index is 1 and the row index is 2, and there are 4 instances visually we have
*
* Instance 0 Instance 1 Instance 2 Instance 3
* q_c q_l q_r ... q_c q_l q_r ... q_c q_l q_r ... q_c q_l q_r ...
* * * * * * * * *
* * * * * * * * *
* * a * b * c * d
* * * * * * * * *
*
* and the function returns the univariate {a, b, c, d}
*
* @param entity_idx A fixed column position in several execution traces.
* @param row_idx A fixed row position in several execution
* @return Univariate<FF, NUM> The univariate whose extensions will be used to construct the combiner.
*/
Univariate<FF, NUM> row_to_univariate(const size_t prover_polynomial_idx, const size_t row_idx) const
{
Univariate<FF, NUM> result;
size_t instance_idx = 0;
for (auto& instance : _data) {
result.evaluations[instance_idx] = instance->prover_polynomials._data[prover_polynomial_idx][row_idx];
instance_idx++;
}
return result;
}
};

template <typename Flavor_, size_t NUM_> struct VerifierInstances_ {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ namespace proof_system::honk {
*
*/
// TODO(https://github.com/AztecProtocol/barretenberg/issues/725): create an Instances class that manages several
// Instance and passes them to ProtoGakaxy prover and verifier so that Instance objects don't need to mantain an index
// Instance and passes them to ProtoGalaxy prover and verifier so that Instance objects don't need to mantain an index
template <class Flavor> class ProverInstance_ {
using Circuit = typename Flavor::CircuitBuilder;
using ProvingKey = typename Flavor::ProvingKey;
Expand Down Expand Up @@ -59,6 +59,7 @@ template <class Flavor> class ProverInstance_ {
, public_inputs(result.folded_public_inputs)
, folding_parameters(result.folding_parameters){};

ProverInstance_() = default;
~ProverInstance_() = default;

std::shared_ptr<VerificationKey> compute_verification_key();
Expand Down
Loading