Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor!: Use circuit builders #501

Merged
merged 65 commits into from
Jun 21, 2023
Merged
Show file tree
Hide file tree
Changes from 64 commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
2b681d2
Initial commit so tests run
codygunton Jun 2, 2023
4fe7ac2
Illustrate aux gate issue
codygunton Jun 2, 2023
ae2f094
Add notes, note last point where circuit checks
codygunton Jun 5, 2023
56ebb56
Merge branch 'master' into cg/big-split
codygunton Jun 12, 2023
a779895
composer~>circuit_constructor
codygunton Jun 12, 2023
bfd910f
StandardHonk test use CCtor
codygunton Jun 12, 2023
7b4ed77
UltraHonk tests use CCtor
codygunton Jun 12, 2023
bc4bcbe
Remove honk composer include from UH tests.
codygunton Jun 12, 2023
33f31ab
Merge branch 'master' into cg/big-split
codygunton Jun 13, 2023
83412a7
Honk doesn't depend on Plonk anymore
codygunton Jun 13, 2023
83cf7d9
Fix honk tests with Grumpkin.
codygunton Jun 13, 2023
08e2741
Honk is split.
codygunton Jun 13, 2023
dc05a2c
Plonk tests split.
codygunton Jun 13, 2023
40e4d69
Fix recursive verifier
codygunton Jun 13, 2023
ecea160
Move ensure_... to finalize_circuit.
codygunton Jun 14, 2023
4e7006d
Move _ensure_ to composer helper.
codygunton Jun 14, 2023
5b10604
Cleanup
codygunton Jun 14, 2023
e3e1d45
Re-add untested code used by circuits.
codygunton Jun 14, 2023
063250b
Split honk benchmarks.
codygunton Jun 14, 2023
624b4fa
Split ultra honk bench.
codygunton Jun 14, 2023
c32e542
Fix include.
codygunton Jun 15, 2023
8a4c34c
Split plonk bench.
codygunton Jun 15, 2023
4b9b7be
Pedersen bench
codygunton Jun 15, 2023
e95b048
Clean up includes
codygunton Jun 15, 2023
0fdd8b5
Cleanup includes (esp composers.hpp)
codygunton Jun 15, 2023
ed6131f
Split simple example
codygunton Jun 16, 2023
ae5e01f
Split acir_format.
codygunton Jun 16, 2023
27f0b68
Split acir_proofs
codygunton Jun 16, 2023
a084cb7
Remove one more composers_fwd.hpp
codygunton Jun 16, 2023
7d5d87c
Split ram_table test.
codygunton Jun 16, 2023
9eb2f1e
Split rom table test.
codygunton Jun 16, 2023
c0503b8
Fix bb.js from Charlie
codygunton Jun 16, 2023
665a54a
Revert "Fix bb.js from Charlie"
codygunton Jun 16, 2023
09baba7
Merge branch 'master' into cg/big-split
codygunton Jun 16, 2023
44851bd
Split j-s example (partial)
codygunton Jun 16, 2023
0d82d1f
Split j-s example (& delete cruft).
codygunton Jun 16, 2023
9788410
Fix little j-s tests.
codygunton Jun 16, 2023
180041f
Reference refactoring issue
codygunton Jun 16, 2023
89c9bde
Split solidity helpers.
codygunton Jun 19, 2023
a9aa020
Split external bench.
codygunton Jun 19, 2023
d0a277f
Split sha256 bench.
codygunton Jun 19, 2023
30e714e
Split transcript tests (temp hide others)
codygunton Jun 19, 2023
bd46581
Unhide vk test
codygunton Jun 19, 2023
e87617c
Split stdlib vk test
codygunton Jun 19, 2023
1fea680
Uncomment verifier tests.
codygunton Jun 19, 2023
77fbc30
Split and refactor stdlib verifier tests
codygunton Jun 19, 2023
61b2c6c
Update composers.hpp
codygunton Jun 20, 2023
3773724
Quick-and-dirty to fix(?) circuits
codygunton Jun 20, 2023
0a13946
Point to circuits feature branch
codygunton Jun 20, 2023
0493ef0
Need to use commit hash(?)
codygunton Jun 20, 2023
efb4119
Undo hack to stop exposing manifest in circuits.
codygunton Jun 21, 2023
daec1bc
Make cast more explicit.
codygunton Jun 21, 2023
b053b79
Need Montgomery conversion.
codygunton Jun 21, 2023
02f9bd4
Merge remote-tracking branch 'origin/master' into cg/big-split
codygunton Jun 21, 2023
86ed54f
WORKTODO: remove unused function
codygunton Jun 21, 2023
4a642c6
Removed hack function.
codygunton Jun 21, 2023
dc25ece
WORKTODO: remove non-circuit-focused test.
codygunton Jun 21, 2023
a33cb5b
WORKTODOs: misc
codygunton Jun 21, 2023
a6ab841
WORKTODO: mv stdlib composers files.
codygunton Jun 21, 2023
22d2ff6
Update packages commit.
codygunton Jun 21, 2023
63a814d
Run acir_format_tests in CI
codygunton Jun 21, 2023
36a8a24
Cleanup
codygunton Jun 21, 2023
e3954de
Try to use automatic workflow
codygunton Jun 21, 2023
74124ba
Whoops, I misunderstood--set packages commit.
codygunton Jun 21, 2023
77742d6
Reformat some comments.
codygunton Jun 21, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,18 @@ jobs:
command: cond_spot_run_tests barretenberg-x86_64-linux-clang-assert 1 stdlib-tests
- *save_logs

acir-format-tests:
docker:
- image: aztecprotocol/alpine-build-image
resource_class: small
steps:
- *checkout
- *setup_env
- run:
name: "Test"
command: cond_spot_run_tests barretenberg-x86_64-linux-clang-assert 1 acir_format_tests
- *save_logs

barretenberg-tests:
docker:
- image: aztecprotocol/alpine-build-image
Expand Down Expand Up @@ -375,6 +387,7 @@ workflows:
- wasm-linux-clang: *defaults
- proof-system-tests: *bb_test
- honk-tests: *bb_test
- acir-format-tests: *bb_test
- barretenberg-tests: *bb_test
- stdlib-tests: *bb_test
- stdlib-recursion-turbo-tests: *bb_test
Expand Down
2 changes: 1 addition & 1 deletion cpp/.aztec-packages-commit
Original file line number Diff line number Diff line change
@@ -1 +1 @@
master
64c8ba700b75df07a8452a6f2eae3d23cf7625a6
2 changes: 0 additions & 2 deletions cpp/.clangd
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,6 @@ Diagnostics:
- readability-function-cognitive-complexity
# It is often nicer to not be explicit
- google-explicit-constructor
CheckOptions:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this option before to avoid a warning we had, but now we don't need to avoid that warning and I added the option incorrectly anyway.

- cppcoreguidelines-special-member-functions.AllowSoleDefaultDtor: True

--- # this divider is necessary
# Disable some checks for Google Test/Bench
Expand Down
4 changes: 2 additions & 2 deletions cpp/src/barretenberg/barretenberg.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@
#include "ecc/curves/grumpkin/grumpkin.hpp"
#include "numeric/random/engine.hpp"
#include "numeric/uint256/uint256.hpp"
#include "plonk/composer/turbo_plonk_composer.hpp"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The follow-up to this PR will include lots of renaming, eg "circuit constructor" becomes "circuit builder" in various places, "composer" becomes "circuit" or "builder" or "circuit_builder" or something in some places, and stays "composer" in other places.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That said, there were certain places where I found it helpful/clarifying to do some renaming in order to get this refactor done, and I leave some of that work in place.

#include "plonk/composer/ultra_plonk_composer.hpp"
#include "proof_system/circuit_constructors/turbo_circuit_constructor.hpp"
#include "proof_system/circuit_constructors/ultra_circuit_constructor.hpp"
#include "plonk/proof_system/types/proof.hpp"
#include "plonk/proof_system/verification_key/verification_key.hpp"
#include "proof_system/types/composer_type.hpp"
Expand Down
94 changes: 49 additions & 45 deletions cpp/src/barretenberg/benchmark/honk_bench/benchmark_utilities.hpp
Original file line number Diff line number Diff line change
@@ -1,11 +1,5 @@
#include "barretenberg/crypto/ecdsa/ecdsa.hpp"
#include "barretenberg/ecc/curves/bn254/fr.hpp"
#include "barretenberg/honk/proof_system/ultra_prover.hpp"
#include "barretenberg/honk/proof_system/ultra_verifier.hpp"
#include <benchmark/benchmark.h>
#include <cstddef>
#include "barretenberg/honk/composer/standard_honk_composer.hpp"
#include "barretenberg/plonk/composer/standard_plonk_composer.hpp"

#include "barretenberg/stdlib/encryption/ecdsa/ecdsa.hpp"
#include "barretenberg/stdlib/hash/keccak/keccak.hpp"
#include "barretenberg/stdlib/primitives/curves/secp256k1.hpp"
Expand Down Expand Up @@ -41,11 +35,13 @@ struct BenchParams {
* @param composer
* @param num_iterations
*/
template <typename Composer> void generate_basic_arithmetic_circuit(Composer& composer, size_t num_gates)
template <typename Builder> void generate_basic_arithmetic_circuit(Builder& builder, size_t num_gates)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it better to be more verbose and use CircuitBuilder, at least for the type name?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I prefer CircuitBuilder but I dont feel strongly. As long as we're not using B..

{
plonk::stdlib::field_t a(plonk::stdlib::witness_t(&composer, barretenberg::fr::random_element()));
plonk::stdlib::field_t b(plonk::stdlib::witness_t(&composer, barretenberg::fr::random_element()));
plonk::stdlib::field_t c(&composer);
proof_system::plonk::stdlib::field_t a(
proof_system::plonk::stdlib::witness_t(&builder, barretenberg::fr::random_element()));
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was some using namespace pollution that let us get away with shortened namespace qualifiers.

proof_system::plonk::stdlib::field_t b(
proof_system::plonk::stdlib::witness_t(&builder, barretenberg::fr::random_element()));
proof_system::plonk::stdlib::field_t c(&builder);
for (size_t i = 0; i < (num_gates / 4) - 4; ++i) {
c = a + b;
c = a * c;
Expand All @@ -57,47 +53,47 @@ template <typename Composer> void generate_basic_arithmetic_circuit(Composer& co
/**
* @brief Generate test circuit with specified number of sha256 hashes
*
* @param composer
* @param builder
* @param num_iterations
*/
template <typename Composer> void generate_sha256_test_circuit(Composer& composer, size_t num_iterations)
template <typename Builder> void generate_sha256_test_circuit(Builder& builder, size_t num_iterations)
{
std::string in;
in.resize(32);
for (size_t i = 0; i < 32; ++i) {
in[i] = 0;
}
proof_system::plonk::stdlib::packed_byte_array<Composer> input(&composer, in);
proof_system::plonk::stdlib::packed_byte_array<Builder> input(&builder, in);
for (size_t i = 0; i < num_iterations; i++) {
input = proof_system::plonk::stdlib::sha256<Composer>(input);
input = proof_system::plonk::stdlib::sha256<Builder>(input);
}
}

/**
* @brief Generate test circuit with specified number of keccak hashes
*
* @param composer
* @param builder
* @param num_iterations
*/
template <typename Composer> void generate_keccak_test_circuit(Composer& composer, size_t num_iterations)
template <typename Builder> void generate_keccak_test_circuit(Builder& builder, size_t num_iterations)
{
std::string in = "abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrstuvwxyz01";

proof_system::plonk::stdlib::byte_array<Composer> input(&composer, in);
proof_system::plonk::stdlib::byte_array<Builder> input(&builder, in);
for (size_t i = 0; i < num_iterations; i++) {
input = proof_system::plonk::stdlib::keccak<Composer>::hash(input);
input = proof_system::plonk::stdlib::keccak<Builder>::hash(input);
}
}

/**
* @brief Generate test circuit with specified number of ecdsa verifications
*
* @param composer
* @param builder
* @param num_iterations
*/
template <typename Composer> void generate_ecdsa_verification_test_circuit(Composer& composer, size_t num_iterations)
template <typename Builder> void generate_ecdsa_verification_test_circuit(Builder& builder, size_t num_iterations)
{
using curve = proof_system::plonk::stdlib::secp256k1<Composer>;
using curve = proof_system::plonk::stdlib::secp256k1<Builder>;
using fr = typename curve::fr;
using fq = typename curve::fq;
using g1 = typename curve::g1;
Expand All @@ -115,22 +111,23 @@ template <typename Composer> void generate_ecdsa_verification_test_circuit(Compo

bool first_result =
crypto::ecdsa::verify_signature<Sha256Hasher, fq, fr, g1>(message_string, account.public_key, signature);
static_cast<void>(first_result); // TODO(Cody): This is not used anywhere.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've started using [[ maybe_unused ]] first_result = ... instead, but I think in this case the entire crypto::ecdsa::verify_signature can be deleted. I think this was debugging code that I forgot to delete.


std::vector<uint8_t> rr(signature.r.begin(), signature.r.end());
std::vector<uint8_t> ss(signature.s.begin(), signature.s.end());
uint8_t vv = signature.v;

typename curve::g1_bigfr_ct public_key = curve::g1_bigfr_ct::from_witness(&composer, account.public_key);
typename curve::g1_bigfr_ct public_key = curve::g1_bigfr_ct::from_witness(&builder, account.public_key);

proof_system::plonk::stdlib::ecdsa::signature<Composer> sig{ typename curve::byte_array_ct(&composer, rr),
typename curve::byte_array_ct(&composer, ss),
proof_system::plonk::stdlib::uint8<Composer>(
&composer, vv) };
proof_system::plonk::stdlib::ecdsa::signature<Builder> sig{ typename curve::byte_array_ct(&builder, rr),
typename curve::byte_array_ct(&builder, ss),
proof_system::plonk::stdlib::uint8<Builder>(
&builder, vv) };

typename curve::byte_array_ct message(&composer, message_string);
typename curve::byte_array_ct message(&builder, message_string);

// Verify ecdsa signature
proof_system::plonk::stdlib::ecdsa::verify_signature<Composer,
proof_system::plonk::stdlib::ecdsa::verify_signature<Builder,
curve,
typename curve::fq_ct,
typename curve::bigfr_ct,
Expand All @@ -141,15 +138,15 @@ template <typename Composer> void generate_ecdsa_verification_test_circuit(Compo
/**
* @brief Generate test circuit with specified number of merkle membership checks
*
* @param composer
* @param builder
* @param num_iterations
*/
template <typename Composer> void generate_merkle_membership_test_circuit(Composer& composer, size_t num_iterations)
template <typename Builder> void generate_merkle_membership_test_circuit(Builder& builder, size_t num_iterations)
{
using namespace proof_system::plonk::stdlib;
using field_ct = field_t<Composer>;
using witness_ct = witness_t<Composer>;
using witness_ct = witness_t<Composer>;
using field_ct = field_t<Builder>;
using witness_ct = witness_t<Builder>;
using witness_ct = witness_t<Builder>;
using MemStore = merkle_tree::MemoryStore;
using MerkleTree_ct = merkle_tree::MerkleTree<MemStore>;

Expand All @@ -163,12 +160,12 @@ template <typename Composer> void generate_merkle_membership_test_circuit(Compos
size_t value = i * 2;
merkle_tree.update_element(idx, value);

field_ct root_ct = witness_ct(&composer, merkle_tree.root());
auto idx_ct = field_ct(witness_ct(&composer, fr(idx))).decompose_into_bits();
field_ct root_ct = witness_ct(&builder, merkle_tree.root());
auto idx_ct = field_ct(witness_ct(&builder, fr(idx))).decompose_into_bits();
auto value_ct = field_ct(value);

merkle_tree::check_membership(
root_ct, merkle_tree::create_witness_hash_path(composer, merkle_tree.get_hash_path(idx)), value_ct, idx_ct);
root_ct, merkle_tree::create_witness_hash_path(builder, merkle_tree.get_hash_path(idx)), value_ct, idx_ct);
}
}

Expand All @@ -177,21 +174,25 @@ template <typename Composer> void generate_merkle_membership_test_circuit(Compos
*
* @details This function assumes state.range refers to num_gates which is the size of the underlying circuit
*
* @tparam Composer
* @tparam Builder
* @param state
* @param test_circuit_function
*/
template <typename Composer>
void construct_proof_with_specified_num_gates(State& state, void (*test_circuit_function)(Composer&, size_t)) noexcept
void construct_proof_with_specified_num_gates(State& state,
void (*test_circuit_function)(typename Composer::CircuitConstructor&,
size_t)) noexcept
{
barretenberg::srs::init_crs_factory("../srs_db/ignition");
auto num_gates = static_cast<size_t>(1 << (size_t)state.range(0));
for (auto _ : state) {
// Constuct circuit and prover; don't include this part in measurement
state.PauseTiming();
auto builder = typename Composer::CircuitConstructor();
test_circuit_function(builder, num_gates);

auto composer = Composer();
test_circuit_function(composer, num_gates);
auto ext_prover = composer.create_prover();
auto ext_prover = composer.create_prover(builder);
state.ResumeTiming();

// Construct proof
Expand All @@ -205,22 +206,25 @@ void construct_proof_with_specified_num_gates(State& state, void (*test_circuit_
* @details This function assumes state.range refers to num_iterations which is the number of times to perform a given
* basic operation in the circuit, e.g. number of hashes
*
* @tparam Composer
* @tparam Builder
* @param state
* @param test_circuit_function
*/
template <typename Composer>
void construct_proof_with_specified_num_iterations(State& state,
void (*test_circuit_function)(Composer&, size_t)) noexcept
void (*test_circuit_function)(typename Composer::CircuitConstructor&,
size_t)) noexcept
{
barretenberg::srs::init_crs_factory("../srs_db/ignition");
auto num_iterations = static_cast<size_t>(state.range(0));
for (auto _ : state) {
// Constuct circuit and prover; don't include this part in measurement
state.PauseTiming();
auto builder = typename Composer::CircuitConstructor();
test_circuit_function(builder, num_iterations);

auto composer = Composer();
test_circuit_function(composer, num_iterations);
auto ext_prover = composer.create_prover();
auto ext_prover = composer.create_prover(builder);
state.ResumeTiming();

// Construct proof
Expand Down
47 changes: 27 additions & 20 deletions cpp/src/barretenberg/benchmark/honk_bench/honk.bench.cpp
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
#include "barretenberg/ecc/curves/bn254/fr.hpp"
#include <benchmark/benchmark.h>
#include <cstddef>
#include "barretenberg/honk/composer/standard_honk_composer.hpp"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renaming and moving comes in the next PR.

#include "barretenberg/honk/composer/composer_helper/standard_honk_composer_helper.hpp"
#include "barretenberg/stdlib/primitives/field/field.hpp"
#include "barretenberg/stdlib/primitives/witness/witness.hpp"

Expand All @@ -10,18 +9,20 @@ using namespace proof_system::plonk::stdlib;

namespace standard_honk_bench {

using Composer = proof_system::honk::StandardHonkComposer;
using Builder = proof_system::StandardCircuitConstructor;
using Composer = proof_system::honk::StandardHonkComposerHelper;

constexpr size_t MIN_LOG_NUM_GATES = 16;
constexpr size_t MAX_LOG_NUM_GATES = 16;
// To get good statistics, number of Repetitions must be sufficient. ~30 Repetitions gives good results.
constexpr size_t NUM_REPETITIONS = 5;

void generate_test_circuit(auto& composer, size_t num_gates)
void generate_test_circuit(auto& builder, size_t num_gates)
{
field_t a(witness_t(&composer, barretenberg::fr::random_element()));
field_t b(witness_t(&composer, barretenberg::fr::random_element()));
field_t c(&composer);
barretenberg::srs::init_crs_factory("../srs_db/ignition");
field_t a(witness_t(&builder, barretenberg::fr::random_element()));
field_t b(witness_t(&builder, barretenberg::fr::random_element()));
field_t c(&builder);
for (size_t i = 0; i < (num_gates / 4) - 4; ++i) {
c = a + b;
c = a * c;
Expand All @@ -38,11 +39,12 @@ void create_prover_standard(State& state) noexcept
for (auto _ : state) {
state.PauseTiming();
auto num_gates = 1 << (size_t)state.range(0);
auto composer = Composer(static_cast<size_t>(num_gates));
generate_test_circuit(composer, static_cast<size_t>(num_gates));
auto builder = Builder(static_cast<size_t>(num_gates));
generate_test_circuit(builder, static_cast<size_t>(num_gates));
state.ResumeTiming();

composer.create_prover();
auto composer = Composer();
composer.create_prover(builder);
}
}
BENCHMARK(create_prover_standard)->DenseRange(MIN_LOG_NUM_GATES, MAX_LOG_NUM_GATES, 1)->Repetitions(NUM_REPETITIONS);
Expand All @@ -55,9 +57,11 @@ void construct_proof_standard(State& state) noexcept
auto num_gates = 1 << (size_t)state.range(0);
for (auto _ : state) {
state.PauseTiming();
auto composer = Composer(static_cast<size_t>(num_gates));
generate_test_circuit(composer, static_cast<size_t>(num_gates));
auto ext_prover = composer.create_prover();
auto builder = Builder(static_cast<size_t>(num_gates));
generate_test_circuit(builder, static_cast<size_t>(num_gates));

auto composer = Composer();
auto ext_prover = composer.create_prover(builder);
state.ResumeTiming();

auto proof = ext_prover.construct_proof();
Expand All @@ -77,11 +81,12 @@ void create_verifier_standard(State& state) noexcept
for (auto _ : state) {
state.PauseTiming();
auto num_gates = 1 << (size_t)state.range(0);
auto composer = Composer(static_cast<size_t>(num_gates));
generate_test_circuit(composer, static_cast<size_t>(num_gates));
auto builder = Builder(static_cast<size_t>(num_gates));
generate_test_circuit(builder, static_cast<size_t>(num_gates));
state.ResumeTiming();

composer.create_verifier();
auto composer = Composer();
composer.create_verifier(builder);
}
}
// BENCHMARK(create_verifier_standard)->DenseRange(MIN_LOG_NUM_GATES, MAX_LOG_NUM_GATES,
Expand All @@ -95,11 +100,13 @@ void verify_proof_standard(State& state) noexcept
for (auto _ : state) {
state.PauseTiming();
auto num_gates = (size_t)state.range(0);
auto composer = Composer(static_cast<size_t>(num_gates));
generate_test_circuit(composer, static_cast<size_t>(num_gates));
auto prover = composer.create_prover();
auto builder = Builder(static_cast<size_t>(num_gates));
generate_test_circuit(builder, static_cast<size_t>(num_gates));

auto composer = Composer();
auto prover = composer.create_prover(builder);
auto proof = prover.construct_proof();
auto verifier = composer.create_verifier();
auto verifier = composer.create_verifier(builder);
state.ResumeTiming();

verifier.verify_proof(proof);
Expand Down
Loading