Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(avm): kernel output opcodes #6416

Merged
merged 80 commits into from
May 28, 2024
Merged
Changes from 1 commit
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
8853132
avm_logderivative
Maddiaa0 Apr 3, 2024
abc3cba
temp: chall line up
Maddiaa0 Apr 4, 2024
8b88dcd
fix: degree too low for lookup relations
Maddiaa0 Apr 5, 2024
c8d9601
chore: rename validate trace proof to check circuit, make another met…
Maddiaa0 Apr 5, 2024
c695905
chore: remove dangling code
Maddiaa0 Apr 5, 2024
40eeb8f
Merge branch 'master' into md/04-03-avm_logderivative
Maddiaa0 Apr 5, 2024
619175b
chore: further cleanup
Maddiaa0 Apr 5, 2024
65c3159
chore: from powdr codegen
Maddiaa0 Apr 8, 2024
61d10f6
temp
Maddiaa0 Apr 11, 2024
7e84550
feat: bb support for public input columns
Maddiaa0 Apr 11, 2024
c054aca
Merge branch 'master' into md/04-11-feat_bb_support_for_public_input_…
Maddiaa0 Apr 11, 2024
f38773f
merge fixy
Maddiaa0 Apr 11, 2024
a4262dc
chore: test structure
Maddiaa0 Apr 11, 2024
cf4bb86
🧹
Maddiaa0 Apr 11, 2024
12a9789
use pilgen
Maddiaa0 Apr 12, 2024
14d15cc
Merge branch 'master' into md/04-11-feat_bb_support_for_public_input_…
Maddiaa0 Apr 12, 2024
b0f8041
feat: example caller and address opcode
Maddiaa0 Apr 12, 2024
d5813a2
feat: generalise builder, move after review
Maddiaa0 Apr 15, 2024
6265c61
Merge branch 'master' into md/04-12-feat_example_caller_and_address_o…
Maddiaa0 Apr 17, 2024
59b221c
fix: add tests for all call context opcodes
Maddiaa0 Apr 18, 2024
b613cf5
Merge branch 'master' into md/04-11-feat_bb_support_for_public_input_…
Maddiaa0 Apr 18, 2024
81a2500
Merge branch 'md/04-11-feat_bb_support_for_public_input_columns' into…
Maddiaa0 Apr 18, 2024
c934c61
chore: update pil comments
Maddiaa0 Apr 18, 2024
a8ceaf8
fix: remove redundant comment
Maddiaa0 Apr 18, 2024
af343db
fix: some negative tests
Maddiaa0 Apr 21, 2024
9388ecc
temp
Maddiaa0 Apr 29, 2024
9e7aad0
Merge branch 'master' into md/04-12-feat_example_caller_and_address_o…
Maddiaa0 May 7, 2024
859a3df
chore: remove l1 gas - no longer exists
Maddiaa0 May 7, 2024
b4a47b3
chore: rearrange where relations live, based on review
Maddiaa0 May 7, 2024
79cf849
Merge branch 'master' into md/04-11-feat_bb_support_for_public_input_…
Maddiaa0 May 7, 2024
faac5b1
chore: update codegen
Maddiaa0 May 7, 2024
7e70e7d
Merge branch 'md/04-11-feat_bb_support_for_public_input_columns' into…
Maddiaa0 May 7, 2024
4bb42b8
feat: add cpp constants gen to calculate kernel offsets
Maddiaa0 May 8, 2024
bccfd67
feat: add global variable opcodes
Maddiaa0 May 8, 2024
2a74110
fix: transaction fee selector test
Maddiaa0 May 8, 2024
1285ccf
fix: update tags to line up with kernel
Maddiaa0 May 8, 2024
60836ca
Merge branch 'master' into md/04-12-feat_example_caller_and_address_o…
Maddiaa0 May 8, 2024
e2bcca8
feat: parallelise check circuit relations
Maddiaa0 May 8, 2024
7bff191
fix: forge fmt after constant gen
Maddiaa0 May 8, 2024
115f17c
fix: no futures for wasm builds
Maddiaa0 May 8, 2024
de663f1
feat: fix segfaults in parallel circuit checking
Maddiaa0 May 9, 2024
c910000
fix: missed log
Maddiaa0 May 9, 2024
fd728c7
fix: incorrect kernel tests
Maddiaa0 May 9, 2024
66f958e
Merge branch 'master' into md/04-11-feat_bb_support_for_public_input_…
Maddiaa0 May 9, 2024
ecdf742
🧹
Maddiaa0 May 9, 2024
3f31357
Merge branch 'md/04-11-feat_bb_support_for_public_input_columns' into…
Maddiaa0 May 9, 2024
d6aeda8
feat: use codegen, vector public inputs - pubs now misaligned
Maddiaa0 May 9, 2024
ea48aca
fix: update codegen
Maddiaa0 May 9, 2024
e1e2cb8
fix: remove legacy codegen files
Maddiaa0 May 10, 2024
3be1242
Merge branch 'master' into md/04-12-feat_example_caller_and_address_o…
Maddiaa0 May 10, 2024
bee84aa
fix: relation tags + removal of function opcode
Maddiaa0 May 10, 2024
449f3da
fmt fix
Maddiaa0 May 10, 2024
a6470d7
kernel_outputs
Maddiaa0 May 10, 2024
94269ae
feat: codegen cols
Maddiaa0 May 10, 2024
3d6633b
feat: first implementation of emit note hash
Maddiaa0 May 14, 2024
25e31bd
feat: add emit nullfier, l2 to l1 msg, unencrypted log
Maddiaa0 May 14, 2024
963dbf6
fix: mixup of emit l2 to l1 msg, l1 to l2 exists
Maddiaa0 May 15, 2024
d465a04
Merge branch 'master' into md/05-10-kernel_outputs
Maddiaa0 May 15, 2024
c45c666
feat: sload and sstore
Maddiaa0 May 15, 2024
036910e
merge master
Maddiaa0 May 15, 2024
9774d95
Merge branch 'master' into md/05-10-kernel_outputs
Maddiaa0 May 15, 2024
c1c4f21
fix: alter exists opcodes to all use fields
Maddiaa0 May 16, 2024
c88d5ef
fix: different in and out tags for exists opcodes
Maddiaa0 May 16, 2024
c5847b2
feat: use pil generated public input columns
Maddiaa0 May 16, 2024
70a6e86
Merge branch 'master' into md/05-10-kernel_outputs
Maddiaa0 May 16, 2024
f8e8349
fix: dirty merge
Maddiaa0 May 16, 2024
289c302
fix: annotate todos
Maddiaa0 May 16, 2024
17640ef
Merge branch 'master' into md/05-10-kernel_outputs
Maddiaa0 May 16, 2024
7e120ad
fix: incorrect offset in emitNoteHash test
Maddiaa0 May 17, 2024
0dc2c58
Merge branch 'master' into md/05-10-kernel_outputs
Maddiaa0 May 17, 2024
6b2bfd6
fix: review
Maddiaa0 May 23, 2024
3e85832
Merge branch 'master' into md/05-10-kernel_outputs
Maddiaa0 May 23, 2024
1922ccb
fix: add more constants to constant gen
Maddiaa0 May 23, 2024
9dca2e2
fix: add call_ptrs
Maddiaa0 May 23, 2024
f9cf428
fmt
Maddiaa0 May 23, 2024
cb1f006
Merge branch 'master' into md/05-10-kernel_outputs
Maddiaa0 May 27, 2024
1bdb64f
fix: share public inputs construction in executor
Maddiaa0 May 27, 2024
17c216b
fix: typo
Maddiaa0 May 27, 2024
c714d7f
Merge branch 'master' into md/05-10-kernel_outputs
Maddiaa0 May 27, 2024
b95b838
Merge branch 'master' into md/05-10-kernel_outputs
Maddiaa0 May 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: bb support for public input columns
  • Loading branch information
Maddiaa0 committed Apr 11, 2024
commit 7e84550641d609ff166b2b4dde2522c636b09763
8 changes: 5 additions & 3 deletions barretenberg/cpp/pil/spike/spike.pil
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@

namespace Spike(1);
namespace Spike(16);

pol constant y;
pol constant first = [1] + [0]*;
pol commit x;
pol public kernel_inputs;
pol public kernel_inputs;

x - first = 0;
Original file line number Diff line number Diff line change
@@ -2,5 +2,6 @@
#define Spike_DECLARE_VIEWS(index) \
using Accumulator = typename std::tuple_element<index, ContainerOverSubrelations>::type; \
using View = typename Accumulator::View; \
[[maybe_unused]] auto Spike_first = View(new_term.Spike_first); \
[[maybe_unused]] auto Spike_kernel_inputs__is_public = View(new_term.Spike_kernel_inputs__is_public); \
[[maybe_unused]] auto Spike_x = View(new_term.Spike_x);
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@

#pragma once
#include "../../relation_parameters.hpp"
#include "../../relation_types.hpp"
#include "./declare_views.hpp"

namespace bb::Spike_vm {

template <typename FF> struct SpikeRow {
FF Spike_first{};
FF Spike_x{};
};

inline std::string get_relation_label_spike(int index)
{
switch (index) {}
return std::to_string(index);
}

template <typename FF_> class spikeImpl {
public:
using FF = FF_;

static constexpr std::array<size_t, 1> SUBRELATION_PARTIAL_LENGTHS{
2,
};

template <typename ContainerOverSubrelations, typename AllEntities>
void static accumulate(ContainerOverSubrelations& evals,
const AllEntities& new_term,
[[maybe_unused]] const RelationParameters<FF>&,
[[maybe_unused]] const FF& scaling_factor)
{

// Contribution 0
{
Spike_DECLARE_VIEWS(0);

auto tmp = (Spike_x - Spike_first);
tmp *= scaling_factor;
std::get<0>(evals) += tmp;
}
}
};

template <typename FF> using spike = Relation<spikeImpl<FF>>;

} // namespace bb::Spike_vm
Original file line number Diff line number Diff line change
@@ -11,12 +11,13 @@
#include "barretenberg/relations/generic_permutation/generic_permutation_relation.hpp"
#include "barretenberg/stdlib_circuit_builders/circuit_builder_base.hpp"

#include "barretenberg/relations/generated/spike/spike.hpp"
#include "barretenberg/vm/generated/spike_flavor.hpp"

namespace bb {

template <typename FF> struct SpikeFullRow {
FF Spike_first;
FF Spike_first{};
FF Spike_kernel_inputs__is_public{};
FF Spike_x{};
};
@@ -31,8 +32,8 @@ class SpikeCircuitBuilder {
using Polynomial = Flavor::Polynomial;
using ProverPolynomials = Flavor::ProverPolynomials;

static constexpr size_t num_fixed_columns = 2;
static constexpr size_t num_polys = 2;
static constexpr size_t num_fixed_columns = 3;
static constexpr size_t num_polys = 3;
std::vector<Row> rows;

void set_trace(std::vector<Row>&& trace) { rows = std::move(trace); }
@@ -58,9 +59,40 @@ class SpikeCircuitBuilder {

[[maybe_unused]] bool check_circuit()
{
// There are no relations, so check circuit does nothing
// auto polys = compute_polynomials();
// const size_t num_rows = polys.get_polynomial_size();

auto polys = compute_polynomials();
const size_t num_rows = polys.get_polynomial_size();

const auto evaluate_relation = [&]<typename Relation>(const std::string& relation_name,
std::string (*debug_label)(int)) {
typename Relation::SumcheckArrayOfValuesOverSubrelations result;
for (auto& r : result) {
r = 0;
}
constexpr size_t NUM_SUBRELATIONS = result.size();

for (size_t i = 0; i < num_rows; ++i) {
Relation::accumulate(result, polys.get_row(i), {}, 1);

bool x = true;
for (size_t j = 0; j < NUM_SUBRELATIONS; ++j) {
if (result[j] != 0) {
std::string row_name = debug_label(static_cast<int>(j));
throw_or_abort(
format("Relation ", relation_name, ", subrelation index ", row_name, " failed at row ", i));
x = false;
}
}
if (!x) {
return false;
}
}
return true;
};

if (!evaluate_relation.template operator()<Spike_vm::spike<FF>>("spike", Spike_vm::get_relation_label_spike)) {
return false;
}

return true;
}
22 changes: 5 additions & 17 deletions barretenberg/cpp/src/barretenberg/vm/generated/spike_flavor.hpp
Original file line number Diff line number Diff line change
@@ -13,8 +13,7 @@
#include "barretenberg/flavor/flavor_macros.hpp"
#include "barretenberg/polynomials/evaluation_domain.hpp"
#include "barretenberg/polynomials/polynomial.hpp"
// TODO: had to include below to make it compile
#include "barretenberg/relations/relation_parameters.hpp"
#include "barretenberg/relations/generated/spike/spike.hpp"
#include "barretenberg/transcript/transcript.hpp"

namespace bb {
@@ -35,17 +34,14 @@ class SpikeFlavor {
using VerifierCommitmentKey = bb::VerifierCommitmentKey<Curve>;
using RelationSeparator = FF;

// NEW!
using PublicInputColumns = std::map<std::string, std::vector<FF>>;

static constexpr size_t NUM_PRECOMPUTED_ENTITIES = 0;
static constexpr size_t NUM_PRECOMPUTED_ENTITIES = 1;
static constexpr size_t NUM_WITNESS_ENTITIES = 2;
static constexpr size_t NUM_WIRES = NUM_WITNESS_ENTITIES + NUM_PRECOMPUTED_ENTITIES;
// We have two copies of the witness entities, so we subtract the number of fixed ones (they have no shift), one for
// the unshifted and one for the shifted
static constexpr size_t NUM_ALL_ENTITIES = 2;
static constexpr size_t NUM_ALL_ENTITIES = 3;

using Relations = std::tuple<>;
using Relations = std::tuple<Spike_vm::spike<FF>>;

static constexpr size_t MAX_PARTIAL_RELATION_LENGTH = compute_max_partial_relation_length<Relations>();

@@ -70,7 +66,7 @@ class SpikeFlavor {

DEFINE_FLAVOR_MEMBERS(DataType, Spike_first)

RefVector<DataType> get_selectors() { return {}; };
RefVector<DataType> get_selectors() { return { Spike_first }; };
RefVector<DataType> get_sigma_polynomials() { return {}; };
RefVector<DataType> get_id_polynomials() { return {}; };
RefVector<DataType> get_table_polynomials() { return {}; };
@@ -101,14 +97,6 @@ class SpikeFlavor {
using Base::Base;

RefVector<DataType> get_to_be_shifted() { return {}; };

void compute_logderivative_inverses(const RelationParameters<FF>& relation_parameters)
{
ProverPolynomials prover_polynomials = ProverPolynomials(*this);
// TODO: also required this when there were no inverses
// Maybe codegen should be selective over which is created
void(relation_parameters.beta);
}
};

using VerificationKey = VerificationKey_<PrecomputedEntities<Commitment>, VerifierCommitmentKey>;
36 changes: 6 additions & 30 deletions barretenberg/cpp/src/barretenberg/vm/generated/spike_prover.cpp
Original file line number Diff line number Diff line change
@@ -44,21 +44,11 @@ SpikeProver::SpikeProver(std::shared_ptr<Flavor::ProvingKey> input_key,
* @brief Add circuit size, public input size, and public inputs to transcript
*
*/
// void SpikeProver::execute_preamble_round(SpikeProver::PublicInputColumns public_inputs)
// TODO(md): as we are sending the commitment to the verifier, we do NOT need to send the public inputs to the verifier
// one by one. We only do so in this case in preparation for when we stop sending the commitment
void SpikeProver::execute_preamble_round(std::vector<FF> public_inputs)
void SpikeProver::execute_preamble_round()
{
const auto circuit_size = static_cast<uint32_t>(key->circuit_size);

transcript->send_to_verifier("circuit_size", circuit_size);

// send all of the public inputs to the prover - this wont work
for (size_t i = 0; i < public_inputs.size(); ++i) {
auto public_input_i = public_inputs[i];
// TODO:
transcript->send_to_verifier("public_input_" + std::to_string(i), public_input_i);
}
}

/**
@@ -74,24 +64,12 @@ void SpikeProver::execute_wire_commitments_round()
witness_commitments.Spike_x = commitment_key->commit(key->Spike_x);

// Send all commitments to the verifier
// transcript->send_to_verifier(commitment_labels.Spike_kernel_inputs__is_public,
// witness_commitments.Spike_kernel_inputs__is_public);
transcript->send_to_verifier(commitment_labels.Spike_kernel_inputs__is_public,
witness_commitments.Spike_kernel_inputs__is_public);
transcript->send_to_verifier(commitment_labels.Spike_x, witness_commitments.Spike_x);
}

void SpikeProver::execute_log_derivative_inverse_round()
{

auto [beta, gamm] = transcript->template get_challenges<FF>("beta", "gamma");
relation_parameters.beta = beta;
relation_parameters.gamma = gamm;

key->compute_logderivative_inverses(relation_parameters);

// Commit to all logderivative inverse polynomials

// Send all commitments to the verifier
}
void SpikeProver::execute_log_derivative_inverse_round() {}

/**
* @brief Run Sumcheck resulting in u = (u_1,...,u_d) challenges and all evaluations at u being calculated.
@@ -134,17 +112,15 @@ HonkProof& SpikeProver::export_proof()
return proof;
}

// TODO: maybe the public inputs can be sent in the circuit builder?
HonkProof& SpikeProver::construct_proof(std::vector<FF> public_inputs)
HonkProof& SpikeProver::construct_proof()
{
// Add circuit size public input size and public inputs to transcript.
execute_preamble_round(public_inputs);
execute_preamble_round();

// Compute wire commitments
execute_wire_commitments_round();

// Compute sorted list accumulator and commitment
execute_log_derivative_inverse_round();

// Fiat-Shamir: alpha
// Run sumcheck subprotocol.
Original file line number Diff line number Diff line change
@@ -14,6 +14,7 @@ namespace bb {
class SpikeProver {

using Flavor = SpikeFlavor;
using FF = Flavor::FF;
using PCS = Flavor::PCS;
using PCSCommitmentKey = Flavor::CommitmentKey;
using ProvingKey = Flavor::ProvingKey;
@@ -23,19 +24,16 @@ class SpikeProver {
using Transcript = Flavor::Transcript;

public:
using FF = Flavor::FF;
using PublicInputColumns = Flavor::PublicInputColumns;

explicit SpikeProver(std::shared_ptr<ProvingKey> input_key, std::shared_ptr<PCSCommitmentKey> commitment_key);

void execute_preamble_round(std::vector<FF> public_inputs);
void execute_preamble_round();
void execute_wire_commitments_round();
void execute_log_derivative_inverse_round();
void execute_relation_check_rounds();
void execute_zeromorph_rounds();

HonkProof& export_proof();
HonkProof& construct_proof(std::vector<FF> public_inputs);
HonkProof& construct_proof();

std::shared_ptr<Transcript> transcript = std::make_shared<Transcript>();

Original file line number Diff line number Diff line change
@@ -32,9 +32,9 @@ using FF = SpikeVerifier::FF;
// This challenge is generated during the sumcheck protocol
//
// TODO: Using the current evaluate_mle implementation is inefficient
FF evaluate_public_input_column(std::vector<FF> evals, std::vector<FF> challenges)
FF evaluate_public_input_column(std::vector<FF> points, std::vector<FF> challenges)
{
Polynomial<FF> polynomial(evals);
Polynomial<FF> polynomial(points);
return polynomial.evaluate_mle(challenges);
}

@@ -71,10 +71,6 @@ bool SpikeVerifier::verify_proof(const HonkProof& proof, std::vector<FF> public_
transcript->template receive_from_prover<Commitment>(commitment_labels.Spike_kernel_inputs__is_public);
commitments.Spike_x = transcript->template receive_from_prover<Commitment>(commitment_labels.Spike_x);

auto [beta, gamm] = transcript->template get_challenges<FF>("beta", "gamma");
relation_parameters.beta = beta;
relation_parameters.gamma = gamm;

// Get commitments to inverses

// Execute Sumcheck Verifier
@@ -105,10 +101,10 @@ bool SpikeVerifier::verify_proof(const HonkProof& proof, std::vector<FF> public_
// at the same challenge point.
// TODO: document this design choice in a github issue

info("sumcheck claimed evaluations", claimed_evaluations);

FF public_column_evaluation = evaluate_public_input_column(public_inputs, multivariate_challenge);
info("public column evaluation: ", public_column_evaluation);
if (public_column_evaluation != claimed_evaluations.Spike_kernel_inputs__is_public) {
return false;
}

// Execute ZeroMorph rounds. See https://hackmd.io/dlf9xEwhTQyE3hiGbq4FsA?view for a complete description of the
// unrolled protocol.
Original file line number Diff line number Diff line change
@@ -15,7 +15,7 @@ class SpikeVerifier {

public:
using FF = Flavor::FF;
using PublicInputColumns = Flavor::PublicInputColumns;
// using PublicInputColumns = Flavor::PublicInputColumns;

explicit SpikeVerifier(std::shared_ptr<VerificationKey> verifier_key = nullptr);
SpikeVerifier(SpikeVerifier&& other) noexcept;
28 changes: 24 additions & 4 deletions barretenberg/cpp/src/barretenberg/vm/tests/spike.test.cpp
Original file line number Diff line number Diff line change
@@ -16,22 +16,33 @@ namespace {
auto& engine = numeric::get_debug_randomness();
}

class SpikeVerifierColumnsCircuitBuilderTests : public ::testing::Test {
protected:
// TODO(640): The Standard Honk on Grumpkin test suite fails unless the SRS is initialised for every test.
void SetUp() override { srs::init_crs_factory("../srs_db/ignition"); };
};

// Test file for testing public inputs evaluations are the same in the verifier and in sumcheck

TEST(SpikeVerifierColumnsCircuitBuilder, VerificationSuccess)
TEST(SpikeVerifierColumnsCircuitBuilderTests, VerificationSuccess)
{
// using FF = SpikeFlavor::FF;
using FF = SpikeFlavor::FF;
using Builder = SpikeCircuitBuilder;
using Row = Builder::Row;
Builder circuit_builder;

srs::init_crs_factory("../srs_db/ignition");

const size_t circuit_size = 16;
std::vector<Row> rows;
std::vector<FF> public_inputs;

// Add to the public input column that is increasing
for (size_t i = 0; i < circuit_size; i++) {
// Make sure the external and trace public inputs are the same
Row row{ .Spike_kernel_inputs__is_public = i };
rows.push_back(row);
public_inputs.emplace_back(i);
}

circuit_builder.set_trace(std::move(rows));
@@ -42,7 +53,16 @@ TEST(SpikeVerifierColumnsCircuitBuilder, VerificationSuccess)
HonkProof proof = prover.construct_proof();

auto verifier = composer.create_verifier(circuit_builder);
bool verified = verifier.verify_proof(proof);
bool verified = verifier.verify_proof(proof, public_inputs);

ASSERT_TRUE(verified);

info("verified: ", verified);
// Assert that the same proof fails with different public inputs
std::vector<FF> failure_public_inputs;
for (size_t i = 0; i < circuit_size; i++) {
// Make sure the external and trace public inputs are NOT same
failure_public_inputs.emplace_back(i + 1);
}
bool verification_failure = verifier.verify_proof(proof, failure_public_inputs);
ASSERT_FALSE(verification_failure);
}