Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Execution Trace #4623

Merged
merged 71 commits into from
Feb 21, 2024
Merged
Changes from 1 commit
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
f0310c2
Initial pass at ETM; construct blocks from builder
ledwards2225 Feb 9, 2024
c40162e
WiP method to test new pkey vs old one
ledwards2225 Feb 10, 2024
82790dc
basic pkey polynomials agree
ledwards2225 Feb 11, 2024
31b911c
simplify wire and sel poly construction
ledwards2225 Feb 12, 2024
c982200
perm polys agree
ledwards2225 Feb 12, 2024
86c8901
pkey using ETM in instance constructor is consistent w original
ledwards2225 Feb 12, 2024
2fd68c8
all polys agree for GU flavor
ledwards2225 Feb 12, 2024
3d3702c
todo comments
ledwards2225 Feb 13, 2024
8bcea37
cleanup plonk circuit size stuff
ledwards2225 Feb 13, 2024
1953ce9
simplify plonk circuit size and share sorted list
ledwards2225 Feb 13, 2024
efc4480
share table poly construction
ledwards2225 Feb 13, 2024
1445daa
move more stuff to methods in plonk
ledwards2225 Feb 13, 2024
285340e
initial structure of new plonk create prover
ledwards2225 Feb 13, 2024
cc525fc
try to fix build errors
ledwards2225 Feb 13, 2024
c806c32
another fix
ledwards2225 Feb 13, 2024
7947f08
woops
ledwards2225 Feb 13, 2024
9855ae4
wtf
ledwards2225 Feb 13, 2024
a4aa824
simplify
ledwards2225 Feb 13, 2024
066eca6
fix lookups size bug, with todo
ledwards2225 Feb 14, 2024
0e6320e
looks good with honk specific trace generate
ledwards2225 Feb 14, 2024
9653d02
Merge branch 'master' into lde/etm_poc
ledwards2225 Feb 14, 2024
c0e15fe
new compute prover method works for plonk, no etm yet
ledwards2225 Feb 14, 2024
7408b26
execution trace works for plonk
ledwards2225 Feb 14, 2024
849336d
Merge branch 'master' into lde/etm_poc
ledwards2225 Feb 14, 2024
be5251d
clean up permutation mapping
ledwards2225 Feb 15, 2024
045270c
Merge branch 'master' into lde/etm_poc
ledwards2225 Feb 15, 2024
8e1b322
test utilize trace everywhere
ledwards2225 Feb 15, 2024
10c0527
simplify unique ptr construction
ledwards2225 Feb 15, 2024
dba3961
exec trace for standard honk
ledwards2225 Feb 15, 2024
54e32fc
Merge branch 'master' into lde/exec_trace
ledwards2225 Feb 15, 2024
e011629
delete old lib methods
ledwards2225 Feb 15, 2024
a46bce8
more dead code deletion
ledwards2225 Feb 15, 2024
0546b61
cleanup
ledwards2225 Feb 15, 2024
86c6f2d
cleanup and go back to changing lookup_gates instead of copy
ledwards2225 Feb 15, 2024
2a86ccf
simplify plonk permuation stuff
ledwards2225 Feb 15, 2024
a2c5a87
fix build
ledwards2225 Feb 15, 2024
e8918dc
yay fix zero_idx bug
ledwards2225 Feb 16, 2024
c5a9b1b
its only tests of locations of data in the proof thats failing..
ledwards2225 Feb 16, 2024
b9284c5
disable js test to see that sweet green
ledwards2225 Feb 16, 2024
b1c15ac
class is static
ledwards2225 Feb 16, 2024
436fba2
split exec trace into hpp/cpp
ledwards2225 Feb 16, 2024
a901ce0
Merge branch 'master' into lde/exec_trace
ledwards2225 Feb 16, 2024
999f3fb
one linking option but circular dep
ledwards2225 Feb 16, 2024
3e5fa8e
typename
ledwards2225 Feb 16, 2024
1188ace
share mem read write population method
ledwards2225 Feb 17, 2024
ca4ebbc
reinstate joinsplit tests
ledwards2225 Feb 18, 2024
ee6378a
how about this
ledwards2225 Feb 18, 2024
e31ea7a
remove some special cases
ledwards2225 Feb 18, 2024
c9b21a1
neutral change breaks plonk acir
ledwards2225 Feb 18, 2024
df6b023
clean up prover instance constructor
ledwards2225 Feb 18, 2024
2ba6853
cleanup
ledwards2225 Feb 18, 2024
1b82b35
Merge branch 'master' into lde/exec_trace
ledwards2225 Feb 18, 2024
df1032b
template and resuse plonk prover construction
ledwards2225 Feb 19, 2024
9a46c19
cleanup and comments
ledwards2225 Feb 19, 2024
a87cfda
move pkey construction out of exec trace to simplify dependency
ledwards2225 Feb 19, 2024
d565070
add TODO with old issue
ledwards2225 Feb 19, 2024
edb6230
update some comments and TODOs
ledwards2225 Feb 19, 2024
6777369
constify some stuff
ledwards2225 Feb 19, 2024
5c32774
comments, cleanup and naming
ledwards2225 Feb 19, 2024
2a2a52a
make proper goblin block and simplify trace block construction
ledwards2225 Feb 19, 2024
8580f20
fix pub input flag
ledwards2225 Feb 19, 2024
b846966
clean up arithmetization
ledwards2225 Feb 19, 2024
4e391d6
use block for goblin gates
ledwards2225 Feb 19, 2024
f6fc114
Merge branch 'master' into lde/exec_trace
ledwards2225 Feb 20, 2024
93c8e8e
dont need to pass dyadic size independently
ledwards2225 Feb 20, 2024
5ee1619
Merge branch 'master' into lde/exec_trace
ledwards2225 Feb 20, 2024
7c22c87
add todo with ref to next issue
ledwards2225 Feb 20, 2024
a98ee4e
Merge branch 'master' into lde/exec_trace
ledwards2225 Feb 21, 2024
55439e9
Notes
ledwards2225 Feb 21, 2024
72c33b0
issue to move some stuff inside trace generation
ledwards2225 Feb 21, 2024
c45bf42
more gcc workarounds
ludamad0 Feb 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
remove some special cases
ledwards2225 committed Feb 18, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit e31ea7a8e13592673678a121e41ed59401c8ac77
Original file line number Diff line number Diff line change
@@ -9,7 +9,7 @@ template <class Flavor>
std::shared_ptr<typename Flavor::ProvingKey> ExecutionTrace_<Flavor>::generate(Builder& builder,
size_t dyadic_circuit_size)
{
auto trace_data = generate_trace_polynomials(builder, dyadic_circuit_size);
auto trace_data = construct_trace_polynomials(builder, dyadic_circuit_size);

if constexpr (IsHonkFlavor<Flavor>) {
return generate_honk_proving_key(trace_data, builder, dyadic_circuit_size);
@@ -18,6 +18,46 @@ std::shared_ptr<typename Flavor::ProvingKey> ExecutionTrace_<Flavor>::generate(B
}
}

template <class Flavor>
typename ExecutionTrace_<Flavor>::TraceData ExecutionTrace_<Flavor>::construct_trace_polynomials(
Builder& builder, size_t dyadic_circuit_size)
{
TraceData trace_data{ dyadic_circuit_size, builder };

auto trace_blocks = create_execution_trace_blocks(builder);

uint32_t offset = 0; // Track offset at which to place each block in the trace polynomials
// For each block in the trace, populate wire polys, copy cycles and selector polys
for (auto& block : trace_blocks) {
auto block_size = static_cast<uint32_t>(block.wires[0].size());
info("block size = ", block_size);

// Update wire polynomials and copy cycles
for (uint32_t wire_idx = 0; wire_idx < NUM_WIRES; ++wire_idx) {
for (uint32_t row_idx = 0; row_idx < block_size; ++row_idx) {
uint32_t var_idx = block.wires[wire_idx][row_idx]; // an index into the variables array
uint32_t real_var_idx = builder.real_variable_index[var_idx];
uint32_t trace_row_idx = row_idx + offset;
// Insert the real witness values from this block into the wire polys at the correct offset
trace_data.wires[wire_idx][trace_row_idx] = builder.get_variable(var_idx);
// Add the address of the witness value to its corresponding copy cycle
trace_data.copy_cycles[real_var_idx].emplace_back(cycle_node{ wire_idx, trace_row_idx });
}
}

// Insert the selector values for this block into the selector polynomials at the correct offset
// WORKTODO: comment about coupling of arith and flavor stuff
for (auto [selector_poly, selector] : zip_view(trace_data.selectors, block.selectors.get())) {
for (size_t row_idx = 0; row_idx < block_size; ++row_idx) {
selector_poly[row_idx + offset] = selector[row_idx];
}
}

offset += block_size;
}
return trace_data;
}

template <class Flavor>
std::shared_ptr<typename Flavor::ProvingKey> ExecutionTrace_<Flavor>::generate_honk_proving_key(
TraceData& trace_data, Builder& builder, size_t dyadic_circuit_size)
@@ -31,9 +71,7 @@ std::shared_ptr<typename Flavor::ProvingKey> ExecutionTrace_<Flavor>::generate_h
for (auto [pkey_selector, trace_selector] : zip_view(proving_key->get_selectors(), trace_data.selectors)) {
pkey_selector = std::move(trace_selector);
}
if constexpr (IsGoblinFlavor<Flavor>) {
proving_key->lagrange_ecc_op = std::move(trace_data.ecc_op_selector);
}

compute_permutation_argument_polynomials<Flavor>(builder, proving_key.get(), trace_data.copy_cycles);

return proving_key;
@@ -65,62 +103,6 @@ std::shared_ptr<typename Flavor::ProvingKey> ExecutionTrace_<Flavor>::generate_p
return proving_key;
}

template <class Flavor>
typename ExecutionTrace_<Flavor>::TraceData ExecutionTrace_<Flavor>::generate_trace_polynomials(
Builder& builder, size_t dyadic_circuit_size)
{
TraceData trace_data{ dyadic_circuit_size, builder };

auto trace_blocks = create_execution_trace_blocks(builder);

uint32_t offset = 0; // Track offset at which to place each block in the trace polynomials
// For each block in the trace, populate wire polys, copy cycles and selector polys
for (auto& block : trace_blocks) {
auto block_size = static_cast<uint32_t>(block.wires[0].size());
info("block size = ", block_size);

// Update wire polynomials and copy cycles
// WORKTODO: order of row/column loops is arbitrary but needs to be row/column to match old copy_cycle code
for (uint32_t row_idx = 0; row_idx < block_size; ++row_idx) {
for (uint32_t wire_idx = 0; wire_idx < NUM_WIRES; ++wire_idx) {
uint32_t var_idx = block.wires[wire_idx][row_idx]; // an index into the variables array
uint32_t real_var_idx = builder.real_variable_index[var_idx];
// Insert the real witness values from this block into the wire polys at the correct offset
trace_data.wires[wire_idx][row_idx + offset] = builder.get_variable(var_idx);
// Add the address of the witness value to its corresponding copy cycle
// WORKTODO: Not adding cycles for wires 3 and 4 here is only needed in order to maintain
// consistency with old version. We can remove this special case and the result is simply that all
// the zeros in wires 3 and 4 over the PI range are copy constrained together.
if (!(block.is_public_input && wire_idx > 1)) {
trace_data.copy_cycles[real_var_idx].emplace_back(cycle_node{ wire_idx, row_idx + offset });
}
}
}

// Insert the selector values for this block into the selector polynomials at the correct offset
// WORKTODO: comment about coupling of arith and flavor stuff
for (auto [selector_poly, selector] : zip_view(trace_data.selectors, block.selectors.get())) {
for (size_t row_idx = 0; row_idx < block_size; ++row_idx) {
selector_poly[row_idx + offset] = selector[row_idx];
}
}

// WORKTODO: this can go away if we just let the goblin op selector be a normal selector. Actually this
// would be a good test case for the concept of gate blocks.
if constexpr (IsGoblinFlavor<Flavor>) {
if (block.is_goblin_op) {
trace_data.ecc_op_selector = Polynomial{ dyadic_circuit_size };
for (size_t row_idx = 0; row_idx < block_size; ++row_idx) {
trace_data.ecc_op_selector[row_idx + offset] = 1;
}
}
}

offset += block_size;
}
return trace_data;
}

template <class Flavor>
std::vector<typename ExecutionTrace_<Flavor>::TraceBlock> ExecutionTrace_<Flavor>::create_execution_trace_blocks(
Builder& builder)
@@ -145,7 +127,7 @@ std::vector<typename ExecutionTrace_<Flavor>::TraceBlock> ExecutionTrace_<Flavor
Selectors ecc_op_selectors;
// Note: there is no selector for ecc ops
ecc_op_selectors.reserve_and_zero(builder.num_ecc_op_gates);
TraceBlock ecc_op_block{ ecc_op_wires, ecc_op_selectors, /*is_public_input=*/false, /*is_goblin_op=*/true };
TraceBlock ecc_op_block{ ecc_op_wires, ecc_op_selectors };
trace_blocks.emplace_back(ecc_op_block);
}

@@ -155,14 +137,14 @@ std::vector<typename ExecutionTrace_<Flavor>::TraceBlock> ExecutionTrace_<Flavor
public_input_selectors.reserve_and_zero(builder.public_inputs.size());
for (auto& idx : builder.public_inputs) {
for (size_t wire_idx = 0; wire_idx < NUM_WIRES; ++wire_idx) {
if (wire_idx < 2) { // first two wires get a copy of the PI
if (wire_idx < 2) { // first two wires get a copy of the public inputs
public_input_wires[wire_idx].emplace_back(idx);
} else { // remaining wires get zeros
} else { // the remaining wires get zeros
public_input_wires[wire_idx].emplace_back(builder.zero_idx);
}
}
}
TraceBlock public_input_block{ public_input_wires, public_input_selectors, /*is_public_input=*/true };
TraceBlock public_input_block{ public_input_wires, public_input_selectors };
trace_blocks.emplace_back(public_input_block);

// Make a block for the basic wires and selectors
Original file line number Diff line number Diff line change
@@ -14,8 +14,6 @@ template <class Arithmetization> struct ExecutionTraceBlock {
using Wires = std::array<std::vector<uint32_t, bb::ContainerSlabAllocator<uint32_t>>, Arithmetization::NUM_WIRES>;
Wires wires;
Arithmetization selectors;
bool is_public_input = false;
bool is_goblin_op = false;
};

template <class Flavor> class ExecutionTrace_ {
@@ -34,7 +32,6 @@ template <class Flavor> class ExecutionTrace_ {
std::array<Polynomial, NUM_WIRES> wires;
std::array<Polynomial, Builder::Selectors::NUM_SELECTORS> selectors;
std::vector<CyclicPermutation> copy_cycles;
Polynomial ecc_op_selector;

TraceData(size_t dyadic_circuit_size, Builder& builder)
{
@@ -47,7 +44,7 @@ template <class Flavor> class ExecutionTrace_ {
}
// Initialize the vector of copy cycles; these are simply collections of indices into the wire polynomials
// whose values are copy constrained to be equal. Each variable represents one cycle.
copy_cycles.resize(builder.variables.size());
copy_cycles.resize(builder.variables.size()); // WORKTODO: real_variables.size()?
}
};

@@ -64,7 +61,7 @@ template <class Flavor> class ExecutionTrace_ {
size_t dyadic_circuit_size)
requires IsPlonkFlavor<Flavor>;

static TraceData generate_trace_polynomials(Builder& builder, size_t dyadic_circuit_size);
static TraceData construct_trace_polynomials(Builder& builder, size_t dyadic_circuit_size);

/**
* @brief Temporary helper method to construct execution trace blocks from existing builder structures
Original file line number Diff line number Diff line change
@@ -47,10 +47,11 @@ template <class Flavor> void ProverInstance_<Flavor>::compute_circuit_size_param
*/
template <class Flavor> void ProverInstance_<Flavor>::construct_ecc_op_wire_polynomials(auto& wire_polynomials)
{
std::array<polynomial, Flavor::NUM_WIRES> op_wire_polynomials;
std::array<Polynomial, Flavor::NUM_WIRES> op_wire_polynomials;
for (auto& poly : op_wire_polynomials) {
poly = static_cast<polynomial>(dyadic_circuit_size);
poly = Polynomial{ dyadic_circuit_size };
}
Polynomial ecc_op_selector{ dyadic_circuit_size };

// The ECC op wires are constructed to contain the op data on the appropriate range and to vanish everywhere else.
// The op data is assumed to have already been stored at the correct location in the convetional wires so the data
@@ -60,13 +61,15 @@ template <class Flavor> void ProverInstance_<Flavor>::construct_ecc_op_wire_poly
for (size_t i = 0; i < num_ecc_op_gates; ++i) {
size_t idx = i + op_wire_offset;
op_wire_polynomials[poly_idx][idx] = wire_polynomials[poly_idx][idx];
ecc_op_selector[idx] = 1;
}
}

proving_key->ecc_op_wire_1 = op_wire_polynomials[0].share();
proving_key->ecc_op_wire_2 = op_wire_polynomials[1].share();
proving_key->ecc_op_wire_3 = op_wire_polynomials[2].share();
proving_key->ecc_op_wire_4 = op_wire_polynomials[3].share();
proving_key->lagrange_ecc_op = ecc_op_selector.share();
}

/**