Account hardener, second class txs (#1537)

* Refactored index.ts to support multiple cli commands/modes. Split out previous logic into migrate_accounts.ts and pulled some logic out of aztec_connect.ts into helper file get_rollup_blocks.ts. Started implementation of fix_accounts.ts for accounts that can have an alias reassigned to their public key

* pulling blocks into worldstate, correct root.

* account fixer now uses SDK, generates keys and aliasHash, constructs AccountTx, performs FFT and gets signing data. Rearranged fixAccounts function

* minor tweak to fix-accounts test script

* For fixAccounts function of account-migrator, refactored the initialization of modules like worker pools, fft, prover, etc to match account_prover.test.ts. This fixed a memory access out-of-bounds error in WASM. Am now computing schnorr signature properly, generating account proof, and verifying account proof

* Updates to fix-accounts-local.sh so it detects when falafel is ready, parses rollup address, runs e2e test and then runs fixAccounts test locally. Can launch full test including ganache, 2 halloumis and a falafel all with a single run of this script

* Began making shift over to Halloumi proofs while investigating why falafel is rejecting proofs generated directly by the account-migrator

* Updated barretenberg rollup_cli and account proofs to accept account txs from halloumi for proof generation. Updated falafel to accept second-class transactions that do not pay a fee (still need to remove them from get_pending_txs and enforce that they only fill up empty space in blocks. fixAccounts is now submitting account txs to halloumi for proof generation before submitting them to falafel for validation and inclusion in rollup blocks. Falafel now has an e2e 'dev' start mode that sets up for e2e but also runs in dev mode for continuous rebuilding

* some cleanup in the account fixer. Started adding code (commented out) to write/read account proofs to/from json files

* Added logic for writing/reading account proofs to json files in batches before sending to falafel. Moved createFundedWalletProvider to utility file as is done for e2e tests. General cleanup in fix_accounts.

* Moved sendSecondClassTxs into a 'limited' rollup provider instantiated in account fixer.

* submitting actual batches of 2nd class txs to falafel

* Moved sendSecondClassTxs into a new class. Moved all account fixing logic into AccountFixer class in account_fixer.ts

* log fix

* Added invertPublicKey wasm bindings. Creating new account first to then be used by many migration TXs for account fixing/hardening

* Fixed account-migrator package deps. Began changes to pull real accounts from mainnet for account hardening/fixing

* broke yarn lock file after merge

* 2nd class txs now only fill up empty slots in rollup block. account fixer force flushes the rollup when in test mode

* rename account fixer to hardener

* added validateAuth for second class txs

* fix account.cpp info text

* rename function to get rollup publish conditions in rollup coordinator. Updated core SDK's pendingAliases to use a Map so that when multiple migration txs are submitted with the same alias, it only updates the SDK with the last tx's alias/acc-public-key pair

* More private helper functions to break up long code blocks in hardener. Added some function header comments.

* fixed falafel bug and broken falafel tests

* separate modes for generation of proofs and submission of proofs

* hardener now hardens 'initial' accounts. Added 'verifyHardened' command

* minor cleanup in sdk and halloumi

* hardener readme and script update

* added pendingSecondClassTxs to status endpoint

* Added falafel-monitoring mode for hardener. Hardener now ensures a unique set of accounts before generating proofs. Other fixes and cleanup to hardener

* hardener cleanup

* debugger working properly

* Added schnorr public key inversion test to barretenberg JS. Added inversion function comments to barretenberg and bJS

* prettier for barretenberg js

* Typos in zk-money and bberg JS (intial -> initial). Split hardener commands into subcommands `harden createHardener` etc. Updated migrator README

* comments and prints cleanup

* revert falafel package.json and start_e2e.sh back to defi-bridge-project version. Commented account hardener and cleaned up harden-command modes (new full-sequence mode that performs all harden steps).

* Tweaks to falafel after code review, bug in rollup_db getTotalTxCount, revert minor/style circuit change. Large updates to Account Hardener after review. Some additional comments and TODOs

* include second class tx count (total and per type) in rollup profile print.

* comment local hardener test script

* fixed bug where second class txs could surpass block-slot limit

* added rollup coordinator tests to check that: 1. rollup does not publish when filled with second class txs, 2. second-class txs are omitted when block is full with 1st class, 3. second-class txs are included when there is empty space and a rollup is going to publish anyway, 3. second class txs are omitted if they would surpass gas/call-data limits

* prettier rollup_coordinator

* compiler warning in rollup coord test

* Hardener now always pulls ALL accounts from ALL blocks and segregates the target accounts for proof generation/verification

* hardener constants updated (batch size and max second-class txs in falafel. Format fix in rolllup profile printer. Added metrics gauge for 2nd-class txs

* Was able to reduce second-class-tx-related changeset in rollup_coordinator.test.ts via use of mockResolvedValueOnce to inject second class txs into rollupDb

* improved hardener script. cleanup in hardener logging

* various yarn package related issues resolved after large merge

* bugs found by ci in barretenberg and falafel after large merge

* type fix in rollup coordinator test

* cleanup for db/account-hardener PR

* revert zkmoney change not appropriate in hardener PR

* minor style fix in account hardener local testing script

* changed terminology: invert -> negate (for negation of public key Y coord)

* fix account hardener after PR #1640

* simplify public key negation

* fixes after merge update

* fix account hardener after Crs->NetCrs rename

src/aztec/crypto/schnorr/c_bind.cpp

@@ -14,6 +14,14 @@ WASM_EXPORT void compute_public_key(uint8_t const* private_key, uint8_t* public_
     write(public_key_buf, pub_key);
 }
 
+WASM_EXPORT void negate_public_key(uint8_t const* public_key_buffer, uint8_t* output)
+{
+    // Negate the public key (effectively negating the y-coordinate of the public key) and return the resulting public
+    // key.
+    auto account_public_key = from_buffer<grumpkin::g1::affine_element>(public_key_buffer);
+    barretenberg::group_elements::write(output, -account_public_key);
+}
+
 WASM_EXPORT void construct_signature(
     uint8_t const* message, size_t msg_len, uint8_t const* private_key, uint8_t* s, uint8_t* e)
 {
@@ -129,4 +137,4 @@ WASM_EXPORT bool multisig_combine_signatures(uint8_t const* message,
         return false;
     }
 }
-}
+}

src/aztec/ecc/curves/secp256k1/secp256k1.test.cpp

@@ -508,9 +508,9 @@ TEST(secp256k1, neg_and_self_neg_0_cmp_regression)
 
 TEST(secp256k1, montgomery_mul_big_bug)
 {
-    secp256k1::fq a(uint256_t{0xfffffffe630dc02f, 0xffffffffffffffff, 0xffffffffffffffff, 0xffffffffffffffff});
+    secp256k1::fq a(uint256_t{ 0xfffffffe630dc02f, 0xffffffffffffffff, 0xffffffffffffffff, 0xffffffffffffffff });
     secp256k1::fq a_sqr = a.sqr();
-    secp256k1::fq expected(uint256_t{0x60381e557e100000, 0x0, 0x0, 0x0});
+    secp256k1::fq expected(uint256_t{ 0x60381e557e100000, 0x0, 0x0, 0x0 });
     EXPECT_EQ((a_sqr == expected), true);
 }
 

src/aztec/ecc/curves/secp256r1/secp256r1.test.cpp

@@ -476,7 +476,7 @@ TEST(secp256r1, montgomery_mul_big_bug)
     a.data[2] = 0xAAAAAAAAAAAAAAAA;
     a.data[3] = 0xFFFFFFFFE38E38E3;
     secp256r1::fr a_sqr = a.sqr();
-    secp256r1::fr expected(uint256_t{0x57abc6aa0349c084, 0x65b21b232a4cb7a5, 0x5ba781948b0fcd6e, 0xd6e9e0644bda12f7});
+    secp256r1::fr expected(uint256_t{ 0x57abc6aa0349c084, 0x65b21b232a4cb7a5, 0x5ba781948b0fcd6e, 0xd6e9e0644bda12f7 });
     EXPECT_EQ((a_sqr == expected), true);
 }
 

src/aztec/rollup/proofs/account/account_tx.cpp

@@ -42,56 +42,6 @@ void account_tx::sign(key_pair<grumpkin::fr, grumpkin::g1> const& keys)
         std::string(message.begin(), message.end()), keys);
 }
 
-void write(std::vector<uint8_t>& buf, account_tx const& tx)
-{
-    using serialize::write;
-    write(buf, tx.merkle_root);
-    write(buf, tx.account_public_key);
-    write(buf, tx.new_account_public_key);
-    write(buf, tx.new_signing_pub_key_1);
-    write(buf, tx.new_signing_pub_key_2);
-    write(buf, tx.alias_hash);
-    write(buf, tx.create);
-    write(buf, tx.migrate);
-    write(buf, tx.account_note_index);
-    write(buf, tx.account_note_path);
-    write(buf, tx.signing_pub_key);
-    write(buf, tx.signature);
-}
-
-void read(uint8_t const*& buf, account_tx& tx)
-{
-    using serialize::read;
-    read(buf, tx.merkle_root);
-    read(buf, tx.account_public_key);
-    read(buf, tx.new_account_public_key);
-    read(buf, tx.new_signing_pub_key_1);
-    read(buf, tx.new_signing_pub_key_2);
-    read(buf, tx.alias_hash);
-    read(buf, tx.create);
-    read(buf, tx.migrate);
-    read(buf, tx.account_note_index);
-    read(buf, tx.account_note_path);
-    read(buf, tx.signing_pub_key);
-    read(buf, tx.signature);
-}
-
-std::ostream& operator<<(std::ostream& os, account_tx const& tx)
-{
-    return os << "merkle_root: " << tx.merkle_root << "\n"
-              << "account_public_key: " << tx.account_public_key << "\n"
-              << "new_account_public_key: " << tx.new_account_public_key << "\n"
-              << "new_signing_pub_key_1: " << tx.new_signing_pub_key_1 << "\n"
-              << "new_signing_pub_key_2: " << tx.new_signing_pub_key_2 << "\n"
-              << "alias_hash: " << tx.alias_hash << "\n"
-              << "create: " << tx.create << "\n"
-              << "migrate: " << tx.migrate << "\n"
-              << "account_note_index: " << tx.account_note_index << "\n"
-              << "account_note_path: " << tx.account_note_path << "\n"
-              << "signing_pub_key: " << tx.signing_pub_key << "\n"
-              << "signature: " << tx.signature << "\n";
-}
-
 } // namespace account
 } // namespace proofs
 } // namespace rollup

src/aztec/rollup/proofs/account/account_tx.hpp

@@ -32,10 +32,57 @@ struct account_tx {
     bool operator==(account_tx const&) const = default;
 };
 
-void read(uint8_t const*& it, account_tx& tx);
-void write(std::vector<uint8_t>& buf, account_tx const& tx);
+template <typename B> inline void read(B& buf, account_tx& tx)
+{
+    using serialize::read;
+    read(buf, tx.merkle_root);
+    read(buf, tx.account_public_key);
+    read(buf, tx.new_account_public_key);
+    read(buf, tx.new_signing_pub_key_1);
+    read(buf, tx.new_signing_pub_key_2);
+    read(buf, tx.alias_hash);
+    read(buf, tx.create);
+    read(buf, tx.migrate);
+    read(buf, tx.account_note_index);
+    read(buf, tx.account_note_path);
+    read(buf, tx.signing_pub_key);
+    read(buf, tx.signature.s);
+    read(buf, tx.signature.e);
+}
 
-std::ostream& operator<<(std::ostream& os, account_tx const& tx);
+template <typename B> inline void write(B& buf, account_tx const& tx)
+{
+    using serialize::write;
+    write(buf, tx.merkle_root);
+    write(buf, tx.account_public_key);
+    write(buf, tx.new_account_public_key);
+    write(buf, tx.new_signing_pub_key_1);
+    write(buf, tx.new_signing_pub_key_2);
+    write(buf, tx.alias_hash);
+    write(buf, tx.create);
+    write(buf, tx.migrate);
+    write(buf, tx.account_note_index);
+    write(buf, tx.account_note_path);
+    write(buf, tx.signing_pub_key);
+    write(buf, tx.signature.s);
+    write(buf, tx.signature.e);
+}
+
+inline std::ostream& operator<<(std::ostream& os, account_tx const& tx)
+{
+    return os << "merkle_root: " << tx.merkle_root << "\n"
+              << "account_public_key: " << tx.account_public_key << "\n"
+              << "new_account_public_key: " << tx.new_account_public_key << "\n"
+              << "new_signing_pub_key_1: " << tx.new_signing_pub_key_1 << "\n"
+              << "new_signing_pub_key_2: " << tx.new_signing_pub_key_2 << "\n"
+              << "alias_hash: " << tx.alias_hash << "\n"
+              << "create: " << tx.create << "\n"
+              << "migrate: " << tx.migrate << "\n"
+              << "account_note_index: " << tx.account_note_index << "\n"
+              << "account_note_path: " << tx.account_note_path << "\n"
+              << "signing_pub_key: " << tx.signing_pub_key << "\n"
+              << "signature: " << tx.signature << "\n";
+}
 
 } // namespace account
 } // namespace proofs

src/aztec/rollup/proofs/account/index.hpp

@@ -4,4 +4,5 @@
 #include "account.hpp"
 #include "c_bind.h"
 #include "compute_circuit_data.hpp"
-#include "create_proof.hpp"
+#include "create_proof.hpp"
+#include "verify.hpp"

src/aztec/rollup/proofs/account/verify.cpp

@@ -0,0 +1,32 @@
+#include "./verify.hpp"
+#include "./account.hpp"
+#include "./account_tx.hpp"
+
+namespace rollup {
+namespace proofs {
+namespace account {
+
+namespace {
+verify_result<Composer> build_circuit(Composer& composer, account_tx& tx, circuit_data const&)
+{
+    verify_result<Composer> result;
+    account_circuit(composer, tx);
+    return result;
+}
+} // namespace
+
+verify_result<Composer> verify_logic(account_tx& tx, circuit_data const& cd)
+{
+    Composer composer = Composer(cd.proving_key, cd.verification_key, cd.num_gates);
+    return verify_logic_internal(composer, tx, cd, "account", build_circuit);
+}
+
+verify_result<Composer> verify(account_tx& tx, circuit_data const& cd)
+{
+    Composer composer = Composer(cd.proving_key, cd.verification_key, cd.num_gates);
+    return verify_internal(composer, tx, cd, "account", true, build_circuit);
+}
+
+} // namespace account
+} // namespace proofs
+} // namespace rollup

src/aztec/rollup/proofs/account/verify.hpp

@@ -0,0 +1,19 @@
+#pragma once
+#include "../verify.hpp"
+#include "./compute_circuit_data.hpp"
+#include "./account.hpp"
+#include <stdlib/types/turbo.hpp>
+
+namespace rollup {
+namespace proofs {
+namespace account {
+
+using namespace plonk::stdlib::types::turbo;
+
+verify_result<Composer> verify_logic(account_tx& tx, circuit_data const& cd);
+
+verify_result<Composer> verify(account_tx& tx, circuit_data const& cd);
+
+} // namespace account
+} // namespace proofs
+} // namespace rollup

src/aztec/rollup/rollup_cli/main.cpp

@@ -6,6 +6,7 @@
 #include <unistd.h>
 
 #include "../proofs/account/compute_circuit_data.hpp"
+#include "../proofs/account/verify.hpp"
 #include "../proofs/join_split/compute_circuit_data.hpp"
 #include "../proofs/claim/get_circuit_data.hpp"
 #include "../proofs/claim/verify.hpp"
@@ -183,6 +184,21 @@ bool create_root_verifier()
     return result.verified;
 }
 
+bool create_account_proof()
+{
+    account::account_tx account_tx;
+    std::cerr << "Reading account tx..." << std::endl;
+    read(std::cin, account_tx);
+
+    auto result = verify(account_tx, account_cd);
+
+    write(std::cout, result.proof_data);
+    write(std::cout, result.verified);
+    std::cout << std::flush;
+
+    return result.verified;
+}
+
 int main(int argc, char** argv)
 {
     std::vector<std::string> args(argv, argv + argc);
@@ -257,6 +273,11 @@ int main(int argc, char** argv)
             create_root_verifier();
             break;
         }
+        case 4: {
+            std::cerr << "Serving request to create account proof..." << std::endl;
+            create_account_proof();
+            break;
+        }
         case 100: {
             // Convert to buffer first, so when we call write we prefix the buffer length.
             std::cerr << "Serving join split vk..." << std::endl;
@@ -282,4 +303,4 @@ int main(int argc, char** argv)
     }
 
     return 0;
-}
+}