Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: unencrypted log filtering #2600

Merged
merged 60 commits into from
Oct 12, 2023
Merged

feat: unencrypted log filtering #2600

merged 60 commits into from
Oct 12, 2023

Conversation

benesjan
Copy link
Contributor

@benesjan benesjan commented Sep 29, 2023

Fixes #1498
Fixes #1500

There is a plenty of filtering options now:
image

Checklist:

Remove the checklist to signal you've completed it. Enable auto-merge if the PR is ready to merge.

  • If the pull request requires a cryptography review (e.g. cryptographic algorithm implementations) I have added the 'crypto' tag.
  • I have reviewed my diff in github, line by line and removed unexpected formatting changes, testing logs, or commented-out code.
  • Every change is related to the PR description.
  • I have linked this pull request to relevant issues (if any exist).

@benesjan benesjan marked this pull request as draft September 29, 2023 16:45
@benesjan benesjan force-pushed the janb/log-filtering branch 8 times, most recently from 03e00d8 to 9964f20 Compare October 5, 2023 10:38
@benesjan benesjan marked this pull request as ready for review October 5, 2023 11:01
@benesjan benesjan force-pushed the janb/log-filtering branch from 9964f20 to de353cb Compare October 5, 2023 12:23
@benesjan benesjan requested a review from spalladino October 5, 2023 18:02
@benesjan benesjan force-pushed the janb/log-filtering branch from de353cb to 9fe305f Compare October 6, 2023 07:13
Copy link
Collaborator

@spalladino spalladino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great @benesjan! I left a few comments.

docs/docs/dev_docs/contracts/syntax/events.md Outdated Show resolved Hide resolved
yarn-project/aztec.js/src/contract/sent_tx.ts Outdated Show resolved Hide resolved
yarn-project/cli/src/index.ts Outdated Show resolved Hide resolved
yarn-project/cli/src/index.ts Show resolved Hide resolved
yarn-project/cli/src/utils.ts Outdated Show resolved Hide resolved
yarn-project/types/src/logs/tx_l2_logs.ts Outdated Show resolved Hide resolved
yarn-project/archiver/src/archiver/archiver_store.ts Outdated Show resolved Hide resolved
yarn-project/cli/src/index.ts Outdated Show resolved Hide resolved
yarn-project/cli/package.json Outdated Show resolved Hide resolved
@benesjan benesjan force-pushed the janb/log-filtering branch from dcc234d to 02e7c2c Compare October 9, 2023 08:46
@benesjan benesjan force-pushed the janb/log-filtering branch from 92221e4 to d791b0b Compare October 9, 2023 10:16
@benesjan benesjan requested a review from PhilWindle October 9, 2023 15:58
yarn-project/cli/src/index.ts Outdated Show resolved Hide resolved
yarn-project/cli/tsconfig.json Outdated Show resolved Hide resolved
yarn-project/types/src/logs/log_id.ts Outdated Show resolved Hide resolved
yarn-project/types/src/logs/log_filter.ts Outdated Show resolved Hide resolved
@benesjan benesjan force-pushed the janb/log-filtering branch 5 times, most recently from c09e88b to 4326479 Compare October 11, 2023 13:50
@AztecBot
Copy link
Collaborator

AztecBot commented Oct 11, 2023

Benchmark results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Values are compared against data from master at commit b30839e9 and shown if the difference exceeds 1%.

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric 8 txs 32 txs 128 txs
l1_rollup_calldata_size_in_bytes 45,444 179,588 716,132
l1_rollup_calldata_gas 222,972 868,040 3,446,384
l1_rollup_execution_gas 842,059 3,595,148 22,201,753
l2_block_processing_time_in_ms 1,044 (+3%) 3,949 (+1%) 15,562 (+1%)
note_successful_decrypting_time_in_ms 324 (-1%) 1,030 (+4%) 3,771 (+2%)
note_trial_decrypting_time_in_ms 26.0 (-10%) ⚠️ 108 (+13%) 139 (+1%)
l2_block_building_time_in_ms 9,017 36,000 150,970 (+1%)
l2_block_rollup_simulation_time_in_ms 6,692 26,674 (-1%) 106,059
l2_block_public_tx_process_time_in_ms 2,285 9,197 (+1%) 44,095 (+1%)

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric 5 blocks 10 blocks
node_history_sync_time_in_ms 14,652 (-10%) 31,785 (-10%)
note_history_successful_decrypting_time_in_ms ⚠️ 2,355 (-20%) ⚠️ 4,774 (-22%)
note_history_trial_decrypting_time_in_ms 123 (-2%) 147 (-7%)
node_database_size_in_bytes 1,652,083 1,197,411
pxe_database_size_in_bytes 27,188 54,187

Circuits stats

Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit circuit_simulation_time_in_ms circuit_input_size_in_bytes circuit_output_size_in_bytes
private-kernel-init 44.0 (-9%) 56,577 14,745
private-kernel-ordering 22.0 (-9%) 20,137 8,089
base-rollup 852 (-10%) 631,605 811
root-rollup 38.0 (-10%) 4,072 1,097
private-kernel-inner 37.7 (-8%) 72,288 14,745
public-kernel-private-input 47.5 (-9%) 37,359 14,745
public-kernel-non-first-iteration 28.5 (-9%) 37,401 14,745
merge-rollup ⚠️ 0.978 (-15%) 2,592 873

@benesjan benesjan enabled auto-merge (squash) October 11, 2023 14:58
@benesjan benesjan merged commit 7ae554a into master Oct 12, 2023
@benesjan benesjan deleted the janb/log-filtering branch October 12, 2023 08:09
PhilWindle pushed a commit that referenced this pull request Oct 13, 2023
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-packages: 0.8.11</summary>

##
[0.8.11](aztec-packages-v0.8.10...aztec-packages-v0.8.11)
(2023-10-13)


### Features

* **archiver:** Use registry to fetch searchStartBlock
([#2830](#2830))
([e5bc067](e5bc067))
* Configure sandbox for network
([#2818](#2818))
([d393a59](d393a59))
* **docker-sandbox:** Allow forks in sandbox
([#2831](#2831))
([ed8431c](ed8431c)),
closes
[#2726](#2726)
* Goblin Translator Decomposition relation (Goblin Translator part 4)
([#2802](#2802))
([3c3cd9f](3c3cd9f))
* Goblin Translator GenPermSort relation (Goblin Translator part 3)
([#2795](#2795))
([b36fdc4](b36fdc4))
* Goblin translator opcode constraint and accumulator transfer relations
(Goblin Translator part 5)
([#2805](#2805))
([b3d1f28](b3d1f28))
* Goblin Translator Permutation relation (Goblin Translator part 2)
([#2790](#2790))
([9a354c9](9a354c9))
* Integrate ZeroMorph into Honk
([#2774](#2774))
([ea86869](ea86869))
* Purge non native token + reorder params in token portal
([#2723](#2723))
([447dade](447dade))
* Throw compile error if read/write public state from private
([#2804](#2804))
([a3649df](a3649df))
* Unencrypted log filtering
([#2600](#2600))
([7ae554a](7ae554a)),
closes
[#1498](#1498)
[#1500](#1500)
* Update goblin translator circuit builder (Goblin Translator part 1)
([#2764](#2764))
([32c69ae](32c69ae))


### Bug Fixes

* Outdated `noir:clean`
([#2821](#2821))
([2ea199f](2ea199f))


### Miscellaneous

* Benchmark tx sizes in p2p pool
([#2810](#2810))
([f63219c](f63219c))
* Change acir_tests branch to point to master
([#2815](#2815))
([73f229d](73f229d))
* Fix typo
([#2839](#2839))
([5afdf91](5afdf91))
* From &lt; genesis allowed in getBlocks
([#2816](#2816))
([5622b50](5622b50))
* Remove Ultra Grumpkin flavor
([#2825](#2825))
([bde77b8](bde77b8))
* Remove work queue from honk
([#2814](#2814))
([bca7d12](bca7d12))
* Spell check
([#2817](#2817))
([4777a11](4777a11))


### Documentation

* Slight changes to update portal page
([#2799](#2799))
([eb65819](eb65819))
* Update aztec_connect_sunset.mdx
([#2808](#2808))
([5f659a7](5f659a7))
</details>

<details><summary>barretenberg.js: 0.8.11</summary>

##
[0.8.11](barretenberg.js-v0.8.10...barretenberg.js-v0.8.11)
(2023-10-13)


### Miscellaneous

* **barretenberg.js:** Synchronize aztec-packages versions
</details>

<details><summary>barretenberg: 0.8.11</summary>

##
[0.8.11](barretenberg-v0.8.10...barretenberg-v0.8.11)
(2023-10-13)


### Features

* Goblin Translator Decomposition relation (Goblin Translator part 4)
([#2802](#2802))
([3c3cd9f](3c3cd9f))
* Goblin Translator GenPermSort relation (Goblin Translator part 3)
([#2795](#2795))
([b36fdc4](b36fdc4))
* Goblin translator opcode constraint and accumulator transfer relations
(Goblin Translator part 5)
([#2805](#2805))
([b3d1f28](b3d1f28))
* Goblin Translator Permutation relation (Goblin Translator part 2)
([#2790](#2790))
([9a354c9](9a354c9))
* Integrate ZeroMorph into Honk
([#2774](#2774))
([ea86869](ea86869))
* Update goblin translator circuit builder (Goblin Translator part 1)
([#2764](#2764))
([32c69ae](32c69ae))


### Miscellaneous

* Change acir_tests branch to point to master
([#2815](#2815))
([73f229d](73f229d))
* Remove Ultra Grumpkin flavor
([#2825](#2825))
([bde77b8](bde77b8))
* Remove work queue from honk
([#2814](#2814))
([bca7d12](bca7d12))
* Spell check
([#2817](#2817))
([4777a11](4777a11))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
AztecBot added a commit to AztecProtocol/barretenberg that referenced this pull request Oct 14, 2023
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-packages: 0.8.11</summary>

##
[0.8.11](AztecProtocol/aztec-packages@aztec-packages-v0.8.10...aztec-packages-v0.8.11)
(2023-10-13)


### Features

* **archiver:** Use registry to fetch searchStartBlock
([#2830](AztecProtocol/aztec-packages#2830))
([e5bc067](AztecProtocol/aztec-packages@e5bc067))
* Configure sandbox for network
([#2818](AztecProtocol/aztec-packages#2818))
([d393a59](AztecProtocol/aztec-packages@d393a59))
* **docker-sandbox:** Allow forks in sandbox
([#2831](AztecProtocol/aztec-packages#2831))
([ed8431c](AztecProtocol/aztec-packages@ed8431c)),
closes
[#2726](AztecProtocol/aztec-packages#2726)
* Goblin Translator Decomposition relation (Goblin Translator part 4)
([#2802](AztecProtocol/aztec-packages#2802))
([3c3cd9f](AztecProtocol/aztec-packages@3c3cd9f))
* Goblin Translator GenPermSort relation (Goblin Translator part 3)
([#2795](AztecProtocol/aztec-packages#2795))
([b36fdc4](AztecProtocol/aztec-packages@b36fdc4))
* Goblin translator opcode constraint and accumulator transfer relations
(Goblin Translator part 5)
([#2805](AztecProtocol/aztec-packages#2805))
([b3d1f28](AztecProtocol/aztec-packages@b3d1f28))
* Goblin Translator Permutation relation (Goblin Translator part 2)
([#2790](AztecProtocol/aztec-packages#2790))
([9a354c9](AztecProtocol/aztec-packages@9a354c9))
* Integrate ZeroMorph into Honk
([#2774](AztecProtocol/aztec-packages#2774))
([ea86869](AztecProtocol/aztec-packages@ea86869))
* Purge non native token + reorder params in token portal
([#2723](AztecProtocol/aztec-packages#2723))
([447dade](AztecProtocol/aztec-packages@447dade))
* Throw compile error if read/write public state from private
([#2804](AztecProtocol/aztec-packages#2804))
([a3649df](AztecProtocol/aztec-packages@a3649df))
* Unencrypted log filtering
([#2600](AztecProtocol/aztec-packages#2600))
([7ae554a](AztecProtocol/aztec-packages@7ae554a)),
closes
[#1498](AztecProtocol/aztec-packages#1498)
[#1500](AztecProtocol/aztec-packages#1500)
* Update goblin translator circuit builder (Goblin Translator part 1)
([#2764](AztecProtocol/aztec-packages#2764))
([32c69ae](AztecProtocol/aztec-packages@32c69ae))


### Bug Fixes

* Outdated `noir:clean`
([#2821](AztecProtocol/aztec-packages#2821))
([2ea199f](AztecProtocol/aztec-packages@2ea199f))


### Miscellaneous

* Benchmark tx sizes in p2p pool
([#2810](AztecProtocol/aztec-packages#2810))
([f63219c](AztecProtocol/aztec-packages@f63219c))
* Change acir_tests branch to point to master
([#2815](AztecProtocol/aztec-packages#2815))
([73f229d](AztecProtocol/aztec-packages@73f229d))
* Fix typo
([#2839](AztecProtocol/aztec-packages#2839))
([5afdf91](AztecProtocol/aztec-packages@5afdf91))
* From &lt; genesis allowed in getBlocks
([#2816](AztecProtocol/aztec-packages#2816))
([5622b50](AztecProtocol/aztec-packages@5622b50))
* Remove Ultra Grumpkin flavor
([#2825](AztecProtocol/aztec-packages#2825))
([bde77b8](AztecProtocol/aztec-packages@bde77b8))
* Remove work queue from honk
([#2814](AztecProtocol/aztec-packages#2814))
([bca7d12](AztecProtocol/aztec-packages@bca7d12))
* Spell check
([#2817](AztecProtocol/aztec-packages#2817))
([4777a11](AztecProtocol/aztec-packages@4777a11))


### Documentation

* Slight changes to update portal page
([#2799](AztecProtocol/aztec-packages#2799))
([eb65819](AztecProtocol/aztec-packages@eb65819))
* Update aztec_connect_sunset.mdx
([#2808](AztecProtocol/aztec-packages#2808))
([5f659a7](AztecProtocol/aztec-packages@5f659a7))
</details>

<details><summary>barretenberg.js: 0.8.11</summary>

##
[0.8.11](AztecProtocol/aztec-packages@barretenberg.js-v0.8.10...barretenberg.js-v0.8.11)
(2023-10-13)


### Miscellaneous

* **barretenberg.js:** Synchronize aztec-packages versions
</details>

<details><summary>barretenberg: 0.8.11</summary>

##
[0.8.11](AztecProtocol/aztec-packages@barretenberg-v0.8.10...barretenberg-v0.8.11)
(2023-10-13)


### Features

* Goblin Translator Decomposition relation (Goblin Translator part 4)
([#2802](AztecProtocol/aztec-packages#2802))
([3c3cd9f](AztecProtocol/aztec-packages@3c3cd9f))
* Goblin Translator GenPermSort relation (Goblin Translator part 3)
([#2795](AztecProtocol/aztec-packages#2795))
([b36fdc4](AztecProtocol/aztec-packages@b36fdc4))
* Goblin translator opcode constraint and accumulator transfer relations
(Goblin Translator part 5)
([#2805](AztecProtocol/aztec-packages#2805))
([b3d1f28](AztecProtocol/aztec-packages@b3d1f28))
* Goblin Translator Permutation relation (Goblin Translator part 2)
([#2790](AztecProtocol/aztec-packages#2790))
([9a354c9](AztecProtocol/aztec-packages@9a354c9))
* Integrate ZeroMorph into Honk
([#2774](AztecProtocol/aztec-packages#2774))
([ea86869](AztecProtocol/aztec-packages@ea86869))
* Update goblin translator circuit builder (Goblin Translator part 1)
([#2764](AztecProtocol/aztec-packages#2764))
([32c69ae](AztecProtocol/aztec-packages@32c69ae))


### Miscellaneous

* Change acir_tests branch to point to master
([#2815](AztecProtocol/aztec-packages#2815))
([73f229d](AztecProtocol/aztec-packages@73f229d))
* Remove Ultra Grumpkin flavor
([#2825](AztecProtocol/aztec-packages#2825))
([bde77b8](AztecProtocol/aztec-packages@bde77b8))
* Remove work queue from honk
([#2814](AztecProtocol/aztec-packages#2814))
([bca7d12](AztecProtocol/aztec-packages@bca7d12))
* Spell check
([#2817](AztecProtocol/aztec-packages#2817))
([4777a11](AztecProtocol/aztec-packages@4777a11))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
4 participants