Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: improve PXE contract DB capabilities #11303

Merged
merged 2 commits into from
Jan 17, 2025
Merged

feat: improve PXE contract DB capabilities #11303

merged 2 commits into from
Jan 17, 2025

Conversation

nventuro
Copy link
Contributor

This extends the features introduced in #10867 so that we can also delete entries and copy multiple entries from one place to another. With these two new primitives I also built a Noir DBArray, which is a dynamic array backed by this database supporting pushes and deletion of arbitrary indexes (which we'll need for #10724).

I took the liberty of renaming store and load to dbStore and dbLoad in TS, since I found the original names to be too generic. I kept those in Noir since we call them as pxe_db::store etc. anyway, and don't really expose them much either. I also renamed key to slot, since the actual kv store does have a key, but it's not what we now call the slot (the key is ${contract}:${slot}).

I found it slightly annoying that I had to modify so many TS files to get this working, perhaps a symptom of over-abstraction?

@nventuro nventuro requested review from Thunkar and benesjan January 17, 2025 12:51
@AztecProtocol AztecProtocol deleted a comment from github-actions bot Jan 17, 2025
@AztecProtocol AztecProtocol deleted a comment from github-actions bot Jan 17, 2025
@nventuro nventuro enabled auto-merge (squash) January 17, 2025 13:50
@nventuro nventuro disabled auto-merge January 17, 2025 13:50
Thunkar
Thunkar approved these changes Jan 17, 2025
View reviewed changes
Copy link
Contributor

@Thunkar Thunkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@nventuro nventuro enabled auto-merge (squash) January 17, 2025 14:05
@github-actions GitHub Actions
Copy link
Contributor

Changes to public function bytecode sizes

Generated at commit: 8e618296f04e743b1dfb2df12dac1b40aae34f5d, compared to commit: 77828365f35a6ac6068c7ac87601758264e0c2eb

🧾 Summary (100% most significant diffs)

Program Bytecode size in bytes (+/-) %
AvmTest::sha256_hash +420 ❌ +12.50%
AvmTest::bulk_testing +420 ❌ +1.85%
AvmTest::public_dispatch +420 ❌ +0.68%
Full diff report 👇
Program Bytecode size in bytes (+/-) %
AvmTest::sha256_hash 3,780 (+420) +12.50%
AvmTest::bulk_testing 23,096 (+420) +1.85%
AvmTest::public_dispatch 62,548 (+420) +0.68%

@github-actions GitHub Actions
Copy link
Contributor

Changes to circuit sizes

Generated at commit: 8e618296f04e743b1dfb2df12dac1b40aae34f5d, compared to commit: 77828365f35a6ac6068c7ac87601758264e0c2eb

🧾 Summary (100% most significant diffs)

Program ACIR opcodes (+/-) % Circuit size (+/-) %
private_kernel_tail_to_public +2 ❌ +0.02% +2 ❌ +0.01%
private_kernel_init +2 ❌ +0.01% +2 ❌ +0.01%
private_kernel_inner +2 ❌ +0.01% +2 ❌ +0.00%
Full diff report 👇
Program ACIR opcodes (+/-) % Circuit size (+/-) %
private_kernel_tail_to_public 12,447 (+2) +0.02% 37,944 (+2) +0.01%
private_kernel_init 18,681 (+2) +0.01% 39,848 (+2) +0.01%
private_kernel_inner 28,370 (+2) +0.01% 72,236 (+2) +0.00%

@nventuro nventuro merged commit fab5570 into master Jan 17, 2025
51 checks passed
@nventuro nventuro deleted the nv/pxe-db-improv branch January 17, 2025 14:50
@AztecBot AztecBot mentioned this pull request Jan 17, 2025
Merged
rahul-kothari pushed a commit that referenced this pull request Jan 17, 2025
@AztecBot
chore(master): Release 0.71.0 (#11247)
232c34e 
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-package: 0.71.0</summary>

##
[0.71.0](aztec-package-v0.70.0...aztec-package-v0.71.0)
(2025-01-17)


### Features

* Track block building helpers
([#11190](#11190))
([a749dc1](a749dc1)),
closes
[#11184](#11184)
</details>

<details><summary>barretenberg.js: 0.71.0</summary>

##
[0.71.0](barretenberg.js-v0.70.0...barretenberg.js-v0.71.0)
(2025-01-17)


### Miscellaneous

* **barretenberg.js:** Synchronize aztec-packages versions
</details>

<details><summary>aztec-packages: 0.71.0</summary>

##
[0.71.0](aztec-packages-v0.70.0...aztec-packages-v0.71.0)
(2025-01-17)


### ⚠ BREAKING CHANGES

* `loop` statements (only frontend)
(noir-lang/noir#7092)
* Include kind in `StructDefinition::generics` and fix derivation of Eq
in structs with numeric generics
(noir-lang/noir#7076)
* Attestation collection times out based on sequencer timetable
([#11248](#11248))

### Features

* `loop` statements (only frontend)
(noir-lang/noir#7092)
([a964cd0](a964cd0))
* Add `ConstrainNotEqual` instruction
(noir-lang/noir#7032)
([a964cd0](a964cd0))
* Archive public testnet tx data
([#11192](#11192))
([66f2014](66f2014))
* Backup proof failures to google cloud storage
([#11255](#11255))
([b4775fd](b4775fd)),
closes
[#11062](#11062)
* **docs:** Algolia-&gt;typesense
([#11034](#11034))
([d254f49](d254f49))
* Improve PXE contract DB capabilities
([#11303](#11303))
([fab5570](fab5570))
* **LSP:** Auto-import trait reexport if trait is not visible
(noir-lang/noir#7079)
([a964cd0](a964cd0))
* Process note logs in aztec-nr
([#10651](#10651))
([708139d](708139d))
* Reenable constrained config for roots
([#10605](#10605))
([a6ebc2e](a6ebc2e))
* **spartan:** Add extra accounts
([#11300](#11300))
([7782836](7782836))
* **ssa:** Treat globals as constant in a function's DFG
(noir-lang/noir#7040)
([a964cd0](a964cd0))
* Track block building helpers
([#11190](#11190))
([a749dc1](a749dc1)),
closes
[#11184](#11184)


### Bug Fixes

* Allow implicit associated types on integer type kinds
(noir-lang/noir#7078)
([a964cd0](a964cd0))
* Do not remove memory blocks used as brillig input
(noir-lang/noir#7073)
([a964cd0](a964cd0))
* Ensure 'docker info' works before preceding
([#11286](#11286))
([0b0e81a](0b0e81a))
* Fail in proxy deployment should fail the step
([#11308](#11308))
([b780d75](b780d75))
* Faster polling times for archiver and sequencer
([#11262](#11262))
([d70511e](d70511e))
* Https://github.com/AztecProtocol/aztec-packages/issues/8939
([66f2014](66f2014))
* Idempotent deploy-l1-contracts with initial validators
([#11284](#11284))
([3a3f9c0](3a3f9c0)),
closes
[#11283](#11283)
* Include kind in `StructDefinition::generics` and fix derivation of Eq
in structs with numeric generics
(noir-lang/noir#7076)
([a964cd0](a964cd0))
* Legacy runner start
([#11291](#11291))
([0b2a619](0b2a619))
* Reallocate commitment key to avoid pippenger error
([#11249](#11249))
([8fc2011](8fc2011))
* References to a3 in docs
([#11256](#11256))
([caf88fa](caf88fa))
* Remove bb path override in cli-wallet
([#11280](#11280))
([a6a226e](a6a226e))
* Resolve misc bugs handling phases in avm witgen
([#11218](#11218))
([29bc4bd](29bc4bd))
* Sequencer timetable accounts for spare time
([#11221](#11221))
([f1b9211](f1b9211))
* Validator ignores block limits during reexec
([#11288](#11288))
([920a521](920a521))


### Miscellaneous

* Add circuit input checks to bootstrap.sh
([#11261](#11261))
([a718b15](a718b15))
* Add regression test for
[#6530](#6530)
(noir-lang/noir#7089)
([a964cd0](a964cd0))
* Add test for isuee
[#7090](#7090)
(noir-lang/noir#7091)
([a964cd0](a964cd0))
* Allow passing custom conditions to inlining pass
(noir-lang/noir#7083)
([a964cd0](a964cd0))
* Attestation collection times out based on sequencer timetable
([#11248](#11248))
([946a418](946a418))
* **avm:** Calldata, returndata slices out of range padded with zeros
([#11265](#11265))
([a469c94](a469c94)),
closes
[#10933](#10933)
* Delete external-ci-approved.yml
([#11258](#11258))
([642bce6](642bce6))
* Demote error closing forks to warn
([#11263](#11263))
([a5b7a6a](a5b7a6a))
* Do not make new instruction if it hasn't changed
(noir-lang/noir#7069)
([a964cd0](a964cd0))
* Ensure devnet has unproven config
([#11302](#11302))
([085f782](085f782))
* Fixing `[@safety](https://github.com/safety)` warnings
([#11094](#11094))
([5de24e0](5de24e0))
* Log correlation in traces in google cloud
([#11276](#11276))
([fbcc8ef](fbcc8ef)),
closes
[#11019](#11019)
[#10937](#10937)
* Mark `noir-edwards` as expected to compile
(noir-lang/noir#7085)
([a964cd0](a964cd0))
* Move shared pcs functionality to internal library in solidity and
small refactorings in sumcheck
([#11230](#11230))
([507ae9d](507ae9d))
* Reduce the number of provers in rc-1
([#11296](#11296))
([92e40ff](92e40ff))
* Remove references to padding txs
([#11264](#11264))
([32408f6](32408f6))
* Remove warnings from types and rollup lib crates
([#11269](#11269))
([9f389a7](9f389a7))
* Replace relative paths to noir-protocol-circuits
([8ece166](8ece166))
* Replace relative paths to noir-protocol-circuits
([be42305](be42305))
* Retry deploys
([#11252](#11252))
([23cfbb4](23cfbb4))
* Set failed proof store for spartan deployments
([#11282](#11282))
([f787a52](f787a52))
* Silence "Updated proven chain" log
([#11250](#11250))
([44bd79b](44bd79b))
* Silence circuit return values in CI
([#11259](#11259))
([db3d860](db3d860))
* Stable masternet images
([#11289](#11289))
([07fabe8](07fabe8))
</details>

<details><summary>barretenberg: 0.71.0</summary>

##
[0.71.0](barretenberg-v0.70.0...barretenberg-v0.71.0)
(2025-01-17)


### Bug Fixes

* Reallocate commitment key to avoid pippenger error
([#11249](#11249))
([8fc2011](8fc2011))
* Resolve misc bugs handling phases in avm witgen
([#11218](#11218))
([29bc4bd](29bc4bd))


### Miscellaneous

* **avm:** Calldata, returndata slices out of range padded with zeros
([#11265](#11265))
([a469c94](a469c94)),
closes
[#10933](#10933)
* Move shared pcs functionality to internal library in solidity and
small refactorings in sumcheck
([#11230](#11230))
([507ae9d](507ae9d))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
AztecBot added a commit to AztecProtocol/barretenberg that referenced this pull request Jan 18, 2025
@AztecBot
chore(master): Release 0.71.0 (#11247)
87bd4f2 
🤖 I have created a release *beep* *boop*
---


<details><summary>aztec-package: 0.71.0</summary>

##
[0.71.0](AztecProtocol/aztec-packages@aztec-package-v0.70.0...aztec-package-v0.71.0)
(2025-01-17)


### Features

* Track block building helpers
([#11190](AztecProtocol/aztec-packages#11190))
([a749dc1](AztecProtocol/aztec-packages@a749dc1)),
closes
[#11184](AztecProtocol/aztec-packages#11184)
</details>

<details><summary>barretenberg.js: 0.71.0</summary>

##
[0.71.0](AztecProtocol/aztec-packages@barretenberg.js-v0.70.0...barretenberg.js-v0.71.0)
(2025-01-17)


### Miscellaneous

* **barretenberg.js:** Synchronize aztec-packages versions
</details>

<details><summary>aztec-packages: 0.71.0</summary>

##
[0.71.0](AztecProtocol/aztec-packages@aztec-packages-v0.70.0...aztec-packages-v0.71.0)
(2025-01-17)


### ⚠ BREAKING CHANGES

* `loop` statements (only frontend)
(noir-lang/noir#7092)
* Include kind in `StructDefinition::generics` and fix derivation of Eq
in structs with numeric generics
(noir-lang/noir#7076)
* Attestation collection times out based on sequencer timetable
([#11248](AztecProtocol/aztec-packages#11248))

### Features

* `loop` statements (only frontend)
(noir-lang/noir#7092)
([a964cd0](AztecProtocol/aztec-packages@a964cd0))
* Add `ConstrainNotEqual` instruction
(noir-lang/noir#7032)
([a964cd0](AztecProtocol/aztec-packages@a964cd0))
* Archive public testnet tx data
([#11192](AztecProtocol/aztec-packages#11192))
([66f2014](AztecProtocol/aztec-packages@66f2014))
* Backup proof failures to google cloud storage
([#11255](AztecProtocol/aztec-packages#11255))
([b4775fd](AztecProtocol/aztec-packages@b4775fd)),
closes
[#11062](AztecProtocol/aztec-packages#11062)
* **docs:** Algolia-&gt;typesense
([#11034](AztecProtocol/aztec-packages#11034))
([d254f49](AztecProtocol/aztec-packages@d254f49))
* Improve PXE contract DB capabilities
([#11303](AztecProtocol/aztec-packages#11303))
([fab5570](AztecProtocol/aztec-packages@fab5570))
* **LSP:** Auto-import trait reexport if trait is not visible
(noir-lang/noir#7079)
([a964cd0](AztecProtocol/aztec-packages@a964cd0))
* Process note logs in aztec-nr
([#10651](AztecProtocol/aztec-packages#10651))
([708139d](AztecProtocol/aztec-packages@708139d))
* Reenable constrained config for roots
([#10605](AztecProtocol/aztec-packages#10605))
([a6ebc2e](AztecProtocol/aztec-packages@a6ebc2e))
* **spartan:** Add extra accounts
([#11300](AztecProtocol/aztec-packages#11300))
([7782836](AztecProtocol/aztec-packages@7782836))
* **ssa:** Treat globals as constant in a function's DFG
(noir-lang/noir#7040)
([a964cd0](AztecProtocol/aztec-packages@a964cd0))
* Track block building helpers
([#11190](AztecProtocol/aztec-packages#11190))
([a749dc1](AztecProtocol/aztec-packages@a749dc1)),
closes
[#11184](AztecProtocol/aztec-packages#11184)


### Bug Fixes

* Allow implicit associated types on integer type kinds
(noir-lang/noir#7078)
([a964cd0](AztecProtocol/aztec-packages@a964cd0))
* Do not remove memory blocks used as brillig input
(noir-lang/noir#7073)
([a964cd0](AztecProtocol/aztec-packages@a964cd0))
* Ensure 'docker info' works before preceding
([#11286](AztecProtocol/aztec-packages#11286))
([0b0e81a](AztecProtocol/aztec-packages@0b0e81a))
* Fail in proxy deployment should fail the step
([#11308](AztecProtocol/aztec-packages#11308))
([b780d75](AztecProtocol/aztec-packages@b780d75))
* Faster polling times for archiver and sequencer
([#11262](AztecProtocol/aztec-packages#11262))
([d70511e](AztecProtocol/aztec-packages@d70511e))
* Https://github.com/AztecProtocol/aztec-packages/issues/8939
([66f2014](AztecProtocol/aztec-packages@66f2014))
* Idempotent deploy-l1-contracts with initial validators
([#11284](AztecProtocol/aztec-packages#11284))
([3a3f9c0](AztecProtocol/aztec-packages@3a3f9c0)),
closes
[#11283](AztecProtocol/aztec-packages#11283)
* Include kind in `StructDefinition::generics` and fix derivation of Eq
in structs with numeric generics
(noir-lang/noir#7076)
([a964cd0](AztecProtocol/aztec-packages@a964cd0))
* Legacy runner start
([#11291](AztecProtocol/aztec-packages#11291))
([0b2a619](AztecProtocol/aztec-packages@0b2a619))
* Reallocate commitment key to avoid pippenger error
([#11249](AztecProtocol/aztec-packages#11249))
([8fc2011](AztecProtocol/aztec-packages@8fc2011))
* References to a3 in docs
([#11256](AztecProtocol/aztec-packages#11256))
([caf88fa](AztecProtocol/aztec-packages@caf88fa))
* Remove bb path override in cli-wallet
([#11280](AztecProtocol/aztec-packages#11280))
([a6a226e](AztecProtocol/aztec-packages@a6a226e))
* Resolve misc bugs handling phases in avm witgen
([#11218](AztecProtocol/aztec-packages#11218))
([29bc4bd](AztecProtocol/aztec-packages@29bc4bd))
* Sequencer timetable accounts for spare time
([#11221](AztecProtocol/aztec-packages#11221))
([f1b9211](AztecProtocol/aztec-packages@f1b9211))
* Validator ignores block limits during reexec
([#11288](AztecProtocol/aztec-packages#11288))
([920a521](AztecProtocol/aztec-packages@920a521))


### Miscellaneous

* Add circuit input checks to bootstrap.sh
([#11261](AztecProtocol/aztec-packages#11261))
([a718b15](AztecProtocol/aztec-packages@a718b15))
* Add regression test for
[#6530](AztecProtocol/aztec-packages#6530)
(noir-lang/noir#7089)
([a964cd0](AztecProtocol/aztec-packages@a964cd0))
* Add test for isuee
[#7090](AztecProtocol/aztec-packages#7090)
(noir-lang/noir#7091)
([a964cd0](AztecProtocol/aztec-packages@a964cd0))
* Allow passing custom conditions to inlining pass
(noir-lang/noir#7083)
([a964cd0](AztecProtocol/aztec-packages@a964cd0))
* Attestation collection times out based on sequencer timetable
([#11248](AztecProtocol/aztec-packages#11248))
([946a418](AztecProtocol/aztec-packages@946a418))
* **avm:** Calldata, returndata slices out of range padded with zeros
([#11265](AztecProtocol/aztec-packages#11265))
([a469c94](AztecProtocol/aztec-packages@a469c94)),
closes
[#10933](AztecProtocol/aztec-packages#10933)
* Delete external-ci-approved.yml
([#11258](AztecProtocol/aztec-packages#11258))
([642bce6](AztecProtocol/aztec-packages@642bce6))
* Demote error closing forks to warn
([#11263](AztecProtocol/aztec-packages#11263))
([a5b7a6a](AztecProtocol/aztec-packages@a5b7a6a))
* Do not make new instruction if it hasn't changed
(noir-lang/noir#7069)
([a964cd0](AztecProtocol/aztec-packages@a964cd0))
* Ensure devnet has unproven config
([#11302](AztecProtocol/aztec-packages#11302))
([085f782](AztecProtocol/aztec-packages@085f782))
* Fixing `[@safety](https://github.com/safety)` warnings
([#11094](AztecProtocol/aztec-packages#11094))
([5de24e0](AztecProtocol/aztec-packages@5de24e0))
* Log correlation in traces in google cloud
([#11276](AztecProtocol/aztec-packages#11276))
([fbcc8ef](AztecProtocol/aztec-packages@fbcc8ef)),
closes
[#11019](AztecProtocol/aztec-packages#11019)
[#10937](AztecProtocol/aztec-packages#10937)
* Mark `noir-edwards` as expected to compile
(noir-lang/noir#7085)
([a964cd0](AztecProtocol/aztec-packages@a964cd0))
* Move shared pcs functionality to internal library in solidity and
small refactorings in sumcheck
([#11230](AztecProtocol/aztec-packages#11230))
([507ae9d](AztecProtocol/aztec-packages@507ae9d))
* Reduce the number of provers in rc-1
([#11296](AztecProtocol/aztec-packages#11296))
([92e40ff](AztecProtocol/aztec-packages@92e40ff))
* Remove references to padding txs
([#11264](AztecProtocol/aztec-packages#11264))
([32408f6](AztecProtocol/aztec-packages@32408f6))
* Remove warnings from types and rollup lib crates
([#11269](AztecProtocol/aztec-packages#11269))
([9f389a7](AztecProtocol/aztec-packages@9f389a7))
* Replace relative paths to noir-protocol-circuits
([8ece166](AztecProtocol/aztec-packages@8ece166))
* Replace relative paths to noir-protocol-circuits
([be42305](AztecProtocol/aztec-packages@be42305))
* Retry deploys
([#11252](AztecProtocol/aztec-packages#11252))
([23cfbb4](AztecProtocol/aztec-packages@23cfbb4))
* Set failed proof store for spartan deployments
([#11282](AztecProtocol/aztec-packages#11282))
([f787a52](AztecProtocol/aztec-packages@f787a52))
* Silence "Updated proven chain" log
([#11250](AztecProtocol/aztec-packages#11250))
([44bd79b](AztecProtocol/aztec-packages@44bd79b))
* Silence circuit return values in CI
([#11259](AztecProtocol/aztec-packages#11259))
([db3d860](AztecProtocol/aztec-packages@db3d860))
* Stable masternet images
([#11289](AztecProtocol/aztec-packages#11289))
([07fabe8](AztecProtocol/aztec-packages@07fabe8))
</details>

<details><summary>barretenberg: 0.71.0</summary>

##
[0.71.0](AztecProtocol/aztec-packages@barretenberg-v0.70.0...barretenberg-v0.71.0)
(2025-01-17)


### Bug Fixes

* Reallocate commitment key to avoid pippenger error
([#11249](AztecProtocol/aztec-packages#11249))
([8fc2011](AztecProtocol/aztec-packages@8fc2011))
* Resolve misc bugs handling phases in avm witgen
([#11218](AztecProtocol/aztec-packages#11218))
([29bc4bd](AztecProtocol/aztec-packages@29bc4bd))


### Miscellaneous

* **avm:** Calldata, returndata slices out of range padded with zeros
([#11265](AztecProtocol/aztec-packages#11265))
([a469c94](AztecProtocol/aztec-packages@a469c94)),
closes
[#10933](AztecProtocol/aztec-packages#10933)
* Move shared pcs functionality to internal library in solidity and
small refactorings in sumcheck
([#11230](AztecProtocol/aztec-packages#11230))
([507ae9d](AztecProtocol/aztec-packages@507ae9d))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Thunkar Thunkar Thunkar approved these changes

@benesjan benesjan Awaiting requested review from benesjan

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@nventuro @Thunkar