Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: hash-based JSON circuit #41

Merged
merged 41 commits into from
Dec 11, 2024

Conversation

Autoparallel
Copy link
Contributor

THis one hurt my head

@Autoparallel Autoparallel changed the title feat: hash based JSON verification feat: hash-based JSON circuit Nov 13, 2024
@Autoparallel
Copy link
Contributor Author

This is based on #40 , so that should 100% be merged first.

This needs SERIOUS review and further testing.

@Autoparallel
Copy link
Contributor Author

The basic idea is this:

We want to show that a given sequence of keys for retrieving from a JSON :

  {
     "data": {
         "items": [
             {
                 "data": "Artist",
                 "profile": {
                     "name": "Taylor Swift"
                 }
             }
         ]
     }
  }

determines a unique value inside of the JSON, and furthermore the value is as expected.

For example, we may want to ultimately supply to this circuit:

["data", "items", 0, "profile", "name", "Taylor Swift"]

Ultimately what I have achieved at this point is getting unique hash values at points in the JSON that take into account all the data at that point that we need.

So, for example, we know we need to get:

  • "data" as a KEY at depth 1, so this can only be the case if we sequentially hash the stack [1,0] with the byte sequence [100,97,116,97] (over the next machine iterations). So this uniquely defines this key. (TODO: This does not rollover the previous hash into the next for this exact part yet, I almost have it though).
  • "items" as a KEY follows the same paradigm, but we should completely exit rewrite the hasher state before we get to a new key. This is done. Hitting a comma , while not parsing a string will wipe the hasher state.
  • 0 as an ARRAY INDEX also follows the same paradigm, but there is now just repeated hashing with "[2,n]at indexn. Comma characters ,` are consistently wiping the state as with the key.
  • This continues down until we hit "Taylor Swift" in our example which will be a VALUE, so the only difference is a hashing with stack [1,1] at that point. Once again guaranteeing uniqueness.

@Autoparallel Autoparallel changed the base branch from main to release/v0-7-0 December 9, 2024 20:58
@Autoparallel Autoparallel marked this pull request as ready for review December 11, 2024 02:23
@Autoparallel Autoparallel merged commit 06c6920 into release/v0-7-0 Dec 11, 2024
@Autoparallel Autoparallel deleted the feat/new-json-hash-circuit branch December 11, 2024 02:27
Autoparallel added a commit that referenced this pull request Dec 18, 2024
* Update CHANGELOG.md

* Update package.json

* feat: optimized `HTTPVerification` (#82)

* wip: better HTTP

- Trying to reduce file size and constraints substantially
- Also need to make this so it doesn't matter if there is padding around headers / etc.

* WIP: improving HTTP digesting

* WIP: http rewrite

* WIP: almost working no-header test

* WIP: working start/body

* working tests!

* cleanup

* Update masker.circom

* Update CHANGELOG.md

* feat: hash-based JSON circuit (#41)

* feat: hash based JSON verification

* WIP: save

* resetting for clearer approach

* good save state

* feat: working hash version

Though this will be too expensive, the idea works!

* WIP: need to clear after comma

* WIP: good progress

* WIP: getting keys also now

* feat: (mostly?) working tree hasher

* seems to be correct for spotify

* perf: first optimization

* wip: brain hurty

left a note to myself

* fix: tree hasher seems correct now

* TODO: note to self

* feat: hash based JSON verification

* WIP: save

* resetting for clearer approach

* good save state

* feat: working hash version

Though this will be too expensive, the idea works!

* WIP: need to clear after comma

* WIP: good progress

* WIP: getting keys also now

* feat: (mostly?) working tree hasher

* seems to be correct for spotify

* perf: first optimization

* wip: brain hurty

left a note to myself

* fix: tree hasher seems correct now

* TODO: note to self

* cleanup from rebase

* cleanup

* WIP: seems to monomial correctly

* rename

* add in value to eval at

* WIP: start looking for matches

* made some fixes

* it may be working!

* now i can write tests!

* more tests

* more JSON hasher tests

* cleanup

* remove excess stuff

* adjust builds

* fix: build

* feat: manifest digest verification (#83)

* feat: `PolynomialDigest`

* WIP: working to get through NIVC

* feat: HTTP circuit digesting

* feat: ChaCha circuit digesting

* feat: JSON circuit digesting

* fix: `JSONExtraction`

* IT WORKS

* feat: TS init digest

* feat: separate sequence/value

* cleanup

* fix: builds

* fix: warnings

* Update CHANGELOG.md

* fix: PR feedback

* Fix/http verification (#85)

* fix: `zeroed_data` for `data_digest` in `http_verification`

* add test for 1024

* cleanup readme

---------

Co-authored-by: Sambhav Dusad <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant