Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix and simplify parser buffering #1333

Merged
merged 3 commits into from
Jan 5, 2022
Merged

Fix and simplify parser buffering #1333

merged 3 commits into from
Jan 5, 2022

Conversation

SupunS
Copy link
Member

@SupunS SupunS commented Jan 5, 2022

Port of https://github.com/dapperlabs/cadence-internal/pull/41 authored by Bastian.

Description

Description from original PR:

Joe had identified a bug in the parser: Buffering does not handle nesting properly: https://github.com/jsatdapr/fuzzing/pull/2

The parser gets confused when doing nested lookaheads. This means someone can write code that looks like it says one thing, but does the opposite. The failure path is complex and can probably be tricked into doing arbitrarily interesting things beyond the simple testcases given here. Even still, a script that reads like -1 but runs like +1 is enough to do interesting things already.

Instead of complicating the already complicated buffering logic in the parser even further, fix and simplify it by:

  • Refactoring the lexer to run in advance and lexing all tokens
  • Adding support for rewinding to an earlier token stream position
  • Keeping a stack of token stream positions and buffered errors in the parser
  • Properly pushing and popping the two stacks (token stream positions and buffered errors) for nested bufferings

This also also increases performance to 2x for the FT contract.


  • Targeted PR against master branch
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work
  • Code follows the standards mentioned here
  • Updated relevant documentation
  • Re-reviewed Files changed in the Github PR explorer
  • Added appropriate labels

@github-actions
Copy link

github-actions bot commented Jan 5, 2022

Cadence Benchstat comparison

This branch with compared with the base branch onflow:master commit 2eb9cf7
The command for i in {1..N}; do go test ./... -run=XXX -bench=. -shuffle=on; done was used.
Bench tests were run a total of 7 times on each branch.

Results

old.txtnew.txt
time/opdelta
RuntimeFungibleTokenTransfer-21.77ms ±20%1.77ms ±36%~(p=0.535 n=7+7)
RuntimeResourceDictionaryValues-218.6ms ± 3%18.9ms ± 4%~(p=0.535 n=7+7)
QualifiedIdentifierCreation/One_level-23.58ns ± 3%3.55ns ± 4%~(p=0.620 n=7+7)
QualifiedIdentifierCreation/Three_levels-2181ns ± 3%185ns ± 4%~(p=0.068 n=7+7)
ContractInterfaceFungibleToken-253.1µs ± 3%53.3µs ± 5%~(p=0.699 n=6+6)
CheckContractInterfaceFungibleTokenConformance-2189µs ± 3%186µs ± 5%~(p=0.366 n=6+7)
NewInterpreter/new_interpreter-21.36µs ± 6%1.34µs ± 3%~(p=0.334 n=7+7)
NewInterpreter/new_sub-interpreter-22.61µs ± 4%2.65µs ± 5%~(p=0.534 n=7+6)
InterpretRecursionFib-23.06ms ± 3%3.14ms ± 3%~(p=0.128 n=7+7)
ParseDeploy/decode_hex-21.63ms ± 4%1.55ms ± 6%−5.14%(p=0.007 n=7+7)
ParseDeploy/byte_array-239.2ms ± 3%26.3ms ± 5%−33.09%(p=0.001 n=7+7)
ParseArray-225.9ms ± 3%17.2ms ±15%−33.47%(p=0.001 n=7+7)
ParseInfix-227.9µs ± 4%13.0µs ±10%−53.40%(p=0.001 n=7+7)
ParseFungibleToken-2537µs ± 6%245µs ± 4%−54.40%(p=0.001 n=7+6)
 
alloc/opdelta
RuntimeFungibleTokenTransfer-2238kB ± 0%273kB ± 0%+14.60%(p=0.001 n=7+7)
RuntimeResourceDictionaryValues-24.04MB ± 0%4.05MB ± 0%+0.42%(p=0.001 n=7+6)
QualifiedIdentifierCreation/One_level-20.00B 0.00B ~(all equal)
QualifiedIdentifierCreation/Three_levels-264.0B ± 0%64.0B ± 0%~(all equal)
ContractInterfaceFungibleToken-226.5kB ± 0%26.5kB ± 0%~(p=0.462 n=7+7)
CheckContractInterfaceFungibleTokenConformance-265.7kB ± 0%65.7kB ± 0%~(p=0.070 n=7+7)
NewInterpreter/new_interpreter-2720B ± 0%720B ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-21.18kB ± 0%1.18kB ± 0%~(all equal)
InterpretRecursionFib-21.24MB ± 0%1.24MB ± 0%~(p=0.192 n=7+7)
 
allocs/opdelta
RuntimeFungibleTokenTransfer-24.54k ± 0%4.54k ± 0%~(p=0.592 n=7+7)
RuntimeResourceDictionaryValues-2102k ± 0%102k ± 0%~(p=0.070 n=7+6)
QualifiedIdentifierCreation/One_level-20.00 0.00 ~(all equal)
QualifiedIdentifierCreation/Three_levels-22.00 ± 0%2.00 ± 0%~(all equal)
ContractInterfaceFungibleToken-2457 ± 0%457 ± 0%~(all equal)
CheckContractInterfaceFungibleTokenConformance-21.07k ± 0%1.07k ± 0%~(all equal)
NewInterpreter/new_interpreter-211.0 ± 0%11.0 ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-236.0 ± 0%36.0 ± 0%~(all equal)
InterpretRecursionFib-225.0k ± 0%25.0k ± 0%~(all equal)
 

@SupunS SupunS marked this pull request as ready for review January 5, 2022 22:06
@SupunS SupunS requested a review from turbolent as a code owner January 5, 2022 22:06
@SupunS SupunS merged commit ef9f285 into master Jan 5, 2022
@SupunS SupunS deleted the supun/port-internal-41 branch January 5, 2022 22:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants