Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic when testRuntimeInterface methods need hooks but lack them #1399

Conversation

robert-e-davidson3
Copy link
Contributor

@robert-e-davidson3 robert-e-davidson3 commented Feb 4, 2022

Description

The runtime test interface has hooks for the FVM methods of the "Interface" API. Most have defaults for when hooks aren't set. A few instead throw invalid memory address or nil pointer dereference. This PR gives them defaults where appropriate else a more descriptive panic.

Update: Switching to make all of the methods panic when they don't have a hook.


  • Targeted PR against master branch
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work
  • Code follows the standards mentioned here
  • Updated relevant documentation
  • Re-reviewed Files changed in the Github PR explorer
  • Added appropriate labels

@@ -409,22 +409,37 @@ func (i *testRuntimeInterface) Hash(data []byte, tag string, hashAlgorithm HashA
}

func (i *testRuntimeInterface) SetCadenceValue(owner common.Address, key string, value cadence.Value) (err error) {
if i.setCadenceValue == nil {
panic("must specify testRuntimeInterface.setCadenceValue")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea! Maybe even do this consistently for all functions, currently some just return a default of no hook is provided. Better to have to explicitly provide functions that are called in the tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I just switched some to having defaults! I can switch them all over though. I agree that being explicit is better since defaults tend to hide bugs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, better to switch to panics, it's a great idea!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A LOT of tests fail when I eliminate the defaults. I created #1400 so we can account for the work and maybe assign it later depending on priority. It's a constrained enough problem that I should be able to it without needing much if any help but I don't want to push more urgent stuff back.

However, there are a few more methods that currently panic with a generic message instead of a nice one. I'm going to add them to this PR before asking for another review (hopefully super quick) then merging into master.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good! Thank you for trying and opening the separate PR 👍

@turbolent
Copy link
Member

@robert-e-davidson3 Please follow the checklist in the PR template as well as you can. It helps with avoiding mistakes, labeling helps with generating the release notes, etc. Thank you!

@robert-e-davidson3 robert-e-davidson3 force-pushed the robert/panic-when-runtime-tests-are-missing-needed-hooks branch from a128c91 to 1ac03c9 Compare February 4, 2022 00:51
@robert-e-davidson3 robert-e-davidson3 self-assigned this Feb 4, 2022
@github-actions
Copy link

github-actions bot commented Feb 4, 2022

Cadence Benchstat comparison

This branch with compared with the base branch onflow:master commit 8ecdaf3
The command for i in {1..N}; do go test ./... -run=XXX -bench=. -shuffle=on; done was used.
Bench tests were run a total of 7 times on each branch.

Results

old.txtnew.txt
time/opdelta
ContractInterfaceFungibleToken-235.4µs ± 2%36.5µs ± 1%+3.10%(p=0.009 n=6+6)
RuntimeResourceDictionaryValues-212.9ms ± 5%13.0ms ± 8%~(p=1.000 n=7+7)
RuntimeFungibleTokenTransfer-21.19ms ±25%1.21ms ±29%~(p=0.620 n=7+7)
ParseInfix-28.94µs ± 8%9.01µs ± 6%~(p=0.871 n=7+7)
ParseArray-211.5ms ± 7%11.5ms ± 5%~(p=1.000 n=7+7)
ParseFungibleToken-2163µs ± 3%167µs ± 5%~(p=0.310 n=6+6)
ParseDeploy/byte_array-219.1ms ±11%19.5ms ±19%~(p=0.902 n=7+7)
ParseDeploy/decode_hex-21.05ms ± 8%1.07ms ± 7%~(p=0.259 n=7+7)
QualifiedIdentifierCreation/One_level-22.03ns ± 1%2.17ns ± 8%~(p=0.221 n=6+7)
QualifiedIdentifierCreation/Three_levels-2125ns ± 4%125ns ± 3%~(p=0.804 n=7+7)
CheckContractInterfaceFungibleTokenConformance-2122µs ± 8%122µs ± 7%~(p=0.620 n=7+7)
NewInterpreter/new_interpreter-2964ns ± 9%980ns ± 5%~(p=0.302 n=7+7)
NewInterpreter/new_sub-interpreter-21.91µs ± 9%1.95µs ± 6%~(p=0.259 n=7+7)
InterpretRecursionFib-22.25ms ±16%2.23ms ± 1%~(p=0.432 n=7+5)
 
alloc/opdelta
RuntimeResourceDictionaryValues-24.05MB ± 0%4.05MB ± 0%~(p=0.620 n=7+7)
RuntimeFungibleTokenTransfer-2278kB ± 0%278kB ± 0%~(p=0.777 n=7+7)
QualifiedIdentifierCreation/One_level-20.00B 0.00B ~(all equal)
QualifiedIdentifierCreation/Three_levels-264.0B ± 0%64.0B ± 0%~(all equal)
CheckContractInterfaceFungibleTokenConformance-266.2kB ± 0%66.2kB ± 0%~(all equal)
ContractInterfaceFungibleToken-226.6kB ± 0%26.6kB ± 0%~(all equal)
NewInterpreter/new_interpreter-2768B ± 0%768B ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-21.24kB ± 0%1.24kB ± 0%~(all equal)
InterpretRecursionFib-21.24MB ± 0%1.24MB ± 0%~(p=0.580 n=6+7)
 
allocs/opdelta
RuntimeResourceDictionaryValues-2102k ± 0%102k ± 0%~(p=1.000 n=7+7)
RuntimeFungibleTokenTransfer-24.62k ± 0%4.62k ± 0%~(p=1.000 n=7+7)
QualifiedIdentifierCreation/One_level-20.00 0.00 ~(all equal)
QualifiedIdentifierCreation/Three_levels-22.00 ± 0%2.00 ± 0%~(all equal)
CheckContractInterfaceFungibleTokenConformance-21.07k ± 0%1.07k ± 0%~(all equal)
ContractInterfaceFungibleToken-2458 ± 0%458 ± 0%~(all equal)
NewInterpreter/new_interpreter-212.0 ± 0%12.0 ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-238.0 ± 0%38.0 ± 0%~(all equal)
InterpretRecursionFib-225.0k ± 0%25.0k ± 0%~(all equal)
 

@robert-e-davidson3 robert-e-davidson3 force-pushed the robert/panic-when-runtime-tests-are-missing-needed-hooks branch from 117d5c1 to 1ac03c9 Compare February 4, 2022 02:14
@@ -409,22 +409,37 @@ func (i *testRuntimeInterface) Hash(data []byte, tag string, hashAlgorithm HashA
}

func (i *testRuntimeInterface) SetCadenceValue(owner common.Address, key string, value cadence.Value) (err error) {
if i.setCadenceValue == nil {
panic("must specify testRuntimeInterface.setCadenceValue")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good! Thank you for trying and opening the separate PR 👍

@codecov-commenter
Copy link

codecov-commenter commented Feb 4, 2022

Codecov Report

Merging #1399 (201ab58) into master (8ecdaf3) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1399   +/-   ##
=======================================
  Coverage   73.08%   73.08%           
=======================================
  Files         280      280           
  Lines       38943    38943           
=======================================
  Hits        28463    28463           
  Misses       9037     9037           
  Partials     1443     1443           
Flag Coverage Δ
unittests 73.08% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8ecdaf3...201ab58. Read the comment docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants