Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix JSON encoding of type values with recursive type #1705

Merged

Conversation

turbolent
Copy link
Member

@turbolent turbolent commented Jun 7, 2022

Closes #1704

Description

Preparation of static types may run into a recursion. Track results and return the type ID in the recursive case.


  • Targeted PR against master branch
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work
  • Code follows the standards mentioned here
  • Updated relevant documentation
  • Re-reviewed Files changed in the Github PR explorer
  • Added appropriate labels

@turbolent turbolent self-assigned this Jun 7, 2022
@github-actions
Copy link

github-actions bot commented Jun 7, 2022

Cadence Benchstat comparison

This branch with compared with the base branch onflow:master commit ec9fb29
The command for i in {1..N}; do go test ./... -run=XXX -bench=. -benchmem -shuffle=on; done was used.
Bench tests were run a total of 7 times on each branch.

Results

old.txtnew.txt
time/opdelta
CheckContractInterfaceFungibleTokenConformance-2176µs ± 2%178µs ± 5%~(p=0.534 n=6+7)
ContractInterfaceFungibleToken-250.5µs ± 2%51.0µs ± 5%~(p=0.836 n=6+7)
InterpretRecursionFib-23.33ms ± 8%3.44ms ± 7%~(p=0.259 n=7+7)
NewInterpreter/new_interpreter-21.33µs ± 7%1.32µs ± 5%~(p=0.779 n=7+7)
NewInterpreter/new_sub-interpreter-22.74µs ± 5%2.70µs ± 5%~(p=0.710 n=7+7)
ParseArray-210.4ms ± 1%10.4ms ± 3%~(p=1.000 n=5+6)
ParseDeploy/byte_array-215.5ms ± 6%15.4ms ± 3%~(p=1.000 n=7+6)
ParseDeploy/decode_hex-21.41ms ± 2%1.43ms ± 6%~(p=0.366 n=6+7)
ParseFungibleToken/With_memory_metering-2255µs ±10%252µs ± 4%~(p=0.620 n=7+7)
ParseFungibleToken/Without_memory_metering-2191µs ± 2%194µs ± 6%~(p=0.731 n=6+7)
ParseInfix-28.89µs ± 2%9.06µs ± 6%~(p=0.366 n=6+7)
QualifiedIdentifierCreation/One_level-23.37ns ± 6%3.32ns ± 3%~(p=0.535 n=7+7)
QualifiedIdentifierCreation/Three_levels-2170ns ± 4%171ns ± 3%~(p=0.445 n=6+7)
RuntimeFungibleTokenTransfer-21.45ms ±31%1.51ms ±25%~(p=0.805 n=7+7)
RuntimeResourceDictionaryValues-28.08ms ± 4%7.87ms ± 4%~(p=0.128 n=7+7)
Transfer-2114ns ± 3%112ns ± 5%~(p=0.119 n=7+7)
 
alloc/opdelta
CheckContractInterfaceFungibleTokenConformance-266.3kB ± 0%66.3kB ± 0%~(p=0.070 n=7+7)
ContractInterfaceFungibleToken-226.7kB ± 0%26.7kB ± 0%~(all equal)
InterpretRecursionFib-21.25MB ± 0%1.25MB ± 0%~(p=1.000 n=7+7)
NewInterpreter/new_interpreter-2872B ± 0%872B ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-21.38kB ± 0%1.38kB ± 0%~(all equal)
ParseArray-22.83MB ± 2%2.86MB ± 0%~(p=0.945 n=7+6)
ParseDeploy/byte_array-24.33MB ± 0%4.50MB ± 4%~(p=0.128 n=6+7)
ParseDeploy/decode_hex-2214kB ± 0%214kB ± 0%~(p=0.729 n=7+7)
ParseFungibleToken/With_memory_metering-236.3kB ± 0%36.3kB ± 0%~(p=0.612 n=7+7)
ParseFungibleToken/Without_memory_metering-236.3kB ± 0%36.3kB ± 0%−0.04%(p=0.042 n=7+6)
ParseInfix-22.17kB ± 0%2.17kB ± 0%~(p=0.788 n=7+7)
QualifiedIdentifierCreation/One_level-20.00B 0.00B ~(all equal)
QualifiedIdentifierCreation/Three_levels-264.0B ± 0%64.0B ± 0%~(all equal)
RuntimeFungibleTokenTransfer-2237kB ± 1%237kB ± 0%~(p=0.710 n=7+7)
RuntimeResourceDictionaryValues-22.25MB ± 0%2.25MB ± 0%~(p=0.097 n=7+7)
Transfer-248.0B ± 0%48.0B ± 0%~(all equal)
 
allocs/opdelta
CheckContractInterfaceFungibleTokenConformance-21.07k ± 0%1.07k ± 0%~(all equal)
ContractInterfaceFungibleToken-2460 ± 0%460 ± 0%~(all equal)
InterpretRecursionFib-223.8k ± 0%23.8k ± 0%~(all equal)
NewInterpreter/new_interpreter-212.0 ± 0%12.0 ± 0%~(all equal)
NewInterpreter/new_sub-interpreter-240.0 ± 0%40.0 ± 0%~(all equal)
ParseArray-270.0k ± 0%70.0k ± 0%~(p=0.592 n=7+7)
ParseDeploy/byte_array-2105k ± 0%105k ± 0%~(p=0.505 n=7+7)
ParseDeploy/decode_hex-278.0 ± 0%78.0 ± 0%~(all equal)
ParseFungibleToken/With_memory_metering-21.06k ± 0%1.06k ± 0%~(all equal)
ParseFungibleToken/Without_memory_metering-21.06k ± 0%1.06k ± 0%~(all equal)
ParseInfix-266.0 ± 0%66.0 ± 0%~(all equal)
QualifiedIdentifierCreation/One_level-20.00 0.00 ~(all equal)
QualifiedIdentifierCreation/Three_levels-22.00 ± 0%2.00 ± 0%~(all equal)
RuntimeFungibleTokenTransfer-24.52k ± 0%4.52k ± 0%~(p=1.000 n=7+7)
RuntimeResourceDictionaryValues-237.6k ± 0%37.6k ± 0%~(p=0.522 n=7+7)
Transfer-21.00 ± 0%1.00 ± 0%~(all equal)
 

@turbolent turbolent marked this pull request as ready for review June 7, 2022 15:55
@codecov
Copy link

codecov bot commented Jun 7, 2022

Codecov Report

Merging #1705 (a36cd96) into master (ec9fb29) will increase coverage by 0.00%.
The diff coverage is 96.92%.

@@           Coverage Diff           @@
##           master    #1705   +/-   ##
=======================================
  Coverage   76.63%   76.64%           
=======================================
  Files         292      292           
  Lines       61195    61207   +12     
=======================================
+ Hits        46898    46910   +12     
+ Misses      12684    12683    -1     
- Partials     1613     1614    +1     
Flag Coverage Δ
unittests 76.64% <96.92%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
encoding/json/encode.go 94.44% <96.42%> (-0.26%) ⬇️
runtime/convertTypes.go 61.50% <100.00%> (ø)
types.go 78.20% <100.00%> (ø)
runtime/interpreter/storage.go 70.63% <0.00%> (+1.58%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ec9fb29...a36cd96. Read the comment docs.

Comment on lines 664 to 670
switch typ := typ.(type) {
case cadence.CompositeType, cadence.InterfaceType:
if _, ok := results[typ]; ok {
return typ.ID()
}
results[typ] = struct{}{}
}
Copy link
Member

@SupunS SupunS Jun 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if these two checks are swapped, just to make sure any unhandled/ newly-added types are gracefully handled?

Suggested change
switch typ := typ.(type) {
case cadence.CompositeType, cadence.InterfaceType:
if _, ok := results[typ]; ok {
return typ.ID()
}
results[typ] = struct{}{}
}
if _, ok := results[typ]; ok {
switch typ := typ.(type) {
case cadence.CompositeType, cadence.InterfaceType:
return typ.ID()
default:
panic(fmt.Errorf("unsupported recursive type: %s", typ))
}
}
results[typ] = struct{}{}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, good idea, it would be great to ensure all recursive types are supported!

This suggestions does not quite work as-is: It records all types as "seen", so e.g. even a primitive type like cadence.IntType is erroneously considered an unsupported recursive type.

Tried to fix this by only recording the recursive types:

	var supportedRecursiveType bool
	switch typ.(type) {
	case cadence.CompositeType, cadence.InterfaceType:
		supportedRecursiveType = true
	}

	if _, ok := results[typ]; ok {
		if !supportedRecursiveType {
			panic(fmt.Errorf("failed to prepare type: unsupported recursive type: %T", typ))
		}

		return typ.ID()
	}

	if supportedRecursiveType {
		results[typ] = struct{}{}
	}

That then because we always check any type (results[typ]), this panics for some types, e.g. functions, which are not pointers at the moment:

panic: runtime error: hash of unhashable type cadence.FunctionType

I guess we should keep the logic above and make all types hashable?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SupunS Tried this in a36cd96. What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, I wasn't thinking of the other types.

This looks good to me. Thanks for adding that!

@turbolent turbolent merged commit 7ce0f63 into master Jun 7, 2022
@turbolent turbolent deleted the bastian/1704-json-encode-type-value-recursive-type branch June 7, 2022 18:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Overflow in json encode on Type
3 participants