Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SAW will not quickcheck for type Integer #1058

Closed
msaaltink opened this issue Feb 2, 2021 · 3 comments
Closed

SAW will not quickcheck for type Integer #1058

msaaltink opened this issue Feb 2, 2021 · 3 comments
Assignees
Labels
subsystem: saw-core Issues related to the saw-core representation or the saw-core subsystem

Comments

@msaaltink
Copy link
Contributor

msaaltink commented Feb 2, 2021

Cryptol happily tests functions on Integers:

Cryptol> :check \ (x:Integer)-> x+1 == x
Using random testing.
(\(x : Integer) -> x + 1 == x) 240 = False

SAW, in contrast, does not:

sawscript> prove (quickcheck 10) {{ \ (x:Integer)-> x+1 == x }}
[19:56:22.922] WARNING: using quickcheck to prove goal...

Stack trace:
"prove" (<stdin>:1:1-1:6):
"quickcheck" (<stdin>:1:8-1:18):
quickcheck:
term has non-testable type:
[...]

This might be related to #669.

The Z types are similar; Cryptol can test them but SAW will not.

@brianhuffman
Copy link
Contributor

This is a consequence of saw-script/saw-core and cryptol having completely separate implementations of quickcheck generators. The ones in the saw-core package is just way out of date, and was never updated to include type Integer or Z n.

I could go and add types Integer and Z n to the saw-core test generators. But probably a better long-term approach is to just reimplement saw-script's quickcheck tactic using Cryptol's quickcheck, and remove the out-of-date saw-core one completely. This would ensure that quickcheck remains consistent in both settings, even as new types (e.g. floating-point numbers and rationals) are added.

@brianhuffman
Copy link
Contributor

The saw-core module Verifier/SAW/Testing/Random.hs even says right at the top that it is "based on 'Cryptol.Testing.Random'". Clearly this is unnecessary code duplication that we should get rid of.

@brianhuffman
Copy link
Contributor

The current plan is to go ahead and implement the quick fix to add support for type Integer to the saw-core quickcheck. This will require it to switch from FiniteType to FirstOrderType for recognizing types that can be quantified over. We should still plan to unify the quickcheck code with cryptol in the long term.

@brianhuffman brianhuffman self-assigned this Feb 15, 2021
@brianhuffman brianhuffman added the subsystem: saw-core Issues related to the saw-core representation or the saw-core subsystem label Feb 15, 2021
robdockins added a commit that referenced this issue Mar 4, 2021
robdockins added a commit that referenced this issue Mar 8, 2021
robdockins added a commit that referenced this issue Mar 9, 2021
robdockins added a commit that referenced this issue Mar 11, 2021
robdockins added a commit that referenced this issue Mar 11, 2021
@mergify mergify bot closed this as completed in 4485d7c Mar 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
subsystem: saw-core Issues related to the saw-core representation or the saw-core subsystem
Projects
None yet
Development

No branches or pull requests

2 participants