Support non-constant arbitrary functions #171

non · 2015-06-12T22:16:05Z

This PR adds support for better Arbitrary[FunctionN[...]] instances. It does a few things:

Removes most use of scala.util.Random.
Implement a simple linear-congruent RNG (Knuth's MMIX), that uses 64-bits of state.
Change Gen methods to explicitly receive (and pass) RNG seeds (Long values)
Introduce Cogen type to permutate RNG seed states.
Introduce a bunch of implicit Cogen instances for many built-in types.
Use codegen to wire up new Gen and Arbitrary instances for functions
Add a line to build.sbt to get tests running faster for ScalaJS.

There is a bit of sketchiness in Gen due to its use of Option[A] instead of A. I was forced to use .get (like a few other existing methods do) which is not ideal. I can't tell if you'd expect this to be a major problem for ScalaCheck users.

We should also add a bunch more Cogen instances. I chose not to do this until I got a sense of whether this PR was going in the right direction.

Finally, while I bet most people using arbitrary functions would use primitive types, there is a bunch of work around updating documentation to explain this approach (and how to define custom Cogen instances).

Fixes #136

non · 2015-06-12T22:42:45Z

Argh! Tests passed locally -- will investigate why they are failing for Travis.

non · 2015-06-12T22:46:44Z

Oh, OK. These are all MIMA failures.

This change is definitely not binary-compatible.

xuwei-k · 2015-06-12T23:57:53Z

awesome! But maybe there is a same issue as my library.

scalaprops/scalaprops#6

scala> Gen.listOfN(10, implicitly[Arbitrary[Int => Int]].arbitrary).sample.get.map((1 to 10).map(_)) foreach println
Vector(-2147483648, 1, -2147483648, 205452885, 2043426884, 1, -1632521114, 1, -2147483648, 243620675)
Vector(-2147483648, 1, -2147483648, 205452885, 2043426884, 1, -1632521114, 1, -2147483648, 243620675)
Vector(-2147483648, 1, -2147483648, 205452885, 2043426884, 1, -1632521114, 1, -2147483648, 243620675)
Vector(-2147483648, 1, -2147483648, 205452885, 2043426884, 1, -1632521114, 1, -2147483648, 243620675)
Vector(-2147483648, 1, -2147483648, 205452885, 2043426884, 1, -1632521114, 1, -2147483648, 243620675)
Vector(-2147483648, 1, -2147483648, 205452885, 2043426884, 1, -1632521114, 1, -2147483648, 243620675)
Vector(-2147483648, 1, -2147483648, 205452885, 2043426884, 1, -1632521114, 1, -2147483648, 243620675)
Vector(-2147483648, 1, -2147483648, 205452885, 2043426884, 1, -1632521114, 1, -2147483648, 243620675)
Vector(-2147483648, 1, -2147483648, 205452885, 2043426884, 1, -1632521114, 1, -2147483648, 243620675)
Vector(-2147483648, 1, -2147483648, 205452885, 2043426884, 1, -1632521114, 1, -2147483648, 243620675)

non · 2015-06-13T00:25:05Z

Ouch! Good catch. Yes, that's definitely a bug.

xuwei-k · 2015-06-13T00:45:07Z

src/main/scala/org/scalacheck/Cogen.scala

+import language.higherKinds
+import language.implicitConversions
+
+sealed trait Cogen[-T] {


sealed abstract class is better for keep binary compatibility.

xuwei-k · 2015-06-13T01:28:07Z

following expression should returns List((false,false), (false,true), (true,false), (true,true))

scala> org.scalacheck.Gen.listOfN(100000, implicitly[Arbitrary[Boolean => Boolean]].arbitrary).sample.get.map{f => (f(true), f(false))}.distinct.sorted
res0: List[(Boolean, Boolean)] = List((false,true), (true,false))

scala> org.scalacheck.Gen.listOfN(100000, implicitly[Arbitrary[Boolean => Boolean]].arbitrary).sample.get.map{f => (f(true), f(false))}.distinct.sorted
res1: List[(Boolean, Boolean)] = List((false,true), (true,false))

scala> org.scalacheck.Gen.listOfN(100000, implicitly[Arbitrary[Boolean => Boolean]].arbitrary).sample.get.map{f => (f(true), f(false))}.distinct.sorted
res2: List[(Boolean, Boolean)] = List((false,true), (true,false))

rickynils · 2015-06-13T10:06:09Z

@non This is great stuff! I have worked in a similar direction in the scalacheck2 branch, but there I've made even more changes to Gen to also support finite generators. Due to lack of time, lots of stuff remains to be done there, though.

So the question is how to go forward. The realistic approach is probably to try to get the immutable gen and cogen into a 1.13 release, and keep my other changes for 2.0.

I'll make an attempt to dig into your solution as soon as possible.

non · 2015-06-13T18:19:51Z

There does seem to be a problem with Gen.oneOf combined with functions, which is demonstrated by @xuwei-k's second issue. I'm looking into it.

@rickynils that sounds fine. Let me know if you see any issues or want me to make any changes.

rickynils · 2015-06-15T18:04:22Z

@non If you rebase this PR, you can bump versionNumber in build.sbt to 1.13.0, and set previousArtifact to None. That should take care of the failing Travis tests.

This commit introduces a (seed: Long) parameter to many methods that need to generate random values, as well as an Rng object that implements a simple linear-congruent RNG. The jvm tests pass. Currently getting permgen/OOM errors in the js tests, not sure if this indicates a problem with the implementation. Still to do: there are still probably places where seeds are not being threaded through correctly.

This type allows values to perturb the RNG, meaning that we can build something resembling a true Gen[A => B]. The basic strategy is as follows: Given a Cogen[A] and a Gen[B], we can create an A => B. When we are given a particular A value, we use Cogen[A] to permute the RNG seed we have, and then use Gen[B] to generate a B value. In this way, the B values created are dependent on the A values provided, and we have a "real" function. This commit doesn't actually do any of that yet, but it does introduce a Cogen[A] type, and define a bunch of useful instances.

There is some ugliness in here (for example, .retrieve.get is a pattern I've copied from elsewhere in the code). However, the instances I've produced do seem to work. The next step is to figure out how to use code-gen to create these methods, as well as updating the Arbitrary code.

This commit adds support for code-generation for Gen. To use it, you'd do something like: import scalacheck._ import Arbitrary.arbitrary Gen.function1[Int, Int](arbitrary[Int]) // returns a Gen[Int => Int] instance // uses implicit Cogen[Int] The only step that is left (besides writing tests) is to start generating new Arbitrary[_] instances that use these methods to provide better arbitrary functions.

This commit updates the code-generation script to use the Gen.functionN methods to produce better arbitrary functions. This is a breaking API change. We probably need to add more Cogen instances for end-users to avoid breaking their tests. However, Scalacheck's own tests still pass. At this point the feature is added, and what is needed is clean up and fleshing out the stable of Cogen instances.

@xuwei-k

Thanks to @xuwei-k for catching this.

non · 2015-06-16T17:55:09Z

Alright, I just did what you suggested: I rebased against upstream/master and edited build.sbt.

I'm still working on the issue that @xuwei-k noticed with Arbitrary[Boolean => Boolean]. I'll push a commit and comment here once I have that issue fixed.

This commit swaps out the 64-bit linear-congruent RNG for a variant on Bob Jenkin's simple RNG [1]. It provides much better support for the kind of reseeding that Cogen needs. [1] http://burtleburtle.net/bob/rand/smallprng.html

This commit adds properties to ensure that Cogen + Gen are producing all the possible functions for types. It ensures that our RNG is doing a good job using the entropy from function arguments to reseed itself to produce novel output values.

non · 2015-06-17T21:53:41Z

These last commits fix the issue that @xuwei-k found. At this point I consider the PR feature complete.

@rickynils is there anything else you'd like me to do? Scalacheck probably needs documentation explaining how Cogen[A] works but I wasn't sure if you wanted me to tackle that in this PR or not.

xuwei-k · 2015-06-20T04:13:32Z

👍

Support non-constant arbitrary functions

jedesah · 2015-06-21T19:44:11Z

This is awesome!

rickynils · 2015-06-21T21:24:01Z

Now that snapshots are published automatically, you can try out this in 1.13.0-ea2bbe6-SNAPSHOT

scalacheck generate non-constant Function Arbitrary since 1.13.0 typelevel/scalacheck#171

carlpaten · 2018-07-24T18:40:15Z

I found this page while looking for documentation on the topic of Cogen. I don't think there is such a thing at the moment, but you can check this Haskell blog post to get the gist of it.

xuwei-k reviewed Jun 13, 2015
View reviewed changes

non added 9 commits June 16, 2015 13:52

Patch build.sbt so js tests can pass.

653192b

Fix some tests I broke due to Gen API changes.

9b0bb56

Fix repeated seed bug with function gen.

8100fd2

Thanks to @xuwei-k for catching this.

Bump version number, unset previous artifact.

b2b5759

non force-pushed the topic/immutable-rng branch from 509c349 to b2b5759 Compare June 16, 2015 17:53

non added 2 commits June 17, 2015 16:53

Better RNG, better Cogen, better results.

96e0913

This commit swaps out the 64-bit linear-congruent RNG for a variant on Bob Jenkin's simple RNG [1]. It provides much better support for the kind of reseeding that Cogen needs. [1] http://burtleburtle.net/bob/rand/smallprng.html

Add test for function randonness.

59153fa

This commit adds properties to ensure that Cogen + Gen are producing all the possible functions for types. It ensures that our RNG is doing a good job using the entropy from function arguments to reseed itself to produce novel output values.

Remove commented-out code.

ed0d885

rickynils added a commit that referenced this pull request Jun 21, 2015

Merge pull request #171 from non/topic/immutable-rng

ea2bbe6

Support non-constant arbitrary functions

rickynils merged commit ea2bbe6 into typelevel:master Jun 21, 2015

non mentioned this pull request Jul 9, 2015

Remove mutable random number generator in Gen #67

Closed

ceedubs mentioned this pull request Feb 12, 2016

java.util.NoSuchElementException: None.get #218

Closed

mosesn mentioned this pull request Mar 16, 2016

Stop takeWhile and dropWhile at predicate match twitter/util#154

Closed

xuwei-k mentioned this pull request Mar 19, 2016

Find a way to avert the CI failure No output has been received in the last 10 minutes typelevel/cats#810

Closed

xuwei-k mentioned this pull request Jul 24, 2016

#52 suggestion for testing laws highj/highj#54

Merged

mosesn mentioned this pull request Oct 25, 2016

upgrade to new scalatest and scalacheck twitter/algebird#566

Closed

xuwei-k added a commit to xuwei-k/scalaz-specs2 that referenced this pull request Nov 11, 2016

remove Function1IntInt

f931be3

scalacheck generate non-constant Function Arbitrary since 1.13.0 typelevel/scalacheck#171

xuwei-k mentioned this pull request Nov 11, 2016

remove Function1IntInt typelevel/scalaz-specs2#27

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support non-constant arbitrary functions #171

Support non-constant arbitrary functions #171

non commented Jun 12, 2015

non commented Jun 12, 2015

non commented Jun 12, 2015

xuwei-k commented Jun 12, 2015

non commented Jun 13, 2015

xuwei-k Jun 13, 2015

xuwei-k commented Jun 13, 2015

rickynils commented Jun 13, 2015

non commented Jun 13, 2015

rickynils commented Jun 15, 2015

non commented Jun 16, 2015

non commented Jun 17, 2015

xuwei-k commented Jun 20, 2015

jedesah commented Jun 21, 2015

rickynils commented Jun 21, 2015

carlpaten commented Jul 24, 2018

Support non-constant arbitrary functions #171

Support non-constant arbitrary functions #171

Conversation

non commented Jun 12, 2015

non commented Jun 12, 2015

non commented Jun 12, 2015

xuwei-k commented Jun 12, 2015

non commented Jun 13, 2015

xuwei-k Jun 13, 2015

Choose a reason for hiding this comment

xuwei-k commented Jun 13, 2015

rickynils commented Jun 13, 2015

non commented Jun 13, 2015

rickynils commented Jun 15, 2015

non commented Jun 16, 2015

non commented Jun 17, 2015

xuwei-k commented Jun 20, 2015

jedesah commented Jun 21, 2015

rickynils commented Jun 21, 2015

carlpaten commented Jul 24, 2018