Implement labelledExamples

[dwijnand]

Fixes #163

This introduces some duplication but, more generally, I'm not sure it's doing exactly the right thing.
Particularly: how is it meant to behave on a failing property? What about None results? Also, should it only
print the journal logs, as it's doing, or also the result logs? And did I make all the correct modifications in
my "takeSmallest" copy? Lots of questions.

I'd gladly have this land in a "we're not totally sure"/experimental feature and dogfood it in a project of
mine, which might give some better insights on what design choices are better. (But I can also copy this impl
into my project, so I wouldn't be blocked either - pure FP ftw.)

[charleso]

@dwijnand I'm definitely ok with taking some experimental code. I think my only preference here would not be to duplicate any of that logic though (those 2 functions in particular are critical to how it all hangs together). If we can either share what we can with common functions and possibly have some parameters/callbacks that might avoid us having to keep the two in sync. In particular, I know those functions have been refactored for Future support.

#147

I don't have any specific suggestions though I'm afraid.

[dwijnand]

Alright, I'll seek to dedup.

[dwijnand]

Can this be merged and released? I think I left this in a state I was very happy with: useful and deduped - but happy to have another look if a fresh review spots anything.

[eed3si9n]

Is there a way of doing partest neg equivalent to see what gets printed on the screen when some test fails?

[dwijnand]

No, there's no setup for testing the behaviour of runner, just unit tests for the core's behaviour. (See also test/README.md 🙂)

[eed3si9n]

My interest is in the negative behavior like in this case how the example affect the test failure output, not so much the details of different runners.

[dwijnand]

Before:

[info] + hedgehog.LabelledExamplesTest.testReverse: OK, passed 100 tests
[info] > 72% nonempty
[info] > 28% empty
[info] - hedgehog.LabelledExamplesTest.testReverseFail: Falsified after 20 passed tests
[info] > List(a, b)
[info] > === Not Equal ===
[info] > --- lhs ---
[info] > List(b, a)
[info] > --- rhs ---
[info] > List(a, b)
[info] > 65% empty
[info] > 35% nonempty

After:

[info] + hedgehog.LabelledExamplesTest.testReverse: OK, passed 100 tests
[info] > 69% nonempty List(a)
[info] > 31% empty List()
[info] - hedgehog.LabelledExamplesTest.testReverseFail: Falsified after 19 passed tests
[info] > List(a, b)
[info] > === Not Equal ===
[info] > --- lhs ---
[info] > List(b, a)
[info] > --- rhs ---
[info] > List(a, b)
[info] > 78% empty List()
[info] > 21% nonempty List(a)

using

    , property("testReverse", testReverse).withExamples
    , property("testReverseFail", testReverseFail).withExamples

...

  def testReverse: Property =
    for {
      xs <- Gen.alpha.list(Range.linear(0, 10)).forAll
        .classify("empty", _.isEmpty)
        .classify("nonempty", _.nonEmpty)
    } yield xs.reverse.reverse ==== xs

  def testReverseFail: Property =
    for {
      xs <- Gen.alpha.list(Range.linear(0, 10)).forAll
        .classify("empty", _.isEmpty)
        .classify("nonempty", _.nonEmpty)
    } yield xs.reverse ==== xs

[eed3si9n]

🚢 it

[kevin-lee]

Looks good! @dwijnand Would you mind updating docs as well?

[charleso]

👍 Looks good @dwijnand. You should definitely feel comfortable merging if and someone else are happy with it. Don't wait for me.

Just one final (probably unhelpful) thought was whether it would just be easier to always collect the examples on every run which might simply that code. It might not make sense, or not worth spending on more time on it, and in any case feel free to ignore me. :)

Thanks again!

[kevin-lee]

That minimum coverage with cover works really well so the test failed. 😆

[info] - hedgehog.examples.ReverseTest.reverse: Falsified after 100 passed tests
[info] > Insufficient coverage.
[info] > 95% nonempty 50% ✓ List(a)
[info] > 5% empty 50% ✗ List()

[kevin-lee]

@charleso I think it's good to merge it now and we can consider your thought afterwards.
Dale created this PR about a month ago and it's a nice feature to have so we should probably not spend more time on this.

[kevin-lee]

@dwijnand As Charles said, please feel free to merge it. If you want to merge it but can't do it, please let me know.

[kevin-lee]

v0.5.0 has been released.
https://search.maven.org/search?q=g:qa.hedgehog%20AND%20a:hedgehog-*%20AND%20v:0.5.0

[charleso]

No longer used? :)

[dwijnand]

Where did that come from? #175