Add classifications to properties #253

felixmulder · 2019-03-16T19:16:33Z

Was recommended this great talk by John Hughes on labelling properties, thought that would be cool to have in Hedgehog too.

hedgehog/hedgehog.cabal

felixmulder · 2019-03-17T00:31:33Z

Tried a bunch of things to work around the bug in GHC 8.2.1. Couldn't get it unborken tonight. How important is 8.2.1 support? 8.2.2 works fine...

moodmosaic · 2019-03-17T19:35:13Z

Thanks for sending a pull request 👍 Consider adding a [WIP] on the title if it's not finalized (from what I can tell you keep adding commits). Also, considering squashing all those wat commits once you're done.

felixmulder · 2019-03-17T19:36:46Z

Lol, yes sorry - I wanted to fix the 8.2.1 issue before cleaning it up. I'll add the WIP tag

…

On Sun, 17 Mar 2019 at 20:35, Nikos Baxevanis ***@***.***> wrote: Thanks for sending a pull request 👍 Consider adding a *[WIP]* on the title if it's not finalized (from what I can tell you keep adding commits). Also, considering squashing all those wat commits once you're done. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#253 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABdYwVk9GBvb7KhgnLVMV6rP7PxFLCT0ks5vXpjygaJpZM4b3-3G> .

felixmulder · 2019-03-17T21:10:30Z

@moodmosaic - I ended up figuring out which line was offending 8.2.1. I filter it out on 8.2.x series, maybe there's a more precise way to do this, but I can't figure one out.

So tl;dr, ready for review

moodmosaic

Thanks for working on this 👍 🚀 I left some high level feedback (and some nitpicks) though I haven't gotten into the implementation details yet. Let's wait for @jystic first.

hedgehog/hedgehog.cabal

hedgehog/src/Hedgehog/Internal/Property.hs

hedgehog/src/Hedgehog/Internal/Runner.hs

felixmulder · 2019-03-18T08:33:35Z

@moodmosaic - I think I addressed your comments.

If this patch is accepted, I'd like to continue work on the feature. Things I'd like to do:

Allow turning coverage warning into failure
Refactor tuple to data (Journal was a good name 👍)

jacobstanley

Sorry for all the style nitpicks, I should really have a style guide. I didn't comment on all the abbreviations but you please find / fix them all. The codebase is pristine at the moment and I want to keep it that way. Generally this looks pretty good though, would love to have this feature!

Fwiw I'm happy to make the style fixups myself in another PR if you can't be bothered.

hedgehog/src/Hedgehog/Internal/Property.hs

hedgehog/src/Hedgehog/Internal/Report.hs

felixmulder · 2019-03-19T21:18:02Z

I'll amend the request in the morning (Stockholm)! Cheers for reviewing!

hedgehog/src/Hedgehog/Internal/Property.hs

felixmulder · 2019-03-20T06:52:56Z

@jystic - good morning, that should be all of it. Hope I didn't miss anything 🙏

hedgehog/src/Hedgehog/Internal/Report.hs

felixmulder · 2019-04-18T17:40:41Z

@jystic - rebased and ready for your verdict 😃

jacobstanley

Just a few items, would also like to make sure @HuwCampbell is happy enough with this before going ahead. I would like to align this with the approach taken in #262, but I don't think we need to hold this PR up because of that. All those changes wouldn't affect the public API so I don't mind if we do it over time.

hedgehog/src/Hedgehog/Internal/Property.hs

jacobstanley · 2019-04-19T23:20:01Z

hedgehog/src/Hedgehog/Internal/Property.hs

+classify :: (MonadTest m, HasCallStack) => Bool -> String -> m ()
+classify = cover 0
+
+cover :: (MonadTest m, HasCallStack) => Double -> Bool -> String -> m ()


I would like to have a comment for this as it will be visible in the haddock.

@mankyKitty maybe we should mention this in the style guide, public facing functions should have a comment. Internal ones I think it's developer's discretion.

Sounds good

hedgehog/src/Hedgehog/Internal/Runner.hs

felixmulder · 2019-04-20T07:03:43Z

@jystic - done 👍

jacobstanley · 2019-04-21T10:17:15Z

Great, so I started adding a few newtypes around Int and String and just fixing a few nits before merge and got a bit carried away, spent all day on it but the failure output is much prettier now. @moodmosaic and I spent a long time bikeshedding type names 😆

An example:

It also prints the current coverage while the test is running, you'll need to run the example to see. I moved the test to hedgehog-examples because we actually want them to fail so we can check the output.

felixmulder · 2019-04-21T10:58:23Z

hedgehog/src/Hedgehog/Internal/Property.hs

+coverageFailures tests (Coverage kvs) =
+  filter (not . classifierCovered tests) (Map.elems kvs)
+
+-- | Records the proportion of tests which satisfy a given condiiton.


condiiton

jacobstanley · 2019-04-21T20:12:03Z

@HuwCampbell I decided I will merge this and do the Log integration as a separate PR as it will be easier to review I think. I won't do a release until this happens so I don't think it's a big issue.

jacobstanley · 2019-04-21T23:04:51Z

Semigroup/Monoid 7.10 fun :)

Issue 71: #71 Taken from a combination of the following PRs: - hedgehogqa/haskell-hedgehog#253 - hedgehogqa/haskell-hedgehog#262

moodmosaic · 2022-08-08T10:35:27Z

hedgehog/src/Hedgehog/Internal/Property.hs

+    Map.singleton name (Classifier name mlocation minimum_ cover_)
+
+coverPercentage :: TestCount -> CoverCount -> CoverPercentage
+coverPercentage (TestCount tests) (CoverCount count) =


@felixmulder, couldn't we just write this as

coverPercentage :: TestCount -> CoverCount -> CoverPercentage coverPercentage (TestCount tests) (CoverCount count) = CoverPercentage (fromIntegral count / fromIntegral tests * 100)

What thousandths means?

Hmm, it does seem to me like that would be reasonable! I reckon this was done in order to get rid of decimals.

I'm trying to troubleshoot this, that was the reason I asked.

felixmulder commented Mar 16, 2019

View reviewed changes

hedgehog/hedgehog.cabal Outdated Show resolved Hide resolved

felixmulder force-pushed the topic/add-classifications branch 6 times, most recently from 2c04184 to 07548d7 Compare March 17, 2019 00:29

felixmulder changed the title ~~Add classifications to properties~~ [WIP] Add classifications to properties Mar 17, 2019

felixmulder force-pushed the topic/add-classifications branch from 188470b to a1c79e5 Compare March 17, 2019 20:50

felixmulder changed the title ~~[WIP] Add classifications to properties~~ Add classifications to properties Mar 17, 2019

moodmosaic reviewed Mar 18, 2019

View reviewed changes

jacobstanley requested changes Mar 19, 2019

View reviewed changes

jacobstanley reviewed Mar 19, 2019

View reviewed changes

hedgehog/src/Hedgehog/Internal/Report.hs Outdated Show resolved Hide resolved

felixmulder commented Mar 20, 2019

View reviewed changes

hedgehog/src/Hedgehog/Internal/Property.hs Outdated Show resolved Hide resolved

felixmulder force-pushed the topic/add-classifications branch 5 times, most recently from 8736454 to c001c80 Compare March 20, 2019 06:51

jacobstanley reviewed Mar 20, 2019

View reviewed changes

hedgehog/src/Hedgehog/Internal/Report.hs Outdated Show resolved Hide resolved

jacobstanley reviewed Mar 20, 2019

View reviewed changes

hedgehog/src/Hedgehog/Internal/Report.hs Outdated Show resolved Hide resolved

jacobstanley reviewed Mar 20, 2019

View reviewed changes

hedgehog/src/Hedgehog/Internal/Report.hs Outdated Show resolved Hide resolved

felixmulder force-pushed the topic/add-classifications branch from c001c80 to 59fc191 Compare March 20, 2019 07:02

felixmulder force-pushed the topic/add-classifications branch 2 times, most recently from 7dbd8f5 to 9e8723d Compare April 18, 2019 17:00

jacobstanley requested changes Apr 19, 2019

View reviewed changes

felixmulder force-pushed the topic/add-classifications branch from 9e8723d to 59d79a6 Compare April 20, 2019 06:42

felixmulder commented Apr 21, 2019

View reviewed changes

jacobstanley approved these changes Apr 21, 2019

View reviewed changes

Add classifications to properties

2bdf852

jacobstanley force-pushed the topic/add-classifications branch from b1e21bf to 1c08e08 Compare April 21, 2019 22:05

charleso mentioned this pull request Apr 21, 2019

Add classifications hedgehogqa/scala-hedgehog#71

Closed

jacobstanley force-pushed the topic/add-classifications branch from 1c08e08 to 451dc74 Compare April 21, 2019 22:32

More types, prettier output

d44f72a

jacobstanley force-pushed the topic/add-classifications branch from 451dc74 to d44f72a Compare April 21, 2019 22:47

jacobstanley merged commit ada4caa into hedgehogqa:master Apr 21, 2019

felixmulder deleted the topic/add-classifications branch April 22, 2019 06:20

charleso added a commit to hedgehogqa/scala-hedgehog that referenced this pull request Aug 22, 2019

Add classification/labelling of test cases

16a5dd2

Issue 71: #71 Taken from a combination of the following PRs: - hedgehogqa/haskell-hedgehog#253 - hedgehogqa/haskell-hedgehog#262

charleso added a commit to hedgehogqa/scala-hedgehog that referenced this pull request Aug 22, 2019

Add classification/labelling of test cases

316c27f

Issue 71: #71 Taken from a combination of the following PRs: - hedgehogqa/haskell-hedgehog#253 - hedgehogqa/haskell-hedgehog#262

charleso added a commit to hedgehogqa/scala-hedgehog that referenced this pull request Aug 22, 2019

Add classification/labelling of test cases

9637830

Issue 71: #71 Taken from a combination of the following PRs: - hedgehogqa/haskell-hedgehog#253 - hedgehogqa/haskell-hedgehog#262

charleso mentioned this pull request Aug 22, 2019

Add classification/labelling of test cases hedgehogqa/scala-hedgehog#110

Merged

charleso added a commit to hedgehogqa/scala-hedgehog that referenced this pull request Aug 22, 2019

Add classification/labelling of test cases

183bcba

Issue 71: #71 Taken from a combination of the following PRs: - hedgehogqa/haskell-hedgehog#253 - hedgehogqa/haskell-hedgehog#262

charleso added a commit to hedgehogqa/scala-hedgehog that referenced this pull request Aug 22, 2019

Add classification/labelling of test cases

4f15c4f

Issue 71: #71 Taken from a combination of the following PRs: - hedgehogqa/haskell-hedgehog#253 - hedgehogqa/haskell-hedgehog#262

charleso added a commit to hedgehogqa/scala-hedgehog that referenced this pull request Aug 22, 2019

Add classification/labelling of test cases

132657a

Issue 71: #71 Taken from a combination of the following PRs: - hedgehogqa/haskell-hedgehog#253 - hedgehogqa/haskell-hedgehog#262

charleso added a commit to hedgehogqa/scala-hedgehog that referenced this pull request Aug 22, 2019

Add classification/labelling of test cases

a00786c

Issue 71: #71 Taken from a combination of the following PRs: - hedgehogqa/haskell-hedgehog#253 - hedgehogqa/haskell-hedgehog#262

charleso added a commit to hedgehogqa/scala-hedgehog that referenced this pull request Aug 22, 2019

Add classification/labelling of test cases

4c5e355

Issue 71: #71 Taken from a combination of the following PRs: - hedgehogqa/haskell-hedgehog#253 - hedgehogqa/haskell-hedgehog#262

moodmosaic mentioned this pull request Jun 7, 2022

Observe test case distribution #127

Closed

moodmosaic reviewed Aug 8, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add classifications to properties #253

Add classifications to properties #253

felixmulder commented Mar 16, 2019 •

edited

Loading

felixmulder commented Mar 17, 2019

moodmosaic commented Mar 17, 2019

felixmulder commented Mar 17, 2019 via email

felixmulder commented Mar 17, 2019

moodmosaic left a comment

felixmulder commented Mar 18, 2019 •

edited

Loading

jacobstanley left a comment •

edited

Loading

felixmulder commented Mar 19, 2019

felixmulder commented Mar 20, 2019

felixmulder commented Apr 18, 2019

jacobstanley left a comment

jacobstanley Apr 19, 2019

mankyKitty Apr 20, 2019

felixmulder commented Apr 20, 2019

jacobstanley commented Apr 21, 2019

felixmulder Apr 21, 2019

jacobstanley Apr 21, 2019

jacobstanley commented Apr 21, 2019

jacobstanley commented Apr 21, 2019

moodmosaic Aug 8, 2022

felixmulder Aug 9, 2022

moodmosaic Aug 9, 2022

Add classifications to properties #253

Add classifications to properties #253

Conversation

felixmulder commented Mar 16, 2019 • edited Loading

felixmulder commented Mar 17, 2019

moodmosaic commented Mar 17, 2019

felixmulder commented Mar 17, 2019 via email

felixmulder commented Mar 17, 2019

moodmosaic left a comment

Choose a reason for hiding this comment

felixmulder commented Mar 18, 2019 • edited Loading

jacobstanley left a comment • edited Loading

Choose a reason for hiding this comment

felixmulder commented Mar 19, 2019

felixmulder commented Mar 20, 2019

felixmulder commented Apr 18, 2019

jacobstanley left a comment

Choose a reason for hiding this comment

jacobstanley Apr 19, 2019

Choose a reason for hiding this comment

mankyKitty Apr 20, 2019

Choose a reason for hiding this comment

felixmulder commented Apr 20, 2019

jacobstanley commented Apr 21, 2019

felixmulder Apr 21, 2019

Choose a reason for hiding this comment

jacobstanley Apr 21, 2019

Choose a reason for hiding this comment

jacobstanley commented Apr 21, 2019

jacobstanley commented Apr 21, 2019

moodmosaic Aug 8, 2022

Choose a reason for hiding this comment

felixmulder Aug 9, 2022

Choose a reason for hiding this comment

moodmosaic Aug 9, 2022

Choose a reason for hiding this comment

felixmulder commented Mar 16, 2019 •

edited

Loading

felixmulder commented Mar 18, 2019 •

edited

Loading

jacobstanley left a comment •

edited

Loading