Show fscheck num tests and distribution on test pass

[rynoV]

Previously for passing fscheck properties, the number of tests passed wouldn't be shown, and there was no option to show the test case distribution (e.g. percentage trivial tests, etc)

Now quietOnSuccess config can be set to false to show this info. New maxRejected config is also accepted

More generally, tests are enabled to show any message on pass via the PassWithMessage exception. This is pretty hacky so this is more of a proof of concept. The alternative was changing the core test types to optionally return a message, which seemed like too big of a change to make on my own.

[farlee2121]

I feel unsure about using pass exceptions. If we do go that route, I think it'd be better to give TestResult.Passed an optional message rather than have two union cases for passed tests. Multiple pass cases feels like it'll lead to easy mistakes. The same apply for multiple pass printers: It'd be better to have one pass printer with an optional message parameter

A notable con to some PassException is that we wouldn't be able to hide it, and then it would be available for other assertions. The message might add temptation to use the PassException even when execution should not be shortcutted

What are some alternative solutions?

• OPT: Using the logger to print the message
  • I'm trying to deglobalize the logger, and the property test methods don't have a ready way to compose the logger without user involvement
• OPT: As you mentioned, Return an optional message from test methods
  • This is a high-impact change. It'd also be a bit roundabout since none of the Check methods appear to return the test data directly.
• OPT: Call Check.One directly
• OPT: Create an FsCheck assertion that returns test distribution data
• Others?

Some questions that might help navigate these choices:

• How often do you want this distribution data?
• What does it help you do?
• Is there another way to achieve the same top-level value you need?

[rynoV]

I really only need to see the distribution as a debug tool, I only need to see the distribution in the console during development, so using the logger if possible sounds good. I'm not sure of the best way to handle the logger. Currently my tests are creating the Expecto logger at startup and passing it down through all the tests, wrapped in the logger API required by the app code.

I updated the PR with a solution that would work for my use case, which is for the property test config to include a logger field, and if that is set it is used when tests pass, but not tied to this

[farlee2121]

Such a logger on FsCheckConfig breaks Information Hiding.
Callers have to know about the internals of the library to understand what the parameter will do.
Even if they understand what it does, it only has a very specific application.

Do you need this diagnostic information often?

[rynoV]

Ideally every property test would show at least the number of tests that passed, so that's frequent but more of a "nice to have". Test distribution is less frequently needed.

How about adding the TestResult to FsCheckConfig.finishedTest? Then I could use my logger as needed, and it would enable the user to handle each of the different test result types.

Or if it's not desirable to expose FsCheck's type directly, the minimum I would want is the number of tests int and stamps (int * string list) seq. Or maybe we make an Expecto.FsCheckTestResult that mirrors fscheck's TestResult?