Skip to content

Commit

Permalink
Improve documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
rabuckley authored Nov 13, 2024
1 parent 7a0516a commit b9a8b04
Show file tree
Hide file tree
Showing 23 changed files with 1,175 additions and 102 deletions.
138 changes: 138 additions & 0 deletions readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
# QuickCheck for .NET

From the original [Haskell version](https://hackage.haskell.org/package/QuickCheck),

> QuickCheck is a library for random testing of program properties. The programmer provides a specification of the program, in the form of properties which functions should satisfy, and QuickCheck then tests that the properties hold in a large number of randomly generated cases.
This library aims to provide similar property testing functionality for .NET.

## Installation

QuickCheck is available as a [NuGet package](https://www.nuget.org/packages/QuickCheck/). You can add it to an existing project as you would any other public nuget package. Using the .NET CLI:

```shell
dotnet add package QuickCheck
```

## Example

Say I've written a function to reverse `Memory<int>`. A property of a working reverse function is that reversing a reversed sequence should give the original sequence. We can check that this is the case for our function using QuickCheck.

```csharp
using QuickCheck;

// The function to test
static Memory<T> Reverse<T>(Memory<T> memory)
{
var span = memory.Span;
var reversed = new T[span.Length];

for (var i = 0; i < span.Length; i++)
{
reversed[span.Length - i - 1] = span[i];
}

return new Memory<T>(reversed);
}

var qc = QuickChecker.CreateEmpty();

// Create and add a generator for `Memory<int>`
var generator = new ArbitraryMemoryGenerator<int>(
ArbitraryInt32Generator.Default,
Random.Shared,
maximumSize: 32);

qc.AddGenerator(generator);

// Test function and validate that for all Memory<int> (Reverse(Reverse(memory)) == memory)
var result = qc.Run(
target: static (Memory<int> memory) => Reverse(Reverse(memory)).Span.SequenceEqual(memory.Span),
validate: static (result) => result);

Console.WriteLine(result.Type); // Success
```

## Usage

The main entry point for interacting with the library is the `QuickChecker` class.

To construct an instance with the default generators pre-registered, use an overload of the `CreateDefault` method.

```csharp
using QuickCheck;

// Using the default options
var qc = QuickChecker.CreateDefault();

// Using custom options
var configuredQc = QuickChecker.CreateDefault(options =>
{
options.RunCount = 1000;
options.Random = new Random(42);
});
```

To construct an instance with no pre-registered generators, use one of the `CreateEmpty` overloads.

```csharp
using QuickCheck;

// Using the default options
var qc = QuickChecker.CreateEmpty();

// Using custom options
var configuredQc = QuickChecker.CreateEmpty(options =>
{
options.RunCount = 1000;
options.Random = new Random(42);
});
```

### Generators

To test your functions, the `QuickChecker` must be able to generate values for every argument type of the target function.

This library provides a number of built-in generators for common types, including many numeric types, `char`, and `string`. These are added when using `CreateDefault` overloads.

Generic generators are available for `List<T>`, `Memory<T>`, and tuples, when given a generator for `T`. These must be added manually.

To add a generator to the `QuickChecker` instance, use the `AddGenerator` method:

```csharp
using QuickCheck.Generators;

var qc = QuickChecker.CreateEmpty();
qc.AddGenerator(ArbitraryInt32Generator.Default);
```

### Testing a Function

With the `QuickChecker` configured, you can test a function by calling the `Run` method.

Say I wrote a method to divide two integers, but return 0 if the divisor is 0. However, I forgot to handle the case where the divisor is 0.

```csharp
using QuickCheck;

// If b == 0, return 0, else return a / b. Oops!
static int NotSoSafeDivide(int a, int b) => a / b;

var qc = QuickChecker.CreateDefault();

var result = qc.Run(static (int a, int b) => NotSoSafeDivide(a, b));

Console.WriteLine(result.IsError); // True
Console.WriteLine(result.Exception is DivideByZeroException); // True
```

## Resources

The original QuickCheck library in Haskell:

- [Haskell's QuickCheck](https://hackage.haskell.org/package/QuickCheck)

I learned about QuickCheck from Jon Gjengset's great video on the Rust implementation:

- [Jon Gjengset YouTube video on QuickCheck in Rust](https://www.youtube.com/watch?v=64t-gPC33cc)
- [QuickCheck in Rust](https://github.com/BurntSushi/quickcheck)
25 changes: 16 additions & 9 deletions src/QuickCheck/ArbitraryValueGenerator.cs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ protected ArbitraryValueGenerator(Random random)
Random = random;
}

/// <summary>
/// The pseudo-random number generator to use when generating values.
/// </summary>
protected Random Random { get; }

/// <summary>
Expand All @@ -18,22 +21,26 @@ protected ArbitraryValueGenerator(Random random)
/// <summary>
/// Chooses a random <see cref="T"/> from <paramref name="options"/>.
/// </summary>
/// <param name="options"></param>
/// <param name="options">The options to choose from.</param>
public virtual T Choose(ReadOnlySpan<T> options)
{
return Random.GetItems(options, 1)[0];
}

/// <summary>
/// <para>
/// Shrinks the value <paramref name="from"/> to a smaller value.
/// </para>
/// <para>
/// If a value cannot be shrunk, an empty enumerable should be returned.
/// </para>
/// </summary>
/// <param name="from">The value to shrink from.</param>
/// <returns>
/// An enumerable of values smaller than <paramref name="from"/>.
/// </returns>
public virtual IEnumerable<T> Shrink(T from)
{
return [];
}
}

public abstract class ArbitraryMultipleValueGenerator<T, TCollection> : ArbitraryValueGenerator<TCollection>
where TCollection : IEnumerable<T>
{
protected ArbitraryMultipleValueGenerator(Random random) : base(random)
{
}
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
using System.Diagnostics.CodeAnalysis;

namespace QuickCheck;
namespace QuickCheck.Exceptions;

/// <summary>
/// Thrown when a <see cref="QuickChecker"/> is missing a required <see cref="IArbitraryValueGenerator{T}"/>.
Expand All @@ -12,16 +12,16 @@ public sealed class MissingGeneratorException : Exception
/// </summary>
public Type Type { get; }

public MissingGeneratorException(string message, Type type) : base(message)
private MissingGeneratorException(string message, Type type) : base(message)
{
Type = type;
}

[DoesNotReturn]
public static void Throw(Type type)
internal static void Throw(Type type)
{
throw new MissingGeneratorException(
$"No generator is registered for type '{type}'. Add one to your {nameof(QuickChecker)} before running.",
$"No generator is registered for type '{type.Name}'. Add one to your {nameof(QuickChecker)} before running.",
type);
}
}
}
41 changes: 41 additions & 0 deletions src/QuickCheck/Exceptions/QuickCheckRunException.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
using System.Diagnostics.CodeAnalysis;

namespace QuickCheck.Exceptions;

/// <summary>
/// Thrown when an unexpected error occurs during the execution of a
/// <see cref="QuickChecker"/> test run.
/// </summary>
public sealed class QuickCheckRunException : Exception
{
private QuickCheckRunException(string message) : base(message)
{
}

private QuickCheckRunException(string message, Exception innerException) :
base(message, innerException)
{
}

[DoesNotReturn]
internal static void Throw(string message)
{
throw new QuickCheckRunException(message);
}

[DoesNotReturn]
internal static void Throw(string message, Exception innerException)
{
throw new QuickCheckRunException(message, innerException);
}

[DoesNotReturn]
internal static void ThrowValidationFunctionThrew(
string input,
Exception exception)
{
Throw(
$"The provided validation function threw on the output target method with input '{input}'.",
exception);
}
}
8 changes: 4 additions & 4 deletions src/QuickCheck/Generators/ArbitraryMemoryGenerator.cs
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,18 @@ namespace QuickCheck.Generators;
public class ArbitraryMemoryGenerator<T> : ArbitraryValueGenerator<Memory<T>>
{
private readonly ArbitraryValueGenerator<T> _valueGenerator;
private readonly int _size;
private readonly int _maximumSize;

public ArbitraryMemoryGenerator(ArbitraryValueGenerator<T> valueGenerator, Random random, int size) :
public ArbitraryMemoryGenerator(ArbitraryValueGenerator<T> valueGenerator, Random random, int maximumSize) :
base(random)
{
_valueGenerator = valueGenerator;
_size = size;
_maximumSize = maximumSize;
}

public override Memory<T> Generate()
{
var length = Random.Next(0, _size);
var length = Random.Next(0, _maximumSize);
var array = new T[length];

for (var i = 0; i < length; i++)
Expand Down
49 changes: 49 additions & 0 deletions src/QuickCheck/Generators/ArbitraryNullableClassGenerator.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
namespace QuickCheck.Generators;

/// <summary>
/// A generator for nullable classes.
/// </summary>
/// <typeparam name="T"></typeparam>
public sealed class ArbitraryNullableClassGenerator<T> : ArbitraryValueGenerator<T?>
where T : class
{
private readonly IArbitraryValueGenerator<T> _generator;
private readonly double _nullProbability;

/// <summary>
/// Initializes a new instance of the <see cref="ArbitraryNullableClassGenerator{T}"/> class.
/// </summary>
/// <param name="generator">A generator for non-nullable instances of <typeparamref name="T"/>.</param>
/// <param name="random">A pseudo-random number generator.</param>
/// <param name="nullProbability">The probability of generating a null value.</param>
public ArbitraryNullableClassGenerator(
IArbitraryValueGenerator<T> generator,
Random random,
double nullProbability = 0.25) : base(random)
{
_generator = generator;
_nullProbability = nullProbability;
}

public override T? Generate()
{
return Random.NextDouble() < _nullProbability
? null
: _generator.Generate();
}

public override IEnumerable<T?> Shrink(T? from)
{
if (from is null)
{
yield break;
}

yield return null;

foreach (var shrunk in _generator.Shrink(from))
{
yield return shrunk;
}
}
}
43 changes: 43 additions & 0 deletions src/QuickCheck/Generators/ArbitraryNullableStructGenerator.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
namespace QuickCheck.Generators;

/// <summary>
/// A generator for nullable structs.
/// </summary>
/// <typeparam name="T"></typeparam>
public sealed class ArbitraryNullableStructGenerator<T> : ArbitraryValueGenerator<T?>
where T : struct
{
private readonly IArbitraryValueGenerator<T> _generator;
private readonly double _nullProbability;

public ArbitraryNullableStructGenerator(
IArbitraryValueGenerator<T> generator,
Random random,
double nullProbability = 0.25) : base(random)
{
_generator = generator;
_nullProbability = nullProbability;
}

public override T? Generate()
{
return Random.NextDouble() < _nullProbability
? null
: _generator.Generate();
}

public override IEnumerable<T?> Shrink(T? from)
{
if (from is null)
{
yield break;
}

yield return null;

foreach (var shrunk in _generator.Shrink(from.Value))
{
yield return shrunk;
}
}
}
Loading

0 comments on commit b9a8b04

Please sign in to comment.