From 6e12759574ce2f372eb6b1de09bca6899e877880 Mon Sep 17 00:00:00 2001 From: Lilith Orion Hafner Date: Fri, 23 Feb 2024 13:35:17 -0600 Subject: [PATCH 1/2] add some docs --- README.md | 73 +++++++++++++++++++++++++++++++++++++++++-- docs/src/index.md | 79 ++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 148 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index a8f01823..8ad48680 100644 --- a/README.md +++ b/README.md @@ -5,13 +5,80 @@ [![Build Status](https://github.com/LilithHafner/Chairmarks.jl/actions/workflows/CI.yml/badge.svg?branch=main)](https://github.com/LilithHafner/Chairmarks.jl/actions/workflows/CI.yml?query=branch%3Amain) [![Coverage](https://codecov.io/gh/LilithHafner/Chairmarks.jl/branch/main/graph/badge.svg)](https://codecov.io/gh/LilithHafner/Chairmarks.jl) -Benchmarks with back support. +Benchmarks with back support. Often hundreds of times faster than BenchmarkTools.jl without compromising on accuracy. + +## Precise + +Capable of detecting 1% difference in runtime in ideal conditions + +```julia +julia> f(n) = sum(rand() for _ in 1:n) +f (generic function with 1 method) + +julia> @b f(1000) +1.074 μs + +julia> @b f(1000) +1.075 μs + +julia> @b f(1000) +1.076 μs + +julia> @b f(1010) +1.086 μs + +julia> @b f(1010) +1.087 μs + +julia> @b f(1010) +1.087 μs +``` + +## Concise + +Chairmarks uses a concise pipeline syntax to define benchmarks. When providing a single argument, that argument is automatically wrapped in a function for higher performance and executed + +```julia +julia> @b sort(rand(100)) +1.500 μs (3 allocs: 2.625 KiB) +``` + +When providing two arguments, the first is setup code and only the runtime of the second is measured + +```julia +julia> @b rand(100) sort +1.018 μs (2 allocs: 1.750 KiB) +``` + +You may use `_` in the later arguments to refer to the output of previous arguments + +```julia +julia> @b rand(100) sort(_, by=x -> exp(-x)) +5.521 μs (2 allocs: 1.750 KiB) +``` + +A third argument can run a "teardown" function to integrate testing into the benchmark and ensure that the benchmarked code is behaving correctly + +```julia +julia> @b rand(100) sort(_, by=x -> exp(-x)) issorted(_) || error() +ERROR: +Stacktrace: + [1] error() +[...] + +julia> @b rand(100) sort(_, by=x -> exp(-x)) issorted(_, rev=true) || error() +5.358 μs (2 allocs: 1.750 KiB) +``` + +See the [docstring of `@b`](https://chairmarks.lilithhafner.com/dev/#Chairmarks.@b-Tuple) for more info + +## Efficient | | Chairmarks.jl | BenchmarkTools.jl | Ratio |-----------|--------|---------------|--------| -|Width | Narrow | Wide | 2–4x -|Back Support | Almost Always | Sometimes | N/A |[TTFX](contrib/ttfx_rm_rf_julia.sh) | 3.4s | 13.4s | 4x | Load time | 4.2ms | 131ms | 31x | TTFX excluding precompile time | 43ms | 1118ms | 26x | minimum runtime | 34μs | 459ms | 13,500x +|Width | Narrow | Wide | 2–4x +|Back Support | Almost Always | Sometimes | N/A diff --git a/docs/src/index.md b/docs/src/index.md index 3a83f530..f19adedf 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -4,7 +4,84 @@ CurrentModule = Chairmarks # Chairmarks -Documentation for [Chairmarks](https://github.com/LilithHafner/Chairmarks.jl). +[Chairmarks.jl](https://github.com/LilithHafner/Chairmarks.jl) provides benchmarks with back support. Often hundreds of times faster than BenchmarkTools.jl without compromising on accuracy. + +## Precise + +Capable of detecting 1% difference in runtime in ideal conditions + +```julia +julia> f(n) = sum(rand() for _ in 1:n) +f (generic function with 1 method) + +julia> @b f(1000) +1.074 μs + +julia> @b f(1000) +1.075 μs + +julia> @b f(1000) +1.076 μs + +julia> @b f(1010) +1.086 μs + +julia> @b f(1010) +1.087 μs + +julia> @b f(1010) +1.087 μs +``` + +## Concise + +Chairmarks uses a concise pipeline syntax to define benchmarks. When providing a single argument, that argument is automatically wrapped in a function for higher performance and executed + +```julia +julia> @b sort(rand(100)) +1.500 μs (3 allocs: 2.625 KiB) +``` + +When providing two arguments, the first is setup code and only the runtime of the second is measured + +```julia +julia> @b rand(100) sort +1.018 μs (2 allocs: 1.750 KiB) +``` + +You may use `_` in the later arguments to refer to the output of previous arguments + +```julia +julia> @b rand(100) sort(_, by=x -> exp(-x)) +5.521 μs (2 allocs: 1.750 KiB) +``` + +A third argument can run a "teardown" function to integrate testing into the benchmark and ensure that the benchmarked code is behaving correctly + +```julia +julia> @b rand(100) sort(_, by=x -> exp(-x)) issorted(_) || error() +ERROR: +Stacktrace: + [1] error() +[...] + +julia> @b rand(100) sort(_, by=x -> exp(-x)) issorted(_, rev=true) || error() +5.358 μs (2 allocs: 1.750 KiB) +``` + +See [`@b`](@ref) for more info + +## Efficient + +| | Chairmarks.jl | BenchmarkTools.jl | Ratio +|-----------|--------|---------------|--------| +|[TTFX](contrib/ttfx_rm_rf_julia.sh) | 3.4s | 13.4s | 4x +| Load time | 4.2ms | 131ms | 31x +| TTFX excluding precompile time | 43ms | 1118ms | 26x +| minimum runtime | 34μs | 459ms | 13,500x +|Width | Narrow | Wide | 2–4x +|Back Support | Almost Always | Sometimes | N/A + ```@index ``` From 717c375ea7c843f7961fc52de2165ec2a240fb8f Mon Sep 17 00:00:00 2001 From: Lilith Orion Hafner Date: Fri, 23 Feb 2024 13:40:36 -0600 Subject: [PATCH 2/2] fix link --- docs/src/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/index.md b/docs/src/index.md index f19adedf..8e6bb577 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -75,7 +75,7 @@ See [`@b`](@ref) for more info | | Chairmarks.jl | BenchmarkTools.jl | Ratio |-----------|--------|---------------|--------| -|[TTFX](contrib/ttfx_rm_rf_julia.sh) | 3.4s | 13.4s | 4x +|[TTFX](https://github.com/LilithHafner/Chairmarks.jl/blob/main/contrib/ttfx_rm_rf_julia.sh) | 3.4s | 13.4s | 4x | Load time | 4.2ms | 131ms | 31x | TTFX excluding precompile time | 43ms | 1118ms | 26x | minimum runtime | 34μs | 459ms | 13,500x