From c5d184dc37c8f51edb86bc623b24461e3d669846 Mon Sep 17 00:00:00 2001 From: Bradley Dice Date: Thu, 20 Jan 2022 12:19:23 -0600 Subject: [PATCH 1/2] Update benchmarking guide to use nvbench. --- cpp/docs/BENCHMARKING.md | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/cpp/docs/BENCHMARKING.md b/cpp/docs/BENCHMARKING.md index ed99ff5f1be..6285528d121 100644 --- a/cpp/docs/BENCHMARKING.md +++ b/cpp/docs/BENCHMARKING.md @@ -1,19 +1,25 @@ # Unit Benchmarking in libcudf -Unit benchmarks in libcudf are written using [Google Benchmark](https://github.com/google/benchmark). +Unit benchmarks in libcudf are written using [NVBench](https://github.com/NVIDIA/nvbench). +While many existing benchmarks are written using +[Google Benchmark](https://github.com/google/benchmark), new benchmarks should use nvbench. -Google Benchmark provides many options for specifying ranges of parameters to benchmarks to test -with varying parameters, as well as to control the time unit reported, among other options. Refer to -other benchmarks in `cpp/benchmarks` to understand the options. +The NVBench library is similar to Google Benchmark, but has several quality of life improvements +when doing GPU benchmarking such as displaying the fraction of peak memory bandwidth achieved and +details about the GPU hardware. + +Both NVBench and Google Benchmark provide many options for specifying ranges of parameters to +benchmark, as well as to control the time unit reported, among other options. Refer to existing +benchmarks in `cpp/benchmarks` to understand the options. ## Directory and File Naming The naming of unit benchmark directories and source files should be consistent with the feature being benchmarked. For example, the benchmarks for APIs in `copying.hpp` should live in -`cudf/cpp/benchmarks/copying`. Each feature (or set of related features) should have its own +`cpp/benchmarks/copying`. Each feature (or set of related features) should have its own benchmark source file named `_benchmark.cu/cpp`. For example, -`cudf/cpp/src/copying/scatter.cu` has benchmarks in -`cudf/cpp/benchmarks/copying/scatter_benchmark.cu`. +`cpp/src/copying/scatter.cu` has benchmarks in +`cpp/benchmarks/copying/scatter_benchmark.cu`. In the interest of improving compile time, whenever possible, test source files should be `.cpp` files because `nvcc` is slower than `gcc` in compiling host code. Note that `thrust::device_vector` From 77e69b821d9003c8a00dd6d7f824d267f6d35d14 Mon Sep 17 00:00:00 2001 From: Bradley Dice Date: Thu, 20 Jan 2022 12:28:15 -0600 Subject: [PATCH 2/2] Update cpp/docs/BENCHMARKING.md --- cpp/docs/BENCHMARKING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cpp/docs/BENCHMARKING.md b/cpp/docs/BENCHMARKING.md index 6285528d121..a0df9917064 100644 --- a/cpp/docs/BENCHMARKING.md +++ b/cpp/docs/BENCHMARKING.md @@ -2,7 +2,7 @@ Unit benchmarks in libcudf are written using [NVBench](https://github.com/NVIDIA/nvbench). While many existing benchmarks are written using -[Google Benchmark](https://github.com/google/benchmark), new benchmarks should use nvbench. +[Google Benchmark](https://github.com/google/benchmark), new benchmarks should use NVBench. The NVBench library is similar to Google Benchmark, but has several quality of life improvements when doing GPU benchmarking such as displaying the fraction of peak memory bandwidth achieved and