Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Create gbenchmarks for cudf strings APIs #5698

Closed
19 tasks done
davidwendt opened this issue Jul 15, 2020 · 2 comments
Closed
19 tasks done

[FEA] Create gbenchmarks for cudf strings APIs #5698

davidwendt opened this issue Jul 15, 2020 · 2 comments
Assignees
@davidwendt
Labels
feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. Performance Performance related issue strings strings issues (C++ and Python) tests Unit testing for project

Comments

@davidwendt
Copy link
Contributor

davidwendt commented Jul 15, 2020 •
edited
Loading

There are currently no benchmarks for any of the strings-specific libcudf APIs.

Propose creating the following gbenchmarks:

This is not all the APIs but covers most if not all the key kernels that would show the performance impact of code changes here.
@davidwendt davidwendt added feature request New feature or request Needs Triage Need team to review and classify libcudf Affects libcudf (C++/CUDA) code. strings strings issues (C++ and Python) labels Jul 15, 2020
@davidwendt davidwendt changed the title Create gbenchmarks for cudf strings APIs [FEA] Create gbenchmarks for cudf strings APIs Jul 15, 2020
@harrism harrism added tech debt Performance Performance related issue tests Unit testing for project and removed Needs Triage Need team to review and classify labels Jul 19, 2020
@karthikeyann
Copy link
Contributor

For converters, benchmarks for from_durations, to_durations is already present
cpp/benchmarks/string/convert_durations_benchmark.cpp

@davidwendt davidwendt mentioned this issue Dec 18, 2020
Merged
rapids-bot bot pushed a commit that referenced this issue Jan 4, 2021
@davidwendt
Create sort gbenchmark for strings column(#7040)
af41136 
Reference #7027 and #5698

This adds a strings column to the current gbenchmark for sort. This will help measure improvements or changes over time to the column and strings comparator functions.

No code logic changed or added.

Authors:
  - davidwendt <[email protected]>

Approvers:
  - Vukasin Milovanovic
  - Devavret Makkar
  - Keith Kraus

URL: #7040
@davidwendt davidwendt mentioned this issue Feb 3, 2021
Merged
@davidwendt davidwendt self-assigned this Feb 4, 2021
@davidwendt davidwendt mentioned this issue Feb 4, 2021
Merged
rapids-bot bot pushed a commit that referenced this issue Feb 5, 2021
@davidwendt
Add UTF-8 chars to create_random_column<string_view> benchmark utility (
7e0437d 
#7292)

This updates the `create_random_column<string_view>` benchmark generate utility to support multi-byte UTF-8 characters. The original code only created columns with ASCII characters. The update also adds the space character which will be useful for text-based benchmarks in the future. Only 10 UTF-8 characters are included so a default distribution will still be mostly ASCII strings.

This will help in providing accurate measurements for adding benchmarks for strings APIs #5698.

This change also includes renaming `DURATION_TO_STRING_BENCH_SRC` to just `STRINGS_BENCH` since this benchmark will be folded into a general strings benchmark executable.

Authors:
  - David (@davidwendt)

Approvers:
  - Keith Kraus (@kkraus14)
  - Vukasin Milovanovic (@vuule)
  - Conor Hoekstra (@codereport)

URL: #7292
@davidwendt davidwendt mentioned this issue Feb 9, 2021
Closed
rapids-bot bot pushed a commit that referenced this issue Feb 10, 2021
@davidwendt
Add gbenchmark for cudf::strings::to_lower (#7316)
f7b3f75 
Reference #5698 
This creates a gbenchmark for the `cudf::strings::to_lower`. The device logic is the same for `cudf::strings::to_upper` and `cudf::strings::swapcase` so this a good measure for the 3 APIs.

This PR is dependent on changes in PR #7292 
These are mostly in the `generate_benchmark_input.cpp`

The initial results were as follows:
```
--------------------------------------------------------------------------------------------------
Benchmark                                        Time             CPU   Iterations UserCounters...
--------------------------------------------------------------------------------------------------
StringCase/strings/4096/manual_time          0.278 ms        0.296 ms         2514 bytes_per_second=248.756M/s
StringCase/strings/32768/manual_time         0.289 ms        0.307 ms         2421 bytes_per_second=1.86625G/s
StringCase/strings/262144/manual_time        0.419 ms        0.438 ms         1662 bytes_per_second=10.2869G/s
StringCase/strings/2097152/manual_time        2.59 ms         2.61 ms          269 bytes_per_second=13.3449G/s
StringCase/strings/16777216/manual_time       25.9 ms         25.9 ms           27 bytes_per_second=10.6531G/s
```

The `convert_case` code here is a bit old. I changed it to use the more efficient `make_strings_children` utility and found the performance improved by 2x

```
--------------------------------------------------------------------------------------------------
Benchmark                                        Time             CPU   Iterations UserCounters...
--------------------------------------------------------------------------------------------------
StringCase/strings/4096/manual_time          0.117 ms        0.135 ms         5877 bytes_per_second=592.795M/s
StringCase/strings/32768/manual_time         0.122 ms        0.140 ms         5641 bytes_per_second=4.42664G/s
StringCase/strings/262144/manual_time        0.274 ms        0.292 ms         2535 bytes_per_second=15.768G/s
StringCase/strings/2097152/manual_time        1.59 ms         1.61 ms          441 bytes_per_second=21.759G/s
StringCase/strings/16777216/manual_time       12.1 ms         12.1 ms           58 bytes_per_second=22.8626G/s
```

So these changes are also included in this PR.

Authors:
  - David (@davidwendt)

Approvers:
  - Conor Hoekstra (@codereport)
  - Vukasin Milovanovic (@vuule)
  - Mark Harris (@harrism)

URL: #7316
@davidwendt davidwendt mentioned this issue Feb 10, 2021
Merged
rapids-bot bot pushed a commit that referenced this issue Feb 26, 2021
@davidwendt
Add gbenchmark for strings split/split_record functions (#7427)
99f0b03 
Reference #5698
This creates a gbenchmark for the `cudf::strings::split` and `cudf::strings::split_record` functions.

This PR also includes changes to `split.cu` to use `device_uvector` for temporary buffers instead of `device_vector` and other minor code cleanup like adding more `const` decls.

Authors:
  - David (@davidwendt)

Approvers:
  - Keith Kraus (@kkraus14)
  - Mark Harris (@harrism)
  - Paul Taylor (@trxcllnt)

URL: #7427
rapids-bot bot pushed a commit that referenced this issue Feb 26, 2021
@davidwendt
Add gbenchmarks for strings filter functions (#7438)
862559f 
Reference #5698
This creates a gbenchmark for the `cudf::strings::filter_characters`, `cudf::strings::filter_characters_of_type`,  and `cudf::strings::strip` functions.

This PR also includes changes to `strip.cu` and `filter_chars` to use the more efficient `make_strings_children` utility. This improved performance on these functions by 2x on average.

Authors:
  - David (@davidwendt)

Approvers:
  - Keith Kraus (@kkraus14)
  - Conor Hoekstra (@codereport)
  - Karthikeyan (@karthikeyann)

URL: #7438
@davidwendt davidwendt mentioned this issue Mar 5, 2021
Merged
rapids-bot bot pushed a commit that referenced this issue Mar 9, 2021
@davidwendt
Add gbenchmarks for strings extract function (#7522)
4897a25 
Reference #5698
This creates a gbenchmark for `cudf::strings::extract` function. The benchmarks measures various sized rows as well as strings lengths. It also has measurements for small, medium, and large regex instructions. The extract performance is effected by the number of instructions in the regex pattern.

Authors:
  - David (@davidwendt)

Approvers:
  - Keith Kraus (@kkraus14)
  - Karthikeyan (@karthikeyann)
  - Mark Harris (@harrism)

URL: #7522
This was referenced Mar 9, 2021
Merged
Merged
Merged
@davidwendt davidwendt mentioned this issue Mar 16, 2021
Merged
rapids-bot bot pushed a commit that referenced this issue Mar 18, 2021
@davidwendt
Add gbenchmarks for string substrings functions (#7603)
4723051 
Reference #5698
This creates a gbenchmark for the 4 variations of `cudf::strings::slice_strings()` API. The benchmarks measures various sized rows as well as strings lengths. 
This PR also includes changes to `substring.cu` implementation cleaning up the code and using the more efficient `make_strings_children`. This change improved performance for all 4 functions on average by 2-3x.

Authors:
  - David (@davidwendt)

Approvers:
  - Nghia Truong (@ttnghia)
  - @nvdbaranec
  - Keith Kraus (@kkraus14)

URL: #7603
@davidwendt davidwendt mentioned this issue Mar 18, 2021
Merged
rapids-bot bot pushed a commit that referenced this issue Mar 18, 2021
@davidwendt
Add gbenchmark for strings::concatenate (#7560)
f5a4214 
Reference #5698
This creates a gbenchmark for `cudf::strings::concatenate` function. The benchmarks measures various sized rows as well as strings lengths. This PR also includes some changes to `combine.cu` for cleaning up the code and replacing `device_vector` usages with `device_uvector`.

Authors:
  - David (@davidwendt)

Approvers:
  - Nghia Truong (@ttnghia)
  - @nvdbaranec
  - Keith Kraus (@kkraus14)

URL: #7560
rapids-bot bot pushed a commit that referenced this issue Mar 22, 2021
@davidwendt
Add gbenchmark for converting strings to/from timestamps (#7641)
c21bd0e 
Reference #5698
This creates a gbenchmark for `cudf::strings::to_timestamps()` and `cudf::strings::from_timestamps()` functions. 
The benchmarks measures converting to/from a fixed format for days, seconds, ms, us, and ns.

Authors:
  - David (@davidwendt)

Approvers:
  - Nghia Truong (@ttnghia)
  - Conor Hoekstra (@codereport)
  - Vukasin Milovanovic (@vuule)
  - Jake Hemstad (@jrhemstad)

URL: #7641
rapids-bot bot pushed a commit that referenced this issue Mar 23, 2021
@davidwendt
Add gbenchmark for cudf::strings::translate function (#7617)
2bf22d1 
Reference #5698
This creates a gbenchmark for the `cudf::strings::translate()` API. The benchmarks measures various sized rows as well as strings lengths and translate table sizes.
This PR also includes changes to `translate.cu` implementation cleaning up the code and using the more efficient make_strings_children. This change improved performance for all 4 functions on average by 2-3x.
A further improvement was to sort the translation table input to more quickly lookup matches in device code. This added another 2x improvement when using longer translate tables.

Authors:
  - David (@davidwendt)

Approvers:
  - Nghia Truong (@ttnghia)
  - Mark Harris (@harrism)

URL: #7617
rapids-bot bot pushed a commit that referenced this issue Mar 23, 2021
@davidwendt
Add gbenchmarks for strings replace regex functions (#7541)
dd1f3ea 
Reference #5698
This creates a gbenchmark for `cudf::strings::replace_re` functions and the `cudf::strings::replace_with_backrefs` function. The benchmarks measures various sized rows as well as strings lengths. This PR also includes changes to source files for these 3 functions cleaning up the code and replacing `device_vector` usages with `device_uvector` and `device_span`.

Authors:
  - David (@davidwendt)

Approvers:
  - Mark Harris (@harrism)
  - Conor Hoekstra (@codereport)

URL: #7541
hyperbolic2346 pushed a commit to hyperbolic2346/cudf that referenced this issue Mar 25, 2021
@davidwendt @hyperbolic2346
Add gbenchmarks for strings extract function (rapidsai#7522)
4f702a0 
Reference rapidsai#5698
This creates a gbenchmark for `cudf::strings::extract` function. The benchmarks measures various sized rows as well as strings lengths. It also has measurements for small, medium, and large regex instructions. The extract performance is effected by the number of instructions in the regex pattern.

Authors:
  - David (@davidwendt)

Approvers:
  - Keith Kraus (@kkraus14)
  - Karthikeyan (@karthikeyann)
  - Mark Harris (@harrism)

URL: rapidsai#7522
This was referenced May 28, 2021
Merged
Merged
rapids-bot bot pushed a commit that referenced this issue Jun 2, 2021
@davidwendt
Add benchmark for strings/integers convert APIs (#8402)
1ba59bd 
Reference #5698
This PR adds benchmark tests for `cudf::strings::to_integers` and `cudf::strings::from_integers`. The current `convert_floats_benchmark.cpp` was refactored to work with floats and integer types and changed to `convert_numerics_benchmark.cpp`

Authors:
  - David Wendt (https://github.com/davidwendt)

Approvers:
  - Nghia Truong (https://github.com/ttnghia)
  - Mike Wilson (https://github.com/hyperbolic2346)

URL: #8402
rapids-bot bot pushed a commit that referenced this issue Jun 3, 2021
@davidwendt
Add benchmark for strings/fixed_point convert APIs (#8417)
347cf49 
Reference #5698
This PR adds benchmark tests for `cudf::strings::to_fixed_point` and `cudf::strings::from_fixed_point`.
The benchmarks include measurements for both `decimal32` and `decimal64` types.

Authors:
  - David Wendt (https://github.com/davidwendt)

Approvers:
  - Conor Hoekstra (https://github.com/codereport)
  - Mark Harris (https://github.com/harrism)

URL: #8417
@davidwendt
Copy link
Contributor Author

This is now complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@davidwendt davidwendt

Labels
feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. Performance Performance related issue strings strings issues (C++ and Python) tests Unit testing for project
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@harrism @karthikeyann @davidwendt