Skip to content

Commit

Permalink
Add exception when trying to create large strings with cudf::test::st…
Browse files Browse the repository at this point in the history
…rings_column_wrapper (#16049)

Throws an exception in the `cudf::test::strings_column_wrapper` if the column size (accumulated offset values) would exceed max size_type.
Large strings created by the wrapper are not supported and discouraged due to the size and time impact on testing and CI.

Authors:
  - David Wendt (https://github.com/davidwendt)

Approvers:
  - Yunsong Wang (https://github.com/PointKernel)
  - Muhammad Haseeb (https://github.com/mhaseeb123)

URL: #16049
  • Loading branch information
davidwendt authored Jun 26, 2024
1 parent d53e409 commit bfaddd3
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion cpp/include/cudf_test/column_wrapper.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -314,7 +314,12 @@ auto make_chars_and_offsets(StringsIterator begin, StringsIterator end, Validity
for (auto str = begin; str < end; ++str) {
std::string tmp = (*v++) ? std::string(*str) : std::string{};
chars.insert(chars.end(), std::cbegin(tmp), std::cend(tmp));
offsets.push_back(offsets.back() + tmp.length());
auto const last_offset = static_cast<std::size_t>(offsets.back());
auto const next_offset = last_offset + tmp.length();
CUDF_EXPECTS(
next_offset < static_cast<std::size_t>(std::numeric_limits<cudf::size_type>::max()),
"Cannot use strings_column_wrapper to build a large strings column");
offsets.push_back(static_cast<cudf::size_type>(next_offset));
}
return std::pair(std::move(chars), std::move(offsets));
};
Expand Down

0 comments on commit bfaddd3

Please sign in to comment.