Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Faster struct row comparator #10164

Merged
merged 58 commits into from
Mar 22, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
933c974
First commit
devavret Aug 26, 2021
a1636e5
testing and profiling deep single hierarchy struct
devavret Aug 27, 2021
d59f54c
Merge branch 'branch-22.02' into struct-row-comp
devavret Jan 12, 2022
765dd8d
Merge branch 'branch-22.02' into struct-row-comp
devavret Jan 12, 2022
3d21daf
Make the sandboxed test compile again
devavret Jan 14, 2022
9f32e6b
Update my row_comparator with nullate
devavret Jan 15, 2022
53d3c90
Merge branch 'branch-22.02' into struct-row-comp
devavret Jan 21, 2022
022e2a4
Basic verticalization utility and experimental namespace
devavret Jan 24, 2022
7fef643
clean up most of row operators that I didn't change.
devavret Jan 26, 2022
930d8de
Sliced column test
devavret Jan 27, 2022
0ecc4f8
column order and null precendence support
devavret Jan 28, 2022
ff36d2d
Manually managed stack
devavret Jan 28, 2022
cd0f938
New depth based method to avoid superimpose nulls
devavret Feb 2, 2022
7b8e060
Put sort2 impl in separate TU
devavret Feb 2, 2022
25eb237
Merge branch 'branch-22.04' into struct-row-comp
devavret Feb 2, 2022
d2937cf
Merge branch 'branch-22.04' into struct-row-comp
devavret Feb 10, 2022
d55c9c7
Move verticalization code to row_comparator.cpp
devavret Feb 15, 2022
3bd749e
Owning row lex operator
devavret Feb 22, 2022
613d664
merge fixes
devavret Feb 23, 2022
2ef3ac7
Move struct logic out of main row loop and into element_relational_co…
devavret Feb 24, 2022
5577431
pushing even more logic into element_relational_comparator
devavret Feb 24, 2022
f037bc0
More optimizations.
devavret Feb 24, 2022
8c54a85
review changes
devavret Feb 24, 2022
9d24a87
Checks to ensure tables can be compared
devavret Feb 24, 2022
294b0cf
Another attempt at new API
devavret Mar 2, 2022
a4c799a
Remove stack based struct comparator + cleanups
devavret Mar 7, 2022
ecb2eb0
thrust::pair -> cuda::std::pair
devavret Mar 7, 2022
34a6564
optional device spans
devavret Mar 7, 2022
fa4abb4
Prevent device comparator construction from any table_device_view
devavret Mar 7, 2022
b213210
Nullate default and fix for non nested depth
devavret Mar 7, 2022
6f9bedd
Fix an unsurfaced bug about depth passing
devavret Mar 7, 2022
be69ffa
Switch over sort impl to new comparator
devavret Mar 8, 2022
76d535a
Copyright changes to satiate ci
devavret Mar 8, 2022
78d10fc
Migrate struct sort benchmark to nvbench
devavret Mar 8, 2022
15920ee
Avoid optional::value in favor of *
devavret Mar 8, 2022
d01fc30
throw when trying to sort List
devavret Mar 8, 2022
ac2eb0d
Leftover change for struct sort nvbench
devavret Mar 8, 2022
076c4c1
struct without null pushdown test
devavret Mar 9, 2022
e8a9202
Remove temporary sort2_test
devavret Mar 9, 2022
a4b1167
Remove temporary sort2 files
devavret Mar 9, 2022
62f6914
leftover sort2 in cmake
devavret Mar 9, 2022
8f628ae
cleanup benchmark headers
devavret Mar 9, 2022
dc7d125
Docs
devavret Mar 9, 2022
fa7d940
Merge branch 'branch-22.04' into struct-row-comp
devavret Mar 10, 2022
76c883f
Apply suggestions from code review
devavret Mar 14, 2022
98b253b
rmm pool in benchmark + style fixes
devavret Mar 14, 2022
3255dc5
Merge branch 'branch-22.04' into struct-row-comp
devavret Mar 14, 2022
52e3a35
Review changes
devavret Mar 15, 2022
9470f06
More review changes
devavret Mar 15, 2022
7c897c3
Review changes req by @vyasr
devavret Mar 17, 2022
e0467c7
add a runtime is_relationally_comparable funtion
devavret Mar 17, 2022
fc1e993
Review changes
devavret Mar 18, 2022
096593f
Review changes
devavret Mar 18, 2022
f539647
Avoid WAR of storing a table_device_view
devavret Mar 18, 2022
01be0bc
Rename struct_linearize to decompose_structs and Improve docs
devavret Mar 18, 2022
de95530
review changes req by @ttnghia
devavret Mar 21, 2022
6c45cd4
Namespace changes and making element comparator private
devavret Mar 21, 2022
9bfd08e
Update cpp/include/cudf/table/experimental/row_operators.cuh
devavret Mar 22, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -470,6 +470,7 @@ add_library(
src/structs/structs_column_factories.cu
src/structs/structs_column_view.cpp
src/structs/utilities.cpp
src/table/row_operators.cu
src/table/table.cpp
src/table/table_device_view.cu
src/table/table_view.cpp
Expand Down
1 change: 1 addition & 0 deletions cpp/benchmarks/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ ConfigureBench(SEARCH_BENCH search/search.cpp)
# ##################################################################################################
# * sort benchmark --------------------------------------------------------------------------------
ConfigureBench(SORT_BENCH sort/rank.cpp sort/sort.cpp sort/sort_strings.cpp)
ConfigureNVBench(SORT_NVBENCH sort/sort_structs.cpp)

# ##################################################################################################
# * quantiles benchmark
Expand Down
84 changes: 84 additions & 0 deletions cpp/benchmarks/sort/sort_structs.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
/*
* Copyright (c) 2022, NVIDIA CORPORATION.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

#include <benchmarks/fixture/rmm_pool_raii.hpp>

#include <cudf/detail/sorting.hpp>

#include <cudf_test/column_utilities.hpp>
#include <cudf_test/column_wrapper.hpp>

#include <nvbench/nvbench.cuh>

#include <random>

void nvbench_sort_struct(nvbench::state& state)
{
cudf::rmm_pool_raii pool_raii;

using Type = int;
using column_wrapper = cudf::test::fixed_width_column_wrapper<Type>;
std::default_random_engine generator;
std::uniform_int_distribution<int> distribution(0, 100);

const cudf::size_type n_rows{static_cast<cudf::size_type>(state.get_int64("NumRows"))};
const cudf::size_type n_cols{1};
const cudf::size_type depth{static_cast<cudf::size_type>(state.get_int64("Depth"))};
const bool nulls{static_cast<bool>(state.get_int64("Nulls"))};

// Create columns with values in the range [0,100)
std::vector<column_wrapper> columns;
columns.reserve(n_cols);
std::generate_n(std::back_inserter(columns), n_cols, [&]() {
auto const elements = cudf::detail::make_counting_transform_iterator(
0, [&](auto row) { return distribution(generator); });
if (!nulls) return column_wrapper(elements, elements + n_rows);
auto valids =
cudf::detail::make_counting_transform_iterator(0, [](auto i) { return i % 10 != 0; });
return column_wrapper(elements, elements + n_rows, valids);
});

std::vector<std::unique_ptr<cudf::column>> cols;
std::transform(columns.begin(), columns.end(), std::back_inserter(cols), [](column_wrapper& col) {
return col.release();
});

std::vector<std::unique_ptr<cudf::column>> child_cols = std::move(cols);
// Lets add some layers
for (int i = 0; i < depth; i++) {
std::vector<bool> struct_validity;
std::uniform_int_distribution<int> bool_distribution(0, 100 * (i + 1));
std::generate_n(
std::back_inserter(struct_validity), n_rows, [&]() { return bool_distribution(generator); });
cudf::test::structs_column_wrapper struct_col(std::move(child_cols), struct_validity);
child_cols = std::vector<std::unique_ptr<cudf::column>>{};
child_cols.push_back(struct_col.release());
}

// Create table view
auto const input = cudf::table(std::move(child_cols));

state.exec(nvbench::exec_tag::sync, [&](nvbench::launch& launch) {
rmm::cuda_stream_view stream_view{launch.get_stream()};
cudf::detail::sorted_order(input, {}, {}, stream_view, rmm::mr::get_current_device_resource());
});
}

NVBENCH_BENCH(nvbench_sort_struct)
.set_name("sort_struct")
.add_int64_power_of_two_axis("NumRows", {10, 18, 26})
.add_int64_axis("Depth", {1, 8})
.add_int64_axis("Nulls", {0, 1});
Loading