Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement lists::concatenate_list_elements #8231

Merged
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
dc6d88d
Rename `concatenate_rows.hpp` into `combine.hpp` and add `concatenate…
ttnghia May 5, 2021
17b4533
Update doxygen
ttnghia May 7, 2021
1f53f06
WIP
ttnghia May 7, 2021
dff283c
Merge branch 'lists_concatenate_by_keys' into lists_concatenate_list_…
ttnghia May 10, 2021
de1c910
Rename files
ttnghia May 10, 2021
f55f35d
Complete doxygen
ttnghia May 10, 2021
4950406
Finish unit tests
ttnghia May 12, 2021
41ed5eb
Finish implementation
ttnghia May 12, 2021
216a8ab
Merge branch 'branch-0.20' into lists_concatenate_list_elements
ttnghia May 12, 2021
afce24e
Update path to `concatenate_rows` in java and python sides
ttnghia May 13, 2021
906a220
Merge branch 'branch-0.20' into lists_concatenate_list_elements
ttnghia May 14, 2021
9321c84
Merge branch 'branch-0.20' into lists_concatenate_list_elements
ttnghia May 18, 2021
215f777
Add a detail header which expose stream parameter
ttnghia May 18, 2021
b7c7b72
Consolidate `lists::concatenate_rows` using `lists::interleave_column…
ttnghia May 18, 2021
fde86d3
Reimplement list concatenation and ddd nested lists column to unit tests
ttnghia May 18, 2021
075bf41
Modify comments
ttnghia May 18, 2021
846fed3
Rewrite logical operations
ttnghia May 18, 2021
b35b9d9
Merge branch 'branch-21.06' into lists_concatenate_list_elements
ttnghia May 18, 2021
e76af2b
Change `concatenate_rows.pxd` to `combine.pxd`
ttnghia May 19, 2021
4b623e2
Rewrite doxygen
ttnghia May 20, 2021
0bb3be8
Fix a bug when the row is an empty list of list
ttnghia May 20, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion conda/recipes/libcudf/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ test:
- test -f $PREFIX/include/cudf/lists/detail/drop_list_duplicates.hpp
- test -f $PREFIX/include/cudf/lists/detail/interleave_columns.hpp
- test -f $PREFIX/include/cudf/lists/detail/sorting.hpp
- test -f $PREFIX/include/cudf/lists/concatenate_rows.hpp
- test -f $PREFIX/include/cudf/lists/combine.hpp
- test -f $PREFIX/include/cudf/lists/count_elements.hpp
- test -f $PREFIX/include/cudf/lists/explode.hpp
- test -f $PREFIX/include/cudf/lists/drop_list_duplicates.hpp
Expand Down
3 changes: 2 additions & 1 deletion cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,8 @@ add_library(cudf
src/join/join.cu
src/join/semi_join.cu
src/lists/contains.cu
src/lists/concatenate_rows.cu
src/lists/combine/concatenate_list_elements.cu
src/lists/combine/concatenate_rows.cu
src/lists/copying/concatenate.cu
src/lists/copying/copying.cu
src/lists/copying/gather.cu
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
namespace cudf {
namespace lists {
/**
* @addtogroup lists_concatenate_rows
* @addtogroup lists_combine
* @{
* @file
*/
Expand Down Expand Up @@ -53,16 +53,45 @@ enum class concatenate_null_policy { IGNORE, NULLIFY_OUTPUT_ROW };
*
* @param input Table of lists to be concatenated.
* @param null_policy The parameter to specify whether a null list element will be ignored from
* concatenation, or any concatenation involving a null list element will result in a null list.
* concatenation, or any concatenation involving a null element will result in a null list.
* @param mr Device memory resource used to allocate the returned column's device memory.
* @return A new column in which each row is a list resulted from concatenating all list elements in
* the corresponding row of the input table.
* the corresponding row of the input table.
*/
std::unique_ptr<column> concatenate_rows(
table_view const& input,
concatenate_null_policy null_policy = concatenate_null_policy::IGNORE,
rmm::mr::device_memory_resource* mr = rmm::mr::get_current_device_resource());

/**
* @brief Concatenating multiple lists on the same row of a lists column into a single list.
*
* Given a lists column where each row in the column is a list of lists of entries, an output lists
* column is generated by concatenating all the list elements at the same row together. If any row
* contains null list elements, the concatenation process will either ignore those null elements, or
* will simply set the entire resulting row to be a null element.
*
* @code{.pseudo}
* l = [ [{1, 2}, {3, 4}, {5}], [{6}, {}, {7, 8, 9}] ]
* r = lists::concatenate_list_elements(l);
* r is [ {1, 2, 3, 4, 5}, {6, 7, 8, 9} ]
* @endcode
*
* @throws cudf::logic_error if the input column is not a two-level depth lists column.
* @throws cudf::logic_error if the input lists column contains nested typed entry.
*
* @param input The lists column containing lists of list elements to concatenate.
* @param null_policy The parameter to specify whether a null list element will be ignored from
* concatenation, or any concatenation involving a null element will result in a null list.
* @param mr Device memory resource used to allocate the returned column's device memory.
* @return A new column in which each row is a list resulted from concatenating all list elements in
* the corresponding row of the input lists column.
*/
std::unique_ptr<column> concatenate_list_elements(
column_view const& input,
concatenate_null_policy null_policy = concatenate_null_policy::IGNORE,
rmm::mr::device_memory_resource* mr = rmm::mr::get_current_device_resource());

/** @} */ // end of group
} // namespace lists
} // namespace cudf
2 changes: 1 addition & 1 deletion cpp/include/doxygen_groups.h
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@
* @}
* @defgroup lists_apis Lists
* @{
* @defgroup lists_concatenate_rows Combining
* @defgroup lists_combine Combining
* @defgroup lists_extract Extracting
* @defgroup lists_contains Searching
* @defgroup lists_gather Gathering
Expand Down
Loading