Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement concatenate_rows for list type #8049

Merged
merged 51 commits into from
May 3, 2021
Merged

Implement concatenate_rows for list type #8049

merged 51 commits into from
May 3, 2021

Conversation

ttnghia
Copy link
Contributor

@ttnghia ttnghia commented Apr 23, 2021 •
edited
Loading

This PR closes #7767. It implements lists::concatenate_rows that performs concatenation of all list elements at the same rows from the given table of list elements.

For example:

s1 = [{0, 1}, {2, 3, 4}, {5}, {}, {6, 7}]
s2 = [{8}, {9}, {}, {10, 11, 12}, {13, 14, 15, 16}]
r = lists::concatenate_rows( table_view{s1, s2} )
r is now [{0, 1, 8}, {2, 3, 4, 9}, {5}, {10, 11, 12}, {6, 7, 13, 14, 15, 16}]

Currently, only lists columns of one depth level are supported.

ttnghia added 30 commits April 13, 2021 16:47
@ttnghia
Add lists/interleave_columns.cuh|cu files
4e7336b
@ttnghia
Remove headers from interleave_columns_tests.cpp
4921a8d
@ttnghia
Add skeleton for interleave_columns that supports list_view
e61170e
@ttnghia
Finish doxygen and declaration for lists::interleave_columns
07beb16
@ttnghia
Add call to lists::interleave_columns
90f6973
@ttnghia
WIP for lists::interleave_columns
f70fb41
@ttnghia
Finish InvalidInput test
7aa2120
@ttnghia
Rewrite input validation
3065577
@ttnghia
Rewrite InvalidInput test and add skeleton for other tests
180cc21
@ttnghia
Rewrite ConcatenateEmptyColumns test
b7f994e
@ttnghia
Finish SimpleInputNoNull and SimpleInputStringsColumnsNoNull tests
c5a5814
@ttnghia
Finish SimpleInputWithNulls test
ea08f43
@ttnghia
Finish SimpleInputStringsColumnsWithNulls
83305be
@ttnghia
Finish SlicedColumnsInputNoNull and SlicedColumnsInputWithNulls tests
76c4876
@ttnghia
Rewrite doxygen
414dc9b
@ttnghia
Add a function to compute the offsets of the output lists column
dcf95ef
@ttnghia
Fix doxygen again
64dad79
@ttnghia
Finish skeleton for lists::detail::interleave_columns
b5dd17c
@ttnghia
Refactor
7908f64
@ttnghia
Implementation for fixed with
8c4b458
@ttnghia
Fix error
45ed175
@ttnghia
Draft
9ac0143
@ttnghia
Fix a bug in unit tests
a406543
@ttnghia
working
e1c0a5a
@ttnghia
Merge branch 'branch-0.20' into lists_interleave
00be57d
@ttnghia
Complete
ac31b2e
@ttnghia
Implement interleave_columns for list type
f9a02be
@ttnghia
Implement lists::concatenate_rows for row-wise list column concaten…
fdc5cd7 
…ation
@ttnghia
Use thrust::identity and thrust::copy
48a6b14
@ttnghia
Implement lists::concatenate_rows for row-wise list column concaten…
3d3b6e8 
…ation
@ttnghia ttnghia changed the title Implement concatenate_rows for list type [skip ci] Implement concatenate_rows for list type Apr 28, 2021
@ttnghia ttnghia requested a review from a team as a code owner April 28, 2021 19:37
@github-actions github-actions bot added the conda label Apr 28, 2021
@ttnghia ttnghia requested a review from devavret April 28, 2021 23:39
@ttnghia ttnghia requested a review from mythrocks April 30, 2021 15:49
@ttnghia
Copy link
Contributor Author

ttnghia commented May 3, 2021

Rerun tests.

@ttnghia ttnghia removed 0 - Blocked Cannot progress due to external reasons 5 - Merge After Dependencies CMake CMake build issue labels May 3, 2021
@codecov
Copy link

codecov bot commented May 3, 2021 •
edited
Loading

Codecov Report

Merging #8049 (718bdba) into branch-0.20 (51336df) will increase coverage by 0.01%.
The diff coverage is n/a.

❗ Current head 718bdba differs from pull request most recent head 1d29866. Consider uploading reports for the commit 1d29866 to get more accurate results
Impacted file tree graph

@@               Coverage Diff               @@
##           branch-0.20    #8049      +/-   ##
===============================================
+ Coverage        82.88%   82.90%   +0.01%     
===============================================
  Files              103      103              
  Lines            17668    17877     +209     
===============================================
+ Hits             14645    14821     +176     
- Misses            3023     3056      +33
Impacted Files Coverage Δ
python/cudf/cudf/core/abc.py 91.66% <ø> (+0.17%) ⬆️
python/cudf/cudf/core/algorithms.py 82.35% <ø> (ø)
python/cudf/cudf/core/column/__init__.py 100.00% <ø> (ø)
python/cudf/cudf/core/column/categorical.py 92.37% <ø> (+0.13%) ⬆️
python/cudf/cudf/core/column/column.py 88.20% <ø> (-0.44%) ⬇️
python/cudf/cudf/core/column/datetime.py 88.03% <ø> (-1.88%) ⬇️
python/cudf/cudf/core/column/decimal.py 90.29% <ø> (-2.64%) ⬇️
python/cudf/cudf/core/column/interval.py 91.66% <ø> (+0.55%) ⬆️
python/cudf/cudf/core/column/lists.py 86.98% <ø> (-0.43%) ⬇️
python/cudf/cudf/core/column/numerical.py 94.72% <ø> (+0.29%) ⬆️
... and 42 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6ab91f2...1d29866. Read the comment docs.

@ttnghia
Copy link
Contributor Author

ttnghia commented May 3, 2021

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 1debb96 into rapidsai:branch-0.20 May 3, 2021
@ttnghia ttnghia deleted the lists_concat branch May 3, 2021 21:52
@sperlingxx sperlingxx mentioned this pull request May 6, 2021
Merged
rapids-bot bot pushed a commit that referenced this pull request May 7, 2021
@sperlingxx
Support listConcatenateByRows in Java package (#8171)
5f9dade 
Current PR is to provide Java API for `cudf::lists::concatenate_rows`, which is added in #8049.

Authors:
  - Alfred Xu (https://github.com/sperlingxx)

Approvers:
  - Jason Lowe (https://github.com/jlowe)
  - Robert (Bobby) Evans (https://github.com/revans2)

URL: #8171
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@davidwendt davidwendt davidwendt left review comments

@raydouglass raydouglass raydouglass approved these changes

@devavret devavret devavret approved these changes

@robertmaynard robertmaynard robertmaynard approved these changes

@harrism harrism Awaiting requested review from harrism

@mythrocks mythrocks Awaiting requested review from mythrocks

Assignees

@ttnghia ttnghia

Labels
3 - Ready for Review Ready for review by team feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[FEA] concatenate lists together
8 participants
@ttnghia @robertmaynard @harrism @devavret @raydouglass @mythrocks @davidwendt @karthikeyann