Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] Fix repr and concat of StructColumn #10042

Merged
merged 8 commits into from
Jan 14, 2022

Conversation

galipremsagar
Copy link
Contributor

Fixes: #8963

This PR fixes a trivial issue in concat where the assumption was that _with_type_metadata is an in-place operation, but it isn't.

@galipremsagar galipremsagar added bug Something isn't working 3 - Ready for Review Ready for review by team Python Affects Python cuDF API. non-breaking Non-breaking change labels Jan 13, 2022
@galipremsagar galipremsagar self-assigned this Jan 13, 2022
@galipremsagar galipremsagar requested a review from a team as a code owner January 13, 2022 19:45
@galipremsagar galipremsagar changed the title [REVIEW] Fix a repr and concat of StructColumn [REVIEW] Fix repr and concat of StructColumn Jan 13, 2022
python/cudf/cudf/core/dataframe.py Outdated Show resolved Hide resolved
python/cudf/cudf/core/dataframe.py Outdated Show resolved Hide resolved
python/cudf/cudf/core/dataframe.py Show resolved Hide resolved
@vyasr vyasr self-requested a review January 13, 2022 20:50
python/cudf/cudf/core/dataframe.py Outdated Show resolved Hide resolved
Comment on lines +1556 to +1557
cudf.core.column.Decimal64Column,
cudf.core.column.StructColumn,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these two types handled the same way? What do they have in common?

Are there other types that might need to be handled here -- and how would developers know to put them here? This may not be something that can be addressed in a code comment, but it seems important to consider. The StructColumn type was seemingly left out from a place it should have been included, until an issue was raised, cause identified, and bug fixed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a common pattern:

# TODO: _copy_type_metadata is a common pattern to apply after the
# roundtrip from libcudf. We should build this into a factory function
# to increase reusability.
result._copy_type_metadata(self)

To answer why only these two types are being handled separately is because _reassign_categories predates _with_type_metadata and will likely need a cleanup with some careful benchmarking to eliminate _reassign_categories which are out of the scope of this PR.

@codecov
Copy link

codecov bot commented Jan 13, 2022

Codecov Report

Merging #10042 (7fbf59b) into branch-22.02 (967a333) will decrease coverage by 0.07%.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff                @@
##           branch-22.02   #10042      +/-   ##
================================================
- Coverage         10.49%   10.41%   -0.08%     
================================================
  Files               119      119              
  Lines             20305    20538     +233     
================================================
+ Hits               2130     2139       +9     
- Misses            18175    18399     +224     
Impacted Files Coverage Δ
python/custreamz/custreamz/kafka.py 29.16% <0.00%> (-0.63%) ⬇️
python/dask_cudf/dask_cudf/sorting.py 92.66% <0.00%> (-0.25%) ⬇️
python/dask_cudf/dask_cudf/core.py 70.85% <0.00%> (-0.17%) ⬇️
python/cudf/cudf/__init__.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/frame.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/index.py 0.00% <0.00%> (ø)
python/cudf/cudf/io/parquet.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/series.py 0.00% <0.00%> (ø)
python/cudf/cudf/utils/utils.py 0.00% <0.00%> (ø)
python/cudf/cudf/utils/dtypes.py 0.00% <0.00%> (ø)
... and 21 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b01c846...7fbf59b. Read the comment docs.

@galipremsagar
Copy link
Contributor Author

rerun tests

Copy link
Contributor

@bdice bdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I copied the comment from DataFrame._concat to Series._concat and committed that suggestion. LGTM!

@galipremsagar galipremsagar added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 3 - Ready for Review Ready for review by team labels Jan 14, 2022
@galipremsagar
Copy link
Contributor Author

@gpucibot merge

@galipremsagar
Copy link
Contributor Author

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 12adb8a into rapidsai:branch-22.02 Jan 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to Merge Testing and reviews complete, ready to merge bug Something isn't working non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] nested column names of a cuDF dataframe created by read_orc are not printed
3 participants