Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardize usage of collections.abc. #10679

Merged
merged 3 commits into from
Apr 18, 2022

Conversation

bdice
Copy link
Contributor

@bdice bdice commented Apr 18, 2022

The codebase currently uses multiple ways of importing collections.abc classes in cudf. This can be problematic because the names in collections.abc can overlap with names in typing, so we need a way to disambiguate the two. This PR standardizes our imports such that abstract base classes follow a pattern so that abc. is always in the name of abstract base classes.

from collections import abc
# Not "import collections.abc" or "from collections.abc import Mapping"

if isinstance(obj, abc.Mapping):
    pass

@bdice bdice added code quality Python Affects Python cuDF API. improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Apr 18, 2022
@bdice bdice requested a review from a team as a code owner April 18, 2022 15:01
@bdice bdice self-assigned this Apr 18, 2022
@bdice bdice requested review from trxcllnt and rgsl888prabhu April 18, 2022 15:01
@@ -279,7 +279,7 @@ cdef csv_reader_options make_csv_reader_options(
)
csv_reader_options_c.set_dtypes(c_dtypes_list)
csv_reader_options_c.set_parse_hex(c_hex_col_indexes)
elif isinstance(dtype, abc.Iterable):
elif isinstance(dtype, abc.Collection):
Copy link
Contributor Author

@bdice bdice Apr 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We must use Collection because an Iterable is not guaranteed to support len(dtype) below. Similarly in json.pyx.

Comment on lines 3004 to 3007
if isinstance(val, abc.Iterable):
idxs.update(val)
elif isinstance(val, str):
idxs.add(val)
Copy link
Contributor Author

@bdice bdice Apr 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The order of these statements may be a bug. Strings meet the requirements of Iterable, so the string code path may never be taken here. Strings should probably be handled first. I fixed this in 816319d.

@codecov
Copy link

codecov bot commented Apr 18, 2022

Codecov Report

Merging #10679 (6044133) into branch-22.06 (94a5d41) will increase coverage by 0.02%.
The diff coverage is 88.63%.

@@               Coverage Diff                @@
##           branch-22.06   #10679      +/-   ##
================================================
+ Coverage         86.38%   86.41%   +0.02%     
================================================
  Files               142      142              
  Lines             22334    22333       -1     
================================================
+ Hits              19294    19299       +5     
+ Misses             3040     3034       -6     
Impacted Files Coverage Δ
python/cudf/cudf/_fuzz_testing/json.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/df_protocol.py 88.48% <33.33%> (ø)
python/cudf/cudf/core/reshape.py 89.82% <80.00%> (-0.27%) ⬇️
python/cudf/cudf/testing/_utils.py 93.85% <80.00%> (ø)
python/cudf/cudf/api/types.py 89.79% <100.00%> (ø)
python/cudf/cudf/comm/gpuarrow.py 79.76% <100.00%> (-0.24%) ⬇️
python/cudf/cudf/core/column/categorical.py 89.77% <100.00%> (ø)
python/cudf/cudf/core/column_accessor.py 93.47% <100.00%> (ø)
python/cudf/cudf/core/cut.py 82.69% <100.00%> (ø)
python/cudf/cudf/core/dataframe.py 93.79% <100.00%> (+0.04%) ⬆️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 94a5d41...6044133. Read the comment docs.

Copy link
Contributor

@galipremsagar galipremsagar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, minor comment.

python/cudf/cudf/core/reshape.py Outdated Show resolved Hide resolved
@bdice bdice requested a review from galipremsagar April 18, 2022 22:01
@galipremsagar galipremsagar added the 5 - Ready to Merge Testing and reviews complete, ready to merge label Apr 18, 2022
@galipremsagar
Copy link
Contributor

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 6c79b59 into rapidsai:branch-22.06 Apr 18, 2022
@bdice bdice deleted the standardize-collections-abc branch April 19, 2022 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to Merge Testing and reviews complete, ready to merge improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants