Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[REVIEW] Upgrade cudf to support pandas 1.4.x versions #10584

Merged
merged 50 commits into from
May 9, 2022
Merged

[REVIEW] Upgrade cudf to support pandas 1.4.x versions #10584

merged 50 commits into from
May 9, 2022

Conversation

galipremsagar
Copy link
Contributor

@galipremsagar galipremsagar commented Apr 4, 2022 •
edited
Loading

This PR:

  • Upgrades pandas version to 1.4.x
  • Raise warning for shift when the fill_value is not of same type.
  • In CategoricalColumn.fillna if there is a mismatch of dtypes, the error is changed to TypeError from ValueError
  • Improvised data & index length mismatch error messages to match pandas.
  • Change default value of skina for a number of operations to True from None.
  • Implemented RangeIndex.sort_values for API compatibility and avoiding materializations.
  • Fix issues in Series constructor where name parameter was not being used when passed explicitly.
  • Fix name handling in Series.reset_index to match pandas behavior.
  • Removed a lot of special casing's in pytests.
  • xfailed some pytests due to pandas regressions.
  • In total fixed 500 pytest failures:

Prev:

= 508 failed, 80961 passed, 2115 skipped, 1187 xfailed, 1948 xpassed, 6880 warnings, 17 errors =

Now:

= 81841 passed, 2059 skipped, 1255 xfailed, 1857 xpassed, 5773 warnings =

@galipremsagar galipremsagar added 2 - In Progress Currently a work in progress improvement Improvement / enhancement to an existing function breaking Breaking change labels Apr 4, 2022
@galipremsagar galipremsagar self-assigned this Apr 4, 2022
@github-actions github-actions bot added conda Python Affects Python cuDF API. labels Apr 4, 2022
@codecov
Copy link

codecov bot commented Apr 4, 2022 •
edited
Loading

Codecov Report

Merging #10584 (05f0a54) into branch-22.06 (8d861ce) will decrease coverage by 0.09%.
The diff coverage is 94.93%.

@@               Coverage Diff                @@
##           branch-22.06   #10584      +/-   ##
================================================
- Coverage         86.40%   86.31%   -0.10%     
================================================
  Files               143      143              
  Lines             22448    22514      +66     
================================================
+ Hits              19396    19432      +36     
- Misses             3052     3082      +30
Impacted Files Coverage Δ
python/cudf/cudf/core/frame.py 93.41% <ø> (ø)
python/cudf/cudf/core/indexed_frame.py 91.70% <ø> (ø)
python/cudf/cudf/core/index.py 92.06% <88.88%> (-0.25%) ⬇️
python/cudf/cudf/core/dataframe.py 93.77% <96.29%> (+0.08%) ⬆️
python/cudf/cudf/core/_compat.py 100.00% <100.00%> (ø)
python/cudf/cudf/core/column/categorical.py 89.97% <100.00%> (ø)
python/cudf/cudf/core/reshape.py 90.11% <100.00%> (ø)
python/cudf/cudf/core/series.py 95.17% <100.00%> (+<0.01%) ⬆️
...thon/dask_cudf/dask_cudf/tests/test_distributed.py 18.86% <0.00%> (-67.93%) ⬇️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8d861ce...05f0a54. Read the comment docs.

@galipremsagar galipremsagar requested review from bdice and shwina May 5, 2022 21:36
@galipremsagar galipremsagar added the 5 - DO NOT MERGE Hold off on merging; see PR for details label May 5, 2022
bdice
bdice approved these changes May 6, 2022
View reviewed changes
Copy link
Contributor

@bdice bdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @galipremsagar! I replied above on one question about numpy pinning. Otherwise LGTM.

galipremsagar
galipremsagar commented May 7, 2022
View reviewed changes
ci/gpu/build.sh Outdated Show resolved Hide resolved
@github-actions github-actions bot removed the gpuCI label May 7, 2022
@galipremsagar galipremsagar removed the 5 - DO NOT MERGE Hold off on merging; see PR for details label May 9, 2022
@galipremsagar galipremsagar added the 5 - Ready to Merge Testing and reviews complete, ready to merge label May 9, 2022
@galipremsagar
Copy link
Contributor Author

rerun tests

2 similar comments
@galipremsagar
Copy link
Contributor Author

rerun tests

@galipremsagar
Copy link
Contributor Author

rerun tests

@galipremsagar
Copy link
Contributor Author

@gpucibot merge

@rapids-bot rapids-bot bot merged commit ac7492e into rapidsai:branch-22.06 May 9, 2022
rapids-bot bot pushed a commit that referenced this pull request May 16, 2022
@karthikeyann
update mangle_dupe_cols behavior in csv reader to match pandas 1.4.0 …
e58d049 
…behavior (#10749)

Fixes #10618

Depends on   #10584

Authors:
  - Karthikeyan (https://github.com/karthikeyann)

Approvers:
  - GALI PREM SAGAR (https://github.com/galipremsagar)
  - Yunsong Wang (https://github.com/PointKernel)

URL: #10749
@shwina shwina mentioned this pull request Jun 2, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bdice bdice bdice approved these changes

@ajschmidt8 ajschmidt8 ajschmidt8 approved these changes

@shwina shwina shwina approved these changes

@isVoid isVoid Awaiting requested review from isVoid isVoid is a code owner automatically assigned from rapidsai/cudf-python-codeowners

@charlesbluca charlesbluca Awaiting requested review from charlesbluca charlesbluca is a code owner automatically assigned from rapidsai/cudf-python-codeowners

Assignees

@galipremsagar galipremsagar

Labels
5 - Ready to Merge Testing and reviews complete, ready to merge breaking Breaking change improvement Improvement / enhancement to an existing function Python Affects Python cuDF API.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@galipremsagar @shwina @bdice @ajschmidt8