Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] First pass of pd.read_orc changes in tests #12103

Merged
merged 4 commits into from
Nov 10, 2022

Conversation

galipremsagar
Copy link
Contributor

@galipremsagar galipremsagar commented Nov 9, 2022

Description

This PR changes calls going via pyarrow and then to_pandas to directly call pd.read_orc. How-ever since pd.read_orc was added in pandas 1.0, we will need to version the call to this constructor. This PR does that.

Partially contributes to #11540

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@galipremsagar galipremsagar added 3 - Ready for Review Ready for review by team Python Affects Python cuDF API. 4 - Needs cuDF (Python) Reviewer improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Nov 9, 2022
@galipremsagar galipremsagar requested a review from a team as a code owner November 9, 2022 14:04
@galipremsagar galipremsagar self-assigned this Nov 9, 2022
@galipremsagar galipremsagar requested review from wence- and bdice November 9, 2022 14:04
@galipremsagar galipremsagar requested a review from vuule November 9, 2022 14:04
@galipremsagar galipremsagar changed the title [REVIEW] First pass of read_orc changes in tests [REVIEW] First pass of pd.read_orc changes in tests Nov 9, 2022
@codecov
Copy link

codecov bot commented Nov 9, 2022

Codecov Report

Base: 87.47% // Head: 88.08% // Increases project coverage by +0.60% 🎉

Coverage data is based on head (b69b791) compared to base (f817d96).
Patch has no changes to coverable lines.

❗ Current head b69b791 differs from pull request most recent head ec43524. Consider uploading reports for the commit ec43524 to get more accurate results

Additional details and impacted files
@@               Coverage Diff                @@
##           branch-22.12   #12103      +/-   ##
================================================
+ Coverage         87.47%   88.08%   +0.60%     
================================================
  Files               133      135       +2     
  Lines             21826    22100     +274     
================================================
+ Hits              19093    19466     +373     
+ Misses             2733     2634      -99     
Impacted Files Coverage Δ
python/cudf/cudf/core/column/interval.py 85.45% <0.00%> (-9.10%) ⬇️
python/cudf/cudf/io/text.py 91.66% <0.00%> (-8.34%) ⬇️
python/cudf/cudf/core/_base_index.py 81.28% <0.00%> (-4.27%) ⬇️
python/cudf/cudf/io/json.py 92.06% <0.00%> (-2.68%) ⬇️
python/cudf/cudf/utils/utils.py 89.91% <0.00%> (-0.69%) ⬇️
python/cudf/cudf/core/column/timedelta.py 90.17% <0.00%> (-0.58%) ⬇️
python/cudf/cudf/core/column/datetime.py 89.21% <0.00%> (-0.51%) ⬇️
python/cudf/cudf/core/column/column.py 87.96% <0.00%> (-0.46%) ⬇️
python/dask_cudf/dask_cudf/core.py 73.72% <0.00%> (-0.41%) ⬇️
python/cudf/cudf/io/parquet.py 90.45% <0.00%> (-0.39%) ⬇️
... and 41 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Contributor

@vuule vuule left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like the change!

Can we expect noticeably lower test execution time with this change? I don't have a good sense of how efficient the conversion is(was).

python/cudf/cudf/tests/test_orc.py Show resolved Hide resolved
Copy link
Contributor

@wence- wence- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! My only comment is whether we need to support pandas pre 1.0 at all?

@galipremsagar
Copy link
Contributor Author

Thanks! My only comment is whether we need to support pandas pre 1.0 at all?

Good catch. Rapids only support pandas >=1.0, so dropped the pre 1.0 code.

@galipremsagar galipremsagar added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 3 - Ready for Review Ready for review by team 4 - Needs cuDF (Python) Reviewer labels Nov 10, 2022
@galipremsagar
Copy link
Contributor Author

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 8ca2bd9 into rapidsai:branch-22.12 Nov 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to Merge Testing and reviews complete, ready to merge improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants