Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[REVIEW] Reduce/Remove reliance on **kwargs and *args in IO readers & writers #12025

Merged
merged 52 commits into from
Nov 1, 2022
Merged

[REVIEW] Reduce/Remove reliance on **kwargs and *args in IO readers & writers #12025

merged 52 commits into from
Nov 1, 2022

Conversation

galipremsagar
Copy link
Contributor

Description

Resolves: #11780

This PR:

  • Reduces reliance on args & kwargs for readers and writers when cudf engine is selected. However, these will have to stay for the purpose of other engines we support in few readers & writers such as pandas & pyarrow engines.
  • Fixes some bugs where dead parameters were still being used.
  • Fixes some bugs where parameters weren't being passed until the cython later in the first place.
  • Updates docs related to newly exposed parameters.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

bdice and others added 24 commits June 1, 2022 07:20
@bdice
Initial work on removing **kwargs from IO code.
400df15
@galipremsagar
Merge remote-tracking branch 'bdice/io-kwargs' into 11780
abc632a
@galipremsagar
Merge remote-tracking branch 'upstream/branch-22.12' into 11780
ce0748c
@galipremsagar
first pass
8466097
@galipremsagar
Merge remote-tracking branch 'upstream/branch-22.12' into 11780
f733a95
@galipremsagar
update avro docstring
29c1bfa
@galipremsagar
add docs
43b5699
@galipremsagar
improve json reader
81c07a2
@galipremsagar
parameter update
e74644a
@galipremsagar
update docstring
367cac6
@galipremsagar
orc update
a2baf55
@galipremsagar
updates
ad35233
@galipremsagar
Merge remote-tracking branch 'upstream/branch-22.12' into 11780
393a43b
@galipremsagar
update
49bbe87
@galipremsagar
Merge remote-tracking branch 'upstream/branch-22.12' into 11780
8511242
@galipremsagar
updates
006d871
@galipremsagar
Merge remote-tracking branch 'upstream/branch-22.12' into 11780
05824ff
@galipremsagar
update
420f452
@galipremsagar
Merge remote-tracking branch 'upstream/branch-22.12' into 11780
643cabe
@galipremsagar
update
d89039d
@galipremsagar
cleanup
8b86691
@galipremsagar
docstring update
ed9c126
@galipremsagar
Merge remote-tracking branch 'upstream/branch-22.12' into 11780
26acc52
@galipremsagar
refactor
99d5da1
@galipremsagar galipremsagar added bug Something isn't working 3 - Ready for Review Ready for review by team Python Affects Python cuDF API. 4 - Needs cuDF (Python) Reviewer breaking Breaking change labels Oct 28, 2022
@galipremsagar
Copy link
Contributor Author

galipremsagar commented Oct 31, 2022 •
edited
Loading

@bdice Should be ready for another round of review.

@galipremsagar galipremsagar requested a review from bdice October 31, 2022 20:25
@ayushdg
Copy link
Member

ayushdg commented Oct 31, 2022

This pr also fixes #6916

@galipremsagar galipremsagar linked an issue Oct 31, 2022 that may be closed by this pull request
[FEA] Expose storage_options kwarg to readers/writers #6916
Closed
wence-
wence- approved these changes Nov 1, 2022
View reviewed changes
Copy link
Contributor

@wence- wence- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two very minor niggles, but otherwise looks great, thanks for this!

bdice
bdice approved these changes Nov 1, 2022
View reviewed changes
Copy link
Contributor

@bdice bdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor suggestions, otherwise LGTM. Thank you for the iterations!

python/cudf/cudf/io/parquet.py Outdated Show resolved Hide resolved
python/cudf/cudf/utils/ioutils.py Outdated Show resolved Hide resolved
python/cudf/cudf/utils/ioutils.py Outdated Show resolved Hide resolved
python/cudf/cudf/utils/ioutils.py Outdated Show resolved Hide resolved
python/cudf/cudf/utils/ioutils.py Outdated Show resolved Hide resolved
@bdice bdice mentioned this pull request Nov 1, 2022
Merged
3 tasks
@galipremsagar galipremsagar added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 4 - Needs cuDF (Python) Reviewer 3 - Ready for Review Ready for review by team labels Nov 1, 2022
@galipremsagar
Copy link
Contributor Author

@gpucibot merge

@rapids-bot rapids-bot bot merged commit d236779 into rapidsai:branch-22.12 Nov 1, 2022
@galipremsagar
Copy link
Contributor Author

Thanks for patiently reviewing this PR @bdice & @wence-, I know this was a tedious one.

rapids-bot bot pushed a commit that referenced this pull request Nov 1, 2022
@bdice
Remove smart quotes from all docstrings. (#12035)
f19bdbc 
This PR removes all "smart quotes" from the library by enforcing a pre-commit hook.

Smart quotes typically arise from copying rendered docstrings from Pandas, because Sphinx automatically transforms straight quotes into smart quotes when rendering the docs as HTML. However, the use of smart quotes is undesirable in code, and makes it difficult to do find-replace transformations if straight and smart quotes are mixed.

I have made suggestions to fix this several times before, so I am making the suggestions more permanent and automatically enforceable via a pre-commit style check:
- #12025 (comment)
- #9817 (comment)
- #9571 (comment)

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - GALI PREM SAGAR (https://github.com/galipremsagar)

URL: #12035
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bdice bdice bdice approved these changes

@wence- wence- wence- approved these changes

@vyasr vyasr Awaiting requested review from vyasr vyasr is a code owner automatically assigned from rapidsai/cudf-python-codeowners

@isVoid isVoid Awaiting requested review from isVoid isVoid is a code owner automatically assigned from rapidsai/cudf-python-codeowners

@vuule vuule Awaiting requested review from vuule

@rjzamora rjzamora Awaiting requested review from rjzamora

Assignees

@galipremsagar galipremsagar

Labels
5 - Ready to Merge Testing and reviews complete, ready to merge breaking Breaking change bug Something isn't working Python Affects Python cuDF API.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[FEA] Remove **kwargs in all IO readers and writers [FEA] Expose storage_options kwarg to readers/writers
4 participants
@galipremsagar @ayushdg @wence- @bdice