-
Notifications
You must be signed in to change notification settings - Fork 915
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEA] Expose storage_options
kwarg to readers/writers
#6916
Comments
This issue has been marked stale due to no recent activity in the past 30d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be marked rotten if there is no activity in the next 60d. |
Can tackle once pandas requirements are updated (being worked on in #7375). |
This issue has been labeled |
The prerequisite PR has been merged. |
Thanks for the ping @vuule. Pushing to 0.20 seems fine. Shouldn't impact workflows, since we already support it when passed via |
This will be really useful! |
@chinmaychandak passing storage_options is already supported today. It just isn't exposed explicitly as a param, which this issue is meant to address. |
No, I meant exposing this as an explicit param would be really useful. Not a blocker by any means. |
Is your feature request related to a problem? Please describe.
cuDF supports reading/writing to multiple data locations aided by
fsspec
. Reading data from remote source like cloud storage often requires additional kwargs passed to fsspec via thestorage_options
argument which is not exposed directly in cuDF but is a hidden kwarg.Since v1.2, pandas has transitioned to using fsspec for a lot of file handling operations and now exposes the
storage_options
argument.Describe the solution you'd like
cuDF can therefore expose this argument rather than using it as a hidden kwarg.
Describe alternatives you've considered
N/A
Additional context
Here are a couple of examples for pandas exposing storage_options from their docs: read_csv , to_parquet.
The text was updated successfully, but these errors were encountered: