Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update io util to convert path like object to string #8275

Merged
merged 2 commits into from
May 19, 2021

Conversation

ayushdg
Copy link
Member

@ayushdg ayushdg commented May 18, 2021

Fixes failing dask-cudf s3 test introduced from changes to fsspec.stringify_path in their 2021.5.0 release.

Problem description:
While reading remote files with dask-cudf the path_or_object param is populated with remote file objects such as s3fs.s3File or gcsfs.GCSFile. Updates to the fsspec.stringify_path util results in returning the string path for such objects resulting in cudf looking for those file paths locally.

Fix:
Fix uses one of the branches from the original stringify_path util that checks if a path like object implement the fsspath protocol and gets the string representation from that.

@github-actions github-actions bot added the Python Affects Python cuDF API. label May 18, 2021
@ayushdg ayushdg marked this pull request as ready for review May 18, 2021 21:31
@ayushdg ayushdg requested a review from a team as a code owner May 18, 2021 21:31
@ayushdg ayushdg requested review from galipremsagar and isVoid May 18, 2021 21:31
@ayushdg ayushdg added 3 - Ready for Review Ready for review by team bug Something isn't working non-breaking Non-breaking change labels May 18, 2021
@codecov
Copy link

codecov bot commented May 18, 2021

Codecov Report

❗ No coverage uploaded for pull request base (branch-21.06@8406522). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff               @@
##             branch-21.06    #8275   +/-   ##
===============================================
  Coverage                ?   82.84%           
===============================================
  Files                   ?      105           
  Lines                   ?    17865           
  Branches                ?        0           
===============================================
  Hits                    ?    14800           
  Misses                  ?     3065           
  Partials                ?        0           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8406522...4a7c652. Read the comment docs.

@kkraus14 kkraus14 added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 3 - Ready for Review Ready for review by team labels May 18, 2021
@kkraus14
Copy link
Collaborator

@gpucibot merge

@ayushdg
Copy link
Member Author

ayushdg commented May 19, 2021

There seem to be errors the decimal orc writer tests unrelated to this pr. Might be a timing issue since the to_orc decimal support merged 4 hours back so it may not have made it to the libcudf build in CI

@ayushdg
Copy link
Member Author

ayushdg commented May 19, 2021

Rerun tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to Merge Testing and reviews complete, ready to merge bug Something isn't working non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants