Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

uniform handling of path argument gievn as a list of strings or a single string in read mode #964

Closed
wants to merge 3 commits into from

Conversation

ghislainp
Copy link
Contributor

Issue #957 was solved by PR #961 when path is a list of string but not when path is a string.
This PR solves this problem.

However, it also changes other behaviors. For instance, paths were sorted and filtered with not fs.isdir(f) only in read mode and only if a string was given with a wildcard. i.e. list of paths in read mode was not filtered+sorted. Write mode was not filtered+sorted. A single path without wildcard was not filtered+sorted.

This PR is an improvement imho with a more uniform handling of all these cases. dirs are filtered out in all the case, and sorting is also applied in all the cases. However, I hope this does not break expected behaviors that I'm ignoring.

@ghislainp
Copy link
Contributor Author

In fact, it is not convenient to always filter out the directory. I'm now using get_fs_token_paths in a case where the target is a (zarr) directory. I think adding a boolean argument 'filter' like 'expand' is the most versatile option.

@martindurant
Copy link
Member

Circling back here. I think it would make a lot of sense to write out the cases you allude to as tests, so that we can confirm that the behaviour we see is what we want.
So we can have a single path or a list of paths as input, and any of these may contain wildcard(s); these may be flagged for expanding, and we may want different behaviours on write (when the paths are expected not to yet exist of be freely over-writable) versus read.

I'm not sure we can add any additional flags to this function, or at least we can't changed the default behaviour, since many other things depend on it in this codebase and elsewhere.

…urn the actual length of written data (this is the case for SFTP with paramiko)
@ghislainp ghislainp closed this by deleting the head repository Oct 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants