Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Copy between filesystems #588

Open
c-thiel opened this issue Mar 29, 2021 · 1 comment
Open

Feature Request: Copy between filesystems #588

c-thiel opened this issue Mar 29, 2021 · 1 comment

Comments

@c-thiel
Copy link

c-thiel commented Mar 29, 2021

It would be great if we could copy between two different remote filesystems with fsspec.

I think this can be achieved with minimal effort if we check if the lpath is a path in a known filesystem in get/put_file functions in fsspec.spec. Then we could just use lpath.fs.open() instead of open(lpath).

@martindurant
Copy link
Member

It is a reasonable request, but not I think that easy to do. Indeed, #41 discusses the idea of a "dispatch" filesystem, where you always give the full URLs where now you give fs-specific paths. The trouble is, most implementations need arguments to work. Either you would have to pass through sets of args for every user method (twice for copy, for origin and target), or have a concept of the "current" instance of each file system type. It would perhaps work better with fsspec path-like objects, where the filesystem is encoded in the object.

aside: I would call this operation "copy" rather than get/put, which to me are synonymous with download/upload.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants