Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mv('filepath','filepath') results in deletion of the file rather than no action/inplace move #711

Closed
NadimNadimNadimNadimNadim opened this issue Mar 15, 2023 · 8 comments · Fixed by fsspec/filesystem_spec#1218

Comments

@NadimNadimNadimNadimNadim

Calling mv() of s3fs.S3FileSystem with path1==path2 results in deletion. This is not the expected behavior, as a POSIX mv results in a message saying they are the same file, and in particular does not result in deletion.

Working example:

session = aiobotocore.session.AioSession(profile="dev")
s3 = s3fs.S3FileSystem(session=session)
path_to_file = 'mybucket/file.txt'
s3.mv(
        path1=path_to_file
        path2=path_to_file
)

Now, the file in my bucket at location path_to_file is deleted.
I have versioning enabled on my bucket for what its worth, so I see the delete marker.

@martindurant
Copy link
Member

hah, that's funny and I suppose not surprising (mv -> cp then rm). You could propose a PR that converts to a no-op if the two paths are identical, or it could be an exception.

@NadimNadimNadimNadimNadim
Copy link
Author

haha yeah that's what I figured was happening. Regarding "propose a PR", I've never contributed to an open-source project before. To clarify, you're suggesting I actually make the code change/PR, correct? Or is there another mechanism to ask for a feature/PR without me making it. Just wondering :)

@martindurant
Copy link
Member

You can ask, but developer time is always limited for open source projects. We could say "don't use the same path twice" and be done :)

But yes, we encourage people to get involved and contribute code changes when they identify a clear problem. https://anaconda.cloud/your-first-oss-contribution

@DavidKatz-il
Copy link

@martindurant I can work on it.
Do you have a preference for whether to raise exceptions or not?

@NadimNadimNadimNadimNadim
Copy link
Author

@DavidKatz-il Sweet! I like the idea of it being no-op if path1==path2.

@NadimNadimNadimNadimNadim
Copy link
Author

or perhaps just not deleting it if path1==path2. that way if someone wants to use this to "touch" files they can.

@DavidKatz-il
Copy link

@martindurant
Should we do this change on the fsspec library here, or only for s3fs?

@martindurant
Copy link
Member

Indeed it probably makes sense to add it upstream in fsspec.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants