Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

at-async interferes with cp #29944

Closed
ararslan opened this issue Nov 6, 2018 · 1 comment
Closed

at-async interferes with cp #29944

ararslan opened this issue Nov 6, 2018 · 1 comment
Assignees
Labels
bug Indicates an unexpected problem or unintended behavior filesystem Underlying file system and functions that use it

Comments

@ararslan
Copy link
Member

ararslan commented Nov 6, 2018

It seems that Julia's cp function copies files in chunks of 64K. When calling cp inside of @async, as Pkg does, a task switch can leave the file in a partial state, i.e. the copy is not continued after the current chunk finishes. This results in files that are truncated to sizes that are multiples of 64K.

This seems only to occur on slower filesystems, where task switches during a copy are more likely, as the copy operation is slower. Also note that running the system cp does not truncate files, since a task switch can't interrupt it.

This issue is difficult to reproduce, as it requires a very specific set of circumstances to occur. It was originally observed and reported in JuliaLang/Pkg.jl#795. More background on the problem can be found in that issue. I'm moving the issue here as it's a problem with Julia (and/or libuv) rather than directly with Pkg.

@ararslan ararslan added bug Indicates an unexpected problem or unintended behavior filesystem Underlying file system and functions that use it labels Nov 6, 2018
@vtjnash
Copy link
Member

vtjnash commented Apr 10, 2019

fixed by #30989

@vtjnash vtjnash closed this as completed Apr 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Indicates an unexpected problem or unintended behavior filesystem Underlying file system and functions that use it
Projects
None yet
Development

No branches or pull requests

3 participants