Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

windows.DeleteFile: Use FileDispositionInformationEx if possible, but fallback if not #16499

Merged
merged 2 commits into from
Jul 23, 2023

Conversation

squeek502
Copy link
Collaborator

@squeek502 squeek502 commented Jul 23, 2023

Using FileDispositionInformationEx (and therefore flags like FILE_DISPOSITION_POSIX_SEMANTICS and FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE) is only supported on NTFS, so the comptime Windows version range check is not enough to determine whether or not the NtSetInformationFile will succeed.

This commit makes DeleteFile always try using FileDispositionInformationEx first, but if INVALID_PARAMETER is received (which is the status that occurs when the filesystem doesn't support FileDispositionInformationEx), then it will fallback and try calling NtSetInformationFile with FileDispositionInformation.

This keeps NTFS as fast as it was before, since it will do at most 1 NtSetInformationFile call, but on non-NTFS filesystems (e.g. FAT32), DeleteFile will need to do 2 NtSetInformationFile calls.

Closes #16497


As mentioned in #16497, the alternative would be to use a NtQueryVolumeInformationFile upfront to check if the filesystem of the file supports posix semantics.

I've implemented the alternative here and benchmarked them using the deleteTree benchmark from #13073:

(fallback is this PR, query is the alternative implementation with NtQueryVolumeInformationFile; benchmark was run on NTFS)

Benchmark 1: rm-opt-fallback.exe
  Time (mean ± σ):      4.574 s ±  0.134 s    [User: 0.290 s, System: 3.950 s]
  Range (min … max):    4.428 s …  4.882 s    10 runs

Benchmark 2: rm-opt-query.exe
  Time (mean ± σ):      4.702 s ±  0.053 s    [User: 0.246 s, System: 4.075 s]
  Range (min … max):    4.586 s …  4.767 s    10 runs

Summary
  'rm-opt-fallback.exe' ran
    1.03 ± 0.03 times faster than 'rm-opt-query.exe'

Not a huge/any difference, and I figured the version that keeps it to 1 NtSetInformationFile call on NTFS would be preferable.

… fallback if not

Using FileDispositionInformationEx (and therefore flags like FILE_DISPOSITION_POSIX_SEMANTICS and FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE) is only supported on NTFS, so the comptime Windows version range check is not enough to determine whether or not the NtSetInformationFile will succeed.

This commit makes DeleteFile always try using FileDispositionInformationEx first, but if INVALID_PARAMETER is received (which is the status that occurs when the filesystem doesn't support FileDispositionInformationEx), then it will fallback and try calling NtSetInformationFile with FileDispositionInformation.

This keeps NTFS as fast as it was before, since it will do at most 1 NtSetInformationFile call, but on non-NTFS filesystems (e.g. FAT32), DeleteFile may need to do 2 NtSetInformationFile calls.

Closes ziglang#16497
These two tests can't be disambiguated at comptime, since the filesystem that the test is running on also matters for whether or not POSIX_SEMANTICS / IGNORE_READONLY_ATTRIBUTE can actually be used (since they are only supported on NTFS).
Copy link
Contributor

@matu3ba matu3ba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: test with file pending semantics does not check, if file has been deleted.

other nit: commit message could mention that windows version < 10 SP1 now use always 2 syscalls instead of 1 as those version dont support posix semantics. so not only fat file systems.

if (delete_result) {
try testing.expectError(error.FileNotFound, tmp.dir.deleteFile("test_file"));
} else |err| {
try testing.expectEqual(@as(anyerror, error.AccessDenied), err);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not strictly test that the deleteFile with file pending semantics succeeds (file does not exist anymore).
The file must be closed for the deletion to be successful (and it should be checked).

@squeek502
Copy link
Collaborator Author

nit: test with file pending semantics does not check, if file has been deleted.

This is not something either of the previous tests did, but I agree that it might be a nice addition to the test.

other nit: commit message could mention that windows version < 10 SP1 now use always 2 syscalls instead of 1 as those version dont support posix semantics. so not only fat file systems.

This isn't true, check the code again. The version check is still there, so on versions < win10_rs1, there is only 1 NtSetInformationFile call.

@andrewrk andrewrk merged commit b35874a into ziglang:master Jul 23, 2023
matu3ba added a commit to matu3ba/zig that referenced this pull request Aug 17, 2023
Also add optimization for happy path with early return.
No idea, why this worked before.
matu3ba added a commit to matu3ba/zig that referenced this pull request Aug 21, 2023
Also add optimization for happy path with early return.
No idea, why this worked before.
matu3ba added a commit to matu3ba/zig that referenced this pull request Aug 21, 2023
Also add optimization for happy path with early return.
No idea, why this worked before.
andrewrk pushed a commit that referenced this pull request Aug 24, 2023
Mitigates #14978.

Uses the same strategy as in #16499 suggested by @squeek502
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

windows: FILE_DISPOSITION_POSIX_SEMANTICS is not supported on FAT filesystems
3 participants