-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[API Proposal]: APIs for working with unix file mode #67837
Comments
Tagging subscribers to this area: @dotnet/area-system-io Issue DetailsBackground and motivationAdd direct support for getting/setting Unix file permissions and special bits. These APIs allow to restrict/extend access to files and directories from .NET. There have been some requests for these APIs: #925, #928, #17540. The tar implementation can make use of them (#65951) without resorting to internal APIs. API Proposalnamespace System.IO;
[System.FlagsAttribute]
public enum UnixFileMode
{
// From https://github.com/dotnet/runtime/issues/65951.
None = 0,
OtherExecute = 1,
OtherWrite = 2,
OtherRead = 4,
GroupExecute = 8,
GroupWrite = 16,
GroupRead = 32,
UserExecute = 64,
UserWrite = 128,
UserRead = 256,
StickyBit = 512,
GroupSpecial = 1024, // or: SetGroup
UserSpecial = 2048, // or: SetUser
// Maybe:
AccessPermissions = {User,Group,Other}{Execute,Read,Write}
}
static partial class Directory
{
// Set mode when creating
public static DirectoryInfo CreateDirectory(string path, UnixFileMode fileMode);
// Get/Set using path
public static UnixFileMode GetUnixFileMode(string path);
public static void SetUnixFileMode(string path, UnixFileMode fileMode);
}
static partial class File
{
// Set mode when creating
public static SafeFileHandle OpenHandle(string path UnixFileMode fileMode, System.IO.FileMode mode = System.IO.FileMode.Create, System.IO.FileAccess access = System.IO.FileAccess.Write, System.IO.FileShare share = System.IO.FileShare.None, System.IO.FileOptions options = System.IO.FileOptions.None, long preallocationSize = (long)0);
// Get/Set using path
public static UnixFileMode GetUnixFileMode(string path);
public static void SetUnixFileMode(string path, UnixFileMode fileMode);
// Get/Set using SafeHandle, (.NET 7) https://github.com/dotnet/runtime/issues/20234
public static UnixFileMode GetUnixFileMode(SafeFileHandle fileHandle);
public static void SetUnixFileMode(SafeFileHandle fileHandle, UnixFileMode fileMode);
}
class FileSystemInfo
{
// Get/Set
public UnixFileMode UnixFileMode { get; set; }
} Notes On Windows/non-Unix:
APIs that create a file/directory (
API UsageExample 1: limit a file to be read/written by the owner only: File.SetUnixFileMode("filename", UnixFileMode.UserRead | UnixFileMode.UserWrite); Example 2: exclude 'Other' access to entries in a directory: var di = new DirectoryInfo("somedir");
foreach (var fi in di.GetFileSystemInfos())
{
fi.UnixFleMode = fi.UnixFileMode & ~(UnixFileMode.OtherExecute | UnixFileMode.OtherRead | UnixFileMode.OtherWrite);
} Example 3: create new files/directories while extracting a tar file with the exact permissions of the tar file. if (entryType == EntryType.Directory)
{
DirectoryInfo directoryInfo = Directory.CreateDirectory(path, entry.Mode); // restricted by umask
di.Mode = entry.Mode;
}
else
{
using var file = File.OpenHandle(path, entry.Mode); // restricted by umask
File.SetUnixFileMode(file, entry,Mode);
} Alternative DesignsNo response RisksNo response
|
@eerhardt @carlossanlop @stephentoub @dotnet/area-system-io ptal. |
File.OpenHandle with UnixFileMode looks useful for fixing the race condition AzureAD/microsoft-authentication-extensions-for-dotnet#174 and could simplify NuGetExtractionFileIO. |
Shouldn't those API be put in a separate TFM |
Those TFMs don't exist. We don't have TFMs for But instead we can use the SupportedOSPlatform and UnsupportedOSPlatform attributes that work with https://docs.microsoft.com/dotnet/standard/analyzers/platform-compat-analyzer. Then you will get |
This looks great @tmds. Thanks for writing this up. I have the following questions:
|
I'll add it in addition to the
When you provide a mode to The .NET example makes two calls. One for If we don't want that, we can perform an Consequently, if the file already exists (e.g.
The |
I meant a public API that .NET developers could call to set the |
Because .NET is multi-threaded, it would end badly. One thread doing 'untar', would set it to While another thread doing |
I can write multi-threaded C code. Does it end badly there as well?
I'm not talking about |
Yes.
By this I meant, use the API to untar a file: #65951.
I don't think there are strong use-cases for this. They should use the |
I had a look. nodejs and python APIs that create files preserve the default behavior of applying umask. Both also provide a umask function. Still, I don't think we should add it in .NET because of the multi-threading issues. The other APIs allow to set the permissions in a thread-safe way without using |
Something like: public static FileStream Create(string path, UnixFileMode permissions, FileOptions options = FileOptions.None)
=> Create(path, permissions, DefaultBufferSize, options);
public static FileStream Create(string path, UnixFileMode permissions, int bufferSize, FileOptions options = FileOptions.None)
=> new FileStream(OpenHandle(path, permissions, FileMode.Create, FileAccess.ReadWrite, FileShare.None, options), FileAccess.ReadWrite, bufferSize); The @eerhardt what do you think? |
Setting umask in ProcessStartInfo would be safe, but if an application cares about that, then it likely wants to control ulimit and file descriptors as well, so just adding umask support there would not be worthwhile. |
We have introduced |
I've included it as a |
Thank you for proposing this. Getting UnixFileMode approved and implemented in 7.0 with all these usages would allow us to remove TarFileMode, which is what ended up being approved here (instead of UnixFileMode): #65951 I think this proposal should explicitly mention that TarFileMode would be removed once UnixFileMode is approved. @bartonjs agree? |
I think this proposal looks good enough to be marked |
Looking again, |
I've added this to the proposal. |
@adamsitnik @jozkee do you have feedback? Can this move to ready for review? |
Does the UnixFileMode sets atomically when the file or directory is created? If not, I think having only
As you are already expanding FileStreamOptions,
Can you elaborate on this? |
Yes, it's atomic, and it is needed to restrict permissions immediately on creation.
I'll keep it in the proposal, and let the API reviewers kick it out if they want.
This is a mask to filter out the access permissions. It could be left out. This is an example use-case: runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarEntry.Unix.cs Lines 45 to 51 in b42adad
I've updated the proposal to include your feedback. |
@jozkee @adamsitnik when it looks good to you, can you move the issue to ready to review? |
That's right. Otherwise all |
Do we really need static partial class Directory
{
- public static UnixFileMode GetUnixFileMode(string path);
- public static void SetUnixFileMode(string path, UnixFileMode mode);
}
static partial class File
{
- public static UnixFileMode GetUnixFileMode(string path);
- public static void SetUnixFileMode(string path, UnixFileMode mode);
}
class FileSystemInfo
{
public UnixFileMode UnixFileMode { get; set; }
} An alternative it to extend
This might confuse some of our users, moreover I am not sure which overload IDE like VS is going to suggest first. If the one that accepts static partial class File
{
- public static SafeFileHandle OpenHandle(string path, UnixFileMode unixCreateMode, FileMode mode = FileMode.CreateNew, FileAccess access = FileAccess.Write, FileShare share = FileShare.None, FileOptions options = FileOptions.None, long preallocationSize = 0);
- public static UnixFileMode GetUnixFileMode(SafeFileHandle fileHandle);
- public static void SetUnixFileMode(SafeFileHandle fileHandle, UnixFileMode mode);
} |
I'll leave these out. var mode = File.GetUnixFileMode(path); becomes var mode = new FileSystemInfo(path).UnixFileMode
I'll leave it out. C# doesn't lend itself to extending an existing method with additional default arguments. Without this overload, there is no way to create a using SafeFileHandle handle = File.OpenHandle(path, unixCreateMode: mode); becomes using FileStream fs = new FileStream(path, new FileStreamOptions { UnixCreateMode = mode });
SafeFileHandle handle = fs.SafeFileHandle;
Operating on a API reviewers can still kick them out. |
@carlossanlop can I replace the |
There's a pattern for it, with the work on the library author.
It doesn't work on virtuals, or for interfaces, but our guidelines say not to use defaulted parameters for either of those cases (except for a CancellationToken parameter) Any existing compilations will call the old method, which just calls the new. Any new compilations will prefer the new method, unless they happened to be specifying every argument in their code, then they recompile to exactly what they were. |
@bartonjs thanks, I'll add back the |
I welcome any name that would make the most sense, @tmds. |
Putting my API Review hat on a bit early... "But isn't it only SetUID/SetGID for (executable) files, and it means something different, like 'sticky' on directories? I think that the POSIX header comments call it 'special', that seems like the more general name." Just sharing 😄 |
https://man7.org/linux/man-pages/man0/sys_stat.h.0p.html shows descriptions of those bits copied from POSIX, I believe. They don't involve "special". The S_ISGID bit has been used for:
(I wonder if the bit has been used for anything on block devices or character devices… like biff sets S_IXUSR and S_IXGRP on a tty.) |
I'm going to replace the names in the proposal by API reviewers can make the final call. |
Hm. Clearly some bad data got shoved into my head at some point. New data accepted, position changed. Rename away 😄 |
It looks like golang calls them (roughly) the same: I don't see Java having names for these. https://docs.oracle.com/javase/8/docs/api/java/nio/file/attribute/PosixFilePermission.html. Looks like there are just |
namespace System.IO;
[System.FlagsAttribute]
public enum UnixFileMode
{
None = 0,
OtherExecute = 1,
OtherWrite = 2,
OtherRead = 4,
GroupExecute = 8,
GroupWrite = 16,
GroupRead = 32,
UserExecute = 64,
UserWrite = 128,
UserRead = 256,
StickyBit = 512,
SetGroup = 1024,
SetUser = 2048
}
public partial class Directory
{
public static DirectoryInfo CreateDirectory(string path, UnixFileMode unixCreateMode);
}
public partial class File
{
public static UnixFileMode GetUnixFileMode(string path);
public static UnixFileMode GetUnixFileMode(SafeFileHandle fileHandle);
public static void SetUnixFileMode(string path, UnixFileMode mode);
public static void SetUnixFileMode(SafeFileHandle fileHandle, UnixFileMode mode);
}
public partial class FileSystemInfo
{
public UnixFileMode UnixFileMode { get; set; }
}
public partial class FileStreamOptions
{
public UnixFileMode? UnixCreateMode { get; set; }
} |
Do we need an equivalent for creating a file? Otherwise, with the APIs I see approved above, it doesn't seem like there's a way for me to create a file and atomically have it have the "right" permissions, e.g. I might want a file locked down to just user, but I'll end up creating a file with the default permissions and by the time I've restricted them, someone else could have accessed it. (Sorry if this was discussed; had to leave the meeting early...) |
FileStreamOptions options = new()
{
Mode = FileMode.Create,
UnixCreateMode = UnixFileMode.UserRead | UnixFileMode.UserWrite
}
File.Open(filePath, options).Dispose(); |
Why is there no |
@jozkee @adamsitnik @eerhardt you can assign this to me.
Let's see where it makes sense as part of the PR. |
"None of the APIs will be marked as Linux only as it seems we can and should make them work on Windows by setting WSL related metadata in NTFS" |
Ah, yes, thanks. |
Thanks. I overread that 🤦🏼♂️. |
Fixed by #69980. |
no chown/chgrp? anyway thanks for this one! |
Background and motivation
Add direct support for getting/setting Unix file permissions and special bits.
These APIs allow to restrict/extend access to files and directories from .NET.
There have been some requests for these APIs: #925, #928, #17540.
The tar implementation can make use of them (#65951) without resorting to internal APIs.
API Proposal
Notes
On Windows/non-Unix:
The
unixCreateMode
arg onDirectory.CreateDirectory
/File.OpenHandle
andFileStreamOptions.UnixCreateMode
are ignored. These APIs can be used in in cross-platform code.FileSystemInfo.UnixFileMode get
returnsUnixFileMode.None
.FileSystemInfo.UnixFileMode set
throws PNSE.APIs that create a file/directory (
Directory.CreateDirectory
,FileStreamOptions.UnixCreateMode
,File.OpenHandle
) have 'POSIX behavior':If the file/directory exists, the mode argument is ignored.
The OS kernel filters the provided mode by the process
umask
. The umask protects users from unintentionally opening up permissions. A regular Linux user has a umask of0002
which prevents it from giving write permissions to other. Use-cases that require the exact mode, must make an additional call toSetUnixFileMode
. See example 3.note: creating and setting permissions is atomic to ensure permissions are applied immediately
UnixFileMode
can replaceTarFileMode
introduced in #65951.AccessPermissionMask
,DefaultFileOpenPermissions
enum values.runtime/src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarEntry.Unix.cs
Lines 45 to 51 in b42adad
runtime/src/libraries/System.Private.CoreLib/src/Microsoft/Win32/SafeHandles/SafeFileHandle.Unix.cs
Lines 167 to 174 in b42adad
API Usage
Example 1: limit a file to be read/written by the owner only:
Example 2: exclude 'Other' access to entries in a directory:
Example 3: create new files/directories while extracting a tar file with the exact permissions of the tar file.
Alternative Designs
No response
Risks
No response
The text was updated successfully, but these errors were encountered: