WASI API considerations

[dumblob]

After looking quickly at the API, I came to conclusion, that it seems to be slightly more unix/POSIX oriented than a really generic API. Maybe there is still time to do something about it 😉.

1. Drawbridge - a modern API from Microsoft with just 45 syscalls

[sunfishcode]

Thanks for the link!

Some parts of Drawbridge operate at a different level of abstraction, for example:

VOID *DkVirtualMemoryAlloc(Addr, Size, AllocType, Prot);
DkVirtualMemoryFree(Addr, Size, FreeType);
DkVirtualMemoryProtect(Addr, Size, Prot);
DkStreamMap(StreamHandle, Addr, ProtFlags, Offset, Size);
DkStreamUnmap(Addr);

The reason we didn't approach WASI at this level is that the core WebAssembly platform with linear memory is already at a higher level; some of the issues are described here.

Some parts look somewhat similar to APIs in WASI:

ULONG DkStreamRead(StreamHandle, Offset, Size, Buffer);
ULONG DkStreamWrite(StreamHandle, Offset, Size, Buffer);

although I wasn't able to find detailed documentation on how these and other APIs work.

Drawbridge also appears to offer process checkpointing with serialization, which is useful functionality that WASI may want to consider.

The paper says "our library OS leverages I/O streams to emulate NT file objects and named pipe objects, as well as a proxied interface to networking sockets." There's unfortunately not a lot of detail here, but we have looked at other systems that do similar things, and I can offer some general thoughts:

The paper doesn't have a lot of detail, so I'm making some assumptions here; please feel welcome to correct me if I'm getting something wrong. What the above seems to imply is that Drawbridge in fact does have fairly complex and possibly Windows-specific APIs, which aren't reflected in the absolute number of system calls, because they're being proxied over stream APIs.

In general, I believe WebAssembly pushes us away from APIs that serialize structured data or complex APIs into streams, at least when communicating between WebAssembly modules. This is because WebAssembly has other ways to do this -- modules can just call each other, both ways, directly. While this does mean that the number of system calls might end up higher than it might otherwise be, it doesn't mean that the system has more features or is more complex. In some ways, this simplifies the system because it takes better advantage of WebAssembly's static type checking.

If there are other specific ideas from Drawbridge you think are worth considering, please highlight them!

[dumblob]

Yes, checkpointing is an interesting feature (security, deployment, ...). There might be also some more information about the use of the "streaming API" in these places:

• https://github.com/thinkcz/pico-toolbox
• https://github.com/thinkcz/DrawBridge
• any place dealing with MSSQL on Linux or with WSL in Windows

Apart from Drawbridge I have some concerns e. g. about the tree-based filesystem-like API (see WebAssembly/WASI#1 (comment) ) and in general about the "too rigid, almost archaic" perception of logical data chunks with serialized data usually called files. The concept of a file might be defined similarly to the following tuple.

name, additional_metadata, metadata_inside, actual_data_inside, serialization_type_of_the_inside, permissions, permissions_scheme

Note name is not included in metadata_inside as metadata_inside are inside of the file - i.e. part of the serialized data chunk. In case of Windows the other "streams" would fit under additional_metadata (e.g. https://blogs.msdn.microsoft.com/ie/2005/03/07/mark-of-the-web/ ). Things like access times, etc. are also under additional_metadata. serialization_type_of_the_inside is more or less a MIME type and its version (e.g. CAD formats evolved over time and have several versions). permissions_scheme might be e.g. ugo|rwx or something more fine-grained like xattr or even more fine-grained (e.g. Samba permissions), etc.

I believe WASI shall abstract over this to simplify things and allow for more straightforward (more efficient) use of the actual data.

---

As a side note - actually I think there is nothing like a file, but rather just a pair of separate chunks of data: RAW deserialized unfiltered data (e.g. pixels in an image), RAW deserialized unfiltered metadata (just additional information about the set of pixels - e.g. GPS coordinates of their origin on Earth), but no compression, no MIME type, no access rights, nothing like that. All this other stuff is just a matter of filtering (compression/decompression, serialization/deserialization, ...) or matter of surrounding (access times, permissions, ...). But this sounds too radical for WASI, so it's just to show the way 😉.

[sunfishcode]

Discussion continues at WebAssembly/WASI#28.