Considerations for WASI standardization phases

[dschuff]

Wasm has a phase process that describes how proposed features progress toward standardization. The process is designed to make it clear how to add to the wasm standard, both in terms of who should be involved and when, and the practical steps needed to ensure that we have all the artifacts we need before we declare things "done".

We'd like to have something similar for standardizing WASI, and it could be broadly similar.
Most of the steps in the process correspond fairly well to what we might want for WASI, and I expect to be uncontroversial but there are a couple of potentially-interesting bits to call out:

1. Aside from spec text and spec tests (which I assume we'll want too) the wasm spec has a reference interpreter, blessed as a single executable standard. Do we want to have an equivalent reference implementation? What should it be and/or what criteria do we want in a reference implementation?

2. To get to stage 4 wasm requires 2 or more "web VMs" to implement the feature. The requirement for multiple implementations helps ensure that a spec is implementable in different codebases with reasonable effort and avoids baking quirks of a particular VM into the standard (which is a good property that we want). Requiring that they be web VMs doesn't make sense for WASI though. So what should the requirement be? What kind of goal or property do we want in an implementation that we consider sufficient to allow a feature to be standardized?

3. At what point and how do we involve the broader wasm CG?

[dschuff]

Regarding 2), I'd say that in general we want to get some diversity of implementations if possible. For example it would be better to get one blockchain embedding and one edge-cache embedding instead of 2 of the same type. But on the other hand if there's an API that only makes sense on blockchain or one particular type of embedding we don't want to preclude that either. Also, all else being equal, implementations that are more widely-used, actively-developed, high-quality, easy-to-understand, and/or open source are better than.... not those things. But that's hard to qualify precisely.

[dschuff]

Regarding 3), I think mirroring the CG/WG split from the main phase process (where the "CG" there is the subgroup here, and the "WG" is the CG), is a good idea. In particular it means that

1. design discussions and other substantive work is expected to happen in the subgroup
2. The CG still has to ratify the subgroup's work, because it is the formally-recognized body

This means that the CG doesn't make change itself, but can send a proposal back to the subgroup.

One thing unresolved by this is, when and how does the WG get involved? Do we plan on turning all of the WASI docs into WG-level artifacts alongside the wasm spec?

[tschneidereit]

One way to look at the process would be to promote parts of WASI out of the subgroup and into the Wasm CG as individual proposals when they're ready. I think that'd be the least-overhead thing to do, but there are some open questions around it. In particular, at what stage of the process do proposals enter the CG? Do we have a mechanism for sending them back? Is there even truly a "sending back-and-forth" going on, or is it more "let's get more eyes on this, but the work continues in the subgroup"? (I think I like that last view best.)

1. Aside from spec text and spec tests (which I assume we'll want too) the wasm spec has a reference interpreter, blessed as a single executable standard. Do we want to have an equivalent reference implementation? What should it be and/or what criteria do we want in a reference implementation?

That requirement only exists for the core spec, no e.g. the JS API, right? ISTM that the situation with WASI is more similar to that of the JS API, and having the requirement might also not make sense.

2. To get to stage 4 wasm requires 2 or more "web VMs" to implement the feature. The requirement for multiple implementations helps ensure that a spec is implementable in different codebases with reasonable effort and avoids baking quirks of a particular VM into the standard (which is a good property that we want). Requiring that they be web VMs doesn't make sense for WASI though. So what should the requirement be? What kind of goal or property do we want in an implementation that we consider sufficient to allow a feature to be standardized?

Yeah, this is interesting :) We might be able to apply some learnings from TC39 here: the staging process in TC39 also has the "at least two implementations" requirement, but what counts as an implementation is deliberately left unspecified. This is because for some proposals there isn't that much verification of implementability required, so e.g. a Babel implementation might be fine, whereas for others, there are much more interesting questions around whether something is implementable across multiple engines in a sufficiently performant way. In those cases, two high-performance engines need to implement.

Regarding 2), I'd say that in general we want to get some diversity of implementations if possible. For example it would be better to get one blockchain embedding and one edge-cache embedding instead of 2 of the same type.

I'm not convinced that'll always be a good guiding principle. What's to say that a blockchain embedding and an edge-cache one won't share the same runtime, and hence implementation of a WASI API? I do however agree that whenever there are real differences to be had, we should aim for having two or more very different implementations, with different use case requirements.

[KronicDeth]

we should aim for having two or more very different implementations, with different use case requirements.

To get very specific, are Fastly's Lucet and Mozilla's WASMtime considered too similar for the 2 implementation rule because Lucet and WASMTime both use Cranelift?

[devsnek]

If those aren't different enough, I've got an implementation written in js: https://github.com/devsnek/node-wasi/

[MarkMcCaskey]

Adding to the runtime diversity: Wasmer intends to fully support Windows which, in my experience so far, has been much trickier than getting it working on Unix-like platforms. Symlinks in particular, but many of the file system details in general, may just not work well in a Windows context without the runtime having a substantial emulation layer between it and the host system.

[sunfishcode]

Yes, there are several implementations planning for WASI on Windows, and also some non-traditional-OS environments as well. So I agree with @tschneidereit's idea that we leave it unspecified exactly what kind of implementations we'll require here in general, and let the Subgroup decide what's needed for features as they progress to that phase in the process.

[dumblob]

Symlinks in particular, but many of the file system details in general, may just not work well in a Windows context without the runtime having a substantial emulation layer between it and the host system.

Maybe I misunderstood, but traditional tree-like filesystem [1] [2] and anything similar to that (e.g. some DB-like API or graph API or whatsoever) shall be separated in an optional module and not required for WASI implementation. So wasmer can easily implement everything else and leave the "fs" module for later time once the WASI standard is more settled and the community can focus on modules rather than the core.

[sunfishcode]

Are there any concerns about @tschneidereit's suggestion here to replace "Web VMs" with "implementations" and leave it up to the committee to decide the specific condition of diversity of "implementations" needed, on a feature-by-feature basis?

[KronicDeth]

replace “Web VMs” with “implementations” and leave it up to the committee

This is fine to me.

[dschuff]

On Wed, May 15, 2019, 3:47 PM Luke Imhoff ***@***.***> wrote: replace “Web VMs” with “implementations” and leave it up to the committee This is fine to me.

Sounds good to me too.

…

[AndrewScheidecker]

Adding to the runtime diversity: Wasmer intends to fully support Windows which, in my experience so far, has been much trickier than getting it working on Unix-like platforms. Symlinks in particular, but many of the file system details in general, may just not work well in a Windows context without the runtime having a substantial emulation layer between it and the host system.

IMO anything that can't be mapped losslessly to Windows files (or native files on other relevant OSes) should be removed from WASI: so advancing the WASI file I/O API to stage 4 should require at least one implementation that exposes Windows native files to WASI.

[MarkMcCaskey]

IMO anything that can't be mapped losslessly to Windows files (or native files on other relevant OSes) should be removed from WASI: so advancing the WASI file I/O API to stage 4 should require at least one implementation that exposes Windows native files to WASI.

I agree that working well on diverse platforms is highly desirable!

The biggest fundamental incompatibility that I'm aware of is that I believe Windows locks files for reading. Some things will be easier on more modern Windows; I think Windows 10 allows symlinks without elevated auth. That said, I don't have very much knowledge about Windows, so it could be user error on my part.

[programmerjake]

The biggest fundamental incompatibility that I'm aware of is that I believe Windows locks files for reading. Some things will be easier on more modern Windows; I think Windows 10 allows symlinks without elevated auth. That said, I don't have very much knowledge about Windows, so it could be user error on my part.

According to the docs for CreateFileW, file locking can be disabled by passing appropriate flags in dwShareMode.

Some of the biggest incompatibilities that I'm aware of are the different semantics of removing an open file (ending up with fd to unnamed file vs. remove operation failing), case sensitivity, and allowed character set (anything but newline and / vs. a rather large set of disallowed characters).

[programmerjake]

According to the docs for CreateFileW, file locking can be disabled by passing appropriate flags in dwShareMode.

Some of the biggest incompatibilities that I'm aware of are the different semantics of removing an open file (ending up with fd to unnamed file vs. remove operation failing), case sensitivity, and allowed character set (anything but newline and / vs. a rather large set of disallowed characters).

See also: https://boostgsoc13.github.io/boost.afio/doc/html/afio/FAQ/deleting_open_files.html

[devsnek]

case sensitivity is more an issue with the partition format than the os. i'm pretty sure ntfs can even flag specific directories as case sensitive.

[sunfishcode]

Let's discuss Windows related topics in other issues, so we can keep this issue focused on the WASI standards process :-). We agreed at the last meeting that changing the requirement to 2 implementations, and letting the Subgroup decide the requirements for those implementations on a feature-by-feature basis is a good approach.