Consider implementing threads support

[MaxDesiatov]

I'm not 100% sure if everything is available in LLVM at this point (I've only found https://reviews.llvm.org/D57874 and https://reviews.llvm.org/D64537 so far), but all browsers except Safari have support for WebAssembly threading and atomics now. It's time to start thinking about adding support for it in SwiftWasm.

I wonder if this is blocked by WebAssembly/wasi-libc#209, or can we rely on those multi-threading APIs without pthreads?

We also need to clarify if WebAssembly platform in SwiftPM should have versions so that users declare compatibility with new WebAssembly features in their Package.swift? How should we name those versions? .wasi(.mvp) and .wasi(.threads), but then the latter implies that WASI itself supports threads, and yet it doesn't. Maybe we should rely on year-based versioning approach, similar to what babel did with EcmaScript features versioning? Like .wasi(.year2020) for our current feature set w/o threads, and .wasi(.year2021) for the future release with threading support? (Exact names for these year-based version cases requires more bikeshedding).

[kateinoigakukun]

I think we need to provide a way to enable various features on wasm.
If thread feature is enabled, we need to link thread enabled version wasi-libc. And if thread feature is disabled, we need to link no-thread version wasi-libc. And we have to provide so many prebuilt wasi-libc libraries for each combinations of features.

[MaxDesiatov]

And then prebuilt stdlib and runtime with and without threading and atomics. 🤔 Maybe separate SwiftWasm distributions so that we don't blow up the distribution size?

[kateinoigakukun]

If we can remove wasi dependency from our toolchain and provide wasi-libc as SwiftPM package, the distribution issue would be resolved, I think.

As far as I know, stdlib doesn't provide threading API, so I think toolchain can be feature flag independent.

[MaxDesiatov]

Foundation (and transitively XCTest) still depend on wasi-libc, even if we remove it from runtime and stdlib. I wish Foundation and XCTest were just SwiftPM packages 😔

[MaxDesiatov]

There's also Dispatch, which is also a part of the standard SDK and isn't distributed via SwiftPM.

[kateinoigakukun]

Hmm, I think Foundation can't provide threading API until wasm thread will be standardized 😢 .

[MaxDesiatov]

The proposal is in the "Proposed Spec Text Available (CG + WG)" stage, and would Foundation expose any platform-specific stuff? Wouldn't we hide it in Foundation under the Thread API and DispatchQueue in Dispatch?

[kateinoigakukun]

Wouldn't we hide it in Foundation under the Thread API and DispatchQueue in Dispatch?

I think if the linked binary has any thread related instruction, the runtime that doesn't support thread feature will crash.

[MaxDesiatov]

For future reference, here's a ticket for SharedArrayBuffer support in WebKit, whcih is one of the things blocking us.

[yonihemi]

Have we seen this before? As of Release 117 Safari Technology Preview supports Wasm threading (SharedArrayBuffer, Atomics, shared Wasm memory) behind a feature flag!

To check it out run

__XPC_JSC_useSharedArrayBuffer=1 open -a "Safari Technology Preview"

[MaxDesiatov]

Great news, I hope it's shipped this year in Safari 15 enabled by default.

I'm still not sure what's the plan in WASI for threads, and whether enabling pthreads is something they're interested in. Maybe we'll have to spawn workers and add shared memory boilerplate on our side instead? 🤔

[kateinoigakukun]

The latest status will be tracked in #5548