Support pulling type information directly from .wasm modules

[weswigham]

To better support wasm modules under node's --experimental-wasm-modules flag.

In this PR, I add a new experimentalWasmModules typescript flag, which allows one to import .wasm files directly in TS code without us issuing an error (this is only supported at runtime in node using the node flag mentioned above). When enabled, we parse the contents of referenced .wasm files for their exported function names and types, and generate a declaration file on-the-fly in memory for it based on those types (and that declaration file is what we then bind/check against).

This is still very much a limited experiment, with only a small subset of possible features implemented thus far, but is far enough along to be able to show some new behavior:

In the above, on the left, I'm watch-rebuilding an assemblyscript program into a wasm module. On the right, I'm consuming that module in the editor, and getting live updates (with auto imports) as the backing .wasm module updates. Repo available here.

In any case, this should be a complete enough implementation to be able to gauge interest in the feature.

TODO:

• Iterate on or amend our readFile compiler API so it can more easily return UInt8Array instances, rather than strings (since wasm file contents are most definitely not strings). (You'll note that right now I have a separate host API call for it, which is very painful). Pretty much our entire file pipeline is geared for strings only (especially in the language service) - this is probably the biggest practical blocker for this.
• Issue parse diagnostics for malformed .wasm files (and introduce more early sanity checks on index bounds)
• Traverse .wasm imports when building the program (wasm files may import other wasm (or JS?) files - while this has no bearing on the types of the exports, it may result in the inclusion of more files in the program (which may in turn impact declaration emit))
• Introduce first-class i32/i64/f32/f64 subtypes of number and strongly check that the inputs/outputs of a wasm module are bounds-checked
• Investigate how exported wasm globals, memories, and tables are exposed to a JS importer (if they are at all)
• Actually emit the generated declaration file when declaration: true is set
• Incremental .wasm parsing/tree reuse in the language service (I had to wholly disable incremental LS operations on wasm files to get the LS to function, since they both need nonstring file reads, and are not updatable with the normal incremental parser)

[weswigham]

@typescript-bot pack this

[typescript-bot]

Heya @weswigham, I've started to run the tarball bundle task on this PR at 0f6b16e. You can monitor the build here.

[typescript-bot]

Hey @weswigham, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/80811/artifacts?artifactName=tgz&fileId=1380FCF7BD82EFBC97A07E1949B757ECB181FE3000C238A5C485E5901051D6B602&fileName=/typescript-4.0.0-insiders.20200728.tgz"
    }
}

and then running npm install.

[trusktr]

This would be amazing to have, especially for AssemblyScript users, but also anyone importing Wasm with type declarations. This would help reduce the barrier of entry for people wanting to try WebAssembly.

What would it do to existing JS projects that rely on number?

[trusktr]

Related: #31713

[ExE-Boss]

The emitted module specifier doesn’t look right.

[weswigham]

The test harness strips the leading /.lib/ because it's on the real disk from the host machine (and has the corresponding full, real path) - it only does this in the test harness~

[weswigham]

@typescript-bot pack this

[typescript-bot]

Heya @weswigham, I've started to run the tarball bundle task on this PR at 0f6b16e. You can monitor the build here.

[typescript-bot]

Hey @weswigham, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/87299/artifacts?artifactName=tgz&fileId=61B1B6F57722D3FD3FAF0A139FA54A9A7D6B0ED73C303429C2A43A9235E7276F02&fileName=/typescript-4.0.0-insiders.20201004.tgz"
    }
}

and then running npm install.

[Jack-Works]

Is it possible to extends this possibility to a set of public APIs that allow custom file types (e.g. CSS Modules, GraphQL, ...) to be able to generate in-memory declarations and join the incremental building / watch mode building?

Does this PR copy the .wasm file to the output folder like what tsc already did to the .json files?

[sandersn]

This experiment is pretty old, so I'm going to close it to reduce the number of open PRs.