Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement type indexing and adapt spec #72

Merged
merged 49 commits into from
Feb 21, 2024
Merged

Implement type indexing and adapt spec #72

merged 49 commits into from
Feb 21, 2024

Conversation

rossberg
Copy link
Collaborator

@rossberg rossberg commented Feb 13, 2024

Sorry for this large PR, this change became a bit of a rabbit hole needing many changes.

PTAL at the spec changes in particular.

Changes to the EL:

  • Added type family declarations: syntax id(params); type definitions then define cases for it, by pattern-matching on their args (analogous to function definitions).
  • Added local variable declarations as a new form of premise: -- var x : t (due to the possibility of local type indices, global var declarations are no longer sufficient). This is ignored in rendering.
  • Renamed Syn to Typ throughout the AST, for more consistency.
  • Added !% hole for suppressing an operand in pretty-printing (only for show hints).

Changes to the IL:

  • Type identifiers can have arguments now: id(args), or VarT of id * arg list.
  • Type definitions define a parameter type and a list of instances, to represent families by pattern matching on arguments (analogous to function clauses).
  • Calls are now n-ary and take a list of "arguments", which can be either expressions or types.
  • Function definitions likewise have a parameter list for their domain.
  • Similarly, bindings can be variable or type identifiers.
  • Tuple types can bind identifiers to account for dependencies (this isn't quite the right level, may clean this up).
  • Renamed Syn to Typ throughout the AST, for more consistency.

Changes to AL:

  • Eliminate VecV.

Changes to specs:

  • Values are now indexed by type, e.g. num_(numtype), lane_(lanetype) etc., which allows checking that the correct kind of constant is used everywhere. This uncovered a couple of problems in the rules, e.g.
    • array.new/init_data not distinguishing between CONST and VCONST in its reduction (this is glossed over in the real spec, because it happens to work syntactically),
    • splat and replace_lane not wrapping their operands for packed types (bug in the real spec).
  • Operators are indexed by type, e.g., binop_(numtype) etc., which makes checking tighter, gets rid of all the xyzopXXX definitions, as well as the tedious _I and _F injections (which we would have needed for every constant otherwise, now that we distinguish between int and float values properly).
  • Edit: Refactored dot, extmul and extadd instructions into generic extunop and extbinop instructions, for more regularity and clarity.
  • Narrowed types of some of the AST definitions to rule out unsupported cases, either by using more refined types or by inserting side conditions. Added a few TODOs with more refinements needed.
  • Refined the types of numeric parameters of various meta functions, e.g., def $vcvtop(shape_1, shape_2, vcvtop, sx?, lnum_($lanetype(shape_1))) : lnum_($lanetype(shape_2)).
  • Some parameter reordering in meta functions to account for type dependencies.
  • Changed reduction of VRELOP to be more abstract, analogous to VBINOP.
  • Had to add a few local variable declarations where the types cannot be deduced.
  • $concat is now a polymorphic operation. :)
  • Added various TODOs.

Many corresponding changes in front- and middlend, some adaptation in backends.

Caveats, as mentioned:

  • I deactivated the subtype elimination pass, since it does not work correctly with type indices. (Arguably, it was already broken, because it happily introduced function invocations in argument patterns on the LHS of definitions.)

  • The interpreter now fails 85% of tests. I suspect this is mostly due to the refactoring of the constant and operator syntax that it hard-codes. I tried to fix the obvious places, but there seem to be other causes that I was not able to debug.

@rossberg rossberg mentioned this pull request Feb 13, 2024
92 tasks
@rossberg
Copy link
Collaborator Author

Is anybody opposed to merging this PR? Not sure if anybody has been looking at it, but f you need more time for that, please let me know. If not, I'd like to merge it early next week. (It contains various changes that kind of block most of the follow-up clean-ups that I have on my TODO list.)

@rossberg
Copy link
Collaborator Author

D'oh, sorry. I just noticed that I accidentally committed one change here the other day that I meant to upload separately, namely refactoring dot/extmul/extadd into generic ext{un/bin}op instructions (a change I wanted to do for a while already). But perhaps that's for the better, putting all breaking changes to the operator/interpreter interplay in one place... :)

@rossberg
Copy link
Collaborator Author

OK, landing now. CI is red because I did not update test expectations for the interpreter to include the failures.

@rossberg rossberg merged commit 121800b into main Feb 21, 2024
5 of 6 checks passed
@rossberg rossberg deleted the params-il branch February 21, 2024 10:26
Alan-Liang pushed a commit to Alan-Liang/spectec that referenced this pull request Sep 1, 2024
* [spec] Add reference types to overview (WebAssembly#1394)

* [interpreter] Remove use of physical equality on characters (WebAssembly#1396)

* Merge SIMD proposal (WebAssembly#1391)

SIMD is [phase 5](WebAssembly/simd#507), merge all the changes back into main spec.

* Remove merge conflict marker

* [spec] Handle v128 in validation algorithm (WebAssembly#1399)

* [spec] Fix instruction table (WebAssembly#1402)

* Add tests for functions without end marker. NFC (WebAssembly#1405)

Inspired by this downstream test in wabt:
WebAssembly/wabt#1775

Fixes: WebAssembly#1404

* Describe correct tail call behavior across modules

Whether tail calls across module boundaries would guarantee tail call behavior was previously an open question, but @thibaudmichaud confirmed that they would guarantee tail call behavior in V8 in WebAssembly/tail-call#15 (comment).

* [interpreter] Fix a typo in README (WebAssembly#1406)

* Add a link to the proposals repo (WebAssembly#1409)

Fixes WebAssembly#1407.

* [spec] Add note regarding parameter names (WebAssembly#1412)

* Comments WIP

* Merge upstream (Wasm-DSL#55)

* Typo

* [spec] Clarifying note on text format (WebAssembly#1420)

Signed-off-by: Adrian Cole <[email protected]>
Co-authored-by: Andreas Rossberg <[email protected]>

* Eps

* Eps

* [test] Fix section size in binary test (WebAssembly#1424)

* Update document README to install six

* Disallow type recursion (Wasm-DSL#56)

* Fix import order

* Add --generate-js-only flag to test runner

This will return early right after generating JS from the wast test
files. It will not attempt to run the tests, or do the round trip
conversion from wasm <-> wast.

This is convenient for proposals to add tests without having to
update the reference interpreter with implementation, and generate those
tests to JS to run in other Wasm engines.

Fixes WebAssembly#1430.

* Remove use of let from func.bind test

* Add call3

* [spec] Fix missing mention of vectype (WebAssembly#1436)

Fixed WebAssembly#1435.

* [spec] Fix single-table limitation in module instantiation (WebAssembly#1434)

* [spec] Fix missing immediate on table.set (WebAssembly#1441)

* [docs] Update syntax in examples (WebAssembly#1442)

* Clarification in proposals README

* [interpreter] Tweak start section AST to match spec

* [spec] Bump release to 2 (WebAssembly#1443)

At yesterday's WG meeting, we decided to make a new release, now switching to the Evergreen model. For administrative and technical reasons having to do with W3C procedure, we decided to bump the release number to 2.

From now on, the standard will iterate at version 2 from the W3C's official perspective. We use minor release numbers internally to distinguish different iterations.

(@ericprud, I hope I understood correctly that the Bikeshed "level" also needed to be bumped to 2.)

* Remove test cases with let

* Sync wpt test (WebAssembly#1449)

* [spec] Fix typo (WebAssembly#1448)

* [proposals] Add missing start to example (WebAssembly#1454)

* [spec] "version 2.0" -> "release 2.0" (WebAssembly#1452)

* [spec] Fix typo (WebAssembly#1458)

* [test] Add assert_trap for unreached valid case (WebAssembly#1460)

* [interpreter] Name the type Utf8.unicode

* [spec] Fix binary format of data/elem tags to allow LEB (WebAssembly#1461)

* [spec] Fix typos in numeric operations (WebAssembly#1467)

* [spec] Fix syntax error in element segments validation rule (WebAssembly#1465)

* [spec] Fix typo in global instance syntax (WebAssembly#1466)

* [spec] Fix typos in module instantiation (WebAssembly#1468)

* [interpreter] Turn into a Dune package (WebAssembly#1459)

* [spec] Fix typos in instruction validation rules (WebAssembly#1462)

* [bib] Update latex .bib file for webassembly 2.0 (WebAssembly#1463)

* [spec] Add missing default for vector types (WebAssembly#1464)

* [spec] Fix typos in binary and text formats (WebAssembly#1469)

* [spec] Fix various typos (WebAssembly#1470)

* TypeError for Global constructor with v128

At the moment the spec requires a `LinkError` to be thrown when the `WebAssembly.Global` constructor is called for type `v128`. This was introduced in WebAssembly/simd#360, but according to the PR description, actually a `TypeError` should be thrown. The PR refers to https://github.com/WebAssembly/simd/blob/master/proposals/simd/SIMD.md#javascript-api-and-simd-values, and there a `TypeError` is required.

* [spec] Fix LEB opcodes in instruction index (WebAssembly#1475)

* [spec] Fix v128.loadX_splat in instruction index (WebAssembly#1477)

* [interpreter] Dune test suite (WebAssembly#1478)

* [interpreter] Fix warning flags for OCaml 4.13 (WebAssembly#1481)

* [interpreter] Simplify lexer and avoid table overflow on some architectures (WebAssembly#1482)

* [spec] Editorial nit (WebAssembly#1484)

* [interpreter] Produce error messages in encoder (WebAssembly#1488)

* [spec] Add missing close paren on table abbreviation (WebAssembly#1486)

Also remove an unnecessary space in the previous table abbreviation.

* [spec] Remove outdated note (WebAssembly#1491)

* Eps

* [interpreter] Factor data and element segments into abstract types (WebAssembly#1492)

* [spec] Update note on module initialization trapping (WebAssembly#1493)

* Fix type equality

* Fix typo

* [spec] Add note about control stack invariant to algorithm (WebAssembly#1498)

* [spec] Tweak tokenisation for text format (WebAssembly#1499)

* [test] Use still-illegal opcode (func-refs) (WebAssembly#1501)

* Fix minor typos and consistency issues in the validation algorithm. (Wasm-DSL#61)

* Add definition of defaultable types (Wasm-DSL#62)

No rules for locals yet, since those are still being discussed.

* Remove func.bind (Wasm-DSL#64)

* Implement 1a (Wasm-DSL#63)

* Subtyping on vector types & heap bottom check (Wasm-DSL#66)

* [interpreter] Bring AST closer to spec

* [spec] Fix typo (WebAssembly#1508)

* WIP

* Remove polymorphic variants

* Minor syntactic consistency fix (Wasm-DSL#68)

Change pseudo equality operator from `==` to `=`.

* Bump version

* [spec] Fix table.copy validation typo (WebAssembly#1511)

* More fixes

* Fix Latex

* Adjust intro

* Spec local initialization (Wasm-DSL#67)

* Add table initialiser (Wasm-DSL#65)

* [spec] Remove outdated note (WebAssembly#1491)

* [interpreter] Factor data and element segments into abstract types (WebAssembly#1492)

* [spec] Update note on module initialization trapping (WebAssembly#1493)

* [spec] Add note about control stack invariant to algorithm (WebAssembly#1498)

* [spec] Tweak tokenisation for text format (WebAssembly#1499)

* [test] Use still-illegal opcode (func-refs) (WebAssembly#1501)

* [spec] Fix typo (WebAssembly#1508)

* [spec] Fix table.copy validation typo (WebAssembly#1511)

* Merge fallout

* Latex fixes

* [spec] Minor copy edit (WebAssembly#1512)

* Spec changelog

* [spec] Trivial editorial fix

* Update embedding

* Oops

* Argh

* Rename Sem to Dyn

* Readd match.mli

* [interpreter] Build wast.js with Js_of_ocaml (WebAssembly#1507)

* [interpreter] Add flag for controlling call budget

* Spec zero byte

* Fix table/elem expansion (Wasm-DSL#71)

* Fix merge artefact

* Restrict init from stack-polymorphism (Wasm-DSL#75)

* [spec] Simplify exec rule for if (WebAssembly#1517)

* [spec] Formatting tweak (WebAssembly#1519)

* [spec] Fix typing rule in appendix (WebAssembly#1516)

* [spec] Fix “invertible” typo (WebAssembly#1520)

* [spec] Correct use of opdtype and stacktype (WebAssembly#1524)

* [spec] Add note to instruction index (WebAssembly#1528)

* Add type annotation to call_ref (Wasm-DSL#76)

* [spec] Tweak wording to avoid first person

* Eps

* Eps2

* Eps3

* Remove unneeded assumption type

* [spec/test] Fix scoping of non-imported globals (WebAssembly#1525)

* Fix test

* A couple of tests

* Performance improvement

* Typo

* Another typo

* [spec] Fix language config

* Fix null subtyping being wrong way around (Wasm-DSL#79)

* [spec] Fix naming typo (WebAssembly#1532)

* Defunctorise types again

* [spec] Add citation for WasmCert (WebAssembly#1533)

* [test] Fix async_index.js

* [test] Enable the i64 tests in imports.wast.

Fixes WebAssembly#1514.

* Minor tweak

* [js-api][web-api] Editorial: Fix some minor issues.

Fixes WebAssembly#1064.

* Update README.md (WebAssembly#1540)

Improve wording.

* [spec] Fix typo in element execution (WebAssembly#1544)

* [spec] Remove obsolete note (WebAssembly#1545)

* cccccc[klghketetivvtnnhvntikigrnueuhdkkukljgjuest/meta/generate_*.js: sync upstream JS with tests (WebAssembly#1546)

* [spec] Editorial tweak

* [test] test segment/table mismatch and externref segment (WebAssembly#1547)

* [interpreter] Remove duplicate token declarations (WebAssembly#1548)

* Update Soundness appendix (Wasm-DSL#72)

* [spec] Formatting eps

* Remove oboslete note in README (Wasm-DSL#82)

* Add `print_i64` to generated spec tests

WebAssembly@82a613d added `print_i64` to the standalone test files, but not to the ones generated by the spec interpreter.

* [test] Tweak binary-leb128 and simd_lane (WebAssembly#1555)

* [spec] Allow explicit keyword definitions (WebAssembly#1553)

Rather than describing keyword tokens as always being defined implicitly by terminal symbols in syntactic productions, describe them as being defined implicitly or explicitly. This accounts for the explicit definitions of `offset` and `align` phrases, which are lexically keywords, later in the chapter.

Fixes WebAssembly#1552.

* [js-api] editorial: adjust link for v128 type

* Factor local init tests to local_init.wast; add more (Wasm-DSL#84)

* Update JS API for no-frills

* [spec] Add missing case for declarative elem segments

Fixes WebAssembly#1562.

* [spec] Hotfix last accidental commit

* [spec] Fix hyperref (WebAssembly#1563)

* [spec] Bump sphinx version to fix Python problem

* [spec] Fix minor errors and inconsistencies (WebAssembly#1564)

* Spacing

* Fix a couple more superfluous brackets

* [spec] Eps

* [interpreter] Refactor parser to handle select & call_indirect correctly (WebAssembly#1567)

* [spec] Remove dead piece of grammar

* [test] elem.wast: force to use exprs in a element (WebAssembly#1561)

* Fix typos in SIMD exec/instructions

* Update interpreter README (WebAssembly#1571)

It previously stated that the formal spec did not exist, but the spec has existed for years now.

* [spec] Remove an obsolete exec step (WebAssembly#1580)

* [test] Optional tableidx for table.{get,set,size,grow,fill} (WebAssembly#1582)

* [spec] Fix abstract grammar for const immediate (WebAssembly#1577)

* [spec] Fix context composition in text format (WebAssembly#1578)

* [spec] Fix label shadowing (WebAssembly#1579)

* Try bumping OCaml

* Try bumping checkout

* Adjust for multi-return

* Tweak reduction rules

* Spec return_call_ref

* Fix

* Text format

* [spec] Fix typos in instruction index (WebAssembly#1584)

* [spec] Fix typo (WebAssembly#1587)

* [spec] Remove inconsistent newline (WebAssembly#1589)

* [interpreter] Remove legacy bigarray linking (WebAssembly#1593)

* [spec] Show scrolls for overflow math blocks (WebAssembly#1594)

* [interpreter] Run JS tests via node.js (WebAssembly#1595)

* [spec] Remove stray `x` indices (WebAssembly#1598)

* [spec] Style tweak for cross-refs

* [spec] Style eps (WebAssembly#1601)

* Separate subsumption from instr sequencing

* State principal types

* Add statements about glbs, lubs, and disjoint hierarchies

* Add missing bot

* [spec] Clarify that atoms can be symbolic (WebAssembly#1602)

* [test] Import v128 global (WebAssembly#1597)

* Update Overview.md

* [js-api] Expose everywhere

* [js-api] Try to clarify NaN/infinity handling. (WebAssembly#1535)

* [web-api] Correct MIME type check. (WebAssembly#1537)

Fixes WebAssembly#1138.

* [ci] Pin nodejs version to avoid fetching failures (WebAssembly#1603)

The issues appears to be related to actions/runner-images#7002.

Co-authored-by: Ms2ger <[email protected]>

* [spec] Add missing value to table.grow reduction rule (WebAssembly#1607)

* [test] Move SIMD linking test to simd dir (WebAssembly#1610)

* Editorial: Clarify the name of the instantiate algorithm.

* Add notes to discourage using synchronous APIs.

* [jsapi] Normative: Always queue a task during asynchronous instantiation

JSC will have to do asynchronous compilation work during some instantiations.
To be consistent, this PR always queues a task to complete instantiation,
except through the synchronous Instance(module) API, to ensure consistency
across platforms.

This patch also cleans up the specification in various surrounding ways:
- Include notes about APIs whose use is discouraged/may be limited

Closes WebAssembly#741
See also webpack/webpack#6433

* [test] Exception -> Tag in wasm-module-builder.js

The section name has changed to the tag section a few years ago. This
adds the corresponding changes added in
WebAssembly/exception-handling#252 and
WebAssembly/exception-handling#256.

* [spec] Fix reduction rule for label (WebAssembly#1612)

Fix WebAssembly#1605.

* [spec] Clarifying note about canonical NaNs (WebAssembly#1614)

* [spec] Tweak crossref

* [test] Fix invalid section ID tests (WebAssembly#1615)

* [tests] Disable node run for now

* [spec] Don't check in generated index, to avoid spurious merge conflicts

* [spec] Rename script

* [ci] deactivate node run for now

* Fix uses of \to; compositionality

* Fix typo in text expansion

* Follow-up fix

* Fix compilation errors after merge.

This commit fixes the errors introduced by the merge of
function-references/main into this tree.

---------

Signed-off-by: Adrian Cole <[email protected]>
Co-authored-by: Andreas Rossberg <[email protected]>
Co-authored-by: Hans Höglund <[email protected]>
Co-authored-by: Ng Zhi An <[email protected]>
Co-authored-by: Ng Zhi An <[email protected]>
Co-authored-by: Sam Clegg <[email protected]>
Co-authored-by: Thomas Lively <[email protected]>
Co-authored-by: Gabor Greif <[email protected]>
Co-authored-by: Andreas Rossberg <[email protected]>
Co-authored-by: Crypt Keeper <[email protected]>
Co-authored-by: Andreas Rossberg <[email protected]>
Co-authored-by: Ng Zhi An <[email protected]>
Co-authored-by: Ben L. Titzer <[email protected]>
Co-authored-by: Keith Winstein <[email protected]>
Co-authored-by: gahaas <[email protected]>
Co-authored-by: r00ster <[email protected]>
Co-authored-by: Timothy McCallum <[email protected]>
Co-authored-by: Julien Cretin <[email protected]>
Co-authored-by: Julien Cretin <[email protected]>
Co-authored-by: Ole Krüger <[email protected]>
Co-authored-by: Jämes Ménétrey <[email protected]>
Co-authored-by: Ivan Panchenko <[email protected]>
Co-authored-by: Ethan Jones <[email protected]>
Co-authored-by: Ole Krüger <[email protected]>
Co-authored-by: aathan <[email protected]>
Co-authored-by: Alberto Fiori <[email protected]>
Co-authored-by: mnordine <[email protected]>
Co-authored-by: cosine <[email protected]>
Co-authored-by: ariez-xyz <[email protected]>
Co-authored-by: Surma <[email protected]>
Co-authored-by: Asumu Takikawa <[email protected]>
Co-authored-by: Ian Henderson <[email protected]>
Co-authored-by: Tom Stuart <[email protected]>
Co-authored-by: James Browning <[email protected]>
Co-authored-by: whirlicote <[email protected]>
Co-authored-by: Ms2ger <[email protected]>
Co-authored-by: Adam Lancaster <[email protected]>
Co-authored-by: Ömer Sinan Ağacan <[email protected]>
Co-authored-by: B Szilvasy <[email protected]>
Co-authored-by: Thomas Lively <[email protected]>
Co-authored-by: Michael Ficarra <[email protected]>
Co-authored-by: YAMAMOTO Takashi <[email protected]>
Co-authored-by: Thomas Lively <[email protected]>
Co-authored-by: candymate <[email protected]>
Co-authored-by: Bongjun Jang <[email protected]>
Co-authored-by: ShinWonho <[email protected]>
Co-authored-by: 서동휘 <[email protected]>
Co-authored-by: Jim Blandy <[email protected]>
Co-authored-by: Heejin Ahn <[email protected]>
Co-authored-by: Daniel Ehrenberg <[email protected]>
Alan-Liang pushed a commit to Alan-Liang/spectec that referenced this pull request Sep 2, 2024
Bring `wasmfx` branch up to date.

* Import bug fixes from wasmfx/specfx
* Synchronise with WebAssembly/spec + wasm-3.0 branch.
* Change syntax of resume tables such that they use `on` rather than `tag`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant