-
Notifications
You must be signed in to change notification settings - Fork 756
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Complete the support for try_table. #6814
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR!
Some initial comments on the code. I haven't read the tests yet.
Aside from those comments the code looks right to me - nice work! |
0b235e4
to
f38aa74
Compare
Ah ah! I found the issue with the refined block types. There was some code missing from |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, one thing to consider here is that any new tests that are added get quickly looked at by our fuzzer. Until we are certain the fuzzer is ready for try_table, please add the new tests here: Lines 355 to 358 in fb6ead8
(You can also run that script locally (instructions at the top) to see if it finds anything, to check if that is needed.) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Spec tests LGTM, thanks!
This includes adding interpreter support for `exnref` values. Since they must carry a `Name tag` in addition to `Literals payload`, they did not fit into existing `GCData`. Therefore, we add another possible type in the big `union` for `ExnData`. Alternatively, we could make a non-`funcref` `Name` value a valid choice for a single `Literal`, but that would require inventing a fake `Type` for such literals, such as `Type::tag`, which would have no spec equivalent. This highlighted the fact that using `Name`s will not work when cross module boundaries (for `funcref` nor `exnref` nor the thrown `WasmException`s). Since fixing this is beyond the scope of this commit, we leave TODO's in place. Other than the above, the changes are straightforward. They are inspired from existing handling of `Try` and `Break`.
f38aa74
to
0f0f5a5
Compare
Thank you for the reviews. I believe I have addressed all the points that were raised. I also completed the translation of I don't consider this WiP anymore. If you're happy with it, I think it's complete. To be fully transparent, there are two things that were not done here: There is a TODO in Possibly more important: I discovered |
Looks like I found an issue but with |
Yeah, that is a larger issue that we need to think about. I agree with your analysis. We can land this without that.
Ah, no need: that was a recent landing that has since been fixed by ignoring f16 in the fuzzer in #6822. Merging latest main should avoid it for you. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work! Here are a few minor comments on the code. I'll look at the tests next.
Btw, please don't squash and force-push, if possible: This is a large PR, and seeing incremental small commits makes it easier to do incremental reviews. We will squash all the commits when landing anyhow.
|
||
(tag $e) | ||
|
||
(tag $any (param (ref any))) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indentation here looks different than the function before. Also, please put the tags first, before functions.
Oops, sorry. Different habits in different repos. I should have asked first. This time I added a commit addressing the latest batch of comments. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the work! Haven't checked the tests yet, but the comments for the code part:
src/ir/linear-execution.h
Outdated
case Expression::Id::TryTableId: { | ||
self->pushTask(SubType::doVisitTryTable, currp); | ||
self->pushTask(SubType::doNoteNonLinear, currp); | ||
break; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't we need to push subtasks for the try_table
's body?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah yes, you're right. Good catch.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does any test cover this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now yes: simplify-locals-eh.wast
covers this.
src/wasm-interpreter.h
Outdated
} | ||
} | ||
if (curr->catchRefs[i]) { | ||
ret.values.push_back(self()->makeExnData(e.tag, e.values)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about calling makeExnData
in visitThrow
and just reuse it when catching/rethrowing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ping 😀
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did that in the branch that changes WasmException
but in which I'm stuck:
sjrd/binaryen@full-support-try-table...sjrd:binaryen:exnref-in-wasmexception
I don't think I can apply this particular suggestion without changing WasmException
, can I?
src/wasm-interpreter.h
Outdated
struct WasmException { | ||
// TODO: handle cross-module calls using something other than a Name here. | ||
Name tag; | ||
Literals values; | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now that we have exnref
, this can simply be
struct WasmException {
WasmException(Literal exn) : exn(exn) {}
Literal exn;
};
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried to something like that, but I'm running into issues that I have a hard time troubleshooting. I get the following errors when running the spec
tests:
executing: /localhome/doeraene/projects/binaryen/bin/wasm-shell /localhome/doeraene/projects/binaryen/test/spec/exception-handling-legacy.wast
('run_command failed (-11)', '0 BUILDING MODULE [line: 1]\n1 CHECKING [line: 288]\n----------------------------------------\nCAUGHT EXCEPTION\n2 CHECKING [line: 289]\n----------------------------------------\nCAUGHT EXCEPTION\n3 CHECKING [line: 290]\n4 CHECKING [line: 291]\n')
Traceback (most recent call last):
File "/localhome/doeraene/projects/binaryen/./check.py", line 214, in run_spec_tests
actual = run_spec_test(wast)
File "/localhome/doeraene/projects/binaryen/./check.py", line 190, in run_spec_test
output = support.run_command(cmd, stderr=subprocess.PIPE)
File "/localhome/doeraene/projects/binaryen/scripts/test/support.py", line 193, in run_command
raise Exception(('run_command failed (%s)' % code, out + str(err or '')))
Exception: ('run_command failed (-11)', '0 BUILDING MODULE [line: 1]\n1 CHECKING [line: 288]\n----------------------------------------\nCAUGHT EXCEPTION\n2 CHECKING [line: 289]\n----------------------------------------\nCAUGHT EXCEPTION\n3 CHECKING [line: 290]\n4 CHECKING [line: 291]\n')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/localhome/doeraene/projects/binaryen/./check.py", line 408, in <module>
sys.exit(main())
File "/localhome/doeraene/projects/binaryen/./check.py", line 391, in main
TEST_SUITES[test]()
File "/localhome/doeraene/projects/binaryen/./check.py", line 220, in run_spec_tests
shared.fail_with_error(str(e))
File "/localhome/doeraene/projects/binaryen/scripts/test/shared.py", line 334, in fail_with_error
raise Exception(msg)
Exception: ('run_command failed (-11)', '0 BUILDING MODULE [line: 1]\n1 CHECKING [line: 288]\n----------------------------------------\nCAUGHT EXCEPTION\n2 CHECKING [line: 289]\n----------------------------------------\nCAUGHT EXCEPTION\n3 CHECKING [line: 290]\n4 CHECKING [line: 291]\n')
I pushed my attempt so far at sjrd/binaryen@full-support-try-table...sjrd:binaryen:exnref-in-wasmexception
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@sjrd You can run individual spec tests using e.g.
bin/wasm-shell test/spec/exception-handling-legacy.wast
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm attending some seminar Tue-Wed, so I can't take a look now, but will take a look soon, unless it's solved by then. But I think this can be done by a follow-up too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@sjrd You can run individual spec tests using e.g.
bin/wasm-shell test/spec/exception-handling-legacy.wast
Thanks, that helps a bit. At least now I know I'm running into a segfault during catch handling inside visitTry
. It's a bit of a heisenbug: depending on how much printing I do, I get further or not. I think I'm out of my depth in terms of understanding how C++ handles exception structs that have a destructor. 😕
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(To be clear, I am not saying you should do the followup - could be anyone.)
src/wasm-interpreter.h
Outdated
WasmException exn; | ||
exn.tag = exnData->tag; | ||
for (auto item : exnData->payload) { | ||
exn.values.push_back(item); | ||
} | ||
throwException(exn); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
WasmException exn; | |
exn.tag = exnData->tag; | |
for (auto item : exnData->payload) { | |
exn.values.push_back(item); | |
} | |
throwException(exn); | |
throwException(exnData); |
This can simplified if we change WasmException
to take ExnData
literal. See my comment on line 49-53.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Haven't read the tests yet, but I wonder why the old -legacy
tests have changed or deleted?
* Do not combine with `--flatten`. * Requires reference types for the `exnref` type.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @aheejin for the review. I addressed everything except the change to WasmException
. I haven't managed to make it work yet. I left a comment with a link to my attempts.
src/ir/linear-execution.h
Outdated
case Expression::Id::TryTableId: { | ||
self->pushTask(SubType::doVisitTryTable, currp); | ||
self->pushTask(SubType::doNoteNonLinear, currp); | ||
break; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah yes, you're right. Good catch.
src/wasm-interpreter.h
Outdated
struct WasmException { | ||
// TODO: handle cross-module calls using something other than a Name here. | ||
Name tag; | ||
Literals values; | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried to something like that, but I'm running into issues that I have a hard time troubleshooting. I get the following errors when running the spec
tests:
executing: /localhome/doeraene/projects/binaryen/bin/wasm-shell /localhome/doeraene/projects/binaryen/test/spec/exception-handling-legacy.wast
('run_command failed (-11)', '0 BUILDING MODULE [line: 1]\n1 CHECKING [line: 288]\n----------------------------------------\nCAUGHT EXCEPTION\n2 CHECKING [line: 289]\n----------------------------------------\nCAUGHT EXCEPTION\n3 CHECKING [line: 290]\n4 CHECKING [line: 291]\n')
Traceback (most recent call last):
File "/localhome/doeraene/projects/binaryen/./check.py", line 214, in run_spec_tests
actual = run_spec_test(wast)
File "/localhome/doeraene/projects/binaryen/./check.py", line 190, in run_spec_test
output = support.run_command(cmd, stderr=subprocess.PIPE)
File "/localhome/doeraene/projects/binaryen/scripts/test/support.py", line 193, in run_command
raise Exception(('run_command failed (%s)' % code, out + str(err or '')))
Exception: ('run_command failed (-11)', '0 BUILDING MODULE [line: 1]\n1 CHECKING [line: 288]\n----------------------------------------\nCAUGHT EXCEPTION\n2 CHECKING [line: 289]\n----------------------------------------\nCAUGHT EXCEPTION\n3 CHECKING [line: 290]\n4 CHECKING [line: 291]\n')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/localhome/doeraene/projects/binaryen/./check.py", line 408, in <module>
sys.exit(main())
File "/localhome/doeraene/projects/binaryen/./check.py", line 391, in main
TEST_SUITES[test]()
File "/localhome/doeraene/projects/binaryen/./check.py", line 220, in run_spec_tests
shared.fail_with_error(str(e))
File "/localhome/doeraene/projects/binaryen/scripts/test/shared.py", line 334, in fail_with_error
raise Exception(msg)
Exception: ('run_command failed (-11)', '0 BUILDING MODULE [line: 1]\n1 CHECKING [line: 288]\n----------------------------------------\nCAUGHT EXCEPTION\n2 CHECKING [line: 289]\n----------------------------------------\nCAUGHT EXCEPTION\n3 CHECKING [line: 290]\n4 CHECKING [line: 291]\n')
I pushed my attempt so far at sjrd/binaryen@full-support-try-table...sjrd:binaryen:exnref-in-wasmexception
I let it run for more than 12,000 iterations without issue. |
The previous implementation caused segmentation faults when exnrefs were manipulated a bit too much.
Oh sorry for the confusion. This was about the new EH, as implemented in an early state of this PR. Porting more tests from old EH to new EH highlighted things I had not yet taken care of in the implementation. This comment is obsolete by now. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly test names and comments nits. I think you can "Add suggestion to batch" on the ones you'd like to accept and batch-commit them, without manually fixing them.
Overnight I ran the fuzzer while allowing it to use the new EH sources as inputs. I did not discover any issue within wasm-opt, but I think I found a bug in the V8 validator (after ~6,000 iterations). It refuses to validate the following module: (module
(type $0 (func))
(func $0
(throw_ref
(ref.null noexn)
)
)
) with the error
I don't think this is correct, because It looks similar to https://issues.chromium.org/issues/332931390 which I had already reported some time ago. |
That does look like a bug. Until v8 fixes it, we'll need to disable fuzzing of exnref. That means disabling all exceptions testing in v8 since we don't have separate feature flags, which is somewhat unfortunate, but we will still be fuzzing it in our interpreter so it's not too bad. To disable it in V8 only, see Lines 840 to 844 in d567578
|
src/ir/possible-contents.h
Outdated
@@ -473,6 +473,12 @@ struct TagLocation { | |||
} | |||
}; | |||
|
|||
// The location of an exnref materialized by a catch_ref or catch_all_ref clause | |||
// of a try_table. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// of a try_table. | |
// of a try_table. No data is stored here (exnrefs contain a stack at runtime, but we don't track that), so this is the same as NullLocation in a way: we just need *a* source of contents for places that receive an exnref. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's been a while since I read GUFA code so my understanding is bad, but why are all exnrefs treated the same? All nulls are actually the same value and can be used interchangeably, but each exnref contains a different payload and tag, no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't the payload separate from the exnref? When we catch with an exnref we jump to a block that gets the payload as multivalues + an exnref at the end IIUC.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GUFA doesn't do any abstract interpretation, AFAICT. It only tracks specific literals, and unknown things of certain types. So all unknown (ref exn)
are the same anyway.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
exnref
conceptually contains the payload, even though it is opaque, no? I think we can treat it in the same way we treat externref
or anyref
, given that they also contain something opaque. How do we treat them in GUFA? Do we treat all of them as the same literal?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the new EH (exnref) scheme, when we rethrow an exnref using throw_ref, we rethrow the whole package, including a tag and a payload. So exnref contains both.
Oh, I didn't realize that. So when we catch, we are sending the payload as values, and also sending it wrapped in the
exnref
object (which also has the stack and tag id)? So we send it both wrapped and unwrapped?
No I think, at least conceptually (because the actual implementation in the engines can differ), we just send a package (=exnref). And when caught, depending on the kind of catch
clauses, we provide the exnref
as is or extract values from it and provide the values.
I think we can treat it in the same way we treat externref or anyref, given that they also contain something opaque.
Makes sense, yeah. That is basically what the code does. E.g. if an import returns an anyref, then we know nothing about it but the type, so we fill that
ResultLocation
(result of a function) withaddRoot
, which just sets it to an unknown value of that type. The code here does the same, but we do need a Location to read the exnref from (it isn't aResultLocation
or anything else we have, yet), so we add that as well.
Still not sure why we need a separate location kind, given that externref
/anyref
doesn't have any. exnref
here is just a return value from a block
, so it can be treated as such, no? Do we need a separate Location
kind for every kind of block return value? Or it can be treated the same way as a return value of a br
, because it sends the info as the block return value.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The issue is, where does the block return value receive the value from? anyref
arrives from e.g. a function result, which we already have a Location type for. But here we are not just adding a new type but also a new control flow construct that sends that value.
But maybe we could instead add a root on the target of the branch, though, is that what you mean? I suppose there is just one such target, and we know exactly what it is. I think that would be safe and correct to do here (that location can't be marked as a root for any other reason), but it does seem a little cleaner to me to keep the code as it is, since the branch target is not conceptually a "root" (e.g. it may receive values from other branches than this, so it needs to combine them all. We do know that this unknown value will "win", but conceptually we are still combining there).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm I see. Then I guess even though all exnref
s are treated as the same constant it wouldn't matter in GUFA because it doesn't do anything with constants, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, we just move constants around, but don't do anything with the values otherwise. So this should be ok as is, I think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm I see. Then I guess even though all
exnref
s are treated as the same constant it wouldn't matter in GUFA because it doesn't do anything with constants, right?
They're not treated like the same constant. They're all treated as some unknown value of type (ref exn)
. GUFA makes a difference between actual constants and unknown values of some type. If we did register them as the same constant, that would be bad, as GUFA would then propagate that constant at all destinations.
I won't have a computer until Tuesday, do I'll come back to this then. For the conflicts, should I resolve them by merging main into this branch, or rebasing on top of main? |
@sjrd Merging would be better, thanks. |
With the latest changes, and with disabling the following lines from the fuzzer (see comments here): binaryen/src/tools/fuzzing/fuzzing.cpp Lines 3304 to 3310 in 2c9c74d
I could successfully run 20,000+ iterations of the fuzzer on this PR, when also allowing the fuzzer to take the new EH tests as inputs. @kripken Do you think that's good enough to remove the following lines: |
Nice! Yes, that seems enough. Likely the fuzzer will eventually find more issues but so long as there is nothing known and high-frequency that fails, that's what we want. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm % any remaining questions @aheejin has, and to enable the fuzzer as just discussed.
Thanks. Fuzzer enabled. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the work!
Thanks from me as well @sjrd , this is great! |
(Motivation: the Scala.js Wasm backend emits
try_table
s, and we would like to pipe our output towasm-opt
.)This includes adding interpreter support for
exnref
values. Since they must carry aName tag
in addition toLiterals payload
, they did not fit into existingGCData
. Therefore, we add another possible type in the bigunion
forExnData
.Alternatively, we could make a non-
funcref
Name
value a valid choice for a singleLiteral
, but that would require inventing a fakeType
for such literals, such asType::tag
, which would have no spec equivalent.Other than the above, the changes are straightforward. They are inspired from existing handling of
Try
andBreak
.Currently WiP:
The new tests inlocal-subtyping.wast
fail. Some pass somewhere is refining the types of blocks that are the target of catch clauses in an incorrect way. The obvious candidates--finalize
,operateOnScopeNameUsesAndSentTypes
--seem to be doing the right thing, so I am missing something somewhere.I would appreciate some guidance about the above questions.
Can someone point me in the right direction for the first one, in particular?Also, is the approach taken for
ExnData
acceptable? Should I do it differently?Thanks in advance