Failure cases, array size limits?

[bvibber]

Array lengths are i32s, but what are the valid ranges? Is it interpreted as a signed number, which may be up to 2^31-1 elements? Or an unsigned number up to 2^32-1 elements?

What happens if an allocation is requested that cannot be fulfilled due to limited memory or address space? array.new_with_rtt and array.new_default_with_rtt are not specified in MVP.md to trap on failure.

Could I try to allocate a ~34-gigabyte 2^32-1 element i64 array? If I try and fail can I recover from it? If I succeed, how many can I make in one process? (cf also #35)

[bvibber]

Note my expectation is that allocations that cannot be fulfilled will trap, but both the current MVP and the SOIL initiative proposal don't explicitly say so. In combination with a fully finished exceptions proposal implementation, a trap would allow recovering from an allocation failure, but it would require both proposals to be implemented together.

[tlively]

We decided on the EH proposal that traps would not be handled as exceptions, so EH won't give you any extra ability to recover from a trap, whether due to allocation failure or otherwise.

[bvibber]

Hmm, if traps can't be caught then handling an allocation failure would require a fallible version of the instruction that branches or returns null on failure?

[tlively]

Yes, that's right. IMO, the MVP should specify trapping on allocation failure, as you suggest, and we should consider fallible allocation instructions in a followup.

[bvibber]

That should work acceptably for my case I think (embedded dynamic scripting language in a host web page). I could catch the trap as an exception at the host JS boundary and throw away the module instance, similar to an OS terminating a process on page fault...

[jakobkummerow]

FWIW, on the web it is common practice to intentionally crash the process on memory exhaustion. The reasoning is that when you can't even allocate any more, then there is no secure+reliable way to handle the situation.

(I'm aware that attempting to allocate a huge array that the application could also do without is a special case that doesn't fit well into this reasoning. I'm not sure how frequent this situation is.)

Aside from that, I'd expect that as the proposal matures, some upper bound will be standardized, at least for web implementations, at https://webassembly.github.io/spec/js-api/index.html#limits. Obviously memory exhaustion might occur even below any such standard-imposed limit.

[bvibber]

Crashing the process, unfortunately, is a denial of service attack against the host web page (brings us back to #35), making the entire exercise of a sandboxed language of limited benefit.

[taralx]

I'm not clear on how this issue is different from #35, honestly.

[bvibber]

It may be fair to merge them back together. I was trying to narrow down the question of how to handle a failure case without worrying about what the specific limits are or if they can be configured somehow.

[bvibber]

One suggestion on the call today was to allocate a 'heap space' similar to a memory, which needs to be referenced for all allocations. This would enable blocking off a smaller heap to use for allocations, and apply limit checks just to those calls. Might or might not be easily implementable in browser engines though.

[fgmccabe]

Having multiple 'gc memories' is already explicitly a potential requirement. This is very important for 'process-oriented' languages like Erlang. But is also potentially useful for Java (with multiple class loaders).

[bvibber]

Ok I'll make two PRs: one that closes #122 in the narrow sense of documenting that allocation failures may be handled in an implementation defined way, up to and including crashing the process or risking DoS of the system. And another to handle #35 by proposing multiple heaps or heap segments that can be limited with a more defined failure mode (which will be more controversial!)

[tlively]

Closing this in favor of the more up-to-date issue on implementation limits, including runtime limits on the size of arrays: #335