heads up re: proposal to add Float16Array to JavaScript

[bakkot]

I'm on TC39, and I'm working on advancing a proposal for Float16Array in JavaScript, which would hold IEEE binary16 floats (also called float16, fp16, f16, half precision floats, etc). I note that this spec has support for float16 values, and there is an outstanding issue in the spec about using ArrayBuffers containing float16s, which is currently impractical due to the lack of Float16Array in JS (the issue links to a thread in which this comment says "It would be good to have the Float16Array support").

The proposal is currently at stage 2, which means TC39 has not yet committed to add it to JS. But since it seems directly relevant here, I wanted to make participants here aware that it may be coming in the hopefully not-to-distant future.

[fdwr]

since it seems directly relevant here, I wanted to make participants here aware

Thanks @bakkot - looking forward to Float16Array over passing raw bits via Uint16Array.

[wchao1115]

@bakkot Please also consider adding comparable typed arrays for float8 and bfloat16 at the same time. These two new data types have gained support from major GPU hardware vendors in recent years. More ML models will be taking advantage of these in the near future. If you are adding float16, these two types should probably be added also.

[anssiko]

I'll assign a v2 label to this issue to signal the WG is supportive of TC39's Float16Array proposal. Once the proposal has matured adequately we should look into adding support in WebNN API.

This half-precision float is particularly useful and demonstrably improves inference performance of e.g. latent text-to-image diffusion models (Stable Diffusion et al.) and super-resolution models (discussed in #127). This is worth prototyping already, results to be shared in this issue to keep interested folks informed.

[bakkot]

Update: the proposal to add Float16Array to JavaScript today reached stage 3, meaning the design is finished, the committee is in favor, and engines can start implementing and shipping it.

[anssiko]

@bakkot thanks for letting us know and congrats! We're in process of updating the WebNN API spec accordingly in #386.

[anssiko]

@bakkot I'm happy to let you know Float16Array just landed into the WebNN API spec as an official reference. Much thanks for specifying this feature important to WebNN API! Please feel free to close this issue if you're satisfied with our resolution.

Also, please let us know if there are substantive developments for Float16Array or other new kinds of TypedArrays or ECMA features that may intersect with this WG's interests. We may get in touch with you in the near future for some proposals.

[bakkot]

Good to hear! I'll go ahead and close this issue.

The main remaining thing needed for Float16Array to be viable is for engines to actually implement it. You can follow along at this issue. I don't have the experience to do so myself, so it'll be up to the authors of the various engines to prioritize that work.

If anyone on this project has experience working with the implementation of TypedArrays in mainstream JS engines and wants to contribute, the issue linked above has pointers to the relevant issues on the the bug trackers for the big three engines.

[anssiko]

(Recent float16 implementation experience via @phoddie in TC39 issue tc39/proposal-float16array#13 and PR tc39/proposal-float16array#14 Feedback welcome in the upstream issue/PR. Implementation status: tc39/proposal-float16array#7)