web,video: Add H264 decoding using the WebCodecs API #16794
Conversation
torokati44
force-pushed
the
external-video-webcodecs
branch
5 times, most recently
from
65c01d8 to
e9c744b
Compare
torokati44
force-pushed
the
external-video-webcodecs
branch
from
e9c744b to
877b6b1
Compare
torokati44
force-pushed
the
external-video-webcodecs
branch
5 times, most recently
from
816e327 to
c96c40c
Compare
torokati44
force-pushed
the
external-video-webcodecs
branch
from
c96c40c to
43a992c
Compare
torokati44
force-pushed
the
external-video-webcodecs
branch
4 times, most recently
from
9afffe8 to
d40c036
Compare
torokati44
force-pushed
the
external-video-webcodecs
branch
2 times, most recently
from
c47534e to
f3bd05b
Compare
torokati44
force-pushed
the
external-video-webcodecs
branch
2 times, most recently
from
35cf89c to
bf5c71c
Compare
torokati44
force-pushed
the
external-video-webcodecs
branch
2 times, most recently
from
74cf97b to
cf85e0f
Compare
|
Rebased on top of #17878. |
torokati44
force-pushed
the
external-video-webcodecs
branch
from
cf85e0f to
033a7c7
Compare
torokati44
force-pushed
the
external-video-webcodecs
branch
from
033a7c7 to
f4edc57
Compare
torokati44
force-pushed
the
external-video-webcodecs
branch
2 times, most recently
from
98302d1 to
d45c3de
Compare
torokati44
force-pushed
the
external-video-webcodecs
branch
5 times, most recently
from
9250fe2 to
8132286
Compare
torokati44
added
the
waiting-on-review
torokati44
force-pushed
the
external-video-webcodecs
branch
from
8132286 to
c6a19de
Compare
| ))); | ||
| } | ||
| _ => { | ||
| panic!("unsupported pixel format: {:?}", output.format().unwrap()); |
There was a problem hiding this comment.
Admittedly I'm not great with Rust, but does it make sense to panic if this is any other format?
https://developer.mozilla.org/en-US/docs/Web/API/VideoFrame/format and https://rustwasm.github.io/wasm-bindgen/api/web_sys/enum.VideoPixelFormat.html lists a lot more formats than just I420 and BGRX. Is it possible to embed a video in any of those other formats with Flash, and if so, wouldn't the whole thing panic when trying to play them?
There was a problem hiding this comment.
Yeah, good point, this is not nice.
There was a problem hiding this comment.
Replaced the panic with a simple error log. We can add conversion from other formats as needed.
I think what we get here depends on the browser more than the encoded content.
These two were chosen because they were returned by Chrome and Firefox in my testing, on my systems.
torokati44
force-pushed
the
external-video-webcodecs
branch
2 times, most recently
from
55e577c to
a326514
Compare
torokati44
force-pushed
the
external-video-webcodecs
branch
from
a326514 to
345b343
Compare
A continuation of #14654.
Not really done yet, just pushing for visibility, and so that it doesn't get forgotten.
(Just look at the commit history lol.)Due to this API being based on callbacks, this currently adds at least one additional (video) frame of delay as compared to a synchronous decoder, such as OpenH264 on desktop. I think this should be acceptable.
It would be possible to wire the callback straight from the browser through the video backend to the renderer backend, to update the texture immediately (off the regular frame pacing schedule), but I think that would be a bit too complicated, and could potentially result in less smooth playback.
Mostly works, in Chrome and in Firefox 129+.
I don't think this could be problematic at all from a patents/licensing/royalty standpoint.
Advances #8891.