[android] Handle mediacodec callback with a handler thread #2044
Conversation
|
Hi guys, this change effectively prevents the decoded frame expired to improve the frame drop. please help to review it. thanks. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2044 +/- ##
==========================================
+ Coverage 58.45% 58.55% +0.10%
==========================================
Files 1904 1904
Lines 94327 94522 +195
==========================================
+ Hits 55138 55352 +214
+ Misses 39189 39170 -19 ☔ View full report in Codecov by Sentry. |
starboard/android/apk/app/src/main/java/dev/cobalt/media/MediaCodecBridge.java
Outdated
Show resolved
Hide resolved
starboard/android/apk/app/src/main/java/dev/cobalt/media/MediaCodecBridge.java
Outdated
Show resolved
Hide resolved
|
@borongc It would be great to understand the performance impact. |
mingchou
force-pushed
the
mediacodec_cb_with_thread
branch
from
fb9e04c to
38710b1
Compare
|
Hi Bo-Rong and Xiaoming, may I have your comments about this commit? It is really helpful for a tight CPU resource. If you need any information, please feel free to let us know. Thanks. |
Hi Topi, thank you for contributing the PR. @borongc will double check that the PR doesn't cause any performance regression during normal playbacks. Please feel free to let us know if it's high priority to get this PR merged, so we can shuffle tasks around accordingly. |
kaidokert
force-pushed
the
mediacodec_cb_with_thread
branch
from
38710b1 to
8547d12
Compare
mingchou
force-pushed
the
mediacodec_cb_with_thread
branch
from
8547d12 to
7c1d952
Compare
|
The pr looks good to me. The only question is if we want to add more attribute in mime string for features. @xiaomings, what do you think? |
It should be ok here. We no longer use mime attribute to configure media features due to that it increases the complexity of the web app when a set of interdependent mime attributes are being maintained. We may still use mime attribute to turn off an feature under emergencies, i.e. the mime attribute shouldn't be used by the webapp under normal situation. This allows us to pass information down to SbPlayer without introducing an extension. When we expose a feature to be used by the web app under normal situations (i.e. to configure the append buffer size), we should avoid using mime attribute, and use dom interface instead. |
borongc
force-pushed
the
mediacodec_cb_with_thread
branch
2 times, most recently
from
db0048a to
2268a44
Compare
|
As there are 3 on_device tests keep failing, we are investigating the possible causes. @Topi-Chen and @mingchou would you be able to help as well? Thank you! |
borongc
force-pushed
the
mediacodec_cb_with_thread
branch
from
58686f7 to
264be75
Compare
mingchou
force-pushed
the
mediacodec_cb_with_thread
branch
from
264be75 to
3743985
Compare
Now the MediaCodec callbacks and the activity lifecycle callbacks are called on the main thread. When pausing YouTube, the MediaCodec callbacks may be blocked until the lifecycle callbacks are done. This may cause frame drops on a resource-limited device when the MediaCodec callback to notify a newly decoded frame is blocked and we run out of decoded frames. Running the MediaCodec callbacks on a handler thread could avoid this problem. Change-Id: I5ffdb1f5a582c3d01964b3f98c99d7aed211674a
borongc
force-pushed
the
mediacodec_cb_with_thread
branch
from
3743985 to
13de765
Compare
Now the MediaCodec callbacks and the activity lifecycle callbacks are called on the main thread. When pausing YouTube, the MediaCodec callbacks may be blocked until the lifecycle callbacks are done. This may cause frame drops on a resource-limited device when the MediaCodec callback to notify a newly decoded frame is blocked and we run out of decoded frames. Running the MediaCodec callbacks on a handler thread could avoid this problem. Change-Id: I5ffdb1f5a582c3d01964b3f98c99d7aed211674a b/316008643 (cherry picked from commit ecde722)
|
Important Creating the cherry pick PR failed! Check the log at https://github.com/youtube/cobalt/actions/runs/7644837688 for details. |
) Now the MediaCodec callbacks and the activity lifecycle callbacks are called on the main thread. When pausing YouTube, the MediaCodec callbacks may be blocked until the lifecycle callbacks are done. This may cause frame drops on a resource-limited device when the MediaCodec callback to notify a newly decoded frame is blocked and we run out of decoded frames. Running the MediaCodec callbacks on a handler thread could avoid this problem. Change-Id: I5ffdb1f5a582c3d01964b3f98c99d7aed211674a b/316008643 (cherry picked from commit ecde722)
Now the MediaCodec callbacks and the activity lifecycle callbacks are called on the main thread. When pausing YouTube, the MediaCodec callbacks may be blocked until the lifecycle callbacks are done. This may cause frame drops on a resource-limited device when the MediaCodec callback to notify a newly decoded frame is blocked and we run out of decoded frames. Running the MediaCodec callbacks on a handler thread could avoid this problem. Change-Id: I5ffdb1f5a582c3d01964b3f98c99d7aed211674a b/316008643 (cherry picked from commit ecde722)
Now the MediaCodec callbacks and the activity lifecycle callbacks are called on the main thread. When pausing YouTube, the MediaCodec callbacks may be blocked until the lifecycle callbacks are done. This may cause frame drops on a resource-limited device when the MediaCodec callback to notify a newly decoded frame is blocked and we run out of decoded frames. Running the MediaCodec callbacks on a handler thread could avoid this problem. Change-Id: I5ffdb1f5a582c3d01964b3f98c99d7aed211674a b/316008643 (cherry picked from commit ecde722)
Now the MediaCodec callbacks and the activity lifecycle callbacks are called on the main thread. When pausing YouTube, the MediaCodec callbacks may be blocked until the lifecycle callbacks are done. This may cause frame drops on a resource-limited device when the MediaCodec callback to notify a newly decoded frame is blocked and we run out of decoded frames. Running the MediaCodec callbacks on a handler thread could avoid this problem. Change-Id: I5ffdb1f5a582c3d01964b3f98c99d7aed211674a b/316008643 (cherry picked from commit ecde722)
…dler thread (#2277) Refer to the original PR: #2044 Now the MediaCodec callbacks and the activity lifecycle callbacks are called on the main thread. When pausing YouTube, the MediaCodec callbacks may be blocked until the lifecycle callbacks are done. This may cause frame drops on a resource-limited device when the MediaCodec callback to notify a newly decoded frame is blocked and we run out of decoded frames. Running the MediaCodec callbacks on a handler thread could avoid this problem. Change-Id: I5ffdb1f5a582c3d01964b3f98c99d7aed211674a b/316008643 Co-authored-by: Stone <[email protected]>
…outube#2044)" This reverts commit ecde722.
…outube#2044)" This reverts commit ecde722.
…outube#2044)" This reverts commit ecde722.
…th a handler thread (#2044)" (#2465) Refer to the original PR: #2458 Reverts #2044 b/326091431 Co-authored-by: Bo-Rong Chen <[email protected]>
) Now the MediaCodec callbacks and the activity lifecycle callbacks are called on the main thread. When pausing YouTube, the MediaCodec callbacks may be blocked until the lifecycle callbacks are done. This may cause frame drops on a resource-limited device when the MediaCodec callback to notify a newly decoded frame is blocked and we run out of decoded frames. Running the MediaCodec callbacks on a handler thread could avoid this problem. Change-Id: I5ffdb1f5a582c3d01964b3f98c99d7aed211674a b/316008643
Now the MediaCodec callbacks and the activity lifecycle callbacks are called on the main thread. When pausing YouTube, the MediaCodec callbacks may be blocked until the lifecycle callbacks are done. This may cause frame drops on a resource-limited device when the MediaCodec callback to notify a newly decoded frame is blocked and we run out of decoded frames. Running the MediaCodec callbacks on a handler thread could avoid this problem.
Change-Id: I5ffdb1f5a582c3d01964b3f98c99d7aed211674a
b/316008643