error in buffer underrun and recovery observations

[jpiesing]

The observation for "Buffer Underrun and Recovery" / duration (below) is too simple.

1) The playback duration shall match the duration of the CMAF Track, i.e. TR[k,S] = TR[k,1] + td[k] taking into account:
    ◦ Missing starting and ending frames.
    ◦ Potential start frames being rendered before playback.
    ◦ Frozen last frame after the playback ended.
    ◦ waiting_duration as the time to wait before appending new data to the buffer after a stall event.

It does not take account of delays like;

• A delay between when the last frame before the under-run is shown and when the video element transitions into waiting or
• A delay between when fragments start being appended after waiting_duration and when the first new frame becomes visible

@yanj-github

[jpiesing]

I suggest replacing the following;

• 

8.17.5 Required Observations
8.17.5.1 General
1) The playback duration shall match the duration of the CMAF Track, i.e. TR[k,S] = TR[k,1] + td[k] taking into account:
◦ Missing starting and ending frames.
◦ Potential start frames being rendered before playback.
◦ Frozen last frame after the playback ended.
◦ waiting_duration as the time to wait before appending new data to the buffer after a stall event.

with

• 

8.17.5 Required Observations
8.17.5.1 General
1) The playback duration shall match the duration of the CMAF Track, i.e. TR[k,S] = TR[k,1] + td[k] taking into account:
◦ Missing starting and ending frames.
◦ Potential start frames being rendered before playback.
◦ Frozen last frame after the playback ended.
◦ variable and implementation specific delays between;
- the freezing of the last frame before the buffer under-runs and the video element transitioning into the waiting state
- new data being appended and playback starting.

i.e.

If t2 is the time (in the recording) when the 2nd frame is first detected, t-u is the time the last frame displayed before the under-run is detected, t-n is the time the first frame after the under-run is detected and t-e is the time the last frame is first detected then the value that should match the CMAF duration is

(t-u minus t2) + (t-e minus t-n) plus 3 frame durations (for the first frame, the last frame before the under-run and the last frame).

The first and last frames are excluded because they can be held on screen for an implementation dependent time.
The last frame before the under-run is also held on the screen for an implementation dependent time.

[gitwjr]

Current observation appears to not take into account 2 additional waiting periods.
Requires input from Resillion.
May require a spec change.
Inclusion in first Launch is at risk.

[yanj-github]

OF only take into account adjustments starting and end of playback not in the middle of playback.

• the freezing of the last frame before the buffer under-runs and the video element transitioning into the waiting state
• new data being appended and playback starting.

I guess OF need to updated to take into account adjustments before the buffer under-runs and playback starts.
This require OF code change.
Do we need to take into account of missing ending and starting frames at buffer under-runs?

[jpiesing]

After taking a more detailed look at a recording, I note the following.

Frame number
in recording
2169 - 2nd frame should be detectable for the first time
3351 - frame 248 visible for the first time which is the frame shown thoughout the underrun
3621- frame 249 first detectable
6473 - last (red) frame first detectable

Playing time excluding 4 frames that may be displayed for longer than normal =((6473-4069)+(3351-2169))/120 = 29.8833333

Playing time corrected for 4 frames at 40ms = 30.04333333

The problem is the difference between 3621 and 3351 = 270 frames = 2.25s.
waiting_duration is 500 so the additional time mentioned in my original problem description are more than 1.5s.

I suggest the spec be changed to define the playback duration be the time the media timeline is advancing, i.e. to exclude the complete time between the first appearance of the last frame decoded before the stall and the first appearance of the first frame decoded after the stall.

[jpiesing]

Here is an updated proposal.

8.17.5.1 General

The playback duration shall match the duration of the CMAF Track, i.e. TR[k,S] = TR[k,1] + td[k] taking into account:

• Missing starting and ending frames.
• Potential start frames being rendered before playback.
• Frozen last frame after the playback ended.
• The last frame before playback stalled and the first frame when playback resumed both being displayed for an implementation-specific period

The time between when playback stalls and when it resumes is not included in the playback duration.

Since github doesn't support underline I can't do a Word style underline / strikethrough but the last line of the current text is removed and replaced by the last 2 lines above.

[yanj-github]

Can we update the spec for low-latency-short-buffer-playback test as well similar to buffer underrun and recovery.

[jpiesing]

Implemented in Draft-CTA-5003-Ae-v2.04.

[jpiesing]

Can we update the spec for low-latency-short-buffer-playback test as well similar to buffer underrun and recovery.

I've looked at low-latency-short-buffer-playback and I can't see how the similar additions would fit. Please can you give an example of what you think would help?

[yanj-github]

Can we update the spec for low-latency-short-buffer-playback test as well similar to buffer underrun and recovery.

I've looked at low-latency-short-buffer-playback and I can't see how the similar additions would fit. Please can you give an example of what you think would help?

All three best devices not went into waiting status, and I don't have other recordings to check.
However, from reading the spec:

• Observe:
o If the video element transitions into playing state, cancel the timeout.
o If the timeout elapses before the video element has transitioned out of the waiting state, throw an error and terminate the test.
• In case the timeout resolves successfully: Append the rest of the CMAF chunks of CMAF track k.

Are we expecting to see "waiting" in playback and device would display a frame longer when "waiting" then transitions into playing state?
Also not clear what is meant by taking into account "The waiting_duration."?

1) The playback duration shall match the duration of the CMAF Track, i.e. TR[k,S] = TR[k,1] + td[k] taking into account:
○ Missing starting and ending frames.
○ Potential start frames being rendered before playback.
○ Frozen last frame after the playback ended.
○ The waiting_duration.

[yanj-github]

low-latency-short-buffer-playback test now has it's own issue #128
And OF code change issue related to buffer underrun and recovery observations is created here cta-wave/device-observation-framework#78