9.4 splicing tests not working when fragment number not matching for video and audio

[yanj-github]

The following tests not working:

• /ceac-local/splicing-of-wave-program-with-baseline-constraints__splice_main_splice_ad_at3_at4.html
• /ca4s-local/splicing-of-wave-program-with-baseline-constraints__splice_main_splice_ad_at3_at4.html
  The error that shows is:
  promise_test: Unhandled rejection with value: object "Error: Segment index in playout out of bounds: 6, but only got 5"

Defined playout is:
"playout": [
[1, 1, 1], [1, 1, 2], [1, 1, 3],
[2, 1, 1], [2, 1, 2], [2, 1, 3],
[1, 1, 4], [1, 1, 5], [1, 1, 6]
]

Video stream has 6 fragment but audio has 5 fragment.

This is linked with issue I think. cta-wave/device-playback-task-force#111

From the spec Tsplice is introduced and this is a new new parameter (used instead of playout): both TR and OF will need to be changed - we propose to maintain the "playout" parameter rather than introduce a new one, "playout" parameter - I think we have agreed on this.
And for video and audio play should match video. - I think currently audio playback is trying to use playout which wont work for different fragment number. Can you help with this please?

In addition to this the Audio and Video Switching Set co-incides with a Fragment Boundary is a separate issue and we don't have right content available yet.

[yanj-github]

@FritzHeiden and @louaybassbouss I would like to have your view on this issue please, thank you.

[jpiesing]

I've marked this as a Blocker because 9.4 is one of those run in HbbTV plugfests. However I note that, in the results from Naples, it failed quite significantly even to complete running successfully.

If it's not fixed then we can't include it in the release, perhaps not even as beta.

[yanj-github]

@jpiesing 's comment from cta-wave/device-playback-task-force#111

I think Tsplice is still needed even if you re-introduce playout[].
Also since video and audio fragments may have different durations, aren't two playout's needed, one for video and one for audio? (This would not be needed if only using "playout").

Would it not be something like this ...

For video playout, playout_video[1] to playout [index of fragment ending at Tsplice] = 1, playout[index of fragment starting at Tsplice] to playout[index of final fragment] = 2.
And the same for audio except the indexes would be different to reflect different fragment lengths.

I think we cannot drop playout as it has triple values (Switching Set, CMAF track number, Fragment number) OF requires them to make right observation if there are more than one video track and video switches as well. Tsplice can indicate the splicing point from main to ad but not others such as CMAF track number. If we strict to no video switches on test 9.4 then Tsplice will be fine.

@FritzHeiden and @louaybassbouss when audio and video have different fragment / track number do you need two playouts one for video and one for audio? If that is the case, can we add "audio_playout"? Playouts can indicates splicing point, so no Tsplice is required.

[FritzHeiden]

@FritzHeiden and @louaybassbouss when audio and video have different fragment / track number do you need two playouts one for video and one for audio? If that is the case, can we add "audio_playout"? Playouts can indicates splicing point, so no Tsplice is required.

@yanj-github I think having a playout dedicated for audio would solve this issue of having different fragment counts in video and audio, yes

[yanj-github]

@FritzHeiden and @louaybassbouss when audio and video have different fragment / track number do you need two playouts one for video and one for audio? If that is the case, can we add "audio_playout"? Playouts can indicates splicing point, so no Tsplice is required.

@yanj-github I think having a playout dedicated for audio would solve this issue of having different fragment counts in video and audio, yes

Thanks @FritzHeiden that is great. We can have "audio_playout" for audio. (please kindly arrange the change carefully this will break current compatibility between OF and test runner.)

@FritzHeiden and @jpiesing Do you think Tsplice is required? I think we can get away without introducing new parameter.
Tsplice is the Audio and Video Switching Set co-incides with a Fragment Boundary. If there is correct content available only playouts should be enough.

e.g: I assumed we have right streams then at [1, 1, 3] the Audio and Video Switching Set co-incides with a Fragment Boundary.
"playout": [
[1, 1, 1], [1, 1, 2], [1, 1, 3],
[2, 1, 1], [2, 1, 2], [2, 1, 3],
[1, 1, 4], [1, 1, 5], [1, 1, 6]
]
"audio_playout": [
[1, 1, 1], [1, 1, 2], [1, 1, 3],
[2, 1, 1], [2, 1, 2], [2, 1, 3],
[1, 1, 4], [1, 1, 5]
]

Let me know if any reason this wont work please?

[FritzHeiden]

Thanks @FritzHeiden that is great. We can have "audio_playout" for audio. (please kindly arrange the change carefully this will break current compatibility between OF and test runner.)

I can put this change in a separate branch for you to test it with the OF

[yanj-github]

Thanks @FritzHeiden let me know the branch name once this has been implemented.

[jpiesing]

I've changed this test from Blocker to Beta as it clearly won't be validated in time for the 1st release. We should discuss removing it completely.

Editing the spec to remove tsplice and insert audio_playout is non-trivial editing. I don't object but it's more than 10 minutes.

[jpiesing]

Removing tsplice is more complex than it seems because of the following line;

Once all fragments are loaded up to time TSplice, set presentation time offset to 0.

If you have playout[] and audio_playout[], when is the presentation time offset set to 0? Is it when all video fragments from the first presentation are appended? Is it when all audio fragments from the first presentation are appended? Does the player have to stop appending at a change of track number in playout[]?

[yanj-github]

@jpiesing and @FritzHeiden I am not sure why we want to set presentation time offset to 0 at presentation 2 and set presentation time offset to tsplice? Can we not make it same as 8.8? The timeline starts from 0 and advancing without set back to 0.
I think the test runner implemented 9.4 same as 8.8 in terms of presentation time. @FritzHeiden and @louaybassbouss can you confirm please?

If we happy with it then I suggest to remove:
Once all fragments are loaded up to time TSplice, set presentation time offset to 0.
and
Once all fragments are loaded up to the end of the second presentation, set presentation time offset to TSplice.

If we wants to keep altering the presentation time, tsplice can be calculated from playouts instead of setting it.
tsplice is equal to k=playout[i=1],i=n] or audio_playout[k=playout[i=1],i=m]. Where an aligned media time playout[k=playout[i=1],i=n] audio_playout[k=playout[i=1],i=m]in the first Audio and Video Switching Set that coincide with a Fragment Boundary and is instructed for playback.

playout[i]: Provides the triple (Switching Set, CMAF track number, Fragment number) for every video playout position i=1,…,N that is to be played out according to the attached test content configuration.
audio_playout[i]: Provides the triple (Switching Set, CMAF track number, Fragment number) for every audio playout position i=1,…,N that is to be played out according to the attached test content configuration.

An aligned media time playout[k=playout[i=1],i=n] audio_playout[k=playout[i=1],i=m]in the first Audio and Video Switching Set that coincide with a Fragment Boundary and is instructed for playback.
Once all fragments are loaded up to time playout[k=playout[i=1],i=n], set presentation time offset to 0.
Once all fragments are loaded up to the end of the second presentation, set presentation time offset to playout[k=playout[i=1],i=n].

[FritzHeiden]

I opened a new PR containing the implementation of audio_playout: #183

The following tests not working:

* /ceac-local/splicing-of-wave-program-with-baseline-constraints__splice_main_splice_ad_at3_at4.html

* /ca4s-local/splicing-of-wave-program-with-baseline-constraints__splice_main_splice_ad_at3_at4.html
  The error that shows is:
  promise_test: Unhandled rejection with value: object "Error: Segment index in playout out of bounds: 6, but only got 5"

@yanj-github I was not able to reproduce this error, as my browser doesn't support ec-3 and the other tests work fine, so you have to test this please.

[yanj-github]

@FritzHeiden I will be able to test is on our device which supports dolby codecs.
Can it stay in branch for now please? I would like to run it on OF.
The new parameter "audio_playout" might cause OF change as well.

[yanj-github]

@FritzHeiden Just noticed current time reported is not correct can you have a look please?

I think audio_playout would required for both /ceac-local/splicing-of-wave-program-with-baseline-constraints__splice_main_splice_ad_at3_at4.html and /ca4s-local/splicing-of-wave-program-with-baseline-constraints__splice_main_splice_ad_at3_at4.html.