When the audio stream duration doesn't match the video stream duration, playback will freeze at the shortest stream duration AND the 'ended' event is not triggered causing loop to fail.

[72lions]

Hi,

on the master branch v1.2.0 and on the development branch, when the video reaches the end and it stops playing, the 'ended' event from the video tag is never triggered.

I believe the reason is the fact that the video never reaches the end.

Example:
By using the Segment List mpd (http://www.digitalprimates.net/dash/streams/gpac/mp4-main-multi-mpd-AV-NBS.mpd) when the video reaches the end and stops it never stops in the end.

I am providing a screenshot:

[rikbauwens]

I guess this is because of the calculation error (issue #217)

[leolucas]

Maybe this will help troubleshoot the problem. When video appears to be stuck, you can see there are an endless number of "timeupdate" events. The current time of each "timeupdate" event is the same even though the video state is playing. During this time, video chunks continue to download so there is no problem with internet connectivity delivering future video chunks. I have added code in the "timeupdate" handler to detect repeated calls and reload the page to re-initialize video playback. After a page reload, the same video that was previously stuck will play past the stuck point and will generate an "ended" event.

[leolucas]

Sometimes when the video stops playing, the total time of the video is incorrect. For example, you might see the current time at 2:11 while the video duration is then incorrect (example: 0:52).

[leolucas]

Can anyone speculate when a 1.3 release will be available? The problem with video stopping in mid-playback is keeping us from using this software. Thanks!

[wilaw]

@leo - we have been trying to stick to a 2-3 month release cycle. This is based on contribution loads and is not fixed. The 1.3 release will basically be the first pass at the refactoring, which from the results I am seeing is a big improvement across the board. My goal would be to get it out before the end of the year, perhaps some time in November? This will depend on Kirk/Mikhail and some of the other key contributors. More hands make lighter work!

Have you tried the refactor branch? Does it solve the video-stoping-mid-playback problem?Playing the video in question on this player (which is based on the refactored branch I believe) result sin good termination:

http://dashplayer.azurewebsites.net/?URL=http://www.digitalprimates.net/dash/streams/gpac/mp4-main-multi-mpd-AV-NBS.mpd

If so, please vote up the move to /dev branch that Kirk just distributed as that will speed up the release date of 1.3

Cheers

Will

On Sep 30, 2014, at 12:30 PM, Leo Lucas [email protected] wrote:

Can anyone speculate when a 1.3 release will be available? The problem with video stopping in mid-playback is keeping us from using this software. Thanks!

—
Reply to this email directly or view it on GitHub.

[leolucas]

@wilaw - thanks for the release information! I have tried the code refactor branch. The refactor branch always fails to trigger the ended event. The release branch randomly fails to trigger the ended event. Maybe the refactor branch results will help you isolate the problem. In the refactor branch, the "playback does not continue" problem always happens near the end of the video. When you look at the "timeupdate" event you either see an endless repeat of the same currentTime of the video OR you see the currentTime count past the end of the video. Here are sample pages showing the normal and refactor branch code. Look at the F12 debug console to watch the currentTime for the video.

Release branch - http://www.e-learningconsulting.com/clients/dash/page.htm
Refactor branch - http://www.e-learningconsulting.com/clients/dash/page-refactor.htm

I can't test the http://dashplayer.azurewebsites.net/?URL=http://www.digitalprimates.net/dash/streams/gpac/mp4-main-multi-mpd-AV-NBS.mpd video because I get a CORS error.

Please let me know if I can help in any way to either isolate or resolve this bug.

Thanks, Leo

[KozhinM]

Update: of course endOfStream is called after the last segment has been appended to the buffer, not all of them.
I have some progress here. I have found that:
• After the player has loaded and appended to the buffer all the segments it calls mediaSource.endOfStream(). This method sets mediaSource.readyState to 'ended' state.
• Usually endOfStream is called several seconds before currentTime reaches the end of the video (because the player buffers ahead).
• During this time something causes mediaSource.readyState to switch back to 'open' state. According to the MSE spec one of the things that may lead to this is appending a segment to source buffer. So I think that sometimes earlier segments are appended to the buffer after the segment with the last index and cause the switch.
• Finally currentTime reaches the end of the video, but 'ended' event is not fired. But if I manually call endOfStream() again, 'ended' event fires immediately and video element switches to 'initial' state (for example in IE it stops firing 'timeupdate' event).
I have not found in the spec that readyState='ended' is a necessary condition for video element to fire 'ended', but it seems this is how it works.
I will investigate why mediaSource switches to 'open' and how to handle this case properly.

[KozhinM]

Fixed at f323386

[leolucas]

Thanks for working on this bug! I downloaded the dash.all.js from the development branch (https://github.com/Dash-Industry-Forum/dash.js/blob/development/dash.all.js) and created a sample page. The video now plays from start to finish (in 5 tests I conducted). But the "ended" event does not get fired. In my sample, I added code to write the currentTime to the console when the timeupdate event is triggered. You can see that the ended is not triggered in most cases. And currentTime will frequently go past the end of the video duration. Please see my sample code at http://www.e-learningconsulting.com/clients/dash/page-230.htm

[KozhinM]

It looks like dash.all.js that you used in your sample was not updated after the fix. I have just updated the min files, so it should work now. Could you please try it it again using the updated dash.all.js and let me know if everything is okay.

[leolucas]

I downloaded the updated dash.all.js and it fixes the problem with the ended event and the video plays from beginning to end. I updated the code here - http://www.e-learningconsulting.com/clients/dash/page-230.htm. Thanks for the super-quick response!

[leolucas]

We did some more testing using the new dash.all.js. If you move the scrubber bar you can get JavaScript errors OR the video gets stuck at 49 seconds. An easy repro case is let the video play to the end and then replay the video from the beginning and move the scrubber bar. Pressing the "Load Video" button will also generate different JavaScript errors. I think the cause of both errors is the need to reset properties when video playback is complete.

[kirkshoop]

Please open a new issue for this

Kirk

Developer
Microsoft Open Technologies, Inc.

[dsparacio]

I am reopening this issue as the solution implemented fails with certain streams. Looks like we are not fully playing out the stream. When this happens this also impeds the loop property on a video element from working as well. Looks like singleComplete is happening too early in certain cases.

[wilaw]

Also fails with

http://dash.edgesuite.net/envivio/dashpr/clear/Manifest.mpd

At the end, it tries to request a segment that does not exist.

FragmentLoader.js:168 GET http://dash.edgesuite.net/envivio/dashpr/clear/video5/65.m4s 404 (Not Found)

[KozhinM]

Here is what I have found - in the following stream
redbull/underwater/underwater.mpd

@mediaPresentationDuration=161.39s and @segmentDuration = 6s

Video representation has 27 segments where the last one has startTime = 156 and duration = 5.28. So the total duration is 161.28 which is 0.11s shorter than declared in the mpd.

At the same time audio has 28 segments where the last one has startTime = 161.28 and duration = 0.11. The total duration perfectly equals @mediaPresentationDuration

When SameTimeRequestRule tries to find a match video request for the last audio request (the one with startTime= 161.28), it fails because there is no video request for which startTime <= 161.28 < endTime

As a result the last audio segment is never loaded => isBufferingCompleted is never true => at the end of the playback hasSufficientBuffer turns false and the playback stalls.

Some time ago I came across a similar issue:
Dash-Industry-Forum/Test-Vectors#18

The main question here is what is an acceptable deviation of actual duration from the duration declared in the mpd. Still have no answer for this.

Regarding
http://dash.edgesuite.net/envivio/dashpr/clear/Manifest.mpd
This is a known issue, video must have 66 segments (indexing from zero), but the segment with index 65 does not exist indeed. The player makes several attempts to load it and then stops trying.

[wilaw]

I think in this case we need to implement some practical guidelines.

Options

1. We could ignore final segments smaller than N seconds in duration. If we set N = 0.02*segmentDuration, the stream would end. This is somewhat arbitrary however.
2. We alter the logic of SameTimeRequest rule so that it knows that if an audio or video adaptation set is complete, then its peer may still have additional segment(s) to retrieve.

Consider the case of a video adaption set that has 10s segments and an audio set that has 2s segments. For a 60s clip, there would be 6 video segments to retrieve and 30 audio segments. After the last video segment is requested at the 50s mark, there would still need to be 4 more audio segments requested.

My preference would be for #2 as that way we do feed all the available data to the source buffer.

To answer your question “ what is an acceptable deviation of actual duration from the duration declared in the mpd.”, the DASH-AVC/264 guidelines allow any segment to deviate by 50% from the declared segmentDuration. Additionally, no restraints on placed on the duration of the last segment of any representation. Here is the exact wording:

Segments have almost equal duration. The maximum tolerance of segment duration is ±50% and the maximum accumulated deviation over multiple segments is ±50% of the signaled segment duration (i.e. the @duration attribute or the S@d in the SegmentTimeline). Such fluctuations in actual segment duration may be caused by for example ad replacement or specific IDR frame placement. Note that the last segment in a representation may be shorter according to ISO/IEC 23009-1 [1].

So it does not declare that there must always be an audio segment for every video segment, or vice versa and if we have any logic that assumes this then we should remove it from the player.

Cheers

Will

On Jan 29, 2015, at 4:29 AM, KozhinM [email protected] wrote:

Here is what I have found - in the following stream
https://mwcssledash-a.akamaihd.net/akamai/redbull/underwater/underwater.mpd

@mediaPresentationDuration=161.39s and @segmentDuration = 6s

Video representation has 27 segments where the last one has startTime = 156 and duration = 5.28. So the total duration is 161.28 which is 0.11s shorter than declared in the mpd.

At the same time audio has 28 segments where the last one has startTime = 161.28 and duration = 0.11. The total duration perfectly equals @mediaPresentationDuration

When SameTimeRequestRule tries to find a match video request for the last audio request (the one with startTime= 161.28), it fails because there is no video request for which startTime <= 161.28 < endTime

As a result the last audio segment is never loaded => isBufferingCompleted is never true => at the end of the playback hasSufficientBuffer turns false and the playback stalls.

Some time ago I came across a similar issue:
Dash-Industry-Forum/Test-Vectors#18

The main question here is what is an acceptable deviation of actual duration from the duration declared in the mpd. Still have no answer for this.

Regarding
http://dash.edgesuite.net/envivio/dashpr/clear/Manifest.mpd
This is a known issue, video must have 66 segments (indexing from zero), but the segment with index 65 does not exist indeed. The player makes several attempts to load it and then stops trying.

—
Reply to this email directly or view it on GitHub.

[KozhinM]

Consider the case of a video adaption set that has 10s segments and an audio set that has 2s segments. For a 60s clip, there would be 6 video segments to retrieve and 30 audio segments. After the last video segment is requested at the 50s mark, there would still need to be 4 more audio segments requested.

I believe that dash.js handles such cases correctly, but the issue that we are discussing here is a bit different.
The problem is not in the number of segments that is different for audio and video. The thing is that a total duration of video stream is shorter than a total duration of audio stream and than a duration of the whole mpd.
In your example above imagine that a clip duration is 61s instead of 60, and audio has 30*2s segments + the last segment 1s. At the same time the video still has only 6*10s segments. At some point SameTimeRequestRule picks the last audio segment with startTime = 60. It searches for a peer, but the last video segment does not fit because the interval of the segment has an excluding endpoint: [50,60).
Dash.js includes startTime, but excludes endTime from the duration interval. It makes sense because there either must be one more video segment with startTime = 60 and 1s duration, or the last video segment must have 11s duration. Both cases would work because 60 would belong either to [60, 61) or [50, 61). It would work even if the video had a single segment [0, 61). So the number of segments is not important, the time they cover is important.
Dash.js assumes that there must be a a segment for any time in the interval [Period@start, Period@start + @mediaPresentationDuration).

To answer your question “ what is an acceptable deviation of actual duration from the duration declared in the mpd.”

Turned out my question was not clear. I am aware of a tolerance of segment duration, I am not sure about duration of the whole representation. I mean what is the delta = @mediaPresentationDuration - sum(seg1.duration + [seg2.duration, …, lastSeg.duration]).

[wilaw]

@KozhinM - ok, I understand now. Let me pose this question to the author of the DASH spec and the DASH IF interop group and get an answer for you.

Cheers

Will

On Jan 29, 2015, at 11:48 PM, KozhinM <[email protected]mailto:[email protected]> wrote:

Turned out my question was not clear. I am aware of a tolerance of segment duration, I am not sure about duration of the whole representation. I mean what is the delta = @mediaPresentationDuration - sum(seg1.duration + [seg2.duration, …, lastSeg.duration]).

[KozhinM]

@wilaw, Thanks! I will be waiting for a response.

[dsparacio]

This was also reported in #97 and #185. I would like to keep this ticket open as it has the most details and is the most current. I will close down #97 and reference this ticket in that issue.

[KozhinM]

@wilaw, @AkamaiDASH, I made a fix for this. It is here:
https://github.com/MSOpenTech/dash.js/commit/bc5e8fb2f584546acdf0c89c7559e0cccf720ce0
Would you mind confirming that it fixes the issue?

[wilaw]

@KozhinM - I'll let Dan test that fix. I did promise earlier in the thread to get a ruling on this from the DASh IF. I wanted to relay some info that I got back. I put there following diagram together to explain the various cases which can arise when duration gaps that may exist for a static manifest between video, audio and the advertised duration (shown by the red lines).

Of the cases below ALL are valid EXCEPT cases #7 and #8. This is somewhat of a surprise to me and I wondered if the current dash.js code base can cope with these? DASH-AVC/264 does not define a threshold for what the size of these gaps may be. The primary take-away is that it is not true that there is always a video segment and an audio segment to describe any time point between t-0 and t=@mediaPresentationDuration. I was wondering if the logic in the SameTimeRequestRule needs to be altered to match that fact?

Cheers

Will

[cid:5E623250-84B4-4793-8BB3-327CA1152A53]
[cid:B11032F6-46C8-4FFB-8B24-92BFA10F5904]

From: KozhinM <[email protected]mailto:[email protected]>
Reply-To: "Dash-Industry-Forum/dash.js" <[email protected]mailto:[email protected]>
Date: Thursday, February 26, 2015 at 5:36 AM
To: "Dash-Industry-Forum/dash.js" <[email protected]mailto:[email protected]>
Cc: Will Law <[email protected]mailto:[email protected]>
Subject: Re: [dash.js] When the audio stream duration doesn't match the video stream duration, playback will freeze at the shortest stream duration AND the 'ended' event is not triggered causing loop to fail. (#230)

@wilawhttps://github.com/wilaw, @AkamaiDASHhttps://github.com/AkamaiDASH, I made a fix for this. It is here:
MSOpenTech@bc5e8fbhttps://github.com/MSOpenTech/dash.js/commit/bc5e8fb2f584546acdf0c89c7559e0cccf720ce0
Would you mind confirming that it fixes the issue?

Reply to this email directly or view it on GitHubhttps://github.com//issues/230#issuecomment-76178551.

[KozhinM]

@wilaw, the diagram you mentioned is not displayed. Could you please re-post it here or send it directly to my email? Once I get it I will be able to say if dash.js satisfies all the valid cases in in the diagram.
For now I can say that dash.js assumes that for any time t in the interval [Period@start; (Period@start + Period@duration)] there are both audio and video segments. In my latest fix dash.js accepts a segmentDuration/2 threshold to recognize that two segments belongs to the same time, e.g. if currentTime = 4.8s, segmentDuration = 2s, videoSeg.startTime = 5s, audioSeg.startTime = 4.7s. In this case SameTimeRequestRule would decide that audio and video segments are peers and both match currentTime even though video.startTime > currentTime. This threshold is based on the fact that the maximum tolerance of segment duration is ±50%.

[dsparacio]

Works for my test stream and I get the ended event every time now. Works on the reporters stream
http://www.digitalprimates.net/dash/streams/gpac/mp4-main-multi-mpd-AV-NBS.mpd

@KozhinM ill have you close once you commit to project. Thanks for the fix!!

[athrowawayaccount]

I just want to note that this bug is back on 2.0.0 release.

[mbrock]

In case anyone ends up in this thread wondering why their video doesn't loop with the latest released Dash.js (2.2.0), I switched to the "nightly" build of 2.3.0 and it works fine.