1.3 building far too much buffer at lowest quality when starting

[wilaw]

This manifest has 5 birtates (500,1000,2000,3000,5000 kbps): http://dash.edgesuite.net/akamai/redbull/underwater/underwater.mpd which are marked via overlays so you can see when it switches.

When I play it on the 1.3 player, from a connection with 100Mbps connectivity, it starts at the very lowest quality (500kbps) and plays that for a full 24s (!) before showing improved quality to the user. We see this reflected in the GET requests.

Underwater2_500k_track1_dashinit.mp4 // loading init
Underwater2_1000k_track1_dashinit.mp4 // loading init
Underwater2_2000k_track1_dashinit.mp4 // loading init
Underwater2_3000k_track1_dashinit.mp4 // loading init
Underwater2_5000k_track1_dashinit.mp4 // loading init
Underwater2_500k_track1_dashinit.mp4 // loading segment #1 at lowest quality
Underwater2_500k_track1_dashinit.mp4 // loading segment #2 at lowest quality
Underwater2_500k_track1_dashinit.mp4 // loading segment #3 at lowest quality
Underwater2_500k_track1_dashinit.mp4 // loading segment #4 at lowest quality
Underwater2_500k_track1_dashinit.mp4 // loading segment #5 at lowest quality
Underwater2_500k_track1_dashinit.mp4 // loading segment #6 at lowest quality
Underwater2_5000k_track1_dashinit.mp4 // only now it switches up to 5Mbps stream after 24s of rendered time and on the 7th segment.
Underwater2_5000k_track1_dashinit.mp4
Underwater2_5000k_track1_dashinit.mp4
Underwater2_5000k_track1_dashinit.mp4

Improved behavior:

1. Even if loads the first segment at 500kbps, it can time that download and use it to make an intelligent decision on the second segment to load , which should be the 5Mbps segment.
2. Even better, when loading both the manifest and the init segments, the player has opportunity to time these downloads and therefore make the very first segment it loads the 5Mbps segment. This is the preferred and requested behavior.

Cheers

Will

[jeroenwijering]

Since loading the init segments by themselves delays loading, I suggest we always pick an initial quality to load. By default, this can be the lowest one, but it should be an option. We talked about that in SF; our consensus was to setup an initial_bitrate. For example, If the publisher sets an initial_bitrate of 1000kbps, the stream starts up with the 1000k one. If the publisher sets an initial_bitrate of 2100kbps, the stream starts up with the 2000k one.

Note we don't want to load the highest quality segment immediately. Startup speed is more important than initial HD quality (assuming the player then quickly converges to HD).

We should also not load all the init segments for all qualities before loading actual video data. First load the startup init segment, then the startup first video segment, then additional inits/segments based upon available bandwidth. If the init segment for a certain quality is not available yet, load it just-in-time.

[wilaw]

Using initial_quality is fine. At Akamai, we use a default value of 1Mbps for HDCore, meaning that in the absence of all other info, the player will start with the largest bitrate <= this value, and then switch up/down from there for second and subsequent fragments. This figure used to be 500kbps but we raise it every year to match the rise in average bandwidth available. (BTW, average in the US is now 10.5Mbps across all connections)

Here is my suggested starting logic

1. If initial_quality has been set externally, use it otherwise assume an initial_quality_video value of 1000kbps and initial_quality_audio of 100kbps.
2. Load the manifest. We could time the download however I don’t think this will give an accurate enough reading since 1) the manifest is tiny and 2) it may be on a different server than the media segments.
3. Select the video and audio representations whose bitrate <= initial_quality value.
4. In parallel, load the init (and sidx if ondemand) segments for these audio and video representations as well as the first audio and video segment
5. Wait to make a decision on the second fragment based upon the throughput estimate achieved by loading the two init segments and the first audio and video segments
6. Switch representations if necessary for the second and subsequent segments.

One note on always loading the lowest quality segment at the start in order to speed up startup. Assume we have a 10Mbps connection (many are more than that, my home is currently 75Mbps) and we can start up at 500Kbps , 2000kbps or 5Mbps. Assume a 2s segment. The 500kbps segment will load in 100ms, the 2000kbps in 400ms and the 5Mbps segment will load in 1000ms. To a human watching , the startup delta between the 500kbps and the 2000kbps will be 300ms, which is difficult to discern, versus the difference in quality between the 500kbps and the 2000kbps streams, which would be easily discerned. If we repeat this analysis assuming a 20Mbps connection, then the delay drops to only 150ms. So my point is that loading the lowest quality representation at start as a rule is generally a bad idea and that a good trade off between start-up time and appropriate quality can be achieved by picking a mid-point bitrate that is neither the lowest nor the highest. A did a quick spreadsheet to look at start up choices for given representation bitrates, segment durations and last mile throughputs. Assuming that we must wait for the entire segment to be downloaded before we can render it (we may want to challenge that assumption) and that the slowest startup we would accept is 1s, then the areas in green give us allowable startup options. We could incorporate this a decision algorithm. The player will know the segment duration and bitrate and will have some estimate of the throughput. With this it could calculate what is the highest bitrate it could start at given a maximum allowable startup time.

Cheers

Will

On Feb 5, 2015, at 9:21 AM, Jeroen Wijering [email protected] wrote:

Since loading the init segments by themselves delays loading, I suggest we always pick an initial quality to load. By default, this can be the lowest one, but it should be an option. We talked about that in SF; our consensus was to setup an initial_bitrate. For example, If the publisher sets an initial_bitrate of 1000kbps, the stream starts up with the 1000k one. If the publisher sets an initial_bitrate of 2100kbps, the stream starts up with the 2000k one.

Note we don't want to load the highest quality segment immediately. Startup speed is more important than initial HD quality (assuming the player then quickly converges to HD).

We should also not load all the init segments for all qualities before loading actual video data. First load the startup init segment, then the startup first video segment, then additional inits/segments based upon available bandwidth. If the init segment for a certain quality is not available yet, load it just-in-time.

—
Reply to this email directly or view it on GitHub.

[jeroenwijering]

Completely agreed; great solution.

*) The default startup bitrates of 1000/100 are based upon real data (from Akamai) and publishers can override them with an option to fit their demographic.
*) The additional steps now focus on (a) getting the video playing ASAP and (b) swiftly ramping to the optimal quality.

[KozhinM]

@jeroenwijering, @wilaw, I have made some changes that implement what was discussed above. They are here:
#412

Could you please take a look and share your thoughts about it.

BTW, I am aware that @AkamaiDASH has also been working on that task, so I am going to wait for his opinion as well before merging this.

P.S. Actually I did merge the PR accidently (#410) , then I reverted the merge, so the commit name in #412 PR looks very awkward. Sorry about that, my fault.

[colde]

Should it perhaps be documented somewhere that these defaults are based on 2 second segments? Or should it take segment length into consideration? If somebody uses 10 second segments or (gasp) even longer ones, then the sensible defaults might be different - if they are still targeting quick startup time.

[wilaw]

The concepts proposed are independent of any segment duration. In the spreadsheet I made, it examined segment durations from 2s up to 10s, so the intent is definitely not to set any sort of threshold that assumes a certain segment length.

[colde]

@wilaw Yeah, i agree the concepts are independent of the segment length. I was just saying it in reference to your calculations of loadtimes in ms. The loadtime over a given connection of a 2 second segment @1000kbit is different from a 10 second segment @1000kbit. (Since it is either loading a 2000kbit segment vs a 10000 kbit segment)

Therefore, for longer segments, in order to facilitate a quicker startup it might be desirable to user lower bitrate segments.

[KozhinM]

How about to take into account both segment duration and a download time tolerance when calculating the initial bitrate. For example, we have a default "average" network bitrate nBitrate= 1000kbps. Then we could add a default segment download time tolerance downloadTolerance, e.g. 0.5s - max download time of a single segment with duration 'sDuration' that we can accept . Based on that, we could calculate the initial bitrate:

initBitrate = (downloadTolerance * nBitrate) / sDuration

For sDuration = 2s
initBitrate = 250kbps
For sDuraton = 10s
initBitrate = 50kbps

[wilaw]

Do we need to wait to receive the last byte of the segment before we can start feeding it to the source buffer and for the MediaSource to start rendering it? With HDS and OSMF, we would bite off chunks ( ~5kB) and process those as they came in. This allowed startup time to be (relatively) independent of segment duration. On xhr.onprogress, can you copy out the contents of the arraybuffer and feed them to the sourceBuffer?

===== below comments only apply if we can't do the above =============

If don't think in 2015 we should be designing a playback engine that intentionally starts at 50kbps for wellc-onnected users, even if theoretically this affords the lowest start-up time. Because it does so at the extreme expense of quality. I'd prefer to the default behavior would be to simply use

initBitrate = nBitrate

and then provide a simple API on MediaPlayer to override this. For most implementations with 3-6s segments, and assuming throughput >= 3*nBitrate, this would give start-up times of 1s-2s. Note that in the US today, average throughput exceeded 10Mbps for the first time this year, which drops those startup times down to 300ms - 600ms. So using nBitrate = 1000 is in fact quite conservative already.

If a provider with 10s segments is willing to sacrifice quality for start-up time, they can set nBitrate to downloadTolerance/sDuration and therefore start in 0.5s with the 50kbps stream. They would also have the flexibility of choosing a different trade-off, by say choosing 2s startup time and 250kbps. I also think 10s segments are a carry-over from HLS defaults, where they were used to lower the file count for all the individual segments. With ondemand profile, the segment duration can instead be built around GOP intervals and encoder efficiency and for that most people are using 2-5s today. From inspection, the Azure default seems to be 2s segments?

The most important factor is that this startup logic is only used for the very first video and audio segment. For second and subsequent requests, switching based on timed-dwnload the first segments should kick in.

Cheers

Will

From: KozhinM <[email protected]mailto:[email protected]>
Reply-To: "Dash-Industry-Forum/dash.js" <[email protected]mailto:[email protected]>
Date: Tuesday, February 17, 2015 at 4:26 AM
To: "Dash-Industry-Forum/dash.js" <[email protected]mailto:[email protected]>
Cc: Will Law <[email protected]mailto:[email protected]>
Subject: Re: [dash.js] 1.3 building far too much buffer at lowest quality when starting (#402)

How about to take into account both segment duration and a download time tolerance when calculating the initial bitrate. For example, we have a default "average" network bitrate nBitrate= 1000kbps. Then we could add a default segment download time tolerance downloadTolerance, e.g. 0.5s - max download time of a single segment with duration 'sDuration' that we can accept . Based on that, we could calculate the initial bitrate:

initBitrate = (downloadTolerance * nBitrate) / sDuration

For sDuration = 2s
initBitrate = 250kbps
For sDuraton = 10s
initBitrate = 50kbps

Reply to this email directly or view it on GitHubhttps://github.com//issues/402#issuecomment-74660173.

[KozhinM]

On xhr.onprogress, can you copy out the contents of the arraybuffer and feed them to the sourceBuffer?

It seems we can't. I did not find any way to extract 'partial' response when xhr.readyState = 3 (processing request). All xhr response properties (response, responseText, responseBody etc.) either null or undefined. The only kind of a workaround is to replace xhr.responceType with 'application/octet-stream'. In chrome this allows to get responseText in onprogress handler. Not sure we would be able to convert in buffer array then, though. This approach does not work in IE.

[dsparacio]

@KozhinM I think the changes that allow for higher quality startup is good start to this issue. But we still grow a larger than needed buffer ~25 seconds at startup. This delays switch up to higher quality if available . Ideally we should keep a 10-15 second buffer at all times, avoiding 10 to 15 spikes above that buffer mark UNLESS we are at the highest quality, then we should build the largest we can. Would you have a moment to discuss some options with me. I can do the work.

[KozhinM]

@AkamaiDASH, certainly I will find some time to discuss this. How about after our bi-weekly call on Tuesday? (before the call we will probably be discussing abandonment logic :) )

[dsparacio]

no an issue anymore