Audio change causes bitrate change

[yairans]

Have you read the FAQ and checked for duplicate issues: Yes

What version of Shaka Player are you using: v2.1.3

Can you reproduce the issue with our latest release version: Yes

Are you using the demo app or your own custom app: Yes

What browser and OS are you using: Chrome 58/ macOS 10.12

What are the manifest and license server URIs: Angel One stream in the demo

What did you do? Load, switch to the lowest bitrate and switch to 'de' audio language

What did you expect to happen?
The video should play with the lowest bitrate.

What actually happened?
The video switched to the highest bitrate.

[joeyparrish]

I can reproduce this easily, but we need to investigate more to decide how we could solve this.

[ismena]

@yairans Do I understand correctly that by "switch to the lowest bitrate" you mean "manually select a variant that has low bitrate"?
I don't think we currently have an assumption that when a new variant is selected due to a language change it should take current variant's bandwidth as a guideline.
This is something we can introduce, but it isn't an obvious conclusion.

If it's something we'd like to have, I can think of two ways of approaching it:

1. Implicit: when new variant is chosen due to a language change, treat current variant's bandwidth as a guideline.
2. Explicit: introduce a way to explicitly set bandwidth guidance (we have bandwidthUpgradeTarget and bandwidthDowngradeTarget configs, but may be we need something more explicit).
   @joeyparrish let me know what you think!

[joeyparrish]

Explicit sounds like too much effort for the app developer. If we can do it at all, I believe in this case it should be implicit.

If the app selects a specific variant (low res, for example) and disables ABR (as the demo does when you select a variant), then it is a surprise to the app developer (and therefore end user) when the res changes to something higher on language change. Even though the user has enough bandwidth, this change in res is essentially an automated one, which is surprising when ABR is turned off. We should, as much as we can, avoid surprising devs & end-users.

This could be a little tricky to fix, since the system behaves very generically today. If there's a language change, AbrManager is given new things to choose from and makes a choice. It all follows from simple rules.

To fix it, we may have to give AbrManager some guidelines or restrictions, or perhaps even cut AbrManager out of the loop entirely on language changes if ABR is disabled. This means more complex rules for this action. Hopefully, we could avoid surprises while also avoiding undue complexity. We'll have to see what we can do.

[yairans]

BTW, I'm not sure is relevant here, but when I switch any track (video, audio and text) I get the 'adaptation' event, unlike the documentation 'AdaptationEvent
Fired when an automatic adaptation causes the active tracks to change. Does not fire when the application calls selectVariantTrack() selectTextTrack(), selectAudioLanguage() or selectTextLanguage().'
Additionally, when I enable/disable the adaption I also get the 'adaptation' event.
All of these can reproduce easily in shaka demo page with the Angel One stream.

[TheModMaker]

There is a similar issue for multi-Period content. If someone manually selects a track, when it switches to the next Period, AbrManager will select the tracks for the new Period. This can result in bandwidth changes.

[joeyparrish]

This is complicated.

With ABR disabled, we could try to maintain the previous resolution or bitrate whenever we are forced to select new streams. This would apply both to language changes as in @yairans's scenario, and to period changes as described by @TheModMaker. But this will not always produce good results.

Assume the following perfectly reasonable-looking periods, in which not all resolutions are available across periods:

period 1 - TV show: 1080p (selected by user), 720p, 480p
period 2 - ad break: 480p only
period 3 - TV show: 1080p, 720p, 480p

When switching to period 2, we can only choose 480p. There are no other streams. When switching to period 3 with ABR disabled, we would stick with 480p in period 3, even though we might have been forced into 1080p by the user during period 1.

This solution could be as surprising to the user as the bug, but at least now we will not ever choose a resolution for which we don't have enough bandwidth. Imagine the inverse of the above periods:

period 1 - TV show: 1080p, 720p, 480p (selected by user)
period 2 - ad break: 720p only
period 3 - TV show: 1080p, 720p, 480p

Here we would wind up at 720p in period 3, even though the user forced the player into 480p in the first period. In period 3, we may end up buffering a lot and never switching down to 480p. This is clearly worse than choosing new streams at the period transition based on current bandwidth.

I feel as though this applies to language transitions as well as period transitions, but I'm not certain.

[joeyparrish]

In the multi-period scenarios, fundamentally, there seems to be a conflict between what you intend to do (choose a particular resolution) and what the API says (choose a particular track). Tracks do not extend across periods, and we don't have a "choose resolution" method.

For languages, it may be easier to resolve. The best-effort "maintain bitrate" goal I suggested above may be good enough if applied solely to selectLanguage(). I will experiment with this.

[joeyparrish]

If the user has selected a particular resolution, it may be a reasonable work-around for applications to listen for TracksChangedEvent and then call selectVariantTrack() with the best match to the user's selection in the app's UI. This would work for both language and period transitions.

It would still be best for the Player to handle this, but it's complicated by the lack of a clear intent to stick to a given resolution. Again, because selectVariantTrack() does not mean "select resolution", and tracks don't extend to other periods or languages.