Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DASH: Prioritize selectionPriority attribute over a "main" Role when ordering AdaptationSets #1082

Merged
merged 1 commit into from
Mar 16, 2022

Conversation

peaBerberian
Copy link
Collaborator

A new platform (from italia) using the RxPlayer recently noticed us that the default audio track chosen on some DASH contents was wrong according to them.

For example, on some American film with two audio tracks, one in english (the original language) and the other in italian (which is dubbed) the RxPlayer would select the english one if no preference was given.
For the application developers, it was an error as the italian audio AdaptationSet had both a selectionPriority higher than the english one and was higher in the MPD.

However, it turned out that the english AdaptationSet had a <Role> element set to "main" - indicating that it was the "main" audio track - and the itialian one also had a <Role> element but set to "dub" this time - indicating that this track is dubbed.

And that's where the DASH-IF IOP are kind of ambiguous.
In the chapter 3.9.5.2. Generic Processing Model, it indicates that to make the initial audio track selection, a player has first to:

The DASH client looks for main content, i.e. any Adaptation Set with annotation
Role@schemeIdUri="urn:mpeg:dash:role:2011" and
Role@value="alternative" is excluded initially for selection.

What's difficult to interpret there is that Role@value="alternative" doesn't seem to exist in the same spec, letting me to believe that it may be referencing an older version of the spec with only two possible Roles: "main" and "alternative" (there's now much more than that).
That and the beginning of the sentence, led me originally to believe that a DASH player is supposed to select a "main" Role initially over any other Role.

However, due to the ambiguity, I looked at what other players were doing:

  • The shaka-player seemed to also put the "main" AdaptationSet first but I'm under the impression that they don't use the selectionPriority attribute at all.
  • dash.js' code is a little more complicated to follow - it seemed that they order both by selectionPriority or by Role depending on the context - but use selectionPriority when testing on their demo page.

In the end, I decided to put selectionPriority over a "main" Role for initial track selection, as it seems to make the more sense (priority ordering AdaptationSet, regardless of if that AdaptationSet is considered the main one), as well as being the more flexible (without that, impossible to prefer a non-main AdaptationSet server-side), even if it could mean "breaking" other usages which had the same comprehension of the specification than me.


This commit implement a solution to order by:

  1. AdaptationSet@selectionPriority ascending, which is 1 by default.
  2. <Role> ("main" over all other AdaptationSet for the same SelectionPriority)
  3. Order in the MPD.

I tried several solutions, but I ended up with the simpler one - which may not be the most efficient in terms of performance - but is by far the most readable:
When parsing AdaptationSets, ordering-related metadata (the selectionPriority attribute etc.) is added alongside it in a tuple parsed object - ordering metadata.
After all AdaptationSets are parsed, they are sorted according to that metadata by type (audio, video, text...) and then the metadata is removed.

@peaBerberian peaBerberian added the DASH Relative to the DASH streaming protocol label Mar 15, 2022
@peaBerberian peaBerberian added this to the 3.27.0 milestone Mar 15, 2022
@peaBerberian peaBerberian added the skip-performance-checks Performance tests are not run on this PR label Mar 15, 2022
A new platform (from italia) using the RxPlayer recently noticed us that
the default audio track chosen on some DASH contents was wrong according
to them.

For example, on some American film with two audio tracks, one in english
(the original language) and the other in italian (which is dubbed) the
RxPlayer would select the english one if no preference was given.
For the application developers, it was an error as the italian
audio `AdaptationSet` had both a `selectionPriority` higher than the
english one and was higher in the MPD.

However, it turned out that the english AdaptationSet had a `<Role>`
element set to "main" - indicating that it was the "main" audio track -
and the itialian one also had a `<Role>` element but set to "dub" this
time - indicating that this track is dubbed.

And that's where the DASH-IF IOP are kind of ambiguous.
In the chapter `3.9.5.2. Generic Processing Model`, it indicates that to
make the initial audio track selection, a player has first to:
```
The DASH client looks for main content, i.e. any Adaptation Set with annotation
Role@schemeIdUri="urn:mpeg:dash:role:2011" and
Role@value="alternative" is excluded initially for selection.
```
What's difficult to interpret there is that `Role@value="alternative"`
doesn't seem to exist in the same spec, letting me to believe that it
may be referencing an older version of the spec with only two possible
Roles: "main" and "alternative" (there's now much more than that).
That and the beginning of the sentence, led me originally to believe
that a DASH player is supposed to select a "main" Role initially over
any other Role.

However, due to the ambiguity, I looked at what other players were
doing:
  - The shaka-player seemed to also put the "main" AdaptationSet first
    but I'm under the impression that they don't use the
    `selectionPriority` attribute at all.
  - dash.js code is a little more complicated to follow - and it seemed
    that they order both by selectionPriority or by Role depending on
    the context - but use `selectionPriority` when testing on their demo
    page.

In the end, I decided to put `selectionPriority` over a "main" Role for
initial track selection, as it seems to make the more sense (priority
ordering AdaptationSet, regardless of if that AdaptationSet is
considered the main one), as well as being the more flexible (without
that, impossible to prefer a non-main AdaptationSet server-side), even
if it could mean "breaking" other usages which had the same
comprehension of the specification than me.

---

This commit implement a solution to order by:
  1. `AdaptationSet@selectionPriority` ascending, which is `1` by
     default.
  2. `<Role>` ("main" over all other AdaptationSet for the same
     `SelectionPriority`)
  3. Order in the MPD.

I tried several solutions, but I ended up with the simpler one - which
may not be the most efficient in terms of performance - but is by far
the most readable:

When parsing AdaptationSets, ordering-related metadata (the
selectionPriority attribute etc.) is added alongside it in a tuple
parsed object - ordering metadata.
After all AdaptationSets are parsed, they are sorted according to that
metadata by type (audio, video, text...) and then the metadata is
removed.
@peaBerberian peaBerberian force-pushed the misc/reorder-adaptation-sets2 branch from dc8ba9e to b53a1dc Compare March 15, 2022 09:27
@peaBerberian peaBerberian added skip-performance-checks Performance tests are not run on this PR and removed skip-performance-checks Performance tests are not run on this PR labels Mar 15, 2022
@peaBerberian peaBerberian merged commit da25f01 into master Mar 16, 2022
@peaBerberian peaBerberian mentioned this pull request Mar 17, 2022
@peaBerberian peaBerberian deleted the misc/reorder-adaptation-sets2 branch June 3, 2022 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DASH Relative to the DASH streaming protocol skip-performance-checks Performance tests are not run on this PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants