Requiring clients to only play content described by the steering server.

[wilaw]

[This issue discovered by CanalPlus and filed here based on their feedback]

In the current DASH ContentSteering Spec, the DASH client first chooses which adaptationSets it wishes to play and then looks for compatible BaseURLs as defined and ranked by the content steering server PATHWAY array. If none of the pathways match, then the spec says that the client

If no @serviceLocation attributes match the manifest, then the client may ignore the current PATHWAY-PRIORITY array and make a default decision about which BaseURL to use.

This makes it impossible to use DASH content steering to force a client to play only certain adaptationSets (for example SD versus HD content) by excluding the invalid serviceLocations form the PATHWAY-PRIORITY array.

In contrast, the HLS Content steering spec says

If a client is currently playing a Variant Stream that does not belong to the applied Pathway, it MUST switch to one that does.

The suggestion is to tighten the DASH spec to constrain the player in to selecting only content enabled by the steering server response. This would enable a new steering use case for content distributors and at the same time pull DASH in to closer alignment with HLS. The change required in the DASH spec are:

1. Replace the text from 7.13 with the following text

If the client is playing content defined by a BaseURL element with a @serviceLocation not equal to the preferred service location, then the client shall switch at the next segment load to retrieving future content from the BaseURL referenced by the preferred service location. Note that existing requests against a prior @serviceLocation should be allowed to complete and forward buffers should not be trimmed. If the preferred service location is not described in the DASH MPD, then the client should attempt to switch to the next highest priority service location. If no @serviceLocation attributes match the current AdaptionSet, then the client must switch to playing content from an alternate adaptionSet which does contain @servicelocations listed in the PATHWAY-PRIORITY array. These rules of only selecting content for playback which is described by @servicelocations listed in the PATHWAY-PRIORITY array also apply at start-up and initial content selection. At all times a DASH player compliant with Content Steering MUST only play content that is described by the steering server response. If no @serviceLocation attributes match any adaptionSet in the manifest, then the client should cease playback. It should still reload the RELOAD-URI after the specified TTL interval in case new service locations are added.

2. Replace the text from 7.14 with the following text

If the manifest contains one or more Location elements with @serviceLocation attributes, then prior to a manifest update, the client shall evaluate the @serviceLocation attributes against the PATHWAY- PRIORITY array and select the Location element, and if necessary the adaptionSet, with the highest matching @serviceLocation. If no @serviceLocation attributes match across the manifest then the client should cease playback. It should still reload the RELOAD-URI after the specified TTL interval in case new service locations are added.

[haudiobe]

IOP WG 2023/08/08

We identified that there are basically three/four different options, and all of three/four have a valid use case:

1. clientRequirement set to FALSE: free to ignore, but if following, you follow the rules
2. clientRequirement set to TRUE: you must follow the rules
3. a new flag set to TRUE: select resources only on permitted pathways, may result in restricted content, and content switching
4. to false: select resources from any pathway, if pathway is not included, you are permitted to stay on resources.

@dsilhavy believes that the third one is complex and not implemented.
@haudiobe believes that the third one is a valid use case, for example to address broadcast/unicast switching, unicast => broadcast
@yreznik third option is not a good idea.

Third option

• may mean that I am out of options on run-time because pathway is not supporting certain content.
• switching may not be seamless
• may mean that the the playback start is delayed because no information available

Invitation for more comments are welcome. We will not remove option 4, but we are open for positive feedback to support option 3 despite some concerns. Maybe also in a new revision.

[haudiobe]

Live TF 2023/08/11

• @wilaw believes that we should only pick option 3 from above. You minimize options.
• @yreznik believes that the use case is not a good idea initially - but may get convinced.

At this stage we believe we should go closer with HLS. However, we need to speak with @dsilhavy and add some guidance into the content steering offering. The new use cases should be documented in more details, but likely need to add this to DASH-IF guidelines.

[dsilhavy]

Sorry I couldn't make the call on Friday. Personally, I am in favor of adding a flag to enable option 3 or option 4. In my opinion, the implementation of option 3 can be tricky, some points I had in mind:

• Potential conflict with attribute queryBeforeStart. Setting queryBeforeStart=false and option 3) seems contradictory.
• Players might work with indices for AdaptationSets as the @id attribute is not mandatory. Adding and removing AdaptationSets might mess with the internal logic of the player. Common filters that are applied at the startup of the player typically remove an AdaptationSet permanently (e.g based on codec or DRM support).
• Multiperiod playback:
  • Assuming we have an MPD with main-ad-main. Now the content steering server needs to know the serviceLocation attribute for the ad period because BaseURL elements need to be explicitly enabled. Not sure if this is a problem though.
  • Same scenario main-ad-main: Assuming I am playing VoD content and got no valid BaseURL for the ad period. Do I jump over the ad period? Do I wait for the duration of the ad period to jump?
• Edit: Adding another point on how this would work with XLink? Is the steering server aware of what the Xlink resolver returns in terms of serviceLocation attributes?
• For live playback: Assuming there are no valid AdaptationSets for a certain amount of time. If the player recovers at some point, do we resume at the live edge? Or do we continue where we left? Relates to timeShiftBufferDepth, player might run out of the DVR window while waiting. Moreover, users will not leave the stream open for more than 2-5 seconds.
• Might mess with the logic for adaptation-set-switching-2016. For instance, Exoplayer creates a virtual AdaptationSet based on all compatible AdaptationSets. Dynamically removing a single AdaptationSet based on the steering response might cause issues.
• Might mess with the ABR logic of the player if specific BaseURLs for Representations are removed during playback.
• Depending on the AdaptationSet, platforms apply certain filters (codec, DRM level). Forcing a switch to a certain AdaptationSet will not always work. For instance, removing the serviceLocation for an SD AdaptationSet might cause some clients to fail in case HD or UHD is not supported on the platform (e.g. due to the DRM security level)
• How long to reload response from the steering server in case no valid options are available? Moreover, users will not wait for the stream to resume in such cases.
• Might delay startup time if we have to wait for content steering response before filtering and initializing all models. Probably not a big issue for JavaScript clients.

That being said, I think we need clear instructions on how a client should behave. And my feeling is that with this change we are moving away from a fairly easy to implement specification/addition to a pretty complex one.

[haudiobe]

Colleagues,

We had good discussion on the Content Steering issue here.

However, we have not yet come to a conclusion and we needed to respond to JTC Broadcast by tomorrow. I responded and asked for 2 weeks additional time.

My suggestion is that we use the call next week on August 29, 2023 to finally resolve the issue. If you have opinions and can not attend, please let me know ahead of time.

We basically have 3 options

1. Use the current specification text
2. Update the specification such that the client implementation is changed to match the use case from CanalPlus
3. Allow both options from the content provider, signaling the client behaviour with a spec.

At this stage, we have diverse opinions. Please discuss in issue.

Thomas

[acbegen]

Posting on behalf of PJ from Theo:

The HLS spec states it is a MUST to stop using any variant which is not in the acceptable pathways. However for an old HLS client, or one not supporting content steering, this will not hold true and it will simply load and play those variants. As a result, the new player requirement is a breaking change in the HLS specification. If one wants to use it, they must very clearly ensure their clients are fully compatible with this capability as most clients (old) will not be.

From that perspective, DASH clients will have a similar issue. Only the new clients which are supporting the capability to block any renditions which are not in a valid pathway will actually block them.

This means you can perfectly require this implementation for full content steering support, but as a user of that feature, one should be careful as they cannot assume that it will be honored by all clients. For example older clients on HbbTV, or a malicious client even which would want to load a higher quality which was blocked through eliminating it from the valid pathways.

In my opinion it is a useful feature to consider, but it is not something which should be used for blocking, only for suggesting. As such, a "should" can be sufficient. In practice, HLS has the same due to the old clients which will be around for ages (even old iOS versions remain active, we support back to iOS 9 ourselves as it is a relevant case for plenty of our customers).

[bbert]

As @dsilhavy said, this new feature could be quite complexed to implement, and it can lead to an important matrix of use cases while considering all inputs (pathway priorities, platforms codec support, DRM, application-level AdaptationSet/Representation filtering).
AFAIK there are other means to perform AdaptationSet/Representation selection (for example DRM, back-end manifest manipulation or patching).
Moreover this was not the initial purpose of content steering spec to address this feature but rather explicity to address delivery problems.

[haudiobe]

Hello Will,

I hope you are doing well.

At CANAL+, we are working on content steering, and we have a first implementation of DASH content steering implemented in our open source video player (RXPlayer : https://github.com/canalplus/rx-player )

We have a doubt in the understanding of an edge case.

Let’s imagine the following edge case (which can occur in some cases) :

• CDN-A : only SD and HD variants
• CDN-B : SD, HD and 4K variants
  The SERVICE-LOCATION-PRIORITY array indicates in priority CDN-A and then CDN-B.
  @defaultServiceLocation is CDN-A

On Apple side, according to the Apple’s explanation here, https://developer.apple.com/videos/play/wwdc2021/10141/ at 7:00 (initial pathway) and 9:00 (after getting the steering manifest), as there is at least one eligible variant in CDN-A, the player will stay in CDN-A and play only SD and HD variants (and never play 4K). And that’s also what I understand from the specification from Roger, for example here https://datatracker.ietf.org/doc/html/draft-pantos-hls-rfc8216bis#section-7.2 “If a client is currently playing a Variant Stream that does not belong to the applied Pathway, it MUST switch to one that does.”

We have a doubt in the DASH specification, if this is the same rule.

“If the client is playing content defined by a BaseURL element with a serviceDescription not equal to the
preferred service location, then the client shall immediately switch to retrieving future content from the
BaseURL referenced by the preferred service location. Note that existing requests against a prior
serviceLocation should be allowed to complete and forward buffers should not be trimmed. If the preferred
service location is not described in the DASH MPD, then the client should attempt to switch to the next highest
priority service location. If no serviceLocations match the manifest, then the client may ignore the current
SERVICE-LOCATION-PRIORITY array. It should still reload the RELOAD-URI after the specified TTL
interval in case new service locations are added. »

If “content” here is meaning segment, then the CDN selection is done after selecting the Period/AdaptationSet and Representation. Meaning that we will select the quality that we want, and select after the best CDN (according to the steering manifest) for getting this quality.
So it seems to be the opposite of the rule on Apple side, if we will understand. And Paul has search in the dash.js project (https://github.com/Dash-Industry-Forum/dash.js/commit/698b5e74a592bcef4dc813cae70697c00ce87701) and apparently they have done the way we understand the DASH specification (and the opposite of what Apple seems to do).

By the way, in dash.js, there is mentions of Cloning Pathway : maybe there is a new version of the specification that will be available ?

Thanks !
And happy to talk with you (some of us will be at IBC for example).

Thanks !
Pierre-Louis

[haudiobe]

Hi Pierre-Louis

Thanks for the CANAL+ implementation of content steering. It is good to see the spec(s) getting implemented in practice.

Yes, there have been updates to the steering specification which include cloning. The DASh Content Steering is being published as ETSI Spec TS 103 998. I have attached the last draft that was submitted. Sect 7.12 details cloning behavior and Appendix A3 gives an example.

You highlight a very interesting potential difference in the steering behavior between HLS and DASH. I agree that the HLS interpretation is that the client can only play content that is available under one of the available pathways. The DASH spec does not explicitly state this behavior. It does state that “If the client is playing content defined by a BaseURL element with a serviceDescription not equal to the preferred service location, then the client shall immediately switch to retrieving future content from the BaseURL referenced by the preferred service location.”. This language is not clear about whether the client should only select content that is available under the preferred pathway. I can say that when we wrote the DASH Content Steering behavior, we did not consider this case, and always assumed that multiple pathways would exist for any given adaptionSet.

All that we would need to align with the HLS content steering would be to add some normative language to Sect 7 to make it clear that clients should make content selection decisions within the constraint of the preferred pathway. We would also need to ask dash.js to change its behavior to match. I think we are too late for this with the current ETSI release, but we could queue it up for the next point release?

I have included @thomas for some oversight of this issue, as well as @zach for awareness from a HLS/DASH interop perspective and @daniel from a dash.js perspective.

I assume CANAL+ would like the DASH behavior to be consistent with HLS, so that you can use steering to force clients to select content from only certain AdaptationSets?

An IBC meeting sounds good. I can host in the Akamai meeting rooms if you like. What days are you and your team at the show and what time would suit you?

Cheers
Will

[haudiobe]

IOP WG 2023/08/29

• Discussion on the comments - we believe we should not add legacy client to the mix.
• Very different opinions. @haudiobe points to 3GPP TS 26.247
• @haudiobe provide MPD examples for offerings with different BaseURLs.
• We need some more time

[haudiobe]

Communication with Roger:

• https://mailarchive.ietf.org/arch/msg/hls-interest/1yo2hpfBOBGsSgRrMO5dU9xtRTA/
• https://mailarchive.ietf.org/arch/msg/hls-interest/qXd9LitBqornEooYFrQQovxqRlM/
• https://mailarchive.ietf.org/arch/msg/hls-interest/CDdwjXKUHK1-cgFMNMplqPJTQGk/

[haudiobe]

I updated 14 as follows

14. If the client is playing Representations defined by a BaseURL element with a @serviceLocation not equal to the preferred service location, then the client shall switch at the next segment load to retrieving future Representations from the BaseURL referenced by the preferred service location. In case the same Representation is present on the preferred service location, the client should continue accessing the same Representation. In case the same Representation is not present on the preferred service location, the DASH client shall switch to a Representation that is presented on the preferred service location, preferably selecting a Representation for which the client can seamlessly switch to. If one or multiple different Representations in the same Adaptation Set is present, the DASH client shall switch to a new Representation in the new Adaptation Set. However, if a new Adaptation Set needs to be selected, and the client identifies that a seamless switch cannot be achieved, while it may be achieved by chosing an Service Locatoin included in the priority list with lower priority list, the DASH client may choose to select a different service location as long as it is included in the prioritized list. Note that existing requests against a prior @serviceLocation should be allowed to complete and forward buffers should not be trimmed. If the preferred service location is not described in the DASH MPD, then the client should attempt to switch to the next highest priority service location. If no @serviceLocation attributes match the manifest, then the client may ignore the current PATHWAY-PRIORITY array and make a default decision about which BaseURL to use. It should still reload the RELOAD-URI after the specified TTL interval in case new service locations are added.

Comments?

[haudiobe]

For more details, please also look at the submitted doc here: https://members.dashif.org/wg/Interoperability/document/4890

It would be great to get some comments.

[wilaw]

@thomas. – I made some small grammar changes in red. 1. If the client is playing Representations defined by a BaseURL element with a @serviceLocation not equal to the preferred service location, then the client shall switch at the next segment load to retrieving future Representations from the BaseURL referenced by the preferred service location. In the case that the same Representation is present on the preferred service location, the client should continue accessing the same Representation. In the case that the same Representation is not present on the preferred service location, the DASH client shall switch to a Representation that is presented on the preferred service location, preferably selecting a Representation for which the client can seamlessly switch to. If one or multiple different Representations in the same Adaptation Set are present, the DASH client shall switch to a new Representation in the new Adaptation Set. However, if a new Adaptation Set needs to be selected, and the client identifies that a seamless switch cannot be achieved, while it may be achieved by choosing a Service Location that is lower in the priority list, the DASH client may choose to select a different Service Location as long as it is included in the prioritized list. Note that existing requests against a prior @serviceLocation should be allowed to complete and forward buffers should not be trimmed. If the preferred service location is not described in the DASH MPD, then the client should attempt to switch to the next highest priority service location. If no @serviceLocation attributes match the manifest, then the client may ignore the current PATHWAY-PRIORITY array and make a default decision about which BaseURL to use. It should still reload the RELOAD-URI after the specified TTL interval in case new service locations are added. 2. 3. I am also concerned with the last sentence “If no @serviceLocation attributes match the manifest, then the client may ignore the current PATHWAY-PRIORITY array and make a default decision about which BaseURL to use. It should still reload the RELOAD-URI after the specified TTL interval in case new service locations are added.”. Shouldn’t this be removed completely? This is the clause that allows the DASH player to ignore the content steering instructions. If we want to align with HLS, then we need to tell the DASH client that it can only play items from Service Locations described in the PATHWAY-PRIORITY array and if none of these match then it should cease playback.

…

-Will From: Thomas Stockhammer ***@***.***> Reply-To: Dash-Industry-Forum/Content-Steering ***@***.***> Date: Sunday, October 15, 2023 at 11:58 AM To: Dash-Industry-Forum/Content-Steering ***@***.***> Cc: "Law, Will" ***@***.***>, Mention ***@***.***> Subject: Re: [Dash-Industry-Forum/Content-Steering] Requiring clients to only play content described by the steering server. (Issue #26) I updated 14 as follows 1. If the client is playing Representations defined by a BaseURL element with a @serviceLocation not equal to the preferred service location, then the client shall switch at the next segment load to retrieving future Representations from the BaseURL referenced by the preferred service location. In case the same Representation is present on the preferred service location, the client should continue accessing the same Representation. In case the same Representation is not present on the preferred service location, the DASH client shall switch to a Representation that is presented on the preferred service location, preferably selecting a Representation for which the client can seamlessly switch to. If one or multiple different Representations in the same Adaptation Set is present, the DASH client shall switch to a new Representation in the new Adaptation Set. However, if a new Adaptation Set needs to be selected, and the client identifies that a seamless switch cannot be achieved, while it may be achieved by chosing an Service Locatoin included in the priority list with lower priority list, the DASH client may choose to select a different service location as long as it is included in the prioritized list. Note that existing requests against a prior @serviceLocation should be allowed to complete and forward buffers should not be trimmed. If the preferred service location is not described in the DASH MPD, then the client should attempt to switch to the next highest priority service location. If no @serviceLocation attributes match the manifest, then the client may ignore the current PATHWAY-PRIORITY array and make a default decision about which BaseURL to use. It should still reload the RELOAD-URI after the specified TTL interval in case new service locations are added. Comments? — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/Dash-Industry-Forum/Content-Steering/issues/26*issuecomment-1763340215__;Iw!!GjvTz_vk!TPiCh8GIc1ZsavIzmZJJd5ZadmQhMY00FA4XHriMiNXRfMANSE2aRGL05bb9PgMpXYjxiS3POQNdbxLvSFNuuA$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AAVCMCU3PMPNHEKV6L7IRW3X7OXU7ANCNFSM6AAAAAA3IT7DRQ__;!!GjvTz_vk!TPiCh8GIc1ZsavIzmZJJd5ZadmQhMY00FA4XHriMiNXRfMANSE2aRGL05bb9PgMpXYjxiS3POQNdbxIW71IKvQ$>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[haudiobe]

IOP WG 2023/10/24

This seems to progress well. The last is interesting. Does it mean in HLS that you never are permitted to access a content for which the service location is not included in the manifest from the Content Steering server?

We also update the text to make it better readable. @haudiobe will update the text and provide a new revision.

[haudiobe]

Updates available here: https://members.dashif.org/wg/Interoperability/document/4890?downloadRevision=

[haudiobe]

On November 16, there was a response on HLS Interest list:

_If the player encounters failures loading Playlists from all Variant Streams of the current Pathway, using its current media selection, then it will penalize the current Pathway and switch to the next-highest priority Pathway in the list. After doing so It would periodically try to switch back to the penalized highest-priority pathway,.

I don’t think this helps you with your use case, though. At the end of the day if your user wants to listen in German, you’ve got to find a way for them to get to the German audio. If German audio is part of the broadcast then you’re good. If not then you need to allow them to load unicast audio, in which case you might as well add the German unicast audio to the broadcast Pathway._

The above basically says that pathway selection is always on top of media selection and you can even only switch down the priority if the "offered" content in the manifest fails to be available on the pathway. It also kind of says that you always only select one pathway. I consider that in DASH we have more flexibility and just giving up all of this would be strange.

Assuming that HLS requires that content is always available on all pathways, but allows that different quality versions of switchable content are possibly only available on a subset of the pathways, then the client behaves as above.

All of this is very difficult to address.

[haudiobe]

Live TF 2023/11/10

Is this example possible? Does DASH allow this?

• http://localhost/ serviceLocation="A"
• http://unicast/audio/german serviceLocation="A"
• http://unicast/ serviceLocation="B"

We could update the client behaviour.

[haudiobe]

Having rechecked the DASH specification, there is no reason to not apply the same serviceLocation value to different base URLs.

This will not be the most common case, but it can help to solve cases for which you want to avoid that specific content is lost when you are forced to switch to a BaseURL. So the proposal is to be silent about same or different content. But we should write down how you can offer content for cases for which you want to support a more flexible offering.

[haudiobe]

After long discussion, we attempted to align to the largest extent with HLS and a new document is submitted here: https://members.dashif.org/wg/Interoperability/mail/thread/2468

https://members.dashif.org/wg/Interoperability/document/4810

[haudiobe]

We believe we addressed this in our published spec here https://www.etsi.org/deliver/etsi_ts/103900_103999/103998/01.01.01_60/ts_103998v010101p.pdf

If there are remaining open issues we encourage to submit a new issue.