Support for signaling HDR / HDR10+ in DASH with SupplementalProperty

[vish91]

System info

Operating System: all
Shaka Packager Version: master

Issue and steps to reproduce the problem

Does Shaka packager support being able to pass additional information to the input while running packaging command ?
For a multi codec DASH manifest , there is optional supplemental property for HDR .

<SupplementalProperty schemeIdUri="http://dashif.org/metadata/hdr" value="SMPTE2094-40"></SupplementalProperty>

DASH - IF Spec 10.3.2.4 HDR metadata.

What is the expected result?
Optional parameter to pass key / value for the Supplemental Property to set.
That property being set in resultant DASH manifest in the appropriate adaptation set.

[FiberOnePL]

Very useful option, I'm in favor of adding hdr info

[sr1990]

+1

Looks like HLS implementation supports specifying transfer characteristics.

shaka-packager/packager/hls/base/master_playlist.cc

Line 244 in d5ca6e8

const std::string video_range = playlist.GetVideoRange();

and

shaka-packager/packager/hls/base/media_playlist.cc

Line 554 in d5ca6e8

std::string MediaPlaylist::GetVideoRange() const {

will add VIDEO-RANGE in m3u8.
Example:
#EXT-X-STREAM-INF:BANDWIDTH=820260,AVERAGE-BANDWIDTH=785273,CODECS="dvh1.05.01",RESOLUTION=640x360,FRAME-RATE=59.940,VIDEO-RANGE=PQ,CLOSED-CAPTIONS=NONE

Similarly, for DASH, one way would be to add a supplemental property to specify transfer characteristics as per DASH-IF IOP v4.3
6.2.5.1. the CICP for transfer characteristics can be indicated in an essential or supplementary property at the AdaptationSet level (given that representations within a set shall not differ for that property), using schemeIdUri="urn:mpeg:mpegB:cicp:TransferCharacteristics"

Example:
<SupplementalProperty schemeIdUri="urn:mpeg:mpegB:cicp:TransferCharacteristics" value="16" />

This supplemental property is also supported by shaka player as per
shaka-project/shaka-player#3726.
VideoConfiguration is created with the transfer function that is mentioned in the supplemental property and is used as a parameter for decodingInfo API.

Implementation could be:

1. Add transfer characteristics as AdaptationSet (adaptation_set.h) data member.
2. While creating the adaptation set
   

   shaka-packager/packager/mpd/base/period.cc

   Line 76 in d5ca6e8

   AdaptationSet* Period::GetOrCreateAdaptationSet(

   
   a. Add transfer characteristics in adaptation set key
   

   shaka-packager/packager/mpd/base/mpd_utils.cc

   Line 146 in d5ca6e8

   std::string GetAdaptationSetKey(const MediaInfo& media_info,

   
   b. Initialize AdaptationSet transfer characteristics data member in

   shaka-packager/packager/mpd/base/period.cc

   Line 200 in d5ca6e8

   bool Period::SetNewAdaptationSetAttributes(

3. While creating mpd
   

   shaka-packager/packager/mpd/base/adaptation_set.cc

   Line 242 in d5ca6e8

   base::Optional<xml::XmlNode> AdaptationSet::GetXml() {

   
   add the supplemental property if transfer characteristics is present.

@vish91 , @joeyparrish / @kqyang , please take a look.

[SteveR-PMP]

@vish91 @joeyparrish @kqyang @sr1990

I came up with some code near the end of RepresentationXmlNode::AddVideoInfo() that will add the SupplementalProperty for transfer characterstics:

  if (video_info.has_transfer_characteristics()) {
    std::string transfer_characteristics =
      base::UintToString(video_info.transfer_characteristics());
    RCHECK(AddDescriptor("SupplementalProperty",
                         "urn:mpeg:mpegB:cicp:TransferCharacteristics",
                         transfer_characteristics));
  }

This follows how it is done in HLS and how SupplementalProperty is done for audio in RepresentationXmlNode::AddAudioChannelInfo()
https://github.com/shaka-project/shaka-packager/blob/d5ca6e84e6ccd543dc9a79ccc8d07b702010f491/packager/mpd/base/xml/xml_node.cc#L556

This should fulfill the request without needing to create a new data member in AdaptationSet. Does anyone see a problem with this solution?

[vish91]

Yeah transfer characteristics work too... infact in the latest DASH-ID IOP I cannot find this HDR metadata signaling, but the signaling using transfer characteristics is still there in the spec.
https://dashif.org/guidelines/iop-v5/

@sr1990 @joeyparrish any thoughts ? what do you do on shaka-player or ExoPlayer side if you know from player perspective what is better.
Also do we know the possible values for Transfer Characteristics and what they are from spec ? its in ISO-IEC spec but haven't gotten my hands into that yet.

[sr1990]

@vish91, check out the shaka player issue shaka-project/shaka-player#3726 and PR that it mentions related to parsing the transfer characteristics.
Don't see exoplayer parsing out VIDEO-RANGE in
https://github.com/google/ExoPlayer/blob/release-v2/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/HlsPlaylistParser.java

For values of transfer characteristics, check out
https://www.itu.int/rec/recommendation.asp?lang=en&parent=T-REC-H.265-202108-I
Table E.4 – Transfer characteristics interpretation using the transfer_characteristics syntax element
Transfer characteristics are parsed out from SPS VUI - check H265Parser::ParseVuiParameters and H264Parser::ParseVUIParameters
Also,

shaka-packager/packager/hls/base/media_playlist.cc

Line 554 in d5ca6e8

std::string MediaPlaylist::GetVideoRange() const {

will probably need to be updated for case 18 which should be HLG

[cosmin]

@vish91 can we consider this closed now that transfer characteristics are in? Or do we still need this specific supplemental property?

[vish91]

yes sure thing. 👍🏽 closed by #1210