@selectionPriority feature support according to DASH IOP v4.3

[XingzhaoYun]

Hi,

As per DASH IOP v4.3 and DASH spec, @selectionPriority is an OD value to express the preference of MPD author for selecting AdaptationSet. The AdaptationSet with higher @selectionPriority value is preferred, otherwise, with the absence of this value, will choose random AdaptationSet.

Will it possible that Shaka is able to support adding @selectionPriority feature so that it allows MPD authors to have more preference choices?

Here I came up with one solution,

To add @selectionPriority attribute for AdaptationSet, we could make use of stream_descriptor for each input, during the mpd building process, AdaptationSet will set @selectionPriority value as the largest value among all the Representations of it, here is the pseudo command line and the consequent manifest it will generate,

./packager.exe \
in=Hevc_Res720p.mp4,stream=video,out=shaka_hevc_res720p.mp4,priority=100
in=Hevc_Res540p.mp4,stream=video,out=shaka_hevc_res540p.mp4,priority=60
in=Hevc_Res360p.mp4,stream=video,out=shaka_hevc_res360p.mp4,priority=90
in=AVC_Res720p.mp4,stream=video,out=shaka_avc_res720p.mp4,priority=80
in=AVC_Res540p.mp4,stream=video,out=shaka_avc_res540p.mp4,priority=50
in=AVC_Res360.mp4,stream=video,out=shaka_avc_res360p.mp4,priority=70
in=DDP_6ch.mp4,stream=audio,out=shaka_ddp_6ch.mp4,priority=10
in=AAC_stereo.mp4,stream=audio,out=shaka_aac_stereo.mp4,priority=9
--mpd_output pseudo_output.mpd

It will have the following pseudo Period,

<Period id="0">
    <!--Audio-ddp-->
    <AdaptationSet id="0" contentType="audio" lang="en" subsegmentAlignment="true" selectionPriority="10">
      <Representation id="0" bandwidth="128289" codecs="ec-3" mimeType="audio/mp4" audioSamplingRate="48000">
        <AudioChannelConfiguration schemeIdUri="urn:mpeg:mpegB:cicp:ChannelConfiguration" value="6"/>
        <BaseURL>shaka_ddp_6ch.mp4</BaseURL>
        <SegmentBase indexRange="735-946" timescale="48000">
          <Initialization range="0-734"/>
        </SegmentBase>
      </Representation>
    </AdaptationSet>
    <!--Audio-aac-->
    <AdaptationSet id="5" contentType="audio" lang="en" subsegmentAlignment="true" selectionPriority="9">
      <Representation id="12" bandwidth="63641" codecs="mp4a.40.2" mimeType="audio/mp4" audioSamplingRate="48000">
        <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
        <BaseURL>shaka_aac_stereo.mp4</BaseURL>
        <SegmentBase indexRange="761-972" timescale="48000">
          <Initialization range="0-760"/>
        </SegmentBase>
      </Representation>
    </AdaptationSet>
    <!--Video-avc-->
    <AdaptationSet id="4" contentType="video" maxWidth="1280" maxHeight="720" frameRate="25000/1000" subsegmentAlignment="true" par="16:9" selectionPriority="80">
      <Representation id="8" bandwidth="570932" codecs="avc1.4d4028" mimeType="video/mp4" sar="1:1" width="640" height="360">
        <BaseURL>shaka_avc_res360p.mp4</BaseURL>
        <SegmentBase indexRange="878-1089" timescale="25000">
          <Initialization range="0-877"/>
        </SegmentBase>
      </Representation>
      <Representation id="9" bandwidth="2301764" codecs="avc1.4d4028" mimeType="video/mp4" sar="1:1" width="960" height="540">
        <BaseURL>shaka_avc_res540p.mp4</BaseURL>
        <SegmentBase indexRange="879-1090" timescale="25000">
          <Initialization range="0-878"/>
        </SegmentBase>
      </Representation>
      <Representation id="11" bandwidth="3357460" codecs="avc1.4d4028" mimeType="video/mp4" sar="1:1" width="1280" height="720">
        <BaseURL>shaka_avc_res720p.mp4</BaseURL>
        <SegmentBase indexRange="880-1091" timescale="25000">
          <Initialization range="0-879"/>
        </SegmentBase>
      </Representation>
    </AdaptationSet>
    <!--Video-hevc-->
    <AdaptationSet id="1" contentType="video" maxWidth="1280" maxHeight="720" frameRate="25000/1000" subsegmentAlignment="true" selectionPriority="100">
      <Representation id="1" bandwidth="167703" codecs="hvc1.2.4.L120.B0" mimeType="video/mp4" sar="1:1" width="640" height="360">
        <BaseURL>shaka_hevc_res360p.mp4</BaseURL>
        <SegmentBase indexRange="958-1169" timescale="25000">
          <Initialization range="0-957"/>
        </SegmentBase>
      </Representation>
      <Representation id="5" bandwidth="1133747" codecs="hvc1.2.4.L120.B0" mimeType="video/mp4" sar="1:1" width="960" height="544">
        <BaseURL>shaka_hevc_res540p.mp4</BaseURL>
        <SegmentBase indexRange="959-1170" timescale="25000">
          <Initialization range="0-958"/>
        </SegmentBase>
      </Representation>
      <Representation id="14" bandwidth="3344411" codecs="hvc1.2.4.L120.B0" mimeType="video/mp4" sar="1:1" width="1280" height="720">
        <BaseURL>shaka_hevc_res720p.mp4</BaseURL>
        <SegmentBase indexRange="959-1170" timescale="25000">
          <Initialization range="0-958"/>
        </SegmentBase>
      </Representation>
    </AdaptationSet>
</Period>

The AdaptationSets set the value of @selectionPriority values as 10, 9 100 and 80, as they are the largest values among their respective AdaptationSet.

[XingzhaoYun]

This feature is only for DASH so far for sure.

[joeyparrish]

Sounds reasonable to me. Interested to send a PR for this?

[XingzhaoYun]

Shaka packager is not expecting to expose AdaptationSet level options to users (at least we didn't find any), but this feature is based on AdaptationSet. Our current proposal is to add priority value to each input/Representation, does this make any sense or might have potential risks?

[XingzhaoYun]

We have implemented this feature as the description above, just want to know, does it need any improvement or any better solution might exist.

[vish91]

Why expose it at Representation ? Is that within the DASH-IF IOP spec ?
At representation it would be difficult to maintain the correct priority across adaptations consider the case of multi key DRM implementations where we will have to check based on codecs or cross adaptations.
Does any players out there support it ?

[XingzhaoYun]

The reason we attach this priority to representation is that we couldn't find any parameter for AdaptationSet, correct me if I'm wrong. DASH-IF IOP spec only mentions this attribute for AdaptationSet, yet we won't be able to know the subordinate structure of the Representation and the AdaptationSet beforehand. As for the multi key DRM case, could you share an instance or example manifest?

[sr1990]

Hi @XingzhaoYun, why are the selection priority values for representations belonging to one adaptation set not the same?

With your approach,
if there is one AVC adaptation set that contains 5 representations with selection priority of 1,1,2,2,1000 mentioned in the packager command and
another HEVC adaptation set that also contains 5 representations with selection priority of 999,999,998,998,999,
result mpd will contain avc with selection priority of 1000 and hevc with selection priority of 999.

Player will select the avc adaptation based on the selection priority even when there are more representations in the hevc adaptation set that have higher selection priority values than avc (based on the packager command).
Fixed selection priority values for representations belonging to the same adaptation set in the packager command makes more sense. Example:

./packager.exe \
in=Hevc_Res720p.mp4,stream=video,out=shaka_hevc_res720p.mp4,priority=100
in=Hevc_Res540p.mp4,stream=video,out=shaka_hevc_res540p.mp4,priority=100
in=Hevc_Res360p.mp4,stream=video,out=shaka_hevc_res360p.mp4,priority=100
in=AVC_Res720p.mp4,stream=video,out=shaka_avc_res720p.mp4,priority=80
in=AVC_Res540p.mp4,stream=video,out=shaka_avc_res540p.mp4,priority=80
in=AVC_Res360.mp4,stream=video,out=shaka_avc_res360p.mp4,priority=80
in=DDP_6ch.mp4,stream=audio,out=shaka_ddp_6ch.mp4,priority=10
in=AAC_stereo.mp4,stream=audio,out=shaka_aac_stereo.mp4,priority=9
--mpd_output pseudo_output.mpd

OR we can mention selection priority along with one representation in the packager command and use it while creating the adaptation set. Example:

./packager.exe \
in=Hevc_Res720p.mp4,stream=video,out=shaka_hevc_res720p.mp4,priority=100
in=Hevc_Res540p.mp4,stream=video,out=shaka_hevc_res540p.mp4
in=Hevc_Res360p.mp4,stream=video,out=shaka_hevc_res360p.mp4
in=AVC_Res720p.mp4,stream=video,out=shaka_avc_res720p.mp4,priority=80
in=AVC_Res540p.mp4,stream=video,out=shaka_avc_res540p.mp4
in=AVC_Res360.mp4,stream=video,out=shaka_avc_res360p.mp4
in=DDP_6ch.mp4,stream=audio,out=shaka_ddp_6ch.mp4,priority=10
in=AAC_stereo.mp4,stream=audio,out=shaka_aac_stereo.mp4,priority=9
--mpd_output pseudo_output.mpd

What do you think?

[cosmin]

Putting an adaptation set level attribute at the representation level doesn't quite feel right, at the same time I agree that we don't really have a better solution at the moment. It might be time to consider the ability to define groups, assign representations to groups and then set group level properties, but might need some more thinking.

Can we rely on the command line ordering to define the adaptation set priority instead?