Assigning audio group in multibitrate HLS

[vasudevavarun]

System info

Operating System: Centos 7.4(64bit)
Shaka Packager Version: 2.4.3

Issue and steps to reproduce the problem

Packager Command:

./packager-linux-2.4.3 'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_2000.mp4,stream=video,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/2000-$Number$.ts,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/2000.m3u8' 'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_700.mp4,stream=video,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/700-$Number$.ts,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/700.m3u8' 'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_250.mp4,stream=video,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/250-$Number$.ts,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/250.m3u8' 'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_2000.mp4,stream=audio,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/128-$Number$.aac,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/128.m3u8,hls_group_id=audio' 'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_250.mp4,stream=audio,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/32-$Number$.aac,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/32.m3u8,hls_group_id=audio' 'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_700.mp4,stream=audio,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/64-$Number$.aac,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/64.m3u8,hls_group_id=audio' 'in=/tmp/Subtitle\ 1.vtt,stream=text,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/$Number$.vtt,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/main.m3u8,hls_group_id=text,hls_name=ENGLISH' --hls_master_playlist_output /opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/a.m3u8

What is the expected result?

#EXTM3U

Generated with https://github.com/google/shaka-packager version v2.4.3-dd98700-release

#EXT-X-MEDIA:TYPE=AUDIO,URI="128.m3u8",GROUP-ID="audio",LANGUAGE="en",NAME="stream_1",AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,URI="64.m3u8",GROUP-ID="audio",LANGUAGE="en",NAME="stream_5",CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,URI="32.m3u8",GROUP-ID="audio",LANGUAGE="en",NAME="stream_3",CHANNELS="2"

#EXT-X-MEDIA:TYPE=SUBTITLES,URI="main.m3u8",GROUP-ID="text",NAME="ENGLISH",AUTOSELECT=YES

#EXT-X-STREAM-INF:BANDWIDTH=2264710,AVERAGE-BANDWIDTH=2189543,CODECS="avc1.4d4029,mp4a.40.2",RESOLUTION=768x576,AUDIO="audio_128",SUBTITLES="text"
2000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=893814,AVERAGE-BANDWIDTH=865367,CODECS="avc1.4d4029,mp4a.40.2",RESOLUTION=640x480,AUDIO="audio_64",SUBTITLES="text"
700.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=421558,AVERAGE-BANDWIDTH=408952,CODECS="avc1.42c01f,mp4a.40.2",RESOLUTION=384x288,AUDIO="audio_32",SUBTITLES="text"
250.m3u8

What happens instead?

#EXTM3U

Generated with https://github.com/google/shaka-packager version v2.4.3-dd98700-release

#EXT-X-MEDIA:TYPE=AUDIO,URI="128.m3u8",GROUP-ID="audio",LANGUAGE="en",NAME="stream_1",AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,URI="64.m3u8",GROUP-ID="audio",LANGUAGE="en",NAME="stream_5",CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,URI="32.m3u8",GROUP-ID="audio",LANGUAGE="en",NAME="stream_3",CHANNELS="2"

#EXT-X-MEDIA:TYPE=SUBTITLES,URI="main.m3u8",GROUP-ID="text",NAME="ENGLISH",AUTOSELECT=YES

#EXT-X-STREAM-INF:BANDWIDTH=2264710,AVERAGE-BANDWIDTH=2189543,CODECS="avc1.4d4029,mp4a.40.2",RESOLUTION=768x576,AUDIO="audio",SUBTITLES="text"
2000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=893814,AVERAGE-BANDWIDTH=865367,CODECS="avc1.4d4029,mp4a.40.2",RESOLUTION=640x480,AUDIO="audio",SUBTITLES="text"
700.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=421558,AVERAGE-BANDWIDTH=408952,CODECS="avc1.42c01f,mp4a.40.2",RESOLUTION=384x288,AUDIO="audio",SUBTITLES="text"
250.m3u8

I want to associate audio group with desired bitrate.
Thanks

[kqyang]

@vasudevavarun You can specify the audio group using hls_group_id stream descriptor: https://google.github.io/shaka-packager/html/tutorials/hls.html#hls-specific-stream-descriptor-fields, i.e.

./packager-linux-2.4.3  \
  'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_2000.mp4,stream=video,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/2000-$Number$.ts,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/2000.m3u8'  \
  'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_700.mp4,stream=video,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/700-$Number$.ts,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/700.m3u8'  \ 
  'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_250.mp4,stream=video,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/250-$Number$.ts,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/250.m3u8'  \ 
  'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_2000.mp4,stream=audio,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/128-$Number$.aac,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/128.m3u8,hls_group_id=audio_128'  \ 
  'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_250.mp4,stream=audio,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/32-$Number$.aac,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/32.m3u8,hls_group_id=audio_32'  \ 
  'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_700.mp4,stream=audio,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/64-$Number$.aac,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/64.m3u8,hls_group_id=audio_64'  \
  'in=/tmp/Subtitle\ 1.vtt,stream=text,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/$Number$.vtt,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/main.m3u8,hls_group_id=text,hls_name=ENGLISH'  \
  --hls_master_playlist_output /opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/a.m3u8

I am closing this issue. Feel free to reply if you have more questions.

[vasudevavarun]

Hi,

Command:

./packager-linux-2.4.3
'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_2000.mp4,stream=video,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/2000-$Number$.ts,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/2000.m3u8,hls_group_id=audio_128'
'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_700.mp4,stream=video,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/700-$Number$.ts,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/700.m3u8,hls_group_id=audio_64'
'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_250.mp4,stream=video,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/250-$Number$.ts,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/250.m3u8,hls_group_id=audio_32'
'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_2000.mp4,stream=audio,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/128-$Number$.aac,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/128.m3u8,hls_group_id=audio_128'
'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_250.mp4,stream=audio,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/32-$Number$.aac,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/32.m3u8,hls_group_id=audio_32'
'in=/opt/video/data/amritatv_ott615/bulk_video/dest/mp4/a_2_700.mp4,stream=audio,segment_template=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/64-$Number$.aac,playlist_name=/opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/64.m3u8,hls_group_id=audio_64'
--hls_master_playlist_output /opt/video/data/amritatv_ott615/bulk_video/dest/hls/a_2/a.m3u8

Expected Behavior:

#EXTM3U

Generated with https://github.com/google/shaka-packager version v2.4.3-dd98700-release

#EXT-X-MEDIA:TYPE=AUDIO,URI="128.m3u8",GROUP-ID="audio_128",LANGUAGE="en",NAME="stream_0",AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,URI="32.m3u8",GROUP-ID="audio_32",LANGUAGE="en",NAME="stream_2",AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,URI="64.m3u8",GROUP-ID="audio_64",LANGUAGE="en",NAME="stream_4",AUTOSELECT=YES,CHANNELS="2"

#EXT-X-STREAM-INF:BANDWIDTH=421558,AVERAGE-BANDWIDTH=408952,CODECS="avc1.42c01f,mp4a.40.2",RESOLUTION=384x288,AUDIO="audio_32"
250.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=893814,AVERAGE-BANDWIDTH=865367,CODECS="avc1.4d4029,mp4a.40.2",RESOLUTION=640x480,AUDIO="audio_64"
700.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2264710,AVERAGE-BANDWIDTH=2189543,CODECS="avc1.4d4029,mp4a.40.2",RESOLUTION=768x576,AUDIO="audio_128"
2000.m3u8

Result:

#EXTM3U

Generated with https://github.com/google/shaka-packager version v2.4.3-dd98700-release

#EXT-X-MEDIA:TYPE=AUDIO,URI="128.m3u8",GROUP-ID="audio_128",LANGUAGE="en",NAME="stream_0",AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,URI="32.m3u8",GROUP-ID="audio_32",LANGUAGE="en",NAME="stream_2",AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,URI="64.m3u8",GROUP-ID="audio_64",LANGUAGE="en",NAME="stream_4",AUTOSELECT=YES,CHANNELS="2"

#EXT-X-STREAM-INF:BANDWIDTH=421558,AVERAGE-BANDWIDTH=408952,CODECS="avc1.42c01f,mp4a.40.2",RESOLUTION=384x288,AUDIO="audio_128"
250.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=893814,AVERAGE-BANDWIDTH=865367,CODECS="avc1.4d4029,mp4a.40.2",RESOLUTION=640x480,AUDIO="audio_128"
700.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2264710,AVERAGE-BANDWIDTH=2189543,CODECS="avc1.4d4029,mp4a.40.2",RESOLUTION=768x576,AUDIO="audio_128"
2000.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=324107,AVERAGE-BANDWIDTH=312862,CODECS="avc1.42c01f,mp4a.40.2",RESOLUTION=384x288,AUDIO="audio_32"
250.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=796363,AVERAGE-BANDWIDTH=769277,CODECS="avc1.4d4029,mp4a.40.2",RESOLUTION=640x480,AUDIO="audio_32"
700.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2167259,AVERAGE-BANDWIDTH=2093453,CODECS="avc1.4d4029,mp4a.40.2",RESOLUTION=768x576,AUDIO="audio_32"
2000.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=356779,AVERAGE-BANDWIDTH=344903,CODECS="avc1.42c01f,mp4a.40.2",RESOLUTION=384x288,AUDIO="audio_64"
250.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=829035,AVERAGE-BANDWIDTH=801318,CODECS="avc1.4d4029,mp4a.40.2",RESOLUTION=640x480,AUDIO="audio_64"
700.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2199931,AVERAGE-BANDWIDTH=2125494,CODECS="avc1.4d4029,mp4a.40.2",RESOLUTION=768x576,AUDIO="audio_64"
2000.m3u8

The audio should be mapped once to the desired video bitrate.

[kqyang]

I see. Shaka packager does not support fixing the audio group for the variant streams right now.

It is possible to add support for it with some simple modifications. Let us know if you are interested in helping us implement it.

[vasudevavarun]

I see. Shaka packager does not support fixing the audio group for the variant streams right now.

It is possible to add support for it with some simple modifications. Let us know if you are interested in helping us implement it.

Sure, Please let me know how can I help.

[kqyang]

It is trivial if we just want to support fixing the audio group id for the variant stream. Here is a sample change to do that:

https://github.com/google/shaka-packager/blob/master/packager/hls/base/master_playlist.cc#L435

  for (const auto& variant : variants) {
    if (video_playlists.empty())
      break;
    content->append("\n");
    for (const auto& playlist : video_playlists) {
      if (!playlist.group_id().empty() && variant.audio_group_id && 
          playlist.group_id() != *variant.audio_group_id) {
        continue;
      }
      BuildStreamInfTag(*playlist, variant, base_url, content);
    }
  }

The above change supports having one audio group id per variant stream only. We may want to support more than one audio group ids per variant stream.

We should also support text group ids. Ideally there should be separate stream descriptors for audio and text, but that requires more changes.

One possible option is to reuse the same stream descriptor hls_group_id, but modify it to support both audio and text, and optionally more than one audios and texts, e.g. use ; to separate the groups and use + to separate audio and text, something like: hls_group_id=audio1;audio2+text1;text2. You may use base::SplitString to split the strings.

[shaka-bot]

@vasudevavarun Does this answer all your questions? Can we close the issue?