Generate multi-period output

[Angels-group]

It would be very nice to get the playlist function. In order to be able to transfer as an argument xml (or json) a file with a list of media files that will be converted to m3u8 or mpd playlist in turn (non-stop stream).
This is one of the most anticipated features.

[joeyparrish]

Let me see if I understand correctly. You want Shaka Streamer to stitch together multiple inputs into a continuous output?

[Angels-group]

Yes something like that.
I imagine it this way: there is a certain json file, to which there is an array of file names and configurations to it.

Just for example:

{
  "input_config": "input.yaml",
  "pipeline_config": "pipeline.yaml",
  "files": [
  {
    "source": "file1.mp4"
  },
  {
    "source": "file2.mp4"
  },
  {
    "source": "file3.mp4"
  }
  ]
}

At the output, we get an hls / dash playlist.
If we used the Live type, the files are encoded one by one. As when playing a playlist in the usual sense. Something like a TV broadcast with a schedule of movies / shows / music videos (and so on). If we choose the VOD type, we get an hls / dash playlist with a duration equal to the sum of the durations of all the files added to the json playlist (which I presented above)

[joeyparrish]

I think for us, this would be a new type of input in the input.yaml file. But I'm not sure how we would design it offhand. I'll rename the issue and leave this as a feature request in our backlog, but we'd be happy to discuss a design here and review your pull request if you want to tackle it yourself.

[joeyparrish]

Rough proposal for the input config:

inputs:
    # The type of input.
  - input_type: concat
    # The media type is required at this level only.
    media_type: video
    list:
      # These only need to have "name" attributes.
      -name: foo1.mp4
      -name: foo2.mp4
       is_interlaced: True  # If you have an interlaced source in the list

    # The type of input.
  - input_type: concat
    # The media type is required at this level only.
    media_type: audio
    language: "de"
    list:
      # These only need to have "name" attributes.
      -name: foo1.mp4
      -name: foo2.mp4
       track_num: 2  # If the 0th audio track is not the one you want here...

[joeyparrish]

There are several ways to concatenate things in ffmpeg, with various limitations.

1. Concat demuxer (same codecs, same time base, "etc" (ffmpeg doc is vague))
2. Concat protocol (same file format, only concatenateable formats like TS supported, analogous to "cat" command)
3. Concat filter (same resolution required, otherwise no restrictions)
4. External ffmpeg process (pre-encoding everything to match parameters first, then streaming it as one stream)

I haven't tested any of this, but I suspect it's possible to introduce AV desync after the first clip if the audio & video streams of the clips don't match in length. For example:

|-AUDIO 1---|-AUDIO 2---|-AUDIO 3---|
|-VIDEO 1-|-VIDEO 2-|-VIDEO 3-|

In this diagram, all audio streams are slightly longer than their video streams. The second clip will be slightly out of sync, and the third even more so.

This is my assumption, anyway, based on concatenating the input streams together. If this turns out to be the case, it will be up to the user to make sure their inputs have the same length in audio & video.

If Shaka Packager ever introduces multi-period DASH support (for something other than preconditioning for ads), that might be a better option. It would allow boundaries between the different inputs, where the alignment can be reset at each period boundary. But we can go ahead and build this based on concatenating with FFmpeg and switch to Packager-based multi-period at a later date.

[joeyparrish]

An example of concatenation with ffmpeg:

ffmpeg \
  -i BigBuckBunny.1080p.webm \
  -i Sintel.2010.720p.Small.mkv \
  -f mpegts \
  -filter_complex "[0:v:0]scale=1920x1080,setsar=1/1[iv0];[1:v:0]scale=1920x1080,setsar=1/1[iv1];[iv0][iv1]concat=n=2:v=1:a=0[outv]" \
  -map "[outv]" -y concat-out.ts

This is video-only, and the filter_complex argument is complex indeed. Each input has to be scaled to the same resolution, and both need their SAR set to 1/1 for the concat part to succeed. Other inputs may possibly need other filters inserted, but those are the two that came up in my experiment.

[mariocynicys]

@joeyparrish Also we may run the transcoding and the packaging for each input on its own. each input produces a DASH and HLS manifest then in some mechanism, we try to combine the multiple DASH/HLS files from multiple inputs into one DASH/HLS file which includes each individual input as a segment in it.
Would that be applicable?

[joeyparrish]

That could certainly work, and in fact, combining MPDs by hand is how this clip was produced several years back:

https://shaka-player-demo.appspot.com/demo/#asset=https://storage.googleapis.com/shaka-demo-assets/heliocentrism/heliocentrism.mpd;build=compiled

But I worry if that is the right approach. It requires an understanding of DASH & HLS structures in Python-land, whereas today that knowledge is entirely delegated to Shaka Packager.

On the other hand, editing text files might be relatively painless, and building the ffmpeg commands to concatenate files and adjust their resolutions to match might be awful in practice.

Does anyone else have an opinion on this?

[mariocynicys]

yeah, surely concatenation using ffmpeg is too complicated and restricted compared to concatenation in the .mpd file, also understanding DASH structure is way easier than understanding the ffmpeg's indeed complex -filter_complex. 😆
for that i would suggest going with the DASH concatenation option, even though it might be later supported by shaka-packager someday, but that at least would force the design to digest the inputs and process it the way shaka-packager would be fed when it starts supporting multi period.

[joeyparrish]

Ideally the solution would also apply to HLS, perhaps using discontinuities to stitch together content.

[mariocynicys]

Exactly, I have tried the #EXT-X-DISCONTINUITY and seems to work flawlessly.