Flatten periods and SegmentTemplate internally

[joeyparrish]

Today, some of our most complex and subtle code is around period transitions. Player and StreamingEngine both have to manage a bunch of corner cases around transition points. The Player may be showing the set of tracks from period 1, while StreamingEngine and AbrManager are dealing with a different set of tracks from period 2.

StreamingEngine has complex code to manage period transitions and synchronize them across stream types (audio, video, text). The only thing that really needs to change between periods is the timestampOffset, append window, and init segment.

If we internally flatten out multi-period DASH, we can have a single list of segments that cross period boundaries. Each segment reference could contain its own append window, timestamp offset, and init segment reference. If the init segment changes between segment N and N+1, StreamingEngine could just fetch and append that init segment before the corresponding media segment. If the append window or timestamp offset change from one segment to the next, StreamingEngine could just pass those parameters to MediaSourceEngine before fetching and appending the next segments. StreamingEngine then would need no period-related state.

A consequence of flattening periods is that we would also have a consistent set of tracks. No manual selection would get undone at a period boundary. There would be no need to ask AbrManager to make choices at period boundaries. If representations in different periods were ill-aligned, these new tracks would map out multiple paths through those representations based on what would be chosen at different bandwidth levels. This mapping would be the most difficult part of the system to get right, but it would be isolated to the DASH implementation.

After this transition, our internal manifest structure would look more like HLS than DASH, which would enable us to then implement features like HLS discontinuities.

Much of this was originally proposed by @vaage internally.

This will unblock #1335, #1308, and #892. It may simplify #999 and #1307. It would fix #856 and #1543.

[joeyparrish]

Note to self: make sure DASH manifest updates are as efficient as possible in the new flattened structure. For example, when updating a SegmentTimeline, don't parse the entire thing and merge indexes if we can start parsing after the last segment from the previous manifest update. This could help a lot on slow devices.

[joeyparrish]

As part of selecting which AdaptationSet/Representation to map to tracks, we should look at Period-connectivity. See DASH 5.3.2.4 and DASH Timing Guidelines section 12.

Having looked more closely at what period-connectivity offers, it may be more complex to use it at this point. It does not indicate the correct transitions from one period to the next on all cases. Rather, it indicates some correct transitions between some periods, but not necessarily covering all streams, and not necessarily for contiguous periods. For example, the diagrams in those guidelines show period 1 connected to period 3, and only for a subset of streams.

So using this information for period flattening would be complex.

Further, it seems that the purpose of period-connectivity information is to indicate that the init segments for certain streams are equivalent for decoder initialization. While this would be a potential optimization, in that we could sometimes avoid fetching an init segment when we change periods, it seems to me that it would be a minor one. And a general period flattening algorithm would naturally connect streams with the same container, codec, and resolution, which are the things that must be in common to reuse the init segment and therefore to have period-connectivity.

So I would expect to wind up with the same connections as we would if we had used the period-connectivity info, but without the optimization of reusing init segments across periods. We may choose to implement that optimization at a later time.

[avelad]

I think that point 5.3.2.2 of https://dashif.org/docs/DASH-IF-IOP-v4.3.pdf could be useful. what do you think?

AssetIdentifier descriptors identify the asset to which a Period belongs. This can be used
for implementation of client functionality that depends on distinguishing between ads and main
content (e.g. progress bar).
Periods with same AssetIdentifier should have identical Adaptation Sets, Initialization
Segments and same DRM information (i.e., DRM systems, licenses). This allows reuse of at least
some initialization data across periods of the same asset, and ensures seamless continuation of
playback if inserted periods have zero duration. Period continuity or connectivity should be sig-
naled, if the content obeys the rules.