SUBSCRIBE(current group) splits past and future

[afrind]

One of the principles the group seemed to agree on is that FETCH is for the past and SUBSCRIBE is for the future. Draft-07 however left in SUBSCRIBE(current group) which spans both and seems to violate the principle. I understand that subscribing starting from the current group is likely to be a very common use case, as groups are MOQT join points.

My concern is that all the reasons we decided FETCH semantics were good for past data are being discarded here. If the current group consists of an large number of subgroups or datagrams, the sender can overwhelm the receiver. If the relay has a cache with gaps, we have not explained the interaction.

My strawman is to respond to SUBSCRIBE(current group) with both a FETCH OK and SUBSCRIBE OK, and objects from (current group, 0) to (current group, live edge) are returned in a FETCH stream using those semantics.

[englishm]

My thought was to resolve this with a new message, JOIN, which is decomposed into a FETCH+SUBSCRIBE at a publisher with sufficient local knowledge to do so.

[englishm]

I'll make slides for Dublin

[ianswett]

I'm not arguing for a specific approach, but people did not like removing current group in the past, ie: #510 (review)

A large number of datagrams is a congestion issue, but not a flow control one. For congestion, I'd expect the relay could lower the subscriber priority once it had too much data for a subscription?

If there's only one peep in a group, then starting at the current group doesn't create any problems. If the peeps were all going to be open at once anyway (ie: 3 or 4 enhancement layers), then the stream usage is no different than starting at the current object. As the number of peeps increases, it does start being problematic. If you're doing one Object per peep, for example.

I'd rather not add a 3rd way to retrieve Objects, but I can see an argument for merging the existing functionality of FETCH and SUBSCRIBE and saying:

• You're getting things before the latest Object on a single stream, and ones after it in peeps or datagrams.

If we want to use track alias, I think we still need to prohibit having two "JOINs" that end beyond the current object? Alternatively, we could just say that you'll only receive one copy of things after the current object, even if there are multiple matching JOINs/SUBSCRIBEs/etc?

[vasilvv]

The problem here is that since the groups are arbitrarily large, asking to subscribe starting with object 0 of the current group is not that different from starting at arbitrary group in the past.

[wilaw]

To illustrate the problem as I see it, which is similar to Victor's statement above

[gwendalsimon]

I distinguish the present to the past and the future. The present is the current group, as seen by the Relay, i.e. the latest object it got from its parent. Let's call it current_ID (in Will's figure, current_ID = 2). Any object with a group ID lesser than current_ID is in the past. They should be obtained by FETCH. Any object with a group ID greater than current_ID is in the future. They typically trigger an error if referenced in a FETCH. The objects with a group ID equal current_ID are the present.

In the present, the Relay has a normative behavior: The Relay sends objects in the current group in the order of the subgroups to each subscriber. Those with a high network connection will be at the latest object, while some subscribers with a much lower connection will be late. In Will's schema, it means that some subscribers are still downloading 2-0 while some others are at 2-1. If we extend it, it means that the Relay must store the ID of the latest sent object within the group for all subscribers. When a new subscriber joins, it is at object 0 while all others will be somewhere between 0 and X (the latest object). It is not a problem, it is the normal behavior.

Yet, I'd see a problem with ascending order in SUBSCRIBE. The subscribers with a slow network will cumulate drift and will eventually be far in the past, at the point that the Relay serves some subscribers in the present (2-1 and 2-0 in Will's schema) and some late subscribers in the past (1-X).

[afrind]

The subscribers with a slow network will cumulate drift

#462 is meant to capture that issue, I think and #582 also discusses it.

[martinduke]

Parked until JOIN/FETCH (#638) is resolved, which may make the answer to this obvious.

[ianswett]

As I pointed out at one virtual interim, I think the past/future problem is one we already have, it's just not as bad.

Specifically, if one issues a SUBSCRIBE and FETCH and the FETCH has a higher priority (think lowish latency live sports) to fetch data from the recent past (ie: 15 second jitter buffer), the latest object could be in the next group or later by the time it can start sending Objects for the subscription

One could close the subscription with too far behind in this case, but you risk closing the subscription right around the time the video playback gets to the live head.

[wilaw]

Specifically, if one issues a SUBSCRIBE and FETCH and the FETCH has a higher priority (think lowish latency live sports) to fetch data from the recent past (ie: 15 second jitter buffer), the latest object could be in the next group or later by the time it can start sending Objects for the subscription

One could close the subscription with too far behind in this case, but you risk closing the subscription right around the time the video playback gets to the live head.

This is why a JOINING FETCH is so useful. It times the subscription so that the latest live object splices neatly with the tail of the FETCH and you avoid the client problem of aligning the two across a RTT gap.

[wilaw]

As a comment on the original post, I think a clear notion that SUBSCRIBE only returns future objects would serve us well. IN a thought experiment, we can imagine a relay that has zero cache. We can then define SUBSCRIBE such that it is the track that such a relay could return. Combined with FETCH for the past and JOINING FETCH for near-live joins, we have a workable system.

[afrind]

Individual Comment:

Specifically, if one issues a SUBSCRIBE and FETCH and the FETCH has a higher priority (think lowish latency live sports) to fetch data from the recent past (ie: 15 second jitter buffer), the latest object could be in the next group or later by the time it can start sending Objects for the subscription

Maybe this is a use case suited to descending FETCH and giving SUBSCRIBE higher priority? Doesn't that give you the most-recent 15s playback buffer in the shortest time? eg if it takes X seconds to download (15-X) seconds of the past while deferring to live data.

[ianswett]

Individual Comment:

Specifically, if one issues a SUBSCRIBE and FETCH and the FETCH has a higher priority (think lowish latency live sports) to fetch data from the recent past (ie: 15 second jitter buffer), the latest object could be in the next group or later by the time it can start sending Objects for the subscription

Maybe this is a use case suited to descending FETCH and giving SUBSCRIBE higher priority? Doesn't that give you the most-recent 15s playback buffer in the shortest time? eg if it takes X seconds to download (15-X) seconds of the past while deferring to live data.

In theory, that sounds optimal though one is going to have the smallest buffer at the beginning of a playback, because an application wants to balance join latency with smooth playbacks, so I think it's more likely an application always starts at some number of groups behind the current group (or maybe at the current group) and increases it's jitter buffer over time if there are rebuffers. To optimize for join latency, you'd want ascending FETCH with a slightly higher priority than the SUBSCRIBE. In that case, you might start playing when you have 2 seconds of buffer even if you eventually want to have 10 or 15 seconds.

[ianswett]

Ideally the fix for this issue should more clearly define what the current object is, which is a topic that came up in a recent PR: #652 (comment)