Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Joining Fetch #638

Open
wants to merge 35 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
619a6ec
Joining Fetch
englishm Nov 27, 2024
4525f9d
Fix client/peer in GOAWAY Timeout
englishm Dec 4, 2024
8e1e9ca
Normalize case
englishm Dec 4, 2024
7f19b5e
Normalize case
englishm Dec 4, 2024
921f4c4
Normalize case
englishm Dec 4, 2024
c285647
Normalize case
englishm Dec 4, 2024
8fa9be8
Remove extra space
englishm Dec 4, 2024
e1093ad
spelling: s/Groupe/Group/
englishm Dec 4, 2024
1522ab7
Update with a more verbose algorithm for clarity
englishm Dec 18, 2024
0a00c36
Simplify algorithm w/ intermediate resolved values
englishm Dec 18, 2024
e0e649e
Previous Group -> Preceding Group Offset
englishm Dec 18, 2024
07e93ce
Fetch error rather than closing session
englishm Dec 18, 2024
d01b841
Fix Resolved Subscribe Start Object (LatestGroup)
englishm Dec 18, 2024
dbc9f9a
Clarify empty fetch case
englishm Dec 18, 2024
204437e
Guidance for certain cases
englishm Dec 18, 2024
578b567
Update draft-ietf-moq-transport.md
englishm Dec 19, 2024
6dae36d
Update draft-ietf-moq-transport.md
englishm Dec 20, 2024
9fa2429
Incorporate feedback
englishm Jan 22, 2025
a50b040
Brief motivating example for Suhas
englishm Jan 22, 2025
6c0790f
Nit: spacing
englishm Jan 22, 2025
eb20e67
fix phrasing nit
englishm Jan 24, 2025
7d4a99d
Cleanup - remove duplicate paragraph
englishm Jan 24, 2025
450b4d7
Clarify properties apply to Fetch and Subscribe
englishm Jan 24, 2025
7db23b5
Group Order for all Fetch Types
englishm Jan 24, 2025
86ec7ff
Remove unnecessary/redundant definitions
englishm Jan 24, 2025
094d800
Merge branch 'main' into me/joining-fetch
ianswett Jan 29, 2025
5b1ec66
Reflow some long lines
ianswett Jan 30, 2025
9cbde3c
Reflow and minor editorial updates
ianswett Jan 30, 2025
e5a2868
Shorten some descriptions
ianswett Jan 30, 2025
d7e8648
Merge branch 'main' into me/joining-fetch
ianswett Jan 30, 2025
f408e38
Update draft-ietf-moq-transport.md
englishm Jan 30, 2025
46f2124
Merge branch 'main' into me/joining-fetch
ianswett Jan 31, 2025
6076f68
Merge branch 'main' into me/joining-fetch
ianswett Feb 3, 2025
df48071
Merge branch 'main' into me/joining-fetch
ianswett Feb 3, 2025
a70455e
Limit Joining Fetch to Latest Object Subscribes
englishm Feb 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 85 additions & 17 deletions draft-ietf-moq-transport.md
Original file line number Diff line number Diff line change
Expand Up @@ -1514,6 +1514,27 @@ objects within a track. The publisher responding to a FETCH is responsible for r
all available Objects. If there are gaps between Objects, the publisher omits them from the
fetch response. All omitted objects have status Object Does Not Exist.

**Fetch Types**

There are two types of Fetch messages:

Standalone Fetch (0x1) : A Fetch of Objects performed independently of any Subscribe.

Joining Fetch (0x2) : A Fetch joined together with a Subscribe. A Joining Fetch
shares the same Subscribe ID as an already-sent Subscribe. A publisher receiving a
Joining Fetch should use properties of the associated Subscribe to determine the
Track Namespace, Track, StartGroup, StartObject, EndGroup, and EndObject for the
Joining Fetch such that it is contiguous with the associated Subscribe and begins
Preceding Group Offset prior.

A Subscriber can use a Joining Fetch to, for example, fill a playback buffer with a
certain number of groups prior to the live edge of a track.

A Joining Fetch is only permitted when the associated Subscribe has the Filter
Type Latest Object.

A Fetch Type other than the above MUST be treated as an error.

A publisher responds to a FETCH request with either a FETCH_OK or a FETCH_ERROR
message. If it responds with FETCH_OK, the publisher creates a new unidirectional
stream that is used to send the Objects. A relay MAY start sending objects immediately
Expand All @@ -1533,29 +1554,31 @@ FETCH Message {
Type (i) = 0x16,
Length (i),
Subscribe ID (i),
Track Namespace (tuple),
Track Name Length (i),
Track Name (..),
Subscriber Priority (8),
Group Order (8),
ianswett marked this conversation as resolved.
Show resolved Hide resolved
StartGroup (i),
StartObject (i),
EndGroup (i),
EndObject (i),
Fetch Type (i),
[Track Namespace (tuple),
Track Name Length (i),
Track Name (..),
StartGroup (i),
StartObject (i),
EndGroup (i),
EndObject (i),]
[Preceding Group Offset (i),]
Number of Parameters (i),
Parameters (..) ...
}
~~~
{: #moq-transport-fetch-format title="MOQT FETCH Message"}

Fields common to all Fetch Types:

* Subscribe ID: The Subscribe ID identifies a given fetch request. Subscribe ID
is a variable length integer that MUST be unique and monotonically increasing
within a session.

* Track Namespace: Identifies the namespace of the track as defined in
({{track-name}}).

* Track Name: Identifies the track name as defined in ({{track-name}}).
within a session. For a Standalone Fetch a new Subscribe ID MUST be used. For
a Joining Fetch, the Subscribe ID MUST correspond to a Subscribe which has already
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I cancel the Subscribe, does it cancel the Fetch? I assume SUBSCRIBE_UPDATE doesn't have any impact on the Fetch?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Canceling the Subscribe would not cancel the Fetch, but if you cancel the Subscribe before sending the Joining Fetch the Joining Fetch would fail with a FETCH_ERROR. SUBSCRIBE_UPDATE would only update the Subscribe with the given id, not the Fetch.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. It might be worth saying "Though they share an ID, cancelling or updating the subscription has no impact on the Fetch and vice versa."?

been sent. If a publisher receives a Joining Fetch with a Subscribe ID that does
not correspond to an existing Subscribe, it MUST respond with a Fetch Error.

* Subscriber Priority: Specifies the priority of a fetch request relative to
other subscriptions or fetches in the same session. Lower numbers get higher
Expand All @@ -1566,6 +1589,17 @@ Ascending (0x1) or Descending (0x2) order by group. See {{priorities}}.
A value of 0x0 indicates the original publisher's Group Order SHOULD be
used. Values larger than 0x2 are a protocol error.

* Fetch Type: Identifies the type of Fetch, whether joining or standalone.

* Parameters: The parameters are defined in {{version-specific-params}}.

Fields present only for Standalone Fetch (0x1):

* Track Namespace: Identifies the namespace of the track as defined in
({{track-name}}).

* Track Name: Identifies the track name as defined in ({{track-name}}).

* StartGroup: The start Group ID.

* StartObject: The start Object ID.
Expand All @@ -1575,8 +1609,11 @@ used. Values larger than 0x2 are a protocol error.
* EndObject: The end Object ID, plus 1. A value of 0 means the entire group is
requested.

* Parameters: The parameters are defined in {{version-specific-params}}.
Field present only for Joining Fetch (0x2):

* Preceding Group Offset: The group offset for the Fetch prior and relative
to the StartGroup of the corresponding Subscribe. A value of 0 indicates
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the StartGroup of the Subscribe is in the future? The current text and algorithm below seem to indicate I could issue a joining fetch for Objects that haven't been published?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good point I had not previously considered. We probably should preclude joining a Fetch to a Subscribe in the future.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made a pass at limiting Joining Fetch to only Subscribes with the Latest Object Filter Type. It definitely reduces the complexity here quite a bit. See a70455e.

the current Group.

Objects that are not yet published will not be retrieved by a FETCH.
The latest available Object is indicated in the FETCH_OK, and is the last
Expand All @@ -1590,9 +1627,40 @@ subgroup ID is not used for ordering.
If StartGroup/StartObject is greater than the latest published Object group,
the publisher MUST return FETCH_ERROR with error code 'No Objects'.

A publisher MUST send fetched groups in group order, either ascending or
descending. Within each group, objects are sent in Object ID order;
subgroup ID is not used for ordering.
### Calculating the Range of a Joining Fetch

A publisher which receives a Fetch message with a Fetch Type of 0x2 treats it
as a Fetch with a range dynamically determined by the Preceding Group Offset
field and field values derived from the corresponding SUBSCRIBE message
(hereafter "the Subscribe").

The following values are used:

* Resolved Subscribe Start Group: the Largest Group ID of the associated Subscribe.
* Resolved Subscribe Start Object: the Largest Object ID of the associated Subscribe.

The Resolved Subscribe Start values for a Joining Fetch MUST correspond to the
referenced Subscribe within the same session so that the ranges of Objects covered
by the Fetch and Subscribe are contiguous and non-overlapping.
If no Objects have been published for the track, so the SUBSCRIBE_OK has a
ContentExists value of 0, the publisher MUST respond to the Joining Fetch with a
FETCH_ERROR.

The publisher receiving a Joining Fetch computes the fetch range as follows:

englishm marked this conversation as resolved.
Show resolved Hide resolved
* Fetch StartGroup: Resolved Subscribe Start Group - Preceding Group Offset
* Fetch StartObject: 0

If Resolved Subscribe Start Object is 0:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Resolved Subscribe Start Object is 0/ Subscribe FilterType is LatestGroup

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not the only case here. This can also be true if the StartObject field of a Subscribe with Filter Type of AbsoluteStart or AbsoluteRange is 0. This is an example of why I'd prefer to retain that above section defining "Resolved Subscribe..." values rather than just referring directly to the Subscribe section's definitions.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine with having the section on Resolved subscribes , but seems bit incorrect to compute those absolute values given those are filters and may not map to the exact values that we mention here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above, I'm thinking of this in terms of computing an absolute value for the requested range so we can delineate which portion would be provided by a Fetch (subject to whatever limitations a standalone Fetch is subject to) and which portion would be provided by a Subscribe (subject to whatever limitations a Subscribe is subject to).

* Fetch EndGroup: Resolved Subscribe Start Group - 1
* Fetch EndObject: 0 (all objects in the group)

Else, if Resolved Subscribe Start Object is 1 or more:
* Fetch EndGroup: Resolved Subscribe Start Group
* Fetch EndObject: Resolved Subscribe Start Object

If Fetch EndGroup < Fetch StartGroup, there are no Objects to retrieve,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's always true that if there are no Objects in the Fetch range, it results in a FETCH_ERROR, so I'm not sure we need to say it again here. If you are going to restate it, I think Fetch EndGroup == Fetch StartGroup && Fetch EndObject < Fetch StartObject also results in a FETCH_ERROR?

so respond with a `FETCH_ERROR`.

## FETCH_CANCEL {#message-fetch-cancel}

Expand Down