Interface for request tracing

[gordon-y-lai]

In our use case it is important to be able to collect some key information regarding the player's network requests (for example, the download request for a media segment). In particular, we would like to measure the start time, header arrival time, completion time, and the response status code for each request individually.

For now we are able to achieve the goals above by registering our own networking plugin (XHR, http/https) through shaka.net.NetworkingEngine.registerScheme(). However, the solution isn't ideal because it will not work properly if there are more than one Shaka player instances on the same page. This is due to the fact that the registration of the networking plugin is handled globally for all player instances, there is no way for us to know which request is associated with which player instance in more complex scenarios.

I believe there are several possible ways Shaka can help to make it easier to trace network requests:

1. When invoking the networking plugin method parse(uri, request, requestType, progressUpdated), pass an extra argument "player" to reference the associated Shaka player instance. This way, we will be able to know which player a request belongs to.

2. Introduce a new networking plugin or request interceptor which can be registered on a player instance-by-instance basis.

3. Introduce new Shaka player events which are triggered whenever a request started/progressed/ended with corresponding status code.

Thanks!

[theodab]

Would request filters and response filters work for your use case? They aren't registered globally. They are mentioned in a few of the tutorials, such as for example the license wrapping tutorial.
The shaka.extern.Request and shaka.extern.Response don't directly link to each other, but you could figure out which response corresponds to which request by checking the originalUri field of the response, and looking for a request with that same URI in its uris array. If you're using range requests, you'll also have to check to see if the range of the request and the Content-Range of the response match up.
And the type field in the filters lets you figure out if the request/response is a segment, a header, etc.

[gordon-y-lai]

@theodab - Thanks for the information. I tried the request/response filters approach as you suggested. I think it should work (since the filters can be registered for a specific player instance) to an extent while there were some shortcomings I observed. In particular:

• For our use case, the response timing metrics include two check points: (1) when the response header arrives (to capture the end-to-end latency since request was made), and (2) when the response body becomes fully available (to estimate the actual download bandwidth).
• It seems that the response filter is invoked after the response body becomes fully available, and there is a timeMs field in the response argument which looks like the elapsed time since the corresponding request was made. If I understand it correctly, we won't be able to measure the timing of check point # 1 as described above with this new approach.
• It is equally important to our use case that failed downloads (and the corresponding HTTP status code) are captured. In my quick tests with bad segment URLs, it appears that the response filters will not be involved in these error conditions. So we will need to find other ways to get notified when requests run into errors.
  Please let me know if I'm missing something.

[theodab]

Ah, I see what you mean. The time that headers become available is something we could be measuring, but we don't at the moment. And we don't really inform the user of a failed download, directly; they only know if a failed download leads to the entire player erroring, but they won't hear if a failed download leads to a retry or similar.

I'll mark this as an enhancement request, then.

To me, adding new player events sounds like the best solution. How would this set of events look to you?

1. 'downloadheadersreceived': Fired when the headers for a download are received. Extra data: the request object, the request type, and the headers (or maybe the response object, if it's available at that stage?).
2. 'downloadfailed': Fired when a download fails. Extra data: the request object, the request type, the error code, and a boolean that indicates whether or not the request was purposefully aborted.

[gordon-y-lai]

The player events sound reasonable. I like to add a few comments:

• If you also add 'downloadrequested' and 'downloadcompleted' events in a similar fashion, then technically it will be possible for us to achieve our goal without registering request/response filters. I guess that's nice but it's not a must have.
• I also like to make sure that the original HTTP response status code (aside from the Shaka error code) is available to the 'downloadfailed' event handler, since the status code can be a crucial piece of information to link the error with what's been reported on the CDN side for example.

[theodab]

Okay. I've added 'downloadheadersreceived' and 'downloadfailed'.
For now, I decided to not add your proposed 'downloadrequested' and 'downloadcompleted' events; since those are replicable using our existing interfaces, they are lower priority features. In case you're still interested I'll leave this issue open to track them.

[github-actions]

Closing due to inactivity. If this is still an issue for you or if you have further questions, the OP can ask shaka-bot to reopen it by including @shaka-bot reopen in a comment.