Skip to content

Commit

Permalink
Improve position estimate when transitioning to another checkpoint
Browse files Browse the repository at this point in the history
When transitioning to the next media position parameter checkpoint
we estimate the position because the audio processor chain no longer
provides access to the actual playout duration.

The estimate using the declared speed and the last checkpoint may
have drifted over time, so we currently estimate relative to the
next checkpoint, which is closer and presumably provides a better
estimate. However, this assumes that these checkpoint are perfectly
aligned without any position jumps.

The current approach has two issues:
 - The next checkpoint may include a position jump by design, e.g.
   if it was set for a new item in the playlist and the duration of
   the current item wasn't perfectly accurate.
 - The sudden switch between two estimation methods may cause a jump
   in the output position, which is visible when we add new media
   position checkpoints to the queue, not when we actually reach the
   playback position of the checkpoint.

We can fix both issues by taking a slightly different approach:
 - Continuously monitor the estimate using the current checkpoint. If
   it starts drifting, we can adjust it directly. This way the estimate
   is always aligned with the actual position.
 - The change above means we can safely switch to using the estimate
   based on the previous checkpoint. This way we don't have to make
   assumptions about the next checkpoint and any position jumps will
   only happen when we actually reach this checkpoint (which is more
   what a user expects to see, e.g. at a playlist item transition).

Issue: #1698
PiperOrigin-RevId: 690979859
  • Loading branch information
tonihei authored and copybara-github committed Oct 29, 2024
1 parent 7a8f05f commit 7c0cffd
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 16 deletions.
4 changes: 4 additions & 0 deletions RELEASENOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@
`DefaultTrackSelector.selectVideoTrack()` method.
* Add `retryCount` parameter to `MediaSourceEventListener.onLoadStarted`
and corresponding `MediaSourceEventListener.EventDispatcher` methods.
* Fix bug where playlist items or periods in multi-period DASH streams
with durations that don't match the actual content could cause frame
freezes at the end of the item
([#1698](https://github.com/androidx/media/issues/1698)).
* Transformer:
* Update parameters of `VideoFrameProcessor.registerInputStream` and
`VideoFrameProcessor.Listener.onInputStreamRegistered` to use `Format`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1732,28 +1732,31 @@ private long applyMediaPositionParameters(long positionUs) {

long playoutDurationSinceLastCheckpointUs =
positionUs - mediaPositionParameters.audioTrackPositionUs;
long estimatedMediaDurationSinceLastCheckpointUs =
Util.getMediaDurationForPlayoutDuration(
playoutDurationSinceLastCheckpointUs, mediaPositionParameters.playbackParameters.speed);
if (mediaPositionParametersCheckpoints.isEmpty()) {
long mediaDurationSinceLastCheckpointUs =
long actualMediaDurationSinceLastCheckpointUs =
audioProcessorChain.getMediaDuration(playoutDurationSinceLastCheckpointUs);
return mediaPositionParameters.mediaTimeUs + mediaDurationSinceLastCheckpointUs;
long currentMediaPositionUs =
mediaPositionParameters.mediaTimeUs + actualMediaDurationSinceLastCheckpointUs;
long mediaDurationEstimateDiffUs =
actualMediaDurationSinceLastCheckpointUs - estimatedMediaDurationSinceLastCheckpointUs;
if (Math.abs(mediaDurationEstimateDiffUs) > 10000) {
// Update current media position parameters if the estimate drifted from the actual
// media duration created by the audio processor chain. This ensures the estimate is always
// fairly accurate and we can rely on it once we enter the else-branch below.
mediaPositionParameters =
new MediaPositionParameters(
mediaPositionParameters.playbackParameters, currentMediaPositionUs, positionUs);
}
return currentMediaPositionUs;
} else {
// The processor chain has been configured with new parameters, but we're still playing audio
// that was processed using previous parameters. We can't scale the playout duration using the
// processor chain in this case, so we fall back to scaling using the previous parameters'
// target speed instead. Since the processor chain may not have achieved the target speed
// precisely, we scale the duration to the next checkpoint (which will always be small) rather
// than the duration from the previous checkpoint (which may be arbitrarily large). This
// limits the amount of error that can be introduced due to a difference between the target
// and actual speeds.
MediaPositionParameters nextMediaPositionParameters =
mediaPositionParametersCheckpoints.getFirst();
long playoutDurationUntilNextCheckpointUs =
nextMediaPositionParameters.audioTrackPositionUs - positionUs;
long mediaDurationUntilNextCheckpointUs =
Util.getMediaDurationForPlayoutDuration(
playoutDurationUntilNextCheckpointUs,
mediaPositionParameters.playbackParameters.speed);
return nextMediaPositionParameters.mediaTimeUs - mediaDurationUntilNextCheckpointUs;
// target speed instead.
return mediaPositionParameters.mediaTimeUs + estimatedMediaDurationSinceLastCheckpointUs;
}
}

Expand Down

0 comments on commit 7c0cffd

Please sign in to comment.