Use SeekParameters in ClippingMediaSource to snap clipping positions to keyframes

[bkraszewski]

Hello,

I have problem with ExoPlayer - ClippedMediaSource.
In application I have a source video.
From this video, I'd like to play content from start to end - in miliseconds, in sample app its:
start: 1140ms end 3140ms

I'd like to concat it with looped part of a video:
start: 3140ms end 34050ms

Looks like perfect scenario for ClippingMediaSource, LoopingMediaSource and ConcatenatingMediaSource.

I created sample application with code used for it:
https://github.com/bkraszewski/ExoPlayerSample/tree/master

Also you can take a look at video captured:
https://www.youtube.com/watch?v=SOBl6biAUgI

Hoverer I noticed transition between start part and looped part is not smooth, there is a visual glitch when switching part. In docs I found that I need to make sure time match keyframes - but can I make it dynamically?
There is a way to find closest frame to exact moment of video ending so I can adjust clipping time?
This video is just a sample, but our IOS team is using similar logic to stitch videos with same start/end params and its smooth.

Testing ENV:
ExoPlayer 2.9.5 (previously tested with 2.9.4, same result)
Android Studio 3.3
Phone: Samsung S8

[tonihei]

ClippingMediaSource currently applies the start and end position as it is. If a source doesn't start with a keyframe, we can't play transitions seamlessly. The reason is that we would need to decode two streams in parallel (the end of the first stream and the start of the second stream from the previous keyframe to the actual start position) to allow such a transition.

However, we do already have a mechanism to snap seek positions to the closest, previous or next keyframe (see SeekParameters). We can potentially reuse this mechanism to allow ClippingMediaSource to snap its clipping positions once it has the required information. I'll mark this issue as an enhancement to track that.

Final note: If you know the exact position of the keyframe in the video, you can set this position as clipping point and set the enableInitialDiscontinuity parameter of ClippingMediaSource to false to get seamless transitions with the current video.

[bkraszewski]

Thanks for reply, can I use snap seek to retrieve information about frames in time window or its implemented internally?

…

On Tue, Feb 12, 2019, 17:51 tonihei ***@***.***> wrote: ClippingMediaSource currently applies the start and end position as it is. If a source doesn't start with a keyframe, we can't play transitions seamlessly. The reason is that we would need to decode two streams in parallel (the end of the first stream and the start of the second stream from the previous keyframe to the actual start position) to allow such a transition. However, we do already have a mechanism to snap seek positions to the closest, previous or next keyframe (see SeekParameters). We can potentially reuse this mechanism to allow ClippingMediaSource to snap its clipping positions once it has the required information. I'll mark this issue as an enhancement to track that. Final note: If you know the exact position of the keyframe in the video, you can set this position as clipping point and set the enableInitialDiscontinuity parameter of ClippingMediaSource to false to get seamless transitions with the current video. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5501 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAvO2fRLV9fvExbH1augPIZbEy7wpKyLks5vMvEbgaJpZM4a3JY1> .

[tonihei]

Interesting question. You could try to pause the player player.setPlayWhenReady(false), set the seek parameters with player.setSeekParameters(SeekParameters.CLOSEST_SYNC) and then seek to the clipping position (player.seekTo(positionMs)). You should receive two callbacks with onPositionDiscontinuity, one with reason SEEK and shortly after that with reason SEEK_ADJUSTMENT (that's the snapping). If you call player.getCurrentPosition() in the second callback, you should be able to obtain the position of the keyframe.

I haven't tested it, but it may work. Note that the keyframe may not be at a millisecond boundary and the returned value could therefore be slightly wrong.

[ojw28]

To clarify, is the idea here that we'd re-use SeekParameters, but that the parameters for clipping would be supplied separately to those that are set on the player (e.g. when constructing the ClippingMediaSource)? I don't think it would make sense to use the ones that are set on the player, particularly given they can be changed during playback.

As an extra question, can #2347 be duplicated onto this issue? This one seems to be tracking a superset of the functionality being requested on that issue.

[tonihei]

Clarification sounds correct, also close the other issue.