[VLCKit/Swiftfin Player] Audio track index incorrect during transcode

[holow29]

Describe the bug

During transcoding/DirectStreaming using VLCKit/Swiftfin player, selected audio track index will be incorrect, causing no audio to play by default. To get audio to play, it appears that the next track in the index needs to be selected. The server is only sending the one track to begin with.

For example, say a file has two audio tracks: x, y.
The track selection will list: none, x, y.
Currently, 'none' will be selected by default graphically while selected track index sent to the server will indicate 'x'. Server will transcode 'x' and send that back to client. Client appears to have something other than 'x' selected, though, causing no audio to play. To get audio, 'y' needs to be selected graphically and this will actually play the transcoded 'x'.

A workaround for now is here: sy6sy2@029af18 VideoPlayerViewModel.swift changes seem to allow audio to play by default, but I am not sure the implications for DirectPlay.

Might be related to #771

Application version

main eb17300

Where did you install the app from?

None

Device information

Multiple simulators and devices

OS version

iOS 16/17, tvOS 16/17

Jellyfin server version

380c367

[LePips]

Yep, this is the issue that's causing the "no audio/wrong audio track", however I've also seen it on instances of direct play.

[holow29]

Doing some basic investigation on this: during transcoding, when VLCUI is building the audioTrackIndex in VLCMediaPlayerExtensions.swift Line 84, 'indexes' array seems to be

{[index: -1,...],
[index: 2,...]}

The question is why this is happening and how to solve for it. The expected behavior in a certain sense would be for audioTrackIndexes to look like:

{[index: -1,...],
[index: 1,...],
[index: 2,...],
etc.}

It might have something to do with the HLS playlist. For transcoding in particular, we might just expect

{[index: -1,...],
[index: 1,...]} 

since Jellyfin is only sending 1 audio track; at some point, there should be some way to deal with switching audio tracks during a transcode, but I am not sure there has been any effort for that yet. It might be pertinent to think of that when looking at solutions to this issue, though, because both efforts might involve an abstraction layer (which might already exist with the proxy - I am not sure what the proxy actually does).

-1 is the disabled state for audio, which is what we are getting by default now. We want track with index 1, but it incorrectly appears with index 2 in the array. Therefore, when it searches for track with index 1 (passed with AudioStreamIndex during VLC configuration or with track change button), nothing is found and it returns index -1 so no audio.

I haven't seen this behavior with DirectPlay; the index appears to have all the tracks. It would be interesting to see how this behaves during remuxing.