HEVC content is unable to be played back (correctly) via the Native Player

[flopsiedev]

Describe the bug

When playing back HEVC/H.265 or 4K UHD content, the Native Player feature will either not play or on mobile, playback with degraded audio.

Steps to Reproduce

1. Open Swiftfin
2. Navigate to the Gear icon in the top-right.
3. Scroll down until you see an option for "Experimental". Open it.
4. Under the "Experimental" section, enable Native Player.
5. Navigate to any media that is formatted in HEVC.
   • For me, this was attempted with untouched 4K UHD disc rips using MakeMKV.
6. Play the content back.

Results

iOS & iPadOS
On these two platforms, it will be black periodically before showing the QuickTime Audio logo. It may come to show that the audio is played back at a noticeably degraded quality.

tvOS
On the Apple TV 4K, due to bug #676, you might notice the same thing with iOS and iPadOS. However, for the time being, it seems though no content gets played back at all, showing a black screen with an error icon next to the playhead.

Application version

1.0.0(79 & 84)

Where did you install the app from?

App Store

Device information

iPhone 14 Pro Max, iPad mini (6th generation), Apple TV 4K (3rd generation)

OS version

iOS 16.3 (20D47), iPadOS 16.3 (20D47), tvOS 16.3 (20K6650), tvOS 16.3 (20K5626c)

Jellyfin server version

10.8.9

[holow29]

You need to turn fmp4 experimental feature on if you want to play that content (HEVC) on native player.

If you still have issues, it might be due to the device profile, which I am still working on: #296

Native Player cannot do tonemapping on HLS content, so if your display does not support HDR10, you need to use VLCKit player - which does not support HDR - (i.e. turn native off) or have the server tonemap/transcode the video.

[flopsiedev]

You need to turn fmp4 experimental feature on if you want to play that content (HEVC) on native player.

If you still have issues, it might be due to the device profile, which I am still working on: #296

Yeah, I definitely still have issues with this, so I'll assume it's due to the device profile. Should I close this item then?

[holow29]

If you could provide media info and server logs that might be helpful. I can play certain 4K UHD remuxes with both features enabled (native player + fmp4 HLS) on iPhone.
There are two issues I can think of:

1. Right now DTS audio is passed to native player, which will cause video to not play. I see Bohemian Rhapsody has Atmos but also a DTS track - it is possible that the server is choosing to send DTS track and that is causing playback issue. This will be fixed by device profile changes.
2. If you don't have audio transcoding enabled (for DirectStream), server might be trying to send Atmos track, which also won't play. If you turn on audio transcoding for your user, the server should transcode the Atmos (TrueHD) track to something playable.

The server log will probably help narrow the issue down either way.