Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add HDR support #2030

Merged
merged 24 commits into from
Mar 22, 2024
Merged

Add HDR support #2030

merged 24 commits into from
Mar 22, 2024

Conversation

shinyquagsire23
Copy link
Contributor

@shinyquagsire23 shinyquagsire23 commented Mar 15, 2024

Adds HDR support by changing compositing framebuffer formats to DXGI_FORMAT_R16G16B16A16_FLOAT (conditionally). Currently, it only works with Windows NVENC.

Tested and working on:

  • RTX 4090
  • Apple Vision Pro
  • Quest 3
  • Quest Pro
  • Pico 4

Other notes:

  • Has to convert to DXGI_FORMAT_P010 and DXGI_FORMAT_NV12 in an extra shader pass, because NVENC is really picky about texture inputs for NV_ENC_BUFFER_FORMAT_YUV420_10BIT and NV_ENC_BUFFER_FORMAT_NV12 and the other 10-bit ones didn't really work.
  • There's a bit of a hack in the YUV pipeline: The Y and UV components are attached to the shader as two output textures, because that's just how it works. But for some ungodly reason, DirectX 11 doesn't raster the entire Y plane unless it is the only texture view attached. If there's two texture views attached, it renders exactly 1/4 of the Y plane. Feels like a driver bug tbh. So there's a hack for that.
  • HDR uses BT.2020 instead of BT.709
  • The transfer characteristic for NVENC was changed to sRGB. I don't think it matters except for video players that don't know the underlying texture is sRGB.
  • The frame render shader has an extra 2 cases for sRGB correction, only active for HDR rendering of sRGB games or SDR rendering of HDR games.
  • The frame render shader also has an optional clamp() to remove extended range color (ie, so HDR/float16 games will reduce banding but not require extra tonemapping, which we don't do, to look correct)
  • The Android client was modified to composite to float16, which reduces banding on Quest 3, and displays extended gamut color on Quest Pro (for some reason it does not reduce banding on Quest Pro??)
  • The Android client also had a minor bugfix as a side-effect: my limited range bugfix was also applied erroneously to the lobby. It had to be turned off for HDR/BT.2020 decoded streams.

Bugs:

  • The YUV render target seems to have a different issue on AMD GPUs, and renders the UV channel incorrectly. It's possible DirectX requires some kind of extra alignment besides the framebuffer size being even. The good news is that my AMD codec patches seem to work, so it's just the P010 texture.

Shaders were compiled with:

"C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\x86\fxc.exe" /T ps_5_0 /O3 /Qstrip_reflect /Qstrip_debug /Qstrip_priv /Qstrip_rootsignature /Fo alvr\server\cpp\platform\win32\ColorCorrectionPixelShader.cso alvr\server\cpp\alvr_server\shader\ColorCorrectionPixelShader.hlsl
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\x86\fxc.exe" /E PS /T ps_5_0 /O3 /Qstrip_reflect /Qstrip_debug /Qstrip_priv /Qstrip_rootsignature /Fo alvr\server\cpp\platform\win32\FrameRenderPS.cso alvr\server\cpp\alvr_server\shader\FrameRender.fx
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\x86\fxc.exe" /T ps_5_0 /O3 /Qstrip_reflect /Qstrip_debug /Qstrip_priv /Qstrip_rootsignature /Fo alvr\server\cpp\platform\win32\CompressAxisAlignedPixelShader.cso alvr\server\cpp\alvr_server\shader\CompressAxisAlignedPixelShader.hlsl

"C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\x86\fxc.exe" /T ps_5_0 /O3 /Qstrip_reflect /Qstrip_debug /Qstrip_priv /Qstrip_rootsignature /Fo alvr\server\cpp\platform\win32\rgbtoyuv420.cso alvr\server\cpp\alvr_server\shader\rgbtoyuv420.hlsl

"C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\x86\fxc.exe" /T vs_5_0 /O3 /Qstrip_reflect /Qstrip_debug /Qstrip_priv /Qstrip_rootsignature /Fo alvr\server\cpp\platform\win32\QuadVertexShader.cso "alvr\server\cpp\platform\win32\d3d-render-utils\QuadVertexShader.hlsl"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\x86\fxc.exe" /E VS /T vs_5_0 /O3 /Qstrip_reflect /Qstrip_debug /Qstrip_priv /Qstrip_rootsignature /Fo alvr\server\cpp\platform\win32\FrameRenderVS.cso alvr\server\cpp\alvr_server\shader\FrameRender.fx

@shinyquagsire23 shinyquagsire23 requested a review from zmerp March 15, 2024 07:48
@barnabwhy
Copy link
Contributor

barnabwhy commented Mar 16, 2024

Testing this on Windows + AMD and my Quest 3, it does not seem to be working very well :(

h264 + HDR:
8c194df930f3e07c2653ebf21097a01b
31571463758f5c357bd35266f1ae16be

Copy link
Member

@zmerp zmerp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style-wise, looks ok to to me.

@zmerp zmerp merged commit 68ea265 into alvr-org:master Mar 22, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants