dxgi exception with Vulkan backend on Windows

[AlfredAn]

Description
When running the "hello-triangle" example, I get the following error message:

[2023-10-16T17:55:54Z ERROR wgpu_hal::auxil::dxgi::exception] ID3D12CommandQueue::ExecuteCommandLists: Using ClearRenderTargetView on Command List (0x000001C77D466B10:'Unnamed ID3D12GraphicsCommandList Object'): Resource state (0x0: D3D12_RESOURCE_STATE_[COMMON|PRESENT]) of resource (0x000001C77D395060:'Unnamed ID3D12Resource Object') (subresource: 0) is invalid for use as a render target.  Expected State Bits (all): 0x4: D3D12_RESOURCE_STATE_RENDER_TARGET, Actual State: 0x0: D3D12_RESOURCE_STATE_[COMMON|PRESENT], Missing State: 0x4: D3D12_RESOURCE_STATE_RENDER_TARGET. [ EXECUTION ERROR #538: INVALID_SUBRESOURCE_STATE]
[2023-10-16T17:55:54Z ERROR wgpu_hal::auxil::dxgi::exception] ID3D12CommandQueue::ExecuteCommandLists: Using IDXGISwapChain::Present on Command List (0x000001C77D3FB510:'Internal DXGI CommandList'): Resource state (0x4: D3D12_RESOURCE_STATE_RENDER_TARGET) of resource (0x000001C77D395060:'Unnamed ID3D12Resource Object') (subresource: 0) is invalid for use as a PRESENT_SOURCE.  Expected State Bits (all): 0x0: D3D12_RESOURCE_STATE_[COMMON|PRESENT], Actual State: 0x4: D3D12_RESOURCE_STATE_RENDER_TARGET, Missing State: 0x0: D3D12_RESOURCE_STATE_[COMMON|PRESENT]. [ EXECUTION ERROR #538: INVALID_SUBRESOURCE_STATE]
[2023-10-16T17:55:54Z ERROR wgpu_hal::auxil::dxgi::exception] ID3D12CommandQueue::Present: Resource state (0x4: D3D12_RESOURCE_STATE_RENDER_TARGET) of resource (0x000001C77D395060:'Unnamed ID3D12Resource Object') (subresource: 0) is invalid for use as a INITIAL|PRESENT.  Expected State Bits (all): 0x0: D3D12_RESOURCE_STATE_[COMMON|PRESENT], Actual State: 0x4: D3D12_RESOURCE_STATE_RENDER_TARGET, Missing State: 0x0: D3D12_RESOURCE_STATE_[COMMON|PRESENT]. [ EXECUTION ERROR #538: INVALID_SUBRESOURCE_STATE]
[2023-10-16T17:55:56Z ERROR wgpu_hal::auxil::dxgi::exception] ID3D12CommandQueue::ExecuteCommandLists: Using ClearRenderTargetView on Command List (0x000001C77D520D60:'Unnamed ID3D12GraphicsCommandList Object'): Resource state (0x0: D3D12_RESOURCE_STATE_[COMMON|PRESENT]) of resource (0x000001C76D0508C0:'Unnamed ID3D12Resource Object') (subresource: 0) is invalid for use as a render target.  Expected State Bits (all): 0x4: D3D12_RESOURCE_STATE_RENDER_TARGET, Actual State: 0x0: D3D12_RESOURCE_STATE_[COMMON|PRESENT], Missing State: 0x4: D3D12_RESOURCE_STATE_RENDER_TARGET. [ EXECUTION ERROR #538: INVALID_SUBRESOURCE_STATE]
[2023-10-16T17:55:56Z ERROR wgpu_hal::auxil::dxgi::exception] ID3D12CommandQueue::ExecuteCommandLists: Using IDXGISwapChain::Present on Command List (0x000001C77D4C4040:'Internal DXGI CommandList'): Resource state (0x4: D3D12_RESOURCE_STATE_RENDER_TARGET) of resource (0x000001C76D0508C0:'Unnamed ID3D12Resource Object') (subresource: 0) is invalid for use as a PRESENT_SOURCE.  Expected State Bits (all): 0x0: D3D12_RESOURCE_STATE_[COMMON|PRESENT], Actual State: 0x4: D3D12_RESOURCE_STATE_RENDER_TARGET, Missing State: 0x0: D3D12_RESOURCE_STATE_[COMMON|PRESENT]. [ EXECUTION ERROR #538: INVALID_SUBRESOURCE_STATE]
[2023-10-16T17:55:56Z ERROR wgpu_hal::auxil::dxgi::exception] ID3D12CommandQueue::Present: Resource state (0x4: D3D12_RESOURCE_STATE_RENDER_TARGET) of resource (0x000001C76D0508C0:'Unnamed ID3D12Resource Object') (subresource: 0) is invalid for use as a INITIAL|PRESENT.  Expected State Bits (all): 0x0: D3D12_RESOURCE_STATE_[COMMON|PRESENT], Actual State: 0x4: D3D12_RESOURCE_STATE_RENDER_TARGET, Missing State: 0x0: D3D12_RESOURCE_STATE_[COMMON|PRESENT]. [ EXECUTION ERROR #538: INVALID_SUBRESOURCE_STATE]

It happens every time the window is resized, including maximizing it. It also appears sometimes when just starting the program, without touching the window. When running the "cube" example, it seems to print the error every frame rather than just when resizing. Setting WGPU_BACKEND to dx12 seems to fix the issue in the cube benchmark, although hello_triangle is unaffected. EDIT: WGPU_BACKEND=vulkan also seems to fix the issue for some reason. Also, Bevy reports the backend to be Vulkan regardless of the value, even though the behavior changes.

I get the same error message in Bevy as well, which is where I noticed it. If I ignore the errors, everything still seems to work.

I've tried both with the latest commit (2f3d17c) and with the 0.17.2 release. I am using stable Rust 1.70.0, which is chosen automatically when running the examples.

Please let me know if I can do anything to help debug this!

Repro steps
Clone repository, run cargo run --bin cube.

Expected vs observed behavior
I expected no error messages in the terminal. Instead I get tons.

Extra materials
N/A

Platform
System info from Bevy:

2023-10-16T18:05:09.654565Z  INFO bevy_render::renderer: AdapterInfo { name: "AMD Radeon RX 6700 XT", vendor: 4098, device: 29663, device_type: DiscreteGpu, driver: "AMD proprietary driver", driver_info: "23.9.3 (AMD proprietary shader compiler)", backend: Vulkan }
2023-10-16T18:05:09.818072Z  INFO bevy_diagnostic::system_information_diagnostics_plugin::internal: SystemInfo { os: "Windows 10 Pro", kernel: "19045", cpu: "12th Gen Intel(R) Core(TM) i7-12700", core_count: "12", memory: "31.8 GiB" }

[cwfitzgerald]

So this is a "bug" in AMD's vulkan swapchain (it's debatable if it's actually a bug, or just something their card doesn't care about) when based on top of a DXGI swapchain. Recently, AMD started using DXGI swapchains all the time, not just on HDR monitors. I believe there is a fix by routing error messages through ID3D12InfoQueue instead of hooking the general windows error message scheme like we do now. (The error would still happen, it just wouldn't be for one of our devices).

WGPU_BACKEND=vulkan works because it doesn't initialize the d3d12 backend, and never hooks windows errors, so we never see this.

[AlfredAn]

So this is a "bug" in AMD's vulkan swapchain (it's debatable if it's actually a bug, or just something their card doesn't care about) when based on top of a DXGI swapchain. Recently, AMD started using DXGI swapchains all the time, not just on HDR monitors. I believe there is a fix by routing error messages through ID3D12InfoQueue instead of hooking the general windows error message scheme like we do now. (The error would still happen, it just wouldn't be for one of our devices).

WGPU_BACKEND=vulkan works because it doesn't initialize the d3d12 backend, and never hooks windows errors, so we never see this.

Thanks for the answer! Will this affect release builds in any way or can I just set the environment variable when testing and not worry about it?

[cwfitzgerald]

It shouldn't actually affect anything, is just kinda loud and annoying.

[DasLixou]

Same here

[cwfitzgerald]

This could be improved by using the new dx12 api: https://microsoft.github.io/DirectX-Specs/d3d/MessageCallback.html#id3d12infoqueue1

[NoFr1ends]

Is there any progess on this? Or at least the idea to suppress these wrong errors?

[fryeb]

Trying to wrap my head around what is going on here. It looks like when running the examples without WGPU_BACKEND set, it initializes all of the backends. And it doesn't drop them until the app closes. Meaning that unused backends are still active.

Once the application has picked the Vulkan backend, would it be possible to drop the DX12 backend? (this would unregister the exception handler and thus ignore the DX12 errors on vulkan?)

[fryeb]

Alternatively #2719 might make this issue redundant. Since this issue only happens when running the vulkan backend with the dx12 backend initialized. If DX12 is the default then Vulkan will only be used when selected explicitly (in which case there will be no DX12 backend initialized).

[cwfitzgerald]

Alternatively #2719 might make this issue redundant

This isn't going to happen, if ever, for a very long time. There a mirad of issues with our interop with dx12 that make it a worse user experience.

Once the application has picked the Vulkan backend, would it be possible to drop the DX12 backend

Not... really? A user can re-request adapters at any moment and we would need to re-initalize it. This all gets very hairy quickly.

My suggestion is one of two things:

• Easier: Only register a dx12 error callback once a dx12 device is created. Once all dx12 devices go away, unregister the callback. Basically using a refcount system. See https://github.com//gfx-rs/wgpu/blob/a157c3cf4b857958e65b669e1313b1b112c8c84a/wgpu-hal/src/auxil/dxgi/exception.rs and the callers of register/unregister
• Harder and only fixes Windows 11. Switch to the ID3D12InfoQueue1's message callback api. This is the correct way of doing error logging, but only works on Win11.

[fryeb]

I've raised a PR that takes the easier path, since it's likely we'd need it anyway as a fallback since the more "correct" option only works on Win11.

It does seem to be fixing the issues on my machine, but I'm not really sure how to validate that the error handling behavior is correct, since I'm not sure of an easy way to deliberately induce DX12 errors to make sure we're catching them.