Capturing Dolby Atmos encoded audio stream generated by Windows Spatial Audio via Windows.Media.Audio APIs

[astarktc]

There are many PC games that support Dolby Atmos through a direct integration from the game's audio engine to Windows' Spatial Audio API:
https://learn.microsoft.com/en-us/windows/win32/api/spatialaudioclient/
https://learn.microsoft.com/en-us/windows/win32/coreaudio/spatial-sound

I believe it is possible that the capability can be built into Sunshine to take advantage of OS-level Dolby Atmos encoding capabilities and capture pre-encoded spatial sound to stream to Moonlight. That is, of course, if Moonlight itself is also updated to receive those streams and process them accordingly on compatible client devices. Windows based Moonlight clients should definitely work I would think, otherwise, the client needs to be able to receive standalone dolby digital plus audio with atmos metadata and pass it on to HDMI eARC to one's surround sound system. This would rule out moonlight-tv for example, as WebOS requires dolby digital audio to be muxed with video to decode, which would add too much latency per @mariotaku

In my research, I was unable to figure out if the dolby audio can be captured from the stream that feeds into the physical sound device (like the HDMI output of my GPU). However, if we look at Windows.Media.Audio (https://learn.microsoft.com/en-us/uwp/api/windows.media.audio?view=winrt-22621) I found several classes that may clue us in to a solution:

SpatialAudioDeviceConfiguration: "Represents a audio device, providing information and control for its current spatial audio setup."
SpatialAudioFormatConfiguration: "Provides a way to configure Spatial Audio Formats."
SpatialAudioFormatSubtype: "Helper class to retrieve the GUIDs for known spatial audio formats."
(SpatialAudioFormatSubtype.DolbyAtmosForHomeTheater Property)
IAudioNodeWithListener: "The interface implemented by audio nodes that support a spatial audio listener."

I'm no programmer, so I'm new to the concept of windows audio graphs, but as I understand it, when a game is running, configured for atmos, and the sound device is set to "Dolby Atmos for Home Theater" mode, there's going to be a AudioDeviceOutputNode that is sending the encoded dolby format, and it'll be connected to a node that represents the audio device (GPU HDMI out).

Conceptually, either that output node can have it's data pulled by more than one other node (such as some generic audio node initiated by Sunshine) or Sunshine itself could configure a virtual audio device within windows that advertises itself as a spatial audio compatible device utilizing the above SpatialAudioDeviceConfiguration class.

If anyone has any licensing/copyright concerns, there's no need to worry. All the built in windows dolby functionality is license free, as it is handled at the OS level. It's meant to be used :)

https://professional.dolby.com/gaming/gaming-getting-started/dolby-atmos-documentation
"Cost and Attribution
There are no additional licensing costs to support Dolby Atmos in games. On PC and Xbox rendering to Dolby Atmos is done at the OS level from the audio passed to Microsoft Spatial Sound from the game's audio engine."

[momala454]

@astarktc do you think the same could be possible for movies that includes an atmos track ?