[Feature Request] Provide a better Steam Remote Play experience

[SaltyBet]

Your system information

• Steam client version (build number or date): Latest Stable Steam Client
• Distribution (e.g. Ubuntu): SteamOS on Steam Deck
• Opted into Steam client beta?: No
• Have you checked for system updates?: Yes

Please describe your issue in as much detail as possible:

TL;DR: The Sunshine/Moonlight experience is consistently superior to that of Steam Remote Play; I wanted to use this issue to both document similarities/differences between then and my experience while using both streaming solutions, and also request Valve developers to investigate why the difference is so noticeable between both streaming solutions, even when using the same set of hardware.

When directly comparing the performance/experience/feel of Steam Remote Play with that of the Sunshine/Moonlight combo, it is like night and day.

I've been reading/testing and wanted to document the similarities/differences I've found so far in this issue.

Table:

Feature	Steam Remote Play	Sunshine/Moonlight
App capture methods	Game D3D12 RGB RGB + scale; Game Delayed OpenGL NV12; Game Vulkan RGB + libyuv + scale; Game polled D3D11 NV12; Game polled D3D11 RGB, Game threaded D3D9 NV12; Steam D3D10 NV12; others?	N/A
Desktop capture methods	Desktop DWM NV12; Desktop PipeWire DMABUF; NvFBC	Desktop Duplication API; KMS; NvFBC; Wayland DMABUF; X11
Encoding support	CPU H264; GPU H264; GPU H265	CPU H264; GPU H264; GPU H265
Protocol	“custom low-latency network protocol” (https://store.steampowered.com/remoteplay/)	Moonlight Protocols (https://games-on-whales.github.io/wolf/stable/protocols/index.html)
Framepacing support	?	Yes
Controller support	Steam Input	ViGEm

I tested the following:

• Host:
  • PC.
  • Windows 10.
  • Latest Stable Steam Client.
  • Tested Both Software Encoding Enabled and Hardware Encoding Enabled.
  • GigE.
• Client:
  • Steam Deck.
  • Latest Stable SteamOS.
  • Latest Stable Steam Client.
  • Hardware Decoding Enabled.
  • WiFi 802.11 ac.
• Stream Test:
  • 1440p.
  • 60 FPS.
  • 40 Mbps.

From what I could see and experiment with:

• Steam Remote Play supports both app capture and desktop capture, while Sunshine/Moonlight seems to support desktop capture exclusively.
• Desktop capture seems to perform better overall on Sunshine/Moonlight when compared to Steam Remote Play.
• Steam Remote Play does not seem to provide a generally available high-performance/low-latency desktop capture method on Windows, while Sunshine/Moonlight provides Desktop Duplication API support for Windows.
• Steam Remote Play seems to provide a generally available high-performance/low-latency desktop capture method on Linux (Desktop PipeWire DMABUF), while Sunshine/Moonlight provides multiple options (KMS; Wayland DMABUF; X11).
• Steam Remote Play performed better using CPU encoding when compared to Steam Remote Play using GPU encoding.
• Sunshine/Moonlight performed better using GPU encoding when compared to Steam Remote Play using CPU encoding.
• Steam Remote Play using GPU encoding shows visible artifacts on screen if standing still after a while.
• Steam Remote Play using GPU encoding seems to fallback to CPU decoding, irrespective of client setting (see Latest Steam OS 3.4 Preview breaks hardware decoding SteamOS#903 (comment)).
• Steam Remote Play does not prove an option for H265 GPU encoding when streaming to the client (see HEVC (H.265) Encoding not available on Steam for Linux? #8731). Curiously, the option appears when using an Android client.
• Steam Remote Play sometimes can show sporadic stuttering, confirmed via the Performance Overlay frametime graph (maybe related to Steam Deck uses different Hardware decoding method between Desktop and Game Mode for Remote Play #8929).
• Sunshine/Moonlight has an option for framepacing support, so stuttering is non-existent; unsure if Steam Remote Play provides an option for this.
• Total perceived latency is much lower with Sunshine/Moonlight.

I truly would like to just use Steam Remote Play, as it is more convenient, but Sunshine/Moonlight provides a clearly a superior experience.

Is there anything Valve developers can do to improve this?

Additional files, below:

Sunshine/Moonlight stats:

Steam Remote Play CPU encoding:

CPU encoding.zip

Steam Remote Play GPU encoding:

GPU encoding artifacting.zip

[joeknock90]

I am with you here @SaltyBet. Great write up and organization. I'll be following this one.

[zacbre]

I have noticed the same exact thing in my testing. Moonlight seems to be superior in streaming latency/performance for now. I really hope valve decides to better the support for remote play considering gamestream is going away.

[trexx]

The difference is night and day. With debugging enabled on Steam Link, Steam can often choose a capture method that introduces latency in the game.
For example, in big picture mode, sometimes it opts to use Desktop DWM capture, which is really laggy but improves once a game is launched.
A game like wreckfest, the capture mode changes mid game sometimes in between loading screens, and sometime it sticks with a less than optimal method. It usually requires that I restart the stream.

To add to the differences, Sunshine supports AV1 encoding now.

Some smaller quality of life improvements are also sunshines web interface for adding new streaming clients remotely.