Improving gaming on the web

[nondebug]

Description

MDN: Gamepad API, Pointer Lock API, Screen Orientation API

Specifications: Gamepad API, Pointer Lock API, Screen Orientation API

Games need reliable input APIs. Gamepad API and Pointer Lock API support are critically important for gaming on the web, especially for cloud gaming. Screen Orientation API, especially orientation locking, is important for gaming on mobile devices where orientation changes can disrupt gameplay.

Tasks:

• Fix Gamepad API failing tests (excluding idlharness-extensions.https.window.html)
• Fix Pointer Lock API failing tests
• Update test environments for Chromium, Firefox, and Safari to enable Screen Orientation API locking tests to pass on wpt.fyi. Currently these tests fail when run on hosts that do not support screen orientation locking.

Specification

Gamepad API, Pointer Lock API, Screen Orientation API are all Working Draft stage standards-track specifications in WebApps WG

Additional Signals

This proposal is primarily driven by feedback from web platform developers regarding a use case (gaming) which is currently too niche to rank highly on surveys. Cloud gaming operators are heavily impacted by breakage in these APIs and have highlighted the need for better testing and interoperability.

The scope for this proposal is limited to fixing the existing failing tests, but is intended as as first step toward addressing the larger issue of poor test coverage. Automated testing is limited as testing most of the API surface would require implementing new test APIs to simulate connected peripherals (gamepads) or system capabilities (pointer locking, screen orientation locking). These test automation APIs should be implemented and new tests should be written in order to continue making progress on this focus area in future Interop iterations. Specifying new test APIs requires standards work and is out of scope for Interop, but can be pursued in parallel with this proposal.

[rejhgadellaa]

I think it's worth mentioning the Fullscreen API here. With regards to support on iOS Safari, MDN reports:

Only available on iPad, not on iPhone. Shows an overlay button which can not be disabled. Swiping down exits fullscreen mode, making it unsuitable for some use cases like games.
MDN: Fullscreen API

Also see: https://webventures.rejh.nl/blog/2024/history-of-safari-show-stoppers/#anchor--2020-add-fullscreen-api-to-ios-display-fullscreen

Update: There is a separate Interop proposal for the FS API

[evanjacover]

As we explore cloud gaming at Jackbox Games, these features do become more critical. We are currently taking the approach of limiting the browsers we support based on some of these features, especially screen orientation.

[AshleyScirra]

Many of the mentioned features are useful for web games made in Construct so we generally support this, but especially gamepad. In particular it would be useful to have good support for button and axis events (see w3c/gamepad#152)

[shalanah]

Related Proposals:

• Fullscreen API #732
• ScreenOrientation: lock() method #738
• Vibration API #718
• Audio + Video API volume control #712

[xingri]

We, NVIDIA Geforce Now cloud gaming, strongly agree with the current proposal and wish to participate in activities that improve the compatibility and interoperability of APIs for cloud gaming on the web.

As an example of cloud gaming's reliance on the Pointer Lock API, all cloud gaming providers experienced a surge in crash issues last July, as shown in this Reddit post. This tracking issue explains the root cause.

Beyond unexpected crashes described above, the Pointer Lock API also suffers from compatibility issues across different browsers, as described in this bug report.

We also advocate for better coverage and cross-browser compatibility for Gamepad APIs, as outlined in the issue.

These examples highlight the very real challenges faced by developers and users due to inconsistencies and shortcomings in key web gaming APIs. Addressing these issues within Interop 2025 is crucial for the continued growth and success of web gaming.

[marcoscaceres]

Not sure if it's a stretch, but adding Screen Wake Lock into the mix might be good too. Agree that Fullscreen is pretty fundamental to all these, but could be handled by #732.

[AshleyScirra]

Also note following Chromium bugs involving Pointer Lock:

• pointerrawupdate does not fire when mouse buttons change with Pointer Lock - https://issues.chromium.org/issues/40817651
• Pointer lock unadjustedMovement breaks pointerrawupdate buttons - https://issues.chromium.org/issues/353249342

[gsnedders]

• Update test environments for Chromium, Firefox, and Safari to enable Screen Orientation API locking tests to pass on wpt.fyi. Currently these tests fail when run on hosts that do not support screen orientation locking.

At least in the case of macOS, I'm pretty sure there is no concept of any orientation lock, for example, thus there's not really any way to update the test environment to achieve that. I don't know what the status quo for GNOME/Wayland or Windows is? (And changing to iOS doesn't seem like "updating" the test environment — that's a totally different test environment.)