Improve local multiplayer support by tracking InputEvents and UI focus per player

[coffeebeats]

Describe the project you are working on

Godot games with support for local multiplayer. I define local multiplayer support as follows:

1. 2 or more players inputting actions via 1 or more input devices (typically 2 devices, but keyboard sharing is not an uncommon request).
2. The game must be able to distinguish which player input events/actions are originating from.
3. Each player must be able to independently navigate menu elements so that developers may take advantage of Godot's standard UI nodes and behaviors.

Describe the problem or limitation you are having in your project

There are two engine limitations that are blocking developers from creating local multiplayer UIs:

1. Only one Control node may be focused at a time. As such, developers cannot rely on the built-in Control node focus feature for multiple players simultaneously.
2. All Control nodes depend on the same UI built-in actions, meaning there's no way for multiple players to separately navigate a UI scene tree (all players would end up triggering the same UI actions).

In addition to these limitations, the following is a pain point for local multiplayer games:

1. Actions must be defined once per player slot, requiring either tedious InputMap modifications that must be kept in sync or non-trivial logic to replicate actions as players join/leave. While there are workarounds, this is toil that should be eliminated by the local multiplayer solution.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

With the above in mind, I think the following criteria will define a comprehensive solution to the problem:

1. Simplify defining actions which can be independently activated by multiple players (not just device - consider sharing a keyboard). This must include the built-in ui_* actions as well as custom in-game actions.
2. All action-related methods on Input and InputEvent should allow for distinguishing between different players.
3. The UI node hierarchy should be separately navigable by different players. Effectively this means that (i) each player can focus their own Control node and (ii) the built-in UI actions used by Control nodes should be player-aware.
4. All focus-related methods on Control should allow for distinguishing between different players.

In addition to the above, the following requirements should be met:

1. These changes should be backwards compatible for GDScript and the Godot editor. I don't think these changes can reasonably be backwards compatible for the underlying C++ methods, unless there's a way to define default arguments I'm not aware of.
2. Games with only a single local player must continue to work as is without requiring any configuration to disable these new features.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

This proposal largely draws on the work @Faless did in godotengine/godot#20091, @reduz's comments in godotengine/godot#20091 (comment), @pouleyKetchoupp's comment in godotengine/godot#29989 (comment), and @redsett's work in godotengine/godot#62421 (along with other comments).

The concept of a "player" layer will be introduced. Implemented as a bitmask, the separate layers identify interactions coming from an independent player. Note that players will usually map 1:1 with devices, but this isn't a hard requirement (consider keyboard sharing or routing both mouse+keyboard and a controller to one player).

The "player" layer will be utilized as follows:

1. InputEvent will get a new player property which is the "player" layer that the event is associated with. This can only be a single layer.

2. ProjectSettings will get new settings which define the mapping from (device type,device index) to a "player" layer. These values will determine which "player" layer an InputEvent gets assigned to.

   NOTE: I suspect using just the device index is the right default value for this mapping: mouse+keyboard, touch input, and the first controller would then all map to the first player, while additional controllers would be assigned to additional players. This is suitable for single player and simple local multiplayer games where the first player might be a "host". However, different mappings can be defined (at runtime!) to better facilitate multiplayer games (e.g. the keyboard might be moved to a new layer if a keyboard user joins the game).

   In addition to the static mapping, players can also construct events on any layer using Input.parse_input_event. To support keyboard sharing, for example, developers could consume events on the keyboard's default "player" layer but then create new InputEventActions on the desired "player" layers using Input.parse_input_event (based on which player the key belongs to).

3. InputMap actions will now implicitly be defined for each "player" layer. This means that developers do not need to configure actions ahead of time or manually duplicate actions; no changes to the InputMap UI are required. Instead, Input will track internally which layers actions were triggered in per-frame (I think we can do this in ActionState?) and developers will provide as an argument the "player" layer(s) they care about when inspecting input action state.

   NOTE: Using specific device IDs for InputEvents added in InputMap could cause actions to be missed if users query for the wrong layer. This won't be a problem in the default case (as we'll see - all layers are queried by default), but it's a potential footgun. I don't think this should influence the design at all, but I want to highlight it. As an aside - I wonder if specifying device IDs in InputMap still has a purpose if this change gets implemented?

4. All Input and InputEvent methods that deal with actions (e.g. Input.is_action_*, InputEvent.is_action_*, Input.action_*, Input.get_action/axis) will all accept a new parameter, player_mask, which is a bitmask of the "player" layers in which to check for input state. This argument will have a default value in GDScript which matches all players and return results consistent with how multiple devices triggering the same action get resolved today; this is how backwards compatibility will be preserved.

   NOTE: @reduz notes in Action Event Layers, multifocus for split screen godot#20091 (comment) that a bitmask argument can be confusing for developers. I think this can be mitigated by providing named constants for the relevant bitmask values, like Input.PLAYER_1, etc. Additionally, due to how backwards compatibility is being designed here, developers will not need to engage with the "player" layer system at all until it's required (when the additional complexity becomes better justified).

   This change will also allow for checking the state of custom, in-game actions without needing to duplicate actions per-player. Character controllers can simply store which player is driving them and use that as an argument when querying action state via Input or InputEvents.

5. "Focus" as a concept within the engine will now exist on "player" layers. This means that each player's focus will be tracked independently. The Control.*_focus methods will be updated with a new parameter, a "player" layer bitmask, that will be used to constrain the set of players that the focus operations apply to. By default this will be all players, preserving backwards compatibility.

   NOTE: Internally, the player_mask argument passed to Control.*_focus methods should also be masked with the Control's own "player" layer bitmask (explained below in 6). Now, Control.*_focus methods will be operating on the intersection of the queried-for player set and the set of players the Control node is configured to handle (both default to all players). I think this will elegantly maintain backwards compatibility while making the semantics of the Control.*_focus methods more consistent.

   For example, if a Button is configured with a "player" layer mask of Input.PLAYER_2, then Button.grab_focus() will only grab player 2's focus (and Button.grab_focus(player_mask=Input.PLAYER_3) would do nothing). For a default Button in a new project, however, Button.grab_focus() will steal focus for all players (matches current behavior).

   This aspect of the solution is likely the most complex to implement. I haven't dug deep into what the exact implementation would look like, but it would likely involve maintaining n separate references to focused Control nodes, where n is the maximum number of players, instead of just 1.

6. Finally, to unlock complex, multi-focus UIs, Control and Viewport nodes will both be updated to include a "player" layer bitmask filter. This bitmask will be used to filter incoming GUI input events, allowing Control nodes to ignore GUI input from players that don't match its setting. By default Control and Viewport nodes will listen on (i.e. accept) input from all players, preserving backwards compatibility. However, developers can modify the bitmask to enable portions of the UI (even within the same Viewport) to only be navigable by specific players. Finally, like mouse events, Control nodes can be set to inherit this value from their parent, all the way up to the Viewport node (Viewport nodes will also use this to filter GUI events).

   NOTE: A Control cannot associate with a "player" layer that its parent does not (my gut feeling is that this is correct, but I'm not sure if it's a strict requirement). This can be implemented by masking a Control's "player" mask with its parent and/or providing configuration warnings in the editor. Additionally, a Control cannot grab focus of a layer it does not associate with; this will be an assertion error and a no-op at runtime.

   I think this change will mainly be implemented within the logic for propagating InputEvents through a scene (is this in Viewport?). Additionally, assuming that Control nodes only receive GUI input events for the configured set of players, then the ui_* action handling within _gui_input methods will work without changes (also assuming the default "all" players mask is used, though we could pass the Control node's "player" bitmask to be safe).

Additional thoughts/commentary

• This solution seems fairly complex to implement, but I think the end result is easy to understand and easy to work with for developers.
• With all due respect to the authors, I believe this approach, where player-specific InputEvents are filtered at the Control level, is a more flexible solution than Viewport-only approaches like Allow multiple SubViewports within a Window to have a focused Control godot#79480 and Introduce Window option to contain multiple Viewports with a focused Control godot#89772.
• I've seen @reduz mention (in Add support for multiple focus layers for Controls godot#62421 (comment) and Action Event Layers, multifocus for split screen godot#20091 (comment)) a use_local_focus property on Viewports that could be used to prevent focus stealing by other Viewports. While this makes sense in the contexts that they were brought up, I feel this will not be required with the solution proposed above. Locking focus within a Viewport is a coarse-grained solution, while developers will now have access to fine-grained solutions via Control-level bitmasks, player-specific InputEvents, and player-specific focus management.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No, there's no way to work around these issues without reimplementing the concept of focus and player-specific actions. Note that player-specific actions can be created for in-game actions, but Control nodes use hard-coded, built-in UI actions that cannot be split amongst players.

Is there a reason why this should be core and not an add-on in the asset library?

This cannot be implemented without changes to the engine. Additionally, allowing Godot's Control nodes to work with local multiplayer is a feature that could benefit a wide variety of games/developers.

[coffeebeats]

Hi all -

I'm very keen on seeing improved support for menu navigation in local multiplayer games. I've looked at a number of issues/PRs related to this topic, and while there are many good ideas/attempts (notably godotengine/godot#29989, #3555, and #4295), I haven't seen a comprehensive proposal yet.

I've outlined above what I feel is a pretty complete solution to the problem. Given the scope of this proposal, I felt it was better to open a new issue rather than post this as a comment in #4295. Please let me know if this should be moved into that issue instead.

Finally, I'm hopeful that this can help drive discussion towards a concrete proposal that can be implemented; feedback/criticism is more than welcome. If this proposal is approved, I would be happy to work on implementing it if no one else with more experience is interested.

[AThousandShips]

See also:

• Resolve device ID conflict between joypads and mouse and keyboard #7161

[Carsonthemonkey]

This proposal seems good. Allowing per-player Control focus is definitely a must. Curious if you can elaborate a bit on what the benefits of the player layer abstraction is over just using direct device IDs. For general player input, developers can just track the device ID per player which seems simpler to me than mapping the player layer at runtime. Admittedly I am not too familiar with how Control focus works, but could it also work on the device id? Better handling of split keyboard is a great idea, but I don't think I understand fully how the player layer allows for this.

[coffeebeats]

Thanks for taking a look! Let me try to better explain the "player layer".

Let's start with the problems we're trying to solve:

1. We need a way to identify which InputEvents correspond to which players in a game.
2. We need a way to filter which players' input is used by various parts of Godot (e.g. Input methods).

First, we cannot use device ID to solve 1. because an InputEvent's device ID does not uniquely identify a player. Devices are indexed separately by device type, meaning the first controller is index 0 and the mouse+keyboard is index 0. As such, we actually want some separate notion of a "player slot"/"player seat" that uniquely identifies a player in the game. This is what this proposal is introducing.

For 2., we want a data structure that lets the developer specify which players' input to consider (including specifying that all players should match). A bitmask is perfect for this use case because (i) it supports the filtering requirements, (ii) is performant, and (iii) Godot already makes use of them so it's conceptually familiar to Godot developers.

Putting these together, the "player layer" is a bitmask (just like the physics layer!) where each bit represents a specific player in the game. Now, Control nodes and Input methods can use that "player layer" bitmask to check whether incoming InputEvents match.

Finally, to make this work seamlessly, Godot needs to know internally how to map from a device type + device ID to a player ID (i.e. one bit of the bitmask), which is why I proposed a new ProjectSettings section to do just that. Then developers can define in configuration things like "player 1" is controller index 0 or "player 2" and "player 3" are both mapped from mouse+keyboard index 0 (it's also possible to change this at run time, of course).

Hopefully that clarifies things somewhat - please let me know if you still have questions.