"Send text" user intent

[jugglinmike]

Due to concerns expressed by potential implementers, we're interested in structuring AT Driver such that screen readers can be in conformance without allowing the simulation of keyboard key presses.

Today, the "pressKeys" command is the only means to simulate user input, so it takes a bit of imagination to consider how an implementation would enable screen reader automation without it. Such screen readers could provide all of their meaningful functionality through high-level commands (or perhaps intents, should that refactoring land). For instance, there might be one command for "read next item" and another for "navigate to next heading" and so on. To the extent that raw keyboard access poses a security risk, this stable of high-level commands might be called "safe."

We're not prepared to propose standard APIs for those high-level commands (though we expect implementation experience will inform future extensions for the specification). Even so, we'd like to support implementations which choose to implement many high-level commands instead of the general-purpose pressKeys command. Specifically, we think that regardless of the design of the safe commands, there will definitely be a need to enter text into form fields. For this reason, we believe it's prudent to standardize a command for sending text.

As a command, sendText could be limited in a way that mitigates the security risks behind pressKeys. It could only be used to send sequences of printable characters (no modifier keys allowed), and it might even be limited in terms of its target (only UI elements designed for text entry). An implementation which exposed this command would not be granting users total control over the host machine; it could only be used to fill data in form fields.

...or so our thinking goes! I've opened this issue to collect input from the experts who understand their security models better than we in the ARIA-AT community group. So, what do folks think? Is this a viable step toward a more complete automation protocol?

[jugglinmike]

This issue was discussed at the 2024-09-26 meeting of the Browser Testing and Tools Working Group. Minutes of that discussion are available here.

[jugglinmike]

This issue was discussed at the 2024-10-07 meeting of the "Automation" Subgroup of the ARIA-AT Community Group. Minutes of that discussion are available here.

[jugglinmike]

Superseded by gh-81