Should spec a buffering/refresh interval model of gamepad data?

[juj]

Noticed that there has already been quite a bit of discussion of polling vs events model for the Gamepad API. The argument in favor of events over polling has been stated that polling can lose button presses (if they fall in between the polled snapshots) and events can not. I think this is not completely accurate, and being able to lose/miss button presses is somewhat orthogonal issue to events vs polling API.

It would be possible for a browser implementation not to lose any button presses if it accumulates those up internally from OS and buffers them to be returned on the next call to getGamepads(). From the spec viewpoint, it is unclear if this is mandated or prohibited. Even in the absence of browser buffering gamepad button events, there is a question about the granularity of when new live data is allowed to come in.

a) Are conforming implementations of the Gamepad API allowed to miss physical button presses? Or is that considered an implementation bug?

b) Are Gamepad API implementations allowed to buffer up button presses (it gets as events from OS) and return them on the next polled call to navigator.getGamepads() as to not miss any button presses that might otherwise fall in between the polls?

c) Consider the following code:

function gameProcessUserInput() {
    var pads = navigator.getGamepads();
    for(var i = 0; i < pads.length; ++i) {
        var state = pads[i];
        if (!state) continue;
        for(var j = 0; j < state.axes.length) console.log(state.axes[j]);
        for(var j = 0; j < state.buttons.length) console.log(state.buttons[j].value);
    }
}

versus the following code:

function numGamepads() { return navigator.getGamepads().length; }
function getGamepad(i) { return navigator.getGamepads()[i]; }

function gameProcessUserInput() {
    var n = numGamepads();
    for(var i = 0; i < n; ++i) {
        var state = getGamepad(i);
        if (!state) continue;
        for(var j = 0; j < state.axes.length) console.log(state.axes[j]);
        for(var j = 0; j < state.buttons.length) console.log(state.buttons[j].value);
    }
}

In a conforming implementation, are these two functions required to print out the exact same output? Or is it expected that the outputs can differ here? That is, does each call to navigator.getGamepads() (even in subsequent calls in a single function) possibly return new live data? Or is the data provided by the browser allowed to change only e.g. at each turn of requestAnimationFrame() or setTimeout() or setInterval()?

Solving the issue via an event API might be one try, although like it was noted, sending events for analog axes, which often have noise based jitter, might lead to a large constant volume of events being sent. Also games would often need to implement their own state registry buffering of these events in order to check if user pressed button X alone, or Left Trigger + button X, or what the exact value of an analog axis was at the time of a press.

It would be possible to fix the missing button presses trouble in the polling approach by specifying when fresh button state data is allowed to come in, e.g. at each rAF() turn or similar (after exiting any event handler in which getGamepads() was called(?)), which would both allow browsers to buffer up presses for the next poll, and make the above code samples identical.

What are the latest thoughts on this front?

[balazs]

Let me just explain what I believe to know about the problem without answering your questions.

In chromium, as far as I understand, if you process gamepad data in a requestAnimationFrame loop, you will be able to catch any button presses that are clearly distinguishable from jitter. We poll gampads from the OS at 60Hz and we produce a snapshot of the actual state - as provided by the OS, which might already include some buffering and/or delay - which is returned in getGamepads(). The actual timing difference between the poll and the time when requestAnimationFrame fires is arbitrary, but in general if a button was pressed after the last rAF fire and it was hold for at least around 16ms you will see it in the next rAF via getGamepads(). Now as far as I know, the thing about humans pressing buttons is that in practice you always hold it for at least 16ms, thus the theoretical issue with button presses quicker than that is not relevant. Of course all these thinking assumes that our polling is happening at 60Hz no matter what and the same about firing rAF which might always be the case (but we should be close to that for quality gaming experience anyway).

With that in mind, I guess my question to you is: do you know any practical examples where lost gamepad events are actually a real problem?

Irrespective of whether we miss events in practice, the spec should be explicit about the model - such as whether getGamepad() is returning a live object or a snapshot. In case the snapshot model is chosen, it probably should have some text about the frequency of polling and about exactly how and when can you observe the physical gamepad events. So, the problem is definitely not solved, however I would say that it's important to avoid solving theoretical issues instead of the real ones, so first we should clarify whether the phenomenon of missing events is a real problem with the current spec.

[juj]

Thanks for the great comments! I agree that any "superhuman" button presses that only occur for a single digit millisecond duration are of not concern. Not sure whether the OS itself would be registering these either.

Although the "all these thinking assumes that our polling is happening at 60Hz no matter what and the same about firing rAF which might always be the case (but we should be close to that for quality gaming experience anyway)." thought could be read a bit in a all bets are off type of scenario for the Gamepad API if one doesn't have 60fps. I think it's rather the hiccup/GC stutter cases and slow frame rates exactly that are the important consideration. We would not want to miss button presses in the event that for example the JS garbage collector decided to take a 500msec pause in a critical moment in a game.

I'm leaning to thinking that perhaps Gamepad API should add an event model which ensures best possible promise about no lost events. With this, if the OS buffers up gamepad events, then these could be naturally delivered. And .getGamepads() could always return the most recent data that browser has, which would be good for sampling analog signals when only most recent signal value is of interest. (e.g. racing car throttle position).

I would dislike having a snapshot model spec which would go and specify actual millisecond intervals when polling occurs or anything like that. That would feel brittle.

On the topic of theoretical vs concrete, and the superhuman question aside, the question of whether the second code snippet above has a danger of losing presses is a concrete issue. This is more rather a question of whether a browser implementation might be using a separate thread to receive gamepad api messages, and the polled data could change mid-flight when JavaScript code is executing. That could make the two code snippets behave differently. Does the spec (intend to) allow that?

[balazs]

See inline.

On Thu, May 5, 2016 at 1:42 PM, juj [email protected] wrote:

Thanks for the great comments! I agree that any "superhuman" button
presses that only occur for a single digit millisecond duration are of not
concern. Not sure whether the OS itself would be registering these either.

Although the "all these thinking assumes that our polling is happening at
60Hz no matter what and the same about firing rAF which might always be the
case (but we should be close to that for quality gaming experience
anyway)." thought could be read a bit in a all bets are off type of
scenario for the Gamepad API if one doesn't have 60fps. I think it's rather
the hiccup/GC stutter cases and slow frame rates exactly that are the
important consideration. We would not want to miss button presses in the
event that for example the JS garbage collector decided to take a 500msec
pause in a critical moment in a game.

​It's not clear to me that you really better off processing those events
after the fact. In case of missed frames users typically keep pushing the
button. Like you are in a racing game pushing the gas, nothing happens, you
will keep pushing it. Now if the game buffers it up it might get even
worse, you end up accelerating too much so you lose control and start
yelling at your browser. :) I don't have a lot of game development
experience so I might not take all details into account here but my gut
feeling is that if you miss a frame it's better to ignore the input from
that frame too.
​

I'm leaning to thinking that perhaps Gamepad API should add an event model
which ensures best possible promise about no lost events. With this, if the
OS buffers up gamepad events, then these could be naturally delivered. And
.getGamepads() could always return the most recent data that browser has,
which would be good for sampling analog signals when only most recent
signal value is of interest. (e.g. racing car throttle position).

I would dislike having a snapshot model spec which would go and specify
actual millisecond intervals when polling occurs or anything like that.
That would feel brittle.

On the topic of theoretical vs concrete, and the superhuman question
aside, the question of whether the second code snippet above has a danger
of losing presses is a concrete issue. This is more rather a question of
whether a browser implementation might be using a separate thread to
receive gamepad api messages, and the polled data could change mid-flight
when JavaScript code is executing. That could make the two code snippets
behave differently. Does the spec (intend to) allow that?

​With the chromium implementation it's definitely not changing on the fly.
If the browser have a polling thread - which chromium has - than it has to
take care of synchronization, I'm sure the spec doesn't intend to allow the
luck of that. Also I don't think the spec has to be explicit on this, it's
beyond the scope of a spec to define every possible implementation pitfalls.

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#22 (comment)

[luser]

I think we'd be better off spec'ing events as the solution here, and also spec'ing #8 to clarify when data updates are visible to content.

[marcoscaceres]

Should be covered by #152 ... at least, the proposal should address the above.

[razvanphp]

In chromium [...] We poll gampads from the OS at 60Hz and we produce a snapshot of the actual state - as provided by the OS, which might already include some buffering and/or delay - which is returned in getGamepads().

Just for posterity, this is not true anymore, see #46 (comment)

Chrome currently polls at 250 Hz (4ms polling interval):

https://source.chromium.org/chromium/chromium/src/+/main:device/gamepad/gamepad_provider.cc;l=100;drc=d024ced704c7a375bce2d9d6654ae4807bfe934b