SDL_MouseMotionEvent with SDL_WINDOW_ALLOW_HIGHDPI flag

[SDLBugzilla]

This bug report was migrated from our old Bugzilla tracker.

Reported in version: 2.0.8
Reported for operating system, platform: All, x86

Comments on the original bug report:

On 2018-10-01 14:18:37 +0000, SKOMAND wrote:

Hi!

Similar problem with mouse coordinates on high DPI was described in detail already:
https://forums.libsdl.org/viewtopic.php?p=43373

SDL_MouseMotionEvent is generated when logical coordinates are changed only, i.e. it's impossible to process an event of moving the mouse less than one logical pixel aside, which may correspond to several physical pixel on device screen.

E.g. on 4k display with screen scaling to 1920x1080 no event is generated if you move the mouse less than 2 physical pixels aside.

The solution may be in adding float rawX, rawY coordinates to SDL_MouseMotionEvent (or changing the type of X, Y coordinates from Sint32 to float) and generating this event when physical coordinates are changed.

Thus, moving the mouse one physical pixel aside will generate SDL_MouseMotionEvent with corresponding coordinate value = 0.5f for example mentioned above.

Thanks!

On 2019-02-12 11:43:31 +0000, ®om wrote:

I have a possibly related issue.

On MacOS, on secondary screen with hidpi enabled, click events location are not consistent with the logical size of the renderer, leading to incorrect click location: Genymobile/scrcpy#15

Someone proposed a workaround in my app: Genymobile/scrcpy#245 but I think the root cause is in SDL.

Still happens in SDL 2.0.9.

Is it the same problem?

[ericwa]

Is it possible, in terms of ABI breakage, to add fields to SDL_MouseMotionEvent?
I see that d5ec735 added a field to SDL_TouchFingerEvent and the discussion at the bottom of #3037 (comment) suggests it's OK.

Given SDL_Event is 56 bytes =14 x 4-byte values, and SDL_MouseMotionEvent currently has 9, there's room for 5 more 4-byte values. e.g. this would fit:

    float xFloat;           /**< X coordinate, relative to window.
                                 Can be non-integer if the window is on a high DPI monitor with scaling. */
    float yFloat;           /**< Y coordinate, relative to window */
    float xrelFloat;        /**< The relative motion in the X direction */
    float yrelFloat;        /**< The relative motion in the Y direction */
} SDL_MouseMotionEvent;

Likewise for SDL_MouseButtonEvent, it could get:

    float xFloat;           /**< X coordinate, relative to window */
    float yFloat;           /**< Y coordinate, relative to window */
} SDL_MouseButtonEvent;

[stianhoiland]

Any progress on this?

[ericwa]

I actually did a quick prototype of this last year, for Wayland + macOS: https://github.com/ericwa/SDL/commits/float-mouse-coords
Not sure if I got all of the rounding details correct for the legacy codepaths, but at least it's a proof of concept.

[slime73]

Now that development on SDL3 has begun...

I think there are two options here (assuming we stick with using DPI-scaled units, which I believe we tentatively decided to do): change all mouse APIs so coordinate values are in floats, or add additional APIs for precise float coordinates alongside the current integer APIs.

I'm a fan of keeping things simple and having just a single set of APIs, but I can also imagine the mouse APIs are frequently used together with things like the window size and position APIs which don't really have good reason to have high precision float values, so there might be a lot of int <-> float conversions in people's code if mouse APIs only use floats for position.

If the extra variant approach is taken, using a Precise suffix or prefix for those names makes sense to me - something similar is done for mouse wheel events already.

Here's a list of all the existing position-related mouse APIs that will need to change or have Precise variants:

// In SDL_MouseMotionEvent:
Sint32 x
Sint32 y;
Sint32 xrel;
Sint32 yrel;

// In SDL_MouseButtonEvent:
Sint32 x;
Sint32 y;

// In SDL_MouseWheelEvent:
Sint32 mouseX;
Sint32 mouseY;

Uint32 SDL_GetMouseState(int *x, int *y);
Uint32 SDL_GetGlobalMouseState(int *x, int *y);
Uint32 SDL_GetRelativeMouseState(int *x, int *y);
void SDL_WarpMouseInWindow(SDL_Window * window, int x, int y);
int SDL_WarpMouseGlobal(int x, int y);

[1bsyl]

just a thought, not sure if this has been said: currently touch event as reported to a window (and also within the window, i think).
but:

• you can touch outside the window.
• the window can be also across two screens!

So it seems to me it should be linked to a VideoDisplay instead, (but we only have ids for them), so maybe linked to the SDL_DisplayMode of the VideoDisplay.
(probably also for mouse event too etc.)

[slime73]

Direct touch event coordinates are relative to the window though rather than a screen, right?

[1bsyl]

 44 typedef enum
 45 {
 46     SDL_TOUCH_DEVICE_INVALID = -1,
 47     SDL_TOUCH_DEVICE_DIRECT,            /* touch screen with window-relative coordinates */
 48     SDL_TOUCH_DEVICE_INDIRECT_ABSOLUTE, /* trackpad with absolute device coordinates */
 49     SDL_TOUCH_DEVICE_INDIRECT_RELATIVE  /* trackpad with screen cursor-relative coordinates */
 50 } SDL_TouchDeviceType;

[1bsyl]

it is yes

[slouken]

FYI, I'm working on this now.