High DPI support in SDL 3.0 - Take 2

[slouken]

So people keep providing feedback that new high DPI model is hard to wrap their heads around, and adapting existing games and engines is difficult.

I'm thinking about making a change that is closer to what people are used to, but still provides the flexibility to provide high DPI support in a clean way across all supported platforms.

We keep the idea of points, but we separate out the UI scale from the backbuffer pixel size:
Windows: points == pixels, backbuffer and window size are the same, UI scale is the scale shown in the display settings control panel.
macOS, iOS, tvOS: points == points, backbuffer size is an integer multiple of the window size, UI scale is 1.0
Linux (X11): points == pixels, backbuffer and window size are the same, UI scale is the scale shown in the KDE display settings
Linux (Wayland): points == points, backbuffer size is related to the window size by a fixed floating point factor, UI scale is that factor
Android: points == pixels, backbuffer and window size are the same, UI scale is the DPI scale

We keep the existing events and API functions, but windows no longer maintain their size across different DPI displays on platforms that use points == pixels. For the most part people can continue using the API as they have in SDL 2.0, but they have more information to make their application handle UI scaling and different display densities properly.

We could potentially add a mode that converts pixels to UI scaled points, but it would be off by default, and it would be handled at a higher layer in SDL.

@libsdl-org/a-team, @flibitijibibo, @ericwa, thoughts?

[slouken]

To expand on this a bit:
Each display would now have a content scale that is distinct from the window size (in points) and the window backbuffer size (in pixels)

[slouken]

Here is the README-highdpi.md for the new model:

SDL 3.0 has new support for high DPI displays

Displays now have a content display scale.

The display scale is the expected scale for content based on the DPI settings of the display. For example, a 4K display might have a 2.0 (200%) display scale, which means that the user expects UI elements to be twice as big on this display, to aid in readability.

The window size is now distinct from the window pixel size. On most platforms they will be the same, but on Apple platforms the backbuffer will have the full pixel resolution of the underlying display (1x, 2x, 3x, etc.)

The window also has a display scale, which is the content display scale relative to the window pixel size.

For example, a 3840x2160 window displayed at 200% on Windows, and a 1920x1080 window on a 2x display on macOS will both have a pixel size of 3840x2160 and a display scale of 2.0.

You can query the window size using SDL_GetWindowSize(), and when this changes you get an SDL_EVENT_WINDOW_RESIZED event.

You can query the window pixel size using SDL_GetWindowSizeInPixels(), and when this changes you get an SDL_EVENT_WINDOW_PIXEL_SIZE_CHANGED event. You are guaranteed to get a SDL_EVENT_WINDOW_PIXEL_SIZE_CHANGED event when a window is created and resized, and you can use this event to create and resize your graphics context for the window.

You can query the window display scale using SDL_GetWindowDisplayScale(), and when this changes you get an SDL_EVENT_WINDOW_DISPLAY_SCALE_CHANGED event.

[Kontrabant]

Functionally it seems sound and more directly exposes how the underlying platforms work. If I have any criticism, it's that we're back to divergent behavior on different platforms, which could cause headaches for multi-platform development, as, say, someone on Windows wouldn't have any way of testing the paths where the window/pixel size values diverge. Although, if people are finding the existing system too difficult or constrictive to deal with and this simplifies things for them, it's probably worth the tradeoff.

I do like the idea of the separate UI scale value, as previously the X11 global/font scale value wasn't exposed.

[icculus]

If I have any criticism, it's that we're back to divergent behavior on different platforms

I think it's okay to aim for "here's a reasonable way to make it work correctly everywhere without having to stick #ifdefs in your code to handle it."

[AntTheAlchemist]

I've given this a try and it's the most broken and convoluted attempt yet 😢

SDL_DisplayMode::pixel_density is always 1.0. I can't get any scenario where this changes at all, on Windows or Android.

It's impossible to know if SDL_CreateWindow is creating a window based on pixels or screen coordinates, at the moment it seems to use pixels and ignore UI scaling on Windows and Android. I can't see where or how I can influence this.

On Android, I'm getting SDL_EVENT_WINDOW_PIXEL_SIZE_CHANGED events come through on app-return with the Y size the same as the X size, so everything gets stretched.

The only way I can get the actual UI scaling factor was to use SDL_GetDisplayContentScale on the display ID. Every other function just returns 1.0 in every scenario. Nothing seemed to make sense. I tried DPI awareness in the manifest / SDL_WINDOW_HIGH_PIXEL_DENSITY, but nothing seemed to work.

Why can't everything throughout be pixels and only pixels? Don't use screen coordinates anywhere. Give the developer a UI DPI scale, and let the developer scale content themselves.

There are more problems but I had to stop at this point because it's so unworkable. I couldn't work out how to properly set this up in a way that works or makes sense.

[AntTheAlchemist]

I think the biggest confusion is, what's the difference between SDL_EVENT_WINDOW_DISPLAY_SCALE_CHANGED and SDL_EVENT_DISPLAY_CONTENT_SCALE_CHANGED?

[slouken]

Did you see README-highdpi.md?

Conceptually there's no longer any such thing as screen coordinates, everything is in the native window coordinate space. Given that, it's possible to have high density back buffers (on Apple platforms and Wayland) and a UI content scale that the application now needs to handle.

The display content scale changes when the user changes the system display settings. The window display scale changes when either the display content scale or the pixel density changes.

Getting PIXEL_SIZE_CHANGED with Y = X sounds like a bug, I'll look.

[slouken]

Android width/height is fixed in 068d3da. Sorry about that, it would have made things really broken on Android. :)

[slouken]

FWIW, we're talking about adding a mode that automatically applies the content scale like it used to work, but it's a bunch of work and I'm not sure when it'll get done.

[AntTheAlchemist]

@slouken I've read the docs and the code-embedded notes. I still don't understand the difference between content-scale and window pixel-scale. My primary display has a 1.25 scale in Windows, but SDL_GetDesktopDisplayMode's pixel_density is always 1.0; I expected it to be 1.25. The window creation size comes out in pixels, even without SDL_WINDOW_HIGH_PIXEL_DENSITY, which is unexpected. README-highdpi.md mentions "The window has a display scale, which is the scale from the pixel resolution to the desired content size", so the window creation should be affected by the 1.25 content scale, but it's not. I'd expect the window to adopt the display's content scale. I would expect the display's content scale to be in the SDL_DisplayMode struct. I've had to fudge things together to get it to sort of work as expected, so when the Android build broke - that was a face-palm moment 🤦🏼‍♂️. This is only my perspective. I'm happy to put it down to my lack of understanding and wait to see how the wider community respond.

[slouken]

This is a case where you have to forget what you know. Once it clicks for you, please feel free to suggest improvements to the documentation.

The pixel density is the ratio between the number of pixels and the screen coordinates. On Windows, we're always working in pixels and this will be 1:1, or 1.0. On Apple displays this will vary depending on the display and video mode, but could typically be 1x or 2x.

The SDL_WINDOW_HIGH_PIXEL_DENSITY flag is simply a hint about whether you want access to these high density modes, and by default the pixel density will be 1.0.

The content scale is a per-monitor scale that is advisory, SDL doesn't do anything with it internally, but makes it available to the application. It's just passing along the value in the display settings. Both Windows and X11 work in pixels and have a display scale that can be set by users.

In order to get the behavior that you're expecting, you would take the normal size of your window unscaled, multiply it by the content scale, and then use the window display scale to convert between mouse coordinates and pixels, or use the function SDL_ConvertEventToRenderCoordinates() if you're using the SDL 2D rendering API. If you run the SDL test programs you can see this behavior by passing the --auto-scale-content command line parameter.

[slouken]

What you'll notice is that the new model is much closer to what SDL2 provides, and the mental model that developers have had all along, but provides all the information for the application to easily handle high DPI environments.