hidpi awareness for macos

[totaam]

See Is HiDPI possible?

See also Automatically scale windows? - similar problem for wayland vs x11

Optimizing for High Resolution

[totaam]

It might be possible to paint at higher resolution through opengl.
In which case the fix would simply be to scale window positions and dimensions when dealing with GTK, but keep the opengl window backing at the "native" resolution.

[totaam]

It may be possible to enable HIDPI mode for testing - though this may require newer versions of macos:

• How to Enable HiDPI Mode in macOS
• hidpi_osx: Enable HiDPI mode on OS X

The API to call with opengl seems to be Optimizing OpenGL for High Resolution: setWantsBestResolutionOpenGLSurface.
The main difficulty with this is that we would need to use double screen dimensions (or whatever ratio is used) when connecting to the server but then downscale the coordinates for all window operations except for opengl painting.

Another difficulty is to do with how we obtain the opengl context: we initialize it without the window (which may be the cause of some problems: r23441 / #2372), so by the time we have the NSView object it may be too late to call setWantsBestResolutionOpenGLSurface. Not sure how we would go about selecting pixel format attributes if we did it that way - not that we currently support high bit depth anyway..

Example code: PyOpenGL on a Macbook retina display

[totaam]

See also Xpra-org/xpra-html5#13, #2500.

[totaam]

2020-09-24 16:40:38: antoine uploaded file osx-highdpi.png (568.2 KiB)

problem with best resolution

[totaam]

Received by email:

--- src/xpra/platform/darwin/gl_context.py (revision 27534)
+++ src/xpra/platform/darwin/gl_context.py (working copy)
@@ -150,6 +150,7 @@
         if not self.window_context:
             self.nsview_ptr = nsview_ptr
             nsview = objc.objc_object(c_void_p=nsview_ptr)
+            nsview.setWantsBestResolutionOpenGLSurface_(True)
             log("get_paint_context(%s) nsview(%#x)=%s", gdk_window, nsview_ptr, nsview)
             if self.alpha and enable_transparency:
                 self.gl_context.setValues_forParameter_([0], NSOpenGLCPSurfaceOpacity)

I managed to enable proper hidpi rendering resolution. However the scale is off and the view is too low. I tried to correct it by changing glViewport and moving the view with glTranslatef, but the view is clamped to the bottom left corner.
[[Image(osx-highdpi.png)]]

It's not clear to me what type of session it is connected to.
But I guess that we need to adjust the scaling to match the actual window dpi.

[totaam]

2020-09-24 20:15:04: hawski commented

---

Hi,

That's my report above in #1933#comment:5. I'm not sure what you mean by type of session, but it is MacOS Mojave shadow to Void Linux running Xfce.

I think what would be right is for the window to be smaller.

[totaam]

2020-09-24 21:24:55: hawski commented

---

Oh, indeed it's a scaling issue. The window is just bigger then my screen, so it's not offset. It's probably more on to the view or the window, however, because changing the viewport does not affect this.

[totaam]

Oh, indeed it's a scaling issue

We need to know the actual multiplier to use, preferably before creating the window.
Any idea how to do that?

[totaam]

2020-09-25 11:43:13: hawski commented

---

The common way, that is not exactly correct, but still an improvement, would be to use: [[NSScreen mainScreen] backingScaleFactor]

The problem is, that one can have multiple screens attached with varying scale factor.

This article suggest reacting to the changes: http://supermegaultragroovy.com/2012/10/24/coding-for-high-resolution-on-os-x-read-this/ but I don't know if it is applicable to Xpra, as it assumes layer-backed and/or layer-hosting view. I don't know if it makes sense.

I also wonder how newer GDK handles it as at least on the master branch it calls setWantsBestResolutionOpenGLSurface in its internals: https://gitlab.gnome.org/GNOME/gtk/-/blob/master/gdk/macos/gdkmacosglcontext.c#L121 This file got merged two months ago, so probably it's not relevant to Xpra as it is probably part of GTK4.

[totaam]

This article suggest reacting to the changes

That makes sense, but this may well require changes to the window geometry to adapt to the new scaling factor. And that's known to cause problems (resizing loops, etc).
I don't have the hardware to test either.
(so don't hold your breadth - this could take time)

.. probably it's not relevant to Xpra as it is probably part of GTK4

Correct. We can't use GTK for opengl anyway, as this would require a rewrite of the opengl window rendering code...

[totaam]

2020-09-30 11:45:37: hawski commented

---

I scaled 2x render_size in init from xpra/client/gl/gl_window_backing_base.py Then it looks good on single display. I will check later how it works with backingScaleFactor, but it's good enough for me now.

I have massive lag on local network, but it is unrelated to my changes as far as I see, because the same lag is noticeable with --opengl=no.

[mbac]

2020-09-30 11:45:37: hawski commented

I scaled 2x render_size in init from xpra/client/gl/gl_window_backing_base.py Then it looks good on single display. I will check later how it works with backingScaleFactor, but it's good enough for me now.
I have massive lag on local network, but it is unrelated to my changes as far as I see, because the same lag is noticeable with –opengl=no.

Hi,

Could you explain exactly what is meant by “scaling 2x render_size”? I’d like to try this workaround…

[totaam]

@mbac hawski never replied and isn't subscribed to this ticket.
He never shared his tiny change and I don't have the hardware to validate any changes I could make.

[hadrianw]

Oh, hello. I'm here. I manually edited /Applications/Xpra.app//Contents/Resources/lib/python3.8/xpra/client/gl/gl_window_backing_base.py

--- /Applications/Xpra.app/Contents/Resources/lib/python3.8/xpra/client/gl/gl_window_backing_base.py
+++ /Applications/Xpra.app/Contents/Resources/lib/python3.8/xpra/client/gl/gl_window_backing_base.py	2020-09-29 13:27:52.000000000 +0200
@@ -325,7 +325,7 @@
     def init(self, ww : int, wh : int, bw : int, bh : int):
         #re-init gl projection with new dimensions
         #(see gl_init)
-        self.render_size = ww, wh
+        self.render_size = ww * 2, wh * 2
         if self.size!=(bw, bh):
             self.gl_setup = False
             oldw, oldh = self.size

I still use this workaround and did not work further on improving the change.

[totaam]

This change will upscale the window, but this will not take advantage of the extra pixel density available.
For that to work, we would need to also upscale the screen dimensions sent to the server - but not the DPI!
Then, I believe that instead of doubling the render_size, the client should halve the window size.
I need to find a way to test this.

[hadrianw]

I had quite a dirty Xpra.app after my debugging. I diffed python sources against Xpra 4.0.4 source and forgot to mention that the previous patch is also crucial for it to work. So overall:

1. I spawn Xpra like that:

xpra shadow --printing=no --opengl=yes --desktop-scaling=2/3 \
    --swap-keys=no ssh:[email protected]:0

Without my changes I get this:

3. Then I modify things with this patch (I don't know if it is clean, but changes are small enough to do this manually):

--- xpra-4.0.4/xpra/client/gl/gl_window_backing_base.py	2020-05-10 19:00:53.000000000 +0200
+++ /Applications/Xpra.app/Contents/Resources/lib/python3.8/xpra/client/gl/gl_window_backing_base.py	2021-06-07 17:46:33.000000000 +0200
@@ -325,7 +325,7 @@
     def init(self, ww : int, wh : int, bw : int, bh : int):
         #re-init gl projection with new dimensions
         #(see gl_init)
-        self.render_size = ww, wh
+        self.render_size = ww * 2, wh * 2
         if self.size!=(bw, bh):
             self.gl_setup = False
             oldw, oldh = self.size
--- xpra-4.0.4/xpra/platform/darwin/gl_context.py	2020-05-10 19:00:55.000000000 +0200
+++ /Applications/Xpra.app/Contents/Resources/lib/python3.8/xpra/platform/darwin/gl_context.py	2021-06-07 17:38:23.000000000 +0200
@@ -150,6 +150,7 @@
         if not self.window_context:
             self.nsview_ptr = nsview_ptr
             nsview = objc.objc_object(c_void_p=nsview_ptr)
+            nsview.setWantsBestResolutionOpenGLSurface_(True)
             log("get_paint_context(%s) nsview(%#x)=%s", gdk_window, nsview_ptr, nsview)
             if self.alpha and enable_transparency:
                 self.gl_context.setValues_forParameter_([0], NSOpenGLCPSurfaceOpacity)

AFAIR the value of 2 could be taken from [[NSScreen mainScreen] backingScaleFactor], but I don't know what should be done in case of multiple screens of differing scale factors. Like retina on a MacBook and 1080p external monitor.

4. I spawn it the same as in point 1. That's the result:

Then, I believe that instead of doubling the render_size, the client should halve the window size.
I need to find a way to test this.

Tell me what to modify and I'll check. I lost my way around the code.

[totaam]

Ah, now I see!
You're downscaling the server side screen by using --desktop-scaling=2/3 and that's why your change actually does work! It can only work if the server is sending a higher resolution than that of the client's window.

And starting with xpra 4.2 servers, this won't work any more because the server will downscale the image to save bandwidth - since it is not aware that the client really will make use of the extra pixels, the details are in #2052.
The commit above now allows you to switch off this feature with XPRA_DOWNSCALE=0. (server-side switch)

Tell me what to modify and I'll check. I lost my way around the code

Well, for your use-case, there isn't a lot more needed because the window has a fixed size and there is no DPI synchronization to do in shadow mode.
(obviously, the code would need to be cleaned up a bit to get the real scaling values)

The difficulty is to make it work for seamless mode. Simply using the extra pixels by default would make all remote windows shrink by 50%, making them hard to read.
Increasing the DPI accordingly doesn't work very well as the application support is spotty at best. (and most applications need to be started after the new DPI is applied or they don't take it into account)
Many applications would still look too small. Others would be a mishmash of small and large widgets.
The best way forward may be to expose this scaling factor to the server side so that it can decide what to do. For example: using the value when making any scaling adjustments, like disabling the automatic downscaling that was added in 4.2.

[jaeyeun97]

Any updates on this? I'm also getting 36x31 dpi downscaling when I access my linux Xpra server from a MacOS client.

[totaam]

@jaeyeun97 I think that the warning you are talking about has nothing to do with hidpi awareness, see https://github.com/Xpra-org/xpra/blob/master/docs/Usage/Xdummy.md

[vkz]

Hi all. Not exactly clear how to get that magical HIDPI on the Mac. Text looks rugged and far from crisp or even decent.

Could someone post a step by step solution? Way I understood it from previous conversation:

• you start server side with --desktop-scaling=2/3
• apply patch to xpra/client/gl/gl_window_backing_base.py
• apply patch to xpra/platform/darwin/gl_context.py
  But then @totaam mentions it won't work past 4.2 without XPRA_DOWNSCALE=0, which if I read Python code correctly means setting shell env variable.

Or am I missing some recent option or flag or setting that I can use to avoid the above dance?
Thank you

[totaam]

@vkz :

• this only works for xpra shadow
• --desktop-scaling=2/3 is a client side switch
• XPRA_DOWNSCALE=0 is a server side switch

There has been no progress on this.
To make this work properly, I would need hardware to test and time to implement.

[vkz]

ah, I see. So, to clarify, as of now there is no way to get HIDPI on Mac OSX clients?
Not even talking retina here. I've been trying it on the external Samsung U32'' which can do 3840x2160 (but I use scaled to look like 1920x1080 else its too tiny).

Is this specific to XPRA implementation? As in, there's a chance one could get crisper image going with X server + Xquartz client. Of course we'd be missing on all the nice things XPRA gives us like clipboard sharing and persistent sessions etc.

[totaam]

So, to clarify, as of now there is no way to get HIDPI on Mac OSX clients?

Only with xpra shadow and the steps above.

Is this specific to XPRA implementation?

Yes.

As in, there's a chance one could get crisper image going with X server + Xquartz client

Perhaps, I have no idea.

[vkz]

ok, I feel stupid but it only just occurred to me that HIDPI you speak of probably refers to Mac Retina displays which I believe >220ppi, which is not what I'm talking about. My Mac indeed has retina, but I'm running XPRA client on the external 4K display 3840 x 2160 which at 32inch diagonal translates to something like 142ppi. If I made a mistake and butted in the wrong convo, I apologies. Should I move my questions into a separate issue?

[totaam]

Should I move my questions into a separate issue?

@vkz Probably, it doesn't hurt to open a new issue with your details in it. Please see https://github.com/Xpra-org/xpra/wiki/Reporting-Bugs

[totaam]

See #4017 (comment)

[totaam]

Perhaps we can deal with this as part of #2467
Some links:

• wantsBestResolutionOpenGLSurface
• PyOpenGL on a Macbook retina display

[totaam]

Is this still an issue, I would have thought that 854f012 fixed this.

[totaam]

Probably solved by #4167 (comment)

[ShadowJonathan]

Can confirm this has fixed my (direct) issues, i'll have to use the desktop client more to see if there are any more bugs, but thank you for fixing this :)