improve DPI, anti-aliasing, colour management

[totaam]

follow up from #697, #559, #163, #976.

We now have fairly complete data from on the wiki: [DPI] and [DPI Data](../wiki/DPI] and [../wiki/DPI-Data) and [DPI] and [DPI Data](../wiki/DPI] and [../wiki/DPI-Data).

OSX needs doing.

[totaam]

2016-09-22 11:26:29: antoine commented

---

OSX links here:

• pass client DPI preference to server (and use sane defaults of 96dpi) #163#comment:34
• Qt5: Retina display support for Mac OS, iOS and X11 suggests adding NSHighResolutionCapable=true, but I don't think we need this since we use opengl for rendering?
• for testing dpi issues, see How to simulate a retina display (HiDPI mode) in Mac OS X 10.8 Mountain Lion on a non-retina display?
• About High Resolution for OS X
• Type rendering: operating systems has pictures of rendering on different OS with different settings

Somewhat related: Allowing OpenGL applications to utilize the integrated GPU: setting NSSupportsAutomaticGraphicsSwitching=true allows the application to run on the integrated GPU rather than the discrete one (for laptops with dual gpus) - this saves battery life, but since we're having problems with the intel gpus (#1233, especially on osx: #563) we're not going to enable that.

---

• r13815 adds NSHighResolutionCapable=true anyway as this shouldn't do any harm and may help when not using opengl rendering?

• r13816 + r13817 + r13818 add lots of extra display and colourspace information to the "Native_GUI" tool. (which could be used for resizing the shadow server's screen: osx shadow server support for changing resolution #1322)

On my OSX VM, I see a reported DPI of 72x72 for a 1280x1024 screen (but -1 is still shown for DPI and vrefresh using the native gui tool).

[totaam]

2016-09-22 16:11:56: antoine commented

---

A lot of these changes should have been recorded in #1155 - oh well:

• r13820 exposes both global ICC data and per-display ICC data from the client
• r13821 (details in commit message): we set the _ICC_PROFILE X11 root window property
• r13822 now exposes the vertical refresh rate ("vrefresh") for osx clients (doesn't work on my test vm), which may help with things like paint everything together with opengl #792
• r13823 detects workareas on OSX
• r13824 detects the ICC colour profile from the X11 root window property if set

I don't think that OSX has any API for querying the anti-aliasing settings, so that's that.
As for the DPI on OSX, we can continue to rely on the calculated DPI which is based on the display dimensions - these dimensions should be correct and we have no other option anyway.

Still TODO (may go in a separate ticket so we can close this one):

• better multi-display support (lookup and use _ICC_PROFILE_N
• not sure what to improve with DPI.. (anti-aliasing is tracked in handle anti-aliasing and subpixel rendering, honour client component order #559)
• runtime change detection (look for the window property for X11 clients, polling for others.. yuk)

This ticket is now tracking a bit more than just dpi and anti-alias stuff, for lack of a better place...

---

@afarr: things to test:

• the data from NativeGUI_info on OSX should make sense, changes in the colour profile should be reflected in the opaque "icc" binary data blob (shown has hexadecimal)
• X11 clients should have their colour profile forwarded if they have one, ie: xprop -root _ICC_PROFILE should show the exact same value on client and server, more details using "-d screen"
• "vertical-refresh" may now be correct when connected to a real display on OSX
• "workarea" and "workareas" should be detected correctly on OSX and forwarded to the server, it can be verified using xprop -root _NET_WORKAREA - this helps application place their windows in the right place (ie: not overlapping the dock or top menu)

[totaam]

2016-11-16 02:15:25: afarr commented

---

Tested a bit with 1.0 r14155 osx client (pre-dylib errors) alone & with 1.0 r14430 fedora 23 server.

• The data from NativeGUI_info makes some sense, but it looks like it's outputting details for every potential dpi setting for each display (testing with a 2560x1440 display and a "default" 1280x800 laptop retina display which I usually use as "more space" aka 1680x1050)... as well as a number if icc blobs (which don't seem to quite match what I see with xprop on server/session).

• As mentioned above, xprop and NativeGUI_info are outputting in decimal & hex and so they're not clearly matching... and "-d screen" does not seem to be helping to compare.

• "vertical-refresh" seems to be reading as "-1" ... which I assume is not a valid value (I can't recall what we were getting before).

• workarea & workareas seem to correspond to my display set up... and the xprop -root _NET_WORKAREA run server side is exactly what I'd expect.

I'll attach a log of the xprop -root _ICC_PROFILE and the NativeGUI_info for examination...

[totaam]

2016-11-16 02:16:44: afarr uploaded file ticket1086-osx-client-NativeGUI_info.txt (83.1 KiB)

NativeGUI_info for 1.0 r14155

[totaam]

2016-11-16 02:17:22: afarr uploaded file ticket1086-icc-profile.txt (15.3 KiB)

xprop ICC info server side

[totaam]

2016-11-18 14:44:52: antoine changed owner from antoine to afarr

[totaam]

2016-11-18 14:44:52: antoine commented

---

but it looks like it's outputting details for every potential dpi setting for each display
[[BR]]
Not exactly: OSX has the concept of "active", "online" and "main" displays. These sets often contain the same monitors, so they may be listed up to 3 times. And we also lists all the modes supported by each monitor, each time..

---

xprop and NativeGUI_info are outputting in decimal & hex and so they're not clearly matching
[[BR]]
Here's a one liner to convert the xprop output data to hexadecimal:

echo "_ICC_PROFILE(CARDINAL) = 0, 0, 15, 48, 97, 112" | \
    python -c 'import sys,binascii;ints=[int(x.strip()) for x in sys.stdin.read().split("=")[1].split(",") if x.strip()];print(binascii.hexlify(bytearray(ints)))'

You can use it directly list this:

xprop -root _ICC_PROFILE | python -c 'import sys,...'

Or better yet, use r14446 or later server-side and you can just run:

python ./xpra/platform/xposix/gui.py

---

The vrefresh code has not changed, you can get more debug information by running the tool with "-v" or with the XPRA_OSX_DEBUG=1 env var.
-1 means no value.
Let's not worry too much about this one, it isn't used for anything yet.

---

workarea & workareas seem to correspond ..
[[BR]]
They do... but we probably shouldn't be sending negative values to the server as this could mess up window positioning.

This just looks wrong from your attachment:

* workareas                       : [(0, 4, 1680, 1023), (-880, 1050, 2560, 1417)]

What is the arrangement here? Are they really overlapping?

[totaam]

2016-11-19 01:51:39: afarr uploaded file ticket1086_osx-display-arrangement.png (4.4 KiB)

display arrangement

[totaam]

2016-11-19 01:59:09: afarr commented

---

Ok, using the 10.10.5 osx machine to run 1.0 r14448 against a 1.0 r14449 fedora 23 server... I ran python ./xpra/platform/xposix/gui.py and compared with the ICC profile from the NativeGUI_info... and the first 12-15 digits matched & so did the last 23... so I'm going to guess that the ones in between did as well.

As for the display arrangement that generated that workarea... 2560x1440 above a 1680x1050 - like so.

[[Image( ticket1086_osx-display-arrangement.png​)]]

My interpretation was that the lower 1680x1050, as the "primary" monitor, was being designated at 0,4,1680,1023 (presumably the 1050 - 27 for bottom menu or something?) ... and that the 2560x1440 was therefore at (1680 - 2560 = -880), 1050 (top dimension of the 1680x1050?), 2560 (actual width?), 1417 (1440 - 23 ... which isn't the same as the above 27, oddly... but close).

So, yeah... it sort of makes sense. Right?

[totaam]

2016-11-19 17:38:17: antoine commented

---

The visibleframe which we use as "workspace" value is the rectangle is always based on the current user-interface settings and does not include the area currently occupied by the dock and menu bar.
The reason why the values don't add up is because as per Dealing with multiple screens programming, the coordinate system is upside-down and all relative to the "primary screen". Oh joy.

So as of r14462 we use the same calculations as GTK - which should now match the screen geometry which we also get from GTK (in absolute coordinates, with y axis going down from the top-left corner)
I am unable to test this as any secondary screens added via virtualbox aren't detected by maxosx, and I haven't got the mac mini with me right now (I usually do - damn, just when you need it). But with a single screen, the values make sense: a 1280x1024 screen gives: 0, 22, 1280, 911 which leaves 22 pixels for the menu bar at the top and 91 pixels for the dock at the bottom (both look about right).

@afarr: does this give us more sensible values now for your setup as per above?
Side note: please attach the full output of Native_info -v which will include the details of the calculations.
Bonus points for running it on a high-dpi display (aka "retina") since this may show a different value for the backingScaleFactor in verbose mode. (OSX 10.7.x onwards only)
I hope that we're not supposed to use any of the convertRectFromBacking or convertRectToBacking from NSScreen because those don't seem to be accessible via pyobjc (tested on OSX 10.9.x)

[totaam]

2016-11-21 19:35:21: afarr commented

---

Well... I wonder what I'm gonna spend all those bonus points on.

Tested with the 1.0 r14467 client. Ran the NativeGUI_info -v with the same setup as listed above, first with the retina display set at "Default" size:

2016-11-21 11:01:43,007 sending updated screen size to server: 2560x2240 with 1 screens
2016-11-21 11:01:43,007   schadenfreude.local (903x790 mm - DPI: 72x72)
2016-11-21 11:01:43,008     monitor 1 1280x800 at 880x1440 (451x282 mm - DPI: 72x72) workarea: 1280x773 at 880x1463
2016-11-21 11:01:43,008     monitor 2 2560x1440 (903x508 mm - DPI: 72x72) workarea: 2560x1417 at 0x23

And again at "More Space" size:

2016-11-21 11:09:10,434 sending updated screen size to server: 2560x2490 with 1 screens
2016-11-21 11:09:10,435   schadenfreude.local (903x878 mm - DPI: 72x72)
2016-11-21 11:09:10,435     monitor 1 1680x1050 at 880x1440 (592x370 mm - DPI: 72x72) workarea: 1680x1023 at 880x1463
2016-11-21 11:09:10,435     monitor 2 2560x1440 (903x508 mm - DPI: 72x72) workarea: 2560x1417 at 0x23

It does indeed look like the backingScaleFactor=2.0 value is different than the non-retina display (backingScaleFactor=1.0), but the dimensions listed seem to match the approximations that the System Preferences indicate.

I'll attach both the native infos.

[totaam]

2016-11-21 19:36:13: afarr uploaded file ticket1086_native-gui-info_retina-at-default-resolution.txt (85.8 KiB)

NativeGUI_info with retina at "default"

[totaam]

2016-11-21 19:36:56: afarr uploaded file ticket1086_native-gui-info_retina-at-more-space-resolution.txt (85.8 KiB)

NativeGUI_info at More Space resolution

[totaam]

2016-11-22 09:51:53: antoine commented

---

r14469 simplifies the code by making the workareas relative to the screen coordinates they're on (then we don't have to care about the relative coordinates, only adjust for the inverted Y axis)

Let's close this for now before we find something else wrong with it..

[totaam]

2017-02-07 02:21:34: antoine commented

---

As per this chromium ticket: Color mismatch for image displayed in linux machine to OSX machine, chromium should now be loading up the correct colorspace information from the _ICC_PROFILE root window property since this changeset: Support for retreiving the monitor color space on X11.

[totaam]

The workarea code was wrong and was causing some applications to behave strangely (ie: menus would appear lower than they should). Fixed in r15161, 15162 for v1.0.x branch.

[totaam]

2017-03-01 00:30:44: afarr uploaded file ticket1086_triple-display-setup.png (5.9 KiB)

display (triple) set up

[totaam]

2017-03-01 00:31:11: afarr uploaded file ticket1086_NativeGUI_info-osx-2-15170.txt (209.6 KiB)

NativeGUI_info with 2.0 r15170 osx client

[totaam]

2017-03-01 00:37:26: afarr commented

---

Ran the NativeGUI_info again with the 2.0 r15170 osx client (on 10.12.1, in case that makes a difference).

The output looks similar to before, but without negative values.

I went ahead and attached the output... and a screenshot of what the triple display layout looks like.

[[Image( ticket1086_triple-display-setup.png​)]]

Just for comparison sake, connecting to a 2.0 r15189 fedora 25 server, this is the workarea output.

2017-02-28 16:33:01,498  desktop size is 5520x2160 with 1 screen:
2017-02-28 16:33:01,499   schadenfreude.local (1947x762 mm - DPI: 72x72)
2017-02-28 16:33:01,499     monitor 1 1680x1050 at 0x1110 (592x370 mm - DPI: 72x72) workarea: 1680x1023 at 0x23
2017-02-28 16:33:01,499     monitor 2 1600x900 at 80x210 (564x317 mm - DPI: 72x72) workarea: 1600x877 at 0x23
2017-02-28 16:33:01,499     monitor 3 3840x2160 at 1680x0 (1354x762 mm - DPI: 72x72) workarea: 3840x2137 at 0x23
2017-02-28 16:33:01,500  upscaled by 125%, virtual screen size: 4416x1728
2017-02-28 16:33:01,500   schadenfreude.local (1947x762 mm - DPI: 57x57)
2017-02-28 16:33:01,501     monitor 1 1344x840 at 0x888 (592x370 mm - DPI: 57x57) workarea: 1344x818 at 0x18
2017-02-28 16:33:01,501     monitor 2 1280x720 at 64x168 (564x317 mm - DPI: 57x57) workarea: 1280x702 at 0x18
2017-02-28 16:33:01,501     monitor 3 3072x1728 at 1344x0 (1354x762 mm - DPI: 57x57) workarea: 3072x1710 at 0x18

[totaam]

FYI: the workarea code is pretty much useless with multiple monitors because we have a single rectangle value which is supposed to cover all monitors... (see _NET_WORKAREA and multiple monitors), and so in this case we don't set it at all.

[totaam]

Apple killed off subpixel antialiasing in 10.14 and called it a “refinement”. Not sure they know what that word means