Half and quadrant block elements are not aligned on the same grid

[PhMajerus]

Cascadia family version

2111.001

Cascadia family variant(s)

Cascadia Mono (the version without ligatures)

Font file format(s)

Windows Terminal included version (TTF (variable))

Platform

Windows 11 build 22621.382

Other Software

Windows Terminal 1.15.2003.0, but identical problem in the Windows Console (conhost).

What happened?

The block elements characters are supposed to be able to mix together to be used as pseudo-graphics.
The █ character fills the whole cell, the common ▀ and ▄ characters are half blocks, and ▌and▐ are left and right halves, but the less common quadrants should be aligned on the same grid, the whole set of 16 characters (with space or nbsp) should then provide all possible combinations of the 2×2 grid:  ▖▘▌▗▄▚▙▝▞▀▛▐▟▜█.

Unfortunately, the alignment grid isn't the same for all those characters.

"\u2580\u2584\u2596\u2597\u2598\u2599\u259a\u259b\u259c\u259d\u259e\u259f"

The horizontal line splitting upper and lower halves should be identical for all those characters, basically always using the same 2×2 grid with each quadrant on or off like pixels, ideally splitting them evenly (perfectly centered).

For reference, here is the same sequence of characters using the UNSCII font (http://viznut.fi/unscii/):

Note some characters of this set are also shared to form the set of squot/sextant characters (2×3 grid), with U+1FB00..U+1FB3B of the Symbols for Legacy Computing. See #597 and #607.
It would make sense to rebuild them all as a single coherent set, all based on divisions of the reference █ full block character.

[PhMajerus]

@DHowett I'm considering trying to quickly fix these, to have the half blocks and quadrants behave as proper mosaics, and make them align properly with octants, as some are reused as part of the octants set and currently don't align perfectly.

I didn't want to include that in another PR to avoid mixing new glyphs and modifications to existing glyphs together.
Would you still be open to review about 12 glyphs adjustments before closing this release?

It would be some very small changes, just snapping their points to the unified grids. This would give a consistent set of mosaics for the people working on Terminal renderer to avoid them investigating or fixing alignments errors in the coming months that are actually caused by small misalignments in the font itself, and mean users hopefully will not experience misalignments with the new set of pseudopixels.

I think I'll go ahead anyway in my personal repo copy so I can at least use that as a reference later, but if you are open to one more tiny PR, this would ensure we have a consistent set of mosaics for the coming months, as I understand Cascadia Next/Refresh will force quite a gap between releases.

[DHowett]

Go for it! The doors close tomorrow, and I haven’t actually shared the final release binaries for 2404.23 with anyone ;)

[PhMajerus]

I'm somewhat perplexed, and probably need some help, especially with the short deadline to get it merged.
I adjusted the coordinates of half-blocks, quadrants, and a few others, but while VTT shows the new alignments, it seems to have no effect in Windows Terminal...

But if I build the NF variant and select that one in Terminal, then the new versions are used.
However, even with the NF version, they are not aligned with the octants that use the same grid coordinates, as if the coordinates of older quadrants and eights were stretched vertically by a bit, making them taller and then cropping the top of those characters.

To be clear, the overlap of the two different y values for the middle horizontal lines seen in the first screenshot above is fixed, but the characters seem to be stretched upward, making their middle horizontal separation higher than the same y value in characters I added such as the octants.

Do you know what's going on? Is there some existing hinting that makes those characters render differently, or some special handling in Terminal for existing block characters?

The version with updated coordinates is available on my repo: https://github.com/PhMajerus/cascadia-code

[DHowett]

Could it be the rclt ones? No idea other than that...

[PhMajerus]

@DHowett
I only changed the coordinates in the glif files and assumed it would follow the same substitution system as they were already in rclt.
But you're on to something with the rclt.
I applied different changes to "normal" and stypo versions of one of the original quadrants upperRightAndLowerLeftBlock (U+259E), and to one of my octants from the last PR blockOctant-1 (U+1CEA8), and there is something wrong.

Windows Terminal shows the normal version of upperRightAndLowerLeftBlock and the stypo version of blockOctant-1.
This seems wrong to me, because based on the full-block coordinates, the normal version of of upperRightAndLowerLeftBlock doesn't seem to properly fit the cell boundaries.

Visual Studio editor, VS Code, conhost, and MS Word all use the stypo versions of both characters.
And even more weird, conhost seems to sometimes use the normal (non-stypo) version:

The two character with red underline are the same, but the one with the up slope in the command is the normal version, while the one with the down slope in the error message is the stypo version.

Can you clarify which version normal or stypo is expected to be used by Windows Terminal? I was under the impression that it was DWrite-based and that would use the stypo version? However, even the GDI API CreateFontIndirect seems to use stypo versions. Did I mix them up?

I'm even more puzzled about Windows Terminal using the non-stypo upperRightAndLowerLeftBlock and the stypo blockOctant-1.

Could there, be by any chance, a special handling of the classic block elements characters that could explain why they don't behave the same? Like some leftover code from when you were trying to make the characters that were supposed to fill the cell stretch to do so, and which uses a different code path, maybe not using DWrite to render these?

More evidence of something being wrong

Experimenting with Terminal Canary 1.21.240424002-llm, it seems to confirm my suspicions:

The ▞ with the slope is (U+259E), the direction of the slope tell me it is the non-stypo version, and it seems wrong as it extends above its cell, as can be seen on the line where the selection shows the cells height. The stypo version would properly fit the cell, as the top y would be 1900 instead of 2226 in font units. The next character is my top-left octant with a slope showing that one is the stypo version, and meets the top border of its cell nicely as its top y is 1900. The 3rd and last character is the standard full block (U+2588), and again it extends above the top of the cell, exhibiting the same problem as ▞.

And to test it is not something I messed up, here is the same test, still running in Terminal Canary, but with the Cascadia Mono 2111.001 bundled with Terminal Preview from the Store, and with my test version of Cascadia Mono NF uninstalled.

I believe the current Preview version crops characters that extend outside of their cell, but the Canary doesn't (which is a good thing), and so now we clearly see that the block elements have been extending too high, but were less noticeable before I fixed their coordinates because they actually used non-stypo coordinates and only moved points that were on the very top y line.
Considering the font works well in all other apps I tried, I suspect Terminal is doing something wrong here that makes it use the non-stypo version, ignoring the rclt substitution, for the base block elements characters.

[PhMajerus]

Seeing the behavior in Terminal Canary, it is clear to me the adjustments I performed are correct. They make the individual pseudopixels of mosaics / block elements equal sized. The reason they still don't align perfectly with my newer mosaics is some other issue.

I actually noticed there was a problem years ago because ▀ and ▄ were not perfect half blocks, but assumed it was a design glitch.
Then beginning of this year when I started working on legacy computing block elements, I noticed they lined perfectly with each other, but not with some of the older blocks, which was strange because while the existing characters didn't use a consistent set of coordinates for its mosaics, some of them were the same as mine but still didn't align.
This is the reason I removed my first PR #705 that contained a mix of new mosaics but also fixes to unify the coordinates of older block elements, because I noticed the problem was more complex than just fixing the coordinates and didn't want the few fixes to make the review more difficult and prevent the new characters from being available.

Now with over 500 characters added in two PR, which I have been using for months, I'm confident my adjustments to glif files are correct, and Terminal Canary shows that there was a problem with the stypo substitutions of those characters since the very beginning.
This means the problem will be much more noticeable regardless of my fixes, and will require a correction somewhere else, but I don't know if it is in the font or in Terminal's code.
Seeing that the font works great in all other environments I tested, including Visual Studio editor, MS Word, and even Win32 GDI code, I think it's better to fix the font and use that as a foundation to search for the bug.

I'm going to go forward with a PR for those changes, as they don't make the situation worse and fixes the issue I originally documented in this thread.
The pending issue is to find out why the non-stypo version of those characters are being used instead of the stypo variants, but that has been the case for years, we just never noticed it was doing that before trying to make everything align perfectly.

Later I'll document the issue in the Terminal repo as it seems to be something they might have to look into.