transparent background (WIP, Kitty and iTerm2 only)

[benjajaja]

Works well with only with Kitty.

iTerm2 has the problem of not clearing what's between the image and the
next cell. Sixel would have the same problem, but transparency does not
really work anyway; the icy_sixel library has some notion of
transparency in the palette (sixel_dither::set_transparent) but I
don't know how to get it to work.

[j4james]

the icy_sixel library has some notion of
transparency in the palette (sixel_dither::set_transparent) but I
don't know how to get it to work.

@benjajaja I think the problem is that it doesn't set the sixel parameters correctly. You need to have the second parameter set to 1 in order to use transparency, and it looks they just hardcode all the parameters to 0.

So you might be able to get it to work by patching the returned sixel string. If it starts with something like \x1BPq or \x1BP0;0;0q, try replacing that with \x1BP0;1q.

[benjajaja]

@j4james tried that, thank you for the tip! It does enable transparency together with sixel_dither.set_transparency(i32), it's just some random color instead of transparent pixels. That is, sixel_dither should encode only transparent pixels the rgba data to that color, but it doesn't (or I'm using it wrong, idk).

[j4james]

It does enable transparency together with sixel_dither.set_transparency(i32), it's just some random color instead of transparent pixels.

@benjajaja I'm not positive how it works, but just looking at the code, it seems to me that the set_transparency method expects you to specify the palette index of the color that you want to be transparent (see here).

So instead of using ARGB, you'd need to paint the transparent areas of your image with a particular color (e.g. bright purple). But then I'm guessing you'd also need to provide a fixed palette for that image, and the index of the bright purple in that palette would be the value you would pass to set_transparency.

I don't know if there's an easier way to do that, because this seems overly complicated. But maybe the transparency support in that library was never fully implemented.

[benjajaja]

I opened an issue on icy_sixel mkrueger/icy_sixel#6

In any case, we'd have to do something about background not clearing. This doesn't happen with Kitty's transparency, because our Kitty protocol writes one placeholder character per cell. So I guess for sixel and iterm2 we'd have to slice 'n' dice the image into many small images (the size of a cell) or something crazy like that. Probably bad for performance. I've tried DECFRA but it didn't seem to work, most terminals probably don't implement it.

[j4james]

we'd have to do something about background not clearing.

Just on this point, almost all Sixel terminals I've tested can erase the image content with ECH (assuming you're happy with it also erasing the text content in the given area). And those terminals that didn't support ECH correctly were also not capable of rendering transparent Sixel images anyway, so it assumedly wouldn't matter.

XTerm is a bit of an exception in that it supports transparency, but its ECH handling is a bit flaky (sometimes the image can reappear after it has been erased).

Also note that you can detect support for rectangular area operations by looking for 28 in the DA1 report, similar to the way you detect Sixel by looking for a 4. That way you can use a more efficient DECERA on terminals that support it. And in the case of XTerm, DECERA is not just more efficient - it also seems to be less buggy than ECH.

[benjajaja]

@j4james I can't get the DECERA to work in any way here 🤔

[j4james]

@benjajaja Are you by any chance running xterm with the -ti vt340 parameter, or have you set the decTerminalID resource to vt340? Because when you do that, xterm disables the rectangular area operations like DECERA, because the original VT340 terminals didn't support that.

If you want to use both Sixel and rectangular area operations at the same time, you need to use the decGraphicsID option to configure that. It's a bit annoying.

And I've just noticed now that xterm is actually still reporting support for rectangular area operations in the DA1 response even when they're disabled, so that's another bug you'd need to work around. Or perhaps just tell people not to use xterm with -ti vt340.

[j4james]

FYI, this is the test case I'm using to check DECERA support...

Draw a transparent yellow triangle occupying around 40x40 pixels (this assumes a 10x20 cell size, so 4x2 character cells):

echo -e '\e[3;5H\eP0;1q"1;1;40;40#1;2;100;100;0!15?o{~~{o-!12?o{!8~{o-!9?o{!14~{o-!6?o{!20~{o-!3?o{!26~{o-o{!32~{o-\e\\'

Draw an inverted red triangle on top of that, so you can see the overlap:

echo -e '\e[3;5H\eP0;1q"1;1;40;40#1;2;100;0;0BN!32~NB-!3?BN!26~NB-!6?BN!20~NB-!9?BN!14~NB-!12?BN!8~NB-!15?BN!2~NB-\e\\'

Draw the red triangle again, but now prefixed with DECERA to clear the background first:

echo -e '\e[3;5;4;8$z\e[3;5H\eP0;1q"1;1;40;40#1;2;100;0;0BN!32~NB-!3?BN!26~NB-!6?BN!20~NB-!9?BN!14~NB-!12?BN!8~NB-!15?BN!2~NB-\e\\'

[benjajaja]

@j4james I was running this on WezTerm, as I was hoping to get transparency support there. WezTerm doesn't seem to support DECERA. However, I tried Erase Line (EL), and this works! I got lucky because it erases only text, not other images, so it ends up working. So I split the image into horizontal slices, and before rendering each slice on a line I also send EL. This clears any stale characters and leaves no artifacts.

The same could maybe be done for sixels, but again, it would rely on EL clearing only text, not graphics.

Overall I really want this feature in, because I'm working on some program that looks a lot better with transparency, but I think it should be behind some picker.enable_experimental_transparency() option to be clear.

[j4james]

I'm not sure I understand why you'd want to erase only text and not graphics, but EL is supposed to erase both, at least when it comes to Sixel. The original hardware terminals essentially had one large bitmap which you could write to with either text or individual pixels. To erase a line, the terminal would just fill a rectangle on the bitmap, which would erase everything that was previously written to that area. There's no way to distinguish the source of each pixel and treat text differently from graphics.

If you're looking at the Kitty image protocol, though, it works in a completely different way. My understanding is that every image is managed separately from every other image, as well as separately from the text. So you can layer images both over and under text, and you can erase each image individually without necessarily affecting other images or text. I'm not sure what the rule is for EL, but it wouldn't surprise me if it only erases text. I would double check that behavior on Kitty, though, because other terminals may not have all the details correct.

[benjajaja]

My reasoning was that DECERA wasn't working on WezTerm, which is the only terminal in which I can run the iTerm2 protocol, so I was trying other erase sequences. EL just happens to not erase (iTerm2 protocol) graphics on WezTerm. By what you're saying it's probably a bug or an accidental feature. After all, iTerm2 cannot possibly be specified in any old VT standards, and iTerm2 itself has rather limited documentation and specs for its graphics protocol.

The Kitty protocol is completely different from sixel or iTerm2, you're right. Images are placed and deleted, and deleting is not really possible in a ratatui-widget, where there is nowhere to hook into any kind of deletion or "no longer rendered" event/hook. Luckily however Kitty provides some help in the form of "unicode placeholders", where the image is "transmitted" once (we can do that on ratatui) and then can be placed at different places with some "unicode placeholder" sequence. When those are not rendered anymore, the image also is gone without need for any delete/cleanup hooks. Except it's still in memory, but that's a different kind of problem.

---

@j4james, thank you so much again for your input!

I finally got it to work with simple erase characters and cursor movement sequences: ECH for (image width in characters) and CUD for each line in (image height in characters), and then go back with CUU, and then the iTerm2 sequence. Clears everything, probably not as fast as DECERA, but good enough for now.