Neovim theme background colors are being stripped when scrolling

[bitcrazed]

From @jasin on November 30, 2017 15:25

This is a similar problem to the one listed here, but since someone decided to lock that thread I'll start my own.

#1706
@zadjii-msft

Simply put, I am running ubuntu from the windows store

lsb_release -a returns 16.04.3 LTS

You can see the problem in neovim, when the file is scrolled down the background color is stripped. I have no idea if this problem has been fixed in 16.04.3 LTS or if this is only fixed in the insider version?? Any sort of tips on how I can further look into the problem I'd happy to look into it and report back.

Copied from original issue: microsoft/WSL#2710

[bitcrazed]

From @rodrymbo on November 30, 2017 17:47

You could fire up sshd and log in to it with PuTTY or similar and see if the same background problem continues. If it works okay in a terminal but not in the console, then this is a console issue...

Work continues to make the console work more and more like a terminal...

[bitcrazed]

From @zadjii-msft on November 30, 2017 19:12

@jasin What windows build# are you on? There was a bit of churn in that area recently. I think I already fixed that, but if not, knowing the specific build number will help me track it down.

[bitcrazed]

From @jasin on December 1, 2017 11:18

@zadjii-msft Windows Version 10.0.16299 Build 16299

[bitcrazed]

From @zadjii-msft on December 1, 2017 17:16

@jasin could you post what the following python script looks like when you run it in the console?

import sys
import time

def csi(seq):
    sys.stdout.write('\x1b[{}'.format(seq))

def write(s):
    sys.stdout.write(s)

if __name__ == '__main__':
    print('Clearing the line paints to the end?')
    print('\E[42mfoo\E[43m\E[KTest')
    csi('42m')
    write('foo')
    csi('43m')
    csi('K')
    write('Test')
    csi('m')
    print('')
    print('\E[42mfoo\E[43m\E[K\\nTest')
    csi('42m')
    write('foo')
    csi('43m')
    csi('K')
    write('\nTest')
    csi('m')
    print('')
    print('\E[42mfoo\E[43m\E[1KTest')
    csi('42m')
    write('foo')
    csi('43m')
    csi('1K')
    write('Test')
    csi('m')
    print('')
    print('\E[42mfoo\E[43m\E[1K\\nTest')
    csi('42m')
    write('foo')
    csi('43m')
    csi('1K')
    write('\nTest')
    csi('m')
    print('')
    print('\E[42mfoo\E[43m\E[2KTest')
    csi('42m')
    write('foo')
    csi('43m')
    csi('2K')
    write('Test')
    csi('m')
    print('')
    print('\E[42mfoo\E[43m\E[2K\\nTest')
    csi('42m')
    write('foo')
    csi('43m')
    csi('2K')
    write('\nTest')
    csi('m')
    print('')
    print("\x1b[48;2;64;128;255mX\x1b[48;2;128;128;255m\x1b[KX")
    csi('m')
    print('')

You can just paste that into a file like gh2710.py and run it with python gh2710.py

[bitcrazed]

From @jasin on December 1, 2017 19:0

Hope this helps

[bitcrazed]

From @zadjii-msft on December 1, 2017 19:7

Welp, that's exactly how that's supposed to look. Shoot. This must be some other weirdness then.

Just to help investigate, are there any specific plugins you have installed? Have you modified your neovim configuration at all? (I'm not really familiar with neovim, but for normal vim I'm talking about the .vimrc file)

I'll file a bug on my end to make sure this gets tracked down.

[bitcrazed]

From @jasin on December 1, 2017 20:10

Well I am not sure I fully understand the problem so it seems.

I was using neovim during an ssh session to my FreeBSD box. That is where I do my development. I decided to quickly install the neovim package from the unstable ppa and I do not see the issue locally that I am experiencing during an ssh session to my FreeBSD box. I do run zsh shell on that server box.

To note, I did run the py scrypt on my FreeBSD box through ssh session and you said that looked ok, so this must specifically be a neovim/FreeBSD issue over SSH. But that is as far as I am willing to comment. I'm not sure I know enough to look into it further without some possible clues.

[bitcrazed]

From @jasin on December 1, 2017 20:39

Further investigation shows this is not just limited to neovim, I see the issue in vim as well over SSH to FreeBSD, but slightly different, opening up a file now shows me this without even having to scroll. I have also ruled out the shell, makes no difference, didn't think it would, but I double checked using bash shell

[bitcrazed]

From @zadjii-msft on December 1, 2017 20:43

@jasin can you post the output of infocmp on your FreeBSD machine?

[bitcrazed]

From @jasin on December 1, 2017 20:57

Here you go

jasin:tyr~$ infocmp
#       Reconstructed via infocmp from file: /usr/local/share/misc/terminfo.db
xterm-256color|xterm with 256 colors,
        am, bce, ccc, km, mc5i, mir, msgr, npc, xenl,
        colors#0x100, cols#80, it#8, lines#24, pairs#0x7fff,
        acsc=``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
        bel=^G, blink=\E[5m, bold=\E[1m, cbt=\E[Z, civis=\E[?25l,
        clear=\E[H\E[2J, cnorm=\E[?12l\E[?25h, cr=\r,
        csr=\E[%i%p1%d;%p2%dr, cub=\E[%p1%dD, cub1=^H,
        cud=\E[%p1%dB, cud1=\n, cuf=\E[%p1%dC, cuf1=\E[C,
        cup=\E[%i%p1%d;%p2%dH, cuu=\E[%p1%dA, cuu1=\E[A,
        cvvis=\E[?12;25h, dch=\E[%p1%dP, dch1=\E[P, dim=\E[2m,
        dl=\E[%p1%dM, dl1=\E[M, ech=\E[%p1%dX, ed=\E[J, el=\E[K,
        el1=\E[1K, flash=\E[?5h$<100/>\E[?5l, home=\E[H,
        hpa=\E[%i%p1%dG, ht=^I, hts=\EH, ich=\E[%p1%d@,
        il=\E[%p1%dL, il1=\E[L, ind=\n, indn=\E[%p1%dS,
        initc=\E]4;%p1%d;rgb\:%p2%{255}%*%{1000}%/%2.2X/%p3%{255}%*%{1000}%/%2.2X/%p4%{255}%*%{1000}%/%2.2X\E\\,
        invis=\E[8m, is2=\E[!p\E[?3;4l\E[4l\E>, kDC=\E[3;2~,
        kEND=\E[1;2F, kHOM=\E[1;2H, kIC=\E[2;2~, kLFT=\E[1;2D,
        kNXT=\E[6;2~, kPRV=\E[5;2~, kRIT=\E[1;2C, kb2=\EOE, kbs=^H,
        kcbt=\E[Z, kcub1=\EOD, kcud1=\EOB, kcuf1=\EOC, kcuu1=\EOA,
        kdch1=\E[3~, kend=\EOF, kent=\EOM, kf1=\EOP, kf10=\E[21~,
        kf11=\E[23~, kf12=\E[24~, kf13=\E[1;2P, kf14=\E[1;2Q,
        kf15=\E[1;2R, kf16=\E[1;2S, kf17=\E[15;2~, kf18=\E[17;2~,
        kf19=\E[18;2~, kf2=\EOQ, kf20=\E[19;2~, kf21=\E[20;2~,
        kf22=\E[21;2~, kf23=\E[23;2~, kf24=\E[24;2~,
        kf25=\E[1;5P, kf26=\E[1;5Q, kf27=\E[1;5R, kf28=\E[1;5S,
        kf29=\E[15;5~, kf3=\EOR, kf30=\E[17;5~, kf31=\E[18;5~,
        kf32=\E[19;5~, kf33=\E[20;5~, kf34=\E[21;5~,
        kf35=\E[23;5~, kf36=\E[24;5~, kf37=\E[1;6P, kf38=\E[1;6Q,
        kf39=\E[1;6R, kf4=\EOS, kf40=\E[1;6S, kf41=\E[15;6~,
        kf42=\E[17;6~, kf43=\E[18;6~, kf44=\E[19;6~,
        kf45=\E[20;6~, kf46=\E[21;6~, kf47=\E[23;6~,
        kf48=\E[24;6~, kf49=\E[1;3P, kf5=\E[15~, kf50=\E[1;3Q,
        kf51=\E[1;3R, kf52=\E[1;3S, kf53=\E[15;3~, kf54=\E[17;3~,
        kf55=\E[18;3~, kf56=\E[19;3~, kf57=\E[20;3~,
        kf58=\E[21;3~, kf59=\E[23;3~, kf6=\E[17~, kf60=\E[24;3~,
        kf61=\E[1;4P, kf62=\E[1;4Q, kf63=\E[1;4R, kf7=\E[18~,
        kf8=\E[19~, kf9=\E[20~, khome=\EOH, kich1=\E[2~,
        kind=\E[1;2B, kmous=\E[M, knp=\E[6~, kpp=\E[5~,
        kri=\E[1;2A, mc0=\E[i, mc4=\E[4i, mc5=\E[5i, meml=\El,
        memu=\Em, oc=\E]104\007, op=\E[39;49m, rc=\E8,
        rep=%p1%c\E[%p2%{1}%-%db, rev=\E[7m, ri=\EM,
        rin=\E[%p1%dT, ritm=\E[23m, rmacs=\E(B, rmam=\E[?7l,
        rmcup=\E[?1049l, rmir=\E[4l, rmkx=\E[?1l\E>,
        rmm=\E[?1034l, rmso=\E[27m, rmul=\E[24m,
        rs1=\Ec\E]104\007, rs2=\E[!p\E[?3;4l\E[4l\E>, sc=\E7,
        setab=\E[%?%p1%{8}%<%t4%p1%d%e%p1%{16}%<%t10%p1%{8}%-%d%e48;5;%p1%d%;m,
        setaf=\E[%?%p1%{8}%<%t3%p1%d%e%p1%{16}%<%t9%p1%{8}%-%d%e38;5;%p1%d%;m,
        sgr=%?%p9%t\E(0%e\E(B%;\E[0%?%p6%t;1%;%?%p5%t;2%;%?%p2%t;4%;%?%p1%p3%|%t;7%;%?%p4%t;5%;%?%p7%t;8%;m,
        sgr0=\E(B\E[m, sitm=\E[3m, smacs=\E(0, smam=\E[?7h,
        smcup=\E[?1049h, smir=\E[4h, smkx=\E[?1h\E=,
        smm=\E[?1034h, smso=\E[7m, smul=\E[4m, tbc=\E[3g,
        u6=\E[%i%d;%dR, u7=\E[6n, u8=\E[?%[;0123456789]c,
        u9=\E[c, vpa=\E[%i%p1%dd,

[bitcrazed]

From @msundin on December 27, 2017 19:31

I can confirm this bug in Ubuntu 16.04 after Fall Creators update and via SSH to FreeNAS (FreeBSD). When scrolling, the background color (in this case grey) disappears where there is no text. Both in Vim and Neovim. This makes it impossible to use sophisticated color schemes in Vim/Neovim:

[bitcrazed]

From @msundin on December 29, 2017 3:38

I don't know why #1706 is locked? It is still very valid and describes the bug well.

[bitcrazed]

From @zadjii-msft on January 4, 2018 22:17

Okay yea I'm seeing this again.

To limit the repro variables for myself:

• it repro's in normal vim
• switch to a colorscheme with a RGB bg for text - these will make it really obvious:

let g:solarized_termcolors=256
colo solarized

:redraw! will get the whole buffer looking right, and then any scroll up down will only have the right bg filling the top or bottom line

[bitcrazed]

From @sus007 on January 5, 2018 11:18

I too confirm this bug in my system:

• Operating System and version: Windows 10 Pro Insider Preview / Version: 1709 / OS-BUILD: 17025.1000

• Ubuntu 16.04.03 LTS

• VIM- Vi IMproved 8.0 (2016 Sep 12, compiled Jan 02 2017)

I've tried every suggested methods of #1706 , still while scrolling down the entire color-scheme sort of fades out, turns colorless as we can see below.

[bitcrazed]

From @zadjii-msft on January 11, 2018 22:51

I got this figured out. I was off-by-one on copying the RGB attributes of the last run of characters.

Before:

After:

gnome-terminal (for reference):

I'm gonna add some tests and get this code reviewed. Thanks for all the persistence and patience folks!

[sus007]

As of today, with the latest Windows Insiders build, the fade away issue while scrolling has been resolved but still there's this ugly grey bar masking every text in the document while the color stays intact.

• Operating System and version: Windows 10 Pro Insider Preview / Version: 1709 / OS-BUILD: 17074.1002
• Ubuntu 16.04.03 LTS
• VIM- Vi IMproved 8.0 (2016 Sep 12, compiled Jan 02 2017)

[parkovski]

As a temporary workaround, setting term to tmux-256color in .vimrc fixes this for me.

[Feynt]

Happily I found this bug listing. It seems to be an xterm default rendering issue. I added the following to my .bashrc:

(line 115): export TERM=screen-256color

The result is vim properly renders background colours all the way to the end. Using default xterm-256color I get the results of the gif above. Interesting note: This seems to be related to new line characters. Using delete/backspace on a line and causing the text to shift left (specifically left) caused the background to render all the way to the right again on that line specifically.

[zadjii-msft]

I'll add it to my backlog and come back and investigate. I thought I fixed this for Spring Creator's Update, but I don't really know when that's actually getting released....

[Feynt]

It may well have been resolved in the spring update, I'm not running a preview build. Here's hoping for a quick test cycle though.

…

On Fri, Apr 20, 2018, 13:02 Mike Griese, ***@***.***> wrote: I'll add it to my backlog and come back and investigate. I thought I fixed this for Spring Creator's Update, but I don't really know when that's actually getting released.... — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#70 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABNRVnitb6foR4VEkQcpII2oE94Qo5X7ks5tqhSTgaJpZM4SHvdT> .

[jasin]

it would be nice if there was some indication of when this is supposed to be fixed?!?!?

I am running the latest Spring creators update 1709 16299.402
uname -a
Linux roofus 4.4.0-43-Microsoft #1-Microsoft Wed Dec 31 14:42:53 PST 2014 x86_64 x86_64 x86_64 GNU/Linux

I am still seeing the issue, but I have no idea what has been updated. All I seem to get is cumulative security updates. This is starting to get really frustrating. A fix should not be this hard to push out. setting screen-256color does nothing for nvim, it just mangles the rendering worse than before.

[zadjii-msft]

@jasin For the record, 16299 is still Fall Creator's Update. "Spring Creator's Update" (which is I guess just "April Update" now, don't ask me I'm not in marketing) should have a build number that's something like "17134" or greater.

This issue has kinda evolved from one manifestation of the bug to the next similar, yet different incarnation. When I get to it, I'll let you know.

[zadjii-msft]

I can't seem to get this to repro anymore, in any fashion. If someone can get me a repro (color scheme/.vimrc) that this still repros on Insider's with, I'll be happy to re-open, but I think it's fixed now.

[Pablo1107]

• Operating System and version: Windows 10 Home / Version: 1809 / OS-BUILD: 17763.194
• Arch Linux on WSL (Linux Pablo-OMEN 4.4.0-17763-Microsoft broken vertical line char #194-Microsoft Mon Dec 03 17:58:00 PST 2018 x86_64 GNU/Linux)
• vim package version: vim 8.1.0542-1

I suppose you can't repro completely because I'm not using Ubuntu on WSL (which would be normal). But here is the screenshot of the bug.

:redraw! fix it perfectly.

Here my .vimrc (which is basically the default one) and the colorscheme (base16)
https://gist.github.com/Pablo1107/0afb0cb31bb8cb8df6aacd0b8243a762

[ndrewxie]

Issue is still here:

I already put export TERM=screen-256color into my bashrc, and put set t_Co=256 into my vimrc. :redraw! does nothing. I've already source'd both my bashrc and vimrc.

[ndrewxie]

To reproduce the above, install vim-gtk on Debian (latest windows stable), then try installing the summerfruit256 colorscheme

[ndrewxie]

In standard Vim, putting this into the vimrc did the trick:

if &term =~ '256color'
    " Disable Background Color Erase (BCE) so that color schemes
    " work properly when Vim is used inside tmux and GNU screen.
    set t_ut=
endif

[DHowett-MSFT]

Thanks @ndrewxie!

This makes it seem like a tmux (tmux/tmux#109) and screen[1] issue moreso than a conhost issue.

[1] from man 1 screen:

       bce [on|off]

       Change  background-color-erase  setting.  If  "bce"  is  set  to  on,  all  characters cleared by an erase/in‐
       sert/scroll/clear operation will be displayed in the current background color.  Otherwise  the  default  back‐
       ground color is used.

When I have bce off in my .screenrc, I see the same issue:

when I set bce on, however:

[Sangeppato]

I'm on 1903 (Release Preview), using Ubuntu 18.04 in WSL, neovim and "gruvbox" colorscheme. I can still reproduce the issue. The only solution apparently is to put " let g:gruvbox_termcolors=16" in my init.vim, but this completely change all the colors

[idbrii]

I can reproduce this with a minimal vim:
vim -Nu NONE +"colorscheme jellybeans" +"help help" and scrolling around inside the help window and some parts of the background are green instead of dark grey.

Where jellybeans is this specific version. (Also occurs with zellner, xoria256, sandydune, etc. but couldn't repro with any colors shipped with vim7. Probably because vim7 doesn't ship with schemes using 256 colors.)

I'm on Windows 1909 (OS Build 18363.657). (Public version, not insider.)

Using WSL: Ubuntu 14.04.5 LTS

VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Nov 24 2016 16:43:18)
Included patches: 1-52
Extra patches: 8.0.0056

&term == xterm-256color
&t_Co == 256

Running a terminal as bash.exe ("Microsoft Bash Launcher")

Strangely, I only noticed this recently. Maybe after my Windows Update on Friday?

Either of these lines will make the problem go away, but at the cost of reduced colours:

set term=xterm
set t_Co=16

I'm using the bash.exe that came with installing WSL through the Windows Store (2 years ago). I'm not using "Windows Terminal (Preview)" from the Windows Store (testing that, the issue doesn't occur because &term is win32 and &t_Co is 16 -- so it doesn't support 256 colours). The WSL repo asks to report issues here, so I assume my experience is relevant to this bug?

[bitcrazed]

This would be a great issue for @Kayla Cinnamon<mailto:[email protected]> & team to explore. Thanks for filing David. Rich.

[DHowett-MSFT]

@idbrii since this issue is closed and you seem to be having some slightly different symptoms, would you mind filling a new one so that we might track it appropriately? Thanks.