Jansi 1.12+ stopped working with 'cmd.exe' if executed inside Eclipse's Local Terminal

[jjohannes]

We use Jansi in Gradle and got a bug report after updating the Jansi dependency to 1.13.

After some investigation I found that from 1.12+ Jansi stopped working on cmd.exe when executed inside Eclipse's terminal (TM Terminal). See attached screenshot.

Reproduce:
On Windows: Install Eclipse > Install 'TM Terminal' (from Neon Update Site) > Open View 'Terminal' > new 'Local Terminal'

See also:
https://issues.gradle.org/browse/GRADLE-3562
https://discuss.gradle.org/t/console-output-is-not-readble/19625

[jjohannes]

I think I found the cause of the problem, which is actually the upgrade we did in the first place:

It's these two pull requests by @sschuberth
#45 (Detect Cygwin)
gradle/gradle#683 (Upgrade dependency in Gradle to include the above)

In the Eclipse terminal above, the term variable is set to xterm:

C:>echo %term%
xterm

If I do

C:>set term=

Jansi works again.

This is because the Cygwin check is checking for xterm:
https://github.com/fusesource/jansi/pull/45/files#diff-07806ec884b958ca4cde0761a14b198aR110

Can the check maybe be improved? Or is it something the TM terminal should not do? @sschuberth what is your take on this?

[sschuberth]

To me this indeed is something TM terminal should not do. By setting TERM=xterm the terminal basically makes a promise to support embedded color directives (with at least 16 colors), i.e. i should understand ANSI sequences directly. Apparently that's not the case, thus TM terminal should not set TERM at all.

[jjohannes]

I agree and I reported this to the TM terminal team:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=502134

What do you think about making the check more robust anyway by checking if cmd.exe is running in addition to checking for TERM=xterm?

[moberhuber]

Eclipse TM Terminal uses https://github.com/rprichard/winpty for running Windows programs inside its xterm emulation. So for us, setting TERM=xterm is correct (since that's what we natively interpret) but winpty changes that to Windows Console API.

I think that just checking the TERM environment variable in jansi is not sufficient to determine, whether Windows Console API or ANSI Sequences should be sent. It should really check whether it's running inside a Windows Console host or not (I don't know how to check that).

I suspect that as of today, your test would also fail if you open a Cygwin mintty and inside that mintty execute "winpty java -cp jansi-1.13.jar org.fusesource.jansi.Main" as per https://github.com/rprichard/winpty#using-the-unix-adapter ... Or, if you have cygwin sshd running, log into that sshd from a remote Linux box (with real xterm) and then run "winpty java -cp jansi-1.13.jar org.fusesource.jansi.Main" .

I have reached out to @rprichard to get his opinion on this.

[sschuberth]

... but winpty changes that to Windows Console API.

Really? So winpty breaks ANSI support by the underlying terminal?

It should really check whether it's running inside a Windows Console host or not (I don't know how to check that).

I fear that even adding such a check would not cover all cases. Implementing the check by looking for conhost in the parent process tree would probably not work if mintty was started from cmd, for example. Also, Windows 10 actually added ANSI support to the console.

I suspect that as of today, your test would also fail if you open a Cygwin mintty and inside that mintty execute ...

I don't have Cygwin installed, but I do have Git for Windows 2.10 installed, which builds upon MSYS2 and comes with both mintty and winpty. When running from a Git Bash (i.e. mintty terminal) I get a colored Hello World for both

mvn exec:java -pl example -Dexec.mainClass="org.fusesource.jansi.Main"

and also

winpty mvn.cmd exec:java -pl example -Dexec.mainClass="org.fusesource.jansi.Main"

[moberhuber]

Interesting. I installed Git for Windows 2.10 and noticed that it uses winpty "Version 0.2.2 (2016-02-25)". In the Eclipse Terminal, we use a version from 2013-12 so it's much older. I suspect the newer winpty can detect "genuine ANSI" in the client and route it through unmodified. I suggest waiting for input from rprichard, the winpty expert.

I checked, and in fact you can break the mintty / winpty combination as well: execute "winpty cmd" in your Git bash to force a Console app, then run your mvn magic makes it break:

[sschuberth]

I just double-checked on another Windows machine:

winpty mvn.cmd exec:java -pl example -Dexec.mainClass="org.fusesource.jansi.Main"

only returns colored output on my Windows 10 machine, on Windows 8.1 I see literal escape sequences. So the interpretation of the ANSI codes came from Windows 10 in my case ...

[rprichard]

I commented on the Eclipse bug report (https://bugs.eclipse.org/bugs/show_bug.cgi?id=502134). I think Eclipse's TM Terminal should stop setting TERM when it uses winpty. Starting with 0.3.0, the winpty.exe Cygwin/MSYS adapter clears TERM. (rprichard/winpty#43)

winpty works by creating a hidden console buffer and scraping its content using ReadConsoleOutput, which reports the character and color of each cell in the buffer. If a console program writes an escape sequence (and if ENABLE_VIRTUAL_TERMINAL_INPUT is unsupported/unenabled), then each byte of the escape will be written to a successive cell of the screen buffer. That sequence can even wrap around to the next line. Prior to 0.3.0, in at least some situations, winpty copies the escape character through as-is, so the escape sequence may be unreliably interpreted by the terminal in which winpty runs. (rprichard/winpty#47)

To get color working with winpty, the console program needs to colorize its output exactly the same way it would with an ordinary Windows console. Prior to Windows 10, it could use SetConsoleTextAttribute. As of Windows 10, it could try to enable ENABLE_VIRTUAL_TERMINAL_INPUT, which turns on conhost's interpretation of ANSI escapes. If the STD_OUTPUT_HANDLE isn't a console handle at all, then I assume those APIs will simply fail, so there'd be no colored output.

[gnodet]

Closing as inactive and targeted to the unsupported 1.x.