Termios addon to allow for more typical terminal behavior

[upsampled]

I agree with the maintainers regarding #2664 , a full PTY implementation is out-of-scope and is more involved than one would initially realize; however, I believe just a 'termios' implementation would be a huge benefit for the majority of developers and fall within the projects goals.

It's pretty common for developer, me at least, to throw a pty infront of raw application data to get sensical terminal behavior. Most immediately it handles the CR, NL, and CRNL behavior that is seen in a typical shell environment. In these cases I am not really using the pty for anything other than termios flags. These flags are also used by essential, terminal aware cli tools to perform tasks like reading passwords .

Right now it looks like many developers in order to get typical typical terminal behavior are:

• piping a pty to/from a host
• creating custom onData/onmessage functions to implement the behavior they want.

I believe that a termios addon would:

• eliminate the need to pipe a pty in many cases
• allow 'normal' terminal behavior without custom data handlers
  • Just pass they typical termios flags to the addon for normal behavior
• set a better foundation for porting common cli applications
  • Providing a tcgetattr, tcsetattr, and cfmakeraw callback to WASM code should handle terminal aware applications

I am just about to refactor some custom ondata methods to take a termios argument, so i figured I'd ask if it was worth making into a full addon first.

[jerch]

I started investigating into that direction some time ago in https://github.com/jerch/browser-fakepty. Basic things are working, but it still has many loose ends (mostly around proper backpressure/flow control handling and process control shims). I started to shim some process API, see https://github.com/jerch/browser-fakepty/blob/master/src/Shell.ts for example "processes" and a naive shell impl. Note that the code is not even alpha, it is just a testground about possible directions with a fake pty. Progress is currently stalled mainly due to pulling more and more POSIX stuff in while I have not much time to work on it.

Maybe you find there some ideas, how to go about the missing-pty issue..

[Kreijstal]

soooo how is this going?

[RReverser]

set a better foundation for porting common cli applications

• Providing a tcgetattr, tcsetattr, and cfmakeraw callback to WASM code should handle terminal aware applications

For those coming across this issue specifically for Wasm usecase - xterm-pty solves this for Emscripten nowadays, covering line discipline, terminal sizing, and, with an upcoming PR converting things like Ctrl+C to signals as well.

[jerch]

@RReverser Oh interesting - thx for the headsup. May I ask how you are going to solve the signals issue? When I was looking into this, it kinda pulled half of the POSIX landscape into it including process management, which I found rather inconvenient, as JS code execution is not interuptable (which yet again boils down to real blocking semantics not being available in JS/WASM). Only termination with the help of webworkers was doable.
Do you hook in some kernel-like functionality?

[RReverser]

May I ask how you are going to solve the signals issue?

Emscripten already had basic intra-process signal emulation support except for default signal handlers, which I added too. emscripten-core/emscripten#20257. WASI SDK does the same.

Of course, inter-process signals won't work because there are no processes inside Wasm, but this is enough for being able to trigger signal via libc::raise from one place (e.g. terminal keystroke handler in our case) and handle in another, which covers basic stuff like termination, terminal resizing and such.

as JS code execution is not interuptable (which yet again boils down to real blocking semantics not being available in JS/WASM)

There are couple of ways to do it, you can see PR description and docs for more details as it was pretty much full Emscripten integration rewrite rather than just signals. Basically it supports two modes in order to suspend Wasm and read from keyboard or poll - one is Asyncify which rewrites Wasm and allows async JS APIs to be called while suspending the Wasm code on the same thread and another is PROXY_TO_PTHREAD which transparently moves the entire Wasm to separate thread so it can block execution thread of Wasm, but not I/O that's happening on the main thread.

---

(to be clear, I didn't build the project itself, only came across it while looking for options for proper PTY for Emscripten and demos on https://xterm-pty.netlify.app/ looked pretty compelling; I only rewrote Emscripten integration because initial one was somewhat difficult from user side and because I'm rather familiar with Emscripten and Asyncify themselves)

[jerch]

Yeah blocking "syscalls" should be doable with asyncify on the mainthread. And in a worker the asyncify overhead could be avoided by using atomic wait conditions. Still thats only working for actions the code asked for in the first place like reading from a blocking source.

But a POSIX signal can interrupt execution at any time, and would run the signal handler instead. How would that unconditional execution jump in middle of anywhere work with WASM? How would that be triggered and where would it jump to? Does WASI shim some magic kernel-like interface with these superpowers into the binary? Last time I read about WASI it was more around IO, but not about unsolicited execution traps orchestrated from outside of the code (kernel like). Ofc there is a chance that I got WASI concepts partially wrong.

Edit:
What could emulate signal behavior is to inject a maybe-blocking syscall in the code in high frequency, that then could trigger the signal handling (works only at the insert points, thus it is in fact a bad hack).

[RReverser]

Still thats only working for actions the code asked for in the first place like reading from a blocking source.

No, not quite - it will work whenever code pauses, but code does not need to explicitly request a signal. This is the same as with handling any other events on the main thread from Wasm - Wasm needs to yield from time to time with anything, even emscripten_sleep(0) for the browser to handle DOM events, but then during that time any event handlers can run and call back into Wasm.

In this case a keyboard handler sends a signal to Wasm, but yes, Wasm needs to yield to main thread to receive it - but then, it needs to yield to draw data to terminal and read data from terminal anyway or UI would be blocked forever, so this is not a problem in practice.

Anyway, this is probably not the right place for discussion of internal implementation details of a 3rd-party library :) I just wanted to post it here because I figured people subscribed to this issue are likely to be interested in a working Emscripten integration.

[jerch]

Anyway, this is probably not the right place for discussion

Yeah sorry, your are right. Thx for the pointer 😺