connector-proxy: make stdin/stdout streams independent to avoid deadlocks

[mdibaiee]

Currently connector-proxy operates in this way:

1. Keep reading from stdin (runtime sends data to stdin), translate it and send it to the underlying connector
2. Keep reading from stdout of the underlying connector, translate ita nd send it to runtime through stdout

The issue is, the runtime waits for the stdout of connector-proxy to be closed before it closes its stdin. With our current implementation, connector-proxy likewise wants to wait until it has read all from runtime before it exits.

At the moment there is a workaround where if the underlying connector exits, we wait a few seconds and then exit connector-proxy to avoid a deadlock. However, the ideal situation is where we have stdin and stdout as independent parts.

[mdibaiee]

I looked into this today, apparently there is no way in Rust standard library to close stdout of the process (see rust-lang/rust#40032), however it is possible by closing fd 1 which stands for stdout. Care must be taken while doing this since when that file descriptor is closed, the next file descriptor allocation will use that descriptor, which can lead to unintended behaviour.

In this case the best practice is to close the fd and immediately request opening /dev/null on fd 1.

Note that libc::close is unsafe, so we would be introducing unsafe code in connector-proxy if we decide to proceed and implement this logic.

The code will roughly look like this (not tested):

let rc = unsafe {
  libc::freopen("/dev/null", "w", libc::fopen(libc::STDOUT_FILENO, "r"))
};
if rc == -1 {
  Ok(())
} else {
  Err(CloseError { previous: io::Error::last_os_error(), file: None })
}

[jgraettinger]

If this is feeling like more trouble than it's worth, we could do an end-run around this issue by moving more directly to a world where connectors listen on a port over which they serve gRPC, and which the runtime dials.

[mdibaiee]

@jgraettinger yeah I think that's a better approach

[mdibaiee]

Relevant PR: #674 (comment)

[mdibaiee]

Closing this issue since we have gone ahead and switched to TCP communications, so this is no longer applicable. #682