improve TTY write buffering

[tanmaykm]

Noticed initially in some code that printed lot of debug messages on to stdout.

The following results in occasional garbage being printed on the terminal (tried on macos and ubuntu):

for i in 1:100000 println(randstring(100)); end

Seems to work fine if printed on to a file instead.

[rwgardner]

I am getting the same issue although I would actually change the title to include "memory corruption" or something.

for i = 1:100000
   a=[1 2 3 4]
   println("a: ", a)
end

destroys my runtime and terminal.

[bicycle1885]

I also encountered the issue.
Using git bisect, I've found that 1aed903 seems to be the first bad commit.

[Keno]

Notice my comment
Wait a minute. Does libuv do non-blocking writes to TTYs now? We make the assumption that writes to TTYs are thread-blocking in a number of places.. That issue isn't just pedagogical ;)

[JeffBezanson]

This is release-blocking.

[nolta]

Removing the TTY write methods from base/stream.jl appears to fix this problem. But then the test output is screwed up,

    JULIA test/all
    From worker 9:      From worker 3:      From worker 4:      From worker 7:      From worker 5:      From worker 6:       * collections
    From worker 2:       * linalg2
     * linalg3
     * numbers
     * core
     * keywordargs
     * linalg1
    From worker 8:       * strings
    From worker 6:       * hashing
    From worker 6:       * remote
    From worker 6:       * iobuffer
...

This, in turn, seems to be fixed by

diff --git a/base/multi.jl b/base/multi.jl
index 5eafe68..14cecab 100644
--- a/base/multi.jl
+++ b/base/multi.jl
@@ -1035,7 +1035,7 @@ function create_worker(privhost, port, pubhost, stream, config, manage)
             @async begin
                 while !eof(stream)
                     line = readline(stream)
-                    print("\tFrom worker $(wrker.id):\t",bytestring(line))
+                    print("\tFrom worker $(wrker.id):\t $line")
                 end
             end
         end

[carlobaldassi]

@nolta the test output screwup is very similar to #5770 (which weirdly I seemed to be the only one having), and indeed your fix works in my case, so I pushed it, thanks.

[JeffBezanson]

@loladiro could you comment on @nolta 's above suggestion?

[Keno]

Yeah, that would work fine. I haven't gotten a chance yet to look at what the hell libuv did to ttys, but that is certainly the right change if they are now not thread-blocking.

[staticfloat]

This change has already been made: aadabde

[Keno]

I think @JeffBezanson was referring to removing the TTY methods, which I don't think has happened yet.

[staticfloat]

Ah, gotcha.

On Tue, Apr 8, 2014 at 3:52 PM, Keno Fischer [email protected]:

I think @JeffBezanson https://github.com/JeffBezanson was referring to
removing the TTY methods, which I don't think has happened yet.

Reply to this email directly or view it on GitHubhttps://github.com//issues/6375#issuecomment-39911442
.

[JeffBezanson]

Removing the TTY methods is ok, but the fix in multi.jl is not general --- other code that uses print or println does not get line buffering, so its output is still not what one would prefer.

[Keno]

The action item here is to make use of libuvs capability to write multiple buffers in one write request and block on that. Might get to it tomorrow.

[vtjnash]

moving this to 0.4, since the bug is fixed

[vtjnash]

closing this since the original bug was fixed, the followup issue has been improved by adding locking to streams during IO, and further improvement is covered by #3887