Add capture macro for both out and err.

[j-fu]

Hi would be great to have this in Pluto.jl

[iamed2]

LGTM but needs tests

[j-fu]

Yeah, sure.

For the time being I added the macro to my PR to PlutoUI.jl
Would replace it later by Supressor.

But:

For making this work with PIuto I had to remove the if logger.stream == original_stderr test.
Still works from the REPL though. However, this has been guesswork. Wonder why @fonsp ?

[iamed2]

I would guess it needs to work something like this instead:

https://github.com/JuliaLang/IJulia.jl/pull/671/files#diff-54090e4bd9da3cc6b0fbb839b246aeb3R120-R121

and temporarily set the global_logger to a SimpleLogger pointing to the redirected stderr.

[c42f]

This hack seems quite ugly and fragile. It's quite likely to break in future versions as it relies on a lot of implementation details.

It also basically assumes that logger is a Logging.ConsoleLogger because it assumes that logger.stream exists.

I'm not sure what the intent of checking logger.stream == original_stderr, but other than that it would be better to create a new ConsoleLogger, and use the public API Logging.with_logger to install it. Then you can avoid relying on a big pile of implementation detail.

[iamed2]

Yeah this is pre-Julia-1.0 code!

with_logger makes total sense, that's probably better than using global_logger(logger)

[j-fu]

Yeah, kind of more understandable: JuliaPluto/PlutoUI.jl#31 (comment)
However not sure if I can ignore @async ...

[fonsp]

Why not just compose the two:

macro capture(expr)
	quote
		local captured_out, captured_err
		captured_out = @capture_out begin
			captured_err = @capture_err $(esc(expr))
		end
		(captured_out, captured_err)
	end
end

[fonsp]

The problem is that returning a Tuple{String,String} means that the relative order between stdout and stderr messages is lost.

Maybe we could return an Array{Pair{Base.TTY,String},1} instead, for example:

(@capture_both begin
	println(stdout, "hello")
	println(stderr, "asfdasdf")
	println(stdout, "world!")
end) == [
	stdout => "hello",
	stderr => "asdfasdf",
	stdout => "world!",
]

This can't be achieved by simple composition of @capture_out and @capture_err

[j-fu]

Agree, see the point.
This one does the trick to capture both into one buffer but still would get stuck when the stdout buffer is full.
Moreover it doesn't return the Array{Pair{Base.TTY,String},1} . Edit: getting there.

macro capture(expr)
    quote
        original_stdout = stdout
        out_rd, out_wr = redirect_stdout()
        # Write just one character into the streams in order to
        # prevent readavailable from blocking if if stays empty
        print(stdout," ")
        # Redirect both logging output and print(stderr,...)
        # to stdout
	with_logger(SimpleLogger(stdout)) do	
	    redirect_stderr(()->$(esc(expr)),stdout)
	end
        result_out=String(readavailable(out_rd))
	redirect_stdout(original_stdout)
        close(out_wr)
        # ignore the first character...
        result_out[2:end]
    end
end

See also JuliaPluto/PlutoUI.jl@9fa7555

[j-fu]

Using async read now. See the update in

JuliaPluto/PlutoUI.jl@1a194cc

I think this is as close as we can come to the idea to collect stdout and stderr together into one string. Concerning the idea to collect things into an array of pairs with the information about the stream I have this (would be easy to replace the first
type in the pairs by TTY (not sure if this would be helpful) or whatever else.

macro xcapture(expr)
    quote
        original_stdout = stdout
        original_stderr = stderr
        out_rd, out_wr = redirect_stdout()
        err_rd, err_wr = redirect_stderr()
        function myread(out_rd,err_rd)
            buffer=Array{Pair{String,String},1}(undef,0)
            more=true
            while more
                if !eof(err_rd)
                    push!(buffer,Pair("e",readline(err_rd,keep=true)))
                end
                if !eof(out_rd)
                    push!(buffer,Pair("o",readline(out_rd,keep=true)))
                end
                if eof(out_rd) && eof(err_rd)
                    more=false
                end
            end
            buffer
        end
        reader = @async myread(out_rd, err_rd)
        try
            with_logger(SimpleLogger(stderr)) do
                $(esc(expr))
            end
        finally
	    redirect_stdout(original_stdout)
	    redirect_stderr(original_stderr)
            close(out_wr)
            close(err_wr)
        end
        fetch(reader)
    end
end

I did not find a way to synchronize the sequence of lines in the case there are multiple line outputs in to one of stderr and stdout. May be it is possible though.

[j-fu]

I researched what the jl_generating_output stuff does mean. Essentially this is a missing API call which checks
if Julia is in one of its compilation stages (aka generating output as a compiler) or is really calculating.

"""
Check if julia is not in one of its compilation stages set by one of the 
flags "--output-bc", "--output-unopt-bc", "--output-o",	"--output-asm",	 "--output-ji",	 "--output-incremental",
where it would output some transformed version of the code instead of executing it.
"""
jl_not_compiling()=ccall(:jl_generating_output, Cint, ())==0

In fact this IMHO belongs to Base, but at least this could be defined here in Suppressor in order to make the code more understandable. And indeed I understand now that it may be necessary to check for this when running under generic Julia. OTOH it seems to be no problem to omit this test when running code cells in Pluto notebooks.