Document that all IO operations, including print, are asynchronous

[jamesdanged]

This bug is very subtle and took me a couple of hours to trace down to being caused by the "print" method. Basically, when writing asynchronous code using tasks, there is an expectation that your code only yields at well defined points, such as when calling "wait" or "yield". It seems that "print" also can cause your code to yield execution.

The below function fails in Julia 0.3.12:

function test_print_is_async()
  val = 1
  for i = 1:10
    f = () -> begin
      try
        val += 1
        local_val_before = val
        print(" ")
        local_val_after = val
        @assert local_val_before == local_val_after
      catch err
        showerror(STDOUT, err)
        Base.show_backtrace(STDOUT, catch_backtrace())
        throw(err)
      end
    end
    schedule(Task(f))  
  end

end

But if you comment out the print statement, it will pass.

[ScottPJones]

I would think that it would be expected that anything involving IO such as print might yield.
I'm not sure if that is well documented though - you'd have to check.

[jamesdanged]

Hmm, I couldn't find anything in the documentation that would indicate this is meant to be asynchronous. My whole experience with Julia is that by default methods that might take a long time like file IO are blocking and synchronous, unlike say Node, so this really seems like a bug to me. If you want to just document it as asynchronous, that's ok, just please do it in big bold letters!

[jamesdanged]

As a use case, I was implementing a simple resource pool, and there was a block of code I was assuming was atomic. I would be notified a resource was available, and I would then obtain a "lock" on that resource by removing it from the pool. It's very trivial code to write, but it became a nightmare when for some reason it was behaving very stochastically. It turns out it was because the block was not atomic at all because an innocent "print" line was a "yield" in disguise.

[malmaud]

All IO operations yield. libuv, the same library that handles async IO for Node, is also used in Julia whenever read or write are directly or indirectly called on an IO stream.

[jamesdanged]

Ah I see. Thanks for clarifying. I think that documentation for a point like this is critical. The Node api makes the async nature of such calls blindingly obvious by requiring you to provide callbacks. In ES6, yield can only be called within special generator functions which are distinctly marked separately from regular functions with a star: "function*()". Another example is C# where you can only call "await" from within a function marked "async" and the norm furthermore is to name such functions xxxAsync(). The danger with hidden yields is that as a developer it becomes increasingly difficult to easily eye code and determine whether a series of statements will be executed atomically or not which can lead to subtle bugs. Obviously, in Julia it is not idiomatic to use callbacks everywhere, and maybe for IO operations, we can just try to remember that they are async as well as not hide them too deeply within other function stacks. There are tradeoffs with any approach. But please put better documentation!

[malmaud]

If you'd like to submit a pull request with suggested changes to the documentation, we would certainly consider and appreciate it.

[goelakash]

@malmaud Is it for all I/O or only Text I/O?

[StefanKarpinski]

All I/O. In fact, anything that might block is done asynchronously.

[goelakash]

Ok, so is this sufficient - #15238

[stevengj]

There is a discussion on julia-users about whether this is true of file I/O, which uses ios.c. Although ios.h has some stuff for compatibility with libuv, it seems like it is doing blocking I/O operations on files? Any reason why we don't use the libuv file API?

[vtjnash]

See comments on #15091