Progress monitoring fallback to the console

[ChrisRackauckas]

It would be nice to be able to support progress monitoring in the console when Juno is not available. I think the proper way to handle this will be to actually implement this as a standard feature for Juno.jl's progress API, so then any implementation of progress monitoring in Juno will automatically carry over to such an implementation in the console.

Also noted at SciML/DifferentialEquations.jl#121

It would not be too hard if Juno.jl accepts the dependency of ProgressMeter.jl, in which case a Progress type could get a new field of whether to print to the console or use the Juno progress bar (default determined by whether Juno is loaded, but make this a keyword arg on construction so one can switch this around as needed), and a few methods could have a separate branch depending on this boolean which calls the appropriate ProgressMeter.jl commands. It can display variables, so the msg string can probably be displayed as well, so it should be a feature-complete fallback.

[MikeInnes]

I'm not against this, but my main concern would be ProgressMeter.jl's ability to handle the nested case. For the flag we can just use Juno.isactive().

[ChrisRackauckas]

I don't think it can handle the nested case, and you may just not want to throw an error if someone tries it with !Juno.isactive(). I think for most uses there is only one progress bar, so I think that it would still cover a majority of uses.

[Yacalis]

Hi everyone, I'm interested in doing this project as part of the Google Summer of Code, as listed on the tooling projects page, http://julialang.org/soc/projects/tooling.html#progress-metre-improvements. What can I do to get started on this?

[ChrisRackauckas]

@MikeInnes

[Datseris]

Hey People, I am also looking for some kind of fallback for progress meters in the case of REPL :)

[pfitzseb]

Ref JuliaLang/Juleps#30 for some very nice thoughts on a more general design for this. In the short run a fallback to ProgressMeters.jl is certainly possible and I might even get to that sometime ;)

[Datseris]

Hello, I would like to solve this / implement this. Any tips? I have to be able to do everything through the Requires package, because my packages do not have a dependency on Juno.

My first question,s how do I check whether users are running code in Juno or in Repl? I do not know that. I saw the command Juno.isactive() but what if Juno doesn't exist at all, e.g. they are in the repl?

[pfitzseb]

You can use isdefined(Main, :Juno) && Juno.isactive(), but if I were you I wouldn't spend to much time coming up with a super clean solution to this problem -- once JuliaLang/julia#23712 is merged it'll enable a much better interface.

[Datseris]

Cool thanks for letting me know. But how can logging have any connection with progress bars in Juno :S ?

[ChrisRackauckas]

The logging tools are going to have a connection to a standard progress monitoring API which Juno can plug into.

[Datseris]

Great!

[c42f]

If you do want to work on this, you don't need to wait for JuliaLang/julia#23712 to be merged. Instead you can use the package MicroLogging.jl which provides the same interface and works on julia-0.6 (it's being kept in sync with the work in Base). The only thing you won't get from MicroLogging is integration with the Base log messages (eg, syntax depwarns).

In fact, two of my TODO items in the MicroLogging design tracker over at c42f/MicroLogging.jl#5 are to check that the design serves as an effective message transport layer for ProgressMeter and Juno. If someone wants to have a go at the Juno side of this that would be great - I've got my work cut out for me refactoring the four or five different log mechanisms currently in use in Base ;-)

Note that I do need to make a new release of MicroLogging very soon, as the interface has been refined rather a lot since the previous release.

[MikeInnes]

That's great to hear. I think the only real issue from our side is that it needs to support nested / parallel progress bars, but if the design is flexible enough that shouldn't be an issue.

[c42f]

Yes, I think it will support the nested progress bars quite easily as there's lots of freedom to structure your log records. The tricky bit will be getting a common convention for the structure which is simple but serves the needs of various tools. At the moment, I like the fact that

@info "Description" progress=i/N

works and is simple, but it doesn't guarantee that progress==1.0 is ever hit which might be a problem.

[MikeInnes]

So right now our API is

p = Progress()
progress!(p, 0.5) # call this in a loop, or whatever
done!(p)

where you can obviously create as many Progress objects as you want and they will stack up in the UI. There isn't any explicit nesting as such, it's just a special case of concurrency. If we can build that on top of the underlying framework then we're golden, I think.

[c42f]

Ok, I see this is a little more general than I thought. In particular, if you run @async foo() several times, these several separate tasks should show up as separate progress bars. it's definitely doable, the trick is simply agreeing on what exactly constitutes a progress message so that backends can agree on how to harvest progress messages from the log stream.

[MikeInnes]

Makes sense. From our point of view I think the requirements are pretty minimal – a progress message needs only to identity a progress bar and set the value (0-1 or "done").

[oxinabox]

Any update on this these days?

[pfitzseb]

Not yet, but in the process of transitioning Juno to 0.7 I'll definitely take a look at this.

[pfitzseb]

Ok, this is now implemented on the relevant julia-0.7 branches.

If you want to use Juno's progress bars without loading Juno.jl then something like this works:

for i in 1:10
  sleep(0.2)
  @debug "progress bar name" progress=i/10
end

I'd definitely recommend using @debug or Logging.@logmsg x with x<0 to not spam the repl if Juno isn't loaded.

Right now there's no REPL-only fallback, but that could probably be implemented in ProgressMeter.jl or a similar package (the API outlined above seems natural enough to me).

More detailed docs here.