CLI monitor not working

[JanisGailis]

Expected behavior

When invoking an operation that uses the monitor, the progress bar should fill in as the operation progresses and the percentage should be updated. When ctrl+c is pressed, the monitor should throw a Cancellation exception that can be caught by the operation such that a clean up is possible.

Actual behavior

The progress bar on CLI does not progress when an operation that implements a monitor is invoked. Also, the Cancellation exception is not raised on ctrl+c.

Steps to reproduce the problem

1. Checkout branch 291-jg-op-checklist

cate ws init
cate res open testset <local.something>
cate res set testset_monitored test_monitor ds=@testset

2. Observe the progress bar staying at 0%, with alive being printed every 2 seconds.
3. Press ctrl+c, observe alive still being printed.

Alternatively:

1. Run the uc06.sh script and observe how the monitor for LTA operation does not get updated.

Specifications

Cate development version on 291-jg-op-checklist branch (up to date with master)

[JanisGailis]

@mzuehlke, I have no idea at what point this has crept in, the monitor with LTA used to work just fine on CLI. Note that all tests run fine. I also tried accessing monitor._percentage for troubleshooting and got a stack trace complaining about ChildMonitor not having a _percentage. I also checked the ChildMonitor implementation, and it doesn't seem to raise the Cancellation event.

Hope this helps!

[mzuehlke]

@JanisGailis I can see progress and I can terminate the operation using the no_op operation:

cate res set foo no_op

marcoz ~/Scratch/cate/foo $ cate res set foo no_op
Computing nothing: [#########---------------------]  30% Step 3 of 10 doing nothing 

marcoz ~/Scratch/cate/foo $ cate res set foo no_op
Computing nothing:[##############################] 100% Step 10 of 10 doing nothing
Resource "foo" set.

When pressing CTRL+C I get a traceback, which I may have introduced when switching from the REST backend to the Websocket backend.

[JanisGailis]

Yes, no_op seems to be working for me as well.
Next question then. Is the method described in op development guide still valid when using monitors:
http://ect-core.readthedocs.io/en/latest/developer_guide/dg_operation_development.html#monitor-usage

That's how I'm using it in both LTA and this test_monitor thing.

[JanisGailis]

@mzuehlke I found the culprit.

In monitor.progress(number, string), the string is required. If it is not there, the monitor fails silently and also the ongoing operation can't be stopped with ctrl+c.

Please check out 291-jg-op-checklist, there are two operations in ops/test.py that demonstrate this. One called test_monitor_works and the other test_monitor_fails.

I'd like if the string is truly optional. If not, this must be documented somehow and the thing should fail with an exception.

Also, when doing ctrl+c with the working monitor, I guess the stack trace should not be there.

[JanisGailis]

@mzuehlke @forman It is kind of important for me to receive an answer on this one. If it will not be fixed, I'll go ahead and add some message strings to all monitors in all operations to make them work. Otherwise none of them can be cancelled from the CLI at the moment.

[JanisGailis]

The branch doesn't have the monitor testing operations. The bug can easily be recreated by modifying the no_op operation such that it doesn't have the message string in monitor.progress.

[forman]

Janis, I'll have a look asap

[forman]

CTRL+C on Windows doesn't give a stack trace but:

(C:\Users\Norman\Miniconda3\envs\cate) C:\Users\Norman\IdeaProjects\ccitools\cate-core>cate res set x no_op
Computing nothing: [###############---------------]  50% Step 5 of 10 doing nothing                                                                                                                                                               f
orrtl: error (200): program aborting due to control-C event
Image              PC                Routine            Line        Source
libifcoremd.dll    00007FF9556043E4  Unknown               Unknown  Unknown
KERNELBASE.dll     00007FF990DF8FBD  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FF994921FF4  Unknown               Unknown  Unknown
ntdll.dll          00007FF994A51F71  Unknown               Unknown  Unknown
forrtl: error (200): program aborting due to control-C event
Image              PC                Routine            Line        Source
libifcoremd.dll    00007FF9556043E4  Unknown               Unknown  Unknown
KERNELBASE.dll     00007FF990DF8FBD  Unknown               Unknown  Unknown
KERNEL32.DLL       00007FF994921FF4  Unknown               Unknown  Unknown
ntdll.dll          00007FF994A51F71  Unknown               Unknown  Unknown