simple fastplotlib example

[kushalkolar]

very simple example with fastplotlib to show updating frames with randomly generated data.

Closes #85

The jupyterlab server is running on my lab workstation but I'm connecting to it from home just through my browser so the remote frame buffer is being streamed over the internet.

improv-fpl-2023-03-18_01.48.16.mp4

Basically:

Generator: 512 x 512 random frames -> queue -> Processor, sends zmq message

jupyter notebook: receives zmq message, updates image graphic in fastplotlib

https://github.com/kushalkolar/improv/blob/fastplotlib-demo/demos/fastplotlib/fastplotlib.ipynb

[jmxpearson]

This is really cool! I'm going to ask that we wait on merging this until @draelos finishes her review and we merge tui into dev. At that point, we should only need to change the way the demo is invoked. Should be doable in the next week, I think.

I will also want us to think about how we write a test for this. I don't want/need full coverage for demos, but it would be good to have ways to check we aren't breaking them in future, especially since lots of people may want this feature.

[kushalkolar]

Sure! I mostly wanted to make sure that this is the right way to approach sending data to notebooks.

Looking into testing notebooks, this looks like it should work: https://github.com/treebeardtech/nbmake

I'll have to make sure the tests actually execute in the notebook, some of the other jupyter notebook testing options just extract the code and run it in a regular python kernel. When wgpu-py doesn't detect that it's in a notebook context it will start a glfw or Qt event loop to run the plot as a desktop application.

[jmxpearson]

@kushalkolar Can you try this again with TUI merged? That way I can see the status of tests?

[kushalkolar]

I'm running into this with improv run ./fastplotlib.yaml:

(mescore) kushalk@hantman-calcium:~/repos/improv/demos/fastplotlib (kushal-onacid-fpl)
└─ $ ▶ improv run ./fastplotlib.ipynb 
Traceback (most recent call last):
  File "/home/kushalk/python-venvs/mescore/bin/improv", line 8, in <module>
    sys.exit(default_invocation())
  File "/home/kushalk/repos/improv/improv/cli.py", line 100, in default_invocation
    args.func(args)
  File "/home/kushalk/repos/improv/improv/cli.py", line 213, in run
    args.logging_port = logging_port
UnboundLocalError: local variable 'logging_port' referenced before assignment
/usr/local/lib/python3.10/subprocess.py:1070: ResourceWarning: subprocess 874199 is still running
  _warn("subprocess %s is still running" % self.pid,

with improv server I get:

(mescore) kushalk@hantman-calcium:~/repos/improv/demos/fastplotlib (kushal-onacid-fpl)
└─ $ ▶ improv server ./fastplotlib.ipynb 
Traceback (most recent call last):
  File "/home/kushalk/python-venvs/mescore/bin/improv", line 8, in <module>
    sys.exit(default_invocation())
  File "/home/kushalk/repos/improv/improv/cli.py", line 100, in default_invocation
    args.func(args)
  File "/home/kushalk/repos/improv/improv/cli.py", line 127, in run_server
    control_port, output_port = server.createNexus(file=args.configfile, control_port=args.control_port, output_port=args.output_port)
  File "/home/kushalk/repos/improv/improv/nexus.py", line 82, in createNexus
    self.loadConfig(file=file)
  File "/home/kushalk/repos/improv/improv/nexus.py", line 105, in loadConfig
    self.config.createConfig()
  File "/home/kushalk/repos/improv/improv/config.py", line 48, in createConfig
    for name,actor in cfg['actors'].items():
KeyError: 'actors'
sys:1: ResourceWarning: Unclosed socket <zmq.asyncio.Socket(zmq.PUB) at 0x7f8a09012e00>
sys:1: ResourceWarning: Unclosed socket <zmq.asyncio.Socket(zmq.REP) at 0x7f8a09012f80>
/usr/local/lib/python3.10/subprocess.py:1070: ResourceWarning: subprocess 874537 is still running
  _warn("subprocess %s is still running" % self.pid,

[jmxpearson]

I think it's #89. Check the log to see if you have an actor crashing.

The proximate cause is that the client is timing out waiting on the server. @draelos saw this in a case where an actor was erroring and the server never wrote its ports to the log.

[draelos]

I'm not sure calling improv server on a .ipynb file works anyway? But the first error is likely due to what John mentioned. The file global.log should have more traceback.

[jmxpearson]

Correct. You’d need to do  ``` improv server <config yaml> ``` and then use `zmq` inside the notebook (connecting on the ports the server outputs) to send the `setup`, `run`, etc. commands.

[kushalkolar]

This actually works, I think I had a weird character in my yaml config file on my end.

[kushalkolar]

continued in #104