-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
solved mpl plot for Mac and linux environments #5
Conversation
Not getting a MPL plot window in local ipython session. LiveTable is shown, but no LivePlot window. This code: from instrument.startup import *
from instrument.tests.sim_plans import *
RE(sim_print_plan())
RE(sim_count_plan())
RE(sim_rel_scan_plan()) |
@MDecarabas : Can you test with |
Rebuilt the conda environment per the instructions. Getting an ImportError from the main branch(model_instrument) jemian@otz ~/.../prjemian/model_instrument $ git checkout main
Switched to branch 'main'
Your branch is up to date with 'origin/main'.
(model_instrument) jemian@otz ~/.../prjemian/model_instrument $ python -c "from instrument.startup import *; print(f'{bec._plots_enabled=}')"
I Wed-16:18:43 - ############################################################ startup
I Wed-16:18:43 - logging started
I Wed-16:18:43 - logging level = 10
I Wed-16:18:43 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/_logging_setup.py
I Wed-16:18:43 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/aps_functions.py
I Wed-16:18:43 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/helper_functions.py
W Wed-16:18:43 - APS DM setup file does not exist: '/home/dm/etc/dm.setup.sh'
I Wed-16:18:44 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/controls_setup.py
I Wed-16:18:44 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/metadata.py
I Wed-16:18:44 - RunEngine metadata saved in directory: /home/beams/JEMIAN/.config/Bluesky_RunEngine_md
I Wed-16:18:44 - using ophyd control layer: 'pyepics'
bec._plots_enabled=False mpl branch(model_instrument) jemian@otz ~/.../prjemian/model_instrument $ git checkout mpl
Switched to branch 'mpl'
Your branch is up to date with 'origin/mpl'.
(model_instrument) jemian@otz ~/.../prjemian/model_instrument $ python -c "from instrument.startup import *; print(f'{bec._plots_enabled=}')"
I Wed-16:18:24 - ############################################################ startup
I Wed-16:18:24 - logging started
I Wed-16:18:24 - logging level = 10
I Wed-16:18:24 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/_logging_setup.py
I Wed-16:18:24 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/aps_functions.py
I Wed-16:18:24 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/helper_functions.py
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/startup.py", line 24, in <module>
from .core.best_effort_init import bec # noqa: F401
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/core/__init__.py", line 13, in <module>
mpl_setup()
File "/home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/helper_functions.py", line 71, in mpl_setup
plt.switch_backend("qtagg")
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/matplotlib/pyplot.py", line 415, in switch_backend
module = backend_registry.load_backend_module(newbackend)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/matplotlib/backends/registry.py", line 323, in load_backend_module
return importlib.import_module(module_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/matplotlib/backends/backend_qtagg.py", line 9, in <module>
from .qt_compat import QT_API, QtCore, QtGui
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/matplotlib/backends/qt_compat.py", line 130, in <module>
raise ImportError(
ImportError: Failed to import any of the following Qt binding modules: PyQt6, PySide6, PyQt5, PySide2 |
yes, works on my end. |
try again, should work now |
python(model_instrument) jemian@otz ~/.../prjemian/model_instrument $ python ./dev_simple.py
I Thu-12:50:31 - ############################################################ startup
I Thu-12:50:31 - logging started
I Thu-12:50:31 - logging level = 10
I Thu-12:50:31 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/_logging_setup.py
I Thu-12:50:31 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/aps_functions.py
I Thu-12:50:31 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/helper_functions.py
W Thu-12:50:31 - APS DM setup file does not exist: '/home/dm/etc/dm.setup.sh'
I Thu-12:50:32 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/controls_setup.py
I Thu-12:50:32 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/metadata.py
I Thu-12:50:32 - RunEngine metadata saved in directory: /home/beams/JEMIAN/.config/Bluesky_RunEngine_md
I Thu-12:50:32 - using ophyd control layer: 'pyepics'
(instrument.tests.sim_plans) This is a test.
(instrument.tests.sim_plans) motor.position=0 noisy_det.read()=OrderedDict({'noisy_det': {'value': 0.9880332265363881, 'timestamp': 1728582632.191969}}).
Transient Scan ID: 13 Time: 2024-10-10 12:50:35
Persistent Unique Scan ID: '3b0f52d3-abc9-4138-80b1-b8e4f1c1222e'
New stream: 'primary'
+-----------+------------+------------+
| seq_num | time | noisy_det |
+-----------+------------+------------+
| 1 | 12:50:36.0 | 9999.961 |
+-----------+------------+------------+
generator count ['3b0f52d3'] (scan num: 13)
(instrument.tests.sim_plans) motor.position=0 noisy_det.read()=OrderedDict({'noisy_det': {'value': 9999.960851437394, 'timestamp': 1728582636.0214987}}).
(instrument.tests.sim_plans) noisy_det.read_configuration()=OrderedDict({'noisy_det_Imax': {'value': 10000, 'timestamp': 1728582636.1461613}, 'noisy_det_center': {'value': 0, 'timestamp': 1728582636.1471736}, 'noisy_det_sigma': {'value': 1, 'timestamp': 1728582636.1480646}, 'noisy_det_noise': {'value': 'uniform', 'timestamp': 1728582636.149229}, 'noisy_det_noise_multiplier': {'value': 0.1, 'timestamp': 1728582632.1914954}}).
(instrument.tests.sim_plans) noisy_det.noise._enum_strs=('none', 'poisson', 'uniform').
Transient Scan ID: 14 Time: 2024-10-10 12:50:36
Persistent Unique Scan ID: 'be4a1fd2-8041-405c-bb2c-f0563f19470d'
New stream: 'primary'
/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/bluesky/callbacks/best_effort.py:211: UserWarning: Starting a Matplotlib GUI outside of the main thread will likely fail.
fig = self._fig_factory(fig_name)
+-----------+------------+------------+------------+
| seq_num | time | motor | noisy_det |
+-----------+------------+------------+------------+
| 1 | 12:50:44.1 | -2.500 | 439.336 |
| 2 | 12:50:44.1 | -2.000 | 1353.309 |
| 3 | 12:50:44.1 | -1.500 | 3246.435 |
| 4 | 12:50:44.1 | -1.000 | 6065.273 |
| 5 | 12:50:44.2 | -0.500 | 8824.952 |
| 6 | 12:50:44.2 | 0.000 | 9999.999 |
| 7 | 12:50:44.2 | 0.500 | 8824.962 |
| 8 | 12:50:44.2 | 1.000 | 6065.221 |
| 9 | 12:50:44.2 | 1.500 | 3246.552 |
| 10 | 12:50:44.2 | 2.000 | 1353.398 |
| 11 | 12:50:44.2 | 2.500 | 439.459 |
+-----------+------------+------------+------------+
generator rel_scan ['be4a1fd2'] (scan num: 14)
final
Exception ignored in atexit callback: <bound method Gcf.destroy_all of <class 'matplotlib._pylab_helpers.Gcf'>>
Traceback (most recent call last):
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/matplotlib/_pylab_helpers.py", line 81, in destroy_all
manager.destroy()
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/matplotlib/backends/_backend_tk.py", line 562, in destroy
self.canvas._tkcanvas.after_cancel(self.canvas._idle_draw_id)
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/tkinter/__init__.py", line 894, in after_cancel
data = self.tk.call('after', 'info', id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: main thread is not in main loop ipython(model_instrument) jemian@otz ~/.../prjemian/model_instrument $ ipython -i ./dev_simple.py
Python 3.12.7 | packaged by conda-forge | (main, Oct 4 2024, 16:05:46) [GCC 13.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.28.0 -- An enhanced Interactive Python. Type '?' for help.
Activating auto-logging. Current session state plus future input saved.
Filename : /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/.logs/ipython_console.log
Mode : rotate
Output logging : True
Raw input log : False
Timestamping : True
State : active
I Thu-12:51:02 - ############################################################ startup
I Thu-12:51:02 - logging started
I Thu-12:51:02 - logging level = 10
I Thu-12:51:02 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/_logging_setup.py
I Thu-12:51:02 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/aps_functions.py
I Thu-12:51:02 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/helper_functions.py
Exception reporting mode: Minimal
I Thu-12:51:02 - xmode exception level: 'Minimal'
W Thu-12:51:02 - APS DM setup file does not exist: '/home/dm/etc/dm.setup.sh'
I Thu-12:51:02 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/controls_setup.py
I Thu-12:51:03 - /home/beams1/JEMIAN/Documents/projects/prjemian/model_instrument/src/instrument/utils/metadata.py
I Thu-12:51:03 - RunEngine metadata saved in directory: /home/beams/JEMIAN/.config/Bluesky_RunEngine_md
I Thu-12:51:03 - using ophyd control layer: 'pyepics'
(instrument.tests.sim_plans) This is a test.
(instrument.tests.sim_plans) motor.position=0 noisy_det.read()=OrderedDict({'noisy_det': {'value': 1.0117847991417692, 'timestamp': 1728582662.9198499}}).
Transient Scan ID: 15 Time: 2024-10-10 12:51:03
Persistent Unique Scan ID: 'c4d5bc9a-15bb-47d7-a0d5-896e9745f799'
New stream: 'primary'
+-----------+------------+------------+
| seq_num | time | noisy_det |
+-----------+------------+------------+
| 1 | 12:51:03.6 | 9999.909 |
+-----------+------------+------------+
generator count ['c4d5bc9a'] (scan num: 15)
(instrument.tests.sim_plans) motor.position=0 noisy_det.read()=OrderedDict({'noisy_det': {'value': 9999.908617904832, 'timestamp': 1728582663.6753654}}).
(instrument.tests.sim_plans) noisy_det.read_configuration()=OrderedDict({'noisy_det_Imax': {'value': 10000, 'timestamp': 1728582663.8788345}, 'noisy_det_center': {'value': 0, 'timestamp': 1728582663.8800678}, 'noisy_det_sigma': {'value': 1, 'timestamp': 1728582663.8811967}, 'noisy_det_noise': {'value': 'uniform', 'timestamp': 1728582663.8823512}, 'noisy_det_noise_multiplier': {'value': 0.1, 'timestamp': 1728582662.919334}}).
(instrument.tests.sim_plans) noisy_det.noise._enum_strs=('none', 'poisson', 'uniform').
Transient Scan ID: 16 Time: 2024-10-10 12:51:03
Persistent Unique Scan ID: '08748bcf-4c3e-43f7-8c2c-3575960a2719'
New stream: 'primary'
/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/bluesky/callbacks/best_effort.py:211: UserWarning: Starting a Matplotlib GUI outside of the main thread will likely fail.
fig = self._fig_factory(fig_name)
+-----------+------------+------------+------------+
| seq_num | time | motor | noisy_det |
+-----------+------------+------------+------------+
| 1 | 12:51:11.0 | -2.500 | 439.441 |
| 2 | 12:51:11.0 | -2.000 | 1353.450 |
| 3 | 12:51:11.0 | -1.500 | 3246.506 |
| 4 | 12:51:11.0 | -1.000 | 6065.234 |
| 5 | 12:51:11.0 | -0.500 | 8825.039 |
| 6 | 12:51:11.0 | 0.000 | 9999.947 |
| 7 | 12:51:11.0 | 0.500 | 8824.943 |
| 8 | 12:51:11.0 | 1.000 | 6065.373 |
| 9 | 12:51:11.0 | 1.500 | 3246.491 |
| 10 | 12:51:11.0 | 2.000 | 1353.384 |
| 11 | 12:51:11.1 | 2.500 | 439.272 |
+-----------+------------+------------+------------+
generator rel_scan ['08748bcf'] (scan num: 16)
final
In [1]: Traceback (most recent call last):
File "/home/beams/JEMIAN/.conda/envs/model_instrument/bin/ipython", line 8, in <module>
sys.exit(start_ipython())
^^^^^^^^^^^^^^^
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/IPython/__init__.py", line 130, in start_ipython
return launch_new_instance(argv=argv, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/traitlets/config/application.py", line 1075, in launch_instance
app.start()
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/IPython/terminal/ipapp.py", line 317, in start
self.shell.mainloop()
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/IPython/terminal/interactiveshell.py", line 917, in mainloop
self.interact()
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/IPython/terminal/interactiveshell.py", line 902, in interact
code = self.prompt_for_code()
^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/IPython/terminal/interactiveshell.py", line 845, in prompt_for_code
text = self.pt_app.prompt(
^^^^^^^^^^^^^^^^^^^
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/prompt_toolkit/shortcuts/prompt.py", line 1035, in prompt
return self.app.run(
^^^^^^^^^^^^^
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/prompt_toolkit/application/application.py", line 978, in run
result = loop.run_until_complete(coro)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete
self.run_forever()
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/asyncio/base_events.py", line 641, in run_forever
self._run_once()
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/asyncio/base_events.py", line 1948, in _run_once
event_list = self._selector.select(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/prompt_toolkit/eventloop/inputhook.py", line 150, in select
self.inputhook(InputHookContext(self._r, input_is_ready))
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/IPython/terminal/pt_inputhooks/tk.py", line 88, in inputhook
wait_using_filehandler()
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/IPython/terminal/pt_inputhooks/tk.py", line 65, in wait_using_filehandler
root.createfilehandler(inputhook_context.fileno(), _tkinter.READABLE, done)
RuntimeError: Calling Tcl from different apartment
If you suspect this is an IPython 8.28.0 bug, please report it at:
https://github.com/ipython/ipython/issues
or send an email to the mailing list at ipython-dev@python.org
You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.
Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
%config Application.verbose_crash=True
Exception ignored in atexit callback: <bound method Gcf.destroy_all of <class 'matplotlib._pylab_helpers.Gcf'>>
Traceback (most recent call last):
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/matplotlib/_pylab_helpers.py", line 81, in destroy_all
manager.destroy()
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/matplotlib/backends/_backend_tk.py", line 562, in destroy
self.canvas._tkcanvas.after_cancel(self.canvas._idle_draw_id)
File "/home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/tkinter/__init__.py", line 894, in after_cancel
data = self.tk.call('after', 'info', id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: main thread is not in main loop |
Is tkagg suitable? This error is shown above.
|
APS workstation, ipython with these commands: from instrument.startup import *
from instrument.tests.sim_plans import *
RE(sim_print_plan())
RE(sim_count_plan())
RE(sim_rel_scan_plan()) this error /home/beams/JEMIAN/.conda/envs/model_instrument/lib/python3.12/site-packages/bluesky/callbacks/best_effort.py:211: UserWarning: Starting a Matplotlib GUI outside of the main thread will likely fail. More testing on this later. Includes: re-install the environment per the installation instructions. |
Still no problems on home workstation. |
Same problems with fresh install on usaxscontrol. May be related to how session is started. My session starts from Windows Mobaxterm, ssh -X login to otz. |
Tried different login paths:
All fail identically with ipython. |
And, each time it fails, the terminal stops echoing keypresses to the console. Other output artefacts also happen. Have to create a new window to continue. |
I'm looking at a temporary change to import logging
def configure_logging():
"""Temporary logging setup."""
brief_date = "%a-%H:%M:%S"
brief_format = "%(levelname)-.1s %(asctime)s: %(message)s"
logging.basicConfig(
encoding="utf-8",
level=logging.INFO,
format=brief_format,
datefmt=brief_date,
)
logger = logging.getLogger(__name__)
logger.info(__file__)
# calm logging in these other modules
for module in "bluesky databroker ophyd".split():
logging.getLogger(module).setLevel(logging.WARNING) This logs only to the console, but it should help focus on the MPL initialization. |
With this change, this branch works for me. How about your end, too? |
This work was advised by https://docs.python.org/3/howto/logging.html |
Failed on my home workstation.
|
Er, that was before pulling latest changes ... |
After pulling latest changes, fails differently:
The Qt libraries are binary code. Can't see how pip can install them properly. |
I have no idea in that case why it keeps failing on your end. My linux workstation and MacBook are both running fine off of the code. Can you try the below commands in an ipython session without running the instrument package:
|
When pyqt is installed by pip
When pyqt is installed by conda
One big difference is that pip does not install |
Testing with the conda-installed "pyqt>5.15" worked for python test. |
I think when you pip installed you didn't have the latest toml version which contains such a requirement |
jupyter and ipython, good, too, all that's left is to remove the Python pin |
Left the lock and cited reason as it fails to build Pyarrow. It is now pinned to 3.11 |
Ah, one of the pip packages requires ... requires pyarrow. On what OS does pyarrow fail to build? I just installed with no python pin and got this:
|
Hard to relax that python pin if a pip install might try something different. Should the pyarrow (or the package that requires it) be installed by conda so the requirements can be matched? That's really the point of the pin, as I understand it. |
PyArrow's PyPI page states:
That's where conda install is superior to pip. |
We could go with the pin(s) for now, but sometime we need to discard the notion that everything can or should be installed with pip. That notion imposes limitations such as pinning Python to a specific version so that pip installs will succeed. |
FWIW: local QS test was success |
I disagree, but that can be a separate branch. If all is good please approve and then I can merge it in |
I'm not happy adding pins that do something already accomplished by package management software. We must not start that. Clearly conda can resolve the package requirements and it is not necessary to pin to a fixed version. State your technical reasons. |
Pinning makes sure that when the package is deployed, it is done so with the exact set of dependencies the team tested with. Bugs will still happen, but it’s much more reliable and consistent than using loose versioning, where the dependencies can vary and cause unexpected issues. As was shown above in Pyarrow not being able to be resolved. |
According to the pip installation record above, bluesky-queueserver requires pyarrow:
Asked about this on Mattermost. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested at APS and home linux workstations with python, ipython, jupyter notebook, and queueserver. All tests pass.
@@ -47,6 +47,7 @@ dependencies = [ | |||
"ophyd", | |||
"pysumreg", | |||
"qtpy", | |||
"PyQt5>5.15", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
any version of pyqt5, pyqt6, pyside2, or pyside6 should work (and if it does not it is a bug in bluesky).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We're using PyQt5 in other places in our local work. Is any part of bluesky using Qt6?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is a site-level choice, in bluesky/bluesky#1512 we got it to work and had a test, but I'm not sure if the pyqt6 dep survived a testing refactor...
Title