Skip to content

Commit

Permalink
fix(api): raise errors from modules disconnecting
Browse files Browse the repository at this point in the history
When a module gets disconnected, the hardware controller cleans up the
module instance, which stops the poller. What it doesn't do is cancel
anything that was waiting on the next poll, or in fact prevent new
things from waiting on the now-stopped poller. That means that if a
module disconnects
- During a module method waiting for the next poll, that module method
hangs (well, awaits) forever
- During a module method right _before_ waiting for the next poll, that
module method would start waiting and continue waiting forever

This PR forwards cancellations into the registered poll waiters when the
poller task is cancelled, and prevents the registration of new poll
waiters on a cancelled poll task.
  • Loading branch information
sfoster1 committed Feb 7, 2024
1 parent c77a53b commit 7e2ddfa
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions api/src/opentrons/hardware_control/poller.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import logging
from abc import ABC, abstractmethod
from typing import AsyncGenerator, List, Optional
from opentrons_shared_data.errors.exceptions import ModuleCommunicationError


log = logging.getLogger(__name__)
Expand Down Expand Up @@ -48,6 +49,8 @@ async def stop(self) -> None:
async with self._use_read_lock():
task.cancel()
await asyncio.gather(task, return_exceptions=True)
for waiter in self._poll_waiters:
waiter.cancel(msg="Module was removed")

async def wait_next_poll(self) -> None:
"""Wait for the next poll to complete.
Expand All @@ -56,6 +59,9 @@ async def wait_next_poll(self) -> None:
the next complete read. If a read raises an exception,
it will be passed through to `wait_next_poll`.
"""
if not self._poll_forever_task or self._poll_forever_task.done():
raise ModuleCommunicationError(message="Module was removed")

poll_future = asyncio.get_running_loop().create_future()
self._poll_waiters.append(poll_future)
await poll_future
Expand Down

0 comments on commit 7e2ddfa

Please sign in to comment.