Skip to content

Commit

Permalink
Do not attempt to connect to Nanny if in status==init
Browse files Browse the repository at this point in the history
  • Loading branch information
fjetter committed May 24, 2022
1 parent ba74495 commit e3ab594
Showing 1 changed file with 10 additions and 4 deletions.
14 changes: 10 additions & 4 deletions distributed/worker.py
Original file line number Diff line number Diff line change
Expand Up @@ -1450,13 +1450,19 @@ async def close(
# is the other way round. If an external caller wants to close
# nanny+worker, the nanny must be notified first. ==> Remove kwarg
# nanny, see also Scheduler.retire_workers
if self.status in (Status.closed, Status.closing):
if self.status in (Status.closed, Status.closing, Status.failed):
await self.finished()
return

if self.status not in WORKER_ANY_RUNNING:
# We may not be able to reliably communicate with the Nanny, so don't try.
# Otherwise, the Nanny RPC call may hang, especially during worker/nanny startup.
if self.status == Status.init:
# If the worker is still in startup/init and is started by a nanny,
# this means the nanny itself is not up, yet. If the Nanny isn't up,
# yet, it's server will not accept any incoming RPC requests and
# will block until the startup is finished.
# Therefore, this worker trying to communicate with the Nanny during
# startup is not possible and we cannot close it.
# In this case, the Nanny will automatically close after inspecting
# the worker status
nanny = False

disable_gc_diagnosis()
Expand Down

0 comments on commit e3ab594

Please sign in to comment.