Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG]: Dask cluster starts multiple times on launch #542

Closed
blythed opened this issue Jul 26, 2023 · 0 comments
Closed

[BUG]: Dask cluster starts multiple times on launch #542

blythed opened this issue Jul 26, 2023 · 0 comments
Labels
🐛 bug Something isn't working

Comments

@blythed
Copy link
Collaborator

blythed commented Jul 26, 2023 •
edited
Loading

Contact Details [Optional]

No response

System Information

  • Mac OSX
  • Python3.11

What happened?

Dask cluster was reinitiated several times.

Steps to reproduce

See notebook on my repo

Relevant log output

WARNING:root:Could not get an identifier from submitted function, creating one!
INFO:distributed.http.proxy:To route to workers diagnostics web server please install jupyter-server-proxy: python -m pip install jupyter-server-proxy
/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/node.py:182: UserWarning: Port 8787 is already in use.
Perhaps you already have a cluster running?
Hosting the HTTP server on port 62844 instead
  warnings.warn(
INFO:distributed.scheduler:State start
INFO:distributed.scheduler:  Scheduler at:     tcp://127.0.0.1:62845
INFO:distributed.scheduler:  dashboard at:  http://127.0.0.1:62844/status
2023-07-26 15:56:25,753 - distributed.nanny - ERROR - Failed to start process
Traceback (most recent call last):
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 432, in instantiate
    result = await self.process.start()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 720, in start
    await self.process.start()
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 55, in _call_and_set_future
    res = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 215, in _start
    process.start()
  File "/usr/local/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 118, in start
    assert not _current_process._config.get('daemon'), \
AssertionError: daemonic processes are not allowed to have children
ERROR:distributed.nanny:Failed to start process
Traceback (most recent call last):
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 432, in instantiate
    result = await self.process.start()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 720, in start
    await self.process.start()
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 55, in _call_and_set_future
    res = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 215, in _start
    process.start()
  File "/usr/local/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 118, in start
    assert not _current_process._config.get('daemon'), \
AssertionError: daemonic processes are not allowed to have children
2023-07-26 15:56:25,754 - distributed.nanny - ERROR - Failed to start process
Traceback (most recent call last):
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 432, in instantiate
    result = await self.process.start()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 720, in start
    await self.process.start()
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 55, in _call_and_set_future
    res = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 215, in _start
    process.start()
  File "/usr/local/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 118, in start
    assert not _current_process._config.get('daemon'), \
AssertionError: daemonic processes are not allowed to have children
ERROR:distributed.nanny:Failed to start process
Traceback (most recent call last):
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 432, in instantiate
    result = await self.process.start()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 720, in start
    await self.process.start()
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 55, in _call_and_set_future
    res = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 215, in _start
    process.start()
  File "/usr/local/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 118, in start
    assert not _current_process._config.get('daemon'), \
AssertionError: daemonic processes are not allowed to have children
2023-07-26 15:56:25,755 - distributed.nanny - ERROR - Failed to start process
Traceback (most recent call last):
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 432, in instantiate
    result = await self.process.start()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 720, in start
    await self.process.start()
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 55, in _call_and_set_future
    res = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 215, in _start
    process.start()
  File "/usr/local/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 118, in start
    assert not _current_process._config.get('daemon'), \
AssertionError: daemonic processes are not allowed to have children
ERROR:distributed.nanny:Failed to start process
Traceback (most recent call last):
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 432, in instantiate
    result = await self.process.start()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 720, in start
    await self.process.start()
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 55, in _call_and_set_future
    res = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 215, in _start
    process.start()
  File "/usr/local/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 118, in start
    assert not _current_process._config.get('daemon'), \
AssertionError: daemonic processes are not allowed to have children
2023-07-26 15:56:25,756 - distributed.nanny - ERROR - Failed to start process
Traceback (most recent call last):
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 432, in instantiate
    result = await self.process.start()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 720, in start
    await self.process.start()
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 55, in _call_and_set_future
    res = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 215, in _start
    process.start()
  File "/usr/local/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 118, in start
    assert not _current_process._config.get('daemon'), \
AssertionError: daemonic processes are not allowed to have children
ERROR:distributed.nanny:Failed to start process
Traceback (most recent call last):
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 432, in instantiate
    result = await self.process.start()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/nanny.py", line 720, in start
    await self.process.start()
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 55, in _call_and_set_future
    res = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dodo/SuperDuperDB/superduperdb/.venv/lib/python3.11/site-packages/distributed/process.py", line 215, in _start
    process.start()
  File "/usr/local/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 118, in start
    assert not _current_process._config.get('daemon'), \
AssertionError: daemonic processes are not allowed to have children
@blythed blythed added the 🐛 bug Something isn't working label Jul 26, 2023
@blythed blythed mentioned this issue Jul 26, 2023
Merged
3 tasks
@blythed blythed closed this as completed Jul 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
🐛 bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@blythed