RayLauncher.setup() got an unexpected keyword argument 'hydra_context'

[alexcoca]

🐛 Bug

Description

Cannot launch application with hydra ray launcher.

Checklist

• [yes ] I checked on the latest version of Hydra
• [ n/a] I created a minimal repro (See this for tips).

To reproduce

** Minimal Code/Config snippet to reproduce **

** Stack trace/error message **

The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
  @hydra.main(config_name="config", config_path=get_config_path())
2023-07-21 18:19:50,081	WARNING __init__.py:25 -- Package pickle5 becomes unnecessary in Python 3.8 and above. Its presence may confuse libraries including Ray. Please uninstall the package.
Traceback (most recent call last):
  File "/Users/alexandrucoca/.local/share/virtualenvs/witchcraft-eKbweQs-/lib/python3.10/site-packages/hydra/_internal/utils.py", line 220, in run_and_report
    return func()
  File "/Users/alexandrucoca/.local/share/virtualenvs/witchcraft-eKbweQs-/lib/python3.10/site-packages/hydra/_internal/utils.py", line 466, in <lambda>
    lambda: hydra.multirun(
  File "/Users/alexandrucoca/.local/share/virtualenvs/witchcraft-eKbweQs-/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 153, in multirun
    sweeper = Plugins.instance().instantiate_sweeper(
  File "/Users/alexandrucoca/.local/share/virtualenvs/witchcraft-eKbweQs-/lib/python3.10/site-packages/hydra/core/plugins.py", line 140, in instantiate_sweeper
    sweeper.setup(
  File "/Users/alexandrucoca/.local/share/virtualenvs/witchcraft-eKbweQs-/lib/python3.10/site-packages/hydra/_internal/core_plugins/basic_sweeper.py", line 90, in setup
    self.launcher = Plugins.instance().instantiate_launcher(
  File "/Users/alexandrucoca/.local/share/virtualenvs/witchcraft-eKbweQs-/lib/python3.10/site-packages/hydra/core/plugins.py", line 157, in instantiate_launcher
    launcher.setup(
TypeError: RayLauncher.setup() got an unexpected keyword argument 'hydra_context'

Expected Behavior

System information

• Hydra Version : 1.3.2
• Python version : 3.10.11
• Virtual environment type and version :pipenv, version 2023.4.29
• Operating system : macOS

Additional context

The warning about pickle5 is weird given the plugin requires it

I tried several versions of hydra, I don't think it's an environment issue, but you never know!

[odelalleau]

Which version of hydra-ray-launcher are you using? It looks like this was fixed in 2021 (#1581) and latest launcher is from 2022 (https://pypi.org/project/hydra-ray-launcher/#history) so it should have this fix.

[alexcoca]

@odelalleau, strangely, when I pip install hydra-ray-launcher version 0.1.2 gets installed:

Successfully installed hydra-ray-launcher-0.1.2 ray-2.6.0

If I try to install a recent version of it, then I run into some trouble. For example, pip install hydra-ray-launcher==1.2.0 leads to

INFO: pip is looking at multiple versions of hydra-ray-launcher to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement ray[default]==1.12.0 (from hydra-ray-launcher) (from versions: 1.13.0, 2.0.0rc0, 2.0.0rc1, 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.3.0rc0, 2.3.0, 2.3.1, 2.4.0, 2.5.0, 2.5.1, 2.6.0)
ERROR: No matching distribution found for ray[default]==1.12.0

The dependency is indeed pinned to 1.12.0 in the launcher deps, but I do not understand why it cannot be resolved against the package index. Would the launcher not work with later ray versions? 1.13.0 seems to be identified.

update Could this be a python version issue? I see that 1.12.0 does not support python 3.10 - the first ray version that supports it seems to be 2.0.0rc0 according to pypi? Is there any chance to make the launcher compatible with python 3.10?

@odelalleau: Confirmed. I hacked the install and installed ray 1.13.0 and so I was able to install the hydra-ray-launcher==1.2.0 on python 3.10. How do we go about supporting the ray launcher on python 3.10? Bumping ray to at least 1.13.0 should work.

[odelalleau]

Yeah it sounds like the launcher's dependencies indeed need to be updated, thanks for the investigation.

[alexcoca]

Thank you so much for looking into this @odelalleau. Let me know if I can help any further! Please tag me when this is fixed and I can try and test the fix at my end.

[odelalleau]

Thank you so much for looking into this @odelalleau. Let me know if I can help any further! Please tag me when this is fixed and I can try and test the fix at my end.

Just to set expectations, I have no idea when/if someone will push a fix. I'm only a very lightweight contributor myself.

[alexcoca]

@Jasha10 , is there interest to fix this? I can try and make a contribution later on this autumn when I have a bit more time. On the face of it, it seems like a matter of figuring out the right sort of dependencies and bumping things up, but there may be more to it.

cc @omry