Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to launch #272

Merged
merged 12 commits into from
May 6, 2022
Merged

Updates to launch #272

merged 12 commits into from
May 6, 2022

Conversation

jgbos
Copy link
Contributor

@jgbos jgbos commented May 6, 2022

This PR does two things:

  • Adds flag to convert a dataclass input to a DictConfig. This is a fix and response to the cloudpickle issue
  • Refactors launch to comform with Hydra's Compose API. Currently we cannot use compose as it does not support muliturn, but these changes make it easy to update if compose eventually supports multirun.

@rsokl I added cloudpickle to some of the test requirements. I'm not sure how to test without it.

Comment on lines 215 to 220
if len(fields(config)) == 0:
raise ValueError(
"""There is an issue with your dataclass. If you previously executed with a
`hydra/launcher` that utilizes cloudpickle (e.g., hydra-submitit-launcher), there is a known
issue with dataclasses (see: https://github.com/cloudpipe/cloudpickle/issues/386). You will have
to restart your interactive environment. To avoid this issue you can use the option `to_dictconfig=True`."""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check will prevent empty configs from working:

from hydra_zen import make_config

from dataclasses import fields

>>> fields(make_config())
()

This would break some of our basic mushin workflow examples

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, when you get rid of this check, it would be good to add a regression test to ensure that launch always supports an empty config.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could check the length of the fields before and after the run, and then raise a warning to the user.

Copy link
Contributor Author

@jgbos jgbos May 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I was just thinking about that.

@jgbos jgbos requested a review from rsokl May 6, 2022 04:32
+ "`to_dictconfig=True`."
)
):
with pytest.warns(UserWarning):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahh, that's how you do that.

@rsokl rsokl merged commit 8e74625 into main May 6, 2022
@rsokl rsokl deleted the update-launch branch May 6, 2022 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants