Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relative recursive defaults take3 #1170

Merged
merged 91 commits into from
Jan 1, 2021
Merged
Changes from 1 commit
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
4664dfb
--info defaults to show defaults list information
omry Oct 31, 2020
36a53ba
testing loading of raw defaults list from configs
omry Nov 23, 2020
82a3ee7
remove package header manipulation in example config source
omry Nov 23, 2020
a198217
some basic basic defaults tree are passing
omry Nov 23, 2020
1f3c8ee
initial nested defaults test passed
omry Nov 24, 2020
e194f1b
added tests for get_config_path and get_group_path
omry Nov 24, 2020
b1713a7
append tests for defaults tree
omry Nov 24, 2020
4d794e4
simple group override tests for defaults tree
omry Nov 24, 2020
06e52f2
tree_to_list dfs implementation
omry Nov 25, 2020
7b245fa
Computing package in result defaults list
omry Nov 25, 2020
b064f25
testing defaults tree with pacakge override in defaults list
omry Nov 26, 2020
d8909d2
testing result list for default-list package override cases
omry Nov 26, 2020
fe0c9ba
Tests overriding group option with non default package
omry Nov 26, 2020
1b5bdca
test include_nested_config_item_global
omry Nov 27, 2020
38063b0
support key@ \(empty package\) in the overrides parser
omry Nov 27, 2020
cdfc3be
test include_nested_group_global_
omry Nov 27, 2020
70fda95
added test for _global_.foo package override
omry Nov 27, 2020
fde7195
reorg tests a bit
omry Nov 27, 2020
d771357
testing _name_ package override
omry Nov 27, 2020
6df6b99
Testing primary config with a package header
omry Nov 28, 2020
5458eca
testing include_nested_group_pkg_header_foo
omry Nov 28, 2020
12db5bd
testing include_nested_group_pkg_header_foo:override_first_level
omry Nov 28, 2020
4cbbe20
Test for include_nested_group_pkg_header_foo:override_first_level_wit…
omry Nov 28, 2020
0f3ddbc
fixed pacakge header to be absolute + tests
omry Nov 28, 2020
571cd7e
test with both config header and defaults list pkg override
omry Nov 28, 2020
9225beb
Marked override in defaults list - same level
omry Nov 29, 2020
7dfd9c5
Test for overriding nested config groups from parent
omry Nov 29, 2020
00be420
testing external override of overrides in defaults list
omry Nov 30, 2020
ea7c2e5
testing two_group_defaults_different_pkgs
omry Nov 30, 2020
f8c9934
Support for adding hydra/config node
omry Nov 30, 2020
1c83e4c
Handling legacy hydra overrides in defaults list
omry Nov 30, 2020
035e722
Lint passing
omry Dec 1, 2020
4786d60
Experinent use case tests
omry Dec 1, 2020
9c8057d
test_experiment_as_primary_config
omry Dec 1, 2020
4f6795f
Testing config extension use cases
omry Dec 1, 2020
bc69ee7
Testing for cases with MISSING ('???') default element
omry Dec 2, 2020
2d18fe0
Support for placeholder defaults list element
omry Dec 2, 2020
d82036b
Interpolation support
omry Dec 2, 2020
ba068d0
Override delete support (~group, ~group=choice)
omry Dec 3, 2020
ba97a50
lint
omry Dec 3, 2020
56bf0fb
updated all tests to pass
omry Dec 4, 2020
48bbab6
Error handling
omry Dec 4, 2020
a79e1c4
support for missing optional configs
omry Dec 9, 2020
6ef22dc
removed package rename support from overrides
omry Dec 9, 2020
7fdddeb
handling _group_ in package header
omry Dec 9, 2020
748694e
more consistent hanlding of _group_ packages
omry Dec 9, 2020
1135e31
overriding file with global header
omry Dec 9, 2020
a3e7fa2
Handling None primary config
omry Dec 9, 2020
6c670a8
handling defaults list with more than one config item (not config group)
omry Dec 9, 2020
1b500d0
fixed config loader tests
omry Dec 9, 2020
6c099a5
fixed some hydra tests
omry Dec 10, 2020
88e4e96
fixed another hydra test
omry Dec 10, 2020
757a771
fixed completion tests
omry Dec 10, 2020
9b2fbe6
fixed configuring hydra example tests
omry Dec 10, 2020
1e22ed8
handling missing config in defaults list
omry Dec 10, 2020
db74897
proper handling of case where defaults list has only overrides
omry Dec 10, 2020
a4dffca
implement --info defaults
omry Dec 11, 2020
06ffc25
proper handling of _group_ package header
omry Dec 11, 2020
4263a4e
updated todo
omry Dec 11, 2020
c70a55f
fixed core tests in nox
omry Dec 11, 2020
ddbc0f2
Fixed plugin tests, examples and docs to use override: true
omry Dec 11, 2020
0a53e12
Partial integration
omry Dec 9, 2020
b2a53a3
removed composition trace printout (redundant)
omry Dec 11, 2020
c493a0e
added --info defaults-tree
omry Dec 12, 2020
22f0c19
enforces that overrids are last in the defaults list
omry Dec 12, 2020
dcbadd4
Cleaning up previous defaults list implementation
omry Dec 12, 2020
dbe0fd8
fixed to not try to load MISSING defaults
omry Dec 13, 2020
87d86d0
cleaning up package handling in low level config sources - WIP
omry Dec 14, 2020
3dda8d9
removed is_primary_config from config loading apis
omry Dec 15, 2020
68ff784
Clean up pacakge specific logic from config repository
omry Dec 15, 2020
af9fcfb
fixed hydra version in standalone test apps
omry Dec 15, 2020
bd8746f
Fixed error message in invalid overrides in defaults list
omry Dec 15, 2020
b10772d
fixed handling of group with keyword names
omry Dec 15, 2020
7fd0895
Moved missing primary config error to defaults_list
omry Dec 15, 2020
14bdc29
minor configloaderimpl cleanup
omry Dec 15, 2020
372544f
Added primary flag to ResultDefault
omry Dec 15, 2020
a63a19d
added hydra.choices node
omry Dec 15, 2020
68f2682
change default _self_ to end of defaults
omry Dec 16, 2020
d0cb554
removed print # @package _global from --cfg output
omry Dec 24, 2020
775d372
disabled defaults list interpoaltion in non-primary configs
omry Dec 25, 2020
20e47cd
fixing error message for disallowed override in interpolated config
omry Dec 25, 2020
f92a075
fixed interpolation for Config items
omry Dec 28, 2020
29d3f23
Improve error message on validation error during composition
omry Dec 28, 2020
2b857b6
moved defaults list test data
omry Dec 29, 2020
299efb0
fixed error for invalid group override in nested config
omry Dec 29, 2020
f51390c
improve --info
omry Dec 29, 2020
98fb628
improve --hydra-help
omry Dec 29, 2020
b36ecb8
Documentation update
omry Dec 26, 2020
337ee1a
minor updates to error messages
omry Jan 1, 2021
f86a90e
news fragments updated
omry Jan 1, 2021
cbd055f
updated todos.
omry Dec 13, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Testing for cases with MISSING ('???') default element
omry committed Dec 25, 2020
commit bc69ee7653b6969a2789780f7bf6659eaa1aed7f
59 changes: 55 additions & 4 deletions hydra/_internal/new_defaults_list.py
Original file line number Diff line number Diff line change
@@ -146,6 +146,7 @@ def _expand_virtual_root(
repo=repo,
root=new_root,
is_primary_config=False,
skip_missing=False,
overrides=overrides,
)
if subtree.children is None:
@@ -159,10 +160,42 @@ def _expand_virtual_root(
return root


def _check_not_missing(
repo: IConfigRepository,
default: InputDefault,
skip_missing: bool,
) -> bool:
path = default.get_config_path()
if path.endswith("???"):
if skip_missing:
return True
if isinstance(default, GroupDefault):
group_path = default.get_group_path()
options = repo.get_group_options(
group_path,
results_filter=ObjectType.CONFIG,
)
opt_list = "\n".join(["\t" + x for x in options])
msg = dedent(
f"""\
You must specify '{group_path}', e.g, {group_path}=<OPTION>
Available options:
"""
)
raise ConfigCompositionException(msg + opt_list)
elif isinstance(default, ConfigDefault):
raise ValueError(f"Missing ConfigDefault is not supported : {path}")
else:
assert False

return False


def _create_defaults_tree(
repo: IConfigRepository,
root: DefaultsTreeNode,
is_primary_config: bool,
skip_missing: bool,
overrides: Overrides,
) -> DefaultsTreeNode:
parent = root.node
@@ -188,12 +221,14 @@ def _create_defaults_tree(
repo=repo, node=parent, is_primary_config=is_primary_config
)

path = parent.get_config_path()
if _check_not_missing(repo=repo, default=parent, skip_missing=skip_missing):
return root

path = parent.get_config_path()
loaded = repo.load_config(config_path=path, is_primary_config=is_primary_config)

if loaded is None:
missing_config_error(repo, root.node)
config_not_found_error(repo, root.node)

assert loaded is not None
defaults_list = copy.deepcopy(loaded.new_defaults_list)
@@ -245,6 +280,7 @@ def _create_defaults_tree(
repo=repo,
root=new_root,
is_primary_config=False,
skip_missing=skip_missing,
overrides=overrides,
)
if subtree.children is None:
@@ -319,6 +355,7 @@ def _create_defaults_list(
config_name: str,
overrides: Overrides,
prepend_hydra: bool,
skip_missing: bool,
) -> List[ResultDefault]:

root = _create_root(config_name=config_name, with_hydra=prepend_hydra)
@@ -328,6 +365,7 @@ def _create_defaults_list(
root=root,
overrides=overrides,
is_primary_config=True,
skip_missing=skip_missing,
)

output: List[ResultDefault] = []
@@ -341,18 +379,31 @@ def create_defaults_list(
config_name: str,
overrides_list: List[Override],
prepend_hydra: bool,
skip_missing: bool,
) -> DefaultsList:
"""
:param repo:
:param config_name:
:param overrides_list:
:param prepend_hydra:
:param skip_missing: True to skip config group with the value '???' and not fail on them. Useful when sweeping.
:return:
"""
overrides = Overrides(repo=repo, overrides_list=overrides_list)
defaults = _create_defaults_list(
repo, config_name, overrides, prepend_hydra=prepend_hydra
repo,
config_name,
overrides,
prepend_hydra=prepend_hydra,
skip_missing=skip_missing,
)
overrides.ensure_overrides_used()
ret = DefaultsList(defaults=defaults, config_overrides=overrides.config_overrides)
return ret


# TODO: show parent config name in the error (where is my error?)
def missing_config_error(repo: IConfigRepository, element: InputDefault) -> None:
def config_not_found_error(repo: IConfigRepository, element: InputDefault) -> None:
options = None
if isinstance(element, GroupDefault):
group = element.get_group_path()
5 changes: 4 additions & 1 deletion tests/defaults_list/__init__.py
Original file line number Diff line number Diff line change
@@ -23,19 +23,21 @@ def _test_defaults_tree_impl(
input_overrides: List[str],
expected: Any,
prepend_hydra: bool = False,
skip_missing: bool = False,
) -> None:
parser = OverridesParser.create()
repo = create_repo()
root = _create_root(config_name=config_name, with_hydra=prepend_hydra)
overrides_list = parser.parse_overrides(overrides=input_overrides)
overrides = Overrides(repo=repo, overrides_list=overrides_list)

if isinstance(expected, DefaultsTreeNode):
if expected is None or isinstance(expected, DefaultsTreeNode):
result = _create_defaults_tree(
repo=repo,
root=root,
overrides=overrides,
is_primary_config=True,
skip_missing=skip_missing,
)
overrides.ensure_overrides_used()
assert result == expected
@@ -46,5 +48,6 @@ def _test_defaults_tree_impl(
root=root,
overrides=overrides,
is_primary_config=True,
skip_missing=skip_missing,
)
overrides.ensure_overrides_used()
4 changes: 4 additions & 0 deletions tests/defaults_list/test_defaults_list.py
Original file line number Diff line number Diff line change
@@ -56,6 +56,7 @@
# - (Y) Extension from the same config group
# - (Y) Extension from absolute config group
# - (Y) Extension from a nested config group
# TODO: Test missing ('???') in Defaults List
# TODO: Interpolation support
# TODO: Consider delete support
# TODO: Consider package rename support
@@ -142,6 +143,7 @@ def _test_defaults_list_impl(
overrides: List[str],
expected: Any,
prepend_hydra: bool = False,
skip_missing: bool = False,
) -> None:
parser = OverridesParser.create()
repo = create_repo()
@@ -152,6 +154,7 @@ def _test_defaults_list_impl(
config_name=config_name,
overrides_list=overrides_list,
prepend_hydra=prepend_hydra,
skip_missing=skip_missing,
)
assert result.defaults == expected
else:
@@ -161,6 +164,7 @@ def _test_defaults_list_impl(
config_name=config_name,
overrides_list=overrides_list,
prepend_hydra=prepend_hydra,
skip_missing=skip_missing,
)


103 changes: 103 additions & 0 deletions tests/defaults_list/test_defaults_tree.py
Original file line number Diff line number Diff line change
@@ -1093,3 +1093,106 @@ def test_extension_use_cases(
input_overrides=overrides,
expected=expected,
)


@mark.parametrize( # type: ignore
"config_name,overrides,expected",
[
param(
"with_missing",
[],
raises(
ConfigCompositionException,
match=dedent(
"""\
You must specify 'db', e.g, db=<OPTION>
Available options:"""
),
),
id="with_missing",
),
param(
"with_missing",
["db=base_db"],
DefaultsTreeNode(
node=ConfigDefault(path="with_missing"),
children=[
ConfigDefault(path="_self_"),
GroupDefault(group="db", name="base_db"),
],
),
id="with_missing:override",
),
param(
"empty",
["+group1=with_missing"],
raises(
ConfigCompositionException,
match=dedent(
"""\
You must specify 'group1/group2', e.g, group1/group2=<OPTION>
Available options"""
),
),
id="nested_missing",
),
param(
"empty",
["+group1=with_missing", "group1/group2=file1"],
DefaultsTreeNode(
node=ConfigDefault(path="empty"),
children=[
ConfigDefault(path="_self_"),
DefaultsTreeNode(
node=GroupDefault(group="group1", name="with_missing"),
children=[
ConfigDefault(path="_self_"),
GroupDefault(group="group2", name="file1"),
],
),
],
),
id="nested_missing:override",
),
],
)
def test_with_missing(
config_name: str,
overrides: List[str],
expected: DefaultsTreeNode,
) -> None:
_test_defaults_tree_impl(
config_name=config_name,
input_overrides=overrides,
expected=expected,
)


@mark.parametrize( # type: ignore
"config_name,overrides,expected",
[
param(
"with_missing",
[],
DefaultsTreeNode(
node=ConfigDefault(path="with_missing"),
children=[
ConfigDefault(path="_self_"),
GroupDefault(group="db", name="???"),
],
),
id="with_missing",
),
],
)
def test_with_missing_and_skip_missing_flag(
config_name: str,
overrides: List[str],
expected: DefaultsTreeNode,
) -> None:
_test_defaults_tree_impl(
config_name=config_name,
input_overrides=overrides,
expected=expected,
skip_missing=True,
)
2 changes: 2 additions & 0 deletions tests/test_data/default_lists/group1/with_missing.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
defaults:
- group2: ???
2 changes: 2 additions & 0 deletions tests/test_data/default_lists/with_missing.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
defaults:
- db: ???