Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OmegaConf.missing_keys to get a set of missing keys #719

Merged
merged 19 commits into from
Dec 17, 2021
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions docs/source/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -782,6 +782,24 @@ and ``OmegaConf.is_list(cfg)`` is equivalent to ``isinstance(cfg, ListConfig)``.
>>> assert not OmegaConf.is_dict(l)


OmegaConf.missing_keys
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Gives a set of missing keys represented in a dotlist style.
Ohad31415 marked this conversation as resolved.
Show resolved Hide resolved
This utility function can be used at the end of creating configuration, after merging sources and so on,
to check for remaining mandatory fields and prompt a proper error message.
Ohad31415 marked this conversation as resolved.
Show resolved Hide resolved

.. doctest::

>>> OmegaConf.missing_keys({
... "foo": {"bar": "???"},
... "missing": "???",
... "list": ["a", None, "???"]
... })
{'missing', 'list.2', 'foo.bar'}

A plain dict, list or any valid input for `OmegaConf.create` is also acceptable - the function would raise a `ValueError` exception on wrong input.

Jasha10 marked this conversation as resolved.
Show resolved Hide resolved

Debugger integration
--------------------

Expand Down
1 change: 1 addition & 0 deletions news/720.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
New `OmegaConf.missing_keys` method that enables to get a config's set of missing keys represented in a dotlist style.
Ohad31415 marked this conversation as resolved.
Show resolved Hide resolved
33 changes: 32 additions & 1 deletion omegaconf/omegaconf.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
Generator,
List,
Optional,
Set,
Tuple,
Type,
Union,
Expand Down Expand Up @@ -785,6 +786,37 @@ def resolve(cfg: Container) -> None:
)
omegaconf._impl._resolve(cfg)

@staticmethod
def missing_keys(cfg: Union[Container, Any]) -> Set[str]:
"""
Returns a set of missing keys flatten in a dotlist style.
:param cfg: An `OmegaConf.Container`,
or convertible object via `OmegaConf.create` (dict, list, ...).
:return: set of strings of the missing keys.
:raises ValueError: On input not representing a config.
"""
if not isinstance(cfg, Container):
try:
cfg = OmegaConf.create(cfg) # type: ignore
except ValidationError:
raise ValueError(f'Could not create a config out of {cfg}')
missings = set()

def gather(_cfg: Container, prefix: str = "") -> None:
if isinstance(_cfg, ListConfig):
itr = range(len(_cfg))
else:
itr = _cfg

for key in itr:
if OmegaConf.is_missing(_cfg, key):
missings.add(f"{prefix}{key}")
elif OmegaConf.is_config(_cfg[key]):
gather(_cfg[key], prefix=f"{prefix}{key}.")

gather(cfg)
return missings

# === private === #

@staticmethod
Expand Down Expand Up @@ -907,7 +939,6 @@ def flag_override(
names: Union[List[str], str],
values: Union[List[Optional[bool]], Optional[bool]],
) -> Generator[Node, None, None]:

if isinstance(names, str):
names = [names]
if values is None or isinstance(values, bool):
Expand Down
Loading