[Bug] Problem overriding variables with default values that interpolate hydra:job.config.

[shubham-goel]

🐛 Bug

Description

Hydra raises an error when trying to override variables with default values that interpolate hydra:job.config.

Checklist

• I checked on the latest version of Hydra
• I created a minimal repro (See this for tips).

To reproduce

test.py

import hydra
@hydra.main(config_name="test")
def main(cfg):
    print(cfg.a)
main()

test.yaml

a: ${hydra:job.config_name}

test1.yaml

defaults:
  - test
a: 1

python test.py --config-name=test1 causes an error, trace below. The error goes away when I change test.yaml to a: ${hydra.job.config_name} but then python test.py stops working.

Error trace:

$ python test.py --config-name=test1
Traceback (most recent call last):
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/_internal/utils.py", line 212, in run_and_report
    return func()
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/_internal/utils.py", line 365, in <lambda>
    lambda: hydra.run(
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 101, in run
    cfg = self.compose_config(
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 542, in compose_config
    cfg = self.config_loader.load_configuration(
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 150, in load_configuration
    return self._load_configuration_impl(
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 191, in _load_configuration_impl
    cfg, composition_trace = self._compose_config_from_defaults_list(
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 424, in _compose_config_from_defaults_list
    merged = OmegaConf.merge(cfg, loaded.config)
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 355, in merge
    target.merge_with(*others[1:])
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/basecontainer.py", line 427, in merge_with
    self._format_and_raise(key=None, value=None, cause=e)
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/base.py", line 107, in _format_and_raise
    format_and_raise(
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/_utils.py", line 678, in format_and_raise
    _raise(ex, cause)
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/_utils.py", line 588, in _raise
    raise ex  # set end OC_CAUSE=1 for full backtrace
ValueError: HydraConfig was not set
        full_key: job.config_name
        reference_type=Optional[Dict[Union[str, Enum], Any]]
        object_type=dict
        full_key: 
        reference_type=Optional[Dict[Union[str, Enum], Any]]
        object_type=dict

Setting OC_CAUSE=1:

$ OC_CAUSE=1 python test.py --config-name=test1
Traceback (most recent call last):
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/base.py", line 373, in _resolve_simple_interpolation
    value = resolver(root_node, inter_key)
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 388, in caching
    cache[key] if key in cache else resolver(*OmegaConf._tokenize_args(key))
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/core/utils.py", line 153, in <lambda>
    lambda path: OmegaConf.select(cast(DictConfig, HydraConfig.get()), path),
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/core/hydra_config.py", line 31, in get
    raise ValueError("HydraConfig was not set")
ValueError: HydraConfig was not set

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/basecontainer.py", line 425, in merge_with
    self._merge_with(*others)
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/basecontainer.py", line 447, in _merge_with
    BaseContainer._map_merge(self, other)
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/basecontainer.py", line 317, in _map_merge
    target_node = dest_node._dereference_node(
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/base.py", line 135, in _dereference_node
    v = parent._resolve_simple_interpolation(
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/base.py", line 382, in _resolve_simple_interpolation
    self._format_and_raise(key=inter_key, value=None, cause=e)
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/base.py", line 107, in _format_and_raise
    format_and_raise(
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/_utils.py", line 678, in format_and_raise
    _raise(ex, cause)
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/_utils.py", line 588, in _raise
    raise ex  # set end OC_CAUSE=1 for full backtrace
ValueError: HydraConfig was not set
        full_key: job.config_name
        reference_type=Optional[Dict[Union[str, Enum], Any]]
        object_type=dict

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/_internal/utils.py", line 212, in run_and_report
    return func()
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/_internal/utils.py", line 365, in <lambda>
    lambda: hydra.run(
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 101, in run
    cfg = self.compose_config(
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 542, in compose_config
    cfg = self.config_loader.load_configuration(
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 150, in load_configuration
    return self._load_configuration_impl(
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 191, in _load_configuration_impl
    cfg, composition_trace = self._compose_config_from_defaults_list(
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 424, in _compose_config_from_defaults_list
    merged = OmegaConf.merge(cfg, loaded.config)
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/omegaconf.py", line 355, in merge
    target.merge_with(*others[1:])
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/basecontainer.py", line 427, in merge_with
    self._format_and_raise(key=None, value=None, cause=e)
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/base.py", line 107, in _format_and_raise
    format_and_raise(
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/_utils.py", line 678, in format_and_raise
    _raise(ex, cause)
  File "/private/home/shubhamgoel/.conda/envs/py3d-implicit/lib/python3.8/site-packages/omegaconf/_utils.py", line 588, in _raise
    raise ex  # set end OC_CAUSE=1 for full backtrace
ValueError: HydraConfig was not set
        full_key: job.config_name
        reference_type=Optional[Dict[Union[str, Enum], Any]]
        object_type=dict
        full_key: 
        reference_type=Optional[Dict[Union[str, Enum], Any]]
        object_type=dict

Expected Behavior

Both python test.py and python test.py --config-name=test1 should work.

System information

• Hydra Version : 1.1.0dev1
• Python version : 3.8.5
• Virtual environment type and version : conda 4.7.10
• Operating system : Ubuntu 20.04.1 LTS

Additional context

Add any other context about the problem here.

[omry]

Thanks for reporting.
This should be fixing it: omry/omegaconf#487

Can you try to install OmegaConf from that branch and tell me if it solves the problem? (I did not try to reproduce it in Hydra because the cause of the problem was pretty clear to me).