Merging configs unnecessarily tries to access base config interpolation

[1matthewli]

Describe the bug
When merging configs, OmegaConf seems to attempt to access interpolated values in the base config.

To Reproduce

base = OmegaConf.create({'a': '${a}'})
override = OmegaConf.create({'a': ''})
merged = OmegaConf.merge(base, override)

This produces the behavior:

RecursionError: maximum recursion depth exceeded while calling a Python object
	full_key: 
	reference_type=Optional[Dict[Union[str, Enum], Any]]
	object_type=dict

Expected behavior
The expected behavior is to produce the merged config {'a': ''}. I understand that the input config base may be considered as invalid, but it would be great if that did not stop the merging from working correctly.

Additional context

• OmegaConf version: 2.0.6
• Python version: 3.7.9
• Operating system : Ubuntu 16.04.7 LTS
• Please provide a minimal repro

[omry]

This is a feature.
Merging a dict into an interpolation to a dict creates a new node that is the merger of both.

In [9]:  OmegaConf.merge({"a" :"${b}", "b": {"a": 1}}, {"a": {"c": 3}})
Out[9]: {'a': {'a': 1, 'c': 3}, 'b': {'a': 1}}

[omry]

Your config is also broken. a node should not interpolate to itself.

[1matthewli]

I guess I just gave the most minimal example I could think of. Here is one that doesn't involve any broken configs:

base = OmegaConf.create({'a': '', 'b': '${a}'})
override = OmegaConf.create({'a': 'foo', 'nested': {'a': '${a}', 'b': '${a}'}})
OmegaConf.merge(base, override.nested)

[omry]

This kind of merge where you are merging things in different levels is a bad idea and is guaranteed to give you issues in various cases.

As I said, resolving the interpolation during merge is an intentional behavior.

Can you give some context on what you are trying to achieve here?

[omry]

Looking at your second example, you are merging

{'a': '${a}', 'b': '${a}'}

on

{'a': '', 'b': '${a}'}

and you are getting

{'a': '${a}', 'b': '${a}'}

This looks like a perfect interpretation to me.
Please explain what you are expecting to get and why.

Closing as invalid, feel free to follow up with more context.

[omry]

I now noticed that the version you tested against is 2.0.6.
Try this on 2.1.0 dev release (pip install omegaconf --pre --upgrade).

$ ipython
Python 3.8.5 (default, Sep  4 2020, 07:30:14) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.19.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from omegaconf import *

In [2]: __version__
Out[2]: '2.1.0dev21'

In [3]: base = OmegaConf.create({'a': '', 'b': '${a}'})
   ...: override = OmegaConf.create({'a': 'foo', 'nested': {'a': '${a}', 'b': '${a}'}})
   ...: OmegaConf.merge(base, override.nested)
Out[3]: {'a': '${a}', 'b': '${a}'}

[omry]

as a side note, your config is still broken because the result contains self interpolation.

[1matthewli]

Thanks, the 2.1.0 dev release works!

[1matthewli]

If you happen to know off the top of your head, just curious what changed that lets the merging succeed in the newer version despite the interpolation being invalid?

In [3]: base = OmegaConf.create({'a': '', 'b': '${a}'})
   ...: override = OmegaConf.create({'a': 'foo', 'nested': {'a': '${a}', 'b': '${a}'}})
   ...: OmegaConf.merge(base, override.nested)
Out[3]: {'a': '${a}', 'b': '${a}'}

[omry]

Many bugs have been fixed. Many of which are around the merge functionality.
This could be related : #486.