[BUG] RecursiveDictDiffer has a bug - TypeError: string indices must be integers #59017
Comments
|
|
ITJamie
changed the title
ITJamie
changed the title
|
@ITJamie The PR I submitted was for this function: https://github.com/saltstack/salt/blob/master/salt/utils/data.py#L1319 |
|
my bad!. I will indeed try that |
|
@github-abcde sadly i reached a new bug with your code. 🤣 dealing with restconf is slowly killing my soul 👎 |
|
This is caused by comparing differently structured data. |
sagetherage
added
severity-medium
|
@github-abcde do you have time to submit a PR on this? We will gladly review, thank you! |
|
I should have the change still lying around somewhere, will try to find time to submit it in a PR. |
|
Discovered this behavior regarding added/removed nested dictionaries as well. PoC: import salt.utils.dictdiffer as dd
initial = {
'foo': 'bar'
}
new_works = {
'foo': 'bar',
'baz': True
}
new_fails = {
'nested': {
'baz': True,
}
}
works = dd.recursive_diff(initial, new_works)
print(works.diffs)
print(works.added())
fails = dd.recursive_diff(initial, new_fails)
print(fails.diffs)
print(fails.added()){'baz': {'new': True, 'old': '<_null_>'}}
['baz']
{'nested': {'new': {'baz': True}, 'old': '<_null_>'}}
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "salt/salt/utils/dictdiffer.py", line 269, in added
return sorted(_added(self._diffs, prefix=""))
File "salt/salt/utils/dictdiffer.py", line 261, in _added
_added(
File "salt/salt/utils/dictdiffer.py", line 258, in _added
elif diffs[key]["old"] == self.NONE_VALUE:
TypeError: 'bool' object is not subscriptableKnowing about The first is just a slight modification and skips over nested dicts, only including the new parent key: class PatchedRecursiveDiffer(salt.utils.dictdiffer.RecursiveDictDiffer):
def added(self):
def _added(diffs, prefix):
keys = []
for key in diffs.keys():
if isinstance(diffs[key], dict):
if "old" not in diffs[key]:
keys.extend(_added(diffs[key], prefix="{}{}.".format(prefix, key)))
elif diffs[key]["old"] == self.NONE_VALUE:
keys.append("{}{}".format(prefix, key))
return keys
return sorted(_added(self._diffs, prefix=""))
def removed(self):
def _removed(diffs, prefix):
keys = []
for key in diffs.keys():
if isinstance(diffs[key], dict):
if "old" not in diffs[key]:
keys.extend(
_removed(diffs[key], prefix="{}{}.".format(prefix, key))
)
elif diffs[key]["new"] == self.NONE_VALUE:
keys.append("{}{}".format(prefix, key))
return keys
return sorted(_removed(self._diffs, prefix=""))Output using the same data as PoC for failing statement: ['nested']The second one is quite ugly, but includes the possibility to return nested keys as well: class PatchedRecursiveDiffer(salt.utils.dictdiffer.RecursiveDictDiffer):
def added(self, include_nested=False):
return sorted(self._it("old", "new", include_nested))
def removed(self, include_nested=False):
return sorted(self._it("new", "old", include_nested))
def _it(
self, key_a, key_b, include_nested=False, diffs=None, prefix="", is_nested=False
):
keys = []
if diffs is None:
diffs = self.diffs
for key in diffs.keys():
if is_nested:
keys.append("{}{}".format(prefix, key))
if not isinstance(diffs[key], dict):
continue
if is_nested:
keys.extend(
self._it(
key_a,
key_b,
diffs=diffs[key],
prefix="{}{}.".format(prefix, key),
is_nested=is_nested,
include_nested=include_nested,
)
)
elif "old" not in diffs[key]:
keys.extend(
self._it(
key_a,
key_b,
diffs=diffs[key],
prefix="{}{}.".format(prefix, key),
is_nested=is_nested,
include_nested=include_nested,
)
)
elif diffs[key][key_a] == self.NONE_VALUE:
keys.append("{}{}".format(prefix, key))
if isinstance(diffs[key][key_b], dict) and include_nested:
keys.extend(
self._it(
key_a,
key_b,
diffs=diffs[key][key_b],
is_nested=True,
prefix="{}{}.".format(prefix, key),
include_nested=include_nested,
)
)
return keysOutput using the same data as PoC for failing statement, passing ['nested']
['nested', 'nested.baz']This works for arbitrarily nested added/removed dictionaries. |
Description
It is possible to cause a python error with RecursiveDictDiffer with some simple example diffs
Setup
When migrating a new restconf module from a custom differ to the built in RecursiveDictDiffer I have managed to make RecursiveDictDiffer to throw a python error
Steps to Reproduce the behavior
test data (output from minon debug):
code:
Error:
appears when i try to access the .added() data
Expected behavior
A dict to be returned of the new keys added
Versions Report
The text was updated successfully, but these errors were encountered: