Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Date strings are incorrectly returned as datetime objects after being stored and retrieved from Redis cache #2691

Closed
galvana opened this issue Feb 24, 2023 · 0 comments · Fixed by #2695
Assignees
Labels
bug Something isn't working

Comments

@galvana
Copy link
Contributor

galvana commented Feb 24, 2023

Bug Description

A date string such as 2001-11-08 is converted to a datetime object after being stored and retrieved from the Redis cache.

Steps to Reproduce

This is easily reproduced by this test

def test_datetime_string_stays_string(self):
        date = {"birthday": "2001-11-08"}
        decoded = FidesopsRedis.decode_obj(FidesopsRedis.encode_obj(date))
        assert decoded == date
AssertionError: assert {'birthday': datetime.datetime(2001, 11, 8, 0, 0)} == {'birthday': '2001-11-08'}

Root cause

The _custom_decoder in cache.py converts any date string to a datetime object after it is read from the cache. This change was introduced as part of #2643

def _custom_decoder(json_dict: Dict[str, Any]) -> Dict[str, Any]:
    for k, v in json_dict.items():
        try:
            json_dict[k] = datetime.fromisoformat(v)
            continue
        except (TypeError, ValueError):
            pass

Expected behavior

The object should be stored and retrieved without converting the date string to a datetime object.

Potential fix

Store any date string with a prefix like we do when storing bytes or ObjectId so that they are not converted to datetime objects after being read from the cache.

Additional context

This issue has the potential to affect SaaS erasure requests since we can't build a valid payload to update a collection after the access request data has been stored and retrieved from the cache.
TypeError: Object of type datetime is not JSON serializable

@galvana galvana added the bug Something isn't working label Feb 24, 2023
@sanders41 sanders41 self-assigned this Feb 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants