feat(data-classes): replace AttributeValue in DynamoDBStreamEvent with deserialized Python values

[shanab]

Issue number: #1432

Summary

Add DynamoDBImageDeserializer class to data classes. The class can be used to parse DynamoDB StreamRecord's old_image and new_image properties of type Dict[str, AttrValue] to Dict with Python types.

Changes

• Replace AttributeValue with deserialized Python types.

User experience

Users will be able to convert a StreamRecord's complex image type into a Python dictionary with Python types; without needing to recurse through the data, which was a limitation when using AttributeValue.

Please note that the following functions: new_image, old_image and keys will introduce breaking changes to users.

Old usage example:

@event_source(data_class=DynamoDBStreamEvent)
def lambda_handler(event: DynamoDBStreamEvent, context):
    for record in event.records:
        new_image: Dict[str, AttributeValue] = record.dynamodb.new_image
        event_type: AttributeValue = new_image["eventType"].get_value
        if event_type == "PENDING":
            data = get_image_data(new_image)
            send_to_sqs(data)

def get_image_data(image: Dict[str, AttributeValue]) -> Dict[str, Any]:
    # Note: the following line does not work with DynamoDB list and map types.
    # If such types are present, then one will have to recursively call `get_value`
    # to get to the underlying data.
    return {k: v.get_value for k, v in image.items()}

def send_to_sqs(data: Dict):
    body = json.dumps(data)
    # send to sqs

New usage example:

@event_source(data_class=DynamoDBStreamEvent)
def lambda_handler(event: DynamoDBStreamEvent, context):
    for record in event.records:
        new_image: Dict[str, Any] = record.dynamodb.new_image
        if new_image.get("eventType") == "PENDING":
            send_to_sqs(new_image)  # Here new_image is just a Python Dict type

def send_to_sqs(data: Dict[str, Any]):
    body = json.dumps(data)
    # send to sqs

Checklist

If your change doesn't seem to apply, please leave them unchecked.

• Meet tenets criteria
• I have performed a self-review of this change
• Changes have been tested
• Changes are documented
• PR title follows conventional commit semantics

Is this a breaking change?

Yes
RFC issue number:

Checklist:

• Migration process documented
• Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

TODOLIST

• Remove all references of AttributeValue
• Document with examples how new_image, old_image and keys are now different from how they used to work previously.

[heitorlessa]

peer review comments in

[heitorlessa]

Pushed the upgrade guide for this with a note on Decimal too.

[codecov-commenter]

Codecov Report

Base: 99.40% // Head: 99.36% // Decreases project coverage by -0.03% ⚠️

Coverage data is based on head (f40c044) compared to base (2864b46).
Patch coverage: 95.45% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##               v2    #1619      +/-   ##
==========================================
- Coverage   99.40%   99.36%   -0.04%     
==========================================
  Files         127      127              
  Lines        5842     5815      -27     
  Branches      367      370       +3     
==========================================
- Hits         5807     5778      -29     
- Misses         17       18       +1     
- Partials       18       19       +1     

Impacted Files	Coverage Δ
aws_lambda_powertools/utilities/batch/base.py	97.79% <ø> (ø)
...s/utilities/data_classes/dynamo_db_stream_event.py	98.13% <95.45%> (-1.87%)	⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[heitorlessa]

Added static typing for methods signatures and return, added last comments to help remember Decimal logic in the far future, and added a note about this being related to Batch Processing too.

Waiting for CI and we can merge!!!!

[boring-cyborg]

Awesome work, congrats on your first merged pull request and thank you for helping improve everyone's experience!