Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(idempotency): handle lambda timeout scenarios for INPROGRESS records #933

Merged
merged 9 commits into from
Dec 14, 2022

Conversation

jeromevdl
Copy link
Contributor

@jeromevdl jeromevdl commented Aug 5, 2022

Issue #, if available: aws-powertools/powertools-lambda-python#1038

Description of changes:

Using the same mechanism as python to handle functions timeout: see aws-powertools/powertools-lambda-python#1387, adding a timestamp in dynamoDB for the expected time of the function timeout. When the item is INPROGRESS, we check if this timestamp is in the past, to let the execution go and avoid blocking.

Adding integration tests
WIP: To test this feature properly, we need an integration test with real function timeout. I've started to implement integration tests, with the deployment of some infrastructure (based on python integration tests).
This is not finished (still have some error with log4j when running the function...).

Checklist

Breaking change checklist

RFC issue #:

  • Migration process documented
  • Implement warnings (if it can live side by side)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@jeromevdl
Copy link
Contributor Author

jeromevdl commented Oct 27, 2022

I removed the integration tests part, so that this PR focuses on the fix for idempotency.
I will work on another branch / PR for integration tests.

@msailes @rubenfonseca can you have a look at this one?

I still need to update the doc.

@jeromevdl jeromevdl marked this pull request as ready for review October 27, 2022 15:27
@rubenfonseca
Copy link
Contributor

I've checked the code and everything on the logic level made sense to me :) I'm just a little bit rust on the Java syntax 😅 Thanks @jeromevdl!

dependabot bot and others added 4 commits November 4, 2022 12:07
…ls#967)

Bumps `aws.sdk.version` from 2.18.4 to 2.18.7.

Updates `software.amazon.awssdk:bom` from 2.18.4 to 2.18.7
- [Release notes](https://github.com/aws/aws-sdk-java-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-java-v2/blob/master/CHANGELOG.md)
- [Commits](aws/aws-sdk-java-v2@2.18.4...2.18.7)

Updates `http-client-spi` from 2.18.4 to 2.18.7
- [Release notes](https://github.com/aws/aws-sdk-java-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-java-v2/blob/master/CHANGELOG.md)
- [Commits](aws/aws-sdk-java-v2@2.18.4...2.18.7)

Updates `url-connection-client` from 2.18.4 to 2.18.7

---
updated-dependencies:
- dependency-name: software.amazon.awssdk:bom
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:http-client-spi
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: software.amazon.awssdk:url-connection-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…rtools#960)

* add precedence for the envelope over built-in types

* cannot use same envelope for in and out

* remove extra new line

* handle event field names properly
ex: SQS Records with big R

* more explicit exception message
@pull-request-size pull-request-size bot added size/XL and removed size/L labels Nov 4, 2022
@pull-request-size pull-request-size bot added size/L and removed size/XL labels Nov 4, 2022
Copy link
Contributor

@msailes msailes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@msailes msailes merged commit d6115ac into aws-powertools:master Dec 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants