Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: LargeMessageIdempotentE2ET Flaky #1518

Merged
merged 1 commit into from
Nov 17, 2023
Merged

fix: LargeMessageIdempotentE2ET Flaky #1518

merged 1 commit into from
Nov 17, 2023

Conversation

scottgerring
Copy link
Contributor

Issue #, if available:
n/a

Description of changes:

The test fails intermittently. If you look at the state of the world after the test fails, the data it is looking for in the DynamoDB table is there. This indicates that the test is failing because we are not giving the Lambda enough time to complete.

Bumping out timing magic numbers is not very satisfying but I can't see another way of doing this.

Checklist

Breaking change checklist

RFC issue #:

  • Migration process documented
  • Implement warnings (if it can live side by side)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@scottgerring scottgerring changed the title Fix: LargeMessageIdempotentE2ET Flaky fix: LargeMessageIdempotentE2ET Flaky Nov 17, 2023
Copy link

sonarcloud bot commented Nov 17, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

Copy link

💾 Artifacts Size Report

Module Version Size (KB)
powertools-core 1.19.0-SNAPSHOT 9.95
powertools-serialization 1.19.0-SNAPSHOT 17.93
powertools-logging 1.19.0-SNAPSHOT 45.83
powertools-tracing 1.19.0-SNAPSHOT 13.44
powertools-sqs 1.19.0-SNAPSHOT 28.99
powertools-metrics 1.19.0-SNAPSHOT 14.30
powertools-parameters 1.19.0-SNAPSHOT 41.07
powertools-validation 1.19.0-SNAPSHOT 18.90
powertools-cloudformation 1.19.0-SNAPSHOT 16.83
powertools-idempotency 1.19.0-SNAPSHOT 43.48
powertools-large-messages 1.19.0-SNAPSHOT 17.64
powertools-batch 1.19.0-SNAPSHOT 16.89

@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (2602ea7) 79.38% compared to head (c7754cc) 79.49%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1518      +/-   ##
============================================
+ Coverage     79.38%   79.49%   +0.10%     
  Complexity      755      755              
============================================
  Files            93       93              
  Lines          2877     2877              
  Branches        307      307              
============================================
+ Hits           2284     2287       +3     
+ Misses          494      492       -2     
+ Partials         99       98       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@scottgerring scottgerring marked this pull request as ready for review November 17, 2023 12:45
Copy link
Contributor

@jeromevdl jeromevdl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flacky / not flacky, that is the question, but let's go!

@jeromevdl jeromevdl merged commit 0c9b9ab into main Nov 17, 2023
13 checks passed
@jeromevdl jeromevdl deleted the fix-e2e-test-timing branch November 17, 2023 12:46
scottgerring added a commit that referenced this pull request Dec 5, 2023
* Setting up Kotlin environment. Converting test to Kotlin.

* Deploying via SAM successfully.

* Added Kotlin example.

* Removing unused Gradle build file.

* Adding SAM template so can be used as an existing project and Java target compatibility

* Adding SAM template so can be used as an existing project

* Updating guidance to use SAM for build and deploy

* Restructuring separate Java and Kotlin examples.

* Updating core examples readme to represent new structure for Java and Kotlin examples.

* Refactoring application code for efficiency, updating build to cover tests too and is more idiomatic and readme to be more descriptive

* Updating to fix trailing \n

* Updating guidance to be more specific for examples

* Adopting new mechanism for specifying jvm target.

* accommodating new project structure

* Fixing link typo after refactoring

* Setting up Kotlin environment. Converting test to Kotlin.

* Deploying via SAM successfully.

* Added Kotlin example.

* Removing unused Gradle build file.

* Adding SAM template so can be used as an existing project and Java target compatibility

* Adding SAM template so can be used as an existing project

* Updating guidance to use SAM for build and deploy

* Restructuring separate Java and Kotlin examples.

* Updating core examples readme to represent new structure for Java and Kotlin examples.

* Refactoring application code for efficiency, updating build to cover tests too and is more idiomatic and readme to be more descriptive

* Updating to fix trailing \n

* Updating guidance to be more specific for examples

* Adopting new mechanism for specifying jvm target.

* accommodating new project structure

* Fixing link typo after refactoring

* Flattening structure back to original to make merging easier for v2

* Adding build for Kotlin Gradle

* Adding build for Kotlin Gradle - Restructuring Java examples to v1 approach

* Correcting paths

* Adding SNAPSHOT support and local capability for Maven. Testing using Java 1.8

* Reviewed and updated against PR comments.

* Un-commenting examples

* Adding validation step for IaC SAM

* Adding Terraform for Java projects IaC validator and linter

* Adding additional projects for SAM validation and matrix approach

* Refactoring stream function to process input logging example with a Lambda Function URL instead of APIGW.

* Demonstrating Java streaming response

* Refactoring stream function to process input logging example to return

* Update CONTRIBUTING.md

* fix: get trace id from system property when env var is not set (#1503)

* fix: check if XRAY Trace ID is present in System property

* chore: remove erroneous extra char in tests

* fix #1500 (#1506)

* feat: Add support for POWERTOOLS_LOGGER_LOG_EVENT (#1510)

* chore: Addition of Warn Message If Invalid Annotation Key While Tracing #1511 (#1512)

* feat: ALC (#1514)

* handle AWS_LAMBDA_LOG configuration

* ALC documentation + code review

* update doc

* chore:Prep release 1.18.0 (#1515)

* chore:prep release 1.18.0

* update version

* update version in kotlin example

* maven local repo in gradle example

* update changelog

---------

Co-authored-by: scottgerring <[email protected]>

* chore: update version to next snapshot: 1-19.0-SNAPSHOT (#1516)

* update version to next snapshot: 1-19.0-SNAPSHOT

* update version to next snapshot: 1-19.0-SNAPSHOT

* update version to next snapshot: 1-19.0-SNAPSHOT

* building only for LTS

* Add some more margin to the test pause (#1518)

* test: e2e tests with java 21 (#1517)

* e2e tests with java 21

* Run Java21 tests using the Java17 compiler

* Run all of the E2E tests in parallel, not just the first 3

* Try again

* .

* Let's try again

* Add some comment on Java21 to the repo

* Add caveat about lambda runtimes

* Clean up wording a little

---------

Co-authored-by: Scott Gerring <[email protected]>

* update doc for ALC (#1520)

* chore: Testing java21 aspectj pre-release (#1519)

* e2e tests with java 21

* use aspectj 1.9.21-SNAPSHOT

* Fix log4j2.xml missing in logging test for java21

* rollback double runtime

* remove comment

* keep aspectj 1.9.7 in parent for java8 compatibility

* use M1 instead of snapshot

* update documentation for aspectj

* update documentation for aspectj

---------

Co-authored-by: Jerome Van Der Linden <[email protected]>

* chore: Remove build cruft

* Adding context for using RequestStreamHandler

* removing pr_lint

* Update examples/powertools-examples-core/sam/src/main/java/helloworld/AppStream.java

Clarify usage of RequestStreamHandler.

Co-authored-by: Alexey Soshin <[email protected]>

---------

Co-authored-by: Jason Harris <[email protected]>
Co-authored-by: Scott Gerring <[email protected]>
Co-authored-by: Jason Harris <[email protected]>
Co-authored-by: Jérôme Van Der Linden <[email protected]>
Co-authored-by: Michele Ricciardi <[email protected]>
Co-authored-by: Alexey Soshin <[email protected]>
Co-authored-by: jdoherty <[email protected]>
Co-authored-by: Scott Gerring <[email protected]>
Co-authored-by: Jerome Van Der Linden <[email protected]>
scottgerring added a commit that referenced this pull request Feb 22, 2024
* docs: HelloWorldStreamFunction in examples fails with sam (#1532)

* Setting up Kotlin environment. Converting test to Kotlin.

* Deploying via SAM successfully.

* Added Kotlin example.

* Removing unused Gradle build file.

* Adding SAM template so can be used as an existing project and Java target compatibility

* Adding SAM template so can be used as an existing project

* Updating guidance to use SAM for build and deploy

* Restructuring separate Java and Kotlin examples.

* Updating core examples readme to represent new structure for Java and Kotlin examples.

* Refactoring application code for efficiency, updating build to cover tests too and is more idiomatic and readme to be more descriptive

* Updating to fix trailing \n

* Updating guidance to be more specific for examples

* Adopting new mechanism for specifying jvm target.

* accommodating new project structure

* Fixing link typo after refactoring

* Setting up Kotlin environment. Converting test to Kotlin.

* Deploying via SAM successfully.

* Added Kotlin example.

* Removing unused Gradle build file.

* Adding SAM template so can be used as an existing project and Java target compatibility

* Adding SAM template so can be used as an existing project

* Updating guidance to use SAM for build and deploy

* Restructuring separate Java and Kotlin examples.

* Updating core examples readme to represent new structure for Java and Kotlin examples.

* Refactoring application code for efficiency, updating build to cover tests too and is more idiomatic and readme to be more descriptive

* Updating to fix trailing \n

* Updating guidance to be more specific for examples

* Adopting new mechanism for specifying jvm target.

* accommodating new project structure

* Fixing link typo after refactoring

* Flattening structure back to original to make merging easier for v2

* Adding build for Kotlin Gradle

* Adding build for Kotlin Gradle - Restructuring Java examples to v1 approach

* Correcting paths

* Adding SNAPSHOT support and local capability for Maven. Testing using Java 1.8

* Reviewed and updated against PR comments.

* Un-commenting examples

* Adding validation step for IaC SAM

* Adding Terraform for Java projects IaC validator and linter

* Adding additional projects for SAM validation and matrix approach

* Refactoring stream function to process input logging example with a Lambda Function URL instead of APIGW.

* Demonstrating Java streaming response

* Refactoring stream function to process input logging example to return

* Update CONTRIBUTING.md

* fix: get trace id from system property when env var is not set (#1503)

* fix: check if XRAY Trace ID is present in System property

* chore: remove erroneous extra char in tests

* fix #1500 (#1506)

* feat: Add support for POWERTOOLS_LOGGER_LOG_EVENT (#1510)

* chore: Addition of Warn Message If Invalid Annotation Key While Tracing #1511 (#1512)

* feat: ALC (#1514)

* handle AWS_LAMBDA_LOG configuration

* ALC documentation + code review

* update doc

* chore:Prep release 1.18.0 (#1515)

* chore:prep release 1.18.0

* update version

* update version in kotlin example

* maven local repo in gradle example

* update changelog

---------

Co-authored-by: scottgerring <[email protected]>

* chore: update version to next snapshot: 1-19.0-SNAPSHOT (#1516)

* update version to next snapshot: 1-19.0-SNAPSHOT

* update version to next snapshot: 1-19.0-SNAPSHOT

* update version to next snapshot: 1-19.0-SNAPSHOT

* building only for LTS

* Add some more margin to the test pause (#1518)

* test: e2e tests with java 21 (#1517)

* e2e tests with java 21

* Run Java21 tests using the Java17 compiler

* Run all of the E2E tests in parallel, not just the first 3

* Try again

* .

* Let's try again

* Add some comment on Java21 to the repo

* Add caveat about lambda runtimes

* Clean up wording a little

---------

Co-authored-by: Scott Gerring <[email protected]>

* update doc for ALC (#1520)

* chore: Testing java21 aspectj pre-release (#1519)

* e2e tests with java 21

* use aspectj 1.9.21-SNAPSHOT

* Fix log4j2.xml missing in logging test for java21

* rollback double runtime

* remove comment

* keep aspectj 1.9.7 in parent for java8 compatibility

* use M1 instead of snapshot

* update documentation for aspectj

* update documentation for aspectj

---------

Co-authored-by: Jerome Van Der Linden <[email protected]>

* chore: Remove build cruft

* Adding context for using RequestStreamHandler

* removing pr_lint

* Update examples/powertools-examples-core/sam/src/main/java/helloworld/AppStream.java

Clarify usage of RequestStreamHandler.

Co-authored-by: Alexey Soshin <[email protected]>

---------

Co-authored-by: Jason Harris <[email protected]>
Co-authored-by: Scott Gerring <[email protected]>
Co-authored-by: Jason Harris <[email protected]>
Co-authored-by: Jérôme Van Der Linden <[email protected]>
Co-authored-by: Michele Ricciardi <[email protected]>
Co-authored-by: Alexey Soshin <[email protected]>
Co-authored-by: jdoherty <[email protected]>
Co-authored-by: Scott Gerring <[email protected]>
Co-authored-by: Jerome Van Der Linden <[email protected]>

* deps: bump aspectj to 1.9.21 for jdk21 (#1536)

* chore: SAM and Terraform IaC extracted from pr_build and simplified approach. (#1533)

* SAM and Terraform IaC extracted from pr_build and simplified approach.

* Update .github/workflows/pr_iac_lint.yml

Co-authored-by: Scott Gerring <[email protected]>

---------

Co-authored-by: Jason Harris <[email protected]>
Co-authored-by: Scott Gerring <[email protected]>

* chore: Remove empty CDK test (#1542)

* Remove CDK test

* Build core utilities so that sam validator can find them

* Remove CDK test

* Ok just build it all

* Remove the plan ...

* Remove terraform plan from iac lint

---------

Co-authored-by: Jérôme Van Der Linden <[email protected]>

* dependabot on v2 branch (#1548)

* chore: remove unecessary creds acquisition (#1572)

* Merge from main

---------

Co-authored-by: Jason Harris <[email protected]>
Co-authored-by: Jason Harris <[email protected]>
Co-authored-by: Jason Harris <[email protected]>
Co-authored-by: Jérôme Van Der Linden <[email protected]>
Co-authored-by: Michele Ricciardi <[email protected]>
Co-authored-by: Alexey Soshin <[email protected]>
Co-authored-by: jdoherty <[email protected]>
Co-authored-by: Jerome Van Der Linden <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants