Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Filebeat] aws-s3: Stop calling ChangeMessageVisibility after ReceiptHandleIsInvalid #29480

Merged
merged 1 commit into from
Dec 16, 2021
Merged

[Filebeat] aws-s3: Stop calling ChangeMessageVisibility after ReceiptHandleIsInvalid #29480

merged 1 commit into from
Dec 16, 2021

Conversation

andrewkroh
Copy link
Member

@andrewkroh andrewkroh commented Dec 16, 2021 •
edited
Loading

What does this PR do?

This reduced log noise and unnecessary API calls from the aws-s3 input.

  • Stop the keepalive goroutine after ErrCodeReceiptHandleIsInvalid is returned by ChangeMessageVisibility.
  • Add message_receipt_time to log messages associated with processing of a given SQS message.
  • Fix incorrect error being wrapped when ApproximateReceiveCount threshold is reached.

Why is it important?

ReceiptHandleIsInvalid warnings become very noisy in Filebeat logs.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@andrewkroh andrewkroh requested a review from aspacca December 16, 2021 18:25
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Dec 16, 2021
@andrewkroh
Stop calling ChangeMessageVisibility after ReceiptHandleIsInvalid
8581e16 
Stop the keepalive goroutine after ErrCodeReceiptHandleIsInvalid is returned by ChangeMessageVisibility.

Add `message_receipt_time` to log messages associated with processing of a given SQS message.

Fix incorrect error being wrapped when ApproximateReceiveCount threshold is reached.
@mergify
Copy link
Contributor

mergify bot commented Dec 16, 2021

This pull request does not have a backport label. Could you fix it @andrewkroh? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@mergify mergify bot added the backport-skip Skip notification from the automated backport with mergify label Dec 16, 2021
@andrewkroh andrewkroh force-pushed the bugfix/fb/aws-s3-logging branch from 9105c79 to 8581e16 Compare December 16, 2021 18:26
@andrewkroh andrewkroh added backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.0.0 Automated backport with mergify labels Dec 16, 2021
@mergify mergify bot removed the backport-skip Skip notification from the automated backport with mergify label Dec 16, 2021
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-12-16T18:26:40.195+0000

  • Duration: 128 min 15 sec

  • Commit: 8581e16

Test stats 🧪

Test Results
Failed 0
Passed 4444
Skipped 309
Total 4753

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@andrewkroh andrewkroh merged commit a0ef4d3 into elastic:master Dec 16, 2021
mergify bot pushed a commit that referenced this pull request Dec 16, 2021
@andrewkroh
Stop calling ChangeMessageVisibility after ReceiptHandleIsInvalid (#2…
e7d0a64 
…9480)

Stop the keepalive goroutine after ErrCodeReceiptHandleIsInvalid is returned by ChangeMessageVisibility.

Add `message_receipt_time` to log messages associated with processing of a given SQS message.

Fix incorrect error being wrapped when ApproximateReceiveCount threshold is reached.

(cherry picked from commit a0ef4d3)
@mergify mergify bot mentioned this pull request Dec 16, 2021
Merged
andrewkroh added a commit that referenced this pull request Dec 17, 2021
@mergify @andrewkroh
Stop calling ChangeMessageVisibility after ReceiptHandleIsInvalid (#2…
d330343 
…9480) (#29485)

Stop the keepalive goroutine after ErrCodeReceiptHandleIsInvalid is returned by ChangeMessageVisibility.

Add `message_receipt_time` to log messages associated with processing of a given SQS message.

Fix incorrect error being wrapped when ApproximateReceiveCount threshold is reached.

(cherry picked from commit a0ef4d3)

Co-authored-by: Andrew Kroh <[email protected]>
@andrewkroh
Copy link
Member Author

@Mergifyio backport 7.17

mergify bot pushed a commit that referenced this pull request Dec 17, 2021
@andrewkroh
Stop calling ChangeMessageVisibility after ReceiptHandleIsInvalid (#2…
cff234d 
…9480)

Stop the keepalive goroutine after ErrCodeReceiptHandleIsInvalid is returned by ChangeMessageVisibility.

Add `message_receipt_time` to log messages associated with processing of a given SQS message.

Fix incorrect error being wrapped when ApproximateReceiveCount threshold is reached.

(cherry picked from commit a0ef4d3)
@mergify mergify bot mentioned this pull request Dec 17, 2021
Merged
@mergify
Copy link
Contributor

mergify bot commented Dec 17, 2021

backport 7.17

✅ Backports have been created

andrewkroh added a commit that referenced this pull request Dec 17, 2021
@mergify @andrewkroh
[Filebeat] aws-s3: Stop calling ChangeMessageVisibility after Receipt…
fab7933 
…HandleIsInvalid (backport #29480) (#29497)

* Stop calling ChangeMessageVisibility after ReceiptHandleIsInvalid (#29480)

Stop the keepalive goroutine after ErrCodeReceiptHandleIsInvalid is returned by ChangeMessageVisibility.

Add `message_receipt_time` to log messages associated with processing of a given SQS message.

Fix incorrect error being wrapped when ApproximateReceiveCount threshold is reached.

(cherry picked from commit a0ef4d3)

Co-authored-by: Andrew Kroh <[email protected]>
@montaguethomas montaguethomas mentioned this pull request Mar 3, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@efd6 efd6 efd6 approved these changes

@kaiyan-sheng kaiyan-sheng kaiyan-sheng approved these changes

@aspacca aspacca Awaiting requested review from aspacca

Assignees
No one assigned
Labels
backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.0.0 Automated backport with mergify bug Filebeat Filebeat Team:Integrations Label for the Integrations team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@andrewkroh @elasticmachine @kaiyan-sheng @efd6