Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x-pack/metricbeat/module/statsd: Skip invalid statsd packet(s) #35075

Merged
merged 8 commits into from
Jun 14, 2023

Conversation

shmsr
Copy link
Member

@shmsr shmsr commented Apr 12, 2023

What does this PR do?

Previously, invalid statsd metric packets would break the ingestion and further metric packets were left unparsed. Now it'd be able to skip invalid packets and continue the parsing. Also made some nitpick fixes (removal of unused vars, usage of proper methods, migrate away from github.com/pkg/errors etc.). Also, made some preallocation changes which are kind of micro-perf improvement and also made did code reorganization to make the code better; although the latter could be done in a different pull request.

Why is it important?

Previously, during parsing of events sent by a statsd client, one invalid packet could stop the entire parsing then and there even if there are valid packets that are waiting to be parsed.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Run go tests in the concerned package:

$ go test -v ./...

Related issues

@shmsr shmsr requested a review from a team as a code owner April 12, 2023 21:08
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Apr 12, 2023
@botelastic
Copy link

botelastic bot commented Apr 12, 2023

This pull request doesn't have a Team:<team> label.

@shmsr shmsr marked this pull request as draft April 12, 2023 21:08
@shmsr shmsr self-assigned this Apr 12, 2023
@mergify
Copy link
Contributor

mergify bot commented Apr 12, 2023

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @shmsr? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

@shmsr shmsr requested a review from lalit-satapathy April 12, 2023 21:08
@shmsr shmsr added the bugfix label Apr 12, 2023
@shmsr shmsr changed the title x-pack/metricbeat/module/statsd: Skip invalid statsd packet(s) [WIP]: x-pack/metricbeat/module/statsd: Skip invalid statsd packet(s) Apr 12, 2023
@shmsr shmsr changed the title [WIP]: x-pack/metricbeat/module/statsd: Skip invalid statsd packet(s) x-pack/metricbeat/module/statsd: Skip invalid statsd packet(s) Apr 12, 2023
@shmsr
Copy link
Member Author

shmsr commented Apr 12, 2023

While making the changes I'll also found a bug in the statsd's event parser where the parser can't parse all valid values. I'll open an issue. Fix is pretty simple for the same. It's an important one so I'd try to do it ASAP.

@elasticmachine
Copy link
Collaborator

elasticmachine commented Apr 12, 2023

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-06-14T09:16:00.539+0000

  • Duration: 54 min 13 sec

Test stats 🧪

Test Results
Failed 0
Passed 1501
Skipped 90
Total 1591

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@shmsr shmsr force-pushed the fix-statsd-skip-invalid-packet-parsing branch 2 times, most recently from b758041 to 3203dc4 Compare April 17, 2023 19:46
@shmsr shmsr marked this pull request as ready for review May 2, 2023 18:34
@shmsr shmsr requested a review from lalit-satapathy May 15, 2023 05:17
@mergify
Copy link
Contributor

mergify bot commented May 16, 2023

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b fix-statsd-skip-invalid-packet-parsing upstream/fix-statsd-skip-invalid-packet-parsing
git merge upstream/main
git push upstream fix-statsd-skip-invalid-packet-parsing

@shmsr shmsr force-pushed the fix-statsd-skip-invalid-packet-parsing branch from 3f47fce to e46c0b1 Compare May 16, 2023 18:38
CHANGELOG.next.asciidoc Outdated Show resolved Hide resolved
@shmsr shmsr requested a review from ritalwar May 17, 2023 06:09
…parsing

Previously, invalid statsd metric packets would break the ingestion
and further metric packets were left unparsed. Now it'd be able to
skip invalid packets and continue the parsing.
@shmsr shmsr force-pushed the fix-statsd-skip-invalid-packet-parsing branch from 015969c to 55802a6 Compare May 17, 2023 08:29
@shmsr shmsr force-pushed the fix-statsd-skip-invalid-packet-parsing branch from 55802a6 to 04f5645 Compare May 17, 2023 08:30
Copy link
Contributor

@ritalwar ritalwar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@mergify
Copy link
Contributor

mergify bot commented May 23, 2023

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b fix-statsd-skip-invalid-packet-parsing upstream/fix-statsd-skip-invalid-packet-parsing
git merge upstream/main
git push upstream fix-statsd-skip-invalid-packet-parsing

@shmsr shmsr requested a review from a team as a code owner June 7, 2023 18:30
@shmsr shmsr requested a review from lucian-ioan June 12, 2023 07:07
Copy link
Contributor

@lucian-ioan lucian-ioan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@shmsr shmsr enabled auto-merge (squash) June 14, 2023 09:16
@shmsr shmsr merged commit 4a16c6a into elastic:main Jun 14, 2023
@shmsr shmsr deleted the fix-statsd-skip-invalid-packet-parsing branch July 6, 2023 17:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix needs_team Indicates that the issue/PR needs a Team:* label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants