Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new parser called include_message to filter messages #32094

Merged
merged 17 commits into from
Jun 27, 2022

Conversation

kvch
Copy link
Contributor

@kvch kvch commented Jun 24, 2022

What does this PR do?

This PR adds a new parser to filter messages. If a message matches one of the patterns from the list, it is passed on to the next parser, otherwise it is dropped.

Why is it important?

The existing include_lines and exlcude_lines filtering runs after the parsers pipeline, so it is not possible to select messages before parsing them.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Closes #31794

@kvch kvch requested a review from a team as a code owner June 24, 2022 15:55
@kvch kvch requested review from belimawr and fearful-symmetry and removed request for a team June 24, 2022 15:55
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 24, 2022
@mergify mergify bot assigned kvch Jun 24, 2022
@kvch kvch marked this pull request as draft June 24, 2022 15:55
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jun 24, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-06-27T12:36:49.196+0000

  • Duration: 115 min 22 sec

Test stats 🧪

Test Results
Failed 0
Passed 22344
Skipped 1937
Total 24281

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@kvch kvch changed the title Add new parser called filter to filter messages Add new parser called include_message to filter messages Jun 27, 2022
@kvch kvch added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Jun 27, 2022
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jun 27, 2022
@kvch kvch marked this pull request as ready for review June 27, 2022 10:51
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@kvch kvch merged commit c731661 into elastic:main Jun 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Filebeat json parser is trying to parse excluded lines
4 participants