Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix decode_json_fields processor to always add error key #29107

Merged
merged 4 commits into from
Nov 29, 2021
Merged

Fix decode_json_fields processor to always add error key #29107

merged 4 commits into from
Nov 29, 2021

Conversation

belimawr
Copy link
Contributor

@belimawr belimawr commented Nov 23, 2021 •
edited
Loading

What does this PR do?

When the decode_json_fields processor encountered an error while
decoding the JSON it was not always respecting the add_error_key
configuration.

This commit fixes it.

Why is it important?

Errors from decode_json_fields processor are now clearly reported

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
    - [ ] I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

How to test this PR locally

Add the following to you filebeat configuration:

processors:
  - decode_json_fields:
      fields: ["my_json_key"]
      process_array: false
      max_depth: 1
      target: "json_object"
      add_error_key: true

Make sure the my_json_key field of the event is an invalid JSON.

Related issues

Use cases

Screenshots

Logs

@belimawr belimawr added bug backport-v8.0.0 Automated backport with mergify backport-v7.16.0 Automated backport with mergify Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team labels Nov 23, 2021
@belimawr belimawr requested a review from kvch November 23, 2021 18:22
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Nov 23, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented Nov 23, 2021 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-11-25T10:52:26.941+0000

  • Duration: 100 min 39 sec

  • Commit: c469e56

Test stats 🧪

Test Results
Failed 0
Passed 48530
Skipped 4286
Total 52816

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@mergify
Copy link
Contributor

mergify bot commented Nov 24, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b fix-add-error-key upstream/fix-add-error-key
git merge upstream/master
git push upstream fix-add-error-key

@belimawr
Fix decode_json_fields processor to always add error key
6ba9d04 
When the `decode_json_fields` processor encountered an error while
decoding the JSON it was not always respecting the `add_error_key`
configuration.

This commit fixes it.
@belimawr
Add field to error data, input is renamed to data
e5fce6c
@belimawr
changelog entry
a83565c
@belimawr
Add test
c469e56
@belimawr belimawr marked this pull request as ready for review November 25, 2021 10:52
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@belimawr belimawr merged commit 37c6229 into elastic:master Nov 29, 2021
@belimawr belimawr deleted the fix-add-error-key branch November 29, 2021 10:31
mergify bot pushed a commit that referenced this pull request Nov 29, 2021
@belimawr
Fix decode_json_fields processor to always add error key (#29107)
e5a61ba 
When the `decode_json_fields` processor encountered an error while
decoding the JSON it was not always respecting the `add_error_key`
configuration.

This commit fixes it.

(cherry picked from commit 37c6229)
@mergify mergify bot mentioned this pull request Nov 29, 2021
Merged
mergify bot pushed a commit that referenced this pull request Nov 29, 2021
@belimawr
Fix decode_json_fields processor to always add error key (#29107)
0160e7a 
When the `decode_json_fields` processor encountered an error while
decoding the JSON it was not always respecting the `add_error_key`
configuration.

This commit fixes it.

(cherry picked from commit 37c6229)
@mergify mergify bot mentioned this pull request Nov 29, 2021
Merged
belimawr added a commit that referenced this pull request Nov 29, 2021
@mergify @belimawr
Fix decode_json_fields processor to always add error key (#29107) (#…
b314606 
…29178)

When the `decode_json_fields` processor encountered an error while
decoding the JSON it was not always respecting the `add_error_key`
configuration.

This commit fixes it.

(cherry picked from commit 37c6229)

Co-authored-by: Tiago Queiroz <[email protected]>
belimawr added a commit that referenced this pull request Nov 30, 2021
@mergify @belimawr
Fix decode_json_fields processor to always add error key (#29107) (#…
c783d87 
…29177)

When the `decode_json_fields` processor encountered an error while
decoding the JSON it was not always respecting the `add_error_key`
configuration.

This commit fixes it.

(cherry picked from commit 37c6229)

Co-authored-by: Tiago Queiroz <[email protected]>
v1v added a commit to v1v/beats that referenced this pull request Nov 30, 2021
@v1v
Merge remote-tracking branch 'upstream/master' into feature/use-debian-9
53d5763 
* upstream/master: (577 commits)
  Remove Journalbeat (elastic#29131)
  Add note that there is no warranty or support for generator code (elastic#28797)
  packetbeat: preparation for npcap addition (elastic#29017)
  Use the generic helper for opening file to read in filestream (elastic#29180)
  Workflow for macos (elastic#29174)
  Fix `decode_json_fields` processor to always add error key (elastic#29107)
  Workflow for macos (elastic#29156)
  Fix agent download timeout values in yaml files (elastic#29039)
  Added workflow file for builds with macos (elastic#29148)
  CI: enable AWS cloud testing on a PR basis if changes in certain files (elastic#29047)
  Remove links to Journalbeat (elastic#29134)
  Fix rds metadata in cloudwatch metricset (elastic#29106)
  [mergify]: notify conflicts in PRs that are still open (elastic#29122)
  Use NamedWatcher in Agent's k8s provider (elastic#29095)
  override host on statsd metricset (elastic#29103)
  Skip config check in autodiscover for duplicated configurations (elastic#29048)
  Change "filebeat.config.modules.enabled" to "true" (elastic#28769)
  Remove deprecated spool queue from Beats (elastic#28869)
  Add `beat` field back to beat.stats (elastic#29094)
  Revert "Move labels and annotations under kubernetes.namespace. (elastic#27917)" (elastic#29069)
  ...
elasticmachine pushed a commit to nxei/beats that referenced this pull request Dec 1, 2021
@v1v
Merge remote-tracking branch 'upstream/master' into github-actions-build
99c50fb 
* upstream/master:
  [libbeat] Fix add_labels flattening of arrays values (elastic#29211)
  Change elastic-agent pprof default to false (elastic#29155)
  elastic#28472 fix flaky tests in libbeat fmtstr to use time.UTC instead of time.Local (elastic#28473)
  Adopt `parsers` in Filebeat's journald input (elastic#29070)
  [Elastic Agent] Add process error handling guidelines (elastic#29152)
  winlogbeat/sys/winevent: use reflect IsZero method (elastic#29190)
  Remove Journalbeat (elastic#29131)
  Add note that there is no warranty or support for generator code (elastic#28797)
  packetbeat: preparation for npcap addition (elastic#29017)
  Use the generic helper for opening file to read in filestream (elastic#29180)
  Workflow for macos (elastic#29174)
  Fix `decode_json_fields` processor to always add error key (elastic#29107)
@tetianakravchenko
Copy link
Contributor

@belimawr this PR should close this issue #27067, right?

@kvch kvch mentioned this pull request Mar 30, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kvch kvch kvch approved these changes

Assignees
No one assigned
Labels
backport-v7.16.0 Automated backport with mergify backport-v8.0.0 Automated backport with mergify bug review Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@belimawr @elasticmachine @tetianakravchenko @kvch