Skip to content

Commit

Permalink
Cisco asa/ftd: Remove _temp_ fields on failure (elastic#18392)
Browse files Browse the repository at this point in the history
The shared pipeline for Cisco ASA and FTD creates temporary fields under
the _temp_ object. If a failure happens in the middle of the pipeline,
all those fields would be indexed, causing the index mapping to grow too
big.

Fixes elastic#18391

(cherry picked from commit 3605c47)
  • Loading branch information
adriansr committed May 13, 2020
1 parent 26cc4ad commit 3e8c21c
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 0 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Fixing `ingress_controller.` fields to be of type keyword instead of text. {issue}17834[17834]
- Fixed typo in log message. {pull}17897[17897]
- Unescape file name from SQS message. {pull}18370[18370]
- Improve cisco asa and ftd pipelines' failure handler to avoid mapping temporary fields. {issue}18391[18391] {pull}18392[18392]

*Heartbeat*

Expand Down
10 changes: 10 additions & 0 deletions x-pack/filebeat/module/cisco/shared/ingest/asa-ftd-pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1275,6 +1275,16 @@ processors:
ignore_missing: true
on_failure:
# Copy any fields under _temp_.cisco to its final destination. Those can help
# with diagnosing the failure.
- rename:
field: _temp_.cisco
target_field: "cisco.{< .internal_prefix >}"
ignore_failure: true
# Remove _temp_ to avoid adding a lot of unnecessary fields to the index.
- remove:
field: _temp_
ignore_missing: true
- append:
field: "error.message"
value: "{{ _ingest.on_failure_message }}"

0 comments on commit 3e8c21c

Please sign in to comment.