Skip to content

Commit

Permalink
Documentation about the hints annotation for processors (#30561)
Browse files Browse the repository at this point in the history
* documentation for the hints annotation

Signed-off-by: Tetiana Kravchenko <[email protected]>

* add pr link

Signed-off-by: Tetiana Kravchenko <[email protected]>
(cherry picked from commit 3a8cf70)
  • Loading branch information
tetianakravchenko authored and mergify-bot committed Mar 10, 2022
1 parent f8b2289 commit a7149f2
Show file tree
Hide file tree
Showing 2 changed files with 49 additions and 0 deletions.
14 changes: 14 additions & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,20 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...main[Check the HEAD dif

- Add support for '/var/log/pods/' path for add_kubernetes_metadata processor with `resource_type: pod`. {pull}28868[28868]
- Add documentation for add_kubernetes_metadata processors `log_path` matcher. {pull}28868[28868]
- Add support for parsers on journald input {pull}29070[29070]
- Add support in httpjson input for oAuth2ProviderDefault of password grant_type. {pull}29087[29087]
- Add support for filtering in journald input with `unit`, `kernel`, `identifiers` and `include_matches`. {pull}29294[29294]
- Add new `userAgent` and `beatInfo` template functions for httpjson input {pull}29528[29528]
- Add extraction of `related.hosts` to Microsoft 365 Defender ingest pipeline {issue}29859[29859] {pull}29863[29863]
- threatintel module: Add new Recorded Future integration. {pull}30030[30030]
- Add pipeline in FB's supported hints. {pull}30212[30212]

*Auditbeat*


*Filebeat*

- Extend documentation about hints annotation for processors {pull}30561[30561]

*Heartbeat*

Expand Down
35 changes: 35 additions & 0 deletions filebeat/docs/autodiscover-hints.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,41 @@ co.elastic.logs/raw: "[{\"containers\":{\"ids\":[\"${data.container.id}\"]},\"mu
Define a processor to be added to the {beatname_uc} input/module configuration. See <<filtering-and-enhancing-data>> for the list
of supported processors.

If processors configuration uses list data structure, object fields must be enumerated.
For example, hints for the `rename` processor configuration below
[source,yaml]
-----
processors:
- rename:
fields:
- from: "a.g"
to: "e.d"
fail_on_error: true
-----
will look like:
[source,yaml]
-----
co.elastic.logs/processors.rename.fields.0.from: "a.g"
co.elastic.logs/processors.rename.fields.1.to: "e.d"
co.elastic.logs/processors.rename.fail_on_error: 'true'
-----

If processors configuration uses map data structure, enumeration is not needed. For example, the equivalent to the `add_fields` configuration below
[source,yaml]
-----
processors:
- add_fields:
target: project
fields:
name: myproject
-----
is
[source,yaml]
-----
co.elastic.logs/processors.1.add_fields.target: "project"
co.elastic.logs/processors.1.add_fields.fields.name: "myproject"
-----

In order to provide ordering of the processor definition, numbers can be provided. If not, the hints builder will do
arbitrary ordering:

Expand Down

0 comments on commit a7149f2

Please sign in to comment.