Add host.name in the events #7051
Conversation
tsg
added
bug
needs_backport
| @@ -93,6 +94,7 @@ type Settings struct { | |||
| type Annotations struct { | |||
| Beat common.MapStr | |||
| Event common.EventMetadata | |||
| Host common.MapStr | |||
There was a problem hiding this comment.
Hmmm.... do we consider to introduce more namespaces? If so, how about changing Annotations to:
type Annotations struct {
Event common.EventMetadata // additional event fields configured by user
Builtin common.MapStr
}
And then init Annotations with:
Annotations{
Event: config.EventMetaData,
Builtin: common.MapStr{
"beat": ...
"host": ...
},
}
This way the logic how/when the builtin namespaces are merged needs to be done only once.
| if meta := global.beatsMeta; len(meta) > 0 { | ||
| processors.add(makeAddFieldsProcessor("beatsMeta", meta, needsCopy)) | ||
| } | ||
| if meta := global.hostMeta; len(meta) > 0 { | ||
| processors.add(makeAddFieldsProcessor("hostMeta", meta, needsCopy)) |
There was a problem hiding this comment.
This is safe, as needsCopy is true if any global processors are configured.
We can also 'reduce' some copies by introducing needsGlobalCopy := global.alwaysCopy || global.processors != nil and needsCopy := needsGlobalCopy || local.processors != nil.
|
IIUC this change and the add_host_metadata processor could cause mapping conflicts for anyone using LS in their pipeline today. In particular it will cause issues for anyone not using versioned index names (e.g. And it could causes issues with any LS config that expects |
As a solution for elastic#7050, we adding a `host.name` field to all events. This is duplicate information from `beat.name`, but is used to avoid the mapping conflict and to slowly introduce the "host as an object" approach. To remove the duplication, you can remove `beat.name` like this: processors: - drop_fields.fields: ["beat.name"] Closes elastic#7050.
tsg
force-pushed
the
add_host_name_by_default
branch
from
ba9fb18 to
6be41ce
Compare
|
@urso @ruflin Can you have a second look? Also, @andrewkroh has a point above, this is kind of a breaking change. I hope we're not missing other cases like this. cc @jsvd as well. |
|
Jenkins, test this again |
| @@ -36,13 +36,14 @@ exec { go get -u github.com/jstemmer/go-junit-report } | |||
| echo "Building $env:beat" | |||
| exec { go build } "Build FAILURE" | |||
|
|
|||
| # always build the libbeat fields | |||
| cp ..\libbeat\_meta\fields.common.yml ..\libbeat\_meta\fields.generated.yml | |||
There was a problem hiding this comment.
As soon as #6911 is in this should become much simpler.
| "name": name, | ||
| "hostname": beatInfo.Hostname, | ||
| "version": beatInfo.Version, | ||
| Builtin: common.MapStr{ |
There was a problem hiding this comment.
I like that change because it prepares us for more fields coming from ECS.
As a solution for elastic#7050, we're adding a `host.name` field to all events. This is duplicate information from `beat.name`, but is used to avoid the mapping conflict and to slowly introduce the "host as an object" approach. To remove the duplication, you can remove `beat.name` like this: processors: - drop_fields.fields: ["beat.name"] Closes elastic#7050. (cherry picked from commit 15d9539)
tsg
added
v6.3.0
and removed
needs_backport
* Add host.name in the events (#7051) As a solution for #7050, we're adding a `host.name` field to all events. This is duplicate information from `beat.name`, but is used to avoid the mapping conflict and to slowly introduce the "host as an object" approach. To remove the duplication, you can remove `beat.name` like this: processors: - drop_fields.fields: ["beat.name"] Closes #7050. (cherry picked from commit 15d9539) * changelog cleanup
As a solution for elastic#7050, we're adding a `host.name` field to all events. This is duplicate information from `beat.name`, but is used to avoid the mapping conflict and to slowly introduce the "host as an object" approach. To remove the duplication, you can remove `beat.name` like this: processors: - drop_fields.fields: ["beat.name"] Closes elastic#7050.
As a solution for elastic#7050, we're adding a `host.name` field to all events. This is duplicate information from `beat.name`, but is used to avoid the mapping conflict and to slowly introduce the "host as an object" approach. To remove the duplication, you can remove `beat.name` like this: processors: - drop_fields.fields: ["beat.name"] Closes elastic#7050.
|
hi @tsg , We still have issues with Filebeat 6.3.1: |
|
@orarnon This error message looks very unrelated. Please use https://discuss.elastic.co to ask for some help. |
…#7068) * Add host.name in the events (elastic#7051) As a solution for elastic#7050, we're adding a `host.name` field to all events. This is duplicate information from `beat.name`, but is used to avoid the mapping conflict and to slowly introduce the "host as an object" approach. To remove the duplication, you can remove `beat.name` like this: processors: - drop_fields.fields: ["beat.name"] Closes elastic#7050. (cherry picked from commit 1889f9c) * changelog cleanup
As a solution for #7050, we're adding a
host.namefield toall events. This is duplicate information from
beat.name,but is used to avoid the mapping conflict and to slowly
introduce the "host as an object" approach.
To remove the duplication, you can remove
beat.namelike this:Closes #7050.