Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use safe put for k8s metadata #6490

Merged
merged 1 commit into from
Mar 13, 2018
Merged

Use safe put for k8s metadata #6490

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ https://github.com/elastic/beats/compare/v6.0.0-beta2...master[Check the HEAD di
- Fix for kafka logger. {pull}6430[6430]
- Remove double slashes in Windows service script. {pull}6491[6491]
- Fix infinite failure on Kubernetes watch {pull}6504[6504]
- Ensure Kubernetes labels/annotations don't break mapping {pull}6490[6490]

*Auditbeat*

Expand Down
13 changes: 8 additions & 5 deletions libbeat/common/kubernetes/metadata.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
package kubernetes

import "github.com/elastic/beats/libbeat/common"
import (
"github.com/elastic/beats/libbeat/common"
"github.com/elastic/beats/libbeat/common/safemapstr"
)

// MetaGenerator builds metadata objects for pods and containers
type MetaGenerator interface {
Expand Down Expand Up @@ -31,7 +34,7 @@ func (g *metaGenerator) PodMetadata(pod *Pod) common.MapStr {
labelMap := common.MapStr{}
if len(g.labels) == 0 {
for k, v := range pod.Metadata.Labels {
labelMap[k] = v
safemapstr.Put(labelMap, k, v)
}
} else {
labelMap = generateMapSubset(pod.Metadata.Labels, g.labels)
Expand All @@ -54,11 +57,11 @@ func (g *metaGenerator) PodMetadata(pod *Pod) common.MapStr {
}

if len(labelMap) != 0 {
meta["labels"] = common.DeDotJSON(labelMap)
meta["labels"] = labelMap
}

if len(annotationsMap) != 0 {
meta["annotations"] = common.DeDotJSON(annotationsMap)
meta["annotations"] = annotationsMap
}

return meta
Expand All @@ -85,7 +88,7 @@ func generateMapSubset(input map[string]string, keys []string) common.MapStr {
for _, key := range keys {
value, ok := input[key]
if ok {
output[key] = value
safemapstr.Put(output, key, value)
}
}

Expand Down
4 changes: 2 additions & 2 deletions libbeat/common/kubernetes/metadata_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ func TestPodMetadataDeDot(t *testing.T) {
{
pod: &Pod{
Metadata: ObjectMeta{
Labels: map[string]string{"a.key": "a.value"},
Labels: map[string]string{"a.key": "foo", "a": "bar"},
},
},
meta: common.MapStr{"labels": common.MapStr{"a_key": "a.value"}},
meta: common.MapStr{"labels": common.MapStr{"a": common.MapStr{"value": "bar", "key": "foo"}}},
},
}

Expand Down
5 changes: 3 additions & 2 deletions metricbeat/module/kubernetes/event/event.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"github.com/elastic/beats/libbeat/common"
"github.com/elastic/beats/libbeat/common/cfgwarn"
"github.com/elastic/beats/libbeat/common/kubernetes"
"github.com/elastic/beats/libbeat/common/safemapstr"
"github.com/elastic/beats/metricbeat/mb"
)

Expand Down Expand Up @@ -105,7 +106,7 @@ func generateMapStrFromEvent(eve *kubernetes.Event) common.MapStr {
if len(eve.Metadata.Labels) != 0 {
labels := make(common.MapStr, len(eve.Metadata.Labels))
for k, v := range eve.Metadata.Labels {
labels[common.DeDot(k)] = v
safemapstr.Put(labels, k, v)
}

eventMeta["labels"] = labels
Expand All @@ -114,7 +115,7 @@ func generateMapStrFromEvent(eve *kubernetes.Event) common.MapStr {
if len(eve.Metadata.Annotations) != 0 {
annotations := make(common.MapStr, len(eve.Metadata.Annotations))
for k, v := range eve.Metadata.Annotations {
annotations[common.DeDot(k)] = v
safemapstr.Put(annotations, k, v)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should document this under breaking changes? As in case someone filtered for the annotations this queries now stop working. Will still merge the PR but we can discuss this as a follow up.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I agree this is a bug fix in general.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I wonder how we should communicate this, so far without this change the only way to get some field that is renamed by this was using drop_field processor to ensure there are no mapping issues.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we link to the PR in the changelog could you add more details to the PR message? So people ending up on this PR get some background info?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, done!

}

eventMeta["annotations"] = annotations
Expand Down