-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for kubernetes provider to recognize namespace level defaults #16321
Conversation
Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually? |
1 similar comment
Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually? |
dcbe36e
to
6ac6d69
Compare
We have a template that is generally used when creating a PR, could you update this PR to fill out all those sections. Also did (or could) you file a bug for the behavior your trying to solve and can you provide steps on how to test this PR? |
|
||
// Fall back to defaults on the namespace if there were no hints on the pods | ||
if len(hints) == 0 { | ||
if rawAnn, ok := kubeMeta["defaults"]; ok { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It sounds to me that we should do a mix of namespace hints + pod specific hints? Example:
I may have a setting to do json decode for all pods, but for one specific pod in the namespace I may want to tune how multiline goes, without affecting the JSON parsing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for contributing! I commented in the PR. I think we need to define how namespace hints and Pod hints should interact with each other (order of precedence, full override..), let's keep the conversation going!
Once settled, this will need tests & docs
24d5180
to
21af677
Compare
I have added changes such that annotations that are missing on the pod/service are defaulted with the help of namespace level annotations. i have added test cases for the same. @exekias PTAL |
event["config"] = config | ||
} else { | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
empty else?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for working on this! I left a few comments
for k, v := range namespace.GetAnnotations() { | ||
safemapstr.Put(defaults, k, v) | ||
} | ||
kubemeta["defaults"] = defaults |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we need to create a new key for this. metadata package is already able to put namespace annotations under kubernetes.namespace.annotations
right? We can add that to the kubemeta
object, in the same way we always add pod annotations
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
onl in 8.0 we will have namespace
as a MapStr
. until then it would be namespace_annotations
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good catch, that's true! I think we should do the same here
} | ||
|
||
// Look at all the namespace level default annotations and do a merge with priority going to the pod annotations. | ||
if rawNsAnn, ok := kubeMeta["defaults"]; ok { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we would be getting namespace.annotations
here
21af677
to
7440678
Compare
Thank you for the changes, I think this is looking good! let's have the docs in and it's ready to go 🎉 |
7440678
to
9ae203a
Compare
@exekias added docs. |
ok to test |
9ae203a
to
54d0b5c
Compare
…ults (elastic#16321) * Add support for kubernetes provider to recognize namespace level default hints (cherry picked from commit 004d36c)
…ognize namespace level defaults (#17054) * Add support for kubernetes provider to recognize namespace level defaults (#16321) * Add support for kubernetes provider to recognize namespace level default hints (cherry picked from commit 004d36c) * Update CHANGELOG.next.asciidoc Co-authored-by: Vijay Samuel <[email protected]>
How to test this: Deploy Filebeat (or Metricbeat, but Filebeat makes more sense here) in kubernetes, enable hints based autodiscover with For instance, adding this annotation to the namespace:
Should result on Filebeat stop retrieving logs from the pods in that namespace, while the others should still be retrieved. Also we should test that this can be overridden per pod, for instance, disable collection for a whole namespace, but enable it for one of the pods only |
This PR allows namespace level defaults to be configured on the namespace object for either service or pod hints autodiscover. This will need:
to be set on the kubernetes autodiscover provider.
This PR has a known gap where a pod is deployed before the namespace annotations are added. In this case, autodiscover wouldnt be able to add in the defaults. To fix that behavior we would need to ensure that we handle resync events as updates. Currently we ignore resync events as the resource version remains the same.