Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/k8sobjects] ensure the k8s.namespace.name attribute is set for objects retrieved using the watch mode #36432

Open
wants to merge 22 commits into
base: main
Choose a base branch
from

Conversation

bacherfl
Copy link
Contributor

Description

This PR ensures that the structure of log records generated by the k8sobjects receiver is the same, regardless of the mode (watch or pull) being used. This also solves the issue of the k8s.namespace.name attribute not being set for objects retrieved with watch mode.

Link to tracking issue

Fixes #36352

Testing

Added unit tests and adapted e2e tests

… and use same data structure as with pull based mode for log entries

Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
@bacherfl bacherfl changed the title Fix/36352/k8sobjects namespace [receiver/k8sobjects] use same data structure for objects received via pull and watch mode Nov 19, 2024
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
@@ -37,6 +34,7 @@ func watchObjectsToLogData(event *watch.Event, observedAt time.Time, config *K8s
if name != "" {
attrs.PutStr("event.domain", "k8s")
attrs.PutStr("event.name", name)
attrs.PutStr("event.type", string(event.Type))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI: there is an issue regarding reflecting of k8s events to the events API definition - #35857 (comment), maybe it makes sense to include event.type to the events api convention

Copy link
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank's @bacherfl for working on this!

I agree that both pull and watch modes should populate the k8s.namespace.name resource attribute as it was highlighted at #36352.

However, I'm not sure if the existing structure should change, object.metadata.* -> metadata.*.

Isn't this a breaking change? Should we ensure if we really want this or not?
Otherwise, if we can fix the namespace issue without introducing a generic breaking change that would be optimal I think.

Could we clarify the above questions?

.chloggen/k8sobjects-data-structure.yaml Outdated Show resolved Hide resolved
@bacherfl
Copy link
Contributor Author

bacherfl commented Dec 2, 2024

Thank's @bacherfl for working on this!

I agree that both pull and watch modes should populate the k8s.namespace.name resource attribute as it was highlighted at #36352.

However, I'm not sure if the existing structure should change, object.metadata.* -> metadata.*.

Isn't this a breaking change? Should we ensure if we really want this or not? Otherwise, if we can fix the namespace issue without introducing a generic breaking change that would be optimal I think.

Could we clarify the above questions?

Thanks @ChrsMark for the feedback! You're right, changing the structure reported by the watch mode to be the same as with pull would be a breaking change, so I can also adapt the PR to specifically fix the namespace being set, if that is more preferable. In this case I would also add a note in the readme that the structure of the emitted log entries might differ, based on which mode is being used for the receiver

@ChrsMark
Copy link
Member

ChrsMark commented Dec 2, 2024

Thanks @ChrsMark for the feedback! You're right, changing the structure reported by the watch mode to be the same as with pull would be a breaking change, so I can also adapt the PR to specifically fix the namespace being set, if that is more preferable. In this case I would also add a note in the readme that the structure of the emitted log entries might differ, based on which mode is being used for the receiver

Cool! I think it's just fine fixing the namespace issue here and possibly file another issue to evaluate the "inconsistency" between the 2 modes. (in case we decide to change this, it should probably be done with a feature gate etc).

To my mind there should be no difference in the 2 modes from users' perspective but maybe I miss the details here behind this decision.

@bacherfl
Copy link
Contributor Author

bacherfl commented Dec 3, 2024

Thanks @ChrsMark for the feedback! You're right, changing the structure reported by the watch mode to be the same as with pull would be a breaking change, so I can also adapt the PR to specifically fix the namespace being set, if that is more preferable. In this case I would also add a note in the readme that the structure of the emitted log entries might differ, based on which mode is being used for the receiver

Cool! I think it's just fine fixing the namespace issue here and possibly file another issue to evaluate the "inconsistency" between the 2 modes. (in case we decide to change this, it should probably be done with a feature gate etc).

To my mind there should be no difference in the 2 modes from users' perspective but maybe I miss the details here behind this decision.

alright, then I'll adapt the current PR accordingly and will file an issue regarding the two different data structures - will let you know when this one is ready for another review :)

Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
@bacherfl bacherfl changed the title [receiver/k8sobjects] use same data structure for objects received via pull and watch mode [receiver/k8sobjects] ensure the k8s.namespace.name attribute is set for objects retrieved using the watch mode Dec 3, 2024
Signed-off-by: Florian Bacher <[email protected]>
Copy link
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[receiver/k8sobjects] Kubernetes events are missing namespace resource attribute
4 participants