Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Agent] Expose stream.* data in every event #17468

Merged
merged 10 commits into from
Apr 14, 2020

Conversation

michalpristas
Copy link
Contributor

@michalpristas michalpristas commented Apr 3, 2020

What does this PR do?

This PR generates a processor for each input exposing stream.[namespace/type/dataset] values to each event.

Why is it important?

Some modules define like Suricata a single input where mixed events are generated: metrics, logs or alerts. The identification of the data is often done in the ingest pipeline, where the type is generated from fields or values from the data.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Fixes: #16562

@elasticmachine
Copy link
Collaborator

Pinging @elastic/ingest-management (Project:fleet)

@ph ph self-requested a review April 3, 2020 12:38
@hbharding hbharding self-assigned this Apr 3, 2020
@ph ph added the needs_backport PR is waiting to be backported to other branches. label Apr 7, 2020
@ph
Copy link
Contributor

ph commented Apr 7, 2020

@hbharding It is an error that you are assigned to this PR?

Copy link
Contributor

@ph ph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small comment concerning the document block of a function otherwise, LGTM Good workd @michalpristas

@@ -429,6 +436,117 @@ func InjectIndex(indexType string) *InjectIndexRule {
}
}

// InjectStreamProcessorRule expect target to be a collection of fields including
// _meta_index map with dataset, index and namespace keys defined.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this comment need to be changed since we do not have the _meta_index present in the code?

@michalpristas michalpristas merged commit 70fba87 into elastic:master Apr 14, 2020
michalpristas added a commit to michalpristas/beats that referenced this pull request Apr 15, 2020
[Agent] Expose stream.* data in every event (elastic#17468)

(cherry picked from commit 70fba87)
michalpristas added a commit that referenced this pull request Apr 15, 2020
[Agent] Expose stream.* data in every event (#17468)

(cherry picked from commit 70fba87)
@hbharding hbharding removed their assignment Apr 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement needs_backport PR is waiting to be backported to other branches.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Agent] Exposed stream.type, stream.dataset and stream.namespace to every events.
4 participants