Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Logs UI] Add ML module with a common log categorization job #50414

Closed
weltenwort opened this issue Nov 12, 2019 · 1 comment · Fixed by #51905
Closed

[Logs UI] Add ML module with a common log categorization job #50414

weltenwort opened this issue Nov 12, 2019 · 1 comment · Fixed by #51905
Labels
Feature:Logs UI Logs UI feature Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services v7.6.0

Comments

@weltenwort
Copy link
Member

weltenwort commented Nov 12, 2019 •
edited by zube bot
Loading

Summary

Following the investigation in #46610 the categorization can probably be handled using a single ML job.

Job results

The following result data structures could be attributed to a partition:

  • the anomaly record
  • the model_plot with the actual count per bucket and partition

A few result data structures would be shared between the partitions:

  • the bucket
  • the category with its examples

Acceptance criteria

  • A new ML module logs_ui_entry_categorization with one job and one datafeed exists.
  • The job is a categorization job with...
    • a categorization_field_name set to message.
    • a count detector with the by_field_name set to mlcategory and partition_field_name set to event.dataset.
    • a bucket_span of 15 minutes.
    • the model plot enabled.
  • The datafeed for the job...
    • filters the documents for the existence of a message field.
@weltenwort weltenwort added [zube]: Backlog Feature:Logs UI Logs UI feature Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services v7.6.0 labels Nov 12, 2019
@elasticmachine
Copy link
Contributor

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

@weltenwort weltenwort self-assigned this Nov 29, 2019
@zube zube bot unassigned weltenwort Nov 29, 2019
@weltenwort weltenwort mentioned this issue Nov 29, 2019
Merged
4 tasks
weltenwort added a commit that referenced this issue Dec 18, 2019
@weltenwort
[Logs UI] Add setup screen for the categorization tab (#51905)
8863fc2 
This adds the setup screen for the log entry categorization.

closes #50320 
closes #50414
weltenwort added a commit to weltenwort/kibana that referenced this issue Dec 18, 2019
@weltenwort
[Logs UI] Add setup screen for the categorization tab (elastic#51905)
91601eb 
This adds the setup screen for the log entry categorization.

closes elastic#50320 
closes elastic#50414
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Feature:Logs UI Logs UI feature Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services v7.6.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Logs UI] Add setup screen for the categorization tab weltenwort/kibana
3 participants
@sgrodzicki @weltenwort @elasticmachine