Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ML] Rare anomaly detection job wizard #100390

Merged
merged 26 commits into from
Jun 29, 2021
Merged

[ML] Rare anomaly detection job wizard #100390

merged 26 commits into from
Jun 29, 2021

Conversation

jgowdyelastic
Copy link
Member

@jgowdyelastic jgowdyelastic commented May 20, 2021 •
edited
Loading

Adds the option to create a rare detector job using a new wizard.
Three general rare detector types are available, rare, rare in population and frequently rare in population. The later two require a population field to be selected.
A split field can be added to any of the choices.

image

Checklist

Delete any items that are not applicable to this PR.

@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

jgowdyelastic
jgowdyelastic commented Jun 17, 2021
View reviewed changes
x-pack/plugins/ml/public/application/jobs/new_job/pages/job_type/rare_job_icon.tsx

import React from 'react';

export const RareIcon = (
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is currently a copy of the categorisation icon. a new icon needs designing and will be placed here when ready in a follow up PR

@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

@alvarezmelissa87
Copy link
Contributor

alvarezmelissa87 commented Jun 22, 2021 •
edited by jgowdyelastic
Loading

I was able to reproduce the bug you mentione @jgowdyelastic with the error for the model memory limit fetch failing due to the over field not being selected.

image

Not sure if you were planning on updating that in this PR. Works fine once I select the population field.

I was unable to reproduce the estimate bucket span error that @peteharverson was seeing - I tested with gallery dataset and tried rare, rare in population, and frequently rare in population but wasn't able to get an error.

@alvarezmelissa87 this bug is tricky to fix and so i'll look at it in a follow up PR

@alvarezmelissa87
Copy link
Contributor

alvarezmelissa87 commented Jun 22, 2021 •
edited by jgowdyelastic
Loading

Found a small issue with estimate bucket span. If I try to estimate bucket span more than once the Next button becomes disabled and I'm unable to proceed unless I change something else in the form. Not a huge deal but a bit of a pain.

rare-wizard-bug.mp4

@alvarezmelissa87 good spot, this is a new bug in this PR. Hopefully easy to fix.
Fixed in 0862bb9

@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
ml 1698 1718 +20

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
ml 5.9MB 5.9MB +26.4KB

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jgowdyelastic

peteharverson
peteharverson approved these changes Jun 29, 2021
View reviewed changes
Copy link
Contributor

@peteharverson peteharverson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

I raised #103466 to look into the bucket span estimator separately.

@jgowdyelastic jgowdyelastic merged commit 2e00e9c into elastic:master Jun 29, 2021
@jgowdyelastic jgowdyelastic deleted the rare-job-wizard branch June 29, 2021 10:02
kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Jun 29, 2021
@jgowdyelastic @kibanamachine @szabosteve
[ML] Rare anomaly detection job wizard (elastic#100390)
e8d6c94 
* [ML] Rare anomaly detection job wizard

* fixing fields selection

* small improvements

* adding event rate chart to summary step

* [ML] Changes UI text for rare wizard.

* improving detector summary

* fixing translations

* removing comments

* fixing field selection

* fixing advanced wizard

* updating detector text

* fixing bucketspan estimator

* bug fixes

Co-authored-by: Kibana Machine <[email protected]>
Co-authored-by: István Zoltán Szabó <[email protected]>
@kibanamachine kibanamachine mentioned this pull request Jun 29, 2021
Merged
@kibanamachine
Copy link
Contributor

💚 Backport successful

Status Branch Result
✅ 7.x

This backport PR will be merged automatically after passing CI.

kibanamachine added a commit that referenced this pull request Jun 29, 2021
@kibanamachine @jgowdyelastic @szabosteve
[ML] Rare anomaly detection job wizard (#100390) (#103630)
6db2ce0 
* [ML] Rare anomaly detection job wizard

* fixing fields selection

* small improvements

* adding event rate chart to summary step

* [ML] Changes UI text for rare wizard.

* improving detector summary

* fixing translations

* removing comments

* fixing field selection

* fixing advanced wizard

* updating detector text

* fixing bucketspan estimator

* bug fixes

Co-authored-by: Kibana Machine <[email protected]>
Co-authored-by: István Zoltán Szabó <[email protected]>

Co-authored-by: James Gowdy <[email protected]>
Co-authored-by: István Zoltán Szabó <[email protected]>
alvarezmelissa87
alvarezmelissa87 reviewed Jun 29, 2021
View reviewed changes
Copy link
Contributor

@alvarezmelissa87 alvarezmelissa87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ⚡

@lcawl lcawl added release_note:feature Makes this part of the condensed release notes and removed release_note:enhancement labels Jul 6, 2021
This was referenced Jul 14, 2021
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@szabosteve szabosteve szabosteve left review comments

@alvarezmelissa87 alvarezmelissa87 alvarezmelissa87 left review comments

@peteharverson peteharverson peteharverson approved these changes

@qn895 qn895 Awaiting requested review from qn895

Assignees

@jgowdyelastic jgowdyelastic

Labels
auto-backport Deprecated - use backport:version if exact versions are needed Feature:Anomaly Detection ML anomaly detection :ml release_note:feature Makes this part of the condensed release notes review v7.14.0 v8.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@jgowdyelastic @elasticmachine @alvarezmelissa87 @kibanamachine @peteharverson @szabosteve @lcawl