Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Improve error message notifications when insufficient available memory #20518

Closed
sophiec20 opened this issue Jul 6, 2018 · 3 comments
Closed

Comments

@sophiec20
Copy link
Contributor

sophiec20 commented Jul 6, 2018

If an end-user has a cluster with 3x master, 3x data and 1x ML node (say), then if they cannot open an ML job due to running out of memory, then they get a pretty awful error message.

Each nodes returns to say that it cannot open the job because it is not an ML node, and the ML node returns to say it's full. We also get the additional message to say the datafeed could not start. See below.

image

Can we trap this and display a top level error? e.g. Insufficient available memory. More nodes or more memory required or close unused jobs.

I would expect the full error to be useful in troubleshooting and it could be hidden with an "expand to view full details".

Relates elastic/elasticsearch#29950, and possibly #17961

cc @skearns64

@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui

@droberts195
Copy link
Contributor

elastic/elasticsearch#31925 moves the detailed message to a nested exception which will not be the root_cause of the open job failure.

This will work nicely if it is possible for Kibana to display the root_cause by default but also allow access to the nested exception message if desired. But if Kibana does not allow access to the nested exceptions then maybe it's not useful.

@jgowdyelastic
Copy link
Member

#20880 raised to fix this.

Duplicate errors are grouped with the ability to expand to see the full response

image

screen shot 2018-07-16 at 12 39 10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants