Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] DF Analytics Regression results: rerun evaluate endpoint for search bar queries #50991

Merged
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
@import 'pages/analytics_exploration/components/exploration/index';
@import 'pages/analytics_exploration/components/regression_exploration/index';
@import 'pages/analytics_management/components/analytics_list/index';
@import 'pages/analytics_management/components/create_analytics_form/index';
@import 'pages/analytics_management/components/create_analytics_flyout/index';
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,11 @@ import { BehaviorSubject } from 'rxjs';
import { filter, distinctUntilChanged } from 'rxjs/operators';
import { Subscription } from 'rxjs';
import { idx } from '@kbn/elastic-idx';
import { cloneDeep } from 'lodash';
import { ml } from '../../services/ml_api_service';
import { Dictionary } from '../../../common/types/common';
import { getErrorMessage } from '../pages/analytics_management/hooks/use_create_analytics_form';
import { SavedSearchQuery } from '../../contexts/kibana';

export type IndexName = string;
export type IndexPattern = string;
Expand Down Expand Up @@ -38,8 +41,8 @@ export enum INDEX_STATUS {
}

export interface Eval {
meanSquaredError: number | '';
rSquared: number | '';
meanSquaredError: number | string;
rSquared: number | string;
error: null | string;
}

Expand Down Expand Up @@ -119,6 +122,13 @@ export const isRegressionAnalysis = (arg: any): arg is RegressionAnalysis => {
return keys.length === 1 && keys[0] === ANALYSIS_CONFIG_TYPE.REGRESSION;
};

export const isRegressionResultsSearchBoolQuery = (
arg: any
): arg is RegressionResultsSearchBoolQuery => {
const keys = Object.keys(arg);
return keys.length === 1 && keys[0] === 'bool';
};

export interface DataFrameAnalyticsConfig {
id: DataFrameAnalyticsId;
// Description attribute is not supported yet
Expand Down Expand Up @@ -212,27 +222,67 @@ export function getValuesFromResponse(response: RegressionEvaluateResponse) {

return { meanSquaredError, rSquared };
}
interface RegressionResultsSearchBoolQuery {
bool: Dictionary<any>;
}
interface RegressionResultsSearchTermQuery {
term: Dictionary<any>;
}

export type RegressionResultsSearchQuery =
| RegressionResultsSearchBoolQuery
| RegressionResultsSearchTermQuery
| SavedSearchQuery;

export function getEvalQueryBody({
resultsField,
isTraining,
searchQuery,
ignoreDefaultQuery,
}: {
resultsField: string;
isTraining: boolean;
searchQuery?: RegressionResultsSearchQuery;
ignoreDefaultQuery?: boolean;
}) {
let query: RegressionResultsSearchQuery = {
term: { [`${resultsField}.is_training`]: { value: isTraining } },
};

if (searchQuery !== undefined && ignoreDefaultQuery === true) {
query = searchQuery;
} else if (isRegressionResultsSearchBoolQuery(searchQuery)) {
const searchQueryClone = cloneDeep(searchQuery);
searchQueryClone.bool.must.push(query);
query = searchQueryClone;
}
return query;
}

export const loadEvalData = async ({
isTraining,
index,
dependentVariable,
resultsField,
predictionFieldName,
searchQuery,
ignoreDefaultQuery,
}: {
isTraining: boolean;
index: string;
dependentVariable: string;
resultsField: string;
predictionFieldName?: string;
searchQuery?: RegressionResultsSearchQuery;
ignoreDefaultQuery?: boolean;
}) => {
const results: LoadEvaluateResult = { success: false, eval: null, error: null };
const defaultPredictionField = `${dependentVariable}_prediction`;
const predictedField = `${resultsField}.${
predictionFieldName ? predictionFieldName : defaultPredictionField
}`;

const query = { term: { [`${resultsField}.is_training`]: { value: isTraining } } };
const query = getEvalQueryBody({ resultsField, isTraining, searchQuery, ignoreDefaultQuery });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the query matches just one document, I've seen that r_squared is returned by the evaluate endpoint as -Infinity. Can you check if this is correct, and if so, getValuesFromResponse needs to handle this case as currently it throws an error when calling toPrecision and the stats show up as -- (the mean_squared_error is a value and could be displayed).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is caused because if there is only one doc variance is 0 - this was then being divided by 0 in the calculation to get rSquared. Instead of dividing by 0, 0 should just be returned.

A fix for this is up already elastic/elasticsearch#49439

We shouldn't need to do any additional checking in the UI.


const config = {
index,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@import 'regression_exploration';
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
.mlDataFrameAnalyticsRegression__evaluateStat span.euiToolTipAnchor {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alvarezmelissa87 Don't forget that you can pass custom classnames to EuiToolTip anchors with the prop anchorClassName. It's safer than targeting the .eui class directly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the reminder 😄 Updated to use anchorClassName instead - 629d3fd

padding-top: $euiSizeL;
}
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,29 @@ export const ErrorCallout: FC<Props> = ({ error }) => {
</p>
</EuiCallOut>
);
} else if (error.includes('userProvidedQueryBuilder')) {
// query bar syntax is incorrect
errorCallout = (
<EuiCallOut
title={i18n.translate(
'xpack.ml.dataframe.analytics.regressionExploration.queryParsingErrorMessage',
{
defaultMessage: 'Unable to parse query.',
}
)}
color="primary"
>
<p>
{i18n.translate(
'xpack.ml.dataframe.analytics.regressionExploration.queryParsingErrorBody',
{
defaultMessage:
'The query syntax is invalid and returned no results. Please check the query syntax and try again.',
}
)}
</p>
</EuiCallOut>
);
}

return <Fragment>{errorCallout}</Fragment>;
Expand Down
Loading