Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Fix for errors when loading data views which are missing index #147916

Merged
merged 8 commits into from
Jan 17, 2023
Merged

[ML] Fix for errors when loading data views which are missing index #147916

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
ca72308
[ML] Fix for errors when loading data veiws which are missing index
jgowdyelastic Dec 21, 2022
afefe9d
adding comment
jgowdyelastic Dec 21, 2022
d694980
Merge branch '7.17' into fix-for-data-view-missing-index-errors
jgowdyelastic Jan 9, 2023
ab37293
fixing test types
jgowdyelastic Jan 10, 2023
b0d6ee9
Merge branch '7.17' into fix-for-data-view-missing-index-errors
jgowdyelastic Jan 11, 2023
5086eff
fixing dfa delete hook
jgowdyelastic Jan 13, 2023
d7c4be5
Merge branch '7.17' into fix-for-data-view-missing-index-errors
jgowdyelastic Jan 13, 2023
53a0563
Merge branch '7.17' into fix-for-data-view-missing-index-errors
jgowdyelastic Jan 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions ...frame_analytics/pages/analytics_management/components/action_delete/use_delete_action.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,13 @@ export const useDeleteAction = (canDeleteDataFrameAnalytics: boolean) => {

const checkIndexPatternExists = async () => {
try {
const dv = (await dataViews.find(indexName)).find(({ title }) => title === indexName);
if (dv !== undefined) {
setIndexPatternExists(true);
} else {
setIndexPatternExists(false);
if (indexName !== '') {
const dv = (await dataViews.find(indexName)).find(({ title }) => title === indexName);
if (dv !== undefined) {
setIndexPatternExists(true);
} else {
setIndexPatternExists(false);
}
}
setIsLoading(false);
} catch (e) {
Expand Down
4 changes: 2 additions & 2 deletions x-pack/plugins/ml/public/application/jobs/components/custom_url_editor/editor.test.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ import React from 'react';
import { CustomUrlEditor } from './editor';
import { TIME_RANGE_TYPE, URL_TYPE } from './constants';
import { CustomUrlSettings } from './utils';
import { DataView } from '../../../../../../../../src/plugins/data_views/common';
import { DataViewListItem } from '../../../../../../../../src/plugins/data_views/common';

function prepareTest(customUrl: CustomUrlSettings, setEditCustomUrlFn: (url: UrlConfig) => void) {
const savedCustomUrls = [
Expand Down Expand Up @@ -50,7 +50,7 @@ function prepareTest(customUrl: CustomUrlSettings, setEditCustomUrlFn: (url: Url
const indexPatterns = [
{ id: 'pattern1', title: 'Index Pattern 1' },
{ id: 'pattern2', title: 'Index Pattern 2' },
] as DataView[];
] as DataViewListItem[];

const queryEntityFieldNames = ['airline'];

Expand Down
4 changes: 2 additions & 2 deletions x-pack/plugins/ml/public/application/jobs/components/custom_url_editor/editor.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ import { isValidLabel } from '../../../util/custom_url_utils';

import { TIME_RANGE_TYPE, URL_TYPE } from './constants';
import { UrlConfig } from '../../../../../common/types/custom_urls';
import { DataView } from '../../../../../../../../src/plugins/data_views/common';
import { DataViewListItem } from '../../../../../../../../src/plugins/data_views/common';

function getLinkToOptions() {
return [
Expand Down Expand Up @@ -59,7 +59,7 @@ interface CustomUrlEditorProps {
setEditCustomUrl: (url: any) => void;
savedCustomUrls: UrlConfig[];
dashboards: any[];
indexPatterns: DataView[];
indexPatterns: DataViewListItem[];
queryEntityFieldNames: string[];
}

Expand Down
4 changes: 2 additions & 2 deletions x-pack/plugins/ml/public/application/jobs/components/custom_url_editor/utils.d.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
* 2.0.
*/

import { DataView } from '../../../../../../../../src/plugins/data_views/common';
import { DataViewListItem } from '../../../../../../../../src/plugins/data_views/common';
import { UrlConfig } from '../../../../../common/types/custom_urls';
import { Job } from '../../../../../common/types/anomaly_detection_jobs';
import { TimeRangeType } from './constants';
Expand Down Expand Up @@ -34,7 +34,7 @@ export function isValidCustomUrlSettingsTimeRange(timeRangeSettings: any): boole
export function getNewCustomUrlDefaults(
job: Job,
dashboards: any[],
indexPatterns: DataView[]
indexPatterns: DataViewListItem[]
): CustomUrlSettings;
export function getQueryEntityFieldNames(job: Job): string[];
export function isValidCustomUrlSettings(
Expand Down
4 changes: 2 additions & 2 deletions ...k/plugins/ml/public/application/jobs/jobs_list/components/edit_job_flyout/edit_utils.d.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
* 2.0.
*/

import type { DataView } from 'src/plugins/data_views/common';
import type { DataViewListItem } from 'src/plugins/data_views/common';

export function loadSavedDashboards(maxNumber: number): Promise<any[]>;
export function loadIndexPatterns(maxNumber: number): Promise<DataView[]>;
export function loadIndexPatterns(): Promise<DataViewListItem[]>;
4 changes: 2 additions & 2 deletions x-pack/plugins/ml/public/application/jobs/jobs_list/components/edit_job_flyout/edit_utils.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,14 +102,14 @@ export function loadSavedDashboards(maxNumber) {
});
}

export function loadIndexPatterns(maxNumber) {
export function loadIndexPatterns() {
// Loads the list of Kibana index patterns, as used in editing custom URLs.
// TODO - amend loadIndexPatterns in index_utils.js to do the request,
// without needing an Angular Provider.
return new Promise((resolve, reject) => {
const dataViewsContract = getDataViews();
dataViewsContract
.find('*', maxNumber)
.getIdsWithTitle()
.then((dataViews) => {
const sortedDataViews = dataViews.sort((a, b) => a.title.localeCompare(b.title));
resolve(sortedDataViews);
Expand Down
7 changes: 3 additions & 4 deletions ...gins/ml/public/application/jobs/jobs_list/components/edit_job_flyout/tabs/custom_urls.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,10 @@ import { loadSavedDashboards, loadIndexPatterns } from '../edit_utils';
import { openCustomUrlWindow } from '../../../../../util/custom_url_utils';
import { Job } from '../../../../../../../common/types/anomaly_detection_jobs';
import { UrlConfig } from '../../../../../../../common/types/custom_urls';
import { DataView } from '../../../../../../../../../../src/plugins/data_views/common';
import { DataViewListItem } from '../../../../../../../../../../src/plugins/data_views/common';
import { MlKibanaReactContextValue } from '../../../../../contexts/kibana';

const MAX_NUMBER_DASHBOARDS = 1000;
const MAX_NUMBER_INDEX_PATTERNS = 1000;

interface CustomUrlsProps {
job: Job;
Expand All @@ -54,7 +53,7 @@ interface CustomUrlsProps {
interface CustomUrlsState {
customUrls: UrlConfig[];
dashboards: any[];
indexPatterns: DataView[];
indexPatterns: DataViewListItem[];
queryEntityFieldNames: string[];
editorOpen: boolean;
editorSettings?: CustomUrlSettings;
Expand Down Expand Up @@ -100,7 +99,7 @@ class CustomUrlsUI extends Component<CustomUrlsProps, CustomUrlsState> {
);
});

loadIndexPatterns(MAX_NUMBER_INDEX_PATTERNS)
loadIndexPatterns()
.then((indexPatterns) => {
this.setState({ indexPatterns });
})
Expand Down
20 changes: 19 additions & 1 deletion x-pack/plugins/ml/public/application/util/index_utils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,25 @@ let indexPatternsContract: DataViewsContract | null = null;
export async function loadIndexPatterns(indexPatterns: DataViewsContract) {
indexPatternsContract = indexPatterns;
const dataViewsContract = getDataViews();
indexPatternCache = await dataViewsContract.find('*', 10000);
const idsAndTitles = await dataViewsContract.getIdsWithTitle();
Copy link
Member Author

@jgowdyelastic jgowdyelastic Dec 21, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now load all of the ids and titles of the data views and attempt to get the fields for them to work out if the index really exists.
Once we have this filtered list of ids for data views which do exist, we can load the actual data views.

It might be better to use an exists check rather than getFieldsForIndexPattern but that will add additional dependencies to this function.


const dataViewsThatExist = (
await Promise.allSettled(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first time I visit the Data Frame Analytics page, I still see the error toasts - one for each index pattern for which the index doesn't exist.

image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5086eff

// attempt to load the fields for every data view.
// if the index doesn't exist an error is thrown which we can catch.
// This is preferable to the get function which display an
// error toast for every missing index.
idsAndTitles.map(({ title }) => dataViewsContract.getFieldsForIndexPattern({ title }))
)
).reduce<string[]>((acc, { status }, i) => {
if (status === 'fulfilled') {
acc.push(idsAndTitles[i].id);
}
return acc;
}, []);

// load each data view which has a real index behind it.
indexPatternCache = await Promise.all(dataViewsThatExist.map(dataViewsContract.get));
return indexPatternCache;
}

Expand Down