Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a VisAugmenter stats API #4006

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

import { SavedObjectAttributes } from 'opensearch-dashboards/server';

export interface AugmentVisSavedObjectAttributes extends SavedObjectAttributes {
id: string;
title: string;
description?: string;
originPlugin: string;
pluginResource: {
type: string;
id: string;
};
visLayerExpressionFn: {
type: string;
name: string;
};
version: number;
// Following fields are optional since they will get set/removed during the extraction/injection
// of the vis reference
visName?: string;
visId?: string;
visReference?: {
id: string;
name: string;
};
Comment on lines +24 to +29
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make a separate interface for these attributes? The convention for SavedObjectAttributes is to keep this interface identical to that of the saved object itself. If these properties are not going to be stored in the saved object, lets make a separate interface that extends this instead and use that. Also i dont see this being used anywhere, is it used in the AD plugin? if so can we just create the new interface with these properties there?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are used in the saved object actually - the visName, is populated to reference the visualization reference in the references array, and the visId and visReference are populated when pulling out the stored vis in references. This follows the same convention as used by visualization saved object extractReferences/injectReferences logic.

The visualization saved object uses the open-ended SavedObjectAttributes, and uses values there that aren't actually indexed in the saved object. I followed the same. From my understanding, the attributes can be values before/after the reference extraction/injection, based on the arguments to those functions themselves.

References:
visualization extract/inject references: link
augment-vis extract/inject references: link

// Error may be populated if there is some issue when parsing the attribute values
error?: string;
}
12 changes: 12 additions & 0 deletions src/plugins/vis_augmenter/common/constants.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

export const APP_PATH = {
STATS: '/stats',
};
export const APP_API = '/api/vis_augmenter';

// used for limiting results received from the stats API
export const PER_PAGE_REQUEST_NUMBER = 50;
7 changes: 7 additions & 0 deletions src/plugins/vis_augmenter/common/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

export * from './constants';
export { AugmentVisSavedObjectAttributes } from './augment_vis_saved_object_attributes';
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ import {
SavedObject,
SavedObjectOpenSearchDashboardsServices,
} from '../../../saved_objects/public';
import { IIndexPattern } from '../../../data/public';
import { extractReferences, injectReferences } from './saved_augment_vis_references';
import { AugmentVisSavedObjectAttributes } from '../../common';

const name = 'augment-vis';

Expand All @@ -24,28 +24,19 @@ export function createSavedAugmentVisClass(services: SavedObjectOpenSearchDashbo

class SavedAugmentVis extends SavedObjectClass {
public static type: string = name;
public static mapping: Record<string, string> = {
description: 'text',
pluginResourceId: 'text',
visId: 'keyword',
visLayerExpressionFn: 'text',
version: 'integer',
};
public static mapping: AugmentVisSavedObjectAttributes;

constructor(opts: Record<string, unknown> | string = {}) {
if (typeof opts !== 'object') {
opts = { id: opts };
}
constructor(opts: AugmentVisSavedObjectAttributes) {
super({
type: SavedAugmentVis.type,
mapping: SavedAugmentVis.mapping,
extractReferences,
injectReferences,
id: (opts.id as string) || '',
indexPattern: opts.indexPattern as IIndexPattern,
ohltyler marked this conversation as resolved.
Show resolved Hide resolved
defaults: {
description: get(opts, 'description', ''),
pluginResourceId: get(opts, 'pluginResourceId', ''),
originPlugin: get(opts, 'originPlugin', ''),
pluginResource: get(opts, 'pluginResource', {}),
visId: get(opts, 'visId', ''),
visLayerExpressionFn: get(opts, 'visLayerExpressionFn', {}),
version: 1,
Expand All @@ -55,5 +46,5 @@ export function createSavedAugmentVisClass(services: SavedObjectOpenSearchDashbo
}
}

return SavedAugmentVis as new (opts: Record<string, unknown> | string) => SavedObject;
return SavedAugmentVis as new (opts: AugmentVisSavedObjectAttributes) => SavedObject;
}
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,14 @@
* SPDX-License-Identifier: Apache-2.0
*/

import { VisLayerExpressionFn, VisLayerTypes } from '../types';
import { VisLayerTypes } from '../types';
import { VisLayerExpressionFn } from '../expressions';
import {
createSavedAugmentVisLoader,
SavedObjectOpenSearchDashboardsServicesWithAugmentVis,
} from './saved_augment_vis';
import { generateAugmentVisSavedObject, getMockAugmentVisSavedObjectClient } from './utils';
import { ISavedPluginResource } from './types';

describe('SavedObjectLoaderAugmentVis', () => {
const fn = {
Expand All @@ -18,22 +20,74 @@ describe('SavedObjectLoaderAugmentVis', () => {
testArg: 'test-value',
},
} as VisLayerExpressionFn;
const validObj1 = generateAugmentVisSavedObject('valid-obj-id-1', fn, 'test-vis-id');
const validObj2 = generateAugmentVisSavedObject('valid-obj-id-2', fn, 'test-vis-id');
const originPlugin = 'test-plugin';
const pluginResource = {
type: 'test-plugin',
id: 'test-plugin-resource-id',
};
const validObj1 = generateAugmentVisSavedObject(
'valid-obj-id-1',
fn,
'test-vis-id',
originPlugin,
pluginResource
);
const validObj2 = generateAugmentVisSavedObject(
'valid-obj-id-2',
fn,
'test-vis-id',
originPlugin,
pluginResource
);
const invalidFnTypeObj = generateAugmentVisSavedObject(
'invalid-fn-obj-id-1',
{
...fn,
// @ts-ignore
type: 'invalid-type',
},
'test-vis-id'
'test-vis-id',
originPlugin,
pluginResource
);

const missingFnObj = generateAugmentVisSavedObject(
'missing-fn-obj-id-1',
{} as VisLayerExpressionFn,
'test-vis-id'
'test-vis-id',
originPlugin,
pluginResource
);

const missingOriginPluginObj = generateAugmentVisSavedObject(
'missing-origin-plugin-obj-id-1',
fn,
'test-vis-id',
// @ts-ignore
undefined,
pluginResource
);

const missingPluginResourceTypeObj = generateAugmentVisSavedObject(
'missing-plugin-resource-type-obj-id-1',
fn,
'test-vis-id',
// @ts-ignore
originPlugin,
{
id: pluginResource.id,
} as ISavedPluginResource
);

const missingPluginResourceIdObj = generateAugmentVisSavedObject(
'missing-plugin-resource-id-obj-id-1',
fn,
'test-vis-id',
// @ts-ignore
originPlugin,
{
type: pluginResource.type,
} as ISavedPluginResource
);

it('find returns single saved obj', async () => {
Expand Down Expand Up @@ -105,4 +159,36 @@ describe('SavedObjectLoaderAugmentVis', () => {
expect(resp.hits[0].id).toEqual('valid-obj-id-1');
expect(resp.hits[0].error).toEqual('visReference is missing in augment-vis saved object');
});

it('findAll returns obj with missing originPlugin', async () => {
const loader = createSavedAugmentVisLoader({
savedObjectsClient: getMockAugmentVisSavedObjectClient([missingOriginPluginObj]),
} as SavedObjectOpenSearchDashboardsServicesWithAugmentVis);
const resp = await loader.findAll();
expect(resp.hits.length).toEqual(1);
expect(resp.hits[0].id).toEqual('missing-origin-plugin-obj-id-1');
expect(resp.hits[0].error).toEqual('originPlugin is missing in augment-vis saved object');
});

it('findAll returns obj with missing plugin resource type', async () => {
const loader = createSavedAugmentVisLoader({
savedObjectsClient: getMockAugmentVisSavedObjectClient([missingPluginResourceTypeObj]),
} as SavedObjectOpenSearchDashboardsServicesWithAugmentVis);
const resp = await loader.findAll();
expect(resp.hits.length).toEqual(1);
expect(resp.hits[0].id).toEqual('missing-plugin-resource-type-obj-id-1');
expect(resp.hits[0].error).toEqual(
'pluginResource.type is missing in augment-vis saved object'
);
});

it('findAll returns obj with missing plugin resource id', async () => {
const loader = createSavedAugmentVisLoader({
savedObjectsClient: getMockAugmentVisSavedObjectClient([missingPluginResourceIdObj]),
} as SavedObjectOpenSearchDashboardsServicesWithAugmentVis);
const resp = await loader.findAll();
expect(resp.hits.length).toEqual(1);
expect(resp.hits[0].id).toEqual('missing-plugin-resource-id-obj-id-1');
expect(resp.hits[0].error).toEqual('pluginResource.id is missing in augment-vis saved object');
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import {
} from '../../../saved_objects/public';
import { createSavedAugmentVisClass } from './_saved_augment_vis';
import { VisLayerTypes } from '../types';
import { AugmentVisSavedObjectAttributes } from '../../common';

// eslint-disable-next-line @typescript-eslint/no-empty-interface
export interface SavedObjectOpenSearchDashboardsServicesWithAugmentVis
Expand All @@ -21,9 +22,9 @@ export function createSavedAugmentVisLoader(
const { savedObjectsClient } = services;

class SavedObjectLoaderAugmentVis extends SavedObjectLoader {
mapHitSource = (source: Record<string, any>, id: string) => {
mapHitSource = (source: AugmentVisSavedObjectAttributes, id: string) => {
source.id = id;
source.visId = get(source, 'visReference.id', '');
source.visId = get(source, 'visReference.id', '') as string;

if (isEmpty(source.visReference)) {
source.error = 'visReference is missing in augment-vis saved object';
Expand All @@ -33,10 +34,22 @@ export function createSavedAugmentVisLoader(
source.error = 'visLayerExpressionFn is missing in augment-vis saved object';
return source;
}
if (!(get(source, 'visLayerExpressionFn.type', '') in VisLayerTypes)) {
if (!((get(source, 'visLayerExpressionFn.type', '') as string) in VisLayerTypes)) {
source.error = 'Unknown VisLayer expression function type';
return source;
}
if (get(source, 'originPlugin', undefined) === undefined) {
source.error = 'originPlugin is missing in augment-vis saved object';
return source;
}
if (get(source, 'pluginResource.type', undefined) === undefined) {
source.error = 'pluginResource.type is missing in augment-vis saved object';
return source;
}
if (get(source, 'pluginResource.id', undefined) === undefined) {
source.error = 'pluginResource.id is missing in augment-vis saved object';
return source;
}
return source;
};

Expand All @@ -48,7 +61,7 @@ export function createSavedAugmentVisLoader(
*/
mapSavedObjectApiHits(hit: {
references: any[];
attributes: Record<string, unknown>;
attributes: AugmentVisSavedObjectAttributes;
id: string;
}) {
// For now we are assuming only one vis reference per saved object.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import {
injectReferences,
VIS_REFERENCE_NAME,
} from './saved_augment_vis_references';
import { AugmentVisSavedObject } from '../types';
import { AugmentVisSavedObject } from './types';

describe('extractReferences()', () => {
test('extracts nothing if visId is null', () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
*/

import { SavedObjectAttributes, SavedObjectReference } from '../../../../core/public';
import { AugmentVisSavedObject } from '../types';
import { AugmentVisSavedObjectAttributes } from '../../common';
import { AugmentVisSavedObject } from './types';

/**
* Note that references aren't stored in the object's client-side interface (AugmentVisSavedObject).
Expand Down Expand Up @@ -35,7 +36,7 @@ export function extractReferences({
attributes: SavedObjectAttributes;
references: SavedObjectReference[];
}) {
const updatedAttributes = { ...attributes };
const updatedAttributes = { ...attributes } as AugmentVisSavedObjectAttributes;
const updatedReferences = [...references];

// Extract saved object
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,17 @@
import { SavedObject } from '../../../saved_objects/public';
import { VisLayerExpressionFn } from '../expressions';

export interface ISavedPluginResource {
type: string;
id: string;
}

export interface ISavedAugmentVis {
id?: string;
title: string;
description?: string;
pluginResourceId: string;
originPlugin: string;
pluginResource: ISavedPluginResource;
visName?: string;
visId?: string;
visLayerExpressionFn: VisLayerExpressionFn;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
*/

import { getSavedAugmentVisLoader } from '../../services';
import { ISavedAugmentVis } from '../../types';
import { ISavedAugmentVis } from '../types';

/**
* Create an augment vis saved object given an object that
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,24 @@
*/

import { cloneDeep } from 'lodash';
import { VisLayerExpressionFn, ISavedAugmentVis } from '../../';
import { VisLayerExpressionFn, ISavedAugmentVis, ISavedPluginResource } from '../../';
import { VIS_REFERENCE_NAME } from '../saved_augment_vis_references';

const pluginResourceId = 'test-plugin-resource-id';
const title = 'test-title';
const version = 1;

export const generateAugmentVisSavedObject = (
idArg: string,
exprFnArg: VisLayerExpressionFn,
visIdArg: string
visIdArg: string,
originPluginArg: string,
pluginResourceArg: ISavedPluginResource
) => {
return {
id: idArg,
title,
pluginResourceId,
originPlugin: originPluginArg,
pluginResource: pluginResourceArg,
visLayerExpressionFn: exprFnArg,
VIS_REFERENCE_NAME,
visId: visIdArg,
Expand Down
Loading