diff --git a/docs/api/saved-objects/find.asciidoc b/docs/api/saved-objects/find.asciidoc index c43b58d3aa989..73a8fbf0a820f 100644 --- a/docs/api/saved-objects/find.asciidoc +++ b/docs/api/saved-objects/find.asciidoc @@ -54,9 +54,13 @@ experimental[] Retrieve a paginated set of {kib} saved objects by various condit `filter`:: (Optional, string) The filter is a KQL string with the caveat that if you filter with an attribute from your type saved object. - It should look like that savedObjectType.attributes.title: "myTitle". However, If you used a direct attribute of a saved object like `updatedAt`, + It should look like that savedObjectType.attributes.title: "myTitle". However, If you used a direct attribute of a saved object like `updated_at`, you will have to define your filter like that savedObjectType.updatedAt > 2018-12-22. +`aggs`:: + (Optional, string) The aggs will support aggregation string with the caveat that your field from the aggregation will have the attribute from your type saved object, + it should look like this: savedObjectType.attributes.field. However, If you use a direct attribute of a saved object like updatedAt, you will have to define your filter like this: savedObjectType.updated_at. + NOTE: As objects change in {kib}, the results on each page of the response also change. Use the find API for traditional paginated results, but avoid using it to export large amounts of data. diff --git a/docs/development/core/public/kibana-plugin-core-public.savedobjectsclient.find.md b/docs/development/core/public/kibana-plugin-core-public.savedobjectsclient.find.md index ddd8b207e3d78..fc9652b96450f 100644 --- a/docs/development/core/public/kibana-plugin-core-public.savedobjectsclient.find.md +++ b/docs/development/core/public/kibana-plugin-core-public.savedobjectsclient.find.md @@ -9,5 +9,5 @@ Search for objects Signature: ```typescript -find: (options: SavedObjectsFindOptions) => Promise>; +find: (options: SavedObjectsFindOptions) => Promise>; ``` diff --git a/docs/development/core/public/kibana-plugin-core-public.savedobjectsclient.md b/docs/development/core/public/kibana-plugin-core-public.savedobjectsclient.md index 6e53b169b8bed..1ec756f8d743d 100644 --- a/docs/development/core/public/kibana-plugin-core-public.savedobjectsclient.md +++ b/docs/development/core/public/kibana-plugin-core-public.savedobjectsclient.md @@ -24,7 +24,7 @@ The constructor for this class is marked as internal. Third-party code should no | [bulkGet](./kibana-plugin-core-public.savedobjectsclient.bulkget.md) | | (objects?: Array<{
id: string;
type: string;
}>) => Promise<SavedObjectsBatchResponse<unknown>> | Returns an array of objects by id | | [create](./kibana-plugin-core-public.savedobjectsclient.create.md) | | <T = unknown>(type: string, attributes: T, options?: SavedObjectsCreateOptions) => Promise<SimpleSavedObject<T>> | Persists an object | | [delete](./kibana-plugin-core-public.savedobjectsclient.delete.md) | | (type: string, id: string, options?: SavedObjectsDeleteOptions | undefined) => ReturnType<SavedObjectsApi['delete']> | Deletes an object | -| [find](./kibana-plugin-core-public.savedobjectsclient.find.md) | | <T = unknown>(options: SavedObjectsFindOptions) => Promise<SavedObjectsFindResponsePublic<T>> | Search for objects | +| [find](./kibana-plugin-core-public.savedobjectsclient.find.md) | | <T = unknown, A = unknown>(options: SavedObjectsFindOptions) => Promise<SavedObjectsFindResponsePublic<T, unknown>> | Search for objects | | [get](./kibana-plugin-core-public.savedobjectsclient.get.md) | | <T = unknown>(type: string, id: string) => Promise<SimpleSavedObject<T>> | Fetches a single object | ## Methods diff --git a/docs/development/core/public/kibana-plugin-core-public.savedobjectscreateoptions.md b/docs/development/core/public/kibana-plugin-core-public.savedobjectscreateoptions.md index b1b93407d4ff1..85a34708cc0dc 100644 --- a/docs/development/core/public/kibana-plugin-core-public.savedobjectscreateoptions.md +++ b/docs/development/core/public/kibana-plugin-core-public.savedobjectscreateoptions.md @@ -4,7 +4,6 @@ ## SavedObjectsCreateOptions interface - Signature: ```typescript diff --git a/docs/development/core/public/kibana-plugin-core-public.savedobjectsfindresponsepublic.aggregations.md b/docs/development/core/public/kibana-plugin-core-public.savedobjectsfindresponsepublic.aggregations.md new file mode 100644 index 0000000000000..14401b02f25c7 --- /dev/null +++ b/docs/development/core/public/kibana-plugin-core-public.savedobjectsfindresponsepublic.aggregations.md @@ -0,0 +1,11 @@ + + +[Home](./index.md) > [kibana-plugin-core-public](./kibana-plugin-core-public.md) > [SavedObjectsFindResponsePublic](./kibana-plugin-core-public.savedobjectsfindresponsepublic.md) > [aggregations](./kibana-plugin-core-public.savedobjectsfindresponsepublic.aggregations.md) + +## SavedObjectsFindResponsePublic.aggregations property + +Signature: + +```typescript +aggregations?: A; +``` diff --git a/docs/development/core/public/kibana-plugin-core-public.savedobjectsfindresponsepublic.md b/docs/development/core/public/kibana-plugin-core-public.savedobjectsfindresponsepublic.md index 7d75878041264..6f2276194f054 100644 --- a/docs/development/core/public/kibana-plugin-core-public.savedobjectsfindresponsepublic.md +++ b/docs/development/core/public/kibana-plugin-core-public.savedobjectsfindresponsepublic.md @@ -11,13 +11,14 @@ Return type of the Saved Objects `find()` method. Signature: ```typescript -export interface SavedObjectsFindResponsePublic extends SavedObjectsBatchResponse +export interface SavedObjectsFindResponsePublic extends SavedObjectsBatchResponse ``` ## Properties | Property | Type | Description | | --- | --- | --- | +| [aggregations](./kibana-plugin-core-public.savedobjectsfindresponsepublic.aggregations.md) | A | | | [page](./kibana-plugin-core-public.savedobjectsfindresponsepublic.page.md) | number | | | [perPage](./kibana-plugin-core-public.savedobjectsfindresponsepublic.perpage.md) | number | | | [total](./kibana-plugin-core-public.savedobjectsfindresponsepublic.total.md) | number | | diff --git a/docs/development/core/server/kibana-plugin-core-server.savedobjectsclient.find.md b/docs/development/core/server/kibana-plugin-core-server.savedobjectsclient.find.md index 9a4c3df5d2d92..56d76125108d1 100644 --- a/docs/development/core/server/kibana-plugin-core-server.savedobjectsclient.find.md +++ b/docs/development/core/server/kibana-plugin-core-server.savedobjectsclient.find.md @@ -9,7 +9,7 @@ Find all SavedObjects matching the search query Signature: ```typescript -find(options: SavedObjectsFindOptions): Promise>; +find(options: SavedObjectsFindOptions): Promise>; ``` ## Parameters @@ -20,5 +20,5 @@ find(options: SavedObjectsFindOptions): PromiseReturns: -`Promise>` +`Promise>` diff --git a/docs/development/core/server/kibana-plugin-core-server.savedobjectsfindresponse.aggregations.md b/docs/development/core/server/kibana-plugin-core-server.savedobjectsfindresponse.aggregations.md new file mode 100644 index 0000000000000..17a899f4c8280 --- /dev/null +++ b/docs/development/core/server/kibana-plugin-core-server.savedobjectsfindresponse.aggregations.md @@ -0,0 +1,11 @@ + + +[Home](./index.md) > [kibana-plugin-core-server](./kibana-plugin-core-server.md) > [SavedObjectsFindResponse](./kibana-plugin-core-server.savedobjectsfindresponse.md) > [aggregations](./kibana-plugin-core-server.savedobjectsfindresponse.aggregations.md) + +## SavedObjectsFindResponse.aggregations property + +Signature: + +```typescript +aggregations?: A; +``` diff --git a/docs/development/core/server/kibana-plugin-core-server.savedobjectsfindresponse.md b/docs/development/core/server/kibana-plugin-core-server.savedobjectsfindresponse.md index 4ed069d1598fe..7facba01a012d 100644 --- a/docs/development/core/server/kibana-plugin-core-server.savedobjectsfindresponse.md +++ b/docs/development/core/server/kibana-plugin-core-server.savedobjectsfindresponse.md @@ -11,13 +11,14 @@ Return type of the Saved Objects `find()` method. Signature: ```typescript -export interface SavedObjectsFindResponse +export interface SavedObjectsFindResponse ``` ## Properties | Property | Type | Description | | --- | --- | --- | +| [aggregations](./kibana-plugin-core-server.savedobjectsfindresponse.aggregations.md) | A | | | [page](./kibana-plugin-core-server.savedobjectsfindresponse.page.md) | number | | | [per\_page](./kibana-plugin-core-server.savedobjectsfindresponse.per_page.md) | number | | | [saved\_objects](./kibana-plugin-core-server.savedobjectsfindresponse.saved_objects.md) | Array<SavedObjectsFindResult<T>> | | diff --git a/docs/development/core/server/kibana-plugin-core-server.savedobjectsrepository.find.md b/docs/development/core/server/kibana-plugin-core-server.savedobjectsrepository.find.md index d3e93e7af2aa0..5c823b7567918 100644 --- a/docs/development/core/server/kibana-plugin-core-server.savedobjectsrepository.find.md +++ b/docs/development/core/server/kibana-plugin-core-server.savedobjectsrepository.find.md @@ -7,7 +7,7 @@ Signature: ```typescript -find(options: SavedObjectsFindOptions): Promise>; +find(options: SavedObjectsFindOptions): Promise>; ``` ## Parameters @@ -18,7 +18,7 @@ find(options: SavedObjectsFindOptions): PromiseReturns: -`Promise>` +`Promise>` {promise} - { saved\_objects: \[{ id, type, version, attributes }\], total, per\_page, page } diff --git a/docs/development/core/server/kibana-plugin-core-server.savedobjectsutils.createemptyfindresponse.md b/docs/development/core/server/kibana-plugin-core-server.savedobjectsutils.createemptyfindresponse.md index 40e865cb02ce8..23cbebf22aa21 100644 --- a/docs/development/core/server/kibana-plugin-core-server.savedobjectsutils.createemptyfindresponse.md +++ b/docs/development/core/server/kibana-plugin-core-server.savedobjectsutils.createemptyfindresponse.md @@ -9,5 +9,5 @@ Creates an empty response for a find operation. This is only intended to be used Signature: ```typescript -static createEmptyFindResponse: ({ page, perPage, }: SavedObjectsFindOptions) => SavedObjectsFindResponse; +static createEmptyFindResponse: ({ page, perPage, }: SavedObjectsFindOptions) => SavedObjectsFindResponse; ``` diff --git a/docs/development/core/server/kibana-plugin-core-server.savedobjectsutils.md b/docs/development/core/server/kibana-plugin-core-server.savedobjectsutils.md index 83831f65bd41a..481fdd3439cb1 100644 --- a/docs/development/core/server/kibana-plugin-core-server.savedobjectsutils.md +++ b/docs/development/core/server/kibana-plugin-core-server.savedobjectsutils.md @@ -15,7 +15,7 @@ export declare class SavedObjectsUtils | Property | Modifiers | Type | Description | | --- | --- | --- | --- | -| [createEmptyFindResponse](./kibana-plugin-core-server.savedobjectsutils.createemptyfindresponse.md) | static | <T>({ page, perPage, }: SavedObjectsFindOptions) => SavedObjectsFindResponse<T> | Creates an empty response for a find operation. This is only intended to be used by saved objects client wrappers. | +| [createEmptyFindResponse](./kibana-plugin-core-server.savedobjectsutils.createemptyfindresponse.md) | static | <T, A>({ page, perPage, }: SavedObjectsFindOptions) => SavedObjectsFindResponse<T, A> | Creates an empty response for a find operation. This is only intended to be used by saved objects client wrappers. | | [namespaceIdToString](./kibana-plugin-core-server.savedobjectsutils.namespaceidtostring.md) | static | (namespace?: string | undefined) => string | Converts a given saved object namespace ID to its string representation. All namespace IDs have an identical string representation, with the exception of the undefined namespace ID (which has a namespace string of 'default'). | | [namespaceStringToId](./kibana-plugin-core-server.savedobjectsutils.namespacestringtoid.md) | static | (namespace: string) => string | undefined | Converts a given saved object namespace string to its ID representation. All namespace strings have an identical ID representation, with the exception of the 'default' namespace string (which has a namespace ID of undefined). | diff --git a/src/core/public/public.api.md b/src/core/public/public.api.md index 37e57a9ee606e..b7cbd22c870b0 100644 --- a/src/core/public/public.api.md +++ b/src/core/public/public.api.md @@ -1088,7 +1088,7 @@ export class SavedObjectsClient { // Warning: (ae-forgotten-export) The symbol "SavedObjectsClientContract" needs to be exported by the entry point index.d.ts delete: (type: string, id: string, options?: SavedObjectsDeleteOptions | undefined) => ReturnType; // Warning: (ae-forgotten-export) The symbol "SavedObjectsFindOptions" needs to be exported by the entry point index.d.ts - find: (options: SavedObjectsFindOptions_2) => Promise>; + find: (options: SavedObjectsFindOptions_2) => Promise>; get: (type: string, id: string) => Promise>; update(type: string, id: string, attributes: T, { version, migrationVersion, references }?: SavedObjectsUpdateOptions): Promise>; } @@ -1096,6 +1096,8 @@ export class SavedObjectsClient { // @public export type SavedObjectsClientContract = PublicMethodsOf; +// Warning: (ae-missing-release-tag) "SavedObjectsCreateOptions" is exported by the package, but it is missing a release tag (@alpha, @beta, @public, or @internal) +// // @public (undocumented) export interface SavedObjectsCreateOptions { id?: string; @@ -1107,6 +1109,8 @@ export interface SavedObjectsCreateOptions { // @public (undocumented) export interface SavedObjectsFindOptions { + // @alpha + aggs?: Record; defaultSearchOperator?: 'AND' | 'OR'; fields?: string[]; // Warning: (ae-forgotten-export) The symbol "KueryNode" needs to be exported by the entry point index.d.ts @@ -1143,7 +1147,9 @@ export interface SavedObjectsFindOptionsReference { } // @public -export interface SavedObjectsFindResponsePublic extends SavedObjectsBatchResponse { +export interface SavedObjectsFindResponsePublic extends SavedObjectsBatchResponse { + // (undocumented) + aggregations?: A; // (undocumented) page: number; // (undocumented) diff --git a/src/core/public/saved_objects/saved_objects_client.ts b/src/core/public/saved_objects/saved_objects_client.ts index d8b65dbc2330e..777a2647d9e85 100644 --- a/src/core/public/saved_objects/saved_objects_client.ts +++ b/src/core/public/saved_objects/saved_objects_client.ts @@ -39,7 +39,6 @@ type SavedObjectsFindOptions = Omit< type PromiseType> = T extends Promise ? U : never; -/** @public */ export interface SavedObjectsCreateOptions { /** * (Not recommended) Specify an id instead of having the saved objects service generate one for you. @@ -110,7 +109,9 @@ export interface SavedObjectsDeleteOptions { * * @public */ -export interface SavedObjectsFindResponsePublic extends SavedObjectsBatchResponse { +export interface SavedObjectsFindResponsePublic + extends SavedObjectsBatchResponse { + aggregations?: A; total: number; perPage: number; page: number; @@ -317,7 +318,7 @@ export class SavedObjectsClient { * @property {object} [options.hasReference] - { type, id } * @returns A find result with objects matching the specified search. */ - public find = ( + public find = ( options: SavedObjectsFindOptions ): Promise> => { const path = this.getPath(['_find']); @@ -333,6 +334,7 @@ export class SavedObjectsClient { sortField: 'sort_field', type: 'type', filter: 'filter', + aggs: 'aggs', namespaces: 'namespaces', preference: 'preference', }; @@ -349,6 +351,12 @@ export class SavedObjectsClient { query.has_reference = JSON.stringify(query.has_reference); } + // `aggs` is a structured object. we need to stringify it before sending it, as `fetch` + // is not doing it implicitly. + if (query.aggs) { + query.aggs = JSON.stringify(query.aggs); + } + const request: ReturnType = this.savedObjectsFetch(path, { method: 'GET', query, @@ -359,6 +367,7 @@ export class SavedObjectsClient { SavedObjectsFindResponsePublic >( { + aggregations: 'aggregations', saved_objects: 'savedObjects', total: 'total', per_page: 'perPage', diff --git a/src/core/server/saved_objects/routes/find.ts b/src/core/server/saved_objects/routes/find.ts index 915d0cccf7af9..1b2e7a445548a 100644 --- a/src/core/server/saved_objects/routes/find.ts +++ b/src/core/server/saved_objects/routes/find.ts @@ -49,6 +49,7 @@ export const registerFindRoute = (router: IRouter) => { has_reference_operator: searchOperatorSchema, fields: schema.maybe(schema.oneOf([schema.string(), schema.arrayOf(schema.string())])), filter: schema.maybe(schema.string()), + aggs: schema.maybe(schema.string()), namespaces: schema.maybe( schema.oneOf([schema.string(), schema.arrayOf(schema.string())]) ), @@ -74,6 +75,7 @@ export const registerFindRoute = (router: IRouter) => { hasReferenceOperator: query.has_reference_operator, fields: typeof query.fields === 'string' ? [query.fields] : query.fields, filter: query.filter, + aggs: query.aggs ? JSON.parse(query.aggs) : undefined, namespaces, }); diff --git a/src/core/server/saved_objects/service/lib/aggs_utils.test.ts b/src/core/server/saved_objects/service/lib/aggs_utils.test.ts new file mode 100644 index 0000000000000..a273772367a38 --- /dev/null +++ b/src/core/server/saved_objects/service/lib/aggs_utils.test.ts @@ -0,0 +1,249 @@ +/* + * Licensed to Elasticsearch B.V. under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch B.V. licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { validateGetSavedObjectsAggs } from './aggs_utils'; +import { mockMappings } from './filter_utils.test'; + +describe('Filter Utils', () => { + describe('#validateGetSavedObjectsAggs', () => { + test('Validate a simple aggregations', () => { + expect( + validateGetSavedObjectsAggs( + ['foo'], + { aggName: { max: { field: 'foo.attributes.bytes' } } }, + mockMappings + ) + ).toEqual({ + aggName: { + max: { + field: 'foo.bytes', + }, + }, + }); + }); + + test('Validate a nested field in simple aggregations', () => { + expect( + validateGetSavedObjectsAggs( + ['alert'], + { aggName: { cardinality: { field: 'alert.attributes.actions.group' } } }, + mockMappings + ) + ).toEqual({ + aggName: { + cardinality: { + field: 'alert.actions.group', + }, + }, + }); + }); + + test('Validate a nested aggregations', () => { + expect( + validateGetSavedObjectsAggs( + ['alert'], + { + aggName: { + cardinality: { + field: 'alert.attributes.actions.group', + aggs: { + aggName: { + max: { field: 'alert.attributes.actions.group' }, + }, + }, + }, + }, + }, + mockMappings + ) + ).toEqual({ + aggName: { + cardinality: { + field: 'alert.actions.group', + aggs: { + aggName: { + max: { + field: 'alert.actions.group', + }, + }, + }, + }, + }, + }); + }); + + test('Validate an aggregation without the attribute field', () => { + expect( + validateGetSavedObjectsAggs( + ['alert'], + { aggName: { terms: { 'alert.attributes.actions.group': ['myFriend', 'snoopy'] } } }, + mockMappings + ) + ).toEqual({ + aggName: { + terms: { + 'alert.actions.group': ['myFriend', 'snoopy'], + }, + }, + }); + }); + + test('Validate a filter term aggregations', () => { + expect( + validateGetSavedObjectsAggs( + ['foo'], + { aggName: { filter: { term: { 'foo.attributes.bytes': 10 } } } }, + mockMappings + ) + ).toEqual({ + aggName: { + filter: { + term: { 'foo.attributes.bytes': 10 }, + }, + }, + }); + }); + + test('Throw an error when types is not allowed', () => { + expect(() => { + validateGetSavedObjectsAggs( + ['alert'], + { + aggName: { + max: { field: 'foo.attributes.bytes' }, + }, + }, + mockMappings + ); + }).toThrowErrorMatchingInlineSnapshot(`"This type foo is not allowed: Bad Request"`); + }); + + test('Throw an error when add an invalid attributes ', () => { + expect(() => { + validateGetSavedObjectsAggs( + ['foo'], + { + aggName: { + max: { field: 'foo.attributes.bytes', notValid: 'yesIamNotValid' }, + }, + }, + mockMappings + ); + }).toThrowErrorMatchingInlineSnapshot( + `"notValid attribute is not supported in max saved objects aggregation: Bad Request"` + ); + }); + + test('Throw an error when an attributes is not defined correctly', () => { + expect(() => + validateGetSavedObjectsAggs( + ['alert'], + { + aggName: { + terms: { 'alert.attributes.actions.group': ['myFriend', 'snoopy'], missing: 0 }, + }, + }, + mockMappings + ) + ).toThrowErrorMatchingInlineSnapshot(`"Invalid value 0 supplied to : string: Bad Request"`); + }); + + test('Throw an error when aggregation is not defined in SavedObjectsAggs', () => { + expect(() => { + validateGetSavedObjectsAggs( + ['foo'], + { + aggName: { + MySuperAgg: { field: 'foo.attributes.bytes' }, + }, + }, + mockMappings + ); + }).toThrowErrorMatchingInlineSnapshot( + `"This aggregation MySuperAgg is not valid or we did not defined it yet: Bad Request"` + ); + }); + + test('Throw an error when children aggregation is not defined in SavedObjectsAggs', () => { + expect(() => { + validateGetSavedObjectsAggs( + ['foo'], + { + aggName: { + cardinality: { + field: 'foo.attributes.bytes', + aggs: { + aggName: { + MySuperAgg: { field: 'alert.attributes.actions.group' }, + }, + }, + }, + }, + }, + mockMappings + ); + }).toThrowErrorMatchingInlineSnapshot( + `"This aggregation MySuperAgg is not valid or we did not defined it yet: Bad Request"` + ); + }); + + test('Throw an error when you add the script attribute who are not defined in SavedObjectsAggs', () => { + expect(() => { + validateGetSavedObjectsAggs( + ['alert'], + { + aggName: { + cardinality: { field: 'alert.attributes.actions.group' }, + script: 'I want to access that I should not', + }, + }, + mockMappings + ); + }).toThrowErrorMatchingInlineSnapshot( + `"script attribute is not supported in saved objects aggregation: Bad Request"` + ); + }); + + test('Throw an error when you add the script attribute in a nested aggregations who are not defined in SavedObjectsAggs', () => { + expect(() => { + validateGetSavedObjectsAggs( + ['alert'], + { + aggName: { + cardinality: { + field: 'alert.attributes.actions.group', + aggs: { + aggName: { + max: { + field: 'alert.attributes.actions.group', + script: 'I want to access that I should not', + }, + }, + }, + }, + }, + }, + mockMappings + ); + }).toThrowErrorMatchingInlineSnapshot( + `"script attribute is not supported in saved objects aggregation: Bad Request"` + ); + }); + }); +}); diff --git a/src/core/server/saved_objects/service/lib/aggs_utils.ts b/src/core/server/saved_objects/service/lib/aggs_utils.ts new file mode 100644 index 0000000000000..2642da56f6b82 --- /dev/null +++ b/src/core/server/saved_objects/service/lib/aggs_utils.ts @@ -0,0 +1,102 @@ +/* + * Licensed to Elasticsearch B.V. under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch B.V. licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { IndexMapping } from '../../mappings'; + +import { SavedObjectsErrorHelpers } from './errors'; +import { hasFilterKeyError } from './filter_utils'; +import { savedObjectsAggs, validateSavedObjectsTypeAggs } from './saved_objects_aggs_types'; + +export const validateGetSavedObjectsAggs = ( + allowedTypes: string[], + aggs: Record, + indexMapping: IndexMapping +) => { + return validateGetAggFieldValue(allowedTypes, aggs, indexMapping); +}; + +const validateGetAggFieldValue = ( + allowedTypes: string[], + aggs: any, + indexMapping: IndexMapping, + lastKey?: string, + aggType?: string +): unknown => { + return Object.keys(aggs).reduce((acc, key) => { + if (key === 'script') { + throw SavedObjectsErrorHelpers.createBadRequestError( + 'script attribute is not supported in saved objects aggregation' + ); + } + if (typeof aggs[key] === 'object' && aggType === undefined && savedObjectsAggs[key]) { + return { + ...acc, + [key]: validateGetAggFieldValue(allowedTypes, aggs[key], indexMapping, key, key), + }; + } else if ( + typeof aggs[key] === 'object' && + (['aggs', 'aggregations'].includes(key) || aggType === undefined) + ) { + return { + ...acc, + [key]: validateGetAggFieldValue(allowedTypes, aggs[key], indexMapping, key, undefined), + }; + } else if ( + key !== 'field' && + aggType && + savedObjectsAggs[aggType] !== undefined && + savedObjectsAggs[aggType][key] !== undefined + ) { + validateSavedObjectsTypeAggs(savedObjectsAggs[aggType][key], aggs[key]); + return { + ...acc, + [key]: aggs[key], + }; + } else { + if (aggType === undefined || savedObjectsAggs[aggType] === undefined) { + throw SavedObjectsErrorHelpers.createBadRequestError( + `This aggregation ${lastKey} is not valid or we did not defined it yet` + ); + } + const error = hasFilterKeyError( + key === 'field' ? aggs[key] : key, + allowedTypes, + indexMapping + ); + if (error != null) { + if ( + aggType !== undefined && + savedObjectsAggs[aggType] !== undefined && + savedObjectsAggs[aggType][key] === undefined + ) { + throw SavedObjectsErrorHelpers.createBadRequestError( + `${key} attribute is not supported in ${aggType} saved objects aggregation` + ); + } + throw SavedObjectsErrorHelpers.createBadRequestError(error); + } + return { + ...acc, + ...(key === 'field' + ? { [key]: aggs[key].replace('.attributes', '') } + : { [key.replace('.attributes', '')]: aggs[key] }), + }; + } + }, {}); +}; diff --git a/src/core/server/saved_objects/service/lib/filter_utils.test.ts b/src/core/server/saved_objects/service/lib/filter_utils.test.ts index 0608035ce51a2..df7cbda3210f6 100644 --- a/src/core/server/saved_objects/service/lib/filter_utils.test.ts +++ b/src/core/server/saved_objects/service/lib/filter_utils.test.ts @@ -21,9 +21,9 @@ import { esKuery } from '../../es_query'; import { validateFilterKueryNode, validateConvertFilterToKueryNode } from './filter_utils'; -const mockMappings = { +export const mockMappings = { properties: { - updatedAt: { + updated_at: { type: 'date', }, foo: { @@ -104,12 +104,12 @@ describe('Filter Utils', () => { expect( validateConvertFilterToKueryNode( ['foo'], - 'foo.updatedAt: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)', + 'foo.updated_at: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)', mockMappings ) ).toEqual( esKuery.fromKueryExpression( - '(type: foo and updatedAt: 5678654567) and foo.bytes > 1000 and foo.bytes < 8000 and foo.title: "best" and (foo.description: t* or foo.description :*)' + '(type: foo and updated_at: 5678654567) and foo.bytes > 1000 and foo.bytes < 8000 and foo.title: "best" and (foo.description: t* or foo.description :*)' ) ); }); @@ -118,12 +118,12 @@ describe('Filter Utils', () => { expect( validateConvertFilterToKueryNode( ['foo', 'bar'], - 'foo.updatedAt: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)', + 'foo.updated_at: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)', mockMappings ) ).toEqual( esKuery.fromKueryExpression( - '(type: foo and updatedAt: 5678654567) and foo.bytes > 1000 and foo.bytes < 8000 and foo.title: "best" and (foo.description: t* or foo.description :*)' + '(type: foo and updated_at: 5678654567) and foo.bytes > 1000 and foo.bytes < 8000 and foo.title: "best" and (foo.description: t* or foo.description :*)' ) ); }); @@ -132,12 +132,12 @@ describe('Filter Utils', () => { expect( validateConvertFilterToKueryNode( ['foo', 'bar'], - '(bar.updatedAt: 5678654567 OR foo.updatedAt: 5678654567) and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or bar.attributes.description :*)', + '(bar.updated_at: 5678654567 OR foo.updated_at: 5678654567) and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or bar.attributes.description :*)', mockMappings ) ).toEqual( esKuery.fromKueryExpression( - '((type: bar and updatedAt: 5678654567) or (type: foo and updatedAt: 5678654567)) and foo.bytes > 1000 and foo.bytes < 8000 and foo.title: "best" and (foo.description: t* or bar.description :*)' + '((type: bar and updated_at: 5678654567) or (type: foo and updated_at: 5678654567)) and foo.bytes > 1000 and foo.bytes < 8000 and foo.title: "best" and (foo.description: t* or bar.description :*)' ) ); }); @@ -156,11 +156,11 @@ describe('Filter Utils', () => { expect(() => { validateConvertFilterToKueryNode( ['foo', 'bar'], - 'updatedAt: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)', + 'updated_at: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)', mockMappings ); }).toThrowErrorMatchingInlineSnapshot( - `"This key 'updatedAt' need to be wrapped by a saved object type like foo,bar: Bad Request"` + `"This key 'updated_at' need to be wrapped by a saved object type like foo,bar: Bad Request"` ); }); @@ -175,7 +175,7 @@ describe('Filter Utils', () => { test('Validate filter query through KueryNode - happy path', () => { const validationObject = validateFilterKueryNode({ astFilter: esKuery.fromKueryExpression( - 'foo.updatedAt: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)' + 'foo.updated_at: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)' ), types: ['foo'], indexMapping: mockMappings, @@ -186,7 +186,7 @@ describe('Filter Utils', () => { astPath: 'arguments.0', error: null, isSavedObjectAttr: true, - key: 'foo.updatedAt', + key: 'foo.updated_at', type: 'foo', }, { @@ -250,7 +250,7 @@ describe('Filter Utils', () => { test('Return Error if key is not wrapper by a saved object type', () => { const validationObject = validateFilterKueryNode({ astFilter: esKuery.fromKueryExpression( - 'updatedAt: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)' + 'updated_at: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)' ), types: ['foo'], indexMapping: mockMappings, @@ -259,9 +259,9 @@ describe('Filter Utils', () => { expect(validationObject).toEqual([ { astPath: 'arguments.0', - error: "This key 'updatedAt' need to be wrapped by a saved object type like foo", + error: "This key 'updated_at' need to be wrapped by a saved object type like foo", isSavedObjectAttr: true, - key: 'updatedAt', + key: 'updated_at', type: null, }, { @@ -305,7 +305,7 @@ describe('Filter Utils', () => { test('Return Error if key of a saved object type is not wrapped with attributes', () => { const validationObject = validateFilterKueryNode({ astFilter: esKuery.fromKueryExpression( - 'foo.updatedAt: 5678654567 and foo.attributes.bytes > 1000 and foo.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.description :*)' + 'foo.updated_at: 5678654567 and foo.attributes.bytes > 1000 and foo.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.description :*)' ), types: ['foo'], indexMapping: mockMappings, @@ -316,7 +316,7 @@ describe('Filter Utils', () => { astPath: 'arguments.0', error: null, isSavedObjectAttr: true, - key: 'foo.updatedAt', + key: 'foo.updated_at', type: 'foo', }, { @@ -362,7 +362,7 @@ describe('Filter Utils', () => { test('Return Error if filter is not using an allowed type', () => { const validationObject = validateFilterKueryNode({ astFilter: esKuery.fromKueryExpression( - 'bar.updatedAt: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)' + 'bar.updated_at: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)' ), types: ['foo'], indexMapping: mockMappings, @@ -373,7 +373,7 @@ describe('Filter Utils', () => { astPath: 'arguments.0', error: 'This type bar is not allowed', isSavedObjectAttr: true, - key: 'bar.updatedAt', + key: 'bar.updated_at', type: 'bar', }, { @@ -417,7 +417,7 @@ describe('Filter Utils', () => { test('Return Error if filter is using an non-existing key in the index patterns of the saved object type', () => { const validationObject = validateFilterKueryNode({ astFilter: esKuery.fromKueryExpression( - 'foo.updatedAt33: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.header: "best" and (foo.attributes.description: t* or foo.attributes.description :*)' + 'foo.updated_at33: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.header: "best" and (foo.attributes.description: t* or foo.attributes.description :*)' ), types: ['foo'], indexMapping: mockMappings, @@ -426,9 +426,9 @@ describe('Filter Utils', () => { expect(validationObject).toEqual([ { astPath: 'arguments.0', - error: "This key 'foo.updatedAt33' does NOT exist in foo saved object index patterns", + error: "This key 'foo.updated_at33' does NOT exist in foo saved object index patterns", isSavedObjectAttr: false, - key: 'foo.updatedAt33', + key: 'foo.updated_at33', type: 'foo', }, { @@ -494,5 +494,33 @@ describe('Filter Utils', () => { }, ]); }); + + test('Validate multiple items nested filter query through KueryNode', () => { + const validationObject = validateFilterKueryNode({ + astFilter: esKuery.fromKueryExpression( + 'alert.attributes.actions:{ actionTypeId: ".server-log" AND actionRef: "foo" }' + ), + types: ['alert'], + indexMapping: mockMappings, + }); + + // nodes will have errors in the array + expect(validationObject).toEqual([ + { + astPath: 'arguments.1.arguments.0', + error: null, + isSavedObjectAttr: false, + key: 'alert.attributes.actions.actionTypeId', + type: 'alert', + }, + { + astPath: 'arguments.1.arguments.1', + error: null, + isSavedObjectAttr: false, + key: 'alert.attributes.actions.actionRef', + type: 'alert', + }, + ]); + }); }); }); diff --git a/src/core/server/saved_objects/service/lib/filter_utils.ts b/src/core/server/saved_objects/service/lib/filter_utils.ts index be36807f0d02b..5c7e9dae70d7f 100644 --- a/src/core/server/saved_objects/service/lib/filter_utils.ts +++ b/src/core/server/saved_objects/service/lib/filter_utils.ts @@ -120,7 +120,15 @@ export const validateFilterKueryNode = ({ return astFilter.arguments.reduce((kueryNode: string[], ast: KueryNode, index: number) => { if (hasNestedKey && ast.type === 'literal' && ast.value != null) { localNestedKeys = ast.value; + } else if (ast.type === 'literal' && ast.value && typeof ast.value === 'string') { + const key = ast.value.replace('.attributes', ''); + const mappingKey = 'properties.' + key.split('.').join('.properties.'); + const field = get(indexMapping, mappingKey); + if (field != null && field.type === 'nested') { + localNestedKeys = ast.value; + } } + if (ast.arguments) { const myPath = `${path}.${index}`; return [ @@ -132,7 +140,7 @@ export const validateFilterKueryNode = ({ storeValue: ast.type === 'function' && astFunctionType.includes(ast.function), path: `${myPath}.arguments`, hasNestedKey: ast.type === 'function' && ast.function === 'nested', - nestedKeys: localNestedKeys, + nestedKeys: localNestedKeys || nestedKeys, }), ]; } diff --git a/src/core/server/saved_objects/service/lib/repository.ts b/src/core/server/saved_objects/service/lib/repository.ts index d362c02de4915..fde2f39ee287b 100644 --- a/src/core/server/saved_objects/service/lib/repository.ts +++ b/src/core/server/saved_objects/service/lib/repository.ts @@ -69,6 +69,7 @@ import { } from '../../types'; import { SavedObjectTypeRegistry } from '../../saved_objects_type_registry'; import { validateConvertFilterToKueryNode } from './filter_utils'; +import { validateGetSavedObjectsAggs } from './aggs_utils'; import { ALL_NAMESPACES_STRING, FIND_DEFAULT_PAGE, @@ -703,7 +704,9 @@ export class SavedObjectsRepository { * @property {string} [options.preference] * @returns {promise} - { saved_objects: [{ id, type, version, attributes }], total, per_page, page } */ - async find(options: SavedObjectsFindOptions): Promise> { + async find( + options: SavedObjectsFindOptions + ): Promise> { const { search, defaultSearchOperator = 'OR', @@ -721,6 +724,7 @@ export class SavedObjectsRepository { typeToNamespacesMap, filter, preference, + aggs, } = options; if (!type && !typeToNamespacesMap) { @@ -748,7 +752,7 @@ export class SavedObjectsRepository { : Array.from(typeToNamespacesMap!.keys()); const allowedTypes = types.filter((t) => this._allowedTypes.includes(t)); if (allowedTypes.length === 0) { - return SavedObjectsUtils.createEmptyFindResponse(options); + return SavedObjectsUtils.createEmptyFindResponse(options); } if (searchFields && !Array.isArray(searchFields)) { @@ -760,15 +764,23 @@ export class SavedObjectsRepository { } let kueryNode; - - try { - if (filter) { + if (filter) { + try { kueryNode = validateConvertFilterToKueryNode(allowedTypes, filter, this._mappings); + } catch (e) { + if (e.name === 'KQLSyntaxError') { + throw SavedObjectsErrorHelpers.createBadRequestError('KQLSyntaxError: ' + e.message); + } else { + throw e; + } } - } catch (e) { - if (e.name === 'KQLSyntaxError') { - throw SavedObjectsErrorHelpers.createBadRequestError('KQLSyntaxError: ' + e.message); - } else { + } + + let aggsObject = null; + if (aggs) { + try { + aggsObject = validateGetSavedObjectsAggs(allowedTypes, aggs, this._mappings); + } catch (e) { throw e; } } @@ -782,6 +794,7 @@ export class SavedObjectsRepository { preference, body: { seq_no_primary_term: true, + ...(aggsObject != null ? { aggs: aggsObject } : {}), ...getSearchDsl(this._mappings, this._registry, { search, defaultSearchOperator, @@ -814,6 +827,7 @@ export class SavedObjectsRepository { } return { + ...(body.aggregations != null ? { aggregations: body.aggregations } : {}), page, per_page: perPage, total: body.hits.total, @@ -823,7 +837,7 @@ export class SavedObjectsRepository { score: (hit as any)._score, }) ), - } as SavedObjectsFindResponse; + } as SavedObjectsFindResponse; } /** diff --git a/src/core/server/saved_objects/service/lib/saved_objects_aggs_types/bucket_aggs/index.ts b/src/core/server/saved_objects/service/lib/saved_objects_aggs_types/bucket_aggs/index.ts new file mode 100644 index 0000000000000..cb9c5febce6e3 --- /dev/null +++ b/src/core/server/saved_objects/service/lib/saved_objects_aggs_types/bucket_aggs/index.ts @@ -0,0 +1,49 @@ +/* + * Licensed to Elasticsearch B.V. under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch B.V. licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import * as rt from 'io-ts'; + +import { fieldBasic } from '../helpers'; + +export const bucketAggsType: Record> = { + filter: { + term: rt.record(rt.string, rt.any), + }, + histogram: { + ...fieldBasic, + interval: rt.number, + min_doc_count: rt.number, + extended_bounds: rt.type({ min: rt.number, max: rt.number }), + keyed: rt.boolean, + missing: rt.number, + order: rt.record(rt.string, rt.literal('asc', 'desc')), + }, + terms: { + ...fieldBasic, + collect_mode: rt.string, + exclude: rt.unknown, + execution_hint: rt.string, + include: rt.unknown, + missing: rt.string, + min_doc_count: rt.number, + size: rt.number, + show_term_doc_count_error: rt.boolean, + order: rt.record(rt.string, rt.literal('asc', 'desc')), + }, +}; diff --git a/src/core/server/saved_objects/service/lib/saved_objects_aggs_types/helpers.ts b/src/core/server/saved_objects/service/lib/saved_objects_aggs_types/helpers.ts new file mode 100644 index 0000000000000..6485639636ec3 --- /dev/null +++ b/src/core/server/saved_objects/service/lib/saved_objects_aggs_types/helpers.ts @@ -0,0 +1,30 @@ +/* + * Licensed to Elasticsearch B.V. under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch B.V. licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import * as rt from 'io-ts'; +import { failure } from 'io-ts/lib/PathReporter'; + +type ErrorFactory = (message: string) => Error; + +export const fieldBasic = { field: rt.string }; +export const FieldBasicRT = rt.type({ field: rt.string }); + +export const throwErrors = (createError: ErrorFactory) => (errors: rt.Errors) => { + throw createError(failure(errors).join('\n')); +}; diff --git a/src/core/server/saved_objects/service/lib/saved_objects_aggs_types/index.ts b/src/core/server/saved_objects/service/lib/saved_objects_aggs_types/index.ts new file mode 100644 index 0000000000000..a60b38ed4db45 --- /dev/null +++ b/src/core/server/saved_objects/service/lib/saved_objects_aggs_types/index.ts @@ -0,0 +1,42 @@ +/* + * Licensed to Elasticsearch B.V. under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch B.V. licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { fold } from 'fp-ts/lib/Either'; +import { identity } from 'fp-ts/lib/function'; +import { pipe } from 'fp-ts/lib/pipeable'; + +import * as rt from 'io-ts'; + +import { bucketAggsType } from './bucket_aggs'; +import { metricsAggsType } from './metrics_aggs'; + +import { SavedObjectsErrorHelpers } from '../errors'; +import { throwErrors } from './helpers'; + +export const savedObjectsAggs = { + ...metricsAggsType, + ...bucketAggsType, +}; + +export const validateSavedObjectsTypeAggs = (rtType: rt.Any, aggObject: unknown) => { + pipe( + rtType.decode(aggObject), + fold(throwErrors(SavedObjectsErrorHelpers.createBadRequestError), identity) + ); +}; diff --git a/src/core/server/saved_objects/service/lib/saved_objects_aggs_types/metrics_aggs/index.ts b/src/core/server/saved_objects/service/lib/saved_objects_aggs_types/metrics_aggs/index.ts new file mode 100644 index 0000000000000..7de8ed12dcc69 --- /dev/null +++ b/src/core/server/saved_objects/service/lib/saved_objects_aggs_types/metrics_aggs/index.ts @@ -0,0 +1,74 @@ +/* + * Licensed to Elasticsearch B.V. under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch B.V. licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import * as rt from 'io-ts'; + +import { fieldBasic, FieldBasicRT } from '../helpers'; + +/* + * Types for Metrics Aggregations + * + * TODO: + * - Extended Stats Aggregation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-extendedstats-aggregation.html + * - Geo Bounds Aggregation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-geobounds-aggregation.html + * - Geo Centroid Aggregation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-geocentroid-aggregation.html + * - Percentiles Aggregation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-percentile-aggregation.html + * - Percentile Ranks Aggregation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-percentile-rank-aggregation.html + * - Scripted Metric Aggregation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-scripted-metric-aggregation.html + * - Stats Aggregation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-stats-aggregation.html + * - String Stats Aggregation (x-pack) https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-string-stats-aggregation.html + * - Sum Aggregation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-sum-aggregation.html + * - Top Hits Aggregation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-top-hits-aggregation.html + * - Value Count Aggregation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-valuecount-aggregation.html + * - Median Absolute Deviation Aggregation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-median-absolute-deviation-aggregation.html + */ + +export const metricsAggsType: Record> = { + avg: fieldBasic, + weighted_avg: { + value: rt.intersection([FieldBasicRT, rt.partial({ missing: rt.number })]), + weight: rt.intersection([FieldBasicRT, rt.partial({ missing: rt.number })]), + format: rt.string, + value_type: rt.string, + }, + cardinality: fieldBasic, + max: { + ...fieldBasic, + missing: rt.number, + }, + min: { + ...fieldBasic, + missing: rt.number, + }, + top_hits: { + explain: rt.boolean, + from: rt.string, + highlight: rt.any, + seq_no_primary_term: rt.boolean, + size: rt.number, + sort: rt.any, + stored_fields: rt.array(rt.string), + version: rt.boolean, + _name: rt.string, + _source: rt.partial({ + includes: rt.array(rt.string), + excludes: rt.array(rt.string), + }), + }, +}; diff --git a/src/core/server/saved_objects/service/lib/utils.ts b/src/core/server/saved_objects/service/lib/utils.ts index 69abc37089218..e72a64cc247a6 100644 --- a/src/core/server/saved_objects/service/lib/utils.ts +++ b/src/core/server/saved_objects/service/lib/utils.ts @@ -60,10 +60,10 @@ export class SavedObjectsUtils { /** * Creates an empty response for a find operation. This is only intended to be used by saved objects client wrappers. */ - public static createEmptyFindResponse = ({ + public static createEmptyFindResponse = ({ page = FIND_DEFAULT_PAGE, perPage = FIND_DEFAULT_PER_PAGE, - }: SavedObjectsFindOptions): SavedObjectsFindResponse => ({ + }: SavedObjectsFindOptions): SavedObjectsFindResponse => ({ page, per_page: perPage, total: 0, diff --git a/src/core/server/saved_objects/service/saved_objects_client.ts b/src/core/server/saved_objects/service/saved_objects_client.ts index 6cb9823c736e0..601ac8f891158 100644 --- a/src/core/server/saved_objects/service/saved_objects_client.ts +++ b/src/core/server/saved_objects/service/saved_objects_client.ts @@ -130,7 +130,8 @@ export interface SavedObjectsFindResult extends SavedObject { * * @public */ -export interface SavedObjectsFindResponse { +export interface SavedObjectsFindResponse { + aggregations?: A; saved_objects: Array>; total: number; per_page: number; @@ -353,7 +354,9 @@ export class SavedObjectsClient { * * @param options */ - async find(options: SavedObjectsFindOptions): Promise> { + async find( + options: SavedObjectsFindOptions + ): Promise> { return await this._repository.find(options); } diff --git a/src/core/server/saved_objects/types.ts b/src/core/server/saved_objects/types.ts index b16eeb2aa03a6..7bb919b9198c0 100644 --- a/src/core/server/saved_objects/types.ts +++ b/src/core/server/saved_objects/types.ts @@ -109,6 +109,14 @@ export interface SavedObjectsFindOptions { */ defaultSearchOperator?: 'AND' | 'OR'; filter?: string | KueryNode; + /** + * Specify an Elasticsearch aggregation to perform. This alpha API only supports a limited set of aggregation types: metrics, bucket. Additional aggregation types can be contributed to Core. + * @alpha + * @example + * aggs = { latest_version: { max: { field: 'dashboard.attributes.version' } } }; + * SavedObjects.find({type: 'dashboard', aggs: '%7B%22latest_version%22%3A%7B%22max%22%3A%7B%22field%22%3A%22dashboard.attributes.version%22%7D%7D%7D'}) + */ + aggs?: Record; namespaces?: string[]; /** * This map defines each type to search for, and the namespace(s) to search for the type in; this is only intended to be used by a saved diff --git a/src/core/server/server.api.md b/src/core/server/server.api.md index a03e5ec9acd27..fbfe177c65a9d 100644 --- a/src/core/server/server.api.md +++ b/src/core/server/server.api.md @@ -1994,7 +1994,7 @@ export class SavedObjectsClient { static errors: typeof SavedObjectsErrorHelpers; // (undocumented) errors: typeof SavedObjectsErrorHelpers; - find(options: SavedObjectsFindOptions): Promise>; + find(options: SavedObjectsFindOptions): Promise>; get(type: string, id: string, options?: SavedObjectsBaseOptions): Promise>; removeReferencesTo(type: string, id: string, options?: SavedObjectsRemoveReferencesToOptions): Promise; update(type: string, id: string, attributes: Partial, options?: SavedObjectsUpdateOptions): Promise>; @@ -2188,6 +2188,8 @@ export type SavedObjectsFieldMapping = SavedObjectsCoreFieldMapping | SavedObjec // @public (undocumented) export interface SavedObjectsFindOptions { + // @alpha + aggs?: Record; defaultSearchOperator?: 'AND' | 'OR'; fields?: string[]; // Warning: (ae-forgotten-export) The symbol "KueryNode" needs to be exported by the entry point index.d.ts @@ -2224,7 +2226,9 @@ export interface SavedObjectsFindOptionsReference { } // @public -export interface SavedObjectsFindResponse { +export interface SavedObjectsFindResponse { + // (undocumented) + aggregations?: A; // (undocumented) page: number; // (undocumented) @@ -2442,7 +2446,7 @@ export class SavedObjectsRepository { deleteByNamespace(namespace: string, options?: SavedObjectsDeleteByNamespaceOptions): Promise; deleteFromNamespaces(type: string, id: string, namespaces: string[], options?: SavedObjectsDeleteFromNamespacesOptions): Promise; // (undocumented) - find(options: SavedObjectsFindOptions): Promise>; + find(options: SavedObjectsFindOptions): Promise>; get(type: string, id: string, options?: SavedObjectsBaseOptions): Promise>; incrementCounter(type: string, id: string, counterFieldName: string, options?: SavedObjectsIncrementCounterOptions): Promise; removeReferencesTo(type: string, id: string, options?: SavedObjectsRemoveReferencesToOptions): Promise; @@ -2551,7 +2555,7 @@ export interface SavedObjectsUpdateResponse extends Omit({ page, perPage, }: SavedObjectsFindOptions) => SavedObjectsFindResponse; + static createEmptyFindResponse: ({ page, perPage, }: SavedObjectsFindOptions) => SavedObjectsFindResponse; static namespaceIdToString: (namespace?: string | undefined) => string; static namespaceStringToId: (namespace: string) => string | undefined; } diff --git a/test/api_integration/apis/saved_objects/find.js b/test/api_integration/apis/saved_objects/find.js index 8e8730b1e574a..7e0ca6d84e33d 100644 --- a/test/api_integration/apis/saved_objects/find.js +++ b/test/api_integration/apis/saved_objects/find.js @@ -295,6 +295,76 @@ export default function ({ getService }) { })); }); + describe('with a aggs', () => { + it('should return 200 with a valid response', async () => + await supertest + .get( + `/api/saved_objects/_find?type=visualization&per_page=0&aggs=${encodeURIComponent( + JSON.stringify({ + type_count: { max: { field: 'visualization.attributes.version' } }, + }) + )}` + ) + .expect(200) + .then((resp) => { + expect(resp.body).to.eql({ + aggregations: { + type_count: { + value: 1, + }, + }, + page: 1, + per_page: 0, + saved_objects: [], + total: 1, + }); + })); + + it('wrong type should return 400 with Bad Request', async () => + await supertest + .get( + `/api/saved_objects/_find?type=visualization&per_page=0&aggs=${encodeURIComponent( + JSON.stringify({ + type_count: { max: { field: 'dashboard.attributes.version' } }, + }) + )}` + ) + .expect(400) + .then((resp) => { + console.log('body', JSON.stringify(resp.body)); + expect(resp.body).to.eql({ + error: 'Bad Request', + message: 'This type dashboard is not allowed: Bad Request', + statusCode: 400, + }); + })); + + it('adding a wrong attributes should return 400 with Bad Request', async () => + await supertest + .get( + `/api/saved_objects/_find?type=visualization&per_page=0&aggs=${encodeURIComponent( + JSON.stringify({ + type_count: { + max: { + field: 'visualization.attributes.version', + script: 'Oh yes I am going to a script', + }, + }, + }) + )}` + ) + .expect(400) + .then((resp) => { + console.log('body', JSON.stringify(resp.body)); + expect(resp.body).to.eql({ + error: 'Bad Request', + message: + 'script attribute is not supported in saved objects aggregation: Bad Request', + statusCode: 400, + }); + })); + }); + describe('`has_reference` and `has_reference_operator` parameters', () => { before(() => esArchiver.load('saved_objects/references')); after(() => esArchiver.unload('saved_objects/references')); diff --git a/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.ts b/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.ts index ddef9f477433c..c936ad0a195d4 100644 --- a/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.ts +++ b/x-pack/plugins/encrypted_saved_objects/server/saved_objects/encrypted_saved_objects_client_wrapper.ts @@ -188,9 +188,9 @@ export class EncryptedSavedObjectsClientWrapper implements SavedObjectsClientCon return await this.options.baseClient.delete(type, id, options); } - public async find(options: SavedObjectsFindOptions) { + public async find(options: SavedObjectsFindOptions) { return await this.handleEncryptedAttributesInBulkResponse( - await this.options.baseClient.find(options), + await this.options.baseClient.find(options), undefined ); } diff --git a/x-pack/plugins/security/server/saved_objects/secure_saved_objects_client_wrapper.ts b/x-pack/plugins/security/server/saved_objects/secure_saved_objects_client_wrapper.ts index e6e34de4ac9ab..3a9339a54e316 100644 --- a/x-pack/plugins/security/server/saved_objects/secure_saved_objects_client_wrapper.ts +++ b/x-pack/plugins/security/server/saved_objects/secure_saved_objects_client_wrapper.ts @@ -201,7 +201,7 @@ export class SecureSavedObjectsClientWrapper implements SavedObjectsClientContra return await this.baseClient.delete(type, id, options); } - public async find(options: SavedObjectsFindOptions) { + public async find(options: SavedObjectsFindOptions) { if ( this.getSpacesService() == null && Array.isArray(options.namespaces) && @@ -228,7 +228,7 @@ export class SecureSavedObjectsClientWrapper implements SavedObjectsClientContra error: new Error(status), }) ); - return SavedObjectsUtils.createEmptyFindResponse(options); + return SavedObjectsUtils.createEmptyFindResponse(options); } const typeToNamespacesMap = Array.from(typeMap).reduce>( @@ -237,7 +237,7 @@ export class SecureSavedObjectsClientWrapper implements SavedObjectsClientContra new Map() ); - const response = await this.baseClient.find({ + const response = await this.baseClient.find({ ...options, typeToNamespacesMap: undefined, // if the user is fully authorized, use `undefined` as the typeToNamespacesMap to prevent privilege escalation ...(status === 'partially_authorized' && { typeToNamespacesMap, type: '', namespaces: [] }), // the repository requires that `type` and `namespaces` must be empty if `typeToNamespacesMap` is defined diff --git a/x-pack/plugins/spaces/server/saved_objects/spaces_saved_objects_client.ts b/x-pack/plugins/spaces/server/saved_objects/spaces_saved_objects_client.ts index 049bd88085ed5..07861399009f6 100644 --- a/x-pack/plugins/spaces/server/saved_objects/spaces_saved_objects_client.ts +++ b/x-pack/plugins/spaces/server/saved_objects/spaces_saved_objects_client.ts @@ -164,7 +164,7 @@ export class SpacesSavedObjectsClient implements SavedObjectsClientContract { * @property {object} [options.hasReference] - { type, id } * @returns {promise} - { saved_objects: [{ id, type, version, attributes }], total, per_page, page } */ - public async find(options: SavedObjectsFindOptions) { + public async find(options: SavedObjectsFindOptions) { throwErrorIfNamespaceSpecified(options); let namespaces = options.namespaces; @@ -180,12 +180,12 @@ export class SpacesSavedObjectsClient implements SavedObjectsClientContract { } if (namespaces.length === 0) { // return empty response, since the user is unauthorized in this space (or these spaces), but we don't return forbidden errors for `find` operations - return SavedObjectsUtils.createEmptyFindResponse(options); + return SavedObjectsUtils.createEmptyFindResponse(options); } } catch (err) { if (Boom.isBoom(err) && err.output.payload.statusCode === 403) { // return empty response, since the user is unauthorized in any space, but we don't return forbidden errors for `find` operations - return SavedObjectsUtils.createEmptyFindResponse(options); + return SavedObjectsUtils.createEmptyFindResponse(options); } throw err; } @@ -193,7 +193,7 @@ export class SpacesSavedObjectsClient implements SavedObjectsClientContract { namespaces = [this.spaceId]; } - return await this.client.find({ + return await this.client.find({ ...options, type: (options.type ? coerceToArray(options.type) : this.types).filter( (type) => type !== 'space' diff --git a/x-pack/test/saved_object_api_integration/common/fixtures/es_archiver/saved_objects/spaces/data.json b/x-pack/test/saved_object_api_integration/common/fixtures/es_archiver/saved_objects/spaces/data.json index d9d5c6f9c5808..c5f2f5cbb77a4 100644 --- a/x-pack/test/saved_object_api_integration/common/fixtures/es_archiver/saved_objects/spaces/data.json +++ b/x-pack/test/saved_object_api_integration/common/fixtures/es_archiver/saved_objects/spaces/data.json @@ -338,7 +338,8 @@ "index": ".kibana", "source": { "globaltype": { - "title": "My favorite global object" + "title": "My favorite global object", + "version": 1 }, "type": "globaltype", "updated_at": "2017-09-21T18:59:16.270Z" diff --git a/x-pack/test/saved_object_api_integration/common/fixtures/es_archiver/saved_objects/spaces/mappings.json b/x-pack/test/saved_object_api_integration/common/fixtures/es_archiver/saved_objects/spaces/mappings.json index 73f0e536b9295..04fed1905ecc5 100644 --- a/x-pack/test/saved_object_api_integration/common/fixtures/es_archiver/saved_objects/spaces/mappings.json +++ b/x-pack/test/saved_object_api_integration/common/fixtures/es_archiver/saved_objects/spaces/mappings.json @@ -90,6 +90,9 @@ } }, "type": "text" + }, + "version": { + "type": "integer" } } }, diff --git a/x-pack/test/saved_object_api_integration/common/fixtures/saved_object_test_plugin/server/plugin.ts b/x-pack/test/saved_object_api_integration/common/fixtures/saved_object_test_plugin/server/plugin.ts index 45880635586a7..6a1e72bfcf0db 100644 --- a/x-pack/test/saved_object_api_integration/common/fixtures/saved_object_test_plugin/server/plugin.ts +++ b/x-pack/test/saved_object_api_integration/common/fixtures/saved_object_test_plugin/server/plugin.ts @@ -56,7 +56,14 @@ export class Plugin { hidden: false, namespaceType: 'agnostic', management, - mappings, + mappings: { + properties: { + ...mappings.properties, + version: { + type: 'integer', + }, + }, + }, }); core.savedObjects.registerType({ name: 'hiddentype', diff --git a/x-pack/test/saved_object_api_integration/common/suites/find.ts b/x-pack/test/saved_object_api_integration/common/suites/find.ts index c7243d8db25fe..918d3c9c7faf3 100644 --- a/x-pack/test/saved_object_api_integration/common/suites/find.ts +++ b/x-pack/test/saved_object_api_integration/common/suites/find.ts @@ -35,9 +35,10 @@ export interface FindTestCase { total?: number; }; failure?: { - statusCode: 200 | 400; // if the user searches for types and/or namespaces they are not authorized for, they will get a 200 result with those types/namespaces omitted + statusCode: 200 | 400 | 403; // if the user searches for types and/or namespaces they are not authorized for, they will get a 200 result with those types/namespaces omitted reason: 'unauthorized' | 'cross_namespace_not_permitted' | 'bad_request'; }; + typeUseInQueryField?: string; } const TEST_CASES = [ @@ -50,7 +51,7 @@ export const getTestCases = ( currentSpace: undefined, crossSpaceSearch: undefined, } -) => { +): Record => { const crossSpaceIds = crossSpaceSearch?.filter((s) => s !== (currentSpace ?? DEFAULT_SPACE_ID)) ?? []; // intentionally exclude the current space const isCrossSpaceSearch = crossSpaceIds.length > 0; @@ -97,7 +98,7 @@ export const getTestCases = ( successResult: { savedObjects: getExpectedSavedObjects((t) => t.type === 'isolatedtype'), }, - } as FindTestCase, + }, multiNamespaceType: { title: buildTitle('find multi-namespace type'), query: `type=sharedtype&fields=title${namespacesQueryParam}`, @@ -105,20 +106,20 @@ export const getTestCases = ( // expected depends on which spaces the user is authorized against... savedObjects: getExpectedSavedObjects((t) => t.type === 'sharedtype'), }, - } as FindTestCase, + }, namespaceAgnosticType: { title: buildTitle('find namespace-agnostic type'), query: `type=globaltype&fields=title${namespacesQueryParam}`, successResult: { savedObjects: SAVED_OBJECT_TEST_CASES.NAMESPACE_AGNOSTIC }, - } as FindTestCase, + }, hiddenType: { title: buildTitle('find hidden type'), query: `type=hiddentype&fields=name${namespacesQueryParam}`, - } as FindTestCase, + }, unknownType: { title: buildTitle('find unknown type'), query: `type=wigwags${namespacesQueryParam}`, - } as FindTestCase, + }, eachType: { title: buildTitle('find each type'), query: `type=isolatedtype&type=sharedtype&type=globaltype&type=hiddentype&type=wigwags${namespacesQueryParam}`, @@ -127,7 +128,7 @@ export const getTestCases = ( ['isolatedtype', 'sharedtype', 'globaltype'].includes(t.type) ), }, - } as FindTestCase, + }, pageBeyondTotal: { title: buildTitle('find page beyond total'), query: `type=isolatedtype&page=100&per_page=100${namespacesQueryParam}`, @@ -137,24 +138,27 @@ export const getTestCases = ( total: -1, savedObjects: [], }, - } as FindTestCase, + }, unknownSearchField: { title: buildTitle('find unknown search field'), query: `type=url&search_fields=a${namespacesQueryParam}`, - } as FindTestCase, + }, filterWithNamespaceAgnosticType: { title: buildTitle('filter with namespace-agnostic type'), query: `type=globaltype&filter=globaltype.attributes.title:*global*${namespacesQueryParam}`, successResult: { savedObjects: SAVED_OBJECT_TEST_CASES.NAMESPACE_AGNOSTIC }, - } as FindTestCase, + typeUseInQueryField: 'globaltype', + }, filterWithHiddenType: { title: buildTitle('filter with hidden type'), query: `type=hiddentype&fields=name&filter=hiddentype.attributes.title:'hello'${namespacesQueryParam}`, - } as FindTestCase, + typeUseInQueryField: 'hiddentype', + }, filterWithUnknownType: { title: buildTitle('filter with unknown type'), query: `type=wigwags&filter=wigwags.attributes.title:'unknown'${namespacesQueryParam}`, - } as FindTestCase, + typeUseInQueryField: 'wigwags', + }, filterWithDisallowedType: { title: buildTitle('filter with disallowed type'), query: `type=globaltype&filter=dashboard.title:'Requests'${namespacesQueryParam}`, @@ -162,7 +166,49 @@ export const getTestCases = ( statusCode: 400, reason: 'bad_request', }, - } as FindTestCase, + typeUseInQueryField: 'dashboard', + }, + aggsWithNamespaceAgnosticType: { + title: buildTitle('aggs with namespace-agnostic type'), + query: `type=globaltype&aggs=${encodeURIComponent( + JSON.stringify({ + type_count: { max: { field: 'globaltype.attributes.version' } }, + }) + )}${namespacesQueryParam}`, + successResult: { savedObjects: SAVED_OBJECT_TEST_CASES.NAMESPACE_AGNOSTIC }, + typeUseInQueryField: 'globaltype', + }, + aggsWithHiddenType: { + title: buildTitle('aggs with hidden type'), + query: `type=hiddentype&fields=name&aggs=${encodeURIComponent( + JSON.stringify({ + type_count: { max: { field: 'hiddentype.attributes.title' } }, + }) + )}${namespacesQueryParam}`, + typeUseInQueryField: 'hiddentype', + }, + aggsWithUnknownType: { + title: buildTitle('aggs with unknown type'), + query: `type=wigwags&aggs=${encodeURIComponent( + JSON.stringify({ + type_count: { max: { field: 'wigwags.attributes.version' } }, + }) + )}${namespacesQueryParam}`, + typeUseInQueryField: 'wigwags', + }, + aggsWithDisallowedType: { + title: buildTitle('aggs with disallowed type'), + query: `type=globaltype&aggs=${encodeURIComponent( + JSON.stringify({ + type_count: { max: { field: 'dashboard.attributes.version' } }, + }) + )}${namespacesQueryParam}`, + failure: { + statusCode: 400, + reason: 'bad_request', + }, + typeUseInQueryField: 'dashboard', + }, }; }; @@ -196,7 +242,7 @@ export function findTestSuiteFactory(esArchiver: any, supertest: SuperTest) } } else if (failure?.statusCode === 400) { if (failure.reason === 'bad_request') { - const type = (parsedQuery.filter as string).split('.')[0]; + const type = testCase.typeUseInQueryField ?? 'unknown type'; expect(response.body.error).to.eql('Bad Request'); expect(response.body.statusCode).to.eql(failure.statusCode); expect(response.body.message).to.eql(`This type ${type} is not allowed: Bad Request`); diff --git a/x-pack/test/saved_object_api_integration/security_and_spaces/apis/find.ts b/x-pack/test/saved_object_api_integration/security_and_spaces/apis/find.ts index afd4783fab792..bfbd7981d88ec 100644 --- a/x-pack/test/saved_object_api_integration/security_and_spaces/apis/find.ts +++ b/x-pack/test/saved_object_api_integration/security_and_spaces/apis/find.ts @@ -32,12 +32,16 @@ const createTestCases = (currentSpace: string, crossSpaceSearch?: string[]) => { cases.unknownSearchField, cases.filterWithNamespaceAgnosticType, cases.filterWithDisallowedType, + cases.aggsWithNamespaceAgnosticType, + cases.aggsWithDisallowedType, ]; const hiddenAndUnknownTypes = [ cases.hiddenType, cases.unknownType, cases.filterWithHiddenType, cases.filterWithUnknownType, + cases.aggsWithHiddenType, + cases.aggsWithUnknownType, ]; const allTypes = normalTypes.concat(hiddenAndUnknownTypes); return { normalTypes, hiddenAndUnknownTypes, allTypes }; diff --git a/x-pack/test/saved_object_api_integration/security_only/apis/find.ts b/x-pack/test/saved_object_api_integration/security_only/apis/find.ts index aa18f32600949..13cddd868a519 100644 --- a/x-pack/test/saved_object_api_integration/security_only/apis/find.ts +++ b/x-pack/test/saved_object_api_integration/security_only/apis/find.ts @@ -29,12 +29,16 @@ const createTestCases = (crossSpaceSearch?: string[]) => { cases.unknownSearchField, cases.filterWithNamespaceAgnosticType, cases.filterWithDisallowedType, + cases.aggsWithNamespaceAgnosticType, + cases.aggsWithDisallowedType, ]; const hiddenAndUnknownTypes = [ cases.hiddenType, cases.unknownType, cases.filterWithHiddenType, cases.filterWithUnknownType, + cases.aggsWithHiddenType, + cases.aggsWithUnknownType, ]; const allTypes = normalTypes.concat(hiddenAndUnknownTypes); return { normalTypes, hiddenAndUnknownTypes, allTypes };