ES client : use the new type definitions #83808

delvedor · 2020-11-19T15:09:55Z

Summary

The JavaScript client will offer a comprehensive type definition of the Elasticsearch API.
PR in the client library: elastic/elasticsearch-js#1358

Notes for reviewers

If your code still uses the legacy Elasticsearch client provided by the Kibana Core, please migrate to the new Elasticsearch client using this instruction.
If your code already leverages the new Elasticsearch client, please, in a follow-up PR, remove all type definitions imported from the legacy 'elasticseachclient library and manually maintained type definitions. Instead, use typings provided from@elastic/elasticsearch` client:

- import { SearchResponse } from 'elasticsearch';
+ import type { estypes } from '@elastic/elasticsearch';

Also, please review all the @elastic/elasticsearch library type generics used in your code. Most of them either unnecessary or should be refactored:

- const { body } = await client.search<MySearchResponse>(...)
+ const { body } = await client.search(...)

We are at the very beginning of supporting type definitions in the ES client library. Keep in mind, during the code review process, that provided types aren't complete and might contain some mistakes.
If you spot any problems in the imported typings, please mute them with @ts-expect-error @elastic/elasticsearch instruction and report them to https://github.com/elastic/elastic-client-generator/issues/new/choose
Once the type errors are fixed in the upstream library, Kibana will receive the update, and @ts-expect-error @elastic/elasticsearch should be deleted.

If you see @ts-expect-error in this PR (instead of @ts-expect-error @elastic/elasticsearch), it's likely to be a problem in the Kibana code that should be addressed by code owners later. If you are sure, it's a problem in the Client code, please, report the problem to https://github.com/elastic/elastic-client-generator/issues/new/choose

List of known problems

Moved to elastic/elasticsearch-specification#197

Plugin API changes

In the past releases, Kibana Platform provided an Elasticsearch client that didn't have any response typings out of the box.
From now on, the Elasticsearch client is provided with enhanced response type definitions. If your code already leverages the new Elasticsearch client, remove all type definitions imported from the legacy elasticseach client library and all the manually maintained type definitions. Instead, use typings provided from the @elastic/elasticsearch client:

- import { SearchResponse } from 'elasticsearch';
+ import type { estypes } from '@elastic/elasticsearch';

Also, please review all the @elastic/elasticsearch library type generics used in your code. Most of them either unnecessary or should be refactored:

- const { body } = await client.search<MySearchResponse>(...)
+ const { body } = await client.search(...)

joshdover · 2020-12-03T15:05:42Z

@elasticmachine merge upstream

kibanamachine · 2020-12-03T15:05:44Z

merge conflict between base and head

mshustov · 2020-12-07T10:12:48Z

List of incompatibilities

all the properties in the response body are optional. It affects the whole codebase and creates a necessity for the normalization layer for every endpoint.
Let's consider the search request. According to the typings in @elastic/elasticsearch, all the properties in SearchResponse are optional.
But I don't see they are described as optional in the docs https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-api-response-body
I haven't checked all the tests in https://github.com/elastic/elasticsearch/tree/498e264df4e8f3c833dd9e3531a1ec5acd305ebd/rest-api-spec/src/main/resources/rest-api-spec/test/search but I don't see any cases when the hits field is absent from a successful response either.
UPD:

// when `filter_path` is specified, ES doesn't return empty arrays, so if
// there are no search results res.body.hits will be undefined.

from https://github.com/elastic/kibana/pull/78413/files#diff-dface0d15abee0d1b6332a9651bd8999d17d724718b8f583a6a348906ef3e4d8R674-R675 So the example with hits wasn't the best.

SearchResponse property hit: Hit<TDocument> interface doesn't provide found document details (and doesn't use passed TDocument type) - should we switch to documents property instead? I don't see documents description in https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html#search-api-response-body
no response type for scroll
mget response type hits?: MultiGetHit<object>[] is not compatible with the legacy structure described in the official documentation docs: Documents[] https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-multi-get.html#multi-get-api-response-body. Also note that MultiGetHit property names don't match ones under docs property as they don't start with an underscore.
cat.indices doesn't support h request param https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-indices.html#cat-indices-api-query-params
indices.stats doesn't support string[] in metric property https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/16.x/api-indices-stats.html
mappings._meta.managed_by is object, but used to be a string
NodeUsageInformation doesn't contain types for aggregations
IProperty doesn't contain properties fields
GetLicenseResponse doesn't contain start_date_in_millis field
GetAliasResponse doesn't contain aliases field
AsyncSearchSubmit.docvalue_fields: string | string[] but SearchRequest.docvalue_fields: string[]
SearchRequest doesn't contain timeout field
XPackRole doesn't contain applications, name fields
IndicesPrivileges.query doesn't support string
AnalysisConfig doesn't contain per_partition_categorization field
Limits doesn't contain effective_max_model_memory_limit field
some types use an empty Date interface, which should be number
Aggregate doesn't contain value, value_as_string fields

Side notes

items with type 'long' defined as strings in the new typings. although, api.types.d.ts contains type long = number. That creates inconstancy. Maybe string is correct as long numbers in java are 64-bit integer (from -2^63 to 2^63-1) but in js safe integer is 2^53 - 1. We will have to adopt the code until BigInt is supported.
don't export named constant types. Export all the types instead. Otherwise, it shadows the local variables:

import type { types } from '@elastic/elasticsearch';
const types = ... // error!

joshdover · 2020-12-15T23:34:13Z

@delvedor Not sure if you saw the feedback above (or I missed a conversation somewhere). How do we anticipate fixing these issues?

delvedor · 2020-12-16T11:08:08Z

@joshdover yes, I did. Sorry for not writing before.
We do expect some errors in the definitions, and those errors will not be fixed all at the same time before releasing the types. The plan is to release what we have and fix them as soon as we start gathering feedback from the team.
Regarding the response types being optional, that's definitely a problem, which will be addressed very soon.

joshdover · 2020-12-28T21:22:06Z

I've done another pass and pushed my progress to this branch. There are several issues still, but I agree that we do not need to solve all of them to move forward with adopting this.

I've fixed or worked around all of these issues in the core code to get type checking passing in that area. There are still several hundred errors in x-pack and ~30 in OSS code. It would be best to have individual teams resolve the remaining issues.

Before we do that, I think we should discuss & resolve the issues in the first two sections below:

Questions / Discussion

Does the client parse NodeUsageInformation.timestamp from a number to a Date?
CatIndicesResponse returns an object with a records property now, instead of the body just being an array. Is this correct?
client.search should be overloaded so that the SearchResponse.hits.total is the appropriate type based on the rest_total_hits_as_int query parameter
- Currently it is typed as TotalHits | number so the calling code has to know which will be reutrned and do manual casting
- It appears that we have some bugs in our codebase due to assuming that the TotalHits object will be returned when in fact it will be a number

In general, there are many response keys in various APIs that are only present when certain request parameters are set. Ideally, we could leverage function overloading in TypeScript to return a more accurate response type based on which input parameters are specified. This probably won't be trivial to accomplish, but would very powerful to implement if possible.

Major issues (should solve before merging existing PR)

The Client.scroll API is missing from the types
- this one causes a lot of breakages in core.
mget response type hits?: MultiGetHit<object>[] is not compatible with the legacy structure described in the official documentation docs: Documents[] https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-multi-get.html#multi-get-api-response-body. Also note that MultiGetHit property names don't match ones under docs property as they don't start with an underscore.

Minor issues

MultiGetOperation should have these fields as optional rather than required: can_be_flattened, routing, stored_fields, version, version_type
DeleteResponse should have an optional error property which when present has a type property
UpdateRequest does not allow the _source_includes property. I believe we could work around this by using _source instead, but it is part of the ES API, so it should be included in the types.
Sort requires fields that it should not: missing, mode, nested, numeric_type. It should only require order. It should also allow the unmapped_type property.
SearchRequest does not allow the from property for paging orsort in the top-level properties, only within body.
UpdateByQueryRequest does not allow the conflicts property

GetIndexResponse and/or DictionaryResponseBase should create an index signature, I suspect DictionaryResponseBase should be written as:

export interface DictionaryResponseBase<TKey = unknown, TValue = unknown> extends ResponseBase {
  [key: TKey]: TValue
}

UpdateIndexSettingsRequest.body does not allow the index property, only the index_settings property which only allows Record<string, object> which is incompatible with values like { number_of_replicas: 0 }. Same issue with CreateIndexRequest.body.settings.
TypeMapping requires many fields that should be optional: all_field, date_detection, dynamic, dynamic_date_formats, dynamic_templates, index_field, numeric_detection
Client.indices.addBlock API is missing from the types
ReindexOnServerResponse does not include the taskId property
CatTemplatesResponse is missing the templates property
GetTaskResponse requires the property task which may not be present when there is an error. error also does not appear to be present at all.
ReindexDestination requires these properties which should be optional: op_type, pipeline, routing, version_type
ReindexSource requires these properties which should be optional: query, remote, slice, sort, _source
InlineScript requires the params property which should be optional
CatIndicesRecord has a few properties that should be number instead of string: docs.count, docs.deleted, pri.store.size, store.size
Alias requires these properties which should be optional: filter, index_routing, is_hidden, is_write_index, routing, search_routing
IProperty requires these properties which should be optional: local_metadata, meta, name
BulkOperationContainer requires these properties which should be optional: index, create, update, delete
FieldCapabilitiesRequest.fields should allow a string
SearchRequest.body.docvalue_fields should allow a string
IndicesStatsRequest.metrics should allow string[]
NodeUsageInformation is missing aggregations
CatIndicesRequest.h is missing
AggregationContainer should have optional properties when used on SearchRequest.body.aggs
Aggregate used on SearchResponse.body.aggregations does not include any properties except meta. Should include a buckets property.
LicenseInformation is missing properties: expirty_date_in_millis, start_date_in_millis

joshdover · 2020-12-30T17:31:00Z

src/core/server/saved_objects/migrations/core/index_migrator.ts

@@ -143,7 +143,9 @@ async function deleteIndexTemplates({ client, log, obsoleteIndexTemplatePattern
    return;
  }

-  const { body: templates } = await client.cat.templates<Array<{ name: string }>>({
+  const {
+    body: { records: templates },


I'm not sure we should be using this records key in the response or not. Needs to be tested here in other usages of the cat API.

joshdover · 2021-01-15T18:08:01Z

I did another update and round of updates. Was able to remove a lot of @ts-expect-error from Core, so we're getting close!

At this point, the major blocker to continuing is Task Manager's types in x-pack/plugins/task_manager/server/task_store.ts. @elastic/kibana-alerting-services does anyone have time to address these type errors? I think we need to refactor this code to stop using the types defined in x-pack/typings/elasticsearch and use the client-provided types instead (import { estypes } from '@elastic/elasticsearch'). I'm happy to guide anyone through this process, but this code seemed critical enough to give me pause when trying to change it all myself.

Feedback on the elastic/elasticsearch types

@delvedor Here is my current feedback based on Core's usage of the client and a few other spots that get picked up by our type building process.

Major problems

These issues need to be fixed before we can merge this:

MultiGetHit should have _id, _source, _type, _primary_term, _seq_no properties (currently only has properties without _ prefix)
I think it would be very convenient to extract the object part of the union type for SearchRequest['body'] into its own type that is exported from the client library.
I was wrong about the types of some properties on CatIndicesRecord. These properties should be string instead of number: docs.count, docs.deleted, pri.store.size, store.size. Sorry about that!
- As an aside: we have some Core code that expects these to be numbers and is definitely wrong. We will need to fix that!

Minor problems

These issues are less important and can be avoided mostly by using @ts-expect-error for now:

TypeMapping requires some fields that should be optional: _field_names, _meta, _routing, _size, _source
AliasDefinition requires some fields that should be optional: filter, index_routing, is_write_index, routing, search_routing
ScrollRequest does not accept the scroll parameter for the scroll duration
It'd be nice if we could specify a generic for the SearchResponse type to declare when we expect the _id field to only be strings and not number | string
ReindexRequest should allow the optional require_alias property
IProperty should allow nested properties via the properties property
Should Hit._source be optional? I can't seem to create a situation where it is null, only an empty object.
client.asyncSearch.status API is missing
AsyncSearchGetResponse is missing all properties

mikecote · 2021-01-15T19:50:09Z

Thanks for the ping, @joshdover. I've added an item in our project board to help out here. Someone on the team will be in touch soon.

…nt library

gmmorris · 2021-01-20T14:36:38Z

I've pushed a commit with fixes to support Task Manager, but there are a couple of issues in the typing which have forced a bunch of @ts-expect-error:

There appears to be an incompatibility between the _id field on estypes.Hit<SavedObjectsRawDocSource> and SavedObjectsRawDoc. This means that when we use esClient.search<SavedObjectsRawDoc['_source']> to search for SavedObjects (we don't use the SO client.... I can't quite recall why, that predates me and it's possible we can revisit that, but as we do complex updateByQueries we probably still can't) we can't use serializer.rawToSavedObject(doc) without lying to the compiler. (this might be for @elastic/kibana-core to address)
we don't seem to support script based sorting (https://www.elastic.co/guide/en/elasticsearch/reference/6.8/search-request-sort.html#_script_based_sorting) which we need to use. This is missing on both UpdateByQueryRequest and SearchRequest. For now I've lied to the compiler.
The AggregationRange only supports double for from and to fields, but they can be a string too (for time based ranges, such we use).
Aggregations in general don't seem to enjoy the deep type inference that the old types had (thanks to @dgieselaar I believe) - do we plan on introducing them? For now I've had to lie to the compiler (cast to unknown) and reintroduce manual types that we had removed a couple of months ago when we began to use the aggregation types in x-pack/typings/elasticsearch/index.d.ts
The FiltersAggregate doesn't cover the case of a nested AggregationContainer, in which FiltersAggregate would not have a buckets property, but rather a keyed property that's inferred from the request. This was covered by the aggregation inference in the legacy types.

joshdover · 2021-01-29T12:33:03Z

I've done another round of updates and things are looking even better.

I still need to comb through the @ts-expect-error comments that been added in this PR, but I think we're now at the point where the types from the client are good enough to proceed with getting more involvement from other teams.

Feedback on client types for @delvedor

UpdateResponse<T> field get should not be optional
IndexState fields alias and mappings should not be optional
DeleteResponse does not have the optional error property
UpdateByQueryRequest does not allow conflicts property on the body
SourceFilter does not allow regular string arrays
Should ShardStatistics.skipped be optional?

Plugins that we need help with

Right now our type check is held back from running the full type check by any plugins that have migrated to TS references, so this list only includes type errors for plugins that have migrated to TS references. There are likely plenty more errors that we will be able to see only once we have fixed these first.

We need someone from each of these teams to take a look at their areas, prioritized by number of errors (most to least):

@elastic/kibana-security - security plugin has a lot of errors, spaces just a few
@elastic/kibana-app Could someone take a look at the issues in the lens plugin?
Core - I will fix the SOT issues below
Alerting - @gmmorris would you be able to take a look at the Task Manager and Event Log issues?
@elastic/kibana-gis Could someone take a look at the issues in the maps plugin?
@elastic/kibana-app-services Could someone take a look at the issues in the data_enhanced plugin?
@elastic/es-ui Could someone fix the one small issue in the searchprofiler plugin below?

Steps

Pull down this PR: gh pr checkout 83808
Bootstrap: yarn kbn bootstrap
Run type check in your area: node scripts/type_check --project path/to/plugin/tsconfig.json
Fix type issues & commit
- How safe you want this to be is up to you and your team, but ideally we're aiming for as safe as possible.
- In many cases, type errors can be fixed by providing the proper generic to the API call, or by narrowing the types of the arguements to the API call.
- If the request types from the client are wrong or incomplete (which is entirely possible), please add a // @ts-expect-error with a note about why the types are wrong, leave a comment on this PR, and push the change.
Push to this branch: git push delvedor new-types-client

spaces

x-pack/plugins/spaces/server/usage_collection/spaces_usage_collector.ts:47:14 - error TS2322: Type 'SearchResponse<unknown>' is not assignable to type 'SpacesAggregationResponse'.

security

x-pack/plugins/security/server/session_management/session_index.ts:146:38 - error TS2339: Property 'status' does not exist on type 'GetResponse<unknown>'.
x-pack/plugins/security/server/session_management/session_index.ts:153:9 - error TS2698: Spread types may only be created from object types.
x-pack/plugins/security/server/session_management/session_index.ts:219:56 - error TS2339: Property 'status' does not exist on type 'IndexResponse'.
x-pack/plugins/security/server/session_management/session_index.ts:287:13 - error TS2322: Type 'Readonly<{ index_patterns: string; order: number; settings: { index: { number_of_shards: number; number_of_replicas: number; auto_expand_replicas: string; priority: number; refresh_interval: string; hidden: boolean; }; }; mappings: { ...; }; }>' is not assignable to type '{ aliases?: Record<string, Alias> | undefined; index_patterns?: string[] | undefined; mappings?: TypeMapping | undefined; order?: number | undefined; settings?: Record<...> | undefined; version?: number | undefined; }'.
x-pack/plugins/security/server/authentication/api_keys/api_keys.ts:146:83 - error TS2322: Type '{ ids: string[]; }' is not assignable to type '{ id?: string | undefined; name?: string | undefined; owner?: boolean | undefined; realm_name?: string | undefined; username?: string | undefined; }'.
x-pack/plugins/security/server/authentication/api_keys/api_keys.ts:185:5 - error TS2322: Type 'CreateApiKeyResponse' is not assignable to type 'CreateAPIKeyResult'.
x-pack/plugins/security/server/authentication/api_keys/api_keys.ts:243:21 - error TS2322: Type '{ ids: string[]; }' is not assignable to type '{ id?: string | undefined; name?: string | undefined; owner?: boolean | undefined; realm_name?: string | undefined; username?: string | undefined; }'.
x-pack/plugins/security/server/authentication/api_keys/api_keys.ts:277:19 - error TS2322: Type '{ ids: string[]; }' is not assignable to type '{ id?: string | undefined; name?: string | undefined; owner?: boolean | undefined; realm_name?: string | undefined; username?: string | undefined; }'.
x-pack/plugins/security/server/authentication/tokens.ts:58:9 - error TS2339: Property 'refresh_token' does not exist on type 'GetUserAccessTokenResponse'.
x-pack/plugins/security/server/authentication/tokens.ts:59:9 - error TS2339: Property 'authentication' does not exist on type 'GetUserAccessTokenResponse'.
x-pack/plugins/security/server/authentication/tokens.ts:65:22 - error TS2322: Type '"refresh_token"' is not assignable to type '"password" | undefined'.
x-pack/plugins/security/server/authentication/tokens.ts:114:13 - error TS2345: Argument of type '{ body: { refresh_token: string; }; }' is not assignable to parameter of type 'InvalidateUserAccessTokenRequest'.
x-pack/plugins/security/server/authentication/tokens.ts:146:13 - error TS2345: Argument of type '{ body: { token: string; }; }' is not assignable to parameter of type 'InvalidateUserAccessTokenRequest'.
x-pack/plugins/security/server/authentication/providers/base.ts:114:7 - error TS2345: Argument of type 'AuthenticateResponse' is not assignable to parameter of type 'Pick<AuthenticatedUser, "metadata" | "authentication_realm" | "lookup_realm" | "authentication_type" | "username" | "email" | "full_name" | "roles" | "enabled">'.
x-pack/plugins/security/server/authentication/providers/kerberos.ts:154:7 - error TS2739: Type 'GetUserAccessTokenResponse' is missing the following properties from type '{ access_token: string; refresh_token: string; kerberos_authentication_response_token?: string | undefined; authentication: Pick<AuthenticatedUser, "metadata" | ... 7 more ... | "enabled">; }': refresh_token, authentication
x-pack/plugins/security/server/authentication/providers/kerberos.ts:156:19 - error TS2322: Type '"_kerberos"' is not assignable to type '"password" | undefined'.
x-pack/plugins/security/server/authentication/providers/token.ts:68:9 - error TS2339: Property 'refresh_token' does not exist on type 'GetUserAccessTokenResponse'.
x-pack/plugins/security/server/authentication/providers/token.ts:69:9 - error TS2339: Property 'authentication' does not exist on type 'GetUserAccessTokenResponse'.
x-pack/plugins/security/server/authentication/providers/token.ts:75:46 - error TS2322: Type '{ grant_type: "password"; username: string; password: string; }' is not assignable to type '{ grant_type?: "password" | undefined; scope?: string | undefined; }'.
x-pack/plugins/security/server/routes/authorization/roles/get.ts:33:15 - error TS2345: Argument of type 'XPackRole' is not assignable to parameter of type 'ElasticsearchRole'.
  x-pack/plugins/security/common/model/role.ts:29:3
x-pack/plugins/security/server/routes/authorization/roles/get_all.ts:27:48 - error TS2345: Argument of type 'XPackRole' is not assignable to parameter of type 'ElasticsearchRole'.
x-pack/plugins/security/server/routes/authorization/roles/put.ts:75:43 - error TS2339: Property 'applications' does not exist on type 'XPackRole'.
x-pack/plugins/security/server/routes/authorization/roles/put.ts:82:13 - error TS2322: Type 'Pick<ElasticsearchRole, "metadata" | "transient_metadata" | "indices" | "cluster" | "run_as" | "applications">' is not assignable to type '{ applications?: ApplicationPrivileges[] | undefined; cluster?: string[] | undefined; global?: Record<string, any> | undefined; indices?: IndicesPrivileges[] | undefined; metadata?: Record<...> | undefined; run_as?: string[] | undefined; }'.
x-pack/plugins/security/server/routes/indices/get_fields.ts:53:86 - error TS2339: Property 'mapping' does not exist on type 'FieldMapping'.
x-pack/plugins/security/server/routes/indices/get_fields.ts:56:54 - error TS2571: Object is of type 'unknown'.
x-pack/plugins/security/server/routes/users/create_or_update.ts:33:11 - error TS2322: Type 'Readonly<{ metadata?: Record<string, any> | undefined; email?: string | null | undefined; full_name?: string | null | undefined; password?: string | undefined; } & { username: string; roles: string[]; enabled: boolean; }>' is not assignable to type '{ email?: string | undefined; full_name?: string | undefined; metadata?: Record<string, any> | undefined; password?: string | undefined; password_hash?: string | undefined; roles?: string[] | undefined; }'.
x-pack/plugins/security/server/routes/role_mapping/get.ts:40:38 - error TS2339: Property 'role_templates' does not exist on type 'XPackRoleMapping'.
x-pack/plugins/security/server/routes/role_mapping/get.ts:40:65 - error TS7006: Parameter 'entry' implicitly has an 'any' type.
x-pack/plugins/security/server/authorization/check_privileges.ts:56:13 - error TS2322: Type '{ names: string; privileges: string[]; }[]' is not assignable to type 'IndexPrivilegesCheck[]'.
x-pack/plugins/security/server/authorization/check_privileges.ts:69:9 - error TS2345: Argument of type 'import("/Users/jdover/src/kibana/node_modules/@elastic/elasticsearch/api/types").HasPrivilegesResponse' is not assignable to parameter of type 'import("/Users/jdover/src/kibana/x-pack/plugins/security/server/authorization/types").HasPrivilegesResponse'.
x-pack/plugins/security/server/authorization/check_privileges.ts:98:34 - error TS2345: Argument of type 'ResourcePrivileges[]' is not assignable to parameter of type 'HasPrivilegesResponseApplication'.
x-pack/plugins/security/server/authorization/check_privileges.ts:106:9 - error TS2322: Type 'Partial<ResourcePrivileges>' is not assignable to type '{ [privilegeName: string]: boolean; }'.
x-pack/plugins/security/server/authentication/tokens.test.ts:106:9 - error TS2345: Argument of type 'ApiResponse<{ access_token: string; refresh_token: string; authentication: { authentication_realm: UserRealm | { name: string; type: string; }; lookup_realm: UserRealm | { ...; }; ... 7 more ...; metadata?: { ...; } | undefined; }; }, unknown>' is not assignable to parameter of type 'ApiResponse<GetUserAccessTokenResponse, unknown> | TransportRequestPromise<ApiResponse<GetUserAccessTokenResponse, unknown>>'.
x-pack/plugins/security/server/authentication/tokens.test.ts:199:9 - error TS2345: Argument of type 'ApiResponse<{ invalidated_tokens: number; }, unknown>' is not assignable to parameter of type 'ApiResponse<InvalidateUserAccessTokenResponse, unknown> | TransportRequestPromise<ApiResponse<InvalidateUserAccessTokenResponse, unknown>>'.
x-pack/plugins/security/server/authentication/tokens.test.ts:217:9 - error TS2345: Argument of type 'ApiResponse<{ invalidated_tokens: number; }, unknown>' is not assignable to parameter of type 'ApiResponse<InvalidateUserAccessTokenResponse, unknown> | TransportRequestPromise<ApiResponse<InvalidateUserAccessTokenResponse, unknown>>'.
x-pack/plugins/security/server/authentication/tokens.test.ts:232:9 - error TS2345: Argument of type 'ApiResponse<{ invalidated_tokens: number; }, unknown>' is not assignable to parameter of type 'ApiResponse<InvalidateUserAccessTokenResponse, unknown> | TransportRequestPromise<ApiResponse<InvalidateUserAccessTokenResponse, unknown>>'.
x-pack/plugins/security/server/authentication/tokens.test.ts:276:9 - error TS2345: Argument of type 'ApiResponse<{ invalidated_tokens: number; }, unknown>' is not assignable to parameter of type 'ApiResponse<InvalidateUserAccessTokenResponse, unknown> | TransportRequestPromise<ApiResponse<InvalidateUserAccessTokenResponse, unknown>>'.
x-pack/plugins/security/server/authentication/api_keys/api_keys.test.ts:70:9 - error TS2345: Argument of type 'ApiResponse<{}, unknown>' is not assignable to parameter of type 'ApiResponse<InvalidateApiKeyResponse, unknown> | TransportRequestPromise<ApiResponse<InvalidateApiKeyResponse, unknown>>'.
x-pack/plugins/security/server/authentication/api_keys/api_keys.test.ts:111:9 - error TS2345: Argument of type 'ApiResponse<{}, unknown>' is not assignable to parameter of type 'ApiResponse<InvalidateApiKeyResponse, unknown> | TransportRequestPromise<ApiResponse<InvalidateApiKeyResponse, unknown>>'.
x-pack/plugins/security/server/authentication/api_keys/api_keys.test.ts:303:9 - error TS2345: Argument of type 'ApiResponse<{ invalidated_api_keys: string[]; previously_invalidated_api_keys: never[]; error_count: number; }, unknown>' is not assignable to parameter of type 'ApiResponse<InvalidateApiKeyResponse, unknown> | TransportRequestPromise<ApiResponse<InvalidateApiKeyResponse, unknown>>'.
x-pack/plugins/security/server/authentication/api_keys/api_keys.test.ts:329:9 - error TS2345: Argument of type 'ApiResponse<{ invalidated_api_keys: string[]; previously_invalidated_api_keys: never[]; error_count: number; }, unknown>' is not assignable to parameter of type 'ApiResponse<InvalidateApiKeyResponse, unknown> | TransportRequestPromise<ApiResponse<InvalidateApiKeyResponse, unknown>>'.
x-pack/plugins/security/server/authentication/api_keys/api_keys.test.ts:365:9 - error TS2345: Argument of type 'ApiResponse<{ invalidated_api_keys: string[]; previously_invalidated_api_keys: never[]; error_count: number; }, unknown>' is not assignable to parameter of type 'ApiResponse<InvalidateApiKeyResponse, unknown> | TransportRequestPromise<ApiResponse<InvalidateApiKeyResponse, unknown>>'.
x-pack/plugins/security/server/authentication/api_keys/api_keys.test.ts:389:9 - error TS2345: Argument of type 'ApiResponse<{ invalidated_api_keys: string[]; previously_invalidated_api_keys: never[]; error_count: number; }, unknown>' is not assignable to parameter of type 'ApiResponse<InvalidateApiKeyResponse, unknown> | TransportRequestPromise<ApiResponse<InvalidateApiKeyResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/anonymous.test.ts:82:13 - error TS2345: Argument of type 'ApiResponse<{ authentication_realm: UserRealm | { name: string; type: string; }; lookup_realm: UserRealm | { name: string; type: string; }; authentication_provider: AuthenticationProvider | { ...; }; ... 6 more ...; metadata?: { ...; } | undefined; }, unknown>' is not assignable to parameter of type 'ApiResponse<AuthenticateResponse, unknown> | TransportRequestPromise<ApiResponse<AuthenticateResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/anonymous.test.ts:167:13 - error TS2345: Argument of type 'ApiResponse<{ authentication_realm: UserRealm | { name: string; type: string; }; lookup_realm: UserRealm | { name: string; type: string; }; authentication_provider: AuthenticationProvider | { ...; }; ... 6 more ...; metadata?: { ...; } | undefined; }, unknown>' is not assignable to parameter of type 'ApiResponse<AuthenticateResponse, unknown> | TransportRequestPromise<ApiResponse<AuthenticateResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/anonymous.test.ts:183:13 - error TS2345: Argument of type 'ApiResponse<{ authentication_realm: UserRealm | { name: string; type: string; }; lookup_realm: UserRealm | { name: string; type: string; }; authentication_provider: AuthenticationProvider | { ...; }; ... 6 more ...; metadata?: { ...; } | undefined; }, unknown>' is not assignable to parameter of type 'ApiResponse<AuthenticateResponse, unknown> | TransportRequestPromise<ApiResponse<AuthenticateResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/anonymous.test.ts:201:13 - error TS2345: Argument of type 'ApiResponse<{ authentication_realm: UserRealm | { name: string; type: string; }; lookup_realm: UserRealm | { name: string; type: string; }; authentication_provider: AuthenticationProvider | { ...; }; ... 6 more ...; metadata?: { ...; } | undefined; }, unknown>' is not assignable to parameter of type 'ApiResponse<AuthenticateResponse, unknown> | TransportRequestPromise<ApiResponse<AuthenticateResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/anonymous.test.ts:247:15 - error TS2345: Argument of type 'ApiResponse<{ authentication_realm: UserRealm | { name: string; type: string; }; lookup_realm: UserRealm | { name: string; type: string; }; authentication_provider: AuthenticationProvider | { ...; }; ... 6 more ...; metadata?: { ...; } | undefined; }, unknown>' is not assignable to parameter of type 'ApiResponse<AuthenticateResponse, unknown> | TransportRequestPromise<ApiResponse<AuthenticateResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/basic.test.ts:52:9 - error TS2345: Argument of type 'ApiResponse<{ authentication_realm: UserRealm | { name: string; type: string; }; lookup_realm: UserRealm | { name: string; type: string; }; authentication_provider: AuthenticationProvider | { ...; }; ... 6 more ...; metadata?: { ...; } | undefined; }, unknown>' is not assignable to parameter of type 'ApiResponse<AuthenticateResponse, unknown> | TransportRequestPromise<ApiResponse<AuthenticateResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/basic.test.ts:162:9 - error TS2345: Argument of type 'ApiResponse<{ authentication_realm: UserRealm | { name: string; type: string; }; lookup_realm: UserRealm | { name: string; type: string; }; authentication_provider: AuthenticationProvider | { ...; }; ... 6 more ...; metadata?: { ...; } | undefined; }, unknown>' is not assignable to parameter of type 'ApiResponse<AuthenticateResponse, unknown> | TransportRequestPromise<ApiResponse<AuthenticateResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/http.test.ts:125:11 - error TS2345: Argument of type 'ApiResponse<{ authentication_realm: UserRealm | { name: string; type: string; }; lookup_realm: UserRealm | { name: string; type: string; }; authentication_provider: AuthenticationProvider | { ...; }; ... 6 more ...; metadata?: { ...; } | undefined; }, unknown>' is not assignable to parameter of type 'ApiResponse<AuthenticateResponse, unknown> | TransportRequestPromise<ApiResponse<AuthenticateResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/kerberos.test.ts:47:9 - error TS2345: Argument of type 'ApiResponse<{}, unknown>' is not assignable to parameter of type 'ApiResponse<AuthenticateResponse, unknown> | TransportRequestPromise<ApiResponse<AuthenticateResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/kerberos.test.ts:123:9 - error TS2345: Argument of type 'ApiResponse<{ access_token: string; refresh_token: string; authentication: { authentication_realm: UserRealm | { name: string; type: string; }; lookup_realm: UserRealm | { ...; }; ... 7 more ...; metadata?: { ...; } | undefined; }; }, unknown>' is not assignable to parameter of type 'ApiResponse<GetUserAccessTokenResponse, unknown> | TransportRequestPromise<ApiResponse<GetUserAccessTokenResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/kerberos.test.ts:157:9 - error TS2345: Argument of type 'ApiResponse<{ access_token: string; refresh_token: string; kerberos_authentication_response_token: string; authentication: { authentication_realm: UserRealm | { name: string; type: string; }; ... 8 more ...; metadata?: { ...; } | undefined; }; }, unknown>' is not assignable to parameter of type 'ApiResponse<GetUserAccessTokenResponse, unknown> | TransportRequestPromise<ApiResponse<GetUserAccessTokenResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/kerberos.test.ts:347:9 - error TS2345: Argument of type 'ApiResponse<{ authentication_realm: UserRealm | { name: string; type: string; }; lookup_realm: UserRealm | { name: string; type: string; }; authentication_provider: AuthenticationProvider | { ...; }; ... 6 more ...; metadata?: { ...; } | undefined; }, unknown>' is not assignable to parameter of type 'ApiResponse<AuthenticateResponse, unknown> | TransportRequestPromise<ApiResponse<AuthenticateResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/oidc.test.ts:35:7 - error TS2345: Argument of type 'ApiResponse<AuthenticatedUser, unknown>' is not assignable to parameter of type 'ApiResponse<AuthenticateResponse, unknown> | TransportRequestPromise<ApiResponse<AuthenticateResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/pki.test.ts:670:9 - error TS2345: Argument of type 'ApiResponse<{ authentication_realm: UserRealm | { name: string; type: string; }; lookup_realm: UserRealm | { name: string; type: string; }; authentication_provider: AuthenticationProvider | { ...; }; ... 6 more ...; metadata?: { ...; } | undefined; }, unknown>' is not assignable to parameter of type 'ApiResponse<AuthenticateResponse, unknown> | TransportRequestPromise<ApiResponse<AuthenticateResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/saml.test.ts:33:7 - error TS2345: Argument of type 'ApiResponse<AuthenticatedUser, unknown>' is not assignable to parameter of type 'ApiResponse<AuthenticateResponse, unknown> | TransportRequestPromise<ApiResponse<AuthenticateResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/token.test.ts:51:9 - error TS2345: Argument of type 'ApiResponse<{ access_token: string; refresh_token: string; authentication: { authentication_realm: UserRealm | { name: string; type: string; }; lookup_realm: UserRealm | { ...; }; ... 7 more ...; metadata?: { ...; } | undefined; }; }, unknown>' is not assignable to parameter of type 'ApiResponse<GetUserAccessTokenResponse, unknown> | TransportRequestPromise<ApiResponse<GetUserAccessTokenResponse, unknown>>'.
x-pack/plugins/security/server/authentication/providers/token.test.ts:164:9 - error TS2345: Argument of type 'ApiResponse<{ authentication_realm: UserRealm | { name: string; type: string; }; lookup_realm: UserRealm | { name: string; type: string; }; authentication_provider: AuthenticationProvider | { ...; }; ... 6 more ...; metadata?: { ...; } | undefined; }, unknown>' is not assignable to parameter of type 'ApiResponse<AuthenticateResponse, unknown> | TransportRequestPromise<ApiResponse<AuthenticateResponse, unknown>>'.
x-pack/plugins/security/server/session_management/session_index.test.ts:167:9 - error TS2345: Argument of type 'ApiResponse<{}, unknown>' is not assignable to parameter of type 'ApiResponse<DeleteByQueryResponse, unknown> | TransportRequestPromise<ApiResponse<DeleteByQueryResponse, unknown>>'.
x-pack/plugins/security/server/session_management/session_index.test.ts:601:9 - error TS2345: Argument of type 'ApiResponse<{ status: number; }, unknown>' is not assignable to parameter of type 'ApiResponse<GetResponse<unknown>, unknown> | TransportRequestPromise<ApiResponse<GetResponse<unknown>, unknown>>'.
x-pack/plugins/security/server/session_management/session_index.test.ts:609:9 - error TS2345: Argument of type 'ApiResponse<{ status: number; found: false; }, unknown>' is not assignable to parameter of type 'ApiResponse<GetResponse<unknown>, unknown> | TransportRequestPromise<ApiResponse<GetResponse<unknown>, unknown>>'.
x-pack/plugins/security/server/session_management/session_index.test.ts:625:9 - error TS2345: Argument of type 'ApiResponse<{ found: true; status: number; _source: { usernameHash: string; provider: { type: string; name: string; }; idleTimeoutExpiration: number; lifespanExpiration: null; content: string; }; _primary_term: number; _seq_no: number; }, unknown>' is not assignable to parameter of type 'ApiResponse<GetResponse<unknown>, unknown> | TransportRequestPromise<ApiResponse<GetResponse<unknown>, unknown>>'.
x-pack/plugins/security/server/session_management/session_index.test.ts:671:9 - error TS2345: Argument of type 'ApiResponse<{ _primary_term: number; _seq_no: number; }, unknown>' is not assignable to parameter of type 'ApiResponse<CreateResponse, unknown> | TransportRequestPromise<ApiResponse<CreateResponse, unknown>>'.
x-pack/plugins/security/server/session_management/session_index.test.ts:719:9 - error TS2345: Argument of type 'ApiResponse<{ found: true; status: number; _source: { usernameHash: string; provider: { type: string; name: string; }; idleTimeoutExpiration: number; lifespanExpiration: number; content: string; }; _primary_term: number; _seq_no: number; }, unknown>' is not assignable to parameter of type 'ApiResponse<GetResponse<unknown>, unknown> | TransportRequestPromise<ApiResponse<GetResponse<unknown>, unknown>>'.
x-pack/plugins/security/server/session_management/session_index.test.ts:730:9 - error TS2345: Argument of type 'ApiResponse<{ status: number; }, unknown>' is not assignable to parameter of type 'ApiResponse<IndexResponse, unknown> | TransportRequestPromise<ApiResponse<IndexResponse, unknown>>'.
x-pack/plugins/security/server/session_management/session_index.test.ts:764:9 - error TS2345: Argument of type 'ApiResponse<{ _primary_term: number; _seq_no: number; status: number; }, unknown>' is not assignable to parameter of type 'ApiResponse<IndexResponse, unknown> | TransportRequestPromise<ApiResponse<IndexResponse, unknown>>'.

task_manager

x-pack/plugins/task_manager/server/task_store.test.ts:1047:9 - error TS2322: Type '({ _id: string; _source: { type: string; task: { runAt: Date; taskType: string; schedule: undefined; attempts: number; status: string; params: string; state: string; user: string; scope: string[]; ownerId: string; }; }; _seq_no: number; _primary_term: number; sort: (string | number)[]; } | { ...; })[]' is not assignable to type 'Hit<unknown>[]'.

event_log

x-pack/plugins/event_log/server/es/cluster_client_adapter.ts:182:9 - error TS2322: Type 'string | Record<string, unknown>' is not assignable to type '{ aliases?: Record<string, Alias> | undefined; mappings?: TypeMapping | undefined; settings?: Record<string, any> | undefined; } | undefined'.
x-pack/plugins/event_log/server/es/cluster_client_adapter.ts:300:9 - error TS2322: Type '{ size: number; from: number; sort: { [x: string]: { order: "asc" | "desc"; }; }; query: { bool: { filter: never[] | JsonObject; must: ("" | { nested: { path: string; query: { bool: { must: ({ bool: { must_not: { ...; }; }; } | { ...; } | { ...; } | { ...; } | { ...; })[]; }; }; }; range?: undefined; } | { ...; } | ...' is not assignable to type '{ aggs?: Record<string, AggregationContainer> | undefined; aggregations?: Record<string, AggregationContainer> | undefined; ... 26 more ...; pit?: PointInTimeReference | undefined; }'.
x-pack/plugins/event_log/server/es/cluster_client_adapter.ts:305:22 - error TS2339: Property 'value' does not exist on type 'number | TotalHits'.
x-pack/plugins/event_log/server/es/cluster_client_adapter.ts:306:24 - error TS2345: Argument of type '(hit: {    _source: unknown;}) => unknown' is not assignable to parameter of type '(value: Hit<unknown>, index: number, array: Hit<unknown>[]) => unknown'.
x-pack/plugins/event_log/server/es/cluster_client_adapter.test.ts:326:7 - error TS2345: Argument of type 'RequestEvent<{ hits: { hits: never[]; total: { value: number; }; }; }, unknown>' is not assignable to parameter of type 'ApiResponse<SearchResponse<unknown>, unknown> | TransportRequestPromise<ApiResponse<SearchResponse<unknown>, unknown>>'.
x-pack/plugins/event_log/server/es/cluster_client_adapter.test.ts:407:7 - error TS2345: Argument of type 'RequestEvent<{ hits: { hits: never[]; total: { value: number; }; }; }, unknown>' is not assignable to parameter of type 'ApiResponse<SearchResponse<unknown>, unknown> | TransportRequestPromise<ApiResponse<SearchResponse<unknown>, unknown>>'.
x-pack/plugins/event_log/server/es/cluster_client_adapter.test.ts:490:7 - error TS2345: Argument of type 'RequestEvent<{ hits: { hits: never[]; total: { value: number; }; }; }, unknown>' is not assignable to parameter of type 'ApiResponse<SearchResponse<unknown>, unknown> | TransportRequestPromise<ApiResponse<SearchResponse<unknown>, unknown>>'.
x-pack/plugins/event_log/server/es/cluster_client_adapter.test.ts:516:7 - error TS2345: Argument of type 'RequestEvent<{ hits: { hits: never[]; total: { value: number; }; }; }, unknown>' is not assignable to parameter of type 'ApiResponse<SearchResponse<unknown>, unknown> | TransportRequestPromise<ApiResponse<SearchResponse<unknown>, unknown>>'.
x-pack/plugins/event_log/server/es/cluster_client_adapter.test.ts:607:7 - error TS2345: Argument of type 'RequestEvent<{ hits: { hits: never[]; total: { value: number; }; }; }, unknown>' is not assignable to parameter of type 'ApiResponse<SearchResponse<unknown>, unknown> | TransportRequestPromise<ApiResponse<SearchResponse<unknown>, unknown>>'.

saved_objects_tagging

x-pack/plugins/saved_objects_tagging/server/usage/fetch_tag_usage_data.ts:61:23 - error TS2532: Object is possibly 'undefined'.
x-pack/plugins/saved_objects_tagging/server/usage/fetch_tag_usage_data.ts:61:49 - error TS2339: Property 'buckets' does not exist on type 'Aggregate'.
x-pack/plugins/saved_objects_tagging/server/usage/fetch_tag_usage_data.ts:62:24 - error TS7006: Parameter 'bucket' implicitly has an 'any' type.
x-pack/plugins/saved_objects_tagging/server/usage/fetch_tag_usage_data.ts:66:8 - error TS7006: Parameter 'tagBucket' implicitly has an 'any' type.
x-pack/plugins/saved_objects_tagging/server/usage/fetch_tag_usage_data.ts:70:25 - error TS7006: Parameter 'tagId' implicitly has an 'any' type.

lens

x-pack/plugins/lens/server/routes/existing_fields.ts:123:40 - error TS2345: Argument of type 'Hit<unknown>[]' is not assignable to parameter of type '{ [key: string]: unknown; fields: Record<string, unknown[]>; }[]'.
x-pack/plugins/lens/server/routes/existing_fields.ts:191:7 - error TS2322: Type 'Record<string, unknown>' is not assignable to type 'Record<string, ScriptField>'.
x-pack/plugins/lens/server/routes/field_stats.ts:81:15 - error TS2322: Type 'unknown' is not assignable to type 'Record<string, AggregationContainer> | undefined'.
x-pack/plugins/lens/server/usage/visualization_counts.ts:52:19 - error TS2532: Object is possibly 'undefined'.
x-pack/plugins/lens/server/usage/visualization_counts.ts:52:47 - error TS2339: Property 'buckets' does not exist on type 'Aggregate'.
x-pack/plugins/lens/server/usage/task.ts:139:31 - error TS2339: Property 'buckets' does not exist on type 'Aggregate'.
x-pack/plugins/lens/server/usage/task.ts:139:48 - error TS7006: Parameter 'daily' implicitly has an 'any' type.
x-pack/plugins/lens/server/usage/task.ts:141:63 - error TS7006: Parameter 'bucket' implicitly has an 'any' type.
x-pack/plugins/lens/server/usage/task.ts:147:66 - error TS7006: Parameter 'bucket' implicitly has an 'any' type.

data_enhanced

x-pack/plugins/data_enhanced/server/search/eql_search_strategy.ts:37:7 - error TS2578: Unused '@ts-expect-error' directive.
x-pack/plugins/data_enhanced/server/search/eql_search_strategy.ts:74:7 - error TS2578: Unused '@ts-expect-error' directive.

maps

x-pack/plugins/maps/public/classes/sources/es_source/es_source.ts:284:12 - error TS2532: Object is possibly 'undefined'.
x-pack/plugins/maps/public/classes/sources/es_source/es_source.ts:284:44 - error TS2339: Property 'bounds' does not exist on type 'Aggregate'.
x-pack/plugins/maps/public/classes/sources/es_source/es_source.ts:288:18 - error TS2532: Object is possibly 'undefined'.
x-pack/plugins/maps/public/classes/sources/es_source/es_source.ts:288:50 - error TS2339: Property 'bounds' does not exist on type 'Aggregate'.

searchprofiler

x-pack/plugins/searchprofiler/server/routes/profile.ts:46:42 - error TS2345: Argument of type '{ index: string; body: string; }' is not assignable to parameter of type 'SearchRequest'.

cjcenizal · 2021-01-29T17:16:55Z

I'll look into the Search Profiler error.

weltenwort

infra plugin changes LGTM

shahzad31

LGTM, User experience and Uptime changes !!

FrankHassanabad

Did review from our end for detections engine. LGTM for us, thank you for all the good notes and good comments along the way for us to do follow ups.

Also thank you for treading some lightly in the logic but still fixing a few things along the way, we appreciate it.

FrankHassanabad · 2021-03-24T15:51:45Z

x-pack/plugins/security_solution/server/lib/detection_engine/signals/bulk_create_ml_signals.ts

  }));

+  // @ts-expect-error Anomaly is not assignable to EcsAnomaly


cc @rylnd (FYI)

FrankHassanabad · 2021-03-24T15:52:32Z

...urity_solution/server/lib/detection_engine/signals/threshold/get_threshold_bucket_filters.ts

@@ -34,7 +34,7 @@ export const getThresholdBucketFilters = async ({

      bucket.terms.forEach((term) => {
        if (term.field != null) {
-          (filter.bool.filter as ESFilter[]).push({
+          (filter.bool!.filter as ESFilter[]).push({


cc @madirey

This one is fine, as filter.bool is hard-coded directly above this. So we definitely won't see any change in behavior and this should work as expected.

FrankHassanabad · 2021-03-24T15:54:07Z

...ins/security_solution/server/lib/detection_engine/migrations/get_signals_indices_in_range.ts

@@ -60,6 +60,7 @@ export const getSignalsIndicesInRange = async ({
                '@timestamp': {
                  gte: from,
                  lte: 'now',
+                  // @ts-expect-error format doesn't exist in RangeQuery


FrankHassanabad · 2021-03-24T15:54:49Z

x-pack/plugins/security_solution/server/lib/detection_engine/signals/build_event_type_signal.ts

-  if (doc._source.event != null && doc._source.event instanceof Object) {
-    return { ...doc._source.event, kind: 'signal' };
+  if (doc._source?.event != null && doc._source?.event instanceof Object) {
+    return { ...doc._source!.event, kind: 'signal' };


cc @dhurley14 @marshallmain

FrankHassanabad · 2021-03-24T15:54:57Z

x-pack/plugins/security_solution/server/lib/detection_engine/signals/build_event_type_signal.ts

@@ -25,5 +24,5 @@ export const buildEventTypeSignal = (doc: BaseSignalHit): object => {
 * @param doc The document which might be a signal or it might be a regular log
 */
 export const isEventTypeSignal = (doc: BaseSignalHit): boolean => {
-  return doc._source.signal?.rule?.id != null && typeof doc._source.signal?.rule?.id === 'string';
+  return doc._source?.signal?.rule?.id != null && typeof doc._source?.signal?.rule?.id === 'string';


cc @dhurley14 @marshallmain

_source is optional in the new type definitions. Hopefully, it will be fixed in elastic/elasticsearch-specification#197

It is optional, _source can be set to false, and maybe we want to encourage use of the fields API instead anyway?

FrankHassanabad · 2021-03-24T15:56:01Z

x-pack/plugins/security_solution/server/lib/detection_engine/index/get_index_exists.ts

      allow_no_indices: true,
+      body: {
+        terminate_after: 1,
+      },


FrankHassanabad · 2021-03-24T15:56:21Z

x-pack/plugins/security_solution/server/lib/detection_engine/signals/utils.ts

@@ -730,6 +731,7 @@ export const mergeSearchResults = (searchResults: SignalSearchResponse[]) => {
        total: newShards.total + existingShards.total,
        successful: newShards.successful + existingShards.successful,
        failed: newShards.failed + existingShards.failed,
+        // @ts-expect-error @elastic/elaticsearch skipped is optional in ShardStatistics


cc @FrankHassanabad

igoristic

Stack Monitoring looking good. Thank you for ding this!

* Use client from branch * Get type checking working in core * Fix types in other plugins * Update client types + remove type errors from core * migrate Task Manager Elasticsearch typing from legacy library to client library * use SortOrder instead o string in alerts * Update client types + fix core type issues * fix maps ts errors * Update Lens types * Convert Search Profiler body from a string to an object to conform to SearchRequest type. * Fix SOT types * Fix/mute Security/Spaces plugins type errors. * Fix bootstrap types * Fix painless_lab * corrected es typing in Event Log * Use new types from client for inferred search responses * Latest type defs * Integrate latest type defs for APM/UX * fix core errors * fix telemetry errors * fix canvas errors * fix data_enhanced errors * fix event_log errors * mute lens errors * fix or mute maps errors * fix reporting errors * fix security errors * mute errors in task_manager * fix errors in telemetry_collection_xpack * fix errors in data plugins * fix errors in alerts * mute errors in index_management * fix task_manager errors * mute or fix lens errors * fix upgrade_assistant errors * fix or mute errors in index_lifecycle_management * fix discover errors * fix core tests * ML changes * fix core type errors * mute error in kbn-es-archiver * fix error in data plugin * fix error in telemetry plugin * fix error in discover * fix discover errors * fix errors in task_manager * fix security errors * fix wrong conflict resolution * address errors with upstream code * update deps to the last commit * remove outdated comments * fix core errors * fix errors after update * adding more expect errors to ML * pull the lastest changes * fix core errors * fix errors in infra plugin * fix errors in uptime plugin * fix errors in ml * fix errors in xpack telemetry * fix or mute errors in transform * fix errors in upgrade assistant * fix or mute fleet errors * start fixing apm errors * fix errors in osquery * fix telemetry tests * core cleanup * fix asMutableArray imports * cleanup * data_enhanced cleanup * cleanup events_log * cleaup * fix error in kbn-es-archiver * fix errors in kbn-es-archiver * fix errors in kbn-es-archiver * fix ES typings for Hit * fix SO * fix actions plugin * fix fleet * fix maps * fix stack_alerts * fix eslint problems * fix event_log unit tests * fix failures in data_enhanced tests * fix test failure in kbn-es-archiver * fix test failures in index_pattern_management * fixing ML test * remove outdated comment in kbn-es-archiver * fix error type in ml * fix eslint errors in osquery plugin * fix runtime error in infra plugin * revert changes to event_log cluser exist check * fix eslint error in osquery * fixing ML endpoint argument types * fx types * Update api-extractor docs * attempt fix for ese test * Fix lint error * Fix types for ts refs * Fix data_enhanced unit test * fix lens types * generate docs * Fix a number of type issues in monitoring and ml * fix triggers_actions_ui * Fix ILM functional test * Put search.d.ts typings back * fix data plugin * Update typings in typings/elasticsearch * Update snapshots * mute errors in task_manager * mute fleet errors * lens. remove unnecessary ts-expect-errors * fix errors in stack_alerts * mute errors in osquery * fix errors in security_solution * fix errors in lists * fix errors in cases * mute errors in search_examples * use KibanaClient to enforce promise-based API * fix errors in test/ folder * update comment * fix errors in x-pack/test folder * fix errors in ml plugin * fix optional fields in ml api_integartoon tests * fix another casting problem in ml tests * fix another ml test failure * fix fleet problem after conflict resolution * rollback changes in security_solution. trying to fix test * Update type for discover rows * uncomment runtime_mappings as its outdated * address comments from Wylie * remove eslint error due to any * mute error due to incompatibility * Apply suggestions from code review Co-authored-by: John Schulz <[email protected]> * fix type error in lens tests * Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.ts Co-authored-by: Alison Goryachev <[email protected]> * Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts Co-authored-by: Alison Goryachev <[email protected]> * update deps * fix errors in core types * fix errors for the new elastic/elasticsearch version * remove unused type * remove unnecessary manual type cast and put optional chaining back * ML: mute Datafeed is missing indices_options * Apply suggestions from code review Co-authored-by: Josh Dover <[email protected]> * use canary pacakge instead of git commit Co-authored-by: Josh Dover <[email protected]> Co-authored-by: Josh Dover <[email protected]> Co-authored-by: Gidi Meir Morris <[email protected]> Co-authored-by: Nathan Reese <[email protected]> Co-authored-by: Wylie Conlon <[email protected]> Co-authored-by: CJ Cenizal <[email protected]> Co-authored-by: Aleh Zasypkin <[email protected]> Co-authored-by: Dario Gieselaar <[email protected]> Co-authored-by: restrry <[email protected]> Co-authored-by: James Gowdy <[email protected]> Co-authored-by: John Schulz <[email protected]> Co-authored-by: Alison Goryachev <[email protected]> # Conflicts: # package.json # src/core/server/saved_objects/migrationsv2/integration_tests/migration.test.ts # src/plugins/data/server/autocomplete/value_suggestions_route.ts # src/plugins/discover/public/application/angular/doc_table/create_doc_table_react.tsx # x-pack/plugins/apm/server/lib/services/get_service_dependencies/get_destination_map.ts # x-pack/plugins/infra/server/lib/alerting/inventory_metric_threshold/evaluate_condition.ts # x-pack/plugins/security/server/authentication/api_keys/api_keys.ts # yarn.lock

* ES client : use the new type definitions (#83808) * Use client from branch * Get type checking working in core * Fix types in other plugins * Update client types + remove type errors from core * migrate Task Manager Elasticsearch typing from legacy library to client library * use SortOrder instead o string in alerts * Update client types + fix core type issues * fix maps ts errors * Update Lens types * Convert Search Profiler body from a string to an object to conform to SearchRequest type. * Fix SOT types * Fix/mute Security/Spaces plugins type errors. * Fix bootstrap types * Fix painless_lab * corrected es typing in Event Log * Use new types from client for inferred search responses * Latest type defs * Integrate latest type defs for APM/UX * fix core errors * fix telemetry errors * fix canvas errors * fix data_enhanced errors * fix event_log errors * mute lens errors * fix or mute maps errors * fix reporting errors * fix security errors * mute errors in task_manager * fix errors in telemetry_collection_xpack * fix errors in data plugins * fix errors in alerts * mute errors in index_management * fix task_manager errors * mute or fix lens errors * fix upgrade_assistant errors * fix or mute errors in index_lifecycle_management * fix discover errors * fix core tests * ML changes * fix core type errors * mute error in kbn-es-archiver * fix error in data plugin * fix error in telemetry plugin * fix error in discover * fix discover errors * fix errors in task_manager * fix security errors * fix wrong conflict resolution * address errors with upstream code * update deps to the last commit * remove outdated comments * fix core errors * fix errors after update * adding more expect errors to ML * pull the lastest changes * fix core errors * fix errors in infra plugin * fix errors in uptime plugin * fix errors in ml * fix errors in xpack telemetry * fix or mute errors in transform * fix errors in upgrade assistant * fix or mute fleet errors * start fixing apm errors * fix errors in osquery * fix telemetry tests * core cleanup * fix asMutableArray imports * cleanup * data_enhanced cleanup * cleanup events_log * cleaup * fix error in kbn-es-archiver * fix errors in kbn-es-archiver * fix errors in kbn-es-archiver * fix ES typings for Hit * fix SO * fix actions plugin * fix fleet * fix maps * fix stack_alerts * fix eslint problems * fix event_log unit tests * fix failures in data_enhanced tests * fix test failure in kbn-es-archiver * fix test failures in index_pattern_management * fixing ML test * remove outdated comment in kbn-es-archiver * fix error type in ml * fix eslint errors in osquery plugin * fix runtime error in infra plugin * revert changes to event_log cluser exist check * fix eslint error in osquery * fixing ML endpoint argument types * fx types * Update api-extractor docs * attempt fix for ese test * Fix lint error * Fix types for ts refs * Fix data_enhanced unit test * fix lens types * generate docs * Fix a number of type issues in monitoring and ml * fix triggers_actions_ui * Fix ILM functional test * Put search.d.ts typings back * fix data plugin * Update typings in typings/elasticsearch * Update snapshots * mute errors in task_manager * mute fleet errors * lens. remove unnecessary ts-expect-errors * fix errors in stack_alerts * mute errors in osquery * fix errors in security_solution * fix errors in lists * fix errors in cases * mute errors in search_examples * use KibanaClient to enforce promise-based API * fix errors in test/ folder * update comment * fix errors in x-pack/test folder * fix errors in ml plugin * fix optional fields in ml api_integartoon tests * fix another casting problem in ml tests * fix another ml test failure * fix fleet problem after conflict resolution * rollback changes in security_solution. trying to fix test * Update type for discover rows * uncomment runtime_mappings as its outdated * address comments from Wylie * remove eslint error due to any * mute error due to incompatibility * Apply suggestions from code review Co-authored-by: John Schulz <[email protected]> * fix type error in lens tests * Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.ts Co-authored-by: Alison Goryachev <[email protected]> * Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts Co-authored-by: Alison Goryachev <[email protected]> * update deps * fix errors in core types * fix errors for the new elastic/elasticsearch version * remove unused type * remove unnecessary manual type cast and put optional chaining back * ML: mute Datafeed is missing indices_options * Apply suggestions from code review Co-authored-by: Josh Dover <[email protected]> * use canary pacakge instead of git commit Co-authored-by: Josh Dover <[email protected]> Co-authored-by: Josh Dover <[email protected]> Co-authored-by: Gidi Meir Morris <[email protected]> Co-authored-by: Nathan Reese <[email protected]> Co-authored-by: Wylie Conlon <[email protected]> Co-authored-by: CJ Cenizal <[email protected]> Co-authored-by: Aleh Zasypkin <[email protected]> Co-authored-by: Dario Gieselaar <[email protected]> Co-authored-by: restrry <[email protected]> Co-authored-by: James Gowdy <[email protected]> Co-authored-by: John Schulz <[email protected]> Co-authored-by: Alison Goryachev <[email protected]> # Conflicts: # package.json # src/core/server/saved_objects/migrationsv2/integration_tests/migration.test.ts # src/plugins/data/server/autocomplete/value_suggestions_route.ts # src/plugins/discover/public/application/angular/doc_table/create_doc_table_react.tsx # x-pack/plugins/apm/server/lib/services/get_service_dependencies/get_destination_map.ts # x-pack/plugins/infra/server/lib/alerting/inventory_metric_threshold/evaluate_condition.ts # x-pack/plugins/security/server/authentication/api_keys/api_keys.ts # yarn.lock * fix file broken during conflict resolution * fix lint errors and test failure Co-authored-by: Tomas Della Vedova <[email protected]>

kibanamachine · 2021-03-25T16:08:46Z

💛 Build succeeded, but was flaky

continuous-integration/kibana-ci/pull-request
Commit: b65f197
Storybooks Preview
Flaky suites:
- xpack-securitySolutionCypressChrome

Test Failures

Kibana Pipeline / general / Closes and opens alerts.Closing alerts Closes and opens alerts

Link to Jenkins

Stack Trace

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

AssertionError: Timed out retrying after 60000ms: expected '<span.euiBadge.euiBadge--hollow.euiBadge--iconLeft>' to have text '197', but the text was '397'
    at Context.eval (http://localhost:6121/__cypress/tests?p=cypress/integration/detection_alerts/closing.spec.ts:20202:43)

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`discover`	388.9KB	388.9KB	+8.0B
`maps`	2.6MB	2.6MB	+46.0B
`ml`	6.0MB	6.0MB	+73.0B
`osquery`	763.5KB	763.9KB	+352.0B
`securitySolution`	7.2MB	7.2MB	+20.0B
`stackAlerts`	654.6KB	654.6KB	+58.0B
total			+557.0B

History

💚 Build #115103 succeeded b65f197
💛 Build #115060 was flaky f353538
💚 Build #115026 succeeded 815edf0
💚 Build #115018 succeeded 370c5db
💚 Build #114972 succeeded 80f730e

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @joshdover @mshustov

delvedor requested a review from a team November 19, 2020 15:09

mshustov self-assigned this Dec 2, 2020

pgayvallet changed the title ~~WIP: use the new type definitions~~ ES client : use the new type definitions Dec 4, 2020

joshdover self-assigned this Dec 17, 2020

joshdover added 3 commits December 28, 2020 14:17

Use client from branch

9fa2b78

Get type checking working in core

d2ae74f

Fix types in other plugins

5106b09

joshdover reviewed Dec 30, 2020

View reviewed changes

joshdover added 2 commits January 15, 2021 14:11

Merge remote-tracking branch 'upstream/master' into new-types-client

be34ba1

Update client types + remove type errors from core

6710f81

gmmorris mentioned this pull request Jan 19, 2021

Help core team use new Elasticsearch type definitions in Task Manager #88668

Closed

migrate Task Manager Elasticsearch typing from legacy library to clie…

491bd11

…nt library

use SortOrder instead o string in alerts

3f179b2

azasypkin mentioned this pull request Jan 21, 2021

Migrate authentication functionality to a new Elasticsearch client. #87094

Merged

joshdover added 2 commits January 29, 2021 12:01

Merge remote-tracking branch 'upstream/master' into new-types-client

a7fc04b

Update client types + fix core type issues

6d99c9c

Merge branch 'master' into pr/83808

b83b2d2

weltenwort approved these changes Mar 24, 2021

View reviewed changes

shahzad31 approved these changes Mar 24, 2021

View reviewed changes

FrankHassanabad approved these changes Mar 24, 2021

View reviewed changes

FrankHassanabad reviewed Mar 24, 2021

View reviewed changes

mshustov added 4 commits March 24, 2021 17:56

Merge branch 'master' into pr/83808

80f730e

Merge branch 'master' into pr/83808

370c5db

Merge branch 'master' into pr/83808

815edf0

Merge branch 'master' into pr/83808

f353538

igoristic approved these changes Mar 24, 2021

View reviewed changes

Merge branch 'master' into pr/83808

b65f197

mshustov enabled auto-merge (squash) March 25, 2021 06:59

mshustov merged commit 238791b into elastic:master Mar 25, 2021

mshustov mentioned this pull request Mar 25, 2021

[7.x] ES client : use the new type definitions (#83808) #95391

Merged

jgowdyelastic mentioned this pull request Mar 29, 2021

[ML] Type updates after esclient type update #95658

Merged

alisonelizabeth mentioned this pull request Mar 30, 2021

[Snapshot Restore] Migrate to new ES client #95499

Merged

peteharverson mentioned this pull request Mar 30, 2021

[Meta] Switch to the new Elasticsearch client #83910

Closed

74 tasks

This was referenced Mar 29, 2021

[Asset Management] Osquery agent picker #94541

Merged

[Security Solution] Remove graphQL host #94592

Merged

jgowdyelastic mentioned this pull request Jul 13, 2021

[ML] Fix calendar creation during model snapshot restore #105421

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ES client : use the new type definitions #83808

ES client : use the new type definitions #83808

delvedor commented Nov 19, 2020 •

edited by mshustov

Loading

joshdover commented Dec 3, 2020

kibanamachine commented Dec 3, 2020

mshustov commented Dec 7, 2020 •

edited

Loading

joshdover commented Dec 15, 2020

delvedor commented Dec 16, 2020

joshdover commented Dec 28, 2020

joshdover Dec 30, 2020

joshdover commented Jan 15, 2021 •

edited

Loading

mikecote commented Jan 15, 2021

gmmorris commented Jan 20, 2021 •

edited

Loading

joshdover commented Jan 29, 2021 •

edited by gmmorris

Loading

cjcenizal commented Jan 29, 2021

weltenwort left a comment

shahzad31 left a comment

FrankHassanabad left a comment

FrankHassanabad Mar 24, 2021

FrankHassanabad Mar 24, 2021

madirey Mar 24, 2021

FrankHassanabad Mar 24, 2021

FrankHassanabad Mar 24, 2021

FrankHassanabad Mar 24, 2021

mshustov Mar 24, 2021

dgieselaar Mar 24, 2021 •

edited

Loading

FrankHassanabad Mar 24, 2021

FrankHassanabad Mar 24, 2021

igoristic left a comment

kibanamachine commented Mar 25, 2021

Stack Trace

		}));

		// @ts-expect-error Anomaly is not assignable to EcsAnomaly

ES client : use the new type definitions #83808

ES client : use the new type definitions #83808

Conversation

delvedor commented Nov 19, 2020 • edited by mshustov Loading

Summary

Notes for reviewers

List of known problems

Plugin API changes

joshdover commented Dec 3, 2020

kibanamachine commented Dec 3, 2020

mshustov commented Dec 7, 2020 • edited Loading

List of incompatibilities

Side notes

joshdover commented Dec 15, 2020

delvedor commented Dec 16, 2020

joshdover commented Dec 28, 2020

Questions / Discussion

Major issues (should solve before merging existing PR)

Minor issues

Choose a reason for hiding this comment

joshdover commented Jan 15, 2021 • edited Loading

Feedback on the elastic/elasticsearch types

Major problems

Minor problems

mikecote commented Jan 15, 2021

gmmorris commented Jan 20, 2021 • edited Loading

joshdover commented Jan 29, 2021 • edited by gmmorris Loading

Feedback on client types for @delvedor

Plugins that we need help with

Steps

spaces

security

task_manager

event_log

saved_objects_tagging

lens

data_enhanced

maps

searchprofiler

cjcenizal commented Jan 29, 2021

weltenwort left a comment

Choose a reason for hiding this comment

shahzad31 left a comment

Choose a reason for hiding this comment

FrankHassanabad left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dgieselaar Mar 24, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

igoristic left a comment

Choose a reason for hiding this comment

kibanamachine commented Mar 25, 2021

💛 Build succeeded, but was flaky

Stack Trace

Metrics [docs]

Async chunks

History

delvedor commented Nov 19, 2020 •

edited by mshustov

Loading

mshustov commented Dec 7, 2020 •

edited

Loading

joshdover commented Jan 15, 2021 •

edited

Loading

gmmorris commented Jan 20, 2021 •

edited

Loading

joshdover commented Jan 29, 2021 •

edited by gmmorris

Loading

dgieselaar Mar 24, 2021 •

edited

Loading