[ML] New Platform server shim: update job validation routes #57644

alvarezmelissa87 · 2020-02-14T01:02:47Z

Summary

Related meta issue: #49743

Updates all job validation routes to use new platform router and replaces use of legacy elasticsearchPlugin in callWithInternalUserFactory with NP's context.core.elasticsearch.adminClient.callAsInternalUser

Also fixes type in migration doc.

Checklist

Delete any items that are not applicable to this PR.

Unit or functional tests were updated or added to match the most common scenarios

elasticmachine · 2020-02-14T01:02:50Z

Pinging @elastic/ml-ui (:ml)

pgayvallet

LGTM for platform, some nits and comments

EDIT: did not see this was a WIP. just some nits and comments then!

pgayvallet · 2020-02-14T07:14:19Z

x-pack/legacy/plugins/ml/server/models/bucket_span_estimator/bucket_span_estimator.d.ts

+  callAsCurrentUser: IScopedClusterClient['callAsCurrentUser'],
+  callAsInternalUser: IScopedClusterClient['callAsInternalUser'],


NIT: Should use APICaller interface instead (same in other .d.ts file)

import { APICaller } from 'src/core/server'; export function estimateBucketSpanFactory( callAsCurrentUser: APICaller, callAsInternalUser: APICaller,

Updated in f0bac33e423d5a570bfa2ba8157a9d1e677c6afb

pgayvallet · 2020-02-14T07:16:29Z

x-pack/legacy/plugins/ml/server/models/bucket_span_estimator/polled_data_checker.js

-export function polledDataCheckerFactory(callWithRequest) {
+export function polledDataCheckerFactory(callAsInternalUser) {


Is it normal that callWithRequest was renamed to callAsInternalUser? shouldn't it be callAsCurrentUser instead? Or did the behavior change? (same question on some other files)

Yep, you're right - I was thinking callAsInternalUser was getting passed in but it is callAsCurrentUser. I'll update it. Good catch 👌
Fixed in f0bac33e423d5a570bfa2ba8157a9d1e677c6afb

pgayvallet · 2020-02-14T07:20:47Z

x-pack/legacy/plugins/ml/server/models/data_visualizer/data_visualizer.ts

+import { CallAPIOptions } from 'kibana/server';
+import { IScopedClusterClient } from 'src/core/server';


NIT: kibana/server and src/core/server are the same alias. Use only one import line.

updated in f0bac33e423d5a570bfa2ba8157a9d1e677c6afb

pgayvallet · 2020-02-14T07:23:44Z

x-pack/legacy/plugins/ml/server/new_platform/job_validation_schema.ts

+import { anomalyDetectionJobSchema } from './anomaly_detectors_schema';
+
+export const estimateBucketSpanSchema = {
+  aggTypes: schema.arrayOf(schema.nullable(schema.string())),


Question: don't know the functional domain, but schema.arrayOf(schema.nullable(schema.string())) looks strange to me. can/should it really accept something like ['foo', null, 'bar', null] ?

Yep - so this was based on the BucketEstimatorData type interface (https://github.com/elastic/kibana/blob/master/x-pack/legacy/plugins/ml/public/application/services/ml_api_service/index.d.ts#L48) used to define the type of data sent to the bucket span estimator endpoint.

https://github.com/elastic/kibana/blob/master/x-pack/legacy/plugins/ml/public/application/jobs/new_job/pages/components/pick_fields_step/components/bucket_span_estimator/estimate_bucket_span.ts#L31

https://github.com/elastic/kibana/blob/master/x-pack/legacy/plugins/ml/public/application/jobs/new_job/pages/components/pick_fields_step/components/bucket_span_estimator/estimate_bucket_span.ts#L68

pgayvallet · 2020-02-14T07:29:44Z

x-pack/legacy/plugins/ml/server/routes/job_validation.ts

+      validate: {
+        body: schema.object(anomalyDetectionJobSchema),
+      },


(Already present before the PR, but) is there any reason the anomalyDetectionJobSchema is not already wrapped with a schema.object()? I see that some are and some aren't in x-pack/legacy/plugins/ml/server/new_platform/anomaly_detectors_schema.ts

pgayvallet · 2020-02-14T07:32:49Z

x-pack/legacy/plugins/ml/server/routes/job_validation.ts

+ * Routes for job validation
+ */
+export function jobValidationRoutes({ config, xpackMainPlugin, router }: RouteInitialization) {
+  function calculateModelMemoryLimit(context: RequestHandlerContext, payload: any) {


NIT: the handler calling that seems to have a proper body validation schema. Could probably get rid of the any signature here

type youWillNameThatBetterThanMePayload = TypeOf<typeof schema.object(modelMemoryLimitSchema)>

This I did not know about and it is beautiful. I thank you 🙏
Updated in f0bac33e423d5a570bfa2ba8157a9d1e677c6afb

walterra

LGTM

jgowdyelastic

LGTM

kibanamachine · 2020-02-18T16:19:15Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 6814dd8

History

💛 Build #26970 was flaky c60bb3c6666f8b614a18a0e04a7bdc1a5fa3a284
💔 Build #26936 failed f0bac33e423d5a570bfa2ba8157a9d1e677c6afb
💔 Build #26719 failed 4e3fb2f93d23208c6198c47c671f698c40879a5e

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…57644) * wip: convert jobValidation routes to NP * fix typo in np migration docs * replace legacy es plugin with callAsInternalUser from context * add schema info for job validation routes * update types and schema for job + cardinality validation * update bucketSpanEstimator test * update job validation schema

…57874) * wip: convert jobValidation routes to NP * fix typo in np migration docs * replace legacy es plugin with callAsInternalUser from context * add schema info for job validation routes * update types and schema for job + cardinality validation * update bucketSpanEstimator test * update job validation schema

alvarezmelissa87 added :ml v8.0.0 release_note:skip Skip the PR/issue when compiling release notes Feature:NP Migration v7.7.0 labels Feb 14, 2020

alvarezmelissa87 requested review from a team as code owners February 14, 2020 01:02

alvarezmelissa87 self-assigned this Feb 14, 2020

peteharverson mentioned this pull request Feb 14, 2020

[ML] New platform migration #49743

Closed

78 tasks

alvarezmelissa87 changed the title ~~[ML] New Platform server shim: update job validation routes~~ WIP [ML] New Platform server shim: update job validation routes Feb 14, 2020

pgayvallet approved these changes Feb 14, 2020

View reviewed changes

alvarezmelissa87 force-pushed the ml-new-platform-job-validation-routes branch from 4e3fb2f to 728fa12 Compare February 14, 2020 15:56

alvarezmelissa87 changed the title ~~WIP [ML] New Platform server shim: update job validation routes~~ [ML] New Platform server shim: update job validation routes Feb 14, 2020

alvarezmelissa87 requested review from walterra, jgowdyelastic and darnautov February 14, 2020 20:25

walterra approved these changes Feb 17, 2020

View reviewed changes

jgowdyelastic approved these changes Feb 17, 2020

View reviewed changes

alvarezmelissa87 added 7 commits February 18, 2020 09:40

wip: convert jobValidation routes to NP

ff80eb4

fix typo in np migration docs

121ba49

replace legacy es plugin with callAsInternalUser from context

c2075ed

add schema info for job validation routes

ef1c821

update types and schema for job + cardinality validation

a1eade0

update bucketSpanEstimator test

79998d5

update job validation schema

6814dd8

alvarezmelissa87 force-pushed the ml-new-platform-job-validation-routes branch from c60bb3c to 6814dd8 Compare February 18, 2020 14:40

alvarezmelissa87 merged commit 6685bd0 into elastic:master Feb 18, 2020

alvarezmelissa87 mentioned this pull request Feb 18, 2020

[7.x] [ML] New Platform server shim: update job validation routes (#57644) #57874

Merged

alvarezmelissa87 mentioned this pull request Feb 18, 2020

[ML] NP server migration: add job validation route endpoint names to docs #57877

Closed

alvarezmelissa87 deleted the ml-new-platform-job-validation-routes branch February 18, 2020 18:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] New Platform server shim: update job validation routes #57644

[ML] New Platform server shim: update job validation routes #57644

alvarezmelissa87 commented Feb 14, 2020 •

edited

Loading

elasticmachine commented Feb 14, 2020

pgayvallet left a comment •

edited

Loading

pgayvallet Feb 14, 2020

alvarezmelissa87 Feb 14, 2020

pgayvallet Feb 14, 2020

alvarezmelissa87 Feb 14, 2020 •

edited

Loading

pgayvallet Feb 14, 2020

alvarezmelissa87 Feb 14, 2020

pgayvallet Feb 14, 2020

alvarezmelissa87 Feb 14, 2020

pgayvallet Feb 14, 2020

pgayvallet Feb 14, 2020

alvarezmelissa87 Feb 14, 2020 •

edited

Loading

walterra left a comment

jgowdyelastic left a comment

kibanamachine commented Feb 18, 2020

		callAsCurrentUser: IScopedClusterClient['callAsCurrentUser'],
		callAsInternalUser: IScopedClusterClient['callAsInternalUser'],

		export function polledDataCheckerFactory(callWithRequest) {
		export function polledDataCheckerFactory(callAsInternalUser) {

		import { CallAPIOptions } from 'kibana/server';
		import { IScopedClusterClient } from 'src/core/server';

[ML] New Platform server shim: update job validation routes #57644

[ML] New Platform server shim: update job validation routes #57644

Conversation

alvarezmelissa87 commented Feb 14, 2020 • edited Loading

Summary

Checklist

elasticmachine commented Feb 14, 2020

pgayvallet left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alvarezmelissa87 Feb 14, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alvarezmelissa87 Feb 14, 2020 • edited Loading

Choose a reason for hiding this comment

walterra left a comment

Choose a reason for hiding this comment

jgowdyelastic left a comment

Choose a reason for hiding this comment

kibanamachine commented Feb 18, 2020

💚 Build Succeeded

History

alvarezmelissa87 commented Feb 14, 2020 •

edited

Loading

pgayvallet left a comment •

edited

Loading

alvarezmelissa87 Feb 14, 2020 •

edited

Loading

alvarezmelissa87 Feb 14, 2020 •

edited

Loading