[ML] Transforms: Adds date picker to transform wizard for data view with time fields. #149049

walterra · 2023-01-17T15:51:24Z

Summary

Adds a date picker to the transform wizard for data views with time fields.

The PR includes functionality to allow to set if the time range should get applied to the transform configuration or just be used for previews, however for now this is disabled with a hard coded feature flag. So the time range will only be applied to previews, not the final transform configuration.

Some code that referred to pivot but was meant to be used for both pivot/latest so was renamed to just transform*.
The form for latest was already in LatestFunctionForm, but the form for pivot was kept inline <StepDefineForm> component. This PR moves that inline code to PivotFunctionForm.
For functional tests, all date picker related code (previously already some existed to assert the date picker in Discover when displaying transform results) was moved to a new transform.datePicker service.

Checklist

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios
This was checked for breaking API changes and was labeled appropriately

…atest config

x-pack/test/functional/apps/transform/creation_index_pattern.ts

...s/transform/public/app/sections/create_transform/components/step_define/step_define_form.tsx

peteharverson · 2023-01-30T12:17:27Z

...ransform/public/app/sections/create_transform/components/step_define/pivot_function_form.tsx

+                      'The advanced editor allows you to edit the pivot configuration of the transform.',
+                  })}{' '}
+                  <EuiLink href={esTransformPivot} target="_blank">
+                    {i18n.translate('xpack.transform.stepDefineForm.advancedEditorHelpTextLink', {


Nit - is it worth getting pivot in the i18n key here, in case there is text for a help link in the Latest wizard too?

peteharverson · 2023-01-30T15:15:31Z

...ransform/public/app/sections/create_transform/components/step_define/pivot_function_form.tsx

+                onClick={applyPivotChangesHandler}
+                disabled={!isAdvancedPivotEditorApplyButtonEnabled}
+              >
+                {i18n.translate('xpack.transform.stepDefineForm.advancedEditorApplyButtonText', {


As above, is it worth adding pivot in the i18n key here?

There's no reference to latest in the form for latest either so I'd prefer to keep the ids as is to keep the existing translations.

peteharverson · 2023-01-30T17:54:04Z

...ransform/public/app/sections/create_transform/components/step_define/step_define_summary.tsx

@@ -94,6 +98,18 @@ export const StepDefineSummary: FC<Props> = ({
            >
              <span>{searchItems.dataView.getIndexPattern()}</span>
            </EuiFormRow>
+            {isDatePickerApplyEnabled && timeRangeMs && (
+              <EuiFormRow


Can this time picker be made not to go full width rather than taking up the full form row?

At the moment it's set up to get the full width like the search bar below. Suggest to revisit this once the follow ups (frozen option / full time range) are added too. I added a note in the meta issue here: #146187

peteharverson · 2023-01-30T17:57:09Z

Could be an idea to suggest searching over a different time range as part of this info message:

walterra · 2023-01-31T13:42:54Z

@peteharverson The callout for no source documents is rendered by the data grid component itself, it's not aware of outer individual options (like search or date picker), that's why it says the query is not too restrictive as the more technical term that includes all of that.

walterra · 2023-01-31T13:55:40Z

Added an info tooltip to the time range label in 427d418.

walterra · 2023-01-31T13:59:07Z

@peteharverson @qn895 Thanks for your comments, addressed and answered them, please have another look.

qn895 · 2023-01-31T16:27:49Z

x-pack/test/functional/apps/transform/index.ts

@@ -36,16 +36,16 @@ export default function ({ getService, loadTestFile }: FtrProviderContext) {
      await transform.testResources.resetKibanaTimeZone();
    });

-    loadTestFile(require.resolve('./permissions'));
+    // loadTestFile(require.resolve('./permissions'));


Don't forget to re-enable these tests back before merging

Thanks for catching! Fixed it in c64f1b2. (I think with the latest merge of main into this branch that whole code part was refactored away though, it's now organized differently)

qn895 · 2023-01-31T17:04:21Z

LGTM 🎉

peteharverson · 2023-01-31T17:47:54Z

...s/transform/public/app/sections/create_transform/components/step_define/step_define_form.tsx

+                    'xpack.transform.stepDefineForm.datePickerIconTipContent',
+                    {
+                      defaultMessage:
+                        'The time range will be applied to previews only, it will not be part of the final transform configuration.',


Suggest editing to The time range will be applied to previews only and will not be part of the final transform configuration

Fixed in 2fcf33a.

peteharverson · 2023-01-31T18:03:29Z

x-pack/plugins/transform/public/app/common/request.ts

@@ -47,9 +48,15 @@ export interface SimpleQuery {
  };
 }

-export type PivotQuery = SimpleQuery | SavedSearchQuery;
+export interface FilterBasedSimpleQuery {
+  bool: {


Is this a change from the previous 'empty' query that was applied "match_all": {} ? This is causing a side effect when cloning a transform, in that the search field shows the 'advanced' JSON version

rather than the preferable empty KQL bar.

Added a note for a follow up in #146187.

PR is up to fix this issue here: #151665

kibana-ci · 2023-01-31T19:43:11Z

💚 Build Succeeded

Buildkite Build
Commit: 2fcf33a

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`transform`	274	296	+22

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`transform`	348.1KB	360.4KB	+12.3KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`transform`	15.4KB	15.6KB	+157.0B

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`transform`	27	31	+4

Total ESLint disabled count

id	before	after	diff
`transform`	30	34	+4

History

💚 Build #104301 succeeded 7f9d8aa
💚 Build #104198 succeeded 427d418
💚 Build #103320 succeeded f20e96d
💚 Build #103284 succeeded b3a608e1ab30e854866660d941c6e4e2401bd6d1
💚 Build #102872 succeeded 674b55201844f762ee110e8884e01a8fbc6e2032
💚 Build #102722 succeeded cd3077b9f40e4e26c42827d1582586f356dce0e4

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @walterra

…ith time fields. (elastic#149049) Adds a date picker to the transform wizard for data views with time fields. The time range will be applied to previews only.

walterra self-assigned this Jan 17, 2023

walterra force-pushed the 146187a-transform-date-picker branch 8 times, most recently from 5f0c069 to a0b7f86 Compare January 25, 2023 09:01

This was referenced Jan 25, 2023

[ML] Transforms: Wizard performance improvements. (8.7) #146187

Closed

[ML] Fix race condition when updating data grid row count #149518

Merged

walterra force-pushed the 146187a-transform-date-picker branch 2 times, most recently from 674b552 to c01a95d Compare January 27, 2023 10:08

walterra added 17 commits January 27, 2023 12:58

pass through width prop for DatePickerWrapper

8aa39ac

add date picker to transform wizard

cbb394a

fix tslint config

013587e

fix i18n ids

30e3387

add UrlStateProvider

be75626

fix default state

955f704

fix to check mounted state in use_search_items

d27441e

update useDatePicker to return timestamp based range.

ec3c666

apply date picker selection to source document data grid

b6bf9a2

fix tslint config

53aa557

apply date picker selection to transform preview

b553da6

add check for default queries to consider default filter based query

b995ed2

move pivot config form part to component

3b2a524

rename pivot references to transformConfig since they also refer to l…

a8e8e6c

…atest config

expose time range in define step summary

78e1ef5

apply time range in creation step

0bfdbf9

fix jest tests

c1332a2

qn895 reviewed Jan 27, 2023

View reviewed changes

x-pack/test/functional/apps/transform/creation_index_pattern.ts Outdated Show resolved Hide resolved

qn895 reviewed Jan 27, 2023

View reviewed changes

...s/transform/public/app/sections/create_transform/components/step_define/step_define_form.tsx Show resolved Hide resolved

peteharverson reviewed Jan 30, 2023

View reviewed changes

qn895 added a commit to qn895/kibana that referenced this pull request Jan 30, 2023

Merge elastic#149049 into branch

477f674

walterra added 6 commits January 31, 2023 13:51

Merge branch 'main' into 146187a-transform-date-picker

736d1e2

use timefilterServiceMock instead of local mock

3de83d1

refactor assertSuperDatePickerToggleQuickMenuButtonExists

da8373a

assert after date picker quick select apply button was clicked

30f7f7f

make openSuperDatePicker part of quickSelect

6610cd0

fix log text

09f1e77

add info tooltip to time range label

427d418

qn895 reviewed Jan 31, 2023

View reviewed changes

walterra added 2 commits January 31, 2023 18:00

reenable accidently disabled tests.

c64f1b2

Merge branch 'main' into 146187a-transform-date-picker

7f9d8aa

qn895 approved these changes Jan 31, 2023

View reviewed changes

peteharverson reviewed Jan 31, 2023

View reviewed changes

tweak tooltip text

2fcf33a

walterra merged commit 0085aae into elastic:main Feb 1, 2023

kibanamachine added the backport:skip This commit does not require backporting label Feb 1, 2023

walterra deleted the 146187a-transform-date-picker branch February 1, 2023 08:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] Transforms: Adds date picker to transform wizard for data view with time fields. #149049

[ML] Transforms: Adds date picker to transform wizard for data view with time fields. #149049

walterra commented Jan 17, 2023 •

edited

Loading

peteharverson Jan 30, 2023

peteharverson Jan 30, 2023

walterra Jan 31, 2023

peteharverson Jan 30, 2023

walterra Jan 31, 2023

peteharverson commented Jan 30, 2023

walterra commented Jan 31, 2023

walterra commented Jan 31, 2023

walterra commented Jan 31, 2023

qn895 Jan 31, 2023

walterra Jan 31, 2023

qn895 commented Jan 31, 2023

peteharverson Jan 31, 2023

walterra Jan 31, 2023

peteharverson Jan 31, 2023

walterra Feb 1, 2023

walterra Feb 21, 2023

kibana-ci commented Jan 31, 2023

ESLint disabled line counts

Total ESLint disabled count

[ML] Transforms: Adds date picker to transform wizard for data view with time fields. #149049

[ML] Transforms: Adds date picker to transform wizard for data view with time fields. #149049

Conversation

walterra commented Jan 17, 2023 • edited Loading

Summary

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

peteharverson commented Jan 30, 2023

walterra commented Jan 31, 2023

walterra commented Jan 31, 2023

walterra commented Jan 31, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qn895 commented Jan 31, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibana-ci commented Jan 31, 2023

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Page load bundle

ESLint disabled line counts

Total ESLint disabled count

History

walterra commented Jan 17, 2023 •

edited

Loading