[Controls] Dashboard Integration #115991

ThomThomson · 2021-10-21T17:34:25Z

Summary

Integrates Phase 1 of the Controls project with dashboard. This includes the following broad tasks:

Place control group embeddable on the dashboard underneath the edit toolbar.
Output from Controls is correctly applied to dashboard embeddables
- Changing an options list option generates and publishes a filter to its output.
- Control group listens to any filter change in its children and publishes all filters to its output.
- Dashboard listens to control group output for filter changes, applies the filters to its children and refreshes.
- All index patterns are correctly communicated from the child control to the dashboard for the filter manager.
Input from Dashboard embeddable is applied to the Control Group
- filters, query, timeRange, lastReloadRequestTime and viewMode are passed from the dashboard to Control Group
- Control group passes this input down to its children
Control group changes are correctly stored in dashboard state
- unsaved changes are backed up in session storage
- Saving a dashboard saves the control group input (if valid)
- data view references are correctly extracted from / injected into the control group

How to enable

You can find the dashboard controls advanced setting under the Presentation Labs header in Stack Management -> Advanced Settings.

Feature GIFs

Control selections apply to charts

Filters Query & Timerange apply to Controls

Edit and Save

Missing pieces for release

There are still a few missing pieces before this project is release-ready:

Functional testing

functional tests will be added soon, the tests which are needed to cover this project are tracked in #115816

Final design review

Using this PR as a basis for the look and feel of the Controls project integrated on dashboard, the design team will do a final once over to finalize the design.

Some broad upcoming design ideas include:

Fix spacing and alignment of the control group and individual controls on dahsboards. Having controls on a dashboard should be aesthetically pleasing in view mode.
Remove the background color and framing from the control group. Controls should appear to be floating on dashboards like other panels.
Fix coloration of the control group background while dragging to reorder.
Some final fixes to the design of the create / edit window for controls.

Important Phase 2 features

There are some missing features in this version of the Controls project that are not technically necessary for the first release, but can cause some UX problems and confusion.

Ability to ignore parent filtering

By default, the time range, filters, and query settings from the dashboard are passed down to the control group, and the individual controls. Making changes to the filters of the dashboard will force a refresh of the controls. This setting should be made configurable for the dashboard author, in case they want to allow their users to use both types of control. The experimental Input Controls have this feature, but only for the time range.

Automatic Validation

When a user has a selection in their control, and the context of the dashboard changes from for instance a change in the filters, query, time range, or refresh, the control will re-query, but it will not validate the user's selections. This can result in inconsistencies, as the current selection will not be listed in the available options.

Additionally, when filtering by the same field in the dashboard filters, and in a control, the control selections can sometimes override the filter selections. This is especially apparent when the user has selected two options in the control, then adds a filter to the dashboard narrowing the selections down to one. The control selections will override the dashboard selections and the dashboard will show results for both.

Automatic validation will take care of both of these issues. Whenever the control fetches its available options, it will also send a cardinality query for each of its selections. If the cardinality query returns 0 for any term, that term will be considered errorSelected instead of selected, and will be retained in the control, but will not be applied. This feature is necessary for hierarchical chaining.

Missing Range Slider

The experimental Input Controls have a range slider control option which is completely missing in this iteration of the Controls project. Users may expect to be able to add a range slider control.

Missing non-search option for Options List

In the experimental Input Controls, users can add an options list which does not use typeahead functionality. Instead, the author defines a max number of terms to fetch. This option is completely missing from this iteration of the Controls project. Users seem to use this option often, and may be surprised that it is not yet included.

Options List Fields

due to our usage of the new autocomplete API, certain field types are not supported by the Options List Control. Currently the options list control only supports String and Boolean field types. The experimental Input Controls support selection of Date and Number fields as well.

… behind lab

…x jest tests

…boardIntegration

… creating new Control

elasticmachine · 2021-10-23T20:05:17Z

Pinging @elastic/kibana-presentation (Team:Presentation)

…dle size below original value

…same pattern in control group and panel

…boardIntegration

andreadelrio

LGTM from a design perspective.

Some other pending design related issues are #112902, #112899 and #116189.

crob611 · 2021-10-25T23:42:06Z

Here's a recording where I get a single control into a weird state.

Brand new dashboard.

Add Control.

Sampledata ecommerce/customer_gender

Select Male

Save

Male is double selected.

Switch to View Mode prompts about unsaved changes.

Screen.Recording.2021-10-25.at.7.38.06.PM.mov

andreadelrio · 2021-10-26T18:34:57Z

@ThomThomson A couple more things I've found, one design related and the other one isn't.

We should expand the width of the data view picker popover to show more of the data views names. Also need to update the label to say "Data view"

Seems like the search for fields is case sensitive, this isn't the behavior in Discover/Lens. In the example below I'm trying to search for OriginCityName but it shows no results when typing origin. If I search "Orig" it does show up.

…le is destroyed. Do not combine reducers when the reducer already exists. Destroy and unmount control group

ThomThomson · 2021-10-26T21:29:40Z

@crob611, I was able to fix the issue that you noticed. It was caused by combineReducers getting called when the dashboard is re-created after saving, and redux attempting to port over any state which existed under the embeddable's key in the store. Now when embeddables get destroyed they remove all of their state from the redux store.

@andreadelrio, since searching through fields is done clientside, making the field search case insensitive should be pretty straightforward! Also, we definitely should make the data views picker wider, these are perfect candidates for the upcoming design review PR! I can start on some of these once this is merged.

crob611

Have looked over the code several times, and it looks good (just a couple of nits/comments below)

Tested functionality and it works great.

crob611 · 2021-10-25T17:57:28Z

src/plugins/dashboard/public/application/actions/add_to_library_action.test.tsx

@@ -7,7 +7,7 @@
 */

 import { AddToLibraryAction } from '.';
-import { DashboardContainer } from '../embeddable';
+import { DashboardContainer } from '../embeddable/dashboard_container';


What's the reason for only exporting the DashboardContainer type and not the entire class, thus having to change all of these imports to a deeper path?

This is the secret to how I made the initial dashboard plugin bundle less than 100kb. The dashboard embeddable contains a whole bunch of code and resources, so I imported it async in the factory.

That alone cut the bundle size from 230kb to 80kb.

The only places that used the dashboard container class itself without going through the factory were the actions/tests so I just updated them all to import from elsewhere.

crob611 · 2021-10-25T19:26:38Z

src/plugins/dashboard/public/application/lib/sync_dashboard_index_patterns.ts

@@ -77,6 +81,14 @@ export const syncDashboardIndexPatterns = ({
    })
  );

+  if (dashboardContainer.controlGroup) {


Nit: Could this be cleaned up a bit so you're not duping the .pipe.subscribe?

Maybe always make an array of container output and add the control group if it's there and always do combine?

Ah good call! I've cleaned it up using your suggestion

crob611 · 2021-10-25T20:48:06Z

src/plugins/presentation_util/public/components/field_picker/field_picker.tsx

-                      className="presFieldPicker__fieldButton"
-                      onClick={() => setSelectedField(f)}
-                      isActive={f.name === selectedField?.name}
+                      className={classNames('presFieldPicker__fieldButton', {


Can we use emotion for classNames instead of classnames?

I can switch this out throughout the controls section in a follow-up - just gotta look into how it works, as it seems not to be 1 to 1.

…boardIntegration

kibanamachine · 2021-10-27T07:51:21Z

💛 Build succeeded, but was flaky

Test Failures

[job] [logs] OSS CI Group #2 / embeddable explorer dashboard container pie charts

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`dashboard`	219	305	+86
`presentationUtil`	124	330	+206
total			+292

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`dashboard`	134	140	+6
`presentationUtil`	160	243	+83
total			+89

Any counts in public APIs

Total count of every any typed public API. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats any for more detailed information.

id	before	after	diff
`presentationUtil`	3	4	+1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`dashboard`	143.3KB	289.5KB	+146.2KB
`presentationUtil`	43.1KB	234.4KB	+191.2KB
total			+337.4KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`dashboard`	10	12	+2
`presentationUtil`	6	12	+6
total			+8

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`dashboard`	168.4KB	65.5KB	-102.9KB
`presentationUtil`	44.8KB	68.0KB	+23.2KB
total			-79.7KB

Saved Objects .kibana field count

Every field in each saved object type adds overhead to Elasticsearch. Kibana needs to keep the total field count below Elasticsearch's default limit of 1000 fields. Only specify field mappings for the fields you wish to search on or query. See https://www.elastic.co/guide/en/kibana/master/development-plugin-saved-objects.html#_mappings

id	before	after	diff
`dashboard`	17	20	+3

Unknown metric groups

API count

id	before	after	diff
`dashboard`	147	153	+6
`presentationUtil`	187	271	+84
total			+90

async chunk count

id	before	after	diff
`dashboard`	1	3	+2
`presentationUtil`	6	9	+3
total			+5

References to deprecated APIs

id	before	after	diff
`presentationUtil`	74	2	-72

History

💔 Build #1923 failed 2851dfe
💛 Build #1628 was flaky 497c057
💛 Build #1237 was flaky 6a22451
💚 Build #1213 succeeded 3682356
💔 Build #1212 failed 5ba1815608da1afbe8cb42cddc4111e761ff4e31

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…-migrate-away-from-injected-css-js * 'master' of github.com:elastic/kibana: (61 commits) [ML] Nodes overview for the Model Management page (elastic#116361) [Uptime] Uptime index config using kibana.yml (elastic#115775) [Controls] Dashboard Integration (elastic#115991) skip flaky suite (elastic#104260) Include Files in GitHub UI (elastic#115956) skip flaky suite (elastic#116060) [Canvas] By-Value Embeddables (elastic#113827) Skip failing test (elastic#115366) [Osquery] Fix live query search doesn't return relevant results for agents (elastic#116332) [Integrations] Added link in old Add Data description and fixed alignment in cards (elastic#116213) [Actions] Extended ActionTypeRegistry with connector validation to validate config with secrets (elastic#116079) skip flaky suite (elastic#109329) Grant access to machine learning features when base privileges are used (elastic#115444) Skipping failing test (elastic#84957) [RAC][Security Solution] Adds migration to new SecuritySolution rule types (elastic#112113) skip flaky suite (elastic#115366) [Fleet] Marking API spec as experimental (elastic#116331) [Docs] Cleaning up the versions in the upgrade paths. Closes elastic#116223 (elastic#116228) [Reporting] Suppress debug logs in the mock logger (elastic#116012) [Metrics UI] Clear threshold alert groups state when filterQuery changes (elastic#116205) ... # Conflicts: # src/plugins/dashboard/public/application/embeddable/dashboard_container.tsx # src/plugins/dashboard/public/types.ts

This reverts commit 0b2dcdf.

ThomThomson added 3 commits October 20, 2021 22:46

Create filters when controls change. Integrated controls on dashboard…

e8dd9e3

… behind lab

Inject and Extract, tracking of index pattern changes in control group

5570442

Fix types

ab3768d

ThomThomson mentioned this pull request Oct 21, 2021

[Controls] TEMPORARY Integrate on dashboard #115477

Closed

9 tasks

ThomThomson added 4 commits October 21, 2021 22:55

Reduce bundle size for presentation util, update telemetry schema, fi…

b2479ea

…x jest tests

Merge branch 'master' of github.com:elastic/kibana into controls/dash…

99415d5

…boardIntegration

Fix type

53eb2bc

implemented new field picker and data view picker. Default title when…

3682356

… creating new Control

ThomThomson force-pushed the controls/dashboardIntegration branch from 5ba1815 to 3682356 Compare October 22, 2021 17:28

ThomThomson marked this pull request as ready for review October 23, 2021 20:05

ThomThomson requested review from a team as code owners October 23, 2021 20:05

remove accidental change to container.ts. Lower dashboard initial bun…

23542eb

…dle size below original value

botelastic bot added the Feature:Embedding Embedding content via iFrame label Oct 24, 2021

ThomThomson added 2 commits October 24, 2021 12:04

Fix potential duplication of dashboard index patterns when using the …

c6732dc

…same pattern in control group and panel

Merge branch 'master' of github.com:elastic/kibana into controls/dash…

43a9442

…boardIntegration

ThomThomson removed the request for review from a team October 24, 2021 16:11

undo change to container.ts

6a22451

ThomThomson removed the request for review from a team October 24, 2021 16:14

pgayvallet approved these changes Oct 25, 2021

View reviewed changes

ThomThomson mentioned this pull request Oct 25, 2021

[Controls] [Meta] Engineering Phase 1 Implementation Roadmap #99993

Closed

21 tasks

andreadelrio approved these changes Oct 25, 2021

View reviewed changes

This was referenced Oct 25, 2021

[Controls] Accessibility #112899

Open

[Controls] Improve drag and drop between rows #112902

Closed

[Controls] Use EuiSelectable for field picker #116189

Closed

communicate default index pattern when editor loaded

497c057

use consistent IDs for control group. Clear redux state when embeddab…

2851dfe

…le is destroyed. Do not combine reducers when the reducer already exists. Destroy and unmount control group

crob611 approved these changes Oct 26, 2021

View reviewed changes

ThomThomson added 2 commits October 26, 2021 21:09

simplified inline editor, cleaned up index pattern sync

fc2149a

Merge branch 'master' of github.com:elastic/kibana into controls/dash…

44cba5c

…boardIntegration

ThomThomson merged commit 0b2dcdf into elastic:master Oct 27, 2021

ThomThomson mentioned this pull request Oct 27, 2021

Lowered bundle size limits for dashboard and presentationUtil #116478

Merged

9 tasks

brianseeders added a commit to brianseeders/kibana that referenced this pull request Oct 27, 2021

Revert "[Controls] Dashboard Integration (elastic#115991)"

523760d

This reverts commit 0b2dcdf.

hendry-lim mentioned this pull request Dec 28, 2021

FEATURE REQUEST: Add KQL filter on dropdown selection tool #122036

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Controls] Dashboard Integration #115991

[Controls] Dashboard Integration #115991

ThomThomson commented Oct 21, 2021 •

edited

Loading

elasticmachine commented Oct 23, 2021

andreadelrio left a comment •

edited

Loading

crob611 commented Oct 25, 2021

andreadelrio commented Oct 26, 2021 •

edited

Loading

ThomThomson commented Oct 26, 2021

crob611 left a comment

crob611 Oct 25, 2021

ThomThomson Oct 26, 2021 •

edited

Loading

crob611 Oct 25, 2021

ThomThomson Oct 27, 2021

crob611 Oct 25, 2021

ThomThomson Oct 26, 2021

kibanamachine commented Oct 27, 2021

API count

async chunk count

References to deprecated APIs

[Controls] Dashboard Integration #115991

[Controls] Dashboard Integration #115991

Conversation

ThomThomson commented Oct 21, 2021 • edited Loading

Summary

How to enable

Feature GIFs

Missing pieces for release

Functional testing

Final design review

Important Phase 2 features

Ability to ignore parent filtering

Automatic Validation

Missing Range Slider

Missing non-search option for Options List

Options List Fields

elasticmachine commented Oct 23, 2021

andreadelrio left a comment • edited Loading

Choose a reason for hiding this comment

crob611 commented Oct 25, 2021

andreadelrio commented Oct 26, 2021 • edited Loading

ThomThomson commented Oct 26, 2021

crob611 left a comment

Choose a reason for hiding this comment

crob611 Oct 25, 2021

Choose a reason for hiding this comment

ThomThomson Oct 26, 2021 • edited Loading

Choose a reason for hiding this comment

crob611 Oct 25, 2021

Choose a reason for hiding this comment

ThomThomson Oct 27, 2021

Choose a reason for hiding this comment

crob611 Oct 25, 2021

Choose a reason for hiding this comment

ThomThomson Oct 26, 2021

Choose a reason for hiding this comment

kibanamachine commented Oct 27, 2021

💛 Build succeeded, but was flaky

Test Failures

Metrics [docs]

Module Count

Public APIs missing comments

Any counts in public APIs

Async chunks

Public APIs missing exports

Page load bundle

Saved Objects .kibana field count

API count

async chunk count

References to deprecated APIs

History

ThomThomson commented Oct 21, 2021 •

edited

Loading

andreadelrio left a comment •

edited

Loading

andreadelrio commented Oct 26, 2021 •

edited

Loading

ThomThomson Oct 26, 2021 •

edited

Loading