[Lens] Query based annotations #138753

dej611 · 2022-08-12T15:57:41Z

Summary

This PR contains:

integrating fetching query based annotations with Lens
the configuration side of the new query based annotation feature in Lens.

Query based annotations in the chart

Config panel

The Layer UI has changed to contain the dataView picker:

Opening an annotation event will present a button group to switch to the Query based type:

Once selected some additional features will be enabled in the panel to query/enrich the annotation/visualization:

On dataview switch all fields selected are validated against the new dataview and errors are shown to the user in the main workspace:

The query annotations are not fetched yet as it requires some more work to collect all the annotations expressions in a single go rather than per layer.

Tasks

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
This was checked for cross-browser compatibility

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk	Probability	Severity	Mitigation/Notes
Multiple Spaces—unexpected behavior in non-default Kibana Space.	Low	High	Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces.
Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks.	High	Low	Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure.
Code should gracefully handle cases when feature X or plugin Y are disabled.	Medium	High	Unit tests will verify that any feature flag or plugin combination still results in our service operational.
See more potential risk examples

For maintainers

This was checked for breaking API changes and was labeled appropriately

…e/129299

mbondyra

Tested and reviewed (and of course, I put some lines here so hopefully we'll get some other pairs of eyes on it )

flash1293 · 2022-09-08T08:15:30Z

Both config UI and chart crash with a weird error if the data view is not available:

Can also be fixed in a follow-up

flash1293 · 2022-09-08T08:48:29Z

...public/visualizations/xy/xy_config_panel/annotations_config_panel/range_annotation_panel.tsx

+          showTimeSelect
+          selected={value}
+          onChange={onChange}
+          dateFormat="MMM D, YYYY @ HH:mm:ss.SSS"


It would be nice to use the dateFormat advanced setting here instead of hardcoding a format

Opened a follow-up issue for that

flash1293 · 2022-09-08T09:38:26Z

This works really well! The only two things we should still fix on this PR before merging is the migration issue (Marco already working on it) and the "missing data view" issue. @dej611 could you ping me once that works?

dej611 · 2022-09-08T11:21:03Z

Both issues have been resolves with the same fix, which fallbacks to the first dataView id found in references if none is available for the annotation layer.

flash1293

LGTM - no further bugs found

kibana-ci · 2022-09-08T16:22:21Z

💚 Build Succeeded

Buildkite Build
Commit: b61cb1d

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`expressionXY`	144	145	+1
`lens`	898	905	+7
total			+8

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`data`	2441	2444	+3
`eventAnnotation`	163	170	+7
`lens`	532	550	+18
total			+28

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`eventAnnotation`	1.5KB	2.1KB	+542.0B
`expressionXY`	116.0KB	116.9KB	+977.0B
`lens`	1.2MB	1.2MB	+15.2KB
total			+16.7KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`eventAnnotation`	5	3	-2
`expressionXY`	11	9	-2
total			-4

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`data`	430.6KB	430.7KB	+122.0B
`eventAnnotation`	17.7KB	18.7KB	+1006.0B
`expressionXY`	35.1KB	35.6KB	+477.0B
total			+1.6KB

Unknown metric groups

API count

id	before	after	diff
`data`	3131	3144	+13
`eventAnnotation`	163	170	+7
`expressionXY`	152	153	+1
`lens`	618	639	+21
total			+42

History

💛 Build #70375 was flaky 1af226a
💚 Build #70308 succeeded a4c9edc
💛 Build #70182 was flaky 77b5598
💔 Build #70157 failed 64535f2
💔 Build #70114 failed 7ceea02
💔 Build #70090 failed f3f2be6

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* ⚗️ Initial code for query based annotations * 🐛 Solved more conflicts * ⚗️ More scaffolding layout * ⚗️ Initial indexpatetrn move into frame * ⚗️ Make field selection work * 🚧 Fixed almost all dataViews occurrencies, but changeIndexPattern * 🚧 More work on change index pattern * Move lens dataViews state into main state * 🔥 Remove some old cruft from the code * 🐛 Fix dataViews layer change * 🐛 Fix datasourceLayers refs * 🔥 Remove more old cruft * 🐛 Fix bug when loading SO * 🐛 Fix initial existence flag * 🏷️ Fix type issues * 🏷️ Fix types and tests * 🏷️ Fix types issues * ✅ Fix more tests * ✅ Fix with new dataViews structure * ✅ Fix more test mocks * ✅ More tests fixed * 🔥 Removed unused prop * ✅ Down to single broken test suite * 🏷️ Fix type issue * 👌 Integrate selector feedback * ✅ Fix remaining unit tests * 🏷️ fix type issues * 🐛 Fix bug when creating dataview in place * ✨ Update with latest dataview state + fix dataviews picker for annotations * 🐛 Fix edit + remove field flow * Update x-pack/plugins/lens/public/visualizations/xy/types.ts * 📸 Fix snapshot * 🐛 Fix the dataViews switch bug * 🔥 remove old cruft * ♻️ Revert removal from dataviews state branch * ♻️ Load all at once * 🔧 working on persistent state + fix new layer bug * 🔥 remove unused stuff * 🏷️ Fix some typings * 🔧 Fix expression issue * ✅ Add service unit tests * 👌 Integrated feedback * ✨ Add migration code for manual annotations * 🏷️ Fix type issue * ✅ Add some other unit test * 🏷️ Fix more type issues * 🐛 Fix importing issue * ♻️ Make range default color dependant on opint one * 🐛 Fix duplicate fields selection in tooltip section * ✅ Add more unit tests * ✅ Fix broken test * 🏷️ Mute ts error for now * ✅ Fix tests * 🔥 Reduce plugin weight * 🐛 prevent layout shift on panel open * 🐛 Fix extract + inject visualization references * 🏷️ fix type issues * ✨ Add dataview reference migration for annotations * 🔧 Add migration to embedadble * 🏷️ Fix type export * 🐛 Fix more conflicts with main * ✅ Fix tests * 🏷️ Make textField optional * ♻️ Refactor query input to be a shared component * 🐛 Fix missing import * 🐛 fix more import issues * 🔥 remove duplicate code * 🐛 Fix dataView switch bug * 🏷️ Fix type issue * annotations with fetching_event_annotations * portal for kql input fix * timeField goes for default if not filled * limit changes * handle ad-hoc data view references correctly * fix types * adjust tests to datatable format (remove isHidden tests as it's filtered before) * small refactors * fix loading on dashboard * empty is invalid (?) tbd * new tooltip * emptyDatatable * ♻️ Flip field + query inputs * 🏷️ Fix type issue * ✨ Add field validation for text and tooltip fields * tooltip for single annotation * fix tests * fix for non--timefilter dataview * fix annotations test - the cause was that we now don't display label for aggregated annotations ever * use eui elements * newline problem solved * ✅ Add more error tests * 👌 Rename migration state version type * fix types for expression chart * 🐛 Fix i18n id * 🏷️ Fix type issue * fix hidden all annotations * ✅ Fix tests after ishidden removal * 🐛 Revert references migration to an in app solution Co-authored-by: Joe Reuter <[email protected]> Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Marta Bondyra <[email protected]> Co-authored-by: Marta Bondyra <[email protected]>

dej611 added 30 commits June 20, 2022 18:40

⚗️ Initial code for query based annotations

3cb5bea

Merge remote-tracking branch 'upstream/main' into feature/129299

59b7c68

Merge remote-tracking branch 'upstream/main' into feature/129299

db65f76

🐛 Solved more conflicts

d1d62dd

⚗️ More scaffolding layout

53a618b

⚗️ Initial indexpatetrn move into frame

776380a

Merge remote-tracking branch 'upstream/main' into feature/129299

f1d8049

⚗️ Make field selection work

00ad767

Merge remote-tracking branch 'upstream/main' into feature/129299

802a8f4

🚧 Fixed almost all dataViews occurrencies, but changeIndexPattern

b1d1832

Merge remote-tracking branch 'upstream/main' into feature/129299

c891391

🚧 More work on change index pattern

0f6dbe6

Merge remote-tracking branch 'upstream/main' into feature/129299

25fd964

Move lens dataViews state into main state

57d814d

🔥 Remove some old cruft from the code

751c2a2

🐛 Fix dataViews layer change

c7a9ce8

🐛 Fix datasourceLayers refs

79566af

🔥 Remove more old cruft

4822b36

🐛 Fix bug when loading SO

220aa39

🐛 Fix initial existence flag

759f19a

🏷️ Fix type issues

7f590da

🏷️ Fix types and tests

12b91fc

🏷️ Fix types issues

2ee6f6d

✅ Fix more tests

ce7bddd

✅ Fix with new dataViews structure

80647be

✅ Fix more test mocks

63bf020

✅ More tests fixed

3a7f9fb

🔥 Removed unused prop

2fc1f0f

✅ Down to single broken test suite

e9abe5a

Merge remote-tracking branch 'upstream/main' into feature/dataview-state

ca51c4f

mbondyra removed the request for review from a team September 7, 2022 12:40

mbondyra changed the title ~~[Lens] Query based annotation UI configuration panel~~ [Lens] Query based annotations Sep 7, 2022

dej611 and others added 10 commits September 7, 2022 14:50

✅ Add more error tests

3647032

Merge branch 'feature/129299' of github.com:dej611/kibana into featur…

a7f8ef5

…e/129299

👌 Rename migration state version type

08deb3f

fix types for expression chart

415f893

Merge branch 'feature/129299' of github.com:dej611/kibana into featur…

f3f2be6

…e/129299

🐛 Fix i18n id

cc03d2e

Merge branch 'feature/129299' of github.com:dej611/kibana into featur…

7ceea02

…e/129299

🏷️ Fix type issue

f3f6bdb

fix hidden all annotations

cdf64fe

Merge branch 'feature/129299' of github.com:dej611/kibana into featur…

64535f2

…e/129299

dej611 mentioned this pull request Sep 7, 2022

[Meta][Lens] Event Annotations #119240

Open

✅ Fix tests after ishidden removal

77b5598

mbondyra approved these changes Sep 8, 2022

View reviewed changes

Merge remote-tracking branch 'upstream/main' into feature/129299

a4c9edc

flash1293 reviewed Sep 8, 2022

View reviewed changes

🐛 Revert references migration to an in app solution

1af226a

flash1293 approved these changes Sep 8, 2022

View reviewed changes

Merge remote-tracking branch 'upstream/main' into feature/129299

b61cb1d

dej611 merged commit 1a1159b into elastic:main Sep 8, 2022

mbondyra mentioned this pull request Sep 19, 2022

[Lens] Query based event annotations #129299

Closed

dokmic mentioned this pull request Sep 30, 2022

Fix failing functional test of event annotations fetch #142386

Merged

KOTungseth mentioned this pull request Oct 3, 2022

[DOCS] Vis Editors 8.5 #142520

Merged

timductive mentioned this pull request Oct 26, 2023

Event Annotations Phase 1 #169971

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Lens] Query based annotations #138753

[Lens] Query based annotations #138753

dej611 commented Aug 12, 2022 •

edited

Loading

mbondyra left a comment

flash1293 commented Sep 8, 2022

flash1293 Sep 8, 2022

flash1293 Sep 8, 2022

flash1293 commented Sep 8, 2022

dej611 commented Sep 8, 2022

flash1293 left a comment

kibana-ci commented Sep 8, 2022

API count

[Lens] Query based annotations #138753

[Lens] Query based annotations #138753

Conversation

dej611 commented Aug 12, 2022 • edited Loading

Summary

Query based annotations in the chart

Config panel

Tasks

Checklist

Risk Matrix

For maintainers

mbondyra left a comment

Choose a reason for hiding this comment

flash1293 commented Sep 8, 2022

flash1293 Sep 8, 2022

Choose a reason for hiding this comment

flash1293 Sep 8, 2022

Choose a reason for hiding this comment

flash1293 commented Sep 8, 2022

dej611 commented Sep 8, 2022

flash1293 left a comment

Choose a reason for hiding this comment

kibana-ci commented Sep 8, 2022

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Public APIs missing exports

Page load bundle

API count

History

dej611 commented Aug 12, 2022 •

edited

Loading