[Discover-next] Address comments for search bar extensions and query assist #6933

joshuali925 · 2024-06-05T23:04:52Z

Description

Addresses most of the comments in #6894 and #6895

pass dependencies to isEnabled func
add lazy and memo to search bar extensions
move ppl specific strings out from query assist
prevent setstate after hook unmounts
add max-height to search bar extensions
prevent setstate after component unmounts
move ml-commons API to common/index.ts
improve i18n and accessibility usages
add hard-coded suggestions for sample data indices

Pending items:

add tests to query assist
extract AGENT_REQUEST_OPTIONS to UI settings [Discover-next] add query assist to query enhancements plugin #6895 (comment)
add advanced settings for disabling all search bar extensions [Discover-next] add query assist to query enhancements plugin #6895 (comment)

Issues Resolved

Screenshot

Testing the changes

Changelog

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

Signed-off-by: Joshua Li <[email protected]>

github-actions · 2024-06-05T23:05:25Z

ℹ️ Manual Changeset Creation Reminder

Please ensure manual commit for changeset file 6933.yml under folder changelogs/fragments to complete this PR.

If you want to use the available OpenSearch Changeset Bot App to avoid manual creation of changeset file you can install it in your forked repository following this link.

For more information about formatting of changeset files, please visit OpenSearch Auto Changeset and Release Notes Tool.

github-actions · 2024-06-05T23:07:21Z

❌ Changeset File Not Added Yet

Please ensure manual commit for changeset file 6933.yml under folder changelogs/fragments to complete this PR. File still missing.

codecov · 2024-06-05T23:21:24Z

Codecov Report

Attention: Patch coverage is 78.94737% with 4 lines in your changes missing coverage. Please review.

Project coverage is 67.05%. Comparing base (016e0f2) to head (caee590).

Files	Patch %	Lines
...ugins/data/public/ui/query_editor/query_editor.tsx	0.00%	1 Missing ⚠️
...ins/data/public/ui/search_bar_extensions/index.tsx	75.00%	1 Missing ⚠️
.../ui/search_bar_extensions/search_bar_extension.tsx	87.50%	0 Missing and 1 partial ⚠️
...ui/search_bar_extensions/search_bar_extensions.tsx	83.33%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@                  Coverage Diff                   @@
##           feature/discover-next    #6933   +/-   ##
======================================================
  Coverage                  67.05%   67.05%           
======================================================
  Files                       3451     3452    +1     
  Lines                      68216    68228   +12     
  Branches                   11140    11141    +1     
======================================================
+ Hits                       45740    45751   +11     
- Misses                     19856    19857    +1     
  Partials                    2620     2620

Flag	Coverage Δ
Linux_4	`34.81% <ø> (ø)`
Windows_1	`33.10% <ø> (ø)`
Windows_2	`55.08% <ø> (ø)`
Windows_3	`44.83% <78.94%> (+0.01%)`	⬆️
Windows_4	`34.81% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

kavilla · 2024-06-06T02:15:27Z

plugins-extra/query_enhancements/public/query_assist/components/query_assist_input.tsx

+    switch (props.selectedIndex) {
+      case 'opensearch_dashboards_sample_data_ecommerce':
+        return [
+          'How many unique customers placed orders this week?',


A future iteration, if we want to consider making this apart of the registration of sample data. I have the code ready to go: https://github.com/opensearch-project/OpenSearch-Dashboards/blob/feature/discover-next/plugins-extra/query_enhancements/server/plugin.ts#L49

kavilla

good stuff!

…assist (opensearch-project#6933) * pass dependencies to isEnabled func Signed-off-by: Joshua Li <[email protected]> * add lazy and memo to search bar extensions Signed-off-by: Joshua Li <[email protected]> * move ppl specific string out from query assist Signed-off-by: Joshua Li <[email protected]> * prevent setstate after hook unmounts Signed-off-by: Joshua Li <[email protected]> * add max-height to search bar extensions Signed-off-by: Joshua Li <[email protected]> * prevent setstate after component unmounts Signed-off-by: Joshua Li <[email protected]> * move ml-commons API to common/index.ts Signed-off-by: Joshua Li <[email protected]> * improve i18n and accessibility usages Signed-off-by: Joshua Li <[email protected]> * add hard-coded suggestions for sample data indices Signed-off-by: Joshua Li <[email protected]> --------- Signed-off-by: Joshua Li <[email protected]> (cherry picked from commit 4aade0f)

@returns

### Description see #6894 This PR picks #6894, #6895, #6933, #6972 to main. Additionally, - separates extensions from query enhancements - adds banner support - partially revert #6972 as it's pending on the data source commit to main - renames search bar extension to query editor extension A query editor extension can display a UI component above the query editor and/or a banner above the language selector. The component has the ability to read and write discover search bar states to enhance the search experience for users. The configuration is part of UI Enhancements. ```ts export interface QueryEditorExtensionDependencies { /** * Currently selected index patterns. */ indexPatterns?: Array<IIndexPattern | string>; /** * Currently selected data source. */ dataSource?: DataSource; /** * Currently selected query language. */ language: string; } export interface QueryEditorExtensionConfig { /** * The id for the search bar extension. */ id: string; /** * Lower order indicates higher position on UI. */ order: number; /** * A function that determines if the search bar extension is enabled and should be rendered on UI. * @returns whether the extension is enabled. */ isEnabled: (dependencies: QueryEditorExtensionDependencies) => Promise<boolean>; /** * A function that returns the search bar extension component. The component * will be displayed on top of the query editor in the search bar. * @param dependencies - The dependencies required for the extension. * @returns The component the search bar extension. */ getComponent?: (dependencies: QueryEditorExtensionDependencies) => React.ReactElement | null; /** * A function that returns the search bar extension banner. The banner is a * component that will be displayed on top of the search bar. * @param dependencies - The dependencies required for the extension. * @returns The component the search bar extension. */ getBanner?: (dependencies: QueryEditorExtensionDependencies) => React.ReactElement | null; } export interface UiEnhancements { query?: QueryEnhancement; + queryEditorExtension?: QueryEditorExtensionConfig; } ``` Developers can utilize search bar extensions to add additional features to the search bar, such as query assist. Issues resolved: #6077 A search bar extension can display a UI component above the query bar. The component has the ability to read and write discover search bar states to enhance the search experience for users. The configuration is part of Query Enhancements. Signed-off-by: Joshua Li <[email protected]>

@returns

### Description see #6894 This PR picks #6894, #6895, #6933, #6972 to main. Additionally, - separates extensions from query enhancements - adds banner support - partially revert #6972 as it's pending on the data source commit to main - renames search bar extension to query editor extension A query editor extension can display a UI component above the query editor and/or a banner above the language selector. The component has the ability to read and write discover search bar states to enhance the search experience for users. The configuration is part of UI Enhancements. ```ts export interface QueryEditorExtensionDependencies { /** * Currently selected index patterns. */ indexPatterns?: Array<IIndexPattern | string>; /** * Currently selected data source. */ dataSource?: DataSource; /** * Currently selected query language. */ language: string; } export interface QueryEditorExtensionConfig { /** * The id for the search bar extension. */ id: string; /** * Lower order indicates higher position on UI. */ order: number; /** * A function that determines if the search bar extension is enabled and should be rendered on UI. * @returns whether the extension is enabled. */ isEnabled: (dependencies: QueryEditorExtensionDependencies) => Promise<boolean>; /** * A function that returns the search bar extension component. The component * will be displayed on top of the query editor in the search bar. * @param dependencies - The dependencies required for the extension. * @returns The component the search bar extension. */ getComponent?: (dependencies: QueryEditorExtensionDependencies) => React.ReactElement | null; /** * A function that returns the search bar extension banner. The banner is a * component that will be displayed on top of the search bar. * @param dependencies - The dependencies required for the extension. * @returns The component the search bar extension. */ getBanner?: (dependencies: QueryEditorExtensionDependencies) => React.ReactElement | null; } export interface UiEnhancements { query?: QueryEnhancement; + queryEditorExtension?: QueryEditorExtensionConfig; } ``` Developers can utilize search bar extensions to add additional features to the search bar, such as query assist. Issues resolved: #6077 A search bar extension can display a UI component above the query bar. The component has the ability to read and write discover search bar states to enhance the search experience for users. The configuration is part of Query Enhancements. Signed-off-by: Joshua Li <[email protected]> (cherry picked from commit 4f54049) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

@returns

### Description see #6894 This PR picks #6894, #6895, #6933, #6972 to main. Additionally, - separates extensions from query enhancements - adds banner support - partially revert #6972 as it's pending on the data source commit to main - renames search bar extension to query editor extension A query editor extension can display a UI component above the query editor and/or a banner above the language selector. The component has the ability to read and write discover search bar states to enhance the search experience for users. The configuration is part of UI Enhancements. ```ts export interface QueryEditorExtensionDependencies { /** * Currently selected index patterns. */ indexPatterns?: Array<IIndexPattern | string>; /** * Currently selected data source. */ dataSource?: DataSource; /** * Currently selected query language. */ language: string; } export interface QueryEditorExtensionConfig { /** * The id for the search bar extension. */ id: string; /** * Lower order indicates higher position on UI. */ order: number; /** * A function that determines if the search bar extension is enabled and should be rendered on UI. * @returns whether the extension is enabled. */ isEnabled: (dependencies: QueryEditorExtensionDependencies) => Promise<boolean>; /** * A function that returns the search bar extension component. The component * will be displayed on top of the query editor in the search bar. * @param dependencies - The dependencies required for the extension. * @returns The component the search bar extension. */ getComponent?: (dependencies: QueryEditorExtensionDependencies) => React.ReactElement | null; /** * A function that returns the search bar extension banner. The banner is a * component that will be displayed on top of the search bar. * @param dependencies - The dependencies required for the extension. * @returns The component the search bar extension. */ getBanner?: (dependencies: QueryEditorExtensionDependencies) => React.ReactElement | null; } export interface UiEnhancements { query?: QueryEnhancement; + queryEditorExtension?: QueryEditorExtensionConfig; } ``` Developers can utilize search bar extensions to add additional features to the search bar, such as query assist. Issues resolved: #6077 A search bar extension can display a UI component above the query bar. The component has the ability to read and write discover search bar states to enhance the search experience for users. The configuration is part of Query Enhancements. (cherry picked from commit 4f54049) Signed-off-by: Joshua Li <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

@returns

### Description see opensearch-project#6894 This PR picks opensearch-project#6894, opensearch-project#6895, opensearch-project#6933, opensearch-project#6972 to main. Additionally, - separates extensions from query enhancements - adds banner support - partially revert opensearch-project#6972 as it's pending on the data source commit to main - renames search bar extension to query editor extension A query editor extension can display a UI component above the query editor and/or a banner above the language selector. The component has the ability to read and write discover search bar states to enhance the search experience for users. The configuration is part of UI Enhancements. ```ts export interface QueryEditorExtensionDependencies { /** * Currently selected index patterns. */ indexPatterns?: Array<IIndexPattern | string>; /** * Currently selected data source. */ dataSource?: DataSource; /** * Currently selected query language. */ language: string; } export interface QueryEditorExtensionConfig { /** * The id for the search bar extension. */ id: string; /** * Lower order indicates higher position on UI. */ order: number; /** * A function that determines if the search bar extension is enabled and should be rendered on UI. * @returns whether the extension is enabled. */ isEnabled: (dependencies: QueryEditorExtensionDependencies) => Promise<boolean>; /** * A function that returns the search bar extension component. The component * will be displayed on top of the query editor in the search bar. * @param dependencies - The dependencies required for the extension. * @returns The component the search bar extension. */ getComponent?: (dependencies: QueryEditorExtensionDependencies) => React.ReactElement | null; /** * A function that returns the search bar extension banner. The banner is a * component that will be displayed on top of the search bar. * @param dependencies - The dependencies required for the extension. * @returns The component the search bar extension. */ getBanner?: (dependencies: QueryEditorExtensionDependencies) => React.ReactElement | null; } export interface UiEnhancements { query?: QueryEnhancement; + queryEditorExtension?: QueryEditorExtensionConfig; } ``` Developers can utilize search bar extensions to add additional features to the search bar, such as query assist. Issues resolved: opensearch-project#6077 A search bar extension can display a UI component above the query bar. The component has the ability to read and write discover search bar states to enhance the search experience for users. The configuration is part of Query Enhancements. Signed-off-by: Joshua Li <[email protected]>

This PR picks [Discover-next] add query assist to query enhancements plugin opensearch-project/OpenSearch-Dashboards#6895 [Discover-next] Address comments for search bar extensions and query assist opensearch-project/OpenSearch-Dashboards#6933 [Discover-next] Support data sources for query assist opensearch-project/OpenSearch-Dashboards#6972 adds query assist banner a callout that advertises query assist when user has at least 1 language with query assist configured and is not on a configured language adds index selector for local cluster only This is a temporary solution given that in discover the index pattern selector will be removed. Before datasource and dataset selectors are added, query assist will rely on this index pattern selector to determine which index user wants to query. Signed-off-by: Joshua Li <[email protected]>

…assist (opensearch-project#6933) * pass dependencies to isEnabled func Signed-off-by: Joshua Li <[email protected]> * add lazy and memo to search bar extensions Signed-off-by: Joshua Li <[email protected]> * move ppl specific string out from query assist Signed-off-by: Joshua Li <[email protected]> * prevent setstate after hook unmounts Signed-off-by: Joshua Li <[email protected]> * add max-height to search bar extensions Signed-off-by: Joshua Li <[email protected]> * prevent setstate after component unmounts Signed-off-by: Joshua Li <[email protected]> * move ml-commons API to common/index.ts Signed-off-by: Joshua Li <[email protected]> * improve i18n and accessibility usages Signed-off-by: Joshua Li <[email protected]> * add hard-coded suggestions for sample data indices Signed-off-by: Joshua Li <[email protected]> --------- Signed-off-by: Joshua Li <[email protected]> (cherry picked from commit 4aade0f)

@sejli

### Description Introduces the addition of a new core plugin called `Query Enhancements` This plugin seems to be focused on enhancing query capabilities, particularly in areas like PPL (Piped Processing Language), SQL, and natural language query assistance. #### Plugin Configuration and Setup The plugin is set up with the necessary configuration files: - src/plugins/query_enhancements/opensearch_dashboards.json: Defines the plugin's metadata and dependencies. - src/plugins/query_enhancements/common/config.ts: Contains the configuration schema for the plugin. - src/plugins/query_enhancements/public/plugin.tsx: The main plugin file that sets up the public-facing part of the plugin. - src/plugins/query_enhancements/server/plugin.ts: The server-side setup for the plugin. #### Query Assist Feature A significant part of the plugin is dedicated to a "Query Assist" feature: - src/plugins/query_enhancements/public/query_assist/: This directory contains components, hooks, and utilities for the Query Assist feature. - src/plugins/query_enhancements/public/query_assist/components/query_assist_bar.tsx: A React component for the Query Assist bar. - src/plugins/query_enhancements/server/routes/query_assist/: Server-side routing for Query Assist functionality. #### Search Interceptors The plugin introduces several search interceptors: - src/plugins/query_enhancements/public/search/ppl_search_interceptor.ts - src/plugins/query_enhancements/public/search/sql_search_interceptor.ts - src/plugins/query_enhancements/public/search/sql_async_search_interceptor.ts These interceptors likely modify or enhance the search functionality for different query languages. #### Data Source Connection There's a new feature for data source connections: - src/plugins/query_enhancements/public/data_source_connection/: This directory contains components and services for managing data source connections. #### Server-side Search Strategies The plugin implements server-side search strategies for different query types: - src/plugins/query_enhancements/server/search/ppl_search_strategy.ts - src/plugins/query_enhancements/server/search/sql_search_strategy.ts - src/plugins/query_enhancements/server/search/sql_async_search_strategy.ts #### Modifications to Existing Files There are some modifications to existing files outside the new plugin directory: - src/plugins/data/public/index.ts: Exports new types and interfaces. - src/plugins/data/public/ui/index.ts: Exports new UI components. - src/plugins/data/public/ui/query_editor/index.tsx: Modifies the QueryEditor component. #### Testing The plugin includes a comprehensive test setup: - src/plugins/query_enhancements/test/: Contains Jest configuration and setup files for testing. #### Acknowledgements Contains work by @sejli @joshuali925 @Swiddis @paulstn @mengweieric Thx @ashwin-pc for the description summary ### Issues Related #6072 #6074 #6075 Commits include: * init Signed-off-by: Kawika Avilla <[email protected]> * fixing bugs and removing target Signed-off-by: Sean Li <[email protected]> * add target and build to .gitignore Signed-off-by: Sean Li <[email protected]> * Remove SQL Async and Query Assist Going for the light weight solution for 2.15. So took out what wasn't required deliverable. However, I created a branch `feature/discover-next` which has the state where all the features where added so we can port it over post 2.15. Signed-off-by: Kawika Avilla <[email protected]> * adding fix for PPL queries Signed-off-by: Sean Li <[email protected]> * Clean up files and add helper functions Signed-off-by: Kawika Avilla <[email protected]> * final touches Signed-off-by: Kawika Avilla <[email protected]> * [Discover-next] add query assist to query enhancements plugin (#6895) it adds query assist specific logic in query enhancements plugin to show a UI above the PPL search bar if user has configured PPL agent. Issues Resolved: #6820 * add query assist to query enhancements Signed-off-by: Joshua Li <[email protected]> * align language to uppercase Signed-off-by: Joshua Li <[email protected]> * pick PR 6167 Signed-off-by: Joshua Li <[email protected]> * use useState hooks for query assist There is a bug in data explorer `AppContainer` where memorized `DiscoverCanvas` gets unmounted after `setQuery`. PR 6167 works around it by memorizing `AppContainer`. As query assist is no longer being unmounted, we can use proper hooks to persist state now. Signed-off-by: Joshua Li <[email protected]> * Revert "pick PR 6167" This reverts commit acb0d41. Wait for official 6167 to merge to avoid conflict Signed-off-by: Joshua Li <[email protected]> * address comments for PR 6894 Signed-off-by: Joshua Li <[email protected]> --------- Signed-off-by: Joshua Li <[email protected]> (cherry picked from commit 016e0f2) * [Discover-next] Address comments for search bar extensions and query assist (#6933) * pass dependencies to isEnabled func Signed-off-by: Joshua Li <[email protected]> * add lazy and memo to search bar extensions Signed-off-by: Joshua Li <[email protected]> * move ppl specific string out from query assist Signed-off-by: Joshua Li <[email protected]> * prevent setstate after hook unmounts Signed-off-by: Joshua Li <[email protected]> * add max-height to search bar extensions Signed-off-by: Joshua Li <[email protected]> * prevent setstate after component unmounts Signed-off-by: Joshua Li <[email protected]> * move ml-commons API to common/index.ts Signed-off-by: Joshua Li <[email protected]> * improve i18n and accessibility usages Signed-off-by: Joshua Li <[email protected]> * add hard-coded suggestions for sample data indices Signed-off-by: Joshua Li <[email protected]> --------- Signed-off-by: Joshua Li <[email protected]> (cherry picked from commit 4aade0f) * [Discover-next] Support data sources for query assist (#6972) * disable query assist for non-default datasource Signed-off-by: Joshua Li <[email protected]> * disable query assist input when loading Signed-off-by: Joshua Li <[email protected]> * support MDS for query assist Signed-off-by: Joshua Li <[email protected]> * add unit tests for agents Signed-off-by: Joshua Li <[email protected]> * Revert "add unit tests for agents" This reverts commit 983514e. The test configs are not yet setup in query_enhancements plugins. Signed-off-by: Joshua Li <[email protected]> --------- Signed-off-by: Joshua Li <[email protected]> (cherry picked from commit 328e08e) * add query assist banner Signed-off-by: Joshua Li <[email protected]> * implement banner rendering logic Signed-off-by: Joshua Li <[email protected]> * pick opensearch-project/dashboards-observability/pull/1896 Signed-off-by: Joshua Li <[email protected]> * Conditionally render the data source select config Signed-off-by: Kawika Avilla <[email protected]> * restore no space Signed-off-by: Kawika Avilla <[email protected]> * add basic tests Signed-off-by: Joshua Li <[email protected]> * add index selector This is a temporary solution given that in discover the index pattern selector will be removed. Before datasource and dataset selectors are added, query assist will rely on this index pattern selector to determine which index user wants to query. Signed-off-by: Joshua Li <[email protected]> * MDS working with PPL and SQL Utilizing the work from MDS to make PPL and SQL calls to a remote cluster. Signed-off-by: Kawika Avilla <[email protected]> * rename logo to mark Signed-off-by: Joshua Li <[email protected]> * extract supported languages to config Signed-off-by: Joshua Li <[email protected]> * Add missing license headers Signed-off-by: Kawika Avilla <[email protected]> * initial implementation for async queries Signed-off-by: Sean Li <[email protected]> * remove queryId and sessionId from facet Signed-off-by: Sean Li <[email protected]> * fix: Update call out tests with intl wrapper Signed-off-by: Simeon Widdis <[email protected]> * maintenance: Add and apply lint command Signed-off-by: Simeon Widdis <[email protected]> * changing to observables Signed-off-by: Sean Li <[email protected]> * search interceptor and cleanup Signed-off-by: Sean Li <[email protected]> * moving polling class into plugin Signed-off-by: Sean Li <[email protected]> * Some minor clean ups plus lint Signed-off-by: Kawika Avilla <[email protected]> * test: Add mock-heavy tests for sql search strategy Signed-off-by: Simeon Widdis <[email protected]> * test: add GH workflows Signed-off-by: Simeon Widdis <[email protected]> * More clean ups Add some props interfaces and consolidated some of the facets Signed-off-by: Kawika Avilla <[email protected]> * chore: apply auto-lints to plugin Signed-off-by: Simeon Widdis <[email protected]> * Not done Signed-off-by: Kawika Avilla <[email protected]> * working Signed-off-by: Kawika Avilla <[email protected]> * working and styled Signed-off-by: Kawika Avilla <[email protected]> * add connection service Signed-off-by: Kawika Avilla <[email protected]> * Update public/types.ts Co-authored-by: Joshua Li <[email protected]> * Update public/plugin.tsx Co-authored-by: Joshua Li <[email protected]> * Update public/plugin.tsx Co-authored-by: Joshua Li <[email protected]> * fix linter Signed-off-by: Kawika Avilla <[email protected]> * add more tests for query assist (#31) add more tests for query assist update eslint config to make it work increase index selector width Signed-off-by: Joshua Li <[email protected]> * fix!: remove dataSource key in get connection response The `Connection` type does not have the extra `dataSource` object around response, and most of the time code references connection over dataSource. remove the redundent object to make response consistent with type. Signed-off-by: Joshua Li <[email protected]> * change isEnabled to an observable based on selected connection Signed-off-by: Joshua Li <[email protected]> * protect duplicated calls Signed-off-by: Joshua Li <[email protected]> * [build] import via start services Signed-off-by: Kawika Avilla <[email protected]> * Changeset file for PR #7212 created/updated * foobar Signed-off-by: Kawika Avilla <[email protected]> * updating imports still in progress Signed-off-by: Kawika Avilla <[email protected]> * Update more imports Signed-off-by: Kawika Avilla <[email protected]> * fix setting of state Signed-off-by: Kawika Avilla <[email protected]> * fixes bugs Signed-off-by: Kawika Avilla <[email protected]> * Update test Signed-off-by: Kawika Avilla <[email protected]> --------- Signed-off-by: Kawika Avilla <[email protected]> Signed-off-by: Sean Li <[email protected]> Signed-off-by: Joshua Li <[email protected]> Signed-off-by: Simeon Widdis <[email protected]> Co-authored-by: Sean Li <[email protected]> Co-authored-by: Joshua Li <[email protected]> Co-authored-by: Simeon Widdis <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> Signed-off-by: Kawika Avilla <[email protected]>

@sejli

…earch-project#7212) Introduces the addition of a new core plugin called `Query Enhancements` This plugin seems to be focused on enhancing query capabilities, particularly in areas like PPL (Piped Processing Language), SQL, and natural language query assistance. The plugin is set up with the necessary configuration files: - src/plugins/query_enhancements/opensearch_dashboards.json: Defines the plugin's metadata and dependencies. - src/plugins/query_enhancements/common/config.ts: Contains the configuration schema for the plugin. - src/plugins/query_enhancements/public/plugin.tsx: The main plugin file that sets up the public-facing part of the plugin. - src/plugins/query_enhancements/server/plugin.ts: The server-side setup for the plugin. A significant part of the plugin is dedicated to a "Query Assist" feature: - src/plugins/query_enhancements/public/query_assist/: This directory contains components, hooks, and utilities for the Query Assist feature. - src/plugins/query_enhancements/public/query_assist/components/query_assist_bar.tsx: A React component for the Query Assist bar. - src/plugins/query_enhancements/server/routes/query_assist/: Server-side routing for Query Assist functionality. The plugin introduces several search interceptors: - src/plugins/query_enhancements/public/search/ppl_search_interceptor.ts - src/plugins/query_enhancements/public/search/sql_search_interceptor.ts - src/plugins/query_enhancements/public/search/sql_async_search_interceptor.ts These interceptors likely modify or enhance the search functionality for different query languages. There's a new feature for data source connections: - src/plugins/query_enhancements/public/data_source_connection/: This directory contains components and services for managing data source connections. The plugin implements server-side search strategies for different query types: - src/plugins/query_enhancements/server/search/ppl_search_strategy.ts - src/plugins/query_enhancements/server/search/sql_search_strategy.ts - src/plugins/query_enhancements/server/search/sql_async_search_strategy.ts There are some modifications to existing files outside the new plugin directory: - src/plugins/data/public/index.ts: Exports new types and interfaces. - src/plugins/data/public/ui/index.ts: Exports new UI components. - src/plugins/data/public/ui/query_editor/index.tsx: Modifies the QueryEditor component. The plugin includes a comprehensive test setup: - src/plugins/query_enhancements/test/: Contains Jest configuration and setup files for testing. Contains work by @sejli @joshuali925 @Swiddis @paulstn @mengweieric Thx @ashwin-pc for the description summary opensearch-project#6072 opensearch-project#6074 opensearch-project#6075 Commits include: * init Signed-off-by: Kawika Avilla <[email protected]> * fixing bugs and removing target Signed-off-by: Sean Li <[email protected]> * add target and build to .gitignore Signed-off-by: Sean Li <[email protected]> * Remove SQL Async and Query Assist Going for the light weight solution for 2.15. So took out what wasn't required deliverable. However, I created a branch `feature/discover-next` which has the state where all the features where added so we can port it over post 2.15. Signed-off-by: Kawika Avilla <[email protected]> * adding fix for PPL queries Signed-off-by: Sean Li <[email protected]> * Clean up files and add helper functions Signed-off-by: Kawika Avilla <[email protected]> * final touches Signed-off-by: Kawika Avilla <[email protected]> * [Discover-next] add query assist to query enhancements plugin (opensearch-project#6895) it adds query assist specific logic in query enhancements plugin to show a UI above the PPL search bar if user has configured PPL agent. Issues Resolved: opensearch-project#6820 * add query assist to query enhancements Signed-off-by: Joshua Li <[email protected]> * align language to uppercase Signed-off-by: Joshua Li <[email protected]> * pick PR 6167 Signed-off-by: Joshua Li <[email protected]> * use useState hooks for query assist There is a bug in data explorer `AppContainer` where memorized `DiscoverCanvas` gets unmounted after `setQuery`. PR 6167 works around it by memorizing `AppContainer`. As query assist is no longer being unmounted, we can use proper hooks to persist state now. Signed-off-by: Joshua Li <[email protected]> * Revert "pick PR 6167" This reverts commit acb0d41. Wait for official 6167 to merge to avoid conflict Signed-off-by: Joshua Li <[email protected]> * address comments for PR 6894 Signed-off-by: Joshua Li <[email protected]> --------- Signed-off-by: Joshua Li <[email protected]> (cherry picked from commit 016e0f2) * [Discover-next] Address comments for search bar extensions and query assist (opensearch-project#6933) * pass dependencies to isEnabled func Signed-off-by: Joshua Li <[email protected]> * add lazy and memo to search bar extensions Signed-off-by: Joshua Li <[email protected]> * move ppl specific string out from query assist Signed-off-by: Joshua Li <[email protected]> * prevent setstate after hook unmounts Signed-off-by: Joshua Li <[email protected]> * add max-height to search bar extensions Signed-off-by: Joshua Li <[email protected]> * prevent setstate after component unmounts Signed-off-by: Joshua Li <[email protected]> * move ml-commons API to common/index.ts Signed-off-by: Joshua Li <[email protected]> * improve i18n and accessibility usages Signed-off-by: Joshua Li <[email protected]> * add hard-coded suggestions for sample data indices Signed-off-by: Joshua Li <[email protected]> --------- Signed-off-by: Joshua Li <[email protected]> (cherry picked from commit 4aade0f) * [Discover-next] Support data sources for query assist (opensearch-project#6972) * disable query assist for non-default datasource Signed-off-by: Joshua Li <[email protected]> * disable query assist input when loading Signed-off-by: Joshua Li <[email protected]> * support MDS for query assist Signed-off-by: Joshua Li <[email protected]> * add unit tests for agents Signed-off-by: Joshua Li <[email protected]> * Revert "add unit tests for agents" This reverts commit 983514e. The test configs are not yet setup in query_enhancements plugins. Signed-off-by: Joshua Li <[email protected]> --------- Signed-off-by: Joshua Li <[email protected]> (cherry picked from commit 328e08e) * add query assist banner Signed-off-by: Joshua Li <[email protected]> * implement banner rendering logic Signed-off-by: Joshua Li <[email protected]> * pick opensearch-project/dashboards-observability/pull/1896 Signed-off-by: Joshua Li <[email protected]> * Conditionally render the data source select config Signed-off-by: Kawika Avilla <[email protected]> * restore no space Signed-off-by: Kawika Avilla <[email protected]> * add basic tests Signed-off-by: Joshua Li <[email protected]> * add index selector This is a temporary solution given that in discover the index pattern selector will be removed. Before datasource and dataset selectors are added, query assist will rely on this index pattern selector to determine which index user wants to query. Signed-off-by: Joshua Li <[email protected]> * MDS working with PPL and SQL Utilizing the work from MDS to make PPL and SQL calls to a remote cluster. Signed-off-by: Kawika Avilla <[email protected]> * rename logo to mark Signed-off-by: Joshua Li <[email protected]> * extract supported languages to config Signed-off-by: Joshua Li <[email protected]> * Add missing license headers Signed-off-by: Kawika Avilla <[email protected]> * initial implementation for async queries Signed-off-by: Sean Li <[email protected]> * remove queryId and sessionId from facet Signed-off-by: Sean Li <[email protected]> * fix: Update call out tests with intl wrapper Signed-off-by: Simeon Widdis <[email protected]> * maintenance: Add and apply lint command Signed-off-by: Simeon Widdis <[email protected]> * changing to observables Signed-off-by: Sean Li <[email protected]> * search interceptor and cleanup Signed-off-by: Sean Li <[email protected]> * moving polling class into plugin Signed-off-by: Sean Li <[email protected]> * Some minor clean ups plus lint Signed-off-by: Kawika Avilla <[email protected]> * test: Add mock-heavy tests for sql search strategy Signed-off-by: Simeon Widdis <[email protected]> * test: add GH workflows Signed-off-by: Simeon Widdis <[email protected]> * More clean ups Add some props interfaces and consolidated some of the facets Signed-off-by: Kawika Avilla <[email protected]> * chore: apply auto-lints to plugin Signed-off-by: Simeon Widdis <[email protected]> * Not done Signed-off-by: Kawika Avilla <[email protected]> * working Signed-off-by: Kawika Avilla <[email protected]> * working and styled Signed-off-by: Kawika Avilla <[email protected]> * add connection service Signed-off-by: Kawika Avilla <[email protected]> * Update public/types.ts Co-authored-by: Joshua Li <[email protected]> * Update public/plugin.tsx Co-authored-by: Joshua Li <[email protected]> * Update public/plugin.tsx Co-authored-by: Joshua Li <[email protected]> * fix linter Signed-off-by: Kawika Avilla <[email protected]> * add more tests for query assist (opensearch-project#31) add more tests for query assist update eslint config to make it work increase index selector width Signed-off-by: Joshua Li <[email protected]> * fix!: remove dataSource key in get connection response The `Connection` type does not have the extra `dataSource` object around response, and most of the time code references connection over dataSource. remove the redundent object to make response consistent with type. Signed-off-by: Joshua Li <[email protected]> * change isEnabled to an observable based on selected connection Signed-off-by: Joshua Li <[email protected]> * protect duplicated calls Signed-off-by: Joshua Li <[email protected]> * [build] import via start services Signed-off-by: Kawika Avilla <[email protected]> * Changeset file for PR opensearch-project#7212 created/updated * foobar Signed-off-by: Kawika Avilla <[email protected]> * updating imports still in progress Signed-off-by: Kawika Avilla <[email protected]> * Update more imports Signed-off-by: Kawika Avilla <[email protected]> * fix setting of state Signed-off-by: Kawika Avilla <[email protected]> * fixes bugs Signed-off-by: Kawika Avilla <[email protected]> * Update test Signed-off-by: Kawika Avilla <[email protected]> --------- Signed-off-by: Kawika Avilla <[email protected]> Signed-off-by: Sean Li <[email protected]> Signed-off-by: Joshua Li <[email protected]> Signed-off-by: Simeon Widdis <[email protected]> Co-authored-by: Sean Li <[email protected]> Co-authored-by: Joshua Li <[email protected]> Co-authored-by: Simeon Widdis <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> Signed-off-by: Kawika Avilla <[email protected]> (cherry picked from commit 9194d83) Signed-off-by: Sean Li <[email protected]>

@sejli

#7400) Introduces the addition of a new core plugin called `Query Enhancements` This plugin seems to be focused on enhancing query capabilities, particularly in areas like PPL (Piped Processing Language), SQL, and natural language query assistance. The plugin is set up with the necessary configuration files: - src/plugins/query_enhancements/opensearch_dashboards.json: Defines the plugin's metadata and dependencies. - src/plugins/query_enhancements/common/config.ts: Contains the configuration schema for the plugin. - src/plugins/query_enhancements/public/plugin.tsx: The main plugin file that sets up the public-facing part of the plugin. - src/plugins/query_enhancements/server/plugin.ts: The server-side setup for the plugin. A significant part of the plugin is dedicated to a "Query Assist" feature: - src/plugins/query_enhancements/public/query_assist/: This directory contains components, hooks, and utilities for the Query Assist feature. - src/plugins/query_enhancements/public/query_assist/components/query_assist_bar.tsx: A React component for the Query Assist bar. - src/plugins/query_enhancements/server/routes/query_assist/: Server-side routing for Query Assist functionality. The plugin introduces several search interceptors: - src/plugins/query_enhancements/public/search/ppl_search_interceptor.ts - src/plugins/query_enhancements/public/search/sql_search_interceptor.ts - src/plugins/query_enhancements/public/search/sql_async_search_interceptor.ts These interceptors likely modify or enhance the search functionality for different query languages. There's a new feature for data source connections: - src/plugins/query_enhancements/public/data_source_connection/: This directory contains components and services for managing data source connections. The plugin implements server-side search strategies for different query types: - src/plugins/query_enhancements/server/search/ppl_search_strategy.ts - src/plugins/query_enhancements/server/search/sql_search_strategy.ts - src/plugins/query_enhancements/server/search/sql_async_search_strategy.ts There are some modifications to existing files outside the new plugin directory: - src/plugins/data/public/index.ts: Exports new types and interfaces. - src/plugins/data/public/ui/index.ts: Exports new UI components. - src/plugins/data/public/ui/query_editor/index.tsx: Modifies the QueryEditor component. The plugin includes a comprehensive test setup: - src/plugins/query_enhancements/test/: Contains Jest configuration and setup files for testing. Contains work by @sejli @joshuali925 @Swiddis @paulstn @mengweieric Thx @ashwin-pc for the description summary #6072 #6074 #6075 Commits include: * init * fixing bugs and removing target * add target and build to .gitignore * Remove SQL Async and Query Assist Going for the light weight solution for 2.15. So took out what wasn't required deliverable. However, I created a branch `feature/discover-next` which has the state where all the features where added so we can port it over post 2.15. * adding fix for PPL queries * Clean up files and add helper functions * final touches * [Discover-next] add query assist to query enhancements plugin (#6895) it adds query assist specific logic in query enhancements plugin to show a UI above the PPL search bar if user has configured PPL agent. Issues Resolved: #6820 * add query assist to query enhancements * align language to uppercase * pick PR 6167 * use useState hooks for query assist There is a bug in data explorer `AppContainer` where memorized `DiscoverCanvas` gets unmounted after `setQuery`. PR 6167 works around it by memorizing `AppContainer`. As query assist is no longer being unmounted, we can use proper hooks to persist state now. * Revert "pick PR 6167" This reverts commit acb0d41. Wait for official 6167 to merge to avoid conflict * address comments for PR 6894 --------- (cherry picked from commit 016e0f2) * [Discover-next] Address comments for search bar extensions and query assist (#6933) * pass dependencies to isEnabled func * add lazy and memo to search bar extensions * move ppl specific string out from query assist * prevent setstate after hook unmounts * add max-height to search bar extensions * prevent setstate after component unmounts * move ml-commons API to common/index.ts * improve i18n and accessibility usages * add hard-coded suggestions for sample data indices --------- (cherry picked from commit 4aade0f) * [Discover-next] Support data sources for query assist (#6972) * disable query assist for non-default datasource * disable query assist input when loading * support MDS for query assist * add unit tests for agents * Revert "add unit tests for agents" This reverts commit 983514e. The test configs are not yet setup in query_enhancements plugins. --------- (cherry picked from commit 328e08e) * add query assist banner * implement banner rendering logic * pick opensearch-project/dashboards-observability/pull/1896 * Conditionally render the data source select config * restore no space * add basic tests * add index selector This is a temporary solution given that in discover the index pattern selector will be removed. Before datasource and dataset selectors are added, query assist will rely on this index pattern selector to determine which index user wants to query. * MDS working with PPL and SQL Utilizing the work from MDS to make PPL and SQL calls to a remote cluster. * rename logo to mark * extract supported languages to config * Add missing license headers * initial implementation for async queries * remove queryId and sessionId from facet * fix: Update call out tests with intl wrapper * maintenance: Add and apply lint command * changing to observables * search interceptor and cleanup * moving polling class into plugin * Some minor clean ups plus lint * test: Add mock-heavy tests for sql search strategy * test: add GH workflows * More clean ups Add some props interfaces and consolidated some of the facets * chore: apply auto-lints to plugin * Not done * working * working and styled * add connection service * Update public/types.ts * Update public/plugin.tsx * Update public/plugin.tsx * fix linter * add more tests for query assist (#31) add more tests for query assist update eslint config to make it work increase index selector width * fix!: remove dataSource key in get connection response The `Connection` type does not have the extra `dataSource` object around response, and most of the time code references connection over dataSource. remove the redundent object to make response consistent with type. * change isEnabled to an observable based on selected connection * protect duplicated calls * [build] import via start services * Changeset file for PR #7212 created/updated * foobar * updating imports still in progress * Update more imports * fix setting of state * fixes bugs * Update test --------- (cherry picked from commit 9194d83) Signed-off-by: Kawika Avilla <[email protected]> Signed-off-by: Sean Li <[email protected]> Co-authored-by: Kawika Avilla <[email protected]>

joshuali925 added 9 commits June 5, 2024 15:14

pass dependencies to isEnabled func

fe0c5fc

Signed-off-by: Joshua Li <[email protected]>

add lazy and memo to search bar extensions

c0d3bdd

Signed-off-by: Joshua Li <[email protected]>

move ppl specific string out from query assist

1bcc485

Signed-off-by: Joshua Li <[email protected]>

prevent setstate after hook unmounts

17e698f

Signed-off-by: Joshua Li <[email protected]>

add max-height to search bar extensions

d56cc88

Signed-off-by: Joshua Li <[email protected]>

prevent setstate after component unmounts

744afa2

Signed-off-by: Joshua Li <[email protected]>

move ml-commons API to common/index.ts

c671c08

Signed-off-by: Joshua Li <[email protected]>

improve i18n and accessibility usages

ccf9962

Signed-off-by: Joshua Li <[email protected]>

add hard-coded suggestions for sample data indices

caee590

Signed-off-by: Joshua Li <[email protected]>

joshuali925 requested review from ananzh, kavilla, AMoo-Miki, ashwin-pc, joshuarrrr, abbyhu2000, zengyan-amazon, zhongnansu, manasvinibs, ZilongX, Flyingliuhub, curq, bandinib-amzn, SuZhou-Joe, ruanyl, BionIT, xinruiba, zhyuanqi and mengweieric as code owners June 5, 2024 23:04

github-actions bot added the repeat-contributor label Jun 5, 2024

github-actions bot added the failed changeset label Jun 5, 2024

joshuali925 marked this pull request as draft June 5, 2024 23:12

joshuali925 marked this pull request as ready for review June 5, 2024 23:19

kavilla reviewed Jun 6, 2024

View reviewed changes

kavilla approved these changes Jun 6, 2024

View reviewed changes

kavilla merged commit 4aade0f into opensearch-project:feature/discover-next Jun 6, 2024
41 of 69 checks passed

This was referenced Jun 14, 2024

[Discover-next] Add query editor extensions #7034

Merged

Add query assist to query enhancements plugin kavilla/queryEnhancements#14

Merged

sejli mentioned this pull request Jul 23, 2024

[Manual Backport 2.16] [Discover-next] add query enhancements plugin as a core plugin (#7212) #7400

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Discover-next] Address comments for search bar extensions and query assist #6933

[Discover-next] Address comments for search bar extensions and query assist #6933

joshuali925 commented Jun 5, 2024 •

edited

Loading

github-actions bot commented Jun 5, 2024

github-actions bot commented Jun 5, 2024

codecov bot commented Jun 5, 2024 •

edited

Loading

kavilla Jun 6, 2024

kavilla left a comment

[Discover-next] Address comments for search bar extensions and query assist #6933

[Discover-next] Address comments for search bar extensions and query assist #6933

Conversation

joshuali925 commented Jun 5, 2024 • edited Loading

Description

Issues Resolved

Screenshot

Testing the changes

Changelog

Check List

github-actions bot commented Jun 5, 2024

ℹ️ Manual Changeset Creation Reminder

github-actions bot commented Jun 5, 2024

❌ Changeset File Not Added Yet

codecov bot commented Jun 5, 2024 • edited Loading

Codecov Report

kavilla Jun 6, 2024

Choose a reason for hiding this comment

kavilla left a comment

Choose a reason for hiding this comment

joshuali925 commented Jun 5, 2024 •

edited

Loading

codecov bot commented Jun 5, 2024 •

edited

Loading