[Auto Suggest] OpenSearch SQL autosuggest with ANTLR #7336

mengweieric · 2024-07-20T22:06:41Z

Description

OpenSearch SQL auto suggest support for query editor.

Issues Resolved

Screenshot

Testing the changes

Changelog

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

Signed-off-by: Eric <[email protected]>

github-actions · 2024-07-20T22:07:13Z

ℹ️ Manual Changeset Creation Reminder

Please ensure manual commit for changeset file 7336.yml under folder changelogs/fragments to complete this PR.

If you want to use the available OpenSearch Changeset Bot App to avoid manual creation of changeset file you can install it in your forked repository following this link.

For more information about formatting of changeset files, please visit OpenSearch Auto Changeset and Release Notes Tool.

Signed-off-by: Eric <[email protected]>

codecov · 2024-07-20T22:36:37Z

Codecov Report

Attention: Patch coverage is 47.10983% with 183 lines in your changes missing coverage. Please review.

Project coverage is 63.95%. Comparing base (34f1b73) to head (3e5b430).
Report is 5 commits behind head on main.

Files	Patch %	Lines
...ntlr/opensearch_sql/opensearch_sql_autocomplete.ts	25.97%	56 Missing and 1 partial ⚠️
...ata/public/antlr/opensearch_sql/code_completion.ts	7.14%	52 Missing ⚠️
.../plugins/data/public/antlr/opensearch_sql/table.ts	52.94%	34 Missing and 6 partials ⚠️
src/plugins/data/public/antlr/shared/cursor.ts	41.02%	19 Missing and 4 partials ⚠️
src/plugins/data/public/antlr/shared/utils.ts	82.14%	1 Missing and 4 partials ⚠️
.../plugins/data/public/antlr/opensearch_sql/parse.ts	85.00%	3 Missing ⚠️
src/plugins/data/public/plugin.ts	0.00%	2 Missing ⚠️
...s/data/public/antlr/opensearch_sql/symbol_table.ts	95.65%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7336      +/-   ##
==========================================
- Coverage   67.62%   63.95%   -3.67%     
==========================================
  Files        3583     3595      +12     
  Lines       72174    78211    +6037     
  Branches    11810    12327     +517     
==========================================
+ Hits        48805    50017    +1212     
- Misses      20385    25177    +4792     
- Partials     2984     3017      +33

Flag	Coverage Δ
Linux_1	`?`
Linux_2	`55.50% <ø> (ø)`
Linux_3	`41.31% <45.69%> (-3.20%)`	⬇️
Linux_4	`31.55% <12.13%> (-2.53%)`	⬇️
Windows_1	`31.08% <12.13%> (-2.10%)`	⬇️
Windows_2	`55.45% <ø> (ø)`
Windows_3	`41.33% <45.69%> (-3.20%)`	⬇️
Windows_4	`31.55% <12.13%> (-2.53%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Eric <[email protected]>

mengweieric · 2024-07-23T00:43:10Z

need to unblock @paulstn for DQL, will follow up on increasing test coverage.

RyanL1997 · 2024-07-23T01:28:47Z

src/plugins/data/public/antlr/shared/utils.ts

+) =>
+  connectionsService.getSelectedConnection$().pipe(
+    distinctUntilChanged(),
+    switchMap((connection) => {


Is this the place to get multi datasource id?

yes, and if it's undefined, the request will hit local cluster.

Got it. Just wanna make sure the underlying logic can be relied on as a source of truth for the flag of data_source.enabled, since my work changed the registration behavior of datasource management plugin. And I've seen that some of the plugin used the datasource management plugin status to determine if multi-datasource is enabled or not.

RyanL1997 · 2024-07-23T01:30:25Z

src/plugins/data/public/antlr/opensearch_sql/grammar/.antlr/OpenSearchSQLLexer.java

+import org.antlr.v4.runtime.misc.*;
+
+@SuppressWarnings({"all", "warnings", "unchecked", "unused", "cast", "CheckReturnValue", "this-escape"})
+public class OpenSearchSQLLexer extends Lexer {


For my knowledge, why we need this java file?

these are auto-generated files from ANTLR engine for query parsing internally.

RyanL1997 · 2024-07-23T01:32:26Z

package.json

@@ -84,7 +84,8 @@
    "cypress:run-without-security": "env TZ=America/Los_Angeles NO_COLOR=1 cypress run --headless --env SECURITY_ENABLED=false",
    "cypress:run-with-security": "env TZ=America/Los_Angeles NO_COLOR=1 cypress run --headless --env SECURITY_ENABLED=true,openSearchUrl=https://localhost:9200,WAIT_FOR_LOADER_BUFFER_MS=500",
    "osd:ciGroup10": "echo \"dashboard_sanity_test_spec.js\"",
-    "osd:ciGroup11": "echo \"apps/vis_builder/*.js\""
+    "osd:ciGroup11": "echo \"apps/vis_builder/*.js\"",
+    "generate:opensearchsqlantlr": "./node_modules/antlr4ng-cli/index.js -Dlanguage=TypeScript -o ./src/plugins/data/public/antlr/opensearch_sql/.generated -visitor -no-listener -Xexact-output-dir ./src/plugins/data/public/antlr/opensearch_sql/grammar/OpenSearchSQLLexer.g4 ./src/plugins/data/public/antlr/opensearch_sql/grammar/OpenSearchSQLParser.g4"


Are this manual or auto generated? If it is manual, we may need better mechanism in the future to embed this into CI

for now it's manually generated, previously we add this to post commit hook but since the build pipeline does not have java runtime, we switch to manual and only need to run this when there's grammar changes

RyanL1997 · 2024-07-23T01:35:01Z

src/plugins/data/public/antlr/opensearch_sql/grammar/OpenSearchSQLParser.g4

+THE SOFTWARE.
+*/
+
+parser grammar OpenSearchSQLParser;


where is the source of this grammar? Is this coming from the sql backend?

yes, it's from SQL plugin.

zhongnansu · 2024-07-23T02:12:33Z

need to unblock @paulstn for DQL, will follow up on increasing test coverage.

@mengweieric could you create a issue for tracking purpose?

mengweieric · 2024-07-23T02:37:41Z

need to unblock @paulstn for DQL, will follow up on increasing test coverage.

@mengweieric could you create a issue for tracking purpose?

sure. #7387

opensearch-trigger-bot · 2024-07-23T02:40:36Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.x
# Create a new branch
git switch --create backport/backport-7336-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9348bd4276a6daf4a719f3709c3d854e002350e0
# Push it to GitHub
git push --set-upstream origin backport/backport-7336-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-7336-to-2.x.

opensearch-trigger-bot · 2024-07-23T02:40:37Z

The backport to 2.16 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.16 2.16
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.16
# Create a new branch
git switch --create backport/backport-7336-to-2.16
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9348bd4276a6daf4a719f3709c3d854e002350e0
# Push it to GitHub
git push --set-upstream origin backport/backport-7336-to-2.16
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.16

Then, create a pull request where the base branch is 2.16 and the compare/head branch is backport/backport-7336-to-2.16.

abbyhu2000 · 2024-07-23T19:58:42Z

src/plugins/data/public/ui/filter_bar/_global_filter_group.scss

@@ -1,6 +1,7 @@
 // SASSTODO: Probably not the right file for this selector, but temporary until the files get re-organized
 .globalQueryBar {
  padding: 0 $euiSizeS $euiSizeS $euiSizeS;
+  height: 160px;


This line should not be added. This makes dashboard and visualize top nav bar being displayed incorrectly.

abbyhu2000 · 2024-07-23T20:06:10Z

src/plugins/data/public/ui/query_editor/language_switcher.tsx

Is this file needed? This is not used anywhere

abbyhu2000 · 2024-07-23T20:11:41Z

src/plugins/data/public/ui/query_editor/_language_switcher.scss

And this too, it is not used anywhere

…ect#7336) * cherry pick Signed-off-by: Eric <[email protected]> * SQL keyword suggestion Signed-off-by: Eric <[email protected]> * column and values suggestions Signed-off-by: Eric <[email protected]> * adjust values suggestions and ignored token Signed-off-by: Eric <[email protected]> * suggestion type casting Signed-off-by: Eric <[email protected]> * ignore functions Signed-off-by: Eric <[email protected]> * case insensitive Signed-off-by: Eric <[email protected]> * change to use grammar for insensitive case Signed-off-by: Eric <[email protected]> * fix multi-line issue Signed-off-by: Eric <[email protected]> * code cleanup Signed-off-by: Eric <[email protected]> * move ppl out of scope Signed-off-by: Eric <[email protected]> * rename folder and add ignoring rules Signed-off-by: Eric <[email protected]> * resolve type issue Signed-off-by: Eric <[email protected]> * fix ppl suggestion provider issue Signed-off-by: Eric <[email protected]> * remove function suggestion Signed-off-by: Eric <[email protected]> * some code clean up and adding tests Signed-off-by: Eric <[email protected]> * cursor tests Signed-off-by: Eric <[email protected]> * remove testing setup Signed-off-by: Eric <[email protected]> * add changelog Signed-off-by: Eric <[email protected]> * update yarn file Signed-off-by: Eric <[email protected]> * add missing testing library Signed-off-by: Eric <[email protected]> * MDS integration Signed-off-by: Eric <[email protected]> * minor interface change and disable word based suggestion Signed-off-by: Eric <[email protected]> * update ID_LITERAL and recompile grammar Signed-off-by: Eric <[email protected]> * column suggest for agg function Signed-off-by: Eric <[email protected]> * revert version Signed-off-by: Eric <[email protected]> * opensearch sql syntax highlighting Signed-off-by: Eric <[email protected]> * add utility tests Signed-off-by: Eric <[email protected]> * symbol table test Signed-off-by: Eric <[email protected]> * error listener test Signed-off-by: Eric <[email protected]> * parse test Signed-off-by: Eric <[email protected]> * add suggestion provider to single line query editor Signed-off-by: Eric <[email protected]> * remove one test Signed-off-by: Eric <[email protected]> * add connection service type Signed-off-by: Eric <[email protected]> --------- Signed-off-by: Eric <[email protected]> (cherry picked from commit 9348bd4)

* Bump OUI to 1.8.0 (#7363) Signed-off-by: Miki <[email protected]> (cherry picked from commit c1bae75) * [Auto Suggest] OpenSearch SQL autosuggest with ANTLR (#7336) * cherry pick Signed-off-by: Eric <[email protected]> * SQL keyword suggestion Signed-off-by: Eric <[email protected]> * column and values suggestions Signed-off-by: Eric <[email protected]> * adjust values suggestions and ignored token Signed-off-by: Eric <[email protected]> * suggestion type casting Signed-off-by: Eric <[email protected]> * ignore functions Signed-off-by: Eric <[email protected]> * case insensitive Signed-off-by: Eric <[email protected]> * change to use grammar for insensitive case Signed-off-by: Eric <[email protected]> * fix multi-line issue Signed-off-by: Eric <[email protected]> * code cleanup Signed-off-by: Eric <[email protected]> * move ppl out of scope Signed-off-by: Eric <[email protected]> * rename folder and add ignoring rules Signed-off-by: Eric <[email protected]> * resolve type issue Signed-off-by: Eric <[email protected]> * fix ppl suggestion provider issue Signed-off-by: Eric <[email protected]> * remove function suggestion Signed-off-by: Eric <[email protected]> * some code clean up and adding tests Signed-off-by: Eric <[email protected]> * cursor tests Signed-off-by: Eric <[email protected]> * remove testing setup Signed-off-by: Eric <[email protected]> * add changelog Signed-off-by: Eric <[email protected]> * update yarn file Signed-off-by: Eric <[email protected]> * add missing testing library Signed-off-by: Eric <[email protected]> * MDS integration Signed-off-by: Eric <[email protected]> * minor interface change and disable word based suggestion Signed-off-by: Eric <[email protected]> * update ID_LITERAL and recompile grammar Signed-off-by: Eric <[email protected]> * column suggest for agg function Signed-off-by: Eric <[email protected]> * revert version Signed-off-by: Eric <[email protected]> * opensearch sql syntax highlighting Signed-off-by: Eric <[email protected]> * add utility tests Signed-off-by: Eric <[email protected]> * symbol table test Signed-off-by: Eric <[email protected]> * error listener test Signed-off-by: Eric <[email protected]> * parse test Signed-off-by: Eric <[email protected]> * add suggestion provider to single line query editor Signed-off-by: Eric <[email protected]> * remove one test Signed-off-by: Eric <[email protected]> * add connection service type Signed-off-by: Eric <[email protected]> --------- Signed-off-by: Eric <[email protected]> (cherry picked from commit 9348bd4) * Move TopNavLinks to new Nav Bar Discover (#7326) This PR aims to introduce a new Nav bar in Discover that would contain the Date Picker as well as the Navigation links that are currently part of headers. The Navigation Links have been replaced with the corresponding Icons to provide a modern UX. These setting are currently controlled by query:enhancements:enabled Advanced Setting Flag and would be visible when the above flag is turned on. Signed-off-by: Suchit Sahoo <[email protected]> (cherry picked from commit e46e54a) * [navigation-next]fix: breadcrumb issue found in data source management and other similar cases. (#7401) * feat: add scoped breadcrumbs Signed-off-by: SuZhou-Joe <[email protected]> * Changeset file for PR #7401 created/updated * feat: move data source management register logic up a little bit Signed-off-by: SuZhou-Joe <[email protected]> * feat: add unit test Signed-off-by: SuZhou-Joe <[email protected]> * feat: revert the home related change Signed-off-by: SuZhou-Joe <[email protected]> * fix: overview error in all use case when workspace is enabled Signed-off-by: SuZhou-Joe <[email protected]> * fix: update snapshot Signed-off-by: SuZhou-Joe <[email protected]> * Changeset file for PR #7401 created/updated * fix: hide nav groups that should be displayed Signed-off-by: SuZhou-Joe <[email protected]> * feat: hide expand icon in left navigation Signed-off-by: SuZhou-Joe <[email protected]> * feat: update Signed-off-by: SuZhou-Joe <[email protected]> --------- Signed-off-by: SuZhou-Joe <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> Co-authored-by: ZilongX <[email protected]> (cherry picked from commit 3f78c94) * [Bug][Data Source] Move data source manageable feature flag to DSM plugin (#7440) * Move data source manageable feature flag to DSM plugin Signed-off-by: yubonluo <[email protected]> * Changeset file for PR #7440 created/updated --------- Signed-off-by: yubonluo <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> (cherry picked from commit dfcd2e1) * [Discover-next] data set picker (#7426) * update using query manager Signed-off-by: Kawika Avilla <[email protected]> * aggs not working Signed-off-by: Kawika Avilla <[email protected]> * almost there Signed-off-by: Kawika Avilla <[email protected]> * stablish Signed-off-by: Kawika Avilla <[email protected]> * thanks ashwin Signed-off-by: Kawika Avilla <[email protected]> * update ref name Signed-off-by: Kawika Avilla <[email protected]> * fix timefields Signed-off-by: Kawika Avilla <[email protected]> * clean up some console logs Signed-off-by: Kawika Avilla <[email protected]> * go safer route of setting language Signed-off-by: Kawika Avilla <[email protected]> * its working again Signed-off-by: Kawika Avilla <[email protected]> * fix names Signed-off-by: Kawika Avilla <[email protected]> * restore code editor and indices Signed-off-by: Kawika Avilla <[email protected]> * sql df Signed-off-by: Kawika Avilla <[email protected]> * fix external datasources again Signed-off-by: Kawika Avilla <[email protected]> --------- Signed-off-by: Kawika Avilla <[email protected]> (cherry picked from commit 6a079d3) * Fix add filter for numeric scripted field (#7022) * Fix add filter for scripted field Signed-off-by: abbyhu2000 <[email protected]> * add unit test Signed-off-by: abbyhu2000 <[email protected]> * change from nested ternary to if else Signed-off-by: abbyhu2000 <[email protected]> * add more unit tests Signed-off-by: abbyhu2000 <[email protected]> * change unit tests Signed-off-by: abbyhu2000 <[email protected]> * fix typo Signed-off-by: abbyhu2000 <[email protected]> --------- Signed-off-by: abbyhu2000 <[email protected]> (cherry picked from commit 7c1f8fc) * [Bug][Workspace] Add permission validation at workspace detail page (#7435) * [Bug][Workspace] Add permission validate at workspace detail page Signed-off-by: yubonluo <[email protected]> * Changeset file for PR #7435 created/updated --------- Signed-off-by: yubonluo <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> (cherry picked from commit 45d34b5) * Revert "[Discover-next] data set picker (#7426)" (#7479) This reverts commit 6a079d3. --------- Co-authored-by: Miki <[email protected]> Co-authored-by: Eric Wei <[email protected]> Co-authored-by: SuZhou-Joe <[email protected]> Co-authored-by: yuboluo <[email protected]> Co-authored-by: Kawika Avilla <[email protected]> Co-authored-by: Qingyang(Abby) Hu <[email protected]> Co-authored-by: Sean Li <[email protected]>

* Bump OUI to 1.8.0 (#7363) Signed-off-by: Miki <[email protected]> (cherry picked from commit c1bae75) * [Auto Suggest] OpenSearch SQL autosuggest with ANTLR (#7336) * cherry pick Signed-off-by: Eric <[email protected]> * SQL keyword suggestion Signed-off-by: Eric <[email protected]> * column and values suggestions Signed-off-by: Eric <[email protected]> * adjust values suggestions and ignored token Signed-off-by: Eric <[email protected]> * suggestion type casting Signed-off-by: Eric <[email protected]> * ignore functions Signed-off-by: Eric <[email protected]> * case insensitive Signed-off-by: Eric <[email protected]> * change to use grammar for insensitive case Signed-off-by: Eric <[email protected]> * fix multi-line issue Signed-off-by: Eric <[email protected]> * code cleanup Signed-off-by: Eric <[email protected]> * move ppl out of scope Signed-off-by: Eric <[email protected]> * rename folder and add ignoring rules Signed-off-by: Eric <[email protected]> * resolve type issue Signed-off-by: Eric <[email protected]> * fix ppl suggestion provider issue Signed-off-by: Eric <[email protected]> * remove function suggestion Signed-off-by: Eric <[email protected]> * some code clean up and adding tests Signed-off-by: Eric <[email protected]> * cursor tests Signed-off-by: Eric <[email protected]> * remove testing setup Signed-off-by: Eric <[email protected]> * add changelog Signed-off-by: Eric <[email protected]> * update yarn file Signed-off-by: Eric <[email protected]> * add missing testing library Signed-off-by: Eric <[email protected]> * MDS integration Signed-off-by: Eric <[email protected]> * minor interface change and disable word based suggestion Signed-off-by: Eric <[email protected]> * update ID_LITERAL and recompile grammar Signed-off-by: Eric <[email protected]> * column suggest for agg function Signed-off-by: Eric <[email protected]> * revert version Signed-off-by: Eric <[email protected]> * opensearch sql syntax highlighting Signed-off-by: Eric <[email protected]> * add utility tests Signed-off-by: Eric <[email protected]> * symbol table test Signed-off-by: Eric <[email protected]> * error listener test Signed-off-by: Eric <[email protected]> * parse test Signed-off-by: Eric <[email protected]> * add suggestion provider to single line query editor Signed-off-by: Eric <[email protected]> * remove one test Signed-off-by: Eric <[email protected]> * add connection service type Signed-off-by: Eric <[email protected]> --------- Signed-off-by: Eric <[email protected]> (cherry picked from commit 9348bd4) * Move TopNavLinks to new Nav Bar Discover (#7326) This PR aims to introduce a new Nav bar in Discover that would contain the Date Picker as well as the Navigation links that are currently part of headers. The Navigation Links have been replaced with the corresponding Icons to provide a modern UX. These setting are currently controlled by query:enhancements:enabled Advanced Setting Flag and would be visible when the above flag is turned on. Signed-off-by: Suchit Sahoo <[email protected]> (cherry picked from commit e46e54a) * [navigation-next]fix: breadcrumb issue found in data source management and other similar cases. (#7401) * feat: add scoped breadcrumbs Signed-off-by: SuZhou-Joe <[email protected]> * Changeset file for PR #7401 created/updated * feat: move data source management register logic up a little bit Signed-off-by: SuZhou-Joe <[email protected]> * feat: add unit test Signed-off-by: SuZhou-Joe <[email protected]> * feat: revert the home related change Signed-off-by: SuZhou-Joe <[email protected]> * fix: overview error in all use case when workspace is enabled Signed-off-by: SuZhou-Joe <[email protected]> * fix: update snapshot Signed-off-by: SuZhou-Joe <[email protected]> * Changeset file for PR #7401 created/updated * fix: hide nav groups that should be displayed Signed-off-by: SuZhou-Joe <[email protected]> * feat: hide expand icon in left navigation Signed-off-by: SuZhou-Joe <[email protected]> * feat: update Signed-off-by: SuZhou-Joe <[email protected]> --------- Signed-off-by: SuZhou-Joe <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> Co-authored-by: ZilongX <[email protected]> (cherry picked from commit 3f78c94) * [Bug][Data Source] Move data source manageable feature flag to DSM plugin (#7440) * Move data source manageable feature flag to DSM plugin Signed-off-by: yubonluo <[email protected]> * Changeset file for PR #7440 created/updated --------- Signed-off-by: yubonluo <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> (cherry picked from commit dfcd2e1) * [Discover-next] data set picker (#7426) * update using query manager Signed-off-by: Kawika Avilla <[email protected]> * aggs not working Signed-off-by: Kawika Avilla <[email protected]> * almost there Signed-off-by: Kawika Avilla <[email protected]> * stablish Signed-off-by: Kawika Avilla <[email protected]> * thanks ashwin Signed-off-by: Kawika Avilla <[email protected]> * update ref name Signed-off-by: Kawika Avilla <[email protected]> * fix timefields Signed-off-by: Kawika Avilla <[email protected]> * clean up some console logs Signed-off-by: Kawika Avilla <[email protected]> * go safer route of setting language Signed-off-by: Kawika Avilla <[email protected]> * its working again Signed-off-by: Kawika Avilla <[email protected]> * fix names Signed-off-by: Kawika Avilla <[email protected]> * restore code editor and indices Signed-off-by: Kawika Avilla <[email protected]> * sql df Signed-off-by: Kawika Avilla <[email protected]> * fix external datasources again Signed-off-by: Kawika Avilla <[email protected]> --------- Signed-off-by: Kawika Avilla <[email protected]> (cherry picked from commit 6a079d3) * Fix add filter for numeric scripted field (#7022) * Fix add filter for scripted field Signed-off-by: abbyhu2000 <[email protected]> * add unit test Signed-off-by: abbyhu2000 <[email protected]> * change from nested ternary to if else Signed-off-by: abbyhu2000 <[email protected]> * add more unit tests Signed-off-by: abbyhu2000 <[email protected]> * change unit tests Signed-off-by: abbyhu2000 <[email protected]> * fix typo Signed-off-by: abbyhu2000 <[email protected]> --------- Signed-off-by: abbyhu2000 <[email protected]> (cherry picked from commit 7c1f8fc) * [Bug][Workspace] Add permission validation at workspace detail page (#7435) * [Bug][Workspace] Add permission validate at workspace detail page Signed-off-by: yubonluo <[email protected]> * Changeset file for PR #7435 created/updated --------- Signed-off-by: yubonluo <[email protected]> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> (cherry picked from commit 45d34b5) * Revert "[Discover-next] data set picker (#7426)" (#7479) This reverts commit 6a079d3. --------- Co-authored-by: Miki <[email protected]> Co-authored-by: Eric Wei <[email protected]> Co-authored-by: SuZhou-Joe <[email protected]> Co-authored-by: yuboluo <[email protected]> Co-authored-by: Kawika Avilla <[email protected]> Co-authored-by: Qingyang(Abby) Hu <[email protected]> Co-authored-by: Sean Li <[email protected]> (cherry picked from commit 0515fb2) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

opensearch-trigger-bot · 2024-07-25T20:48:39Z

The backport to 2.16 failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.16 2.16
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.16
# Create a new branch
git switch --create backport/backport-7336-to-2.16
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9348bd4276a6daf4a719f3709c3d854e002350e0
# Push it to GitHub
git push --set-upstream origin backport/backport-7336-to-2.16
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.16

Then, create a pull request where the base branch is 2.16 and the compare/head branch is backport/backport-7336-to-2.16.

* Bump OUI to 1.8.0 (#7363) (cherry picked from commit c1bae75) * [Auto Suggest] OpenSearch SQL autosuggest with ANTLR (#7336) * cherry pick * SQL keyword suggestion * column and values suggestions * adjust values suggestions and ignored token * suggestion type casting * ignore functions * case insensitive * change to use grammar for insensitive case * fix multi-line issue * code cleanup * move ppl out of scope * rename folder and add ignoring rules * resolve type issue * fix ppl suggestion provider issue * remove function suggestion * some code clean up and adding tests * cursor tests * remove testing setup * add changelog * update yarn file * add missing testing library * MDS integration * minor interface change and disable word based suggestion * update ID_LITERAL and recompile grammar * column suggest for agg function * revert version * opensearch sql syntax highlighting * add utility tests * symbol table test * error listener test * parse test * add suggestion provider to single line query editor * remove one test * add connection service type --------- (cherry picked from commit 9348bd4) * Move TopNavLinks to new Nav Bar Discover (#7326) This PR aims to introduce a new Nav bar in Discover that would contain the Date Picker as well as the Navigation links that are currently part of headers. The Navigation Links have been replaced with the corresponding Icons to provide a modern UX. These setting are currently controlled by query:enhancements:enabled Advanced Setting Flag and would be visible when the above flag is turned on. (cherry picked from commit e46e54a) * [navigation-next]fix: breadcrumb issue found in data source management and other similar cases. (#7401) * feat: add scoped breadcrumbs * Changeset file for PR #7401 created/updated * feat: move data source management register logic up a little bit * feat: add unit test * feat: revert the home related change * fix: overview error in all use case when workspace is enabled * fix: update snapshot * Changeset file for PR #7401 created/updated * fix: hide nav groups that should be displayed * feat: hide expand icon in left navigation * feat: update --------- (cherry picked from commit 3f78c94) * [Bug][Data Source] Move data source manageable feature flag to DSM plugin (#7440) * Move data source manageable feature flag to DSM plugin * Changeset file for PR #7440 created/updated --------- (cherry picked from commit dfcd2e1) * [Discover-next] data set picker (#7426) * update using query manager * aggs not working * almost there * stablish * thanks ashwin * update ref name * fix timefields * clean up some console logs * go safer route of setting language * its working again * fix names * restore code editor and indices * sql df * fix external datasources again --------- (cherry picked from commit 6a079d3) * Fix add filter for numeric scripted field (#7022) * Fix add filter for scripted field * add unit test * change from nested ternary to if else * add more unit tests * change unit tests * fix typo --------- (cherry picked from commit 7c1f8fc) * [Bug][Workspace] Add permission validation at workspace detail page (#7435) * [Bug][Workspace] Add permission validate at workspace detail page * Changeset file for PR #7435 created/updated --------- (cherry picked from commit 45d34b5) * Revert "[Discover-next] data set picker (#7426)" (#7479) This reverts commit 6a079d3. --------- (cherry picked from commit 0515fb2) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Miki <[email protected]> Co-authored-by: Eric Wei <[email protected]> Co-authored-by: SuZhou-Joe <[email protected]> Co-authored-by: yuboluo <[email protected]> Co-authored-by: Kawika Avilla <[email protected]> Co-authored-by: Qingyang(Abby) Hu <[email protected]> Co-authored-by: Sean Li <[email protected]>

AMoo-Miki · 2024-09-13T15:51:55Z

This was manually backported to 2.x with #7477 and to 2.16 with #7487.

mengweieric added 19 commits July 17, 2024 18:17

cherry pick

df0a054

Signed-off-by: Eric <[email protected]>

SQL keyword suggestion

e5175af

Signed-off-by: Eric <[email protected]>

column and values suggestions

d5daf39

Signed-off-by: Eric <[email protected]>

adjust values suggestions and ignored token

f0de90e

Signed-off-by: Eric <[email protected]>

suggestion type casting

be42ea2

Signed-off-by: Eric <[email protected]>

ignore functions

f215024

Signed-off-by: Eric <[email protected]>

case insensitive

2a452e9

Signed-off-by: Eric <[email protected]>

change to use grammar for insensitive case

7ae59f4

Signed-off-by: Eric <[email protected]>

fix multi-line issue

64c266d

Signed-off-by: Eric <[email protected]>

code cleanup

7c62bf7

Signed-off-by: Eric <[email protected]>

move ppl out of scope

c9da113

Signed-off-by: Eric <[email protected]>

rename folder and add ignoring rules

140114a

Signed-off-by: Eric <[email protected]>

resolve type issue

f1014b7

Signed-off-by: Eric <[email protected]>

fix ppl suggestion provider issue

152a645

Signed-off-by: Eric <[email protected]>

remove function suggestion

79e7782

Signed-off-by: Eric <[email protected]>

some code clean up and adding tests

4d84d3b

Signed-off-by: Eric <[email protected]>

sync main and resolve conflicts

609a2a9

Signed-off-by: Eric <[email protected]>

cursor tests

ce01e2c

Signed-off-by: Eric <[email protected]>

remove testing setup

d7e3bd4

Signed-off-by: Eric <[email protected]>

github-actions bot added the valued-contributor label Jul 20, 2024

github-actions bot added the failed changeset label Jul 20, 2024

mengweieric added 2 commits July 20, 2024 22:21

add changelog

3aa0dff

Signed-off-by: Eric <[email protected]>

update yarn file

d1e2540

Signed-off-by: Eric <[email protected]>

mengweieric added 5 commits July 21, 2024 00:05

add missing testing library

a5d12e5

Signed-off-by: Eric <[email protected]>

MDS integration

a606aa8

Signed-off-by: Eric <[email protected]>

Merge branch 'main' into feature/antlr-autocomplete

12715a2

minor interface change and disable word based suggestion

4a8300b

Signed-off-by: Eric <[email protected]>

update ID_LITERAL and recompile grammar

9cb77b8

Signed-off-by: Eric <[email protected]>

mengweieric added 5 commits July 22, 2024 22:21

parse test

9009922

Signed-off-by: Eric <[email protected]>

add suggestion provider to single line query editor

f28f558

Signed-off-by: Eric <[email protected]>

remove one test

1f9080b

Signed-off-by: Eric <[email protected]>

add connection service type

2fa1dfb

Signed-off-by: Eric <[email protected]>

Merge branch 'main' into feature/antlr-autocomplete

3e5b430

joshuali925 approved these changes Jul 23, 2024

View reviewed changes

RyanL1997 approved these changes Jul 23, 2024

View reviewed changes

BionIT approved these changes Jul 23, 2024

View reviewed changes

RyanL1997 approved these changes Jul 23, 2024

View reviewed changes

zhongnansu approved these changes Jul 23, 2024

View reviewed changes

mengweieric added the backport 2.16 label Jul 23, 2024

mengweieric merged commit 9348bd4 into opensearch-project:main Jul 23, 2024
65 of 69 checks passed

opensearch-trigger-bot bot added the failed backport label Jul 23, 2024

abbyhu2000 reviewed Jul 23, 2024

View reviewed changes

src/plugins/data/public/ui/query_editor/language_switcher.tsx

Copy link

Member

abbyhu2000 Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file needed? This is not used anywhere

abbyhu2000 reviewed Jul 23, 2024

View reviewed changes

src/plugins/data/public/ui/query_editor/_language_switcher.scss

Copy link

Member

abbyhu2000 Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this too, it is not used anywhere

mengweieric added backport 2.16 and removed backport 2.16 labels Jul 25, 2024

AMoo-Miki removed the failed backport label Sep 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Auto Suggest] OpenSearch SQL autosuggest with ANTLR #7336

[Auto Suggest] OpenSearch SQL autosuggest with ANTLR #7336

mengweieric commented Jul 20, 2024 •

edited

Loading

github-actions bot commented Jul 20, 2024

codecov bot commented Jul 20, 2024 •

edited

Loading

mengweieric commented Jul 23, 2024

RyanL1997 Jul 23, 2024

mengweieric Jul 23, 2024

RyanL1997 Jul 23, 2024

RyanL1997 Jul 23, 2024

mengweieric Jul 23, 2024

RyanL1997 Jul 23, 2024

mengweieric Jul 23, 2024

RyanL1997 Jul 23, 2024

mengweieric Jul 23, 2024

zhongnansu commented Jul 23, 2024

mengweieric commented Jul 23, 2024

opensearch-trigger-bot bot commented Jul 23, 2024

opensearch-trigger-bot bot commented Jul 23, 2024

abbyhu2000 Jul 23, 2024

abbyhu2000 Jul 23, 2024

abbyhu2000 Jul 23, 2024

opensearch-trigger-bot bot commented Jul 25, 2024

AMoo-Miki commented Sep 13, 2024

[Auto Suggest] OpenSearch SQL autosuggest with ANTLR #7336

[Auto Suggest] OpenSearch SQL autosuggest with ANTLR #7336

Conversation

mengweieric commented Jul 20, 2024 • edited Loading

Description

Issues Resolved

Screenshot

Testing the changes

Changelog

Check List

github-actions bot commented Jul 20, 2024

ℹ️ Manual Changeset Creation Reminder

codecov bot commented Jul 20, 2024 • edited Loading

Codecov Report

mengweieric commented Jul 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhongnansu commented Jul 23, 2024

mengweieric commented Jul 23, 2024

opensearch-trigger-bot bot commented Jul 23, 2024

opensearch-trigger-bot bot commented Jul 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

opensearch-trigger-bot bot commented Jul 25, 2024

AMoo-Miki commented Sep 13, 2024

mengweieric commented Jul 20, 2024 •

edited

Loading

codecov bot commented Jul 20, 2024 •

edited

Loading