[ESQL] Preparation for technical preview #163248
Assignees
Labels
enhancement
New value added to drive a business result
Feature:ES|QL
ES|QL related features in Kibana
Feature:Lens
impact:high
Addressing this issue will have a high level of impact on the quality/strength of our product.
Meta
Team:Visualizations
Visualization editors, elastic-charts and infrastructure
Comments
stratoula
added
enhancement
|
Pinging @elastic/kibana-visualizations @elastic/kibana-visualizations-external (Team:Visualizations) |
3 tasks
stratoula
added a commit
that referenced
this issue
Aug 28, 2023
## Summary Part of #163248 This PR adds the autocomplete feature for the `ENRICH` command:  If there's no available policy saved then a helpful message is shown: <img width="742" alt="Screenshot 2023-08-25 at 17 23 07" src="https://github.com/elastic/kibana/assets/924948/5b24aa26-81a4-4f1b-a257-d78c548f24a6"> Policy suggestion: <img width="744" alt="Screenshot 2023-08-25 at 17 55 10" src="https://github.com/elastic/kibana/assets/924948/c1a6a72f-b261-4234-97a3-a4a7af9c4824"> <img width="1004" alt="Screenshot 2023-08-25 at 17 55 04" src="https://github.com/elastic/kibana/assets/924948/6d82c666-78cd-445f-bcc7-76a9fdaf7523"> Matching field suggestion: <img width="1091" alt="Screenshot 2023-08-25 at 17 55 59" src="https://github.com/elastic/kibana/assets/924948/2b4bbfcd-6e9f-43f3-a8cc-ddae80d20b89"> <img width="828" alt="Screenshot 2023-08-25 at 17 55 54" src="https://github.com/elastic/kibana/assets/924948/06745377-8f2e-480f-94cb-5a612974da7e"> Some UX notes: * Policy suggestions will show the list of indices defined as info ✅ (`Policy defined on index: ...`) * after the `ON` command the list of available fields is shown and the matching field defined in the chosen policy is shown (`Use to match on <matching_field> on the policy`) ✅ * after the `WITH` command both a new variable is shown and the list of policy enriched fields ✅ * currently there's no way to detect whether an `enrichIdentifier` is a variable or a `policyEnrichedField` so both an assign and a pipe command is suggested after an identifier⚠️ **This will be merged in the feature branch** **To test** Create a policy based on the flights index: ``` PUT /_enrich/policy/my-policy { "match": { "indices": "kibana_sample_data_flights", "match_field": "OriginCountry", "enrich_fields": ["Carrier"] } } PUT /_enrich/policy/my-policy/_execute ``` Now you're all set to use the `enrich` command in ES|QL. Make sure to use another dataView who has a country code field to match the policy one. ### Checklist Delete any items that are not applicable to this PR. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Stratoula Kalafateli <[email protected]>
9 tasks
dej611
added a commit
that referenced
this issue
Aug 29, 2023
## Summary Fixes #163248 Provides better UX for partial autocomplete.  ### Checklist Delete any items that are not applicable to this PR. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/)) - [ ] Any UI touched in this PR does not create any new axe failures (run axe in browser: [FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/), [Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US)) - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server)) - [ ] This was checked for [cross-browser compatibility](https://www.elastic.co/support/matrix#matrix_browsers) ### Risk Matrix Delete this section if it is not applicable to this PR. Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release. When forming the risk matrix, consider some of the following examples and how they may potentially impact the change: | Risk | Probability | Severity | Mitigation/Notes | |---------------------------|-------------|----------|-------------------------| | Multiple Spaces—unexpected behavior in non-default Kibana Space. | Low | High | Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces. | | Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks. | High | Low | Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure. | | Code should gracefully handle cases when feature X or plugin Y are disabled. | Medium | High | Unit tests will verify that any feature flag or plugin combination still results in our service operational. | | [See more potential risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) | ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
3 tasks
stratoula
added a commit
that referenced
this issue
Sep 1, 2023
## Summary Closes #137810 Part of #163248 This PR is the **first iteration** of the new ES|QL language in kibana. The majority of the functionality is based on the existing functionality for SQL (which has already been merged). This builds on top of it to enable the functionality for ES|QL. We decided to remove SQL for now so this PR enables ES|QL and hides SQL. We are not removing the code for now. ### Important notes: - This PR contains basic autocomplete functionality for the ES|QL commands. We want to improve it in follow up PRs - The majority of the tests for SQL were moved to work with ES|QL instead - The search strategy is a very simple endpoint for now as we don't have async search nor pagination - Now that we remove SQL, the ui for ES|QL selection has changed - The documentation for ESQL has been handled by the docs team so it is already reviewed and in sync with the official documentation - ES|QL is disabled in serverless projects for now ### Changes from SQL: - The Discover histogram now is being created with ES|QL (using the date_trunc function). This gives it the ability to be saved on a dashboard and also to be edited inline. - ES|QL sometimes returns some warnings (on the search headers). For example when we are trying to date parse a string that doesnt contain a valid date). These warnings are also reported on the UI. There is a bug in ES and the warning doesnt come always elastic/elasticsearch-internal#1465) - We have 2 types of commands: - Transformational commands (keep, stats) -> when they exist in the query Discover should render the Table view (selected columns) - Non transformational commands (all the others) -> in that case Discover renders the Document view - ESQL switch on advanced settings is now on by default **Discover view with non transformational commands** <img width="1678" alt="image" src="https://github.com/elastic/kibana/assets/17003240/abe100e1-01e9-4fe0-9b89-6d8bdf6443fc"> **Discover view with transformational commands** <img width="1679" alt="image" src="https://github.com/elastic/kibana/assets/17003240/e46af422-daeb-4be5-88cf-522211674ff5"> ### Missing - ESQL autocomplete is not perfect, we are going to work on it on a future PR. Specifically: - There is a bug in autocomplete, sometimes writing a query overwrites the existing one, will deal with it in a future PR - Further improvements ### Checklist - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Alexey Antonov <[email protected]> Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Abdon Pijpelink <[email protected]> Co-authored-by: gchaps <[email protected]> Co-authored-by: Peter Pisljar <[email protected]> Co-authored-by: Marco Liberati <[email protected]> Co-authored-by: Matthias Wilhelm <[email protected]> Co-authored-by: Davis McPhee <[email protected]>
|
I am closing this as the issues described above have been done. I moved the dashboard tasks to another issue for 8.12 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the feature:
8.11 is going to be the tech preview release of ESQL. We want to:
Remove SQL, small changes in Unified search UI (@stratoula ) blocked by ES should be transferred to main Integrate Elasticsearch Query Language, ES|QL elasticsearch#98309
Add the enrich policies in autocomplete => [ES|QL] Add autocomplete for enrich command #164865
blocked byuse [Index Management] Add enrich policies fetch api and expose in plugin api #163556Make the antlr autosuggestions easier to use ( @dej611 )
The text was updated successfully, but these errors were encountered: