Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Controls] Support Numeric Fields in Options List #126795

Closed
ThomThomson opened this issue Mar 3, 2022 · 4 comments · Fixed by #172106
Closed

[Controls] Support Numeric Fields in Options List #126795

ThomThomson opened this issue Mar 3, 2022 · 4 comments · Fixed by #172106
Assignees
@Heenawter
Labels
Feature:Dashboard Dashboard related features Feature:Input Control Input controls visualization impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. loe:medium Medium Level of Effort Project:Controls Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas

Comments

@ThomThomson
Copy link
Contributor

ThomThomson commented Mar 3, 2022 •
edited
Loading

Blocked by #143587

Describe the feature: The Options List Control should support numeric fields.

Describe a specific use case for the feature: Some users use numeric fields for transit numbers, or id numbers. Because these values are distinct, a range slider doesn't completely cover the use case, and an Options List control should be used. Currently you cannot use an Option List Control with a numeric field at all.

Additionally, using a terms aggregation on a number field like this is supported and should result in numerically sorted options.
@ThomThomson ThomThomson added Feature:Dashboard Dashboard related features Feature:Input Control Input controls visualization Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas loe:needs-research This issue requires some research before it can be worked on or estimated impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. Project:Controls labels Mar 3, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-presentation (Team:Presentation)

@ThomThomson ThomThomson mentioned this issue Mar 3, 2022
Closed
23 tasks
@ThomThomson ThomThomson mentioned this issue Jun 20, 2022
Closed
@ThomThomson ThomThomson changed the title [Dashboard][Controls] Support Numeric Fields and field formatters in the Options List [Dashboard][Controls] Support Numeric Fields and Numeric Sorting in Options List Jun 20, 2022
@jack-morrison
Copy link

+1, I would be very happy to see numeric sorted options in Controls! This is a case I use quite a lot with unique ID numbers not suited for a range slider.

@ThomThomson ThomThomson changed the title [Dashboard][Controls] Support Numeric Fields and Numeric Sorting in Options List [Controls] Support Numeric Fields in Options List Oct 18, 2022
@ThomThomson ThomThomson mentioned this issue Oct 18, 2022
Closed
@Heenawter Heenawter mentioned this issue Jan 26, 2023
Closed
@nreese nreese mentioned this issue Feb 16, 2023
Closed
@Heenawter Heenawter mentioned this issue Feb 23, 2023
Closed
@Heenawter Heenawter added impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. and removed impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. labels Mar 7, 2023
@Heenawter Heenawter mentioned this issue Mar 15, 2023
Closed
@ThomThomson ThomThomson mentioned this issue Apr 5, 2023
Closed
@Heenawter Heenawter mentioned this issue Apr 25, 2023
Merged
3 tasks
Heenawter added a commit that referenced this issue Apr 26, 2023
@Heenawter
[Controls] Fix sorting of numeric keyword fields (#155207)
b3f65f7 
Closes #155073

## Summary

### Before

Previously, the options list suggestions were stored as a dictionary
(i.e. an object of key+value pairs) - while this worked for most fields,
unbeknownst to us, Javascript tries to sort numeric keys (regardless of
if they are of type `string` or `number`) based on their value.

This meant that, as part of the parsing process when using an options
list control for a numeric `keyword` field, the results returned by the
ES query were **always** sorted in ascending numeric order regardless of
the sorting method that was picked (note that this is especially obvious
once you "load more", which is what I did for the following
screenshots):


|                        | Ascending | Descending |
|--------------|-----------|------------|
| Alphabetical | <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234391308-6d3a23ee-3495-4eff-810f-216f758b3a58.png"/>
| <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234391213-117163e2-ee97-4f9d-87fa-a63c8cc5459e.png"/>
|
| Doc count | <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234391375-0ccdf72f-83c0-4a87-951e-c2e1e3223006.png"/>
| <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234392997-fea42ffe-5d9d-4a11-968f-e1503f2c0e4f.png"/>
|


### After

This PR converts the options list suggestions to be stored as an
**array** of key/value pairs in order to preserve the order returned
from Elasticsearch - now, you get the expected string-sorted ordering
when using numeric `keyword` fields in an options list control:

|                        | Ascending | Descending |
|--------------|-----------|------------|
| Alphabetical | <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234394182-aa2bfdf4-fe41-441d-bdbf-917173c17627.png"/>
| <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234393421-24cca3e3-0249-4607-9e16-daa274399bdd.png"/>
|
| Doc count | <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234394599-dda01056-5446-497e-abe4-f3839aeb4dd0.png"/>
| <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234394693-42544ef1-eb2b-4d52-8a78-c2ca7d2d7cfa.png"/>
|


Notice in the above that we are now using **string sorting** for the
numeric values when alphabetical sorting is selected, which means you
aren't getting the expected "numeric" sorting - so for example, when
sorted ascending, `"6" > "52"` because it is only comparing the first
character and `"6" > "5"`. This will be handled much better once
[numeric field support](#126795)
is added to options lists.


### Checklist


- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)


### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Heenawter added a commit to Heenawter/kibana that referenced this issue Apr 26, 2023
@Heenawter
[Controls] Fix sorting of numeric keyword fields (elastic#155207)
8b741d6 
Closes elastic#155073

## Summary

### Before

Previously, the options list suggestions were stored as a dictionary
(i.e. an object of key+value pairs) - while this worked for most fields,
unbeknownst to us, Javascript tries to sort numeric keys (regardless of
if they are of type `string` or `number`) based on their value.

This meant that, as part of the parsing process when using an options
list control for a numeric `keyword` field, the results returned by the
ES query were **always** sorted in ascending numeric order regardless of
the sorting method that was picked (note that this is especially obvious
once you "load more", which is what I did for the following
screenshots):

|                        | Ascending | Descending |
|--------------|-----------|------------|
| Alphabetical | <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234391308-6d3a23ee-3495-4eff-810f-216f758b3a58.png"/>
| <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234391213-117163e2-ee97-4f9d-87fa-a63c8cc5459e.png"/>
|
| Doc count | <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234391375-0ccdf72f-83c0-4a87-951e-c2e1e3223006.png"/>
| <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234392997-fea42ffe-5d9d-4a11-968f-e1503f2c0e4f.png"/>
|

### After

This PR converts the options list suggestions to be stored as an
**array** of key/value pairs in order to preserve the order returned
from Elasticsearch - now, you get the expected string-sorted ordering
when using numeric `keyword` fields in an options list control:

|                        | Ascending | Descending |
|--------------|-----------|------------|
| Alphabetical | <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234394182-aa2bfdf4-fe41-441d-bdbf-917173c17627.png"/>
| <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234393421-24cca3e3-0249-4607-9e16-daa274399bdd.png"/>
|
| Doc count | <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234394599-dda01056-5446-497e-abe4-f3839aeb4dd0.png"/>
| <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234394693-42544ef1-eb2b-4d52-8a78-c2ca7d2d7cfa.png"/>
|

Notice in the above that we are now using **string sorting** for the
numeric values when alphabetical sorting is selected, which means you
aren't getting the expected "numeric" sorting - so for example, when
sorted ascending, `"6" > "52"` because it is only comparing the first
character and `"6" > "5"`. This will be handled much better once
[numeric field support](elastic#126795)
is added to options lists.

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

(cherry picked from commit b3f65f7)

# Conflicts:
#	src/plugins/controls/public/options_list/embeddable/options_list_embeddable.tsx
Heenawter referenced this issue Apr 26, 2023
@Heenawter
[8.7] [Controls] Fix sorting of numeric keyword fields (#155207) (#15…
167c328 
…5934)

# Backport

This will backport the following commits from `main` to `8.7`:
- [[Controls] Fix sorting of numeric keyword fields
(#155207)](#155207)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Hannah
Mudge","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-04-26T18:19:46Z","message":"[Controls]
Fix sorting of numeric keyword fields (#155207)\n\nCloses
https://github.com/elastic/kibana/issues/155073\r\n\r\n##
Summary\r\n\r\n### Before\r\n\r\nPreviously, the options list
suggestions were stored as a dictionary\r\n(i.e. an object of key+value
pairs) - while this worked for most fields,\r\nunbeknownst to us,
Javascript tries to sort numeric keys (regardless of\r\nif they are of
type `string` or `number`) based on their value.\r\n\r\nThis meant that,
as part of the parsing process when using an options\r\nlist control for
a numeric `keyword` field, the results returned by the\r\nES query were
**always** sorted in ascending numeric order regardless of\r\nthe
sorting method that was picked (note that this is especially
obvious\r\nonce you \"load more\", which is what I did for the
following\r\nscreenshots):\r\n\r\n\r\n| | Ascending | Descending
|\r\n|--------------|-----------|------------|\r\n| Alphabetical | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391308-6d3a23ee-3495-4eff-810f-216f758b3a58.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391213-117163e2-ee97-4f9d-87fa-a63c8cc5459e.png\"/>\r\n|\r\n|
Doc count | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391375-0ccdf72f-83c0-4a87-951e-c2e1e3223006.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234392997-fea42ffe-5d9d-4a11-968f-e1503f2c0e4f.png\"/>\r\n|\r\n\r\n\r\n###
After\r\n\r\nThis PR converts the options list suggestions to be stored
as an\r\n**array** of key/value pairs in order to preserve the order
returned\r\nfrom Elasticsearch - now, you get the expected string-sorted
ordering\r\nwhen using numeric `keyword` fields in an options list
control:\r\n\r\n| | Ascending | Descending
|\r\n|--------------|-----------|------------|\r\n| Alphabetical | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394182-aa2bfdf4-fe41-441d-bdbf-917173c17627.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234393421-24cca3e3-0249-4607-9e16-daa274399bdd.png\"/>\r\n|\r\n|
Doc count | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394599-dda01056-5446-497e-abe4-f3839aeb4dd0.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394693-42544ef1-eb2b-4d52-8a78-c2ca7d2d7cfa.png\"/>\r\n|\r\n\r\n\r\nNotice
in the above that we are now using **string sorting** for the\r\nnumeric
values when alphabetical sorting is selected, which means you\r\naren't
getting the expected \"numeric\" sorting - so for example,
when\r\nsorted ascending, `\"6\" > \"52\"` because it is only comparing
the first\r\ncharacter and `\"6\" > \"5\"`. This will be handled much
better once\r\n[numeric field
support](https://github.com/elastic/kibana/issues/126795)\r\nis added to
options lists.\r\n\r\n\r\n### Checklist\r\n\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [x] This was
checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
For maintainers\r\n\r\n- [ ] This was checked for breaking API changes
and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"b3f65f79e5017b70fe26e5aa1c2ee1085e68c138","branchLabelMapping":{"^v8.8.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","Feature:Input
Control","Team:Presentation","loe:days","impact:medium","backport:prev-minor","v8.8.0","v8.9.0"],"number":155207,"url":"https://github.com/elastic/kibana/pull/155207","mergeCommit":{"message":"[Controls]
Fix sorting of numeric keyword fields (#155207)\n\nCloses
https://github.com/elastic/kibana/issues/155073\r\n\r\n##
Summary\r\n\r\n### Before\r\n\r\nPreviously, the options list
suggestions were stored as a dictionary\r\n(i.e. an object of key+value
pairs) - while this worked for most fields,\r\nunbeknownst to us,
Javascript tries to sort numeric keys (regardless of\r\nif they are of
type `string` or `number`) based on their value.\r\n\r\nThis meant that,
as part of the parsing process when using an options\r\nlist control for
a numeric `keyword` field, the results returned by the\r\nES query were
**always** sorted in ascending numeric order regardless of\r\nthe
sorting method that was picked (note that this is especially
obvious\r\nonce you \"load more\", which is what I did for the
following\r\nscreenshots):\r\n\r\n\r\n| | Ascending | Descending
|\r\n|--------------|-----------|------------|\r\n| Alphabetical | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391308-6d3a23ee-3495-4eff-810f-216f758b3a58.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391213-117163e2-ee97-4f9d-87fa-a63c8cc5459e.png\"/>\r\n|\r\n|
Doc count | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391375-0ccdf72f-83c0-4a87-951e-c2e1e3223006.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234392997-fea42ffe-5d9d-4a11-968f-e1503f2c0e4f.png\"/>\r\n|\r\n\r\n\r\n###
After\r\n\r\nThis PR converts the options list suggestions to be stored
as an\r\n**array** of key/value pairs in order to preserve the order
returned\r\nfrom Elasticsearch - now, you get the expected string-sorted
ordering\r\nwhen using numeric `keyword` fields in an options list
control:\r\n\r\n| | Ascending | Descending
|\r\n|--------------|-----------|------------|\r\n| Alphabetical | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394182-aa2bfdf4-fe41-441d-bdbf-917173c17627.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234393421-24cca3e3-0249-4607-9e16-daa274399bdd.png\"/>\r\n|\r\n|
Doc count | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394599-dda01056-5446-497e-abe4-f3839aeb4dd0.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394693-42544ef1-eb2b-4d52-8a78-c2ca7d2d7cfa.png\"/>\r\n|\r\n\r\n\r\nNotice
in the above that we are now using **string sorting** for the\r\nnumeric
values when alphabetical sorting is selected, which means you\r\naren't
getting the expected \"numeric\" sorting - so for example,
when\r\nsorted ascending, `\"6\" > \"52\"` because it is only comparing
the first\r\ncharacter and `\"6\" > \"5\"`. This will be handled much
better once\r\n[numeric field
support](https://github.com/elastic/kibana/issues/126795)\r\nis added to
options lists.\r\n\r\n\r\n### Checklist\r\n\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [x] This was
checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
For maintainers\r\n\r\n- [ ] This was checked for breaking API changes
and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"b3f65f79e5017b70fe26e5aa1c2ee1085e68c138"}},"sourceBranch":"main","suggestedTargetBranches":["8.9"],"targetPullRequestStates":[{"branch":"main","label":"v8.8.0","labelRegex":"^v8.8.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/155207","number":155207,"mergeCommit":{"message":"[Controls]
Fix sorting of numeric keyword fields (#155207)\n\nCloses
https://github.com/elastic/kibana/issues/155073\r\n\r\n##
Summary\r\n\r\n### Before\r\n\r\nPreviously, the options list
suggestions were stored as a dictionary\r\n(i.e. an object of key+value
pairs) - while this worked for most fields,\r\nunbeknownst to us,
Javascript tries to sort numeric keys (regardless of\r\nif they are of
type `string` or `number`) based on their value.\r\n\r\nThis meant that,
as part of the parsing process when using an options\r\nlist control for
a numeric `keyword` field, the results returned by the\r\nES query were
**always** sorted in ascending numeric order regardless of\r\nthe
sorting method that was picked (note that this is especially
obvious\r\nonce you \"load more\", which is what I did for the
following\r\nscreenshots):\r\n\r\n\r\n| | Ascending | Descending
|\r\n|--------------|-----------|------------|\r\n| Alphabetical | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391308-6d3a23ee-3495-4eff-810f-216f758b3a58.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391213-117163e2-ee97-4f9d-87fa-a63c8cc5459e.png\"/>\r\n|\r\n|
Doc count | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391375-0ccdf72f-83c0-4a87-951e-c2e1e3223006.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234392997-fea42ffe-5d9d-4a11-968f-e1503f2c0e4f.png\"/>\r\n|\r\n\r\n\r\n###
After\r\n\r\nThis PR converts the options list suggestions to be stored
as an\r\n**array** of key/value pairs in order to preserve the order
returned\r\nfrom Elasticsearch - now, you get the expected string-sorted
ordering\r\nwhen using numeric `keyword` fields in an options list
control:\r\n\r\n| | Ascending | Descending
|\r\n|--------------|-----------|------------|\r\n| Alphabetical | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394182-aa2bfdf4-fe41-441d-bdbf-917173c17627.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234393421-24cca3e3-0249-4607-9e16-daa274399bdd.png\"/>\r\n|\r\n|
Doc count | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394599-dda01056-5446-497e-abe4-f3839aeb4dd0.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394693-42544ef1-eb2b-4d52-8a78-c2ca7d2d7cfa.png\"/>\r\n|\r\n\r\n\r\nNotice
in the above that we are now using **string sorting** for the\r\nnumeric
values when alphabetical sorting is selected, which means you\r\naren't
getting the expected \"numeric\" sorting - so for example,
when\r\nsorted ascending, `\"6\" > \"52\"` because it is only comparing
the first\r\ncharacter and `\"6\" > \"5\"`. This will be handled much
better once\r\n[numeric field
support](https://github.com/elastic/kibana/issues/126795)\r\nis added to
options lists.\r\n\r\n\r\n### Checklist\r\n\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [x] This was
checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
For maintainers\r\n\r\n- [ ] This was checked for breaking API changes
and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"b3f65f79e5017b70fe26e5aa1c2ee1085e68c138"}},{"branch":"8.9","label":"v8.9.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
Heenawter added a commit to Heenawter/kibana that referenced this issue Apr 28, 2023
@Heenawter
[Controls] Fix sorting of numeric keyword fields (elastic#155207)
92362ec 
Closes elastic#155073

## Summary

### Before

Previously, the options list suggestions were stored as a dictionary
(i.e. an object of key+value pairs) - while this worked for most fields,
unbeknownst to us, Javascript tries to sort numeric keys (regardless of
if they are of type `string` or `number`) based on their value.

This meant that, as part of the parsing process when using an options
list control for a numeric `keyword` field, the results returned by the
ES query were **always** sorted in ascending numeric order regardless of
the sorting method that was picked (note that this is especially obvious
once you "load more", which is what I did for the following
screenshots):

|                        | Ascending | Descending |
|--------------|-----------|------------|
| Alphabetical | <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234391308-6d3a23ee-3495-4eff-810f-216f758b3a58.png"/>
| <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234391213-117163e2-ee97-4f9d-87fa-a63c8cc5459e.png"/>
|
| Doc count | <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234391375-0ccdf72f-83c0-4a87-951e-c2e1e3223006.png"/>
| <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234392997-fea42ffe-5d9d-4a11-968f-e1503f2c0e4f.png"/>
|

### After

This PR converts the options list suggestions to be stored as an
**array** of key/value pairs in order to preserve the order returned
from Elasticsearch - now, you get the expected string-sorted ordering
when using numeric `keyword` fields in an options list control:

|                        | Ascending | Descending |
|--------------|-----------|------------|
| Alphabetical | <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234394182-aa2bfdf4-fe41-441d-bdbf-917173c17627.png"/>
| <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234393421-24cca3e3-0249-4607-9e16-daa274399bdd.png"/>
|
| Doc count | <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234394599-dda01056-5446-497e-abe4-f3839aeb4dd0.png"/>
| <img width="320px"
src="https://user-images.githubusercontent.com/8698078/234394693-42544ef1-eb2b-4d52-8a78-c2ca7d2d7cfa.png"/>
|

Notice in the above that we are now using **string sorting** for the
numeric values when alphabetical sorting is selected, which means you
aren't getting the expected "numeric" sorting - so for example, when
sorted ascending, `"6" > "52"` because it is only comparing the first
character and `"6" > "5"`. This will be handled much better once
[numeric field support](elastic#126795)
is added to options lists.

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

(cherry picked from commit b3f65f7)
Heenawter referenced this issue Apr 28, 2023
@Heenawter
[8.8] [Controls] Fix sorting of numeric keyword fields (#155207) (#15…
1d581aa 
…6197)

# Backport

This will backport the following commits from `main` to `8.8`:
- [[Controls] Fix sorting of numeric keyword fields
(#155207)](#155207)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Hannah
Mudge","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-04-26T18:19:46Z","message":"[Controls]
Fix sorting of numeric keyword fields (#155207)\n\nCloses
https://github.com/elastic/kibana/issues/155073\r\n\r\n##
Summary\r\n\r\n### Before\r\n\r\nPreviously, the options list
suggestions were stored as a dictionary\r\n(i.e. an object of key+value
pairs) - while this worked for most fields,\r\nunbeknownst to us,
Javascript tries to sort numeric keys (regardless of\r\nif they are of
type `string` or `number`) based on their value.\r\n\r\nThis meant that,
as part of the parsing process when using an options\r\nlist control for
a numeric `keyword` field, the results returned by the\r\nES query were
**always** sorted in ascending numeric order regardless of\r\nthe
sorting method that was picked (note that this is especially
obvious\r\nonce you \"load more\", which is what I did for the
following\r\nscreenshots):\r\n\r\n\r\n| | Ascending | Descending
|\r\n|--------------|-----------|------------|\r\n| Alphabetical | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391308-6d3a23ee-3495-4eff-810f-216f758b3a58.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391213-117163e2-ee97-4f9d-87fa-a63c8cc5459e.png\"/>\r\n|\r\n|
Doc count | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391375-0ccdf72f-83c0-4a87-951e-c2e1e3223006.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234392997-fea42ffe-5d9d-4a11-968f-e1503f2c0e4f.png\"/>\r\n|\r\n\r\n\r\n###
After\r\n\r\nThis PR converts the options list suggestions to be stored
as an\r\n**array** of key/value pairs in order to preserve the order
returned\r\nfrom Elasticsearch - now, you get the expected string-sorted
ordering\r\nwhen using numeric `keyword` fields in an options list
control:\r\n\r\n| | Ascending | Descending
|\r\n|--------------|-----------|------------|\r\n| Alphabetical | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394182-aa2bfdf4-fe41-441d-bdbf-917173c17627.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234393421-24cca3e3-0249-4607-9e16-daa274399bdd.png\"/>\r\n|\r\n|
Doc count | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394599-dda01056-5446-497e-abe4-f3839aeb4dd0.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394693-42544ef1-eb2b-4d52-8a78-c2ca7d2d7cfa.png\"/>\r\n|\r\n\r\n\r\nNotice
in the above that we are now using **string sorting** for the\r\nnumeric
values when alphabetical sorting is selected, which means you\r\naren't
getting the expected \"numeric\" sorting - so for example,
when\r\nsorted ascending, `\"6\" > \"52\"` because it is only comparing
the first\r\ncharacter and `\"6\" > \"5\"`. This will be handled much
better once\r\n[numeric field
support](https://github.com/elastic/kibana/issues/126795)\r\nis added to
options lists.\r\n\r\n\r\n### Checklist\r\n\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [x] This was
checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
For maintainers\r\n\r\n- [ ] This was checked for breaking API changes
and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"b3f65f79e5017b70fe26e5aa1c2ee1085e68c138","branchLabelMapping":{"^v8.8.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","Feature:Input
Control","Team:Presentation","loe:days","impact:medium","backport:prev-minor","v8.8.0","v8.7.1","v8.9.0"],"number":155207,"url":"https://github.com/elastic/kibana/pull/155207","mergeCommit":{"message":"[Controls]
Fix sorting of numeric keyword fields (#155207)\n\nCloses
https://github.com/elastic/kibana/issues/155073\r\n\r\n##
Summary\r\n\r\n### Before\r\n\r\nPreviously, the options list
suggestions were stored as a dictionary\r\n(i.e. an object of key+value
pairs) - while this worked for most fields,\r\nunbeknownst to us,
Javascript tries to sort numeric keys (regardless of\r\nif they are of
type `string` or `number`) based on their value.\r\n\r\nThis meant that,
as part of the parsing process when using an options\r\nlist control for
a numeric `keyword` field, the results returned by the\r\nES query were
**always** sorted in ascending numeric order regardless of\r\nthe
sorting method that was picked (note that this is especially
obvious\r\nonce you \"load more\", which is what I did for the
following\r\nscreenshots):\r\n\r\n\r\n| | Ascending | Descending
|\r\n|--------------|-----------|------------|\r\n| Alphabetical | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391308-6d3a23ee-3495-4eff-810f-216f758b3a58.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391213-117163e2-ee97-4f9d-87fa-a63c8cc5459e.png\"/>\r\n|\r\n|
Doc count | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391375-0ccdf72f-83c0-4a87-951e-c2e1e3223006.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234392997-fea42ffe-5d9d-4a11-968f-e1503f2c0e4f.png\"/>\r\n|\r\n\r\n\r\n###
After\r\n\r\nThis PR converts the options list suggestions to be stored
as an\r\n**array** of key/value pairs in order to preserve the order
returned\r\nfrom Elasticsearch - now, you get the expected string-sorted
ordering\r\nwhen using numeric `keyword` fields in an options list
control:\r\n\r\n| | Ascending | Descending
|\r\n|--------------|-----------|------------|\r\n| Alphabetical | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394182-aa2bfdf4-fe41-441d-bdbf-917173c17627.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234393421-24cca3e3-0249-4607-9e16-daa274399bdd.png\"/>\r\n|\r\n|
Doc count | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394599-dda01056-5446-497e-abe4-f3839aeb4dd0.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394693-42544ef1-eb2b-4d52-8a78-c2ca7d2d7cfa.png\"/>\r\n|\r\n\r\n\r\nNotice
in the above that we are now using **string sorting** for the\r\nnumeric
values when alphabetical sorting is selected, which means you\r\naren't
getting the expected \"numeric\" sorting - so for example,
when\r\nsorted ascending, `\"6\" > \"52\"` because it is only comparing
the first\r\ncharacter and `\"6\" > \"5\"`. This will be handled much
better once\r\n[numeric field
support](https://github.com/elastic/kibana/issues/126795)\r\nis added to
options lists.\r\n\r\n\r\n### Checklist\r\n\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [x] This was
checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
For maintainers\r\n\r\n- [ ] This was checked for breaking API changes
and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"b3f65f79e5017b70fe26e5aa1c2ee1085e68c138"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.8.0","labelRegex":"^v8.8.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/155207","number":155207,"mergeCommit":{"message":"[Controls]
Fix sorting of numeric keyword fields (#155207)\n\nCloses
https://github.com/elastic/kibana/issues/155073\r\n\r\n##
Summary\r\n\r\n### Before\r\n\r\nPreviously, the options list
suggestions were stored as a dictionary\r\n(i.e. an object of key+value
pairs) - while this worked for most fields,\r\nunbeknownst to us,
Javascript tries to sort numeric keys (regardless of\r\nif they are of
type `string` or `number`) based on their value.\r\n\r\nThis meant that,
as part of the parsing process when using an options\r\nlist control for
a numeric `keyword` field, the results returned by the\r\nES query were
**always** sorted in ascending numeric order regardless of\r\nthe
sorting method that was picked (note that this is especially
obvious\r\nonce you \"load more\", which is what I did for the
following\r\nscreenshots):\r\n\r\n\r\n| | Ascending | Descending
|\r\n|--------------|-----------|------------|\r\n| Alphabetical | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391308-6d3a23ee-3495-4eff-810f-216f758b3a58.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391213-117163e2-ee97-4f9d-87fa-a63c8cc5459e.png\"/>\r\n|\r\n|
Doc count | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234391375-0ccdf72f-83c0-4a87-951e-c2e1e3223006.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234392997-fea42ffe-5d9d-4a11-968f-e1503f2c0e4f.png\"/>\r\n|\r\n\r\n\r\n###
After\r\n\r\nThis PR converts the options list suggestions to be stored
as an\r\n**array** of key/value pairs in order to preserve the order
returned\r\nfrom Elasticsearch - now, you get the expected string-sorted
ordering\r\nwhen using numeric `keyword` fields in an options list
control:\r\n\r\n| | Ascending | Descending
|\r\n|--------------|-----------|------------|\r\n| Alphabetical | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394182-aa2bfdf4-fe41-441d-bdbf-917173c17627.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234393421-24cca3e3-0249-4607-9e16-daa274399bdd.png\"/>\r\n|\r\n|
Doc count | <img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394599-dda01056-5446-497e-abe4-f3839aeb4dd0.png\"/>\r\n|
<img
width=\"320px\"\r\nsrc=\"https://user-images.githubusercontent.com/8698078/234394693-42544ef1-eb2b-4d52-8a78-c2ca7d2d7cfa.png\"/>\r\n|\r\n\r\n\r\nNotice
in the above that we are now using **string sorting** for the\r\nnumeric
values when alphabetical sorting is selected, which means you\r\naren't
getting the expected \"numeric\" sorting - so for example,
when\r\nsorted ascending, `\"6\" > \"52\"` because it is only comparing
the first\r\ncharacter and `\"6\" > \"5\"`. This will be handled much
better once\r\n[numeric field
support](https://github.com/elastic/kibana/issues/126795)\r\nis added to
options lists.\r\n\r\n\r\n### Checklist\r\n\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [x] This was
checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
For maintainers\r\n\r\n- [ ] This was checked for breaking API changes
and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"b3f65f79e5017b70fe26e5aa1c2ee1085e68c138"}},{"branch":"8.7","label":"v8.7.1","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/155934","number":155934,"state":"MERGED","mergeCommit":{"sha":"167c3285d8a94bfce14a7e5e0d89825e38ccef3a","message":"[8.7]
[Controls] Fix sorting of numeric keyword fields (#155207)
(#155934)\n\n# Backport\n\nThis will backport the following commits from
`main` to `8.7`:\n- [[Controls] Fix sorting of numeric keyword
fields\n(#155207)](https://github.com/elastic/kibana/pull/155207)\n\n<!---
Backport version: 8.9.7 -->\n\n### Questions ?\nPlease refer to the
[Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Hannah\nMudge\",\"email\":\"[email protected]\"},\"sourceCommit\":{\"committedDate\":\"2023-04-26T18:19:46Z\",\"message\":\"[Controls]\nFix
sorting of numeric keyword fields
(#155207)\\n\\nCloses\nhttps://github.com//issues/155073\\r\\n\\r\\n##\nSummary\\r\\n\\r\\n###
Before\\r\\n\\r\\nPreviously, the options list\nsuggestions were stored
as a dictionary\\r\\n(i.e. an object of key+value\npairs) - while this
worked for most fields,\\r\\nunbeknownst to us,\nJavascript tries to
sort numeric keys (regardless of\\r\\nif they are of\ntype `string` or
`number`) based on their value.\\r\\n\\r\\nThis meant that,\nas part of
the parsing process when using an options\\r\\nlist control for\na
numeric `keyword` field, the results returned by the\\r\\nES query
were\n**always** sorted in ascending numeric order regardless
of\\r\\nthe\nsorting method that was picked (note that this is
especially\nobvious\\r\\nonce you \\\"load more\\\", which is what I did
for the\nfollowing\\r\\nscreenshots):\\r\\n\\r\\n\\r\\n| | Ascending |
Descending\n|\\r\\n|--------------|-----------|------------|\\r\\n|
Alphabetical |
<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234391308-6d3a23ee-3495-4eff-810f-216f758b3a58.png\\\"/>\\r\\n|\n<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234391213-117163e2-ee97-4f9d-87fa-a63c8cc5459e.png\\\"/>\\r\\n|\\r\\n|\nDoc
count |
<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234391375-0ccdf72f-83c0-4a87-951e-c2e1e3223006.png\\\"/>\\r\\n|\n<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234392997-fea42ffe-5d9d-4a11-968f-e1503f2c0e4f.png\\\"/>\\r\\n|\\r\\n\\r\\n\\r\\n###\nAfter\\r\\n\\r\\nThis
PR converts the options list suggestions to be stored\nas
an\\r\\n**array** of key/value pairs in order to preserve the
order\nreturned\\r\\nfrom Elasticsearch - now, you get the expected
string-sorted\nordering\\r\\nwhen using numeric `keyword` fields in an
options list\ncontrol:\\r\\n\\r\\n| | Ascending |
Descending\n|\\r\\n|--------------|-----------|------------|\\r\\n|
Alphabetical |
<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234394182-aa2bfdf4-fe41-441d-bdbf-917173c17627.png\\\"/>\\r\\n|\n<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234393421-24cca3e3-0249-4607-9e16-daa274399bdd.png\\\"/>\\r\\n|\\r\\n|\nDoc
count |
<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234394599-dda01056-5446-497e-abe4-f3839aeb4dd0.png\\\"/>\\r\\n|\n<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234394693-42544ef1-eb2b-4d52-8a78-c2ca7d2d7cfa.png\\\"/>\\r\\n|\\r\\n\\r\\n\\r\\nNotice\nin
the above that we are now using **string sorting** for
the\\r\\nnumeric\nvalues when alphabetical sorting is selected, which
means you\\r\\naren't\ngetting the expected \\\"numeric\\\" sorting - so
for example,\nwhen\\r\\nsorted ascending, `\\\"6\\\" > \\\"52\\\"`
because it is only comparing\nthe first\\r\\ncharacter and `\\\"6\\\" >
\\\"5\\\"`. This will be handled much\nbetter once\\r\\n[numeric
field\nsupport](https://github.com/elastic/kibana/issues/126795)\\r\\nis
added to\noptions lists.\\r\\n\\r\\n\\r\\n###
Checklist\\r\\n\\r\\n\\r\\n- [x] [Unit
or\nfunctional\\r\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\r\\nwere\nupdated
or added to match the most common scenarios\\r\\n- [x] This was\nchecked
for\n[cross-browser\\r\\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\\r\\n\\r\\n\\r\\n###\nFor
maintainers\\r\\n\\r\\n- [ ] This was checked for breaking API
changes\nand
was\n[labeled\\r\\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\",\"sha\":\"b3f65f79e5017b70fe26e5aa1c2ee1085e68c138\",\"branchLabelMapping\":{\"^v8.8.0$\":\"main\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"bug\",\"release_note:fix\",\"Feature:Input\nControl\",\"Team:Presentation\",\"loe:days\",\"impact:medium\",\"backport:prev-minor\",\"v8.8.0\",\"v8.9.0\"],\"number\":155207,\"url\":\"https://github.com/elastic/kibana/pull/155207\",\"mergeCommit\":{\"message\":\"[Controls]\nFix
sorting of numeric keyword fields
(#155207)\\n\\nCloses\nhttps://github.com//issues/155073\\r\\n\\r\\n##\nSummary\\r\\n\\r\\n###
Before\\r\\n\\r\\nPreviously, the options list\nsuggestions were stored
as a dictionary\\r\\n(i.e. an object of key+value\npairs) - while this
worked for most fields,\\r\\nunbeknownst to us,\nJavascript tries to
sort numeric keys (regardless of\\r\\nif they are of\ntype `string` or
`number`) based on their value.\\r\\n\\r\\nThis meant that,\nas part of
the parsing process when using an options\\r\\nlist control for\na
numeric `keyword` field, the results returned by the\\r\\nES query
were\n**always** sorted in ascending numeric order regardless
of\\r\\nthe\nsorting method that was picked (note that this is
especially\nobvious\\r\\nonce you \\\"load more\\\", which is what I did
for the\nfollowing\\r\\nscreenshots):\\r\\n\\r\\n\\r\\n| | Ascending |
Descending\n|\\r\\n|--------------|-----------|------------|\\r\\n|
Alphabetical |
<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234391308-6d3a23ee-3495-4eff-810f-216f758b3a58.png\\\"/>\\r\\n|\n<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234391213-117163e2-ee97-4f9d-87fa-a63c8cc5459e.png\\\"/>\\r\\n|\\r\\n|\nDoc
count |
<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234391375-0ccdf72f-83c0-4a87-951e-c2e1e3223006.png\\\"/>\\r\\n|\n<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234392997-fea42ffe-5d9d-4a11-968f-e1503f2c0e4f.png\\\"/>\\r\\n|\\r\\n\\r\\n\\r\\n###\nAfter\\r\\n\\r\\nThis
PR converts the options list suggestions to be stored\nas
an\\r\\n**array** of key/value pairs in order to preserve the
order\nreturned\\r\\nfrom Elasticsearch - now, you get the expected
string-sorted\nordering\\r\\nwhen using numeric `keyword` fields in an
options list\ncontrol:\\r\\n\\r\\n| | Ascending |
Descending\n|\\r\\n|--------------|-----------|------------|\\r\\n|
Alphabetical |
<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234394182-aa2bfdf4-fe41-441d-bdbf-917173c17627.png\\\"/>\\r\\n|\n<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234393421-24cca3e3-0249-4607-9e16-daa274399bdd.png\\\"/>\\r\\n|\\r\\n|\nDoc
count |
<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234394599-dda01056-5446-497e-abe4-f3839aeb4dd0.png\\\"/>\\r\\n|\n<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234394693-42544ef1-eb2b-4d52-8a78-c2ca7d2d7cfa.png\\\"/>\\r\\n|\\r\\n\\r\\n\\r\\nNotice\nin
the above that we are now using **string sorting** for
the\\r\\nnumeric\nvalues when alphabetical sorting is selected, which
means you\\r\\naren't\ngetting the expected \\\"numeric\\\" sorting - so
for example,\nwhen\\r\\nsorted ascending, `\\\"6\\\" > \\\"52\\\"`
because it is only comparing\nthe first\\r\\ncharacter and `\\\"6\\\" >
\\\"5\\\"`. This will be handled much\nbetter once\\r\\n[numeric
field\nsupport](https://github.com/elastic/kibana/issues/126795)\\r\\nis
added to\noptions lists.\\r\\n\\r\\n\\r\\n###
Checklist\\r\\n\\r\\n\\r\\n- [x] [Unit
or\nfunctional\\r\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\r\\nwere\nupdated
or added to match the most common scenarios\\r\\n- [x] This was\nchecked
for\n[cross-browser\\r\\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\\r\\n\\r\\n\\r\\n###\nFor
maintainers\\r\\n\\r\\n- [ ] This was checked for breaking API
changes\nand
was\n[labeled\\r\\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\",\"sha\":\"b3f65f79e5017b70fe26e5aa1c2ee1085e68c138\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"8.9\"],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v8.8.0\",\"labelRegex\":\"^v8.8.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/155207\",\"number\":155207,\"mergeCommit\":{\"message\":\"[Controls]\nFix
sorting of numeric keyword fields
(#155207)\\n\\nCloses\nhttps://github.com//issues/155073\\r\\n\\r\\n##\nSummary\\r\\n\\r\\n###
Before\\r\\n\\r\\nPreviously, the options list\nsuggestions were stored
as a dictionary\\r\\n(i.e. an object of key+value\npairs) - while this
worked for most fields,\\r\\nunbeknownst to us,\nJavascript tries to
sort numeric keys (regardless of\\r\\nif they are of\ntype `string` or
`number`) based on their value.\\r\\n\\r\\nThis meant that,\nas part of
the parsing process when using an options\\r\\nlist control for\na
numeric `keyword` field, the results returned by the\\r\\nES query
were\n**always** sorted in ascending numeric order regardless
of\\r\\nthe\nsorting method that was picked (note that this is
especially\nobvious\\r\\nonce you \\\"load more\\\", which is what I did
for the\nfollowing\\r\\nscreenshots):\\r\\n\\r\\n\\r\\n| | Ascending |
Descending\n|\\r\\n|--------------|-----------|------------|\\r\\n|
Alphabetical |
<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234391308-6d3a23ee-3495-4eff-810f-216f758b3a58.png\\\"/>\\r\\n|\n<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234391213-117163e2-ee97-4f9d-87fa-a63c8cc5459e.png\\\"/>\\r\\n|\\r\\n|\nDoc
count |
<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234391375-0ccdf72f-83c0-4a87-951e-c2e1e3223006.png\\\"/>\\r\\n|\n<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234392997-fea42ffe-5d9d-4a11-968f-e1503f2c0e4f.png\\\"/>\\r\\n|\\r\\n\\r\\n\\r\\n###\nAfter\\r\\n\\r\\nThis
PR converts the options list suggestions to be stored\nas
an\\r\\n**array** of key/value pairs in order to preserve the
order\nreturned\\r\\nfrom Elasticsearch - now, you get the expected
string-sorted\nordering\\r\\nwhen using numeric `keyword` fields in an
options list\ncontrol:\\r\\n\\r\\n| | Ascending |
Descending\n|\\r\\n|--------------|-----------|------------|\\r\\n|
Alphabetical |
<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234394182-aa2bfdf4-fe41-441d-bdbf-917173c17627.png\\\"/>\\r\\n|\n<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234393421-24cca3e3-0249-4607-9e16-daa274399bdd.png\\\"/>\\r\\n|\\r\\n|\nDoc
count |
<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234394599-dda01056-5446-497e-abe4-f3839aeb4dd0.png\\\"/>\\r\\n|\n<img\nwidth=\\\"320px\\\"\\r\\nsrc=\\\"https://user-images.githubusercontent.com/8698078/234394693-42544ef1-eb2b-4d52-8a78-c2ca7d2d7cfa.png\\\"/>\\r\\n|\\r\\n\\r\\n\\r\\nNotice\nin
the above that we are now using **string sorting** for
the\\r\\nnumeric\nvalues when alphabetical sorting is selected, which
means you\\r\\naren't\ngetting the expected \\\"numeric\\\" sorting - so
for example,\nwhen\\r\\nsorted ascending, `\\\"6\\\" > \\\"52\\\"`
because it is only comparing\nthe first\\r\\ncharacter and `\\\"6\\\" >
\\\"5\\\"`. This will be handled much\nbetter once\\r\\n[numeric
field\nsupport](https://github.com/elastic/kibana/issues/126795)\\r\\nis
added to\noptions lists.\\r\\n\\r\\n\\r\\n###
Checklist\\r\\n\\r\\n\\r\\n- [x] [Unit
or\nfunctional\\r\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\r\\nwere\nupdated
or added to match the most common scenarios\\r\\n- [x] This was\nchecked
for\n[cross-browser\\r\\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\\r\\n\\r\\n\\r\\n###\nFor
maintainers\\r\\n\\r\\n- [ ] This was checked for breaking API
changes\nand
was\n[labeled\\r\\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\",\"sha\":\"b3f65f79e5017b70fe26e5aa1c2ee1085e68c138\"}},{\"branch\":\"8.9\",\"label\":\"v8.9.0\",\"labelRegex\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->"}},{"branch":"8.9","label":"v8.9.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
@ThomThomson ThomThomson removed the loe:needs-research This issue requires some research before it can be worked on or estimated label Jul 27, 2023
@ThomThomson ThomThomson added the loe:medium Medium Level of Effort label Jul 27, 2023
@srothkin
Copy link

We require this feature to upgrade one of our dashboards that is currently using the soon to be deprecated input controls.

@Heenawter
Copy link
Contributor

FYI this is blocked by #143587 - it was pointed out to us that it wasn't obvious why this issue was marked as blocked, so hopefully that helps to provide some context for folks 👍

@Heenawter Heenawter self-assigned this Nov 23, 2023
@Heenawter Heenawter mentioned this issue Nov 28, 2023
Merged
7 tasks
Heenawter added a commit that referenced this issue Dec 18, 2023
@Heenawter
[Controls] Add numeric options list (#172106)
7d024a7 
Closes #143587
Closes #126795

## Summary

This PR adds support for numeric options lists - i.e. options list
controls that are created with a numeric field. In order to make this
possible, I had to add support for fields to have multiple, overlapping
compatible control types (however, it is currently only number fields
where this applies). When selecting a field that is compatible with
multiple control types, the user is given the option to select which
control type they want, like so:

<p align="center"><img alt="GIF of multiple control types being
available for a single field"
src="https://github.com/elastic/kibana/assets/8698078/9ebb6795-1206-47c4-aaef-32437d27cf59"/></p>

> [!NOTE]
> This system currently defaults to options list controls, since these
are the most common - this is backed up by our telemetry, which shows
that (in the last 30 days), clusters with at least one options list
control occured **ten times more often** than clusters with at least one
range slider control. However, if we decide to introduce more control
types (such as, for example, a date picker control), this assumption may
no longer be the case - at that point, we would need to reevaluate
whether the default should **always** be options list.

### Video


https://github.com/elastic/kibana/assets/8698078/4a876c94-a041-4228-aab8-7c2c1c871071

### Exact Match Searching

Since numeric fields do not have a "prefix" query equivalent, the only
possibility for searching these fields is either (1) a range query or
(2) an exact match / equality query. In this PR, I added support for
equality search - i.e. in order to find a value in a numeric options
list, you must enter the *full*, exact term in order for it to be found;
in the future, we could extend this to include a range search. This is
the exact match searching in action:

<p align="center"><img alt="GIF of exact match searching example on
number field"
src="https://github.com/elastic/kibana/assets/8698078/491feff3-031f-4367-8018-67aab9be55e3"/></p>


Since exact match searching is a generic search query that works for
**all** field types, I added this as an option for **all** field types
that support searching - i.e. keyword fields, number fields, IP fields,
etc. For example, here is the supported search techniques for a keyword
field:


<p align="center"><img alt="GIF of all available search techniques for
keyword field"
src="https://github.com/elastic/kibana/assets/8698078/dbc12cef-d43e-4d9f-a779-a5fe9e75325c"/></p>

> [!TIP]
> Exact match / equality / term searching on float values can lead to
slightly unexpected results - refer to the [documentation on precision
loss](https://www.elastic.co/guide/en/elasticsearch/reference/current/number.html#_which_type_should_i_use)
for a description of why. Be mindful when using an options list control
for float values, and consider whether a range slider might be a better
solution!

Eventually, we should be able to add exact-match searching to date
fields, as well; however, due to [discrepancies that exist in the
unified search bar](#172097)
with respect to how searching date fields is handled, we should ideally
wait until this is resolved so that we can be consistent across all
three search experiences (query bar, filter pills, and controls).


Depending on how the author expects a given control to be used, this
search technique will return results **faster** than some of the other
search types since "search as you type" will, more often than not,
return zero results in this setting - that is why I chose to add it for
keyword fields, as well.


### Searching when `allowExpensiveQueries` setting is `false`
Previously, we had **two separate versions** of our options list search
queries - one for when `allowExpensiveQueries` was `true`, and the other
for when it was `false`. This was a significant amount of tech debt that
was time consuming to maintain, which made it difficult to justify
keeping this around considering **how much** of Kibana relies on this
setting to be `true` (searching for existing Dashboards by title on the
listing page, saving a brand new dashboard, etc.).

Therefore, while we still have **some** functionality when
`allowExpensiveQueries` is `false`, I have refactored this code
significantly to simplify the logic.

> [!IMPORTANT]
> Specifically, options list controls now only support **exact match
searching** when `allowExpensiveQueries` is `false`.

Since this query is the same regardless of the type of field or the
value of `allowExpensiveQueries`, this means we no longer have to
maintain two slightly different versions ("cheap" and "expensive") of
our search queries. This cleans up our tech debt significantly.

### Bundle Size Changes @elastic/kibana-operations 

Changes to bundle size are primarily due to the changes I made to the
`OptionsListEditorOptions` component - since this component is directly
added to the options list factory (which is not async imported), this
impacts the bundle size.

### Checklist

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [x] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)


### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
@Heenawter Heenawter removed the blocked label Feb 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Heenawter Heenawter

Labels
Feature:Dashboard Dashboard related features Feature:Input Control Input controls visualization impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. loe:medium Medium Level of Effort Project:Controls Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Controls] Add numeric options list Heenawter/kibana
5 participants
@srothkin @Heenawter @ThomThomson @elasticmachine @jack-morrison