[Discover] Fix timefield sorting when switching similar index patterns

[dimaanj]

Summary

Fixes #108186
Fixes #117617

Sorting should be set properly after changing index patterns with the same field names, but different timefields. Testing can be done using data provided in the issue.

Checklist

• Unit or functional tests were updated or added to match the most common scenarios

[elasticmachine]

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

[dimaanj]

@elasticmachine merge upstream

[dimaanj]

@elasticmachine merge upstream

[timroes]

ℹ️ This way we get better TS support. Calling that method now, will cause the compiler to understand, that if that returns true timeFieldName must be a string and can't be undefined anymore.

[dimaanj]

@elasticmachine merge upstream

[mattkime]

Thanks, nice improvement! code lgtm!

[kertal]

Thx for all those changes! ❤️ So I did some 🥣 a-la-carte testing

1. I've installed our ecommerce sample data kibana_sample_data_ecommerce (timeField order_date)
2. I created a new data view *ecommerce* (timeField: products.created_on)
3. I've selected kibana_sample_data_ecommerce in Discover, no changes
4. When I switched from kibana_sample_data_ecommerce (no sort change) to *ecommerce* , the documents were now sorted by products.created_on ... this is how it should behave 👍

Before it was still sorted by order_date

1. Now I switched to kibana_sample_data_ecommerce,
2. Sorted by products.product_name.keyword,order_date, so it's a non default sort with the timefield on the second place.
3. Switched to *ecommerce*
4. The result was, that the sorting was kept, which I expected to be fine, but since there's a different time column configured, there's no more way to remove the sorting unless changing the URL:
   
   So I think we need to consider also this case, but I'm not 100% sure how.
   There are 2 ways
5. We remove the timefield of the previous data field in this case (🚀)
6. We replace it with the timefield the user has just selected (❤️ )
   Adding another 2 pair of eyes here @majagrubic @timroes

[dimaanj]

@elasticmachine merge upstream

[kertal]

added 2 hopefully last comments. thx so much 🙏 for your patience

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 4513717
• Storybooks Preview

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
dataViews	526	528	+2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
discover	312.5KB	312.6KB	+140.0B

Unknown metric groups

API count

id	before	after	diff
dataViews	668	671	+3

History

• 💚 Build #8336 succeeded c3512b9
• 💔 Build #8308 failed 74592b7
• 💛 Build #7614 was flaky 74592b7
• 💔 Build #7591 failed e271656
• 💔 Build #7339 failed 63252f6
• 💛 Build #6819 was flaky 60dcfa8

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @dmitriynj

[kertal]

LGTM, thx so much! 👍
Final test by using this data (one of the examples of the issues linked)

Steps to reproduce:
Create 2 indices, and 2 index patterns: timefield is the time field of my_index and timefield_2 is the time field of my_index_2. In my_index_2 also add timefield to the mapping but it won't contain data:

PUT my_index
{
  "mappings": {
    "properties": {
      "timefield": {
        "type": "date"
      }
    }
  }
}

POST my_index/_bulk
{ "index": {} }
{ "timefield": "2021-08-05" }
{ "index": {} }
{ "timefield": "2021-08-04" }
{ "index": {} }
{ "timefield": "2021-08-03" }

PUT my_index_2
{
  "mappings": {
    "properties": {
      "timefield": {
        "type": "date"
      },
      "timefield_2": {
        "type": "date"
      }
    }
  }
}

POST my_index_2/_bulk
{ "index": {} }
{ "timefield_2": "2021-08-05" }
{ "index": {} }
{ "timefield_2": "2021-08-04" }
{ "index": {} }
{ "timefield_2": "2021-08-03" }

Now in Discover, when switching from my_index to my_index_2, Discover sees the current sorting field (timefield) exists in my_index_2 and doesn’t change it. Since there is no data in this field in my_index_2, it shows up as unsorted.

With this PR
It's fixed 🎂 , switching the to my_index_2 also switches the timeField, so it works

[kibanamachine]

💔 Backport failed

Status	Branch	Result
✅	8.0
❌	7.16	Commit could not be cherrypicked due to conflicts

Successful backport PRs will be merged automatically after passing CI.

To backport manually run:
node scripts/backport --pr 116145