Elasticsearch legacy compatible api

[mshustov]

Summary

There is a plan to migrate from https://github.com/elastic/elasticsearch-js-legacy to the new https://github.com/elastic/elasticsearch-js client. Their interfaces are not compatible and migration will require additional work.
The current implementation of the Elasticsearch service introduced another version of the API. Thus, a plugin migrated to New Platform Elasticsearch Service has to adapt to the new API twice:

• when switching from Legacy platform API to New Platform API.
• when switching from New Platform API to elasticsearch-js.
  This PR make New Platform Elasticsearch API compatible with LegacyPlatform to avoid double work for plugins in the future.

Checklist

Use strikethroughs to remove checklist items you don't feel are applicable to this PR.

• [ ] This was checked for cross-browser compatibility, including a check against IE11
• [ ] Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• [ ] Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• [ ] This was checked for keyboard-only and screenreader accessibility

For maintainers

• This was checked for breaking API changes and was labeled appropriately
• This includes a feature addition or change that requires a release note and was labeled appropriately

[elasticmachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request

[elasticmachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request

[elasticmachine]

Pinging @elastic/kibana-platform

[mshustov]

Should we tag it as a breaking change? I found only Security plugin uses this API, although I could miss a plugin that is in progress of the migration.

[joshdover]

Yes we definitely should. There are at least 3 or 4 new in-development plugins. Let's add it to this issue #40768

[mshustov]

done

[elasticmachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request

[elasticmachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request

[azasypkin]

ACK: Reviewing...

[azasypkin]

Changes in Spaces and Security plugins look good to me assuming this is what we really want. But before we move forward with these changes can you please clarify the following things for me?

• What is the main motivation behind these changes and what problem are we trying to solve here? Issue description is a bit ters on that.

• If the current core ElasticsearchService API conflicts with the one we'll have once we migrate to the new elasticsearch-js lib, then how the new API will look like? I thought that once we advance with more and more "scoped" or "context-bound" services provided by the core we'll end up with something like callAsInternalUser and callAsCurrentUser anyway, and consumers won't need to call asScoped manually (most of the time at least). But I'm out of the loop here, sorry if you need to explain that multiple times 🙈

• In general, why the core public API is going to change with the new version of the 3rd-party lib we use under the hood (elasticsearch-js lib)? Aren't we trying to augment consumers from raw elasticsearch-js lib and provide stable interfaces/contracts that don't neceserrily need to change if underlying lib changes? Are we planning to incorporate new major versions of elasticsearch-js client only within new major versions of Kibana as well? As far as I understand elasticsearch-js is still relatively independent or we have plans to converge APIs of ElasticSearchService and elasticsearch-js somehow?

I had a brief conversation with @restrry over Slack and I've got that with these changes we're trying to simplify migration of the legacy plugins to the new platform. But it's still not immediately clear to me in what ways these changes do that since we already have a legacy elasticsearch plugin that serves exactly that purpose. I mean people will still need to change the way they receive cluster client(s), probably adapt to new error types, update tests to use new platform mocks (while migrating Security to NP that caused the main pain I believe cause we had lots of tests to update), etc.

So the only win I see here is that we'll keep same method names and their signatures (for callWithRequest only, callAsInternalUser and callWithInternalUser have the same signature already), but it doesn't feel like a huge one in the grand scheme of things unless I'm missing something.

Having said that, I'm not opposed to what you've done here, just want to have a better understanding of the plan you have 🙂

Thanks a lot!

[mshustov]

closed until we agree on the final design and outline the whole migration plan.