Run Kibana test suite when Stack Monitoring integrations change

[miltonhultgren]

Summary

In elastic/kibana#146000 we added a test setup inside the Kibana repo to allow us to test that the Stack Monitoring UI app works with an unreleased version of a related Integration (Elasticsearch, Kibana, Logstash, Beat).

The downside to that setup is that it requires a manual step of building the package, copying the zip file into Kibana and executing the test suites and if those tests fail they still don't block the merger of the changes to the Integration.

It would be better, if when a PR is opened to one of these integrations, tests could be run for the components that depend on those integrations (in this case, Kibana). So a CI build for the Elasticsearch Integration, would pull down the latest Kibana version from source and run the affected test suites, if any test fails, the CI build fails and the PR is blocked.
The PR can become unblocked if changes are made to Kibana to support the new (and old) formats of the data for example.

Questions

• How can we make the latest package build available to the Kibana ?
• How can we provide the dynamic package configuration to Kibana ? Right now it's hardcoded
• Should we run this verification against main ? all supported minors ?
• How can we create a CI step that only runs for a specific package ? How to parametrize the CI step with the package under test ?

[klacabane]

This is blocked by elastic/kibana#148303, but one could experiment with the Beats test suite