[CI] Add ECS-mappings compatibility tests

[eyalkoren]

#96171 added dynamic templates that cover all ECS mappings.
Those were enhanced by #96712 to be applicable with subobjects: false as well.

@P1llus wrote a test that scans ECS mapping artifact, creates a document with all ECS-matching fields, indexes it and verifies that all mappings are as expected.

We want to add this to CI so that it runs on a weekly basis, thus tests regressions in our ECS templates as well as notifying us when ECS gets updated in such a way that requires modification of the dynamic templates.

cc @felixbarny @ruflin

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[eyalkoren]

@mark-vieira this issue is not scheduled yet for implementation, but some guidance would be helpful to have in advance:

• the tests are currently written in Python, is there a simple way to run Python scripts through CI, or should we convert them to Java and run them as JUnit tests?
• @dakrone proposed that we add these tests to a new subdirectory in https://github.com/elastic/elasticsearch/tree/main/x-pack/plugin/stack/qa and have them running as part of the entire test suite both periodically and in PR builds. Can you approve this location and approach?
• there is a single external dependency for these tests- a generated yaml file from ECS. Since it is accessible through a static URL, my assumption is that we can simply download it for the test, instead of cloning the ECS repo, can you approve?

[mark-vieira]

• the tests are currently written in Python, is there a simple way to run Python scripts through CI, or should we convert them to Java and run them as JUnit tests?

If you want to run as part of Elasticsearch CI then reimplementing as JUnit test would be the preferable solution.

@dakrone proposed that we add these tests to a new subdirectory in https://github.com/elastic/elasticsearch/tree/main/x-pack/plugin/stack/qa and have them running as part of the entire test suite both periodically and in PR builds. Can you approve this location and approach?

That's fine. I also think they could just as well live inside the stack project itself rather than create a separate QA project.

there is a single external dependency for these tests- a generated yaml file from ECS. Since it is accessible through a static URL, my assumption is that we can simply download it for the test, instead of cloning the ECS repo, can you approve?

That's probably fine although I wonder if we'll want to version this. Pointing at a file in main that's changing makes repeatability difficult. How often do the contents of this file change?

[eyalkoren]

That's probably fine although I wonder if we'll want to version this. Pointing at a file in main that's changing makes repeatability difficult. How often do the contents of this file change?

These tests are our way to verify that the ECS dynamic templates keep up with changes in ECS, so we always want to rely on main and fail if new ECS fields are not covered by these templates. We expect that such failures will be rare and the test would be stable, but if we find out this is not the case we can try something else.