[CI] XPackRestIT test {p0=terms_enum/10_basic/Test security} failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic-platform-support #4689 / sles-15_platform-support-unix
• elasticsearch-pull-request #39641 / part-3
• elasticsearch-pull-request #39641 / rest-compatibility
• elasticsearch-pull-request #39439 / part-3

Reproduction Line:

./gradlew ":x-pack:plugin:yamlRestTest" --tests "org.elasticsearch.xpack.test.rest.XPackRestIT.test {p0=terms_enum/10_basic/Test security}" -Dtests.seed=53BDDA28E105FDDF -Dtests.locale=guz-KE -Dtests.timezone=Brazil/East -Druntime.java=23

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: Failure at [terms_enum/10_basic:581]: field [terms] doesn't have length [1]
Expected: <1>
     but: was <0>

Issue Reasons:

• [main] 4 failures in test test {p0=terms_enum/10_basic/Test security} (0.6% fail rate in 709 executions)
• [main] 2 failures in step part-3 (1.9% fail rate in 103 executions)
• [main] 2 failures in pipeline elasticsearch-pull-request (1.7% fail rate in 120 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 4 failures in test test {p0=terms_enum/10_basic/Test security} (0.6% fail rate in 709 executions)
• [main] 2 failures in step part-3 (1.9% fail rate in 103 executions)
• [main] 2 failures in pipeline elasticsearch-pull-request (1.7% fail rate in 120 executions)

Build Scans:

• elasticsearch-periodic-platform-support #4689 / sles-15_platform-support-unix
• elasticsearch-pull-request #39641 / part-3
• elasticsearch-pull-request #39641 / rest-compatibility
• elasticsearch-pull-request #39439 / part-3

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[benwtrent]

Only one of the failures actually looks like a "valid" test failure. The rest were caused by some larger failure (ILM policy bug, ML template failing to create).

I am gonna unmute this.

The only valid test failure that I can find: https://gradle-enterprise.elastic.co/s/3oocp75hqvi34

Gonna see if this replicates or if there are any weird security terms enum commits around the time of that initial failure.

[benwtrent]

The failing build's logs. Honestly, nothing stands out to me at all :(.
build.zip