[CI] IndexLifecycleRestIT {yaml=ilm/40_explain_lifecycle/Test All Indexes Lifecycle Explain} failure

[bizybot]

The test failed in one of my PR build:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+pull-request-2/7623/console
but I can see some intermittent failures though the build is not available.
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob+fast+part2/1195/console

./gradlew ':x-pack:plugin:ilm:qa:rest:restTestRunner' \
--tests "org.elasticsearch.xpack.ilm.IndexLifecycleRestIT.test {yaml=ilm/40_explain_lifecycle/Test All Indexes Lifecycle Explain}" \
-Dtests.seed=147FD374FCB1376A \
-Dtests.security.manager=true \
-Dtests.locale=ja-JP-u-ca-japanese-x-lvariant-JP \
-Dtests.timezone=Indian/Cocos \
-Dcompiler.java=12 \
-Druntime.java=11

with error

org.elasticsearch.xpack.ilm.IndexLifecycleRestIT > test {yaml=ilm/40_explain_lifecycle/Test All Indexes Lifecycle Explain} FAILED
14:29:13     java.lang.AssertionError: Failure at [ilm/40_explain_lifecycle:218]: field [indices.index_with_policy_that_doesnt_exist.step_info.reason] is null
14:29:13 
14:29:13         Caused by:
14:29:13         java.lang.AssertionError: field [indices.index_with_policy_that_doesnt_exist.step_info.reason] is null

[elasticmachine]

Pinging @elastic/es-core-features

[gwbrown]

This doesn't reproduce locally, which combined with the assertion that failed, makes me think this is a timing issue - maybe the lifecycle runner hadn't attempted to run the policy for this index yet when this test ran? I'd try to fix this by adjusting the timeout if this were a Java test, but I'm not sure if that's possible for yaml tests.

[gwbrown]

I'm muting this test on master in #47317

[markharwood]

Another occurrence today on master.

Did not reproduce locally

./gradlew ':x-pack:plugin:ilm:qa:rest:yamlRestTest' --tests "org.elasticsearch.xpack.ilm.IndexLifecycleRestIT.test {yaml=ilm/40_explain_lifecycle/Test All Indexes Lifecycle Explain}" -Dtests.seed=55D813579B15B8D8 -Dtests.security.manager=true -Dtests.locale=vi-VN -Dtests.timezone=SystemV/MST7MDT -Druntime.java=11

java.lang.AssertionError: Failure at [ilm/40_explain_lifecycle:196]: field [indices.another_index.phase] is null
	at __randomizedtesting.SeedInfo.seed([55D813579B15B8D8:DD8C2C8D35E9D520]:0)
	at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.executeSection(ESClientYamlSuiteTestCase.java:414)
	at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.test(ESClientYamlSuiteTestCase.java:391)

Caused by: java.lang.AssertionError: field [indices.another_index.phase] is null

[droberts195]

Another failure in https://gradle-enterprise.elastic.co/s/yhozzcer6ucue that's identical to #47275 (comment):

java.lang.AssertionError: Failure at [ilm/40_explain_lifecycle:196]: field [indices.another_index.phase] is null

[tlrx]

Another failure today on master: https://gradle-enterprise.elastic.co/s/uoe6vaq33g3ng

[cbuescher]

Another one on master: https://gradle-enterprise.elastic.co/s/fydylz5vx5ars