[CI] ShardChangesRestIT testShardChangesNoOperation failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic #5414 / openjdk23_checkpart2_java-matrix
• elasticsearch-pull-request #46805 / part-2
• elasticsearch-pull-request #46798 / part-2
• elasticsearch-pull-request #46791 / part-2

Reproduction Line:

./gradlew ":x-pack:plugin:ccr:javaRestTest" --tests "org.elasticsearch.xpack.ccr.rest.ShardChangesRestIT.testShardChangesNoOperation" -Dtests.seed=138D22242FDE3214 -Dtests.locale=vun-TZ -Dtests.timezone=Etc/Zulu -Druntime.java=23

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.net.SocketTimeoutException: 60,000 milliseconds timeout on connection http-outgoing-0 [ACTIVE]

Issue Reasons:

• [main] 4 failures in test testShardChangesNoOperation (14.8% fail rate in 27 executions)
• [main] 3 failures in step part-2 (15.8% fail rate in 19 executions)
• [main] 3 failures in pipeline elasticsearch-pull-request (16.7% fail rate in 18 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 4 failures in test testShardChangesNoOperation (14.8% fail rate in 27 executions)
• [main] 3 failures in step part-2 (15.8% fail rate in 19 executions)
• [main] 3 failures in pipeline elasticsearch-pull-request (16.7% fail rate in 18 executions)

Build Scans:

• elasticsearch-periodic #5414 / openjdk23_checkpart2_java-matrix
• elasticsearch-pull-request #46805 / part-2
• elasticsearch-pull-request #46798 / part-2
• elasticsearch-pull-request #46791 / part-2

[elasticsearchmachine]

Pinging @elastic/es-storage-engine (Team:StorageEngine)

[salvatore-campagna]

I tried to reproduce it but without managing to have it fail. This test is a bit time-dependent anyway. The shard changes api will wait a bit (depending on timeout) rather than returning an empty response in case at the moment the api call is made there is no change. Waiting is needed to see if a global checkpoint is reached so to return translog operations up to a certain global checkpoint.