[BUG] Shard replication task assignment is not handled properly

[nisgoel-amazon]

What is the bug?
We have seen on managed service side of opensearch in AWS that if there are some stale entries present in persistent task of cluster state.
Then these stale tasks assignment is not handled properly which also cause issue for other tasks to not getting picked up.

[2023-04-03T20:56:23,341][WARN ][o.o.c.s.ClusterApplierService] [9dfc08ec809e11a9df3578add840cf68] failed to notify ClusterStateListener
[aclmbrprimary/Ago-hpPiRzm64zVf3Rf2Yw] IndexNotFoundException[no such index [aclmbrprimary]]
	at org.opensearch.cluster.routing.RoutingTable.shardRoutingTable(RoutingTable.java:171)
	at org.opensearch.replication.task.shard.ShardReplicationExecutor.getAssignment(ShardReplicationExecutor.kt:59)
	at org.opensearch.replication.task.shard.ShardReplicationExecutor.getAssignment(ShardReplicationExecutor.kt:32)
	at org.opensearch.persistent.PersistentTasksClusterService.createAssignment(PersistentTasksClusterService.java:352)
	at org.opensearch.persistent.PersistentTasksClusterService.shouldReassignPersistentTasks(PersistentTasksClusterService.java:420)
	at org.opensearch.persistent.PersistentTasksClusterService.clusterChanged(PersistentTasksClusterService.java:358)

What is the expected behavior?
Exception should be handled properly

What is your host/environment?

• OS: linux
• Version OS-1.3
• Plugins CCR