Cannot delete index, it is still part of the cluster state

[jasontedor]

I've seen this a few times in the last few days. This is a fresh build off of master (72fb93e) and a fresh data directory. The node was started, had some data indexed into it, deleted those indices, and shutdown the node, and then started the node again and saw this stacktrace on startup:

[2016-04-28 19:13:59,265][WARN ][cluster.service          ] [Ape-Man] failed to notify ClusterStateListener                                                   
java.lang.IllegalStateException: Cannot delete index [[data/czEFA9CDRWSpUdU0GPzSIw]], it is still part of the cluster state.
        at org.elasticsearch.indices.IndicesService.verifyIndexIsDeleted(IndicesService.java:686)
        at org.elasticsearch.indices.cluster.IndicesClusterStateService.applyDeletedIndices(IndicesClusterStateService.java:250)
        at org.elasticsearch.indices.cluster.IndicesClusterStateService.clusterChanged(IndicesClusterStateService.java:183)
        at org.elasticsearch.cluster.service.ClusterService.runTasksForExecutor(ClusterService.java:652)
        at org.elasticsearch.cluster.service.ClusterService$UpdateTask.run(ClusterService.java:814)
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:392)
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:237)
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:200)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

Another observation is that there is not a folder on disk with that UUID.

[jasontedor]

There is a tombstone for this index in the cluster state:

  {
        "index" : {
          "index_name" : "data",
          "index_uuid" : "czEFA9CDRWSpUdU0GPzSIw"
        },
        "delete_date_in_millis": 1461885168863
  }

[jasontedor]

Here's one more observation: creating an index with the same name as the deleted index appears to be related to the message here. In particular, if I delete the second index (with the same name) then that message does not appear. But if I create the index again (so a third time), the message does.