[BUG] Reranking makes relevancy worse, somehow

[HenryL27]

What is the bug?

Doing some testing on a cluster, and relevancy seems to decrease...
Anecdotally, the reranking feature has made search quality better, but I've only ever run it on single node clusters. Now, relevency gets worse, e.g. (ndcg@10)

dataset	pure neural	minilm reranker	bge reranker	quantized bge reranker
fiqa	0.3859	0.3098	0.2574	0.0783

How can one reproduce the bug?

cluster with 3 r5.8xlarge

What is the expected behavior?

ndcg@10 should get better

What is your host/environment?

Operating system, version.

Do you have any screenshots?

If applicable, add screenshots to help explain your problem.

Do you have any additional context?

Add any other context about the problem.

[navneet1v]

@HenryL27 when you run the same dataset with single node is the relevancy better?

[HenryL27]

Not sure. deploying the model on only one node doesn't do it tho

[navneet1v]

@HenryL27 is the bug resolved?

[HenryL27]

ran on a single node cluster and I have similar results, so I think it's just a model problem. I think we can close this.