Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ranker scores shouldn't be discarded #2706

Closed
MichelBartels opened this issue Jun 22, 2022 · 3 comments · Fixed by #2048
Closed

Ranker scores shouldn't be discarded #2706

MichelBartels opened this issue Jun 22, 2022 · 3 comments · Fixed by #2048
Labels
topic:predictions format of predictions, score, probability ...

Comments

@MichelBartels
Copy link
Contributor

Is your feature request related to a problem? Please describe.
The reason for using a ranker is that the scores should be more accurate than the retriever scores. For this reason, in that use case you would also want to use the ranker scores later on instead of the retriever score (e.g. when defining a threshold). You currently cannot do that as the ranker scores aren't saved anywhere (see here) as the reader only updates the order and not the score. Having the order of documents not dependent on the scores is also unintuitive.

Describe the solution you'd like
Replace retriever scores with ranker scores.

Describe alternatives you've considered
Having an additional field to save ranker scores.

@MichelBartels MichelBartels added the topic:predictions format of predictions, score, probability ... label Jun 22, 2022
@MichelBartels
Copy link
Contributor Author

It seems I have overlooked #2048 which should solve this issue.

@julian-risch
Copy link
Member

Hi @MichelBartels the corresponding PR #2048 is about to be merged, feel free to test it out and let me know if there are any problems. 🙂

@MichelBartels
Copy link
Contributor Author

Hi @julian-risch, I have already used the PR and can confirm that it is solving this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic:predictions format of predictions, score, probability ...
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants