Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Simplify some arguments of the evaluation #1737

Closed
Jannchie opened this issue Jun 9, 2022 · 5 comments
Closed

[FEATURE] Simplify some arguments of the evaluation #1737

Jannchie opened this issue Jun 9, 2022 · 5 comments
Labels
enhancement New feature or request

Comments

@Jannchie
Copy link
Contributor

Jannchie commented Jun 9, 2022

Description

I found that for metrics like map_at_k, ndcg_at_k, precision_at_k, recall_at_k, the col_rating argument seem unnecessary if the relevancy_method is "top_k". However, the current implementation will report an error if this argument is not passed in:

ValueError: Missing columns in true rating DataFrame

Expected behavior with the suggested feature

I think it's a bit confusing to pass in unnecessary arguments.

A simple but dirty fix would be to change the default arguments DEFAULT_RATING_COL to DEFAULT_USER_COL.

Other Comments

@Jannchie Jannchie added the enhancement New feature or request label Jun 9, 2022
@miguelgfierro
Copy link
Collaborator

@ChuyangKe can you review this?

@yueguoguo
Copy link
Collaborator

Perhaps another way is to use *args and then check the existence of rating column in args when relevancy_method is not "top_k".

@AdityaSoni19031997
Copy link
Contributor

i can work on this issue, Can someone point-me to the correct file? I think the changes has to be cascaded and checked as well as multiple modules might be using these metrics?
Thanks

@miguelgfierro
Copy link
Collaborator

@AdityaSoni190319 can you implement the solution proposed by Le (yueguoguo)?

metrics are here https://github.com/microsoft/recommenders/blob/main/recommenders/evaluation/python_evaluation.py

@AdityaSoni19031997
Copy link
Contributor

@miguelgfierro I have tried to patch this fix and had run the tests for the given methods as well. LMK your feedback or any other changes needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants