Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

torchmetrics.detection.mean_ap.MeanAveragePrecision bug if en(max_detection_thresholds)<3 #2218

Closed
JuanFMontesinos opened this issue Nov 17, 2023 · 7 comments · Fixed by #2219
Assignees
Labels
help wanted Extra attention is needed question Further information is requested

Comments

@JuanFMontesinos
Copy link

🐛 Bug

torchmetrics.detection.mean_ap.MeanAveragePrecision
an arg is max_detection_thresholds
if len(max_detection_thresholds)<3 there is a bug in pycocotools>cocoeval.py
it seem that the size of this list is hardcoded.
image

Soz not to be more proactive.

@JuanFMontesinos JuanFMontesinos added bug / fix Something isn't working help wanted Extra attention is needed labels Nov 17, 2023
Copy link

Hi! thanks for your contribution!, great first issue!

@SkafteNicki
Copy link
Member

Hi @JuanFMontesinos, thanks for raising the issue.
I have created PR #2219 that will raise an error when the class is initialized instead of it failing randomly inside pycocotools. There is really nothing more we can do, because then it would need to be fixed directly in pycocotools. That said, the faster-coco-eval backend already supports this, so a fix for users is also just to change backend.

@SkafteNicki SkafteNicki self-assigned this Nov 17, 2023
@JuanFMontesinos
Copy link
Author

I was imagine so. I think it will be clearer with better documentation so thanks for your PR.
Quick question if you don't mind, how to change the backend?
image
is the only reference to a backend

@SkafteNicki
Copy link
Member

A new backend argument was added in v1.2 of torchmetrics:
https://lightning.ai/docs/torchmetrics/stable/detection/mean_average_precision.html
image

@JuanFMontesinos
Copy link
Author

@SkafteNicki tak!
I think the current docs still point to an older version.

@SkafteNicki
Copy link
Member

@JuanFMontesinos, we are in the process of transferring our docs from read the docs server:
https://torchmetrics.readthedocs.io/en/stable/
to own lightning server:
https://lightning.ai/docs/torchmetrics/stable/
which is reason you could not find the argument in the first place. Sorry for the confusion.

@Borda Borda added question Further information is requested and removed bug / fix Something isn't working labels Nov 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed question Further information is requested
Projects
None yet
4 participants
@Borda @SkafteNicki @JuanFMontesinos and others