Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace thredshold argument to binned metrics #322

Merged
merged 11 commits into from
Jun 28, 2021
Merged

Conversation

SkafteNicki
Copy link
Member

@SkafteNicki SkafteNicki commented Jun 27, 2021

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

What does this PR do?

Fixes #265
Adds threshold argument to binned metrics for manually control which thresholds to use

cc: @maximsch2

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@SkafteNicki SkafteNicki added the enhancement New feature or request label Jun 27, 2021
@pep8speaks
Copy link

pep8speaks commented Jun 27, 2021

Hello @SkafteNicki! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-06-28 21:35:21 UTC

@codecov
Copy link

codecov bot commented Jun 27, 2021

Codecov Report

Merging #322 (3763c4d) into master (578c8f5) will decrease coverage by 20.83%.
The diff coverage is 57.14%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master     #322       +/-   ##
===========================================
- Coverage   96.54%   75.71%   -20.84%     
===========================================
  Files         111      111               
  Lines        3536     3541        +5     
===========================================
- Hits         3414     2681      -733     
- Misses        122      860      +738     
Flag Coverage Δ
Linux 75.71% <57.14%> (-0.10%) ⬇️
Windows 75.71% <57.14%> (-0.10%) ⬇️
cpu 75.71% <57.14%> (-20.79%) ⬇️
gpu ?
macOS 75.71% <57.14%> (-20.79%) ⬇️
pytest 75.71% <57.14%> (-20.84%) ⬇️
python3.6 ?
python3.8 ?
python3.9 ?
torch1.3.1 ?
torch1.4.0 ?
torch1.9.0 ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
torchmetrics/regression/psnr.py 100.00% <ø> (ø)
torchmetrics/regression/ssim.py 100.00% <ø> (ø)
...hmetrics/classification/binned_precision_recall.py 89.04% <57.14%> (-10.96%) ⬇️
torchmetrics/utilities/distributed.py 17.24% <0.00%> (-81.04%) ⬇️
torchmetrics/image/kid.py 17.44% <0.00%> (-79.07%) ⬇️
torchmetrics/image/inception.py 30.00% <0.00%> (-65.00%) ⬇️
torchmetrics/image/fid.py 30.23% <0.00%> (-52.33%) ⬇️
torchmetrics/classification/auc.py 50.00% <0.00%> (-50.00%) ⬇️
torchmetrics/classification/kldivergence.py 37.50% <0.00%> (-50.00%) ⬇️
torchmetrics/functional/classification/auroc.py 46.26% <0.00%> (-40.30%) ⬇️
... and 69 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 578c8f5...3763c4d. Read the comment docs.

@mergify mergify bot removed the has conflicts label Jun 28, 2021
@Borda Borda changed the title Adds thredshold argument to binned metrics Replace thredshold argument to binned metrics Jun 28, 2021
Copy link
Contributor

@tchaton tchaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !

@Borda Borda enabled auto-merge (squash) June 28, 2021 21:45
@Borda Borda merged commit 67eb71b into master Jun 28, 2021
@Borda Borda deleted the binned_thresholds branch June 28, 2021 22:10
@maximsch2
Copy link
Contributor

Awesome, thanks for improving this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API / design enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Exclude thresholds of 0.0 and 1.0 in precision recall curve
7 participants