Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Weighted AUROC to omit empty classes #376

Merged
merged 27 commits into from
Jul 26, 2021

Conversation

BeyondTheProof
Copy link
Contributor

@BeyondTheProof BeyondTheProof commented Jul 15, 2021

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

What does this PR do?

Fixes #348.

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

added logic to omit empty classes with weighted AUROC, as they should have 0 weight anyway
@pep8speaks
Copy link

pep8speaks commented Jul 15, 2021

Hello @BeyondTheProof! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-07-24 21:07:30 UTC

@codecov
Copy link

codecov bot commented Jul 15, 2021

Codecov Report

Merging #376 (8af688d) into master (397a089) will increase coverage by 0.01%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #376      +/-   ##
==========================================
+ Coverage   96.37%   96.39%   +0.01%     
==========================================
  Files         120      120              
  Lines        3756     3771      +15     
==========================================
+ Hits         3620     3635      +15     
  Misses        136      136              
Flag Coverage Δ
Linux 76.61% <23.52%> (-0.23%) ⬇️
Windows 76.61% <23.52%> (-0.23%) ⬇️
cpu 96.34% <100.00%> (+0.01%) ⬆️
gpu 96.31% <100.00%> (+0.01%) ⬆️
macOS 96.34% <100.00%> (+0.01%) ⬆️
pytest 96.39% <100.00%> (+0.01%) ⬆️
python3.6 95.51% <100.00%> (+0.01%) ⬆️
python3.8 96.34% <100.00%> (+0.01%) ⬆️
python3.9 96.23% <100.00%> (+0.01%) ⬆️
torch1.3.1 95.51% <100.00%> (+0.01%) ⬆️
torch1.4.0 95.59% <100.00%> (+0.01%) ⬆️
torch1.9.0 96.23% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
torchmetrics/functional/classification/auroc.py 88.09% <100.00%> (+2.58%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 397a089...8af688d. Read the comment docs.

@Borda Borda added the enhancement New feature or request label Jul 16, 2021
@Borda Borda added this to the v0.5 milestone Jul 16, 2021
Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BeyondTheProof mind check the failing tests?

torchmetrics/functional/classification/auroc.py Outdated Show resolved Hide resolved
Copy link
Member

@SkafteNicki SkafteNicki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, remember to also update the changelog as this influences the end user!

torchmetrics/functional/classification/auroc.py Outdated Show resolved Hide resolved
torchmetrics/functional/classification/auroc.py Outdated Show resolved Hide resolved
@BeyondTheProof
Copy link
Contributor Author

@SkafteNicki I've addressed all your suggestions, and my tests are passing locally. However, code coverage is failing, but I don't know how to address this, and whether it is important or not in this scenario. I would appreciate this workflow approved to make sure everything is A-OK! Thanks again.

@SkafteNicki
Copy link
Member

@BeyondTheProof do not worry about the code coverage. Only test to focus on are the once marked as "required"

Copy link
Member

@SkafteNicki SkafteNicki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM :]

@SkafteNicki SkafteNicki enabled auto-merge (squash) July 24, 2021 13:13
@SkafteNicki SkafteNicki merged commit f42256d into Lightning-AI:master Jul 26, 2021
@Borda Borda added this to the v0.5 milestone Aug 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request ready
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[metrics] AUROC Metric can't handle 0 observations of a class with multiclass classifier
4 participants