Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Metrics] Unification of FBeta #4656

Merged
merged 12 commits into from
Nov 23, 2020
Merged

[Metrics] Unification of FBeta #4656

merged 12 commits into from
Nov 23, 2020

Conversation

SkafteNicki
Copy link
Member

What does this PR do?

Update to fbeta metric:

  • Unification of class and functional backend
  • Added F1 metric as it is the most commonly used metric
  • Renamed class metric Fbeta->FBeta and functionals fbeta_score->fbeta, f1_score->f1 for consistency
  • average argument now also supports weighted and None as options
  • Made averaging more stable towards NaNs (some edge case when reduction='micro')

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?
  • Did you verify new and existing tests pass locally with your changes?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In in short, see following bullet-list:

  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified; Bugfixes should be including in bug-fix release milestones (m.f.X) and features should be included in (m.X.b) releases.

Did you have fun?

Make sure you had fun coding 🙃

@SkafteNicki SkafteNicki added feature Is an improvement or enhancement Metrics labels Nov 13, 2020
@SkafteNicki SkafteNicki added this to the 1.1 milestone Nov 13, 2020
Borda
Borda reviewed Nov 13, 2020
View reviewed changes
Copy link
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is quite hard to see what is move and what is change, but I very welcome this unification, I tried already when it was added :] #4043 (comment)

@Borda Borda added the design Includes a design discussion label Nov 13, 2020
@SkafteNicki
Copy link
Member Author

@Borda yes, I saw your comment and promised long time ago update to the metric.
I have a hard time keeping these unifications PRs small, as it involves moving the old implementation, creating/updating class interface to call functional interface, update docs, update tests ect. so it often involve changes in many files :]

teddykoker
teddykoker approved these changes Nov 16, 2020
View reviewed changes
Copy link
Contributor

@teddykoker teddykoker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work as always :)

@codecov
Copy link

codecov bot commented Nov 22, 2020 •
edited
Loading

Codecov Report

Merging #4656 (9d5b505) into master (cd90dd4) will decrease coverage by 0%.
The diff coverage is 97%.

@@          Coverage Diff           @@
##           master   #4656   +/-   ##
======================================
- Coverage      93%     93%   -0%     
======================================
  Files         117     118    +1     
  Lines        9010    9018    +8     
======================================
+ Hits         8383    8389    +6     
- Misses        627     629    +2

awaelchli
awaelchli approved these changes Nov 22, 2020
View reviewed changes
Copy link
Contributor

@awaelchli awaelchli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 ⚡

pytorch_lightning/metrics/classification/f_beta.py Outdated Show resolved Hide resolved
pytorch_lightning/metrics/classification/f_beta.py Outdated Show resolved Hide resolved
pytorch_lightning/metrics/classification/f_beta.py Outdated Show resolved Hide resolved
pytorch_lightning/metrics/functional/f_beta.py Outdated Show resolved Hide resolved
pytorch_lightning/metrics/functional/f_beta.py Outdated Show resolved Hide resolved
pytorch_lightning/metrics/functional/f_beta.py Outdated Show resolved Hide resolved
@SkafteNicki SkafteNicki merged commit 6831ba9 into Lightning-AI:master Nov 23, 2020
@SkafteNicki SkafteNicki deleted the metrics/fbeta branch November 23, 2020 16:12
@Borda Borda modified the milestones: 1.1, 1.0.x Nov 23, 2020
Borda pushed a commit that referenced this pull request Nov 23, 2020
@SkafteNicki @Borda
[Metrics] Unification of FBeta (#4656)
f00c246 
* implementation

* init files

* more stable reduction

* add tests

* docs

* remove old implementation

* pep8

* changelog

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <[email protected]>

Co-authored-by: Nicki Skafte <[email protected]>
Co-authored-by: Teddy Koker <[email protected]>
Co-authored-by: Adrian Wälchli <[email protected]>

(cherry picked from commit 6831ba9)
Borda pushed a commit that referenced this pull request Nov 23, 2020
@SkafteNicki @Borda
[Metrics] Unification of FBeta (#4656)
9fc4b6d 
* implementation

* init files

* more stable reduction

* add tests

* docs

* remove old implementation

* pep8

* changelog

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <[email protected]>

Co-authored-by: Nicki Skafte <[email protected]>
Co-authored-by: Teddy Koker <[email protected]>
Co-authored-by: Adrian Wälchli <[email protected]>

(cherry picked from commit 6831ba9)
Borda pushed a commit that referenced this pull request Nov 23, 2020
@SkafteNicki @Borda
[Metrics] Unification of FBeta (#4656)
278b9a9 
* implementation

* init files

* more stable reduction

* add tests

* docs

* remove old implementation

* pep8

* changelog

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <[email protected]>

Co-authored-by: Nicki Skafte <[email protected]>
Co-authored-by: Teddy Koker <[email protected]>
Co-authored-by: Adrian Wälchli <[email protected]>

(cherry picked from commit 6831ba9)
@Borda Borda mentioned this pull request Dec 10, 2020
Merged
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Borda Borda Borda left review comments

@justusschock justusschock justusschock approved these changes

@teddykoker teddykoker teddykoker approved these changes

@awaelchli awaelchli awaelchli approved these changes

@ananyahjha93 ananyahjha93 Awaiting requested review from ananyahjha93

@nateraw nateraw Awaiting requested review from nateraw

@SeanNaren SeanNaren Awaiting requested review from SeanNaren

@tchaton tchaton Awaiting requested review from tchaton tchaton is a code owner

@williamFalcon williamFalcon Awaiting requested review from williamFalcon

Assignees
No one assigned
Labels
design Includes a design discussion feature Is an improvement or enhancement
Projects
None yet
Milestone
pl:1.0.x
Development

Successfully merging this pull request may close these issues.
5 participants
@SkafteNicki @awaelchli @Borda @teddykoker @justusschock