Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhance] Imporve efficiency of precision, recall, f1_score and support. #595

Merged
merged 4 commits into from
Dec 13, 2021

Conversation

mzr1996
Copy link
Member

@mzr1996 mzr1996 commented Dec 9, 2021

Motivation

Please describe the motivation of this PR and the goal you want to achieve through this PR.

Modification

Test on data shape: (50000, 1000), which is the validation set size of ImageNet.

metric original speed (ms) new speed (ms)
accuracy 76.7 17.9
precision / recall / f1_score 3770 201
support 531 15.5

BC-breaking

The return value of accuracy is changed to float | list[float] | list[list[float]], which is unified with other metrics. And according to the docstring of accuracy, it should be float, but the original return value is wrong.

Checklist

Before PR:

  • Pre-commit or other linting tools are used to fix the potential lint issues.
  • Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
  • The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
  • The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

  • If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects, like MMDet or MMSeg.
  • CLA has been signed and all committers have signed the CLA in this PR.

@mzr1996 mzr1996 requested a review from 0x4f5da2 December 9, 2021 10:37
@codecov
Copy link

codecov bot commented Dec 10, 2021

Codecov Report

Merging #595 (ff1716e) into master (188aa6e) will increase coverage by 0.02%.
The diff coverage is 89.47%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #595      +/-   ##
==========================================
+ Coverage   80.07%   80.10%   +0.02%     
==========================================
  Files         111      111              
  Lines        6450     6458       +8     
  Branches     1108     1106       -2     
==========================================
+ Hits         5165     5173       +8     
+ Misses       1149     1148       -1     
- Partials      136      137       +1     
Flag Coverage Δ
unittests 80.10% <89.47%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmcls/core/evaluation/eval_metrics.py 82.35% <86.20%> (+1.58%) ⬆️
mmcls/datasets/base_dataset.py 93.18% <100.00%> (ø)
mmcls/models/losses/accuracy.py 91.78% <100.00%> (+0.11%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 188aa6e...ff1716e. Read the comment docs.

Copy link
Contributor

@0x4f5da2 0x4f5da2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mzr1996 mzr1996 merged commit e3cf188 into open-mmlab:master Dec 13, 2021
mzr1996 added a commit to mzr1996/mmpretrain that referenced this pull request Nov 24, 2022
…rt. (open-mmlab#595)

* [Enhance] Imporve efficiency of precision, recall, f1_score and support.

* Fix bugs

* Use np.maximum since torch doesn't have maximum before torch 1.7

* Fix bug
@mzr1996 mzr1996 deleted the accuracy branch December 7, 2022 02:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants