Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce memory in classification metrics when average='micro' #1286

Merged
merged 5 commits into from
Oct 25, 2022

Conversation

SkafteNicki
Copy link
Member

@SkafteNicki SkafteNicki commented Oct 24, 2022

What does this PR do?

Fixes #1279
When average='micro' in multiclass classification we do not need to form the full [n_class, n_class] confmat to do the calculations. This PR introduces a much simpler reduction that scales much better when n_class is a large number.

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@SkafteNicki SkafteNicki added the enhancement New feature or request label Oct 24, 2022
@SkafteNicki SkafteNicki added this to the v0.10 milestone Oct 24, 2022
Copy link
Member

@justusschock justusschock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, can we add a simple test-case asserting that this is actually used properly?

Copy link
Contributor

@lucadiliello lucadiliello left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally tested and worked flawlessly on MPS, reducing memory usage of the metric from 18GB to a few KB.

@codecov
Copy link

codecov bot commented Oct 24, 2022

Codecov Report

Merging #1286 (5775fb7) into master (afc55c4) will decrease coverage by 37%.
The diff coverage is 100%.

Additional details and impacted files
@@           Coverage Diff            @@
##           master   #1286     +/-   ##
========================================
- Coverage      86%     49%    -37%     
========================================
  Files         190     190             
  Lines       11108   11118     +10     
========================================
- Hits         9607    5490   -4117     
- Misses       1501    5628   +4127     

@mergify mergify bot added the ready label Oct 24, 2022
@SkafteNicki
Copy link
Member Author

LGTM, can we add a simple test-case asserting that this is actually used properly?

Do you want test for memory consumption?
Because tests for correctness is already covered by the tests that are already in place.

@SkafteNicki SkafteNicki merged commit acc933b into master Oct 25, 2022
@SkafteNicki SkafteNicki deleted the feature/memory_micro_average branch October 25, 2022 10:41
Borda pushed a commit that referenced this pull request Oct 31, 2022
* base impl

* class change

* change subclasses

* changelog

(cherry picked from commit acc933b)
@Borda
Copy link
Member

Borda commented Nov 7, 2022

Do you want test for memory consumption?

would be great :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request ready
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MulticlassAccuracy using lot of memory with many classes
5 participants