Binned PR-related metrics #128
Conversation
maximsch2
requested review from
ananyahjha93,
Borda,
justusschock,
SkafteNicki and
tchaton
as code owners
Codecov Report
@@ Coverage Diff @@
## master #128 +/- ##
===========================================
- Coverage 96.03% 79.71% -16.33%
===========================================
Files 176 89 -87
Lines 5402 2756 -2646
===========================================
- Hits 5188 2197 -2991
- Misses 214 559 +345
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
|
@maximsch2 how is it going here, ready for review? |
|
@Borda, I need to fix up isort and do a short cleanup, will get to it some time this week. |
…into binned_metrics
|
@maximsch2 how is the clean-up doing? ;] |
|
@Borda , I've moved out the function I wanted to do, feel free to take a look now. Another thing I think I should probably do is to move out test data generation code into a central place, but actually curious about your feedback to see if we should just replace the main one with it? The motivation is to get a reasonably-looking model, so that metrics like recall@precision=0.8 can be computed as the random data never actually reaches that precision :) |
|
Hello @maximsch2! Thanks for updating this PR. There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻 Comment last updated at 2021-04-13 21:55:49 UTC |
|
@Borda , mind taking a look now? |
|
Addressed comments. On docs, is your preference to go for strict alphabetical order? I was putting the binned versions close to non-binned ones so that people will notice them together, but I can move the new one to the to the sorted place as well. |
that is also an option, I do not have strong preference here :] |
* WIP: Binned PR-related metrics * attempt to fix types * switch to linspace to make old pytorch happy * make flake happy * clean up * Add more testing, move test input generation to the approproate place * bugfixes and more stable and thorough tests * flake8 * Reuse python zip-based implementation as it can't be reproduced with torch.where/max * address comments * isort * Add docs and doctests, make APIs same as non-binned versions * pep8 * isort * doctests likes longer title underlines :O * use numpy's nan_to_num * add atol to bleu tests to make them more stable * atol=1e-2 for bleu * add more docs * pep8 * remove nlp test hack * address comments * pep8 * abc * flake8 * remove typecheck * chlog Co-authored-by: Jirka Borovec <[email protected]> Co-authored-by: Nicki Skafte <[email protected]> Co-authored-by: Jirka Borovec <[email protected]>
* Add AverageMeter * Fix type annotation to accomodate Python 3.6 bug * Add tests * Update changelog * Add AverageMeter to docs * fixup! Add AverageMeter to docs * Code review comments * Add tests for scalar case * Fix behavior on PyTorch <1.8 * fixup! Add tests for scalar case * fixup! fixup! Add tests for scalar case * Update CHANGELOG.md * Add Pearson correlation coefficient (#157) * init files * rest * pep8 * changelog * clamp * suggestions * rename * format * _sk_pearsonr * inline * fix sync * fix tests * fix docs * Apply suggestions from code review * Update torchmetrics/functional/regression/pearson.py * atol * update * pep8 * pep8 * chlog * . Co-authored-by: Jirka Borovec <[email protected]> Co-authored-by: Jirka Borovec <[email protected]> * Spearman correlation coefficient (#158) * ranking * init files * update * nearly working * fix tests * pep8 * add docs * fix doctests * fix docs * pep8 * isort * ghlog * Apply suggestions from code review Co-authored-by: Jirka Borovec <[email protected]> * Added changes for Test Differentiability [1/n] (#154) * added test changes * fix style error * fixed typo * added changes for requires_grad * metrics differentiability testing generalization * Update tests/classification/test_accuracy.py Co-authored-by: Nicki Skafte <[email protected]> * fix tests * pep8 * changelog * fix docs * fix tests * pep8 * Apply suggestions from code review Co-authored-by: Nicki Skafte <[email protected]> Co-authored-by: Jirka Borovec <[email protected]> * Binned PR-related metrics (#128) * WIP: Binned PR-related metrics * attempt to fix types * switch to linspace to make old pytorch happy * make flake happy * clean up * Add more testing, move test input generation to the approproate place * bugfixes and more stable and thorough tests * flake8 * Reuse python zip-based implementation as it can't be reproduced with torch.where/max * address comments * isort * Add docs and doctests, make APIs same as non-binned versions * pep8 * isort * doctests likes longer title underlines :O * use numpy's nan_to_num * add atol to bleu tests to make them more stable * atol=1e-2 for bleu * add more docs * pep8 * remove nlp test hack * address comments * pep8 * abc * flake8 * remove typecheck * chlog Co-authored-by: Jirka Borovec <[email protected]> Co-authored-by: Nicki Skafte <[email protected]> Co-authored-by: Jirka Borovec <[email protected]> * version + about (#170) * version + about * flake8 * try * . * fix doc * overload sig * fix * Different import style Co-authored-by: Nicki Skafte <[email protected]> Co-authored-by: Jirka Borovec <[email protected]> Co-authored-by: Jirka Borovec <[email protected]> Co-authored-by: Bhadresh Savani <[email protected]> Co-authored-by: Maxim Grechkin <[email protected]>
Before submitting
What does this PR do?
Fixes #95 .
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃