Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix support for half precision in Perplexity metric #2235

Merged
merged 11 commits into from
Nov 25, 2023

Conversation

SkafteNicki
Copy link
Member

@SkafteNicki SkafteNicki commented Nov 23, 2023

What does this PR do?

Fixes #2224

Before submitting
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?
PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃


📚 Documentation preview 📚: https://torchmetrics--2235.org.readthedocs.build/en/2235/

@SkafteNicki SkafteNicki added the bug / fix Something isn't working label Nov 23, 2023
@SkafteNicki SkafteNicki added this to the v1.2.x milestone Nov 23, 2023
@SkafteNicki SkafteNicki self-assigned this Nov 23, 2023
Copy link

codecov bot commented Nov 23, 2023

Codecov Report

Merging #2235 (e67d254) into master (9c964ec) will decrease coverage by 40%.
Report is 1 commits behind head on master.
The diff coverage is 50%.

Additional details and impacted files
@@           Coverage Diff            @@
##           master   #2235     +/-   ##
========================================
- Coverage      87%     47%    -40%     
========================================
  Files         293     293             
  Lines       16425   16423      -2     
========================================
- Hits        14278    7724   -6554     
- Misses       2147    8699   +6552     

@mergify mergify bot added the ready label Nov 23, 2023
@Borda Borda enabled auto-merge (squash) November 23, 2023 13:50
tests/unittests/text/test_perplexity.py Outdated Show resolved Hide resolved
@Borda
Copy link
Member

Borda commented Nov 24, 2023

this looks strange:

>       assert time_cg < time_no_cg, "using compute groups were not faster"
E       AssertionError: using compute groups were not faster
E       assert 24.99868607521057 < 0.33540940284729004

@mergify mergify bot removed the has conflicts label Nov 25, 2023
@Borda Borda merged commit c35a2fb into master Nov 25, 2023
61 checks passed
@Borda Borda deleted the bugfix/perplexity_dtype branch November 25, 2023 16:16
Borda pushed a commit that referenced this pull request Nov 30, 2023
Borda pushed a commit that referenced this pull request Dec 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug / fix Something isn't working ready topic: Text
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Perplexity dtype restriction too strict
3 participants