fix bf16 issue in text classification pipeline

[chujiezheng]

What does this PR do?

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[chujiezheng]

@amyeroberts Hi, this is the new PR I mentioned in #30518

[amyeroberts]

Thanks for fixing and adding tests!

Some of the CI tests failing are unrelated, and I've triggered re-runs which should hopefully resolve these.

Some are related to this PR. For the quality checks, running make fixup should resolve. For the others, specifically:

FAILED tests/test_pipeline_mixin.py::TextClassificationPipelineTests::test_accepts_torch_bf16 - TypeError: '<' not supported between instances of 'torch.dtype' and 'int'
FAILED tests/test_pipeline_mixin.py::TextClassificationPipelineTests::test_accepts_torch_fp16 - TypeError: '<' not supported between instances of 'torch.dtype' and 'int'

you'll need to debug :)

[chujiezheng]

@amyeroberts Hello, I have fixed the bugs. The failing CI tests seem about tf or flax, but I guess they are unrelated to this PR?

[amyeroberts]

Thanks for adding!

For the failing tests - could you rebase on main to include the most recent upstream changes?

[chujiezheng]

Hi, I synced with the upstream changes, but the failed tests remain.

[chujiezheng]

@amyeroberts Hi, could you help check the PR? I think this PR is quite simple. It can also facilitate many use cases as the bf16-based classifiers are becoming popular.

[amyeroberts]

Hi @chujiezheng,

the PR itself all looks good, but there's some outstanding tests which are still failing which need to be resolved before we can merge. When you're merging in main, is your main branch in-sync with the upstream?

[chujiezheng]

Yes, I did it just a few hours ago. You can notice the sync in the commits.

[amyeroberts]

@chujiezheng It's very odd. The tests are failing are ones which were failing about two weeks ago and have been resolved. I can see that there's merge commits in the history, although from this I'm not sure if it's possible to tell from which commit in main that the merge is happening.

Could you try a rebase:

• Pulling the latest changes from the hugging face main branch: git fetch upstream main
• Go onto the feature branch on your fork git checkout main
• Rebase the changes git rebase upstream/main
• Force push the changes git push -f

[chujiezheng]

The issue still exists...

[amyeroberts]

@chujiezheng Hmm, how odd 🤔 Thanks for trying the rebase. OK, as these tests aren't related to the pipeline, I'm going to forcibly merge. Thanks for your patience and for adding this!