add gather_use_object arguments

[SangbumChoi]

What does this PR do?

Fixes #30766 (issue)
I have described the problem and the solution at the issue!

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@muellerzr and @SunMarc

[qubvel]

Thanks for working on this 🤗

[SunMarc]

Thanks for the PR @SangbumChoi ! I left a comment

[SunMarc]

LGTM ! Just one final nit

[SunMarc]

Thanks for iterating ! I left a small suggestion !

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[SangbumChoi]

@SunMarc Thanks for the clean code!

[SunMarc]

LGTM ! Thanks for iterating !

[amyeroberts]

Thanks for working on this!

Logic looks OK, but we should move where it happens. A test should be added too, to make sure this argument can be passed and is respected

[SangbumChoi]

@amyeroberts For the test part where should I add for the extra kwargs or arguments?

https://github.com/huggingface/transformers/blob/main/tests/trainer/test_trainer.py

Maybe here?

[amyeroberts]

@SangbumChoi Yep - test_trainer.py is perfect 👌 It might be hard to verify if the argument is respected, but if we can pass eval_use_gather_object=True, have the trainer do a run (small model, dataset etc.) and evaluate I think it's good enough

[SangbumChoi]

@amyeroberts @SunMarc Even though the CI says pass the test I'm little bit skeptical about where did the test_trainer.py exactly correspends in the CI... I can't see it from the tests_torch.
https://app.circleci.com/pipelines/github/huggingface/transformers/96590/workflows/8552cb16-a227-4ef5-9919-308482684d63/jobs/1275319

Do I need some extra flag or commit for this?

[amyeroberts]

@SangbumChoi The test_trainer.py tests should run in the test_torch run. If you look at the test artefacts e.g. the test_outputs.txt for the parallel run 1, you can find the test_trainer.py tests have run

[SangbumChoi]

@amyeroberts I think I don't have access to test_outputs.txt for the parallel run 1, but I confirmed that the CI passed. Thanks again for the detail description!

[gw5] [ 83%] PASSED tests/trainer/test_trainer.py::TrainerIntegrationTest::test_eval_use_gather_object 
tests/trainer/test_trainer.py::TrainerIntegrationTest::test_evaluate 

[amyeroberts]

Thanks for adding this feature + tests!

[SunMarc]

Thanks for iterating !

[muellerzr]

Thanks for your PR! I didn't get a chance to take a look at this as I was OOO. Please see my comments here as I think we need a follow-up to avoid bad practices + clarify this very, very specific use-case