update API to use latest TRL

[kashif]

fixes #180 and updates the API to use updated transformers and TRL versions #178

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[alvarobartt]

LGTM thanks for the update @kashif!

Left some minor comments, but IMO we should keep some dataclass fields as those override existing defaults in either trl or transformers, or just ignore those and don't override the existing defaults. Additionally, I saw that the hub_model_revision is only available for DPOConfig so I'd either remove that for DPOConfig or add it to SFTConfig and ORPOConfig too.

Thanks again 🤗

[kashif]

@alvarobartt do you know where the hub_model_revision is being used?

[alvarobartt]

@alvarobartt do you know where the hub_model_revision is being used?

So AFAIK we used it in the past to e.g. push different versions of a fine-tune under diff branches, but probably not widely used, so I'd let @lewtun or anyone else chime in! But I believe we can maybe just go ahead with the current changes and leave that for a separate PR in case we want to keep that, WDYT? 🤗 Thanks again for the PR!