Fix some TFWhisperModelIntegrationTests

[ydshieh]

What does this PR do?

(Probably since the introduction of GenerationConfig), some TF Whisper Integration tests fail with error

ValueError: The following `model_kwargs` are not used by the model: ['language', 'task'] (note: typos in the generate arguments will also show up in this list)

From my understanding, we should pass some arguments via generation_kwargs.

Note that WhisperForConditionalGeneartion has its custom generate but TFWhisperForConditionalGeneartion doesn't.

⚠️ When I try to apply the same changes to PyTorch Whisper test methods, it fails as the output is different.

We are somehow in a trouble of the inconsistency between PT/TF.

(Not sure if we should overwrite generate in TFWhisperForConditionalGeneartion.)

[sgugger]

LGTM, thanks for fixing!

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[amyeroberts]

LGTM- thanks for fixing!

Regarding the PyTorch tests failing with the same inputs - how do they fail?

[ydshieh]

LGTM- thanks for fixing!

Regarding the PyTorch tests failing with the same inputs - how do they fail?

It gives different outputs

[ydshieh]

I am very sorry, I was doing something stupid and made wrong claims. The generation_kwargs is an argument to the __init__ of GenerationConfig instead of the generate method.

(Some of) The previous failing tests pass as the 2 problematic arguments to generate are not passed, but this is wrong logic.

I will look in more depth.

[ydshieh]

Well, I finally decided to overwrite generate for TFWhisperForConditionalGeneartion.

[ArthurZucker]

Thanks! Needs testing for prompt_ids otherwise lgtm! Also we don't need part of the legacy things for timestamps since it was never supported in TF (just in case)

[amyeroberts]

Thanks for adding the generate logic and fixing this!

+1 on @ArthurZucker's comment about adding the tests