Generation: get special tokens from model config

[zucchini-nlp]

What does this PR do?

Fixed #30892 . The error, described in the linked issue, was caused by passing a GenerationConfig type instead of a kwarg. If a user passes GenerationConfig we do not update it with model's generation config, thus cannot get special tokens used by the model. Two options were:

1. Update the user's GenerationConfig by that of models in the _prepare_generation_config
2. Try to get special tokens from the config, if not then from self.config

I think the first option can mess-up with what the user wants when generating, for ex by adding a do_sample=True from model's generation config. So I went for option-2

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ArthurZucker]

Are you sure this is the only argument that we need to get from the model's generation config for BC?

[gante]

I see what went wrong in my previous PR 😭 Thank you for taking this issue @zucchini-nlp!

For full BC, we need further changes. The removed _get_decoder_start_token_id function used bos_token_id as a fallback, and bos_token_id could be loaded from the config (like decoder_start_token_id). Furthermore, this logic only was used when self.config.is_encoder_decoder is True.

So the needed changes for BC are:

1. first check if self.config.is_encoder_decoder is True.
2. If 1. is true, get BOTH decoder_start_token_id and bos_token_id with fallback to self.generation_config
3. if 1. is false, don't use a fallback to self.generation_config
4. The line decoder_start_token_id = decoder_start_token_id if decoder_start_token_id is not None else bos_token_id can stay as it is :)

[gante]

Are you sure this is the only argument that we need to get from the model's generation config for BC?

@ArthurZucker see my comment above :D

[zucchini-nlp]

@ArthurZucker @gante done, I brought back the _get_decoder_start_token from prev version and checked it works when user passes decoder_start_tokens, or when user does not pass anything.

[gante]

Thank you for iterating on the fix 💛

(for a perfect PR, only a test is missing, to ensure we don't repeat the mistake)

[ArthurZucker]

Approving and merging to patch transformers today

[ArthurZucker]

Suggested change

	return
	return None

explicit but I guess it's the same