[gpt] Gpt2 fix half precision causal mask

[younesbelkada]

What does this PR do?

Applies a similar fix than #23136 but for GPT2.

To reproduce:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("gpt2", device_map="auto", load_in_8bit=True)
inputs = torch.LongTensor([[1, 1, 1], [1, 2, 1]]).to(0)

print(model(inputs))

The explanation is the same as the tagged PR:

When going for low_cpu_mem_usage each parameter is force-casted to the expected dtype, which is force-set to torch.float16 for 8bit models.

Therefore, for 8bit models (and also half-precision models) the causal mask is always force casted to float16 as it is part of the model's state dict, hence expected to be loaded from the Hub if the mask is available on the state dict.

The fix is to add persistant=False and add a field _keys_to_ignore_on_unexpected (for removing the warnings) to avoid loading that causal mask from the state dict and assign it to the buffer, and all causal masks that are saved as buffers should do the same to avoid unexpected behaviors.

Some users reported that they were also able to reproduce on PyTorch main branch but without load_in_8bit, I didn't managed to reproduce that way, I will have a deeper look

cc @amyeroberts

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[amyeroberts]

Thanks for fixing! I just have a question about _keys_to_ignore_on_load_missing for decision transformer

[amyeroberts]

Should r"h\.\d+\.attn\.masked_bias", r"h\.\d+\.attn\.bias" be in _keys_to_ignore_on_load_missing?

[younesbelkada]

Ah you are right it should probably be in keys_to_ignore_on_load_unexpected only; will modify that!

[amyeroberts]

LGTM! Thanks for fixing :)

[amyeroberts]

LGTM! Thanks for fixing :)