Llama 2 BetterTransformer Broken

[AjayP13]

System Info

On transformers==4.34.0, patching meta-llama/Llama-2-7b-chat-hf with BetterTransformer and then running results in the error:

Error: TypeError: llama_forward() got an unexpected keyword argument 'padding_mask'

Who can help?

@younesbelkada

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction (minimal, reproducible, runnable)

N/A

Expected behavior

N/A

[younesbelkada]

Thanks for reporting, this should be already fixed on the main branch of transformers, can you try to install optimum from source?
We are also migrating BetterTransformer to transformers core. Check out this PR: huggingface/transformers#26572 for more details

[fxmarty]

Hi @AjayP13, thank you. Transformers added an argument padding_mask that broke BetterTransformer. The bug is fixed on the main branch, but we still did not do a pypi release.

[hengqujushi]

Same here, looking forward to this fix PR being merged into the main branch soon.

[fxmarty]

The fix PR is already in main!

[fxmarty]

#1421