LLaVA OV: fix unpadding precision

[zucchini-nlp]

What does this PR do?

Fixes #34625. There was a small precision error in unpadding because the modeling code casts the size to list, while processing code works with tensors. This PR casts everything to list to match the calculations

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qubvel]

Thanks for the fix! Just a question re types

[zucchini-nlp]

sorry @qubvel , wrong tag for review

[ArthurZucker]

Thanks, does it fix #34625 entirely? (let's close it if so!)

[Scyther-07]

Hey @zucchini-nlp,
I am running the following code but it is throwing me a TypeError:

from transformers import AutoProcessor, LlavaForConditionalGeneration, BitsAndBytesConfig, LlavaNextProcessor
model_id = "llava-hf/llava-v1.6-mistral-7b-hf"
processor = AutoProcessor.from_pretrained(model_id)

Error:
TypeError: LlavaNextProcessor.__init__() got an unexpected keyword argument 'image_token'

Now, I looked at the commits of this PR and it looks good to me. The issue might be with your last PR #33424. Kindly look into this.

[zucchini-nlp]

@Scyther-07 hey, which transformers version you are using?

[Scyther-07]

@Scyther-07 hey, which transformers version you are using?

It's 4.39.3.
I just noticed that the above code works fine on Google Colab but throws an error in the Kaggle Notebook. I don't know what to make of it. I think I should shift to Colab.

[zucchini-nlp]

@Scyther-07 hmm, the 4.39.3 should throw error indeed and you need at least v4.43 to bypass the error. In fact we are currently changing the way inputs for VLMs are processed, thus I'd recommend to use the latest transformers after release. It will be v4.47 around next 1-2 weeks, not released yet now :)

[Scyther-07]

Yeah, it worked. Too foolish of me. Thanks for the help!