convert qwen 110b gptq checkpoint的时候，qkv_bias 的shape不能被3整除

[CallmeZhangChenchen]

[jershi425]

@CallmeZhangChenchen Thank you for the feedback. We have not supported Qwen-110B yet. We will support and validate it soon.

[Tlntin]

i got same error in qwen1.5-32b, i think may caused by GQA(group query attention).
I fix up it in this, pr link

[nv-guomingz]

This week's update will contain the fixing submmited by @Tlntin .

[Hukongtao]

The model can be converted and built into the engine normally, but the inference results are garbled. Have you ever encountered this?

[Tlntin]

The model can be converted and built into the engine normally, but the inference results are garbled. Have you ever encountered this?

i think you need update auto-gptq and transformers to latest.

[Hukongtao]

The model can be converted and built into the engine normally, but the inference results are garbled. Have you ever encountered this?

i think you need update auto-gptq and transformers to latest.

still have the problem

[Tlntin]

The model can be converted and built into the engine normally, but the inference results are garbled. Have you ever encountered this?

i think you need update auto-gptq and transformers to latest.

still have the problem

after update, have you build engine again?

[Hukongtao]

The model can be converted and built into the engine normally, but the inference results are garbled. Have you ever encountered this?

i think you need update auto-gptq and transformers to latest.

still have the problem

after update, have you build engine again?

Yes, I re-update rank0.safetensors and rank0.engine

[Hukongtao]

The v0.10.0 version may have some problems. I used the latest code from the main branch and it was aligned.

[tianzuishiwo]

auto-gpt

I meet the same problems，do you solve the problem? 兄弟！

[tianzuishiwo]

The model can be converted and built into the engine normally, but the inference results are garbled. Have you ever encountered this?

I meet the same problems，do you solve the problem? 兄弟！