run but baaaaaaaaaaaad

[werruww]

https://github.com/werruww/run-Qwen2-72B-Instruct-on-16gb-vram/blob/main/suc_Qwen2_72B_Instruct_AQLM_PV_1bit_1x16%20(2).ipynb

It worked but the results were disastrous and the answers were very bad and incomprehensible despite changing the tokenizer to the original model tokenizer and despite that the answers are incomprehensible and very bad. Is there a solution?

[werruww]

https://github.com/werruww/run-Qwen2-72B-Instruct-on-16gb-vram/blob/main/suc_Qwen2_72B_Instruct_AQLM_PV_1bit_1x16%20(3).ipynb

All 1bit models have very bad answers.

How to improve answers and outputs