(Maybe fixed) UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 451: character maps to <undefined>"

[MOULINIER]

Describe the bug

This error (UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 451: character maps to <undefined>) was thrown when I launched "TheBloke_Magicoder-S-DS-6.7B-GPTQ_gptq-4bit-64g-actorder_True", it also happened with the deepseek model.

I maybe fixed it (at least it works for me) by changing the metadata = json.loads(open(path, 'r').read()) by `metadata = json.loads(open(path, 'r', encoding='utf-8').read()) in the ``models_settings.py`

Is there an existing issue for this?

• I have searched the existing issues

Reproduction

Lauch TheBloke_Magicoder-S-DS-6.7B-GPTQ_gptq-4bit-64g-actorder_True

Screenshot

No response

Logs

" File "W:\text-generation-webui-main\modules\ui_model_menu.py", line 209, in load_model_wrapper

shared.model, shared.tokenizer = load_model(selected_model, loader)

                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "W:\text-generation-webui-main\modules\models.py", line 78, in load_model

metadata = get_model_metadata(model_name)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "W:\text-generation-webui-main\modules\models_settings.py", line 105, in get_model_metadata

metadata = json.loads(open(path, 'r').read())

                      ^^^^^^^^^^^^^^^^^^^^^^

File "W:\text-generation-webui-main\installer_files\env\Lib\encodings\cp1252.py", line 23, in decode

return codecs.charmap_decode(input,self.errors,decoding_table)[0]

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 451: character maps to <undefined>" ```

System Info

RTX 3060 ti, windows 10

[VertexMachine]

Same issue, same fix, same bug report here: #4947

[MOULINIER]

Same issue, same fix, same bug report here: #4947

Sorry, I had the error yesterday and only had the time to make the issue today, didn't read the new issues in between !

[NicolasMejiaPetit]

I'm having the same issue with deepseek coder 1.3 , weirdly enough other models work. But i really needed deepseek coder1.3b. :(. Actuslly niether does deepseek 6b work, they wore working for me before i updated web ui. Yeah just got done testing with deepseek coder 1.3, 5.4 and 6.7 same error. even with encoding set to utf 8.