Flag quantize-nbits returns error

[citruslab]

Getting error when using flag --quantize-nbits 6.
ValueError: Input contains NaN, infinity or a value too large for dtype('float64').
I tried downgrading python, torch and scikit-learn but nothing changes. Anyone has a tip on how to solve this?

[MenKuch]

Which model are you using? Have you tried using vanilla SD 1.5 or 2.1 as a test?

[citruslab]

Thanks for replying, I've tried with many different models and I always get the same type of error. I've tried with SD 1.5 and even just running the example code from https://huggingface.co/blog/fast-diffusers-coreml#converting-and-optimizing-custom-models which I guess should just work...

[MenKuch]

This usually means that something is wrong with your environment. Could you post your macOS version, Mac model and the python environment you are using?

[citruslab]

MBP M2 Max, 64GB RAM, Sonoma v14.0, Python 3.10.10, scikit-learn 1.1.2.

Edit: I just updated to macOS 14.1.1 and gave it another try but the results are the same.

It creates the mlpackage files, then goes into:

INFO:main:Converted safety_checker
INFO:main:Quantizing weights to 6-bit precision
INFO:main:Quantizing text_encoder to 6-bit precision
INFO:main:Quantizing text_encoder
Running compression pass palettize_weights:...

And then returns the error shortly after that:

File "/Users/.../miniconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/sklearn/utils/validation.py", line 146, in _assert_all_finite
raise ValueError(msg_err)
ValueError: Input X contains infinity or a value too large for dtype('float64').

[citruslab]

Downgrading from transformers 4.35.2 to 4.34.1 solved this problem.

[y-ich]

I still have a same error.
Any transformers version 4.29.2 or newer fails.

The target model is runwayml/stable-diffusion-v1-5.

My environment is,
M2 MBA 16GB RAM
macOS Sonoma 14.2.1
Python 3.8.17
ml-stable-diffusion commit 7449ce4
coremltools 7.1
torch 2.1.0
transformers 4.29.2, 4.30.2, 4.31.0, 4.32.1, 4.33.3,3.34.1,3.35.2,3.36.2
scipy 1.10.1
scikit-learn 1.1.3
pytest = 7.4.3
invisible-watermark 0.2.0
safetensors 0.4.1
matplotlib 3.7.4

[hubin858130]

I still have a same error. Any transformers version 4.29.2 or newer fails.

The target model is runwayml/stable-diffusion-v1-5.

My environment is, M2 MBA 16GB RAM macOS Sonoma 14.2.1 Python 3.8.17 ml-stable-diffusion commit 7449ce4 coremltools 7.1 torch 2.1.0 transformers 4.29.2, 4.30.2, 4.31.0, 4.32.1, 4.33.3,3.34.1,3.35.2,3.36.2 scipy 1.10.1 scikit-learn 1.1.3 pytest = 7.4.3 invisible-watermark 0.2.0 safetensors 0.4.1 matplotlib 3.7.4

I also have the same environment and the same issues

[y-ich]

I found that I had forgot to remove previous *.mlpackage.

transformers 4.34.1 works.

So same as citruslab san's workaround, my current understanding is that transformers 4.35.0 or later does not work with ml-stable-diffusion in related with --quantize-nbits option.

[jaycoolslm]

Do you not get this error when downgrading @y-ich ?

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
diffusers 0.29.0 requires huggingface-hub>=0.23.2, but you have huggingface-hub 0.17.3 which is incompatible.