using --quantize-nbits, stuck after generated mlpackage files

[JLCAIYE]

Good morning, I am trying to quantize the wights to 6 bits, after I have mlpackage files, the process will stuck, here is the error:

zsh: bus error python -m python_coreml_stable_diffusion.torch2coreml --convert-unet -o
(coreml_stable_diffusion) caiyuelai@CAIYUEdeMacBook-Pro ml-stable-diffusion % /Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '

and command I used: python -m python_coreml_stable_diffusion.torch2coreml --convert-unet --convert-text-encoder --convert-vae-decoder --convert-safety-checker --model-version CAIYE123/demov15 -o sdv15bits --bundle-resources-for-swift-cli --quantize-nbits 6 --chunk-unet --check-output-correctness

thank you

[atiorh]

Hello!
1-) What is your Mac chip version and RAM size?
2-) Could you please retry without --chunk-unet and report the outcome?
3-) Technically, you don't need --chunk-unet for v1-5 after 6-bit quantization but if you still want it for some reason, you can run the same command twice: first without this argument and then with this argument so memory pressure from conversion and chunking do not compound.

[atiorh]

Side note: --check-output-correctness is yet another memory pressure point but that one is useful to sanity check the resulting artifacts.

[JLCAIYE]

Hello! 1-) What is your Mac chip version and RAM size? 2-) Could you please retry without --chunk-unet and report the outcome? 3-) Technically, you don't need --chunk-unet for v1-5 after 6-bit quantization but if you still want it for some reason, you can run the same command twice: first without this argument and then with this argument so memory pressure from conversion and chunking do not compound.

thank you! im using m1 8gb, i will try it right now.

[JLCAIYE]

Side note: --check-output-correctness is yet another memory pressure point but that one is useful to sanity check the resulting artifacts.

I tried not use --chunk-unet and --check-ouput-correctness, it shows up a new error.

INFO:main:Quantizing weights to 6-bit precision
INFO:main:Quantizing text_encoder to 6-bit precision
INFO:main:Quantizing text_encoder
Running compression pass palettize_weights: 5%| | 10/197 [00:01<00:22, 8.32 o
Traceback (most recent call last):
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/caiyuelai/ml-stable-diffusion/python_coreml_stable_diffusion/torch2coreml.py", line 1409, in
main(args)
File "/Users/caiyuelai/ml-stable-diffusion/python_coreml_stable_diffusion/torch2coreml.py", line 1271, in main
quantize_weights(args)
File "/Users/caiyuelai/ml-stable-diffusion/python_coreml_stable_diffusion/torch2coreml.py", line 148, in quantize_weights
_quantize_weights(
File "/Users/caiyuelai/ml-stable-diffusion/python_coreml_stable_diffusion/torch2coreml.py", line 184, in _quantize_weights
model = ct.optimize.coreml.palettize_weights(mlmodel, config=config).save(out_path)
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/coremltools/optimize/coreml/_post_training_quantization.py", line 262, in palettize_weights
return _apply_graph_pass(mlmodel, weight_palettizer)
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/coremltools/optimize/coreml/_post_training_quantization.py", line 67, in _apply_graph_pass
graph_pass.apply(prog)
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/coremltools/optimize/coreml/_quantization_passes.py", line 104, in apply
apply_block(f)
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/coremltools/converters/mil/mil/passes/helper.py", line 60, in wrapper
return func(*args)
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/coremltools/optimize/coreml/_quantization_passes.py", line 101, in apply_block
self.transform_op(op)
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/coremltools/optimize/coreml/_quantization_passes.py", line 537, in transform_op
lut_params = self.compress(
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/coremltools/optimize/coreml/_quantization_passes.py", line 505, in compress
lut, indices = compress_kmeans(val, nbits)
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/coremltools/optimize/coreml/_quantization_passes.py", line 426, in compress_kmeans
lut, indices = _get_kmeans_lookup_table_and_weight(nbits, val)
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/coremltools/models/neural_network/quantization_utils.py", line 424, in _get_kmeans_lookup_table_and_weight
kmeans = KMeans(
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/sklearn/base.py", line 1151, in wrapper
return fit_method(estimator, *args, **kwargs)
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/sklearn/cluster/_kmeans.py", line 1471, in fit
X = self._validate_data(
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/sklearn/base.py", line 604, in _validate_data
out = check_array(X, input_name="X", **check_params)
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/sklearn/utils/validation.py", line 959, in check_array
_assert_all_finite(
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/sklearn/utils/validation.py", line 124, in _assert_all_finite
_assert_all_finite_element_wise(
File "/Users/caiyuelai/anaconda3/envs/coreml_stable_diffusion/lib/python3.8/site-packages/sklearn/utils/validation.py", line 173, in _assert_all_finite_element_wise
raise ValueError(msg_err)
ValueError: Input X contains infinity or a value too large for dtype('float64')

and it did not convert my personal model which I put on the huggingface. I am really confused on it.

[atiorh]

Great! It was indeed a memory pressure issue with your 8GB RAM Mac it seems

[atiorh]

Your new issue was fixed #246 a few hours ago so if you pull the latest main branch, this issue should go away. Please let me know how it goes.

[JLCAIYE]

Your new issue was fixed #246 a few hours ago so if you pull the latest main branch, this issue should go away. Please let me know how it goes.

it works, thank you, I just have one more stupid question. I put my personal model on the huggingface and try to convert it. But every time it convert a default stable diffusion v1.5. Do you know why cause it?

[atiorh]

Unsure about why this might happen but you can point to a local folder for --model-version and it will definitely pick up whatever model is in that folder

[JLCAIYE]

awesome! Really appreciate!