TypeError: Could not locate class 'QConv2D'.

[OliverFaust]

Hi,

When I run the QKerasTutorial:
https://github.com/google/qkeras/blob/master/notebook/QKerasTutorial.ipynb
I get the following error:

TypeError: Could not locate class 'QConv2D'. Make sure custom classes are decorated with @keras.saving.register_keras_serializable(). ...

Below is the list of installed packages and the complete console trace.

Thank you for your help in advance.
Oliver

pip list

Package                       Version
----------------------------- ------------
absl-py                       1.4.0
astunparse                    1.6.3
cachetools                    5.3.2
certifi                       2023.11.17
charset-normalizer            3.3.2
dm-tree                       0.1.8
flatbuffers                   23.5.26
gast                          0.5.4
google-auth                   2.25.2
google-auth-oauthlib          1.2.0
google-pasta                  0.2.0
grpcio                        1.60.0
h5py                          3.10.0
idna                          3.6
joblib                        1.3.2
keras                         2.15.0
keras-tuner                   1.4.6
kt-legacy                     1.0.5
libclang                      16.0.6
Markdown                      3.5.1
MarkupSafe                    2.1.3
ml-dtypes                     0.2.0
networkx                      3.2.1
numpy                         1.26.2
oauthlib                      3.2.2
opt-einsum                    3.3.0
packaging                     23.2
parse                         1.6.5
pip                           23.2
protobuf                      4.23.4
pyasn1                        0.5.1
pyasn1-modules                0.3.0
pyparser                      1.0
pyparsing                     3.1.1
QKeras                        0.9.0
requests                      2.31.0
requests-oauthlib             1.3.1
rsa                           4.9
scikit-learn                  1.3.2
scipy                         1.11.4
setuptools                    68.1.2
six                           1.16.0
tensorboard                   2.15.1
tensorboard-data-server       0.7.2
tensorflow                    2.15.0.post1
tensorflow-estimator          2.15.0
tensorflow-io-gcs-filesystem  0.34.0
tensorflow-model-optimization 0.7.5
termcolor                     2.4.0
threadpoolctl                 3.2.0
tqdm                          4.66.1
typing_extensions             4.9.0
urllib3                       2.1.0
Werkzeug                      3.0.1
wheel                         0.42.0
wrapt                         1.14.1

python3 QKerasTutorial.py 
2023-12-17 22:13:39.560279: I external/local_tsl/tsl/cuda/cudart_stub.cc:31] Could not find cuda drivers on your machine, GPU will not be used.
2023-12-17 22:13:39.619692: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2023-12-17 22:13:39.619738: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2023-12-17 22:13:39.621197: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2023-12-17 22:13:39.628727: I external/local_tsl/tsl/cuda/cudart_stub.cc:31] Could not find cuda drivers on your machine, GPU will not be used.
2023-12-17 22:13:39.628945: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-12-17 22:13:40.651978: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2023-12-17 22:13:42.765716: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 188160000 exceeds 10% of free system memory.
Epoch 1/3
469/469 [==============================] - 62s 130ms/step - loss: 0.2192 - accuracy: 0.9368 - val_loss: 0.0792 - val_accuracy: 0.9766
Epoch 2/3
469/469 [==============================] - 53s 113ms/step - loss: 0.0657 - accuracy: 0.9800 - val_loss: 0.0611 - val_accuracy: 0.9800
Epoch 3/3
469/469 [==============================] - 44s 93ms/step - loss: 0.0457 - accuracy: 0.9863 - val_loss: 0.0550 - val_accuracy: 0.9831
Model: "model"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_1 (InputLayer)        [(None, 28, 28, 1)]       0         
                                                                 
 conv2d_1 (Conv2D)           (None, 26, 26, 18)        180       
                                                                 
 act_1 (Activation)          (None, 26, 26, 18)        0         
                                                                 
 conv2d_2 (Conv2D)           (None, 24, 24, 32)        5216      
                                                                 
 act_2 (Activation)          (None, 24, 24, 32)        0         
                                                                 
 flatten (Flatten)           (None, 18432)             0         
                                                                 
 dense (Dense)               (None, 10)                184330    
                                                                 
 softmax (Activation)        (None, 10)                0         
                                                                 
=================================================================
Total params: 189726 (741.12 KB)
Trainable params: 189726 (741.12 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
2023-12-17 22:16:24.997914: W external/local_tsl/tsl/framework/cpu_allocator_impl.cc:83] Allocation of 188160000 exceeds 10% of free system memory.
Epoch 1/10
469/469 [==============================] - 84s 175ms/step - loss: 0.2676 - accuracy: 0.9229 - val_loss: 0.1157 - val_accuracy: 0.9656
Epoch 2/10
469/469 [==============================] - 81s 173ms/step - loss: 0.1002 - accuracy: 0.9705 - val_loss: 0.0775 - val_accuracy: 0.9770
Epoch 3/10
469/469 [==============================] - 80s 171ms/step - loss: 0.0710 - accuracy: 0.9791 - val_loss: 0.0659 - val_accuracy: 0.9809
Epoch 4/10
469/469 [==============================] - 78s 166ms/step - loss: 0.0579 - accuracy: 0.9829 - val_loss: 0.0653 - val_accuracy: 0.9781
Epoch 5/10
469/469 [==============================] - 78s 166ms/step - loss: 0.0473 - accuracy: 0.9860 - val_loss: 0.0536 - val_accuracy: 0.9824
Epoch 6/10
469/469 [==============================] - 80s 170ms/step - loss: 0.0395 - accuracy: 0.9880 - val_loss: 0.0612 - val_accuracy: 0.9813
Epoch 7/10
469/469 [==============================] - 80s 171ms/step - loss: 0.0343 - accuracy: 0.9894 - val_loss: 0.0480 - val_accuracy: 0.9836
Epoch 8/10
469/469 [==============================] - 81s 173ms/step - loss: 0.0302 - accuracy: 0.9906 - val_loss: 0.0632 - val_accuracy: 0.9817
Epoch 9/10
469/469 [==============================] - 85s 182ms/step - loss: 0.0262 - accuracy: 0.9920 - val_loss: 0.0461 - val_accuracy: 0.9846
Epoch 10/10
469/469 [==============================] - 80s 170ms/step - loss: 0.0211 - accuracy: 0.9938 - val_loss: 0.0496 - val_accuracy: 0.9844
<keras.src.engine.functional.Functional object at 0x7fa7ea059ad0>
Traceback (most recent call last):
  File "/home/oliver/ARM/hls4ml-tutorial/hls4ml-tutorial/QKerasTutorial.py", line 358, in <module>
    qmodel = model_quantize(model, config, 4, transfer_weights=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oliver/ARM/hls4ml-tutorial/hls4ml-tutorial/cleanEnv/lib/python3.11/site-packages/qkeras/utils.py", line 805, in model_quantize
    qmodel = quantized_model_from_json(json.dumps(jm), custom_objects)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oliver/ARM/hls4ml-tutorial/hls4ml-tutorial/cleanEnv/lib/python3.11/site-packages/qkeras/utils.py", line 894, in quantized_model_from_json
    qmodel = model_from_json(json_string, custom_objects=custom_objects)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oliver/ARM/hls4ml-tutorial/hls4ml-tutorial/cleanEnv/lib/python3.11/site-packages/keras/src/saving/legacy/model_config.py", line 125, in model_from_json
    return deserialize_from_json(json_string, custom_objects=custom_objects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oliver/ARM/hls4ml-tutorial/hls4ml-tutorial/cleanEnv/lib/python3.11/site-packages/keras/src/layers/serialization.py", line 299, in deserialize_from_json
    return deserialize(config, custom_objects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oliver/ARM/hls4ml-tutorial/hls4ml-tutorial/cleanEnv/lib/python3.11/site-packages/keras/src/layers/serialization.py", line 276, in deserialize
    return serialization_lib.deserialize_keras_object(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oliver/ARM/hls4ml-tutorial/hls4ml-tutorial/cleanEnv/lib/python3.11/site-packages/keras/src/saving/serialization_lib.py", line 609, in deserialize_keras_object
    return deserialize_keras_object(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oliver/ARM/hls4ml-tutorial/hls4ml-tutorial/cleanEnv/lib/python3.11/site-packages/keras/src/saving/serialization_lib.py", line 728, in deserialize_keras_object
    instance = cls.from_config(inner_config)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oliver/ARM/hls4ml-tutorial/hls4ml-tutorial/cleanEnv/lib/python3.11/site-packages/keras/src/engine/training.py", line 3330, in from_config
    inputs, outputs, layers = functional.reconstruct_from_config(
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oliver/ARM/hls4ml-tutorial/hls4ml-tutorial/cleanEnv/lib/python3.11/site-packages/keras/src/engine/functional.py", line 1493, in reconstruct_from_config
    process_layer(layer_data)
  File "/home/oliver/ARM/hls4ml-tutorial/hls4ml-tutorial/cleanEnv/lib/python3.11/site-packages/keras/src/engine/functional.py", line 1474, in process_layer
    layer = deserialize_layer(layer_data, custom_objects=custom_objects)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oliver/ARM/hls4ml-tutorial/hls4ml-tutorial/cleanEnv/lib/python3.11/site-packages/keras/src/layers/serialization.py", line 276, in deserialize
    return serialization_lib.deserialize_keras_object(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oliver/ARM/hls4ml-tutorial/hls4ml-tutorial/cleanEnv/lib/python3.11/site-packages/keras/src/saving/serialization_lib.py", line 705, in deserialize_keras_object
    cls = _retrieve_class_or_fn(
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/oliver/ARM/hls4ml-tutorial/hls4ml-tutorial/cleanEnv/lib/python3.11/site-packages/keras/src/saving/serialization_lib.py", line 827, in _retrieve_class_or_fn
    raise TypeError(
TypeError: Could not locate class 'QConv2D'. Make sure custom classes are decorated with `@keras.saving.register_keras_serializable()`. Full object config: {'module': 'keras.layers', 'class_name': 'QConv2D', 'config': {'name': 'conv2d_1', 'trainable': True, 'dtype': 'float32', 'filters': 18, 'kernel_size': [3, 3], 'strides': [1, 1], 'padding': 'valid', 'data_format': 'channels_last', 'dilation_rate': [1, 1], 'groups': 1, 'activation': 'linear', 'use_bias': True, 'kernel_initializer': {'module': 'keras.initializers', 'class_name': 'GlorotUniform', 'config': {'seed': None}, 'registered_name': None}, 'bias_initializer': {'module': 'keras.initializers', 'class_name': 'Zeros', 'config': {}, 'registered_name': None}, 'kernel_regularizer': None, 'bias_regularizer': None, 'activity_regularizer': None, 'kernel_constraint': None, 'bias_constraint': None, 'kernel_quantizer': 'stochastic_binary', 'bias_quantizer': 'quantized_po2(4)'}, 'registered_name': None, 'build_config': {'input_shape': [None, 28, 28, 1]}, 'name': 'conv2d_1', 'inbound_nodes': [[['input_1', 0, 0, {}]]]}

[JochiSt]

Hi,
are there any updates (or even a solution) on this issue?
I'm facing the same issue when using model_quantize in my code.

[rappysaha]

Hello,

I am facing exact same issue. Any update will be awesome.

Thank you

[AaronLReid]

Hi there, I have the exact issue also. Would love to know a solution.

[DuKsTa]

I was facing the same error. Turns out there was a problem with the JSON parsing.

This should do it:

config = { "QConv2D": { "conv2d_1": { "kernel_quantizer": "stochastic_binary", "bias_quantizer": "quantized_po2(4)" }, "conv2d_2": { "kernel_quantizer": "stochastic_binary", "bias_quantizer": "quantized_po2(4)" } }, "QDense": { "dense": { "kernel_quantizer": "quantized_bits(4,0,1)", "bias_quantizer": "quantized_bits(4)" } }

And then you should the same for the rest of the layers