Compress the model but met KeyError: 'layer_0' from tabulate.py

[zhao-w-en]

Hello developer，
I want to compress the model for training acceleration, but I ran into this problem halfway through. In the two steps of compression, the first step ‘compress’ did not report an error, but the second step 'freeze' failed. The error is shown below.
File "/home/zhaow/anaconda3/envs/dpmd/lib/python3.9/site-packages/deepmd/descriptor/se_a.py", line 320, in enable_compression
self.table = DPTabulate(
File "/home/zhaow/anaconda3/envs/dpmd/lib/python3.9/site-packages/deepmd/utils/tabulate.py", line 101, in __init__
for item in self.matrix["layer_" + str(self.layer_size)]:
KeyError: 'layer_0'
I checked my descriptor setting but did not find the problem. What should I do next? It’s worth mentioning that I have successfully performed compression training before, and the sudden mistake made me feel puzzled.
"descriptor": {
"type": "se_e2_a",
"sel": [ 68, 46, 92],
"rcut_smth": 5.80,
"rcut": 6.00,
"neuron": [32, 64, 128],
"resnet_dt": false,
"axis_neuron": 16,
"seed": 1,
},

[njzjz]

@denghuilu

[zhao-w-en]

At present, I have done some tests and found that model.ckpt.meta/index/data files has been replaced after stage1 of compression. I think it should be in preparation for the freezing of the compressed model. So after stage2 fails, I manually freeze the model. The model can be successfully frozen and used for accelerated training.
what can I do to prevent failure in stage2 ?
The error message is as follows:

Traceback (most recent call last):
  File "/home/zhaow/anaconda3/envs/dpmd/bin/dp", line 10, in <module>
    sys.exit(main())
  File "/home/zhaow/anaconda3/envs/dpmd/lib/python3.9/site-packages/deepmd/entrypoints/main.py", line 447, in main
    compress(**dict_args)
  File "/home/zhaow/anaconda3/envs/dpmd/lib/python3.9/site-packages/deepmd/entrypoints/compress.py", line 143, in compress
    freeze(checkpoint_folder=checkpoint_folder, output=output, node_names=None)
  File "/home/zhaow/anaconda3/envs/dpmd/lib/python3.9/site-packages/deepmd/entrypoints/freeze.py", line 163, in freeze
    input_graph_def = graph.as_graph_def()
  File "/home/zhaow/anaconda3/envs/dpmd/lib/python3.9/site-packages/tensorflow/python/framework/ops.py", line 3374, in as_graph_def
    result, _ = self._as_graph_def(from_version, add_shapes)
  File "/home/zhaow/anaconda3/envs/dpmd/lib/python3.9/site-packages/tensorflow/python/framework/ops.py", line 3291, in _as_graph_def
    graph.ParseFromString(compat.as_bytes(data))
google.protobuf.message.DecodeError: Error parsing message

[denghuilu]

At present, I have done some tests and found that model.ckpt.meta/index/data files has been replaced after stage1 of compression. I think it should be in preparation for the freezing of the compressed model. So after stage2 fails, I manually freeze the model. The model can be successfully frozen and used for accelerated training. what can I do to prevent failure in stage2 ? The error message is as follows:

Traceback (most recent call last):
  File "/home/zhaow/anaconda3/envs/dpmd/bin/dp", line 10, in <module>
    sys.exit(main())
  File "/home/zhaow/anaconda3/envs/dpmd/lib/python3.9/site-packages/deepmd/entrypoints/main.py", line 447, in main
    compress(**dict_args)
  File "/home/zhaow/anaconda3/envs/dpmd/lib/python3.9/site-packages/deepmd/entrypoints/compress.py", line 143, in compress
    freeze(checkpoint_folder=checkpoint_folder, output=output, node_names=None)
  File "/home/zhaow/anaconda3/envs/dpmd/lib/python3.9/site-packages/deepmd/entrypoints/freeze.py", line 163, in freeze
    input_graph_def = graph.as_graph_def()
  File "/home/zhaow/anaconda3/envs/dpmd/lib/python3.9/site-packages/tensorflow/python/framework/ops.py", line 3374, in as_graph_def
    result, _ = self._as_graph_def(from_version, add_shapes)
  File "/home/zhaow/anaconda3/envs/dpmd/lib/python3.9/site-packages/tensorflow/python/framework/ops.py", line 3291, in _as_graph_def
    graph.ParseFromString(compat.as_bytes(data))
google.protobuf.message.DecodeError: Error parsing message

Which version of deepmd-kit are you using? Please confirm this error with the latest released package.

[denghuilu]

Trained DP model and input file if convenient. [email protected], that's my email address.

[zhao-w-en]

I've send a e-mail but I'm not sure if you received it...

[J-You]

hello, this error could be reproduced in v.2.0.1, does it be fixed in the latest version?

[njzjz]

There are indeed some fixes for this error in the latest version.

[weimq1]

May I ask how your problem was finally solved?