Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error loading #61

Open
claragarciamoll opened this issue Nov 21, 2024 · 1 comment
Open

Error loading #61

claragarciamoll opened this issue Nov 21, 2024 · 1 comment

Comments

@claragarciamoll
Copy link

Hello,

First of all, congratulations on your amazing job. I am currently researching on estimating the 3D human pose and using this information to animate an avatar replicating the human's movements. To do it, I need to use lightweight software such as MotionAGFormer-XS or MotionAGFormer-S. However, when I use both of them, an error appears when running the code:

RuntimeError: Error(s) in loading state_dict for DataParallel:
	Missing key(s) in state_dict: "module.layers.12.att_spatial.layer_scale_1", "module.layers.12.att_spatial.layer_scale_2", "module.layers.12.att_spatial.norm1.weight", "module.layers.12.att_spatial.norm1.bias", "module.layers.12.att_spatial.mixer.proj.weight", "module.layers.12.att_spatial.mixer.proj.bias", "module.layers.12.att_spatial.mixer.qkv.weight", "module.layers.12.att_spatial.norm2.weight", "module.layers.12.att_spatial.norm2.bias", "module.layers.12.att_spatial.mlp.fc1.weight", "module.layers.12.att_spatial.mlp.fc1.bias", "module.layers.12.att_spatial.mlp.fc2.weight", "module.layers.12.att_spatial.mlp.fc2.bias", "module.layers.12.att_temporal.layer_scale_1", "module.layers.12.att_temporal.layer_scale_2", "module.layers.12.att_temporal.norm1.weight", "module.layers.12.att_temporal.norm1.bias", "module.layers.12.att_temporal.mixer.proj.weight", "module.layers.12.att_temporal.mixer.proj.bias", "module.layers.12.att_temporal.mixer.qkv.weight", "module.layers.12.att_temporal.norm2.weight", "module.layers.12.att_temporal.norm2.bias", "module.layers.12.att_temporal.mlp.fc1.weight", "module.layers.12.att_temporal.mlp.fc1.bias", "module.layers.12.att_temporal.mlp.fc2.weight", "module.layers.12.att_temporal.mlp.fc2.bias", "module.layers.12.graph_spatial.layer_scale_1", "module.layers.12.graph_spatial.layer_scale_2", "module.layers.12.graph_spatial.norm1.weight", "module.layers.12.graph_spatial.norm1.bias", "module.layers.12.graph_spatial.mixer.U.weight", "module.layers.12.graph_spatial.mixer.U.bias", "module.layers.12.graph_spatial.mixer.V.weight", "module.layers.12.graph_spatial.mixer.V.bias", "module.layers.12.graph_spatial.mixer.batch_norm.weight", "module.layers.12.graph_spatial.mixer.batch_norm.bias", "module.layers.12.graph_spatial.mixer.batch_norm.running_mean", "module.layers.12.graph_spatial.mixer.batch_norm.running_var", "module.layers.12.graph_spatial.norm2.weight", "module.layers.12.graph_spatial.norm2.bias", "module.layers.12.graph_spatial.mlp.fc1.weight", "module.layers.12.graph_spatial.mlp.fc1.bias", "module.layers.12.graph_spatial.mlp.fc2.weight", "module.layers.12.graph_spatial.mlp.fc2.bias", "module.layers.12.graph_temporal.layer_scale_1", "module.layers.12.graph_temporal.layer_scale_2", "module.layers.12.graph_temporal.norm1.weight", "module.layers.12.graph_temporal.norm1.bias", "module.layers.12.graph_temporal.mixer.U.weight", "module.layers.12.graph_temporal.mixer.U.bias", "module.layers.12.graph_temporal.mixer.V.weight", "module.layers.12.graph_temporal.mixer.V.bias", "module.layers.12.graph_temporal.mixer.batch_norm.weight", "module.layers.12.graph_temporal.mixer.batch_norm.bias", "module.layers.12.graph_temporal.mixer.batch_norm.running_mean", "module.layers.12.graph_temporal.mixer.batch_norm.running_var", "module.layers.12.graph_temporal.norm2.weight", "module.layers.12.graph_temporal.norm2.bias", "module.layers.12.graph_temporal.mlp.fc1.weight", "module.layers.12.graph_temporal.mlp.fc1.bias", "module.layers.12.graph_temporal.mlp.fc2.weight", "module.layers.12.graph_temporal.mlp.fc2.bias", "module.layers.12.fusion.weight", "module.layers.12.fusion.bias", "module.layers.13.att_spatial.layer_scale_1", "module.layers.13.att_spatial.layer_scale_2", "module.layers.13.att_spatial.norm1.weight", "module.layers.13.att_spatial.norm1.bias", "module.layers.13.att_spatial.mixer.proj.weight", "module.layers.13.att_spatial.mixer.proj.bias", "module.layers.13.att_spatial.mixer.qkv.weight", "module.layers.13.att_spatial.norm2.weight", "module.layers.13.att_spatial.norm2.bias", "module.layers.13.att_spatial.mlp.fc1.weight", "module.layers.13.att_spatial.mlp.fc1.bias", "module.layers.13.att_spatial.mlp.fc2.weight", "module.layers.13.att_spatial.mlp.fc2.bias", "module.layers.13.att_temporal.layer_scale_1", "module.layers.13.att_temporal.layer_scale_2", "module.layers.13.att_temporal.norm1.weight", "module.layers.13.att_temporal.norm1.bias", "module.layers.13.att_temporal.mixer.proj.weight", "module.layers.13.att_temporal.mixer.proj.bias", "module.layers.13.att_temporal.mixer.qkv.weight", "module.layers.13.att_temporal.norm2.weight", "module.layers.13.att_temporal.norm2.bias", "module.layers.13.att_temporal.mlp.fc1.weight", "module.layers.13.att_temporal.mlp.fc1.bias", "module.layers.13.att_temporal.mlp.fc2.weight", "module.layers.13.att_temporal.mlp.fc2.bias", "module.layers.13.graph_spatial.layer_scale_1", "module.layers.13.graph_spatial.layer_scale_2", "module.layers.13.graph_spatial.norm1.weight", "module.layers.13.graph_spatial.norm1.bias", "module.layers.13.graph_spatial.mixer.U.weight", "module.layers.13.graph_spatial.mixer.U.bias", "module.layers.13.graph_spatial.mixer.V.weight", "module.layers.13.graph_spatial.mixer.V.bias", "module.layers.13.graph_spatial.mixer.batch_norm.weight", "module.layers.13.graph_spatial.mixer.batch_norm.bias", "module.layers.13.graph_spatial.mixer.batch_norm.running_mean", "module.layers.13.graph_spatial.mixer.batch_norm.running_var", "module.layers.13.graph_spatial.norm2.weight", "module.layers.13.graph_spatial.norm2.bias", "module.layers.13.graph_spatial.mlp.fc1.weight", "module.layers.13.graph_spatial.mlp.fc1.bias", "module.layers.13.graph_spatial.mlp.fc2.weight", "module.layers.13.graph_spatial.mlp.fc2.bias", "module.layers.13.graph_temporal.layer_scale_1", "module.layers.13.graph_temporal.layer_scale_2", "module.layers.13.graph_temporal.norm1.weight", "module.layers.13.graph_temporal.norm1.bias", "module.layers.13.graph_temporal.mixer.U.weight", "module.layers.13.graph_temporal.mixer.U.bias", "module.layers.13.graph_temporal.mixer.V.weight", "module.layers.13.graph_temporal.mixer.V.bias", "module.layers.13.graph_temporal.mixer.batch_norm.weight", "module.layers.13.graph_temporal.mixer.batch_norm.bias", "module.layers.13.graph_temporal.mixer.batch_norm.running_mean", "module.layers.13.graph_temporal.mixer.batch_norm.running_var", "module.layers.13.graph_temporal.norm2.weight", "module.layers.13.graph_temporal.norm2.bias", "module.layers.13.graph_temporal.mlp.fc1.weight", "module.layers.13.graph_temporal.mlp.fc1.bias", "module.layers.13.graph_temporal.mlp.fc2.weight", "module.layers.13.graph_temporal.mlp.fc2.bias", "module.layers.13.fusion.weight", "module.layers.13.fusion.bias", "module.layers.14.att_spatial.layer_scale_1", "module.layers.14.att_spatial.layer_scale_2", "module.layers.14.att_spatial.norm1.weight", "module.layers.14.att_spatial.norm1.bias", "module.layers.14.att_spatial.mixer.proj.weight", "module.layers.14.att_spatial.mixer.proj.bias", "module.layers.14.att_spatial.mixer.qkv.weight", "module.layers.14.att_spatial.norm2.weight", "module.layers.14.att_spatial.norm2.bias", "module.layers.14.att_spatial.mlp.fc1.weight", "module.layers.14.att_spatial.mlp.fc1.bias", "module.layers.14.att_spatial.mlp.fc2.weight", "module.layers.14.att_spatial.mlp.fc2.bias", "module.layers.14.att_temporal.layer_scale_1", "module.layers.14.att_temporal.layer_scale_2", "module.layers.14.att_temporal.norm1.weight", "module.layers.14.att_temporal.norm1.bias", "module.layers.14.att_temporal.mixer.proj.weight", "module.layers.14.att_temporal.mixer.proj.bias", "module.layers.14.att_temporal.mixer.qkv.weight", "module.layers.14.att_temporal.norm2.weight", "module.layers.14.att_temporal.norm2.bias", "module.layers.14.att_temporal.mlp.fc1.weight", "module.layers.14.att_temporal.mlp.fc1.bias", "module.layers.14.att_temporal.mlp.fc2.weight", "module.layers.14.att_temporal.mlp.fc2.bias", "module.layers.14.graph_spatial.layer_scale_1", "module.layers.14.graph_spatial.layer_scale_2", "module.layers.14.graph_spatial.norm1.weight", "module.layers.14.graph_spatial.norm1.bias", "module.layers.14.graph_spatial.mixer.U.weight", "module.layers.14.graph_spatial.mixer.U.bias", "module.layers.14.graph_spatial.mixer.V.weight", "module.layers.14.graph_spatial.mixer.V.bias", "module.layers.14.graph_spatial.mixer.batch_norm.weight", "module.layers.14.graph_spatial.mixer.batch_norm.bias", "module.layers.14.graph_spatial.mixer.batch_norm.running_mean", "module.layers.14.graph_spatial.mixer.batch_norm.running_var", "module.layers.14.graph_spatial.norm2.weight", "module.layers.14.graph_spatial.norm2.bias", "module.layers.14.graph_spatial.mlp.fc1.weight", "module.layers.14.graph_spatial.mlp.fc1.bias", "module.layers.14.graph_spatial.mlp.fc2.weight", "module.layers.14.graph_spatial.mlp.fc2.bias", "module.layers.14.graph_temporal.layer_scale_1", "module.layers.14.graph_temporal.layer_scale_2", "module.layers.14.graph_temporal.norm1.weight", "module.layers.14.graph_temporal.norm1.bias", "module.layers.14.graph_temporal.mixer.U.weight", "module.layers.14.graph_temporal.mixer.U.bias", "module.layers.14.graph_temporal.mixer.V.weight", "module.layers.14.graph_temporal.mixer.V.bias", "module.layers.14.graph_temporal.mixer.batch_norm.weight", "module.layers.14.graph_temporal.mixer.batch_norm.bias", "module.layers.14.graph_temporal.mixer.batch_norm.running_mean", "module.layers.14.graph_temporal.mixer.batch_norm.running_var", "module.layers.14.graph_temporal.norm2.weight", "module.layers.14.graph_temporal.norm2.bias", "module.layers.14.graph_temporal.mlp.fc1.weight", "module.layers.14.graph_temporal.mlp.fc1.bias", "module.layers.14.graph_temporal.mlp.fc2.weight", "module.layers.14.graph_temporal.mlp.fc2.bias", "module.layers.14.fusion.weight", "module.layers.14.fusion.bias", "module.layers.15.att_spatial.layer_scale_1", "module.layers.15.att_spatial.layer_scale_2", "module.layers.15.att_spatial.norm1.weight", "module.layers.15.att_spatial.norm1.bias", "module.layers.15.att_spatial.mixer.proj.weight", "module.layers.15.att_spatial.mixer.proj.bias", "module.layers.15.att_spatial.mixer.qkv.weight", "module.layers.15.att_spatial.norm2.weight", "module.layers.15.att_spatial.norm2.bias", "module.layers.15.att_spatial.mlp.fc1.weight", "module.layers.15.att_spatial.mlp.fc1.bias", "module.layers.15.att_spatial.mlp.fc2.weight", "module.layers.15.att_spatial.mlp.fc2.bias", "module.layers.15.att_temporal.layer_scale_1", "module.layers.15.att_temporal.layer_scale_2", "module.layers.15.att_temporal.norm1.weight", "module.layers.15.att_temporal.norm1.bias", "module.layers.15.att_temporal.mixer.proj.weight", "module.layers.15.att_temporal.mixer.proj.bias", "module.layers.15.att_temporal.mixer.qkv.weight", "module.layers.15.att_temporal.norm2.weight", "module.layers.15.att_temporal.norm2.bias", "module.layers.15.att_temporal.mlp.fc1.weight", "module.layers.15.att_temporal.mlp.fc1.bias", "module.layers.15.att_temporal.mlp.fc2.weight", "module.layers.15.att_temporal.mlp.fc2.bias", "module.layers.15.graph_spatial.layer_scale_1", "module.layers.15.graph_spatial.layer_scale_2", "module.layers.15.graph_spatial.norm1.weight", "module.layers.15.graph_spatial.norm1.bias", "module.layers.15.graph_spatial.mixer.U.weight", "module.layers.15.graph_spatial.mixer.U.bias", "module.layers.15.graph_spatial.mixer.V.weight", "module.layers.15.graph_spatial.mixer.V.bias", "module.layers.15.graph_spatial.mixer.batch_norm.weight", "module.layers.15.graph_spatial.mixer.batch_norm.bias", "module.layers.15.graph_spatial.mixer.batch_norm.running_mean", "module.layers.15.graph_spatial.mixer.batch_norm.running_var", "module.layers.15.graph_spatial.norm2.weight", "module.layers.15.graph_spatial.norm2.bias", "module.layers.15.graph_spatial.mlp.fc1.weight", "module.layers.15.graph_spatial.mlp.fc1.bias", "module.layers.15.graph_spatial.mlp.fc2.weight", "module.layers.15.graph_spatial.mlp.fc2.bias", "module.layers.15.graph_temporal.layer_scale_1", "module.layers.15.graph_temporal.layer_scale_2", "module.layers.15.graph_temporal.norm1.weight", "module.layers.15.graph_temporal.norm1.bias", "module.layers.15.graph_temporal.mixer.U.weight", "module.layers.15.graph_temporal.mixer.U.bias", "module.layers.15.graph_temporal.mixer.V.weight", "module.layers.15.graph_temporal.mixer.V.bias", "module.layers.15.graph_temporal.mixer.batch_norm.weight", "module.layers.15.graph_temporal.mixer.batch_norm.bias", "module.layers.15.graph_temporal.mixer.batch_norm.running_mean", "module.layers.15.graph_temporal.mixer.batch_norm.running_var", "module.layers.15.graph_temporal.norm2.weight", "module.layers.15.graph_temporal.norm2.bias", "module.layers.15.graph_temporal.mlp.fc1.weight", "module.layers.15.graph_temporal.mlp.fc1.bias", "module.layers.15.graph_temporal.mlp.fc2.weight", "module.layers.15.graph_temporal.mlp.fc2.bias", "module.layers.15.fusion.weight", "module.layers.15.fusion.bias".

It seems that the code when loading the model is not correlated with the weights. Do you have any idea? Also, just in case you changed the code, could you provide me with the code to load the model so that I can use those weights?

I appreciate any help you can provide.

@SoroushMehraban
Copy link
Member

@claragarciamoll Sorry for the late response. Based on the error you provided, it seems that you are using the checkpoint from MotionAGFromer-XS but the model you load is MotionAGformer-B. Are you sure you are using MotionAGFormer-XS in initialization? You have to use the config of MotionAGFormer-XS

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants