Poor results with pretrained models

[FHSatis]

Hi there,

When trying to run visualisation with pretrained models I am getting really poor results.

Command:
python demo.py --config-file ../configs/ovis/video_maskformer2_R50_bs32_8ep_frame.yaml --input ../datasets/ovis/test/0d11ec15 --output demo_outputs --opts MODEL.WEIGHTS ../pretrained/minvis_ovis_R50.pth

Output:

Console Log:

Weight format of MaskFormerHead have changed! Please upgrade your models. Applying automatic conversion now ...
WARNING [09/30 17:56:38 fvcore.common.checkpoint]: Skip loading parameter 'sem_seg_head.pixel_decoder.adapter_1.weight' to the model due to incompatible shapes: (256, 256, 1, 1) in the checkpoint but (256, 192, 1, 1) in the model! You might want to double check if this is expected.
WARNING [09/30 17:56:38 fvcore.common.checkpoint]: Skip loading parameter 'sem_seg_head.pixel_decoder.input_proj.0.0.weight' to the model due to incompatible shapes: (256, 2048, 1, 1) in the checkpoint but (256, 1536, 1, 1) in the model! You might want to double check if this is expected.
WARNING [09/30 17:56:38 fvcore.common.checkpoint]: Skip loading parameter 'sem_seg_head.pixel_decoder.input_proj.1.0.weight' to the model due to incompatible shapes: (256, 1024, 1, 1) in the checkpoint but (256, 768, 1, 1) in the model! You might want to double check if this is expected.
WARNING [09/30 17:56:38 fvcore.common.checkpoint]: Skip loading parameter 'sem_seg_head.pixel_decoder.input_proj.2.0.weight' to the model due to incompatible shapes: (256, 512, 1, 1) in the checkpoint but (256, 384, 1, 1) in the model! You might want to double check if this is expected.
WARNING [09/30 17:56:38 fvcore.common.checkpoint]: Skip loading parameter 'sem_seg_head.predictor.query_embed.weight' to the model due to incompatible shapes: (100, 256) in the checkpoint but (200, 256) in the model! You might want to double check if this is expected.
WARNING [09/30 17:56:38 fvcore.common.checkpoint]: Skip loading parameter 'sem_seg_head.predictor.query_feat.weight' to the model due to incompatible shapes: (100, 256) in the checkpoint but (200, 256) in the model! You might want to double check if this is expected.

WARNING [09/30 17:56:38 fvcore.common.checkpoint]: Some model parameters or buffers are not found in the checkpoint:
backbone.layers.0.blocks.0.attn.proj.{bias, weight}
backbone.layers.0.blocks.0.attn.qkv.{bias, weight}
backbone.layers.0.blocks.0.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.0.blocks.0.mlp.fc1.{bias, weight}
backbone.layers.0.blocks.0.mlp.fc2.{bias, weight}
backbone.layers.0.blocks.0.norm1.{bias, weight}
backbone.layers.0.blocks.0.norm2.{bias, weight}
backbone.layers.0.blocks.1.attn.proj.{bias, weight}
backbone.layers.0.blocks.1.attn.qkv.{bias, weight}
backbone.layers.0.blocks.1.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.0.blocks.1.mlp.fc1.{bias, weight}
backbone.layers.0.blocks.1.mlp.fc2.{bias, weight}
backbone.layers.0.blocks.1.norm1.{bias, weight}
backbone.layers.0.blocks.1.norm2.{bias, weight}
backbone.layers.0.downsample.norm.{bias, weight}
backbone.layers.0.downsample.reduction.weight
backbone.layers.1.blocks.0.attn.proj.{bias, weight}
backbone.layers.1.blocks.0.attn.qkv.{bias, weight}
backbone.layers.1.blocks.0.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.1.blocks.0.mlp.fc1.{bias, weight}
backbone.layers.1.blocks.0.mlp.fc2.{bias, weight}
backbone.layers.1.blocks.0.norm1.{bias, weight}
backbone.layers.1.blocks.0.norm2.{bias, weight}
backbone.layers.1.blocks.1.attn.proj.{bias, weight}
backbone.layers.1.blocks.1.attn.qkv.{bias, weight}
backbone.layers.1.blocks.1.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.1.blocks.1.mlp.fc1.{bias, weight}
backbone.layers.1.blocks.1.mlp.fc2.{bias, weight}
backbone.layers.1.blocks.1.norm1.{bias, weight}
backbone.layers.1.blocks.1.norm2.{bias, weight}
backbone.layers.1.downsample.norm.{bias, weight}
backbone.layers.1.downsample.reduction.weight
backbone.layers.2.blocks.0.attn.proj.{bias, weight}
backbone.layers.2.blocks.0.attn.qkv.{bias, weight}
backbone.layers.2.blocks.0.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.0.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.0.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.0.norm1.{bias, weight}
backbone.layers.2.blocks.0.norm2.{bias, weight}
backbone.layers.2.blocks.1.attn.proj.{bias, weight}
backbone.layers.2.blocks.1.attn.qkv.{bias, weight}
backbone.layers.2.blocks.1.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.1.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.1.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.1.norm1.{bias, weight}
backbone.layers.2.blocks.1.norm2.{bias, weight}
backbone.layers.2.blocks.10.attn.proj.{bias, weight}
backbone.layers.2.blocks.10.attn.qkv.{bias, weight}
backbone.layers.2.blocks.10.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.10.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.10.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.10.norm1.{bias, weight}
backbone.layers.2.blocks.10.norm2.{bias, weight}
backbone.layers.2.blocks.11.attn.proj.{bias, weight}
backbone.layers.2.blocks.11.attn.qkv.{bias, weight}
backbone.layers.2.blocks.11.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.11.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.11.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.11.norm1.{bias, weight}
backbone.layers.2.blocks.11.norm2.{bias, weight}
backbone.layers.2.blocks.12.attn.proj.{bias, weight}
backbone.layers.2.blocks.12.attn.qkv.{bias, weight}
backbone.layers.2.blocks.12.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.12.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.12.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.12.norm1.{bias, weight}
backbone.layers.2.blocks.12.norm2.{bias, weight}
backbone.layers.2.blocks.13.attn.proj.{bias, weight}
backbone.layers.2.blocks.13.attn.qkv.{bias, weight}
backbone.layers.2.blocks.13.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.13.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.13.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.13.norm1.{bias, weight}
backbone.layers.2.blocks.13.norm2.{bias, weight}
backbone.layers.2.blocks.14.attn.proj.{bias, weight}
backbone.layers.2.blocks.14.attn.qkv.{bias, weight}
backbone.layers.2.blocks.14.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.14.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.14.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.14.norm1.{bias, weight}
backbone.layers.2.blocks.14.norm2.{bias, weight}
backbone.layers.2.blocks.15.attn.proj.{bias, weight}
backbone.layers.2.blocks.15.attn.qkv.{bias, weight}
backbone.layers.2.blocks.15.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.15.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.15.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.15.norm1.{bias, weight}
backbone.layers.2.blocks.15.norm2.{bias, weight}
backbone.layers.2.blocks.16.attn.proj.{bias, weight}
backbone.layers.2.blocks.16.attn.qkv.{bias, weight}
backbone.layers.2.blocks.16.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.16.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.16.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.16.norm1.{bias, weight}
backbone.layers.2.blocks.16.norm2.{bias, weight}
backbone.layers.2.blocks.17.attn.proj.{bias, weight}
backbone.layers.2.blocks.17.attn.qkv.{bias, weight}
backbone.layers.2.blocks.17.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.17.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.17.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.17.norm1.{bias, weight}
backbone.layers.2.blocks.17.norm2.{bias, weight}
backbone.layers.2.blocks.2.attn.proj.{bias, weight}
backbone.layers.2.blocks.2.attn.qkv.{bias, weight}
backbone.layers.2.blocks.2.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.2.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.2.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.2.norm1.{bias, weight}
backbone.layers.2.blocks.2.norm2.{bias, weight}
backbone.layers.2.blocks.3.attn.proj.{bias, weight}
backbone.layers.2.blocks.3.attn.qkv.{bias, weight}
backbone.layers.2.blocks.3.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.3.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.3.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.3.norm1.{bias, weight}
backbone.layers.2.blocks.3.norm2.{bias, weight}
backbone.layers.2.blocks.4.attn.proj.{bias, weight}
backbone.layers.2.blocks.4.attn.qkv.{bias, weight}
backbone.layers.2.blocks.4.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.4.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.4.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.4.norm1.{bias, weight}
backbone.layers.2.blocks.4.norm2.{bias, weight}
backbone.layers.2.blocks.5.attn.proj.{bias, weight}
backbone.layers.2.blocks.5.attn.qkv.{bias, weight}
backbone.layers.2.blocks.5.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.5.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.5.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.5.norm1.{bias, weight}
backbone.layers.2.blocks.5.norm2.{bias, weight}
backbone.layers.2.blocks.6.attn.proj.{bias, weight}
backbone.layers.2.blocks.6.attn.qkv.{bias, weight}
backbone.layers.2.blocks.6.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.6.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.6.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.6.norm1.{bias, weight}
backbone.layers.2.blocks.6.norm2.{bias, weight}
backbone.layers.2.blocks.7.attn.proj.{bias, weight}
backbone.layers.2.blocks.7.attn.qkv.{bias, weight}
backbone.layers.2.blocks.7.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.7.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.7.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.7.norm1.{bias, weight}
backbone.layers.2.blocks.7.norm2.{bias, weight}
backbone.layers.2.blocks.8.attn.proj.{bias, weight}
backbone.layers.2.blocks.8.attn.qkv.{bias, weight}
backbone.layers.2.blocks.8.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.8.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.8.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.8.norm1.{bias, weight}
backbone.layers.2.blocks.8.norm2.{bias, weight}
backbone.layers.2.blocks.9.attn.proj.{bias, weight}
backbone.layers.2.blocks.9.attn.qkv.{bias, weight}
backbone.layers.2.blocks.9.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.2.blocks.9.mlp.fc1.{bias, weight}
backbone.layers.2.blocks.9.mlp.fc2.{bias, weight}
backbone.layers.2.blocks.9.norm1.{bias, weight}
backbone.layers.2.blocks.9.norm2.{bias, weight}
backbone.layers.2.downsample.norm.{bias, weight}
backbone.layers.2.downsample.reduction.weight
backbone.layers.3.blocks.0.attn.proj.{bias, weight}
backbone.layers.3.blocks.0.attn.qkv.{bias, weight}
backbone.layers.3.blocks.0.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.3.blocks.0.mlp.fc1.{bias, weight}
backbone.layers.3.blocks.0.mlp.fc2.{bias, weight}
backbone.layers.3.blocks.0.norm1.{bias, weight}
backbone.layers.3.blocks.0.norm2.{bias, weight}
backbone.layers.3.blocks.1.attn.proj.{bias, weight}
backbone.layers.3.blocks.1.attn.qkv.{bias, weight}
backbone.layers.3.blocks.1.attn.{relative_position_bias_table, relative_position_index}
backbone.layers.3.blocks.1.mlp.fc1.{bias, weight}
backbone.layers.3.blocks.1.mlp.fc2.{bias, weight}
backbone.layers.3.blocks.1.norm1.{bias, weight}
backbone.layers.3.blocks.1.norm2.{bias, weight}
backbone.norm0.{bias, weight}
backbone.norm1.{bias, weight}
backbone.norm2.{bias, weight}
backbone.norm3.{bias, weight}
backbone.patch_embed.norm.{bias, weight}
backbone.patch_embed.proj.{bias, weight}
sem_seg_head.pixel_decoder.adapter_1.norm.{bias, weight}
sem_seg_head.pixel_decoder.adapter_1.weight
sem_seg_head.pixel_decoder.input_proj.0.0.{bias, weight}
sem_seg_head.pixel_decoder.input_proj.0.1.{bias, weight}
sem_seg_head.pixel_decoder.input_proj.1.0.{bias, weight}
sem_seg_head.pixel_decoder.input_proj.1.1.{bias, weight}
sem_seg_head.pixel_decoder.input_proj.2.0.{bias, weight}
sem_seg_head.pixel_decoder.input_proj.2.1.{bias, weight}
sem_seg_head.pixel_decoder.layer_1.norm.{bias, weight}
sem_seg_head.pixel_decoder.layer_1.weight
sem_seg_head.pixel_decoder.mask_features.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.0.linear1.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.0.linear2.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.0.norm1.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.0.norm2.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.0.self_attn.attention_weights.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.0.self_attn.output_proj.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.0.self_attn.sampling_offsets.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.0.self_attn.value_proj.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.1.linear1.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.1.linear2.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.1.norm1.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.1.norm2.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.1.self_attn.attention_weights.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.1.self_attn.output_proj.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.1.self_attn.sampling_offsets.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.1.self_attn.value_proj.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.2.linear1.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.2.linear2.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.2.norm1.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.2.norm2.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.2.self_attn.attention_weights.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.2.self_attn.output_proj.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.2.self_attn.sampling_offsets.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.2.self_attn.value_proj.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.3.linear1.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.3.linear2.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.3.norm1.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.3.norm2.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.3.self_attn.attention_weights.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.3.self_attn.output_proj.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.3.self_attn.sampling_offsets.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.3.self_attn.value_proj.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.4.linear1.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.4.linear2.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.4.norm1.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.4.norm2.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.4.self_attn.attention_weights.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.4.self_attn.output_proj.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.4.self_attn.sampling_offsets.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.4.self_attn.value_proj.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.5.linear1.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.5.linear2.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.5.norm1.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.5.norm2.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.5.self_attn.attention_weights.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.5.self_attn.output_proj.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.5.self_attn.sampling_offsets.{bias, weight}
sem_seg_head.pixel_decoder.transformer.encoder.layers.5.self_attn.value_proj.{bias, weight}
sem_seg_head.pixel_decoder.transformer.level_embed
sem_seg_head.predictor.query_embed.weight
sem_seg_head.predictor.query_feat.weight
WARNING [09/30 17:56:38 fvcore.common.checkpoint]: The checkpoint state_dict contains keys that are not used by the model:
  backbone.res2.0.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res2.0.conv1.weight
  backbone.res2.0.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res2.0.conv2.weight
  backbone.res2.0.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res2.0.conv3.weight
  backbone.res2.0.shortcut.norm.{bias, running_mean, running_var, weight}
  backbone.res2.0.shortcut.weight
  backbone.res2.1.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res2.1.conv1.weight
  backbone.res2.1.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res2.1.conv2.weight
  backbone.res2.1.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res2.1.conv3.weight
  backbone.res2.2.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res2.2.conv1.weight
  backbone.res2.2.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res2.2.conv2.weight
  backbone.res2.2.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res2.2.conv3.weight
  backbone.res3.0.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res3.0.conv1.weight
  backbone.res3.0.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res3.0.conv2.weight
  backbone.res3.0.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res3.0.conv3.weight
  backbone.res3.0.shortcut.norm.{bias, running_mean, running_var, weight}
  backbone.res3.0.shortcut.weight
  backbone.res3.1.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res3.1.conv1.weight
  backbone.res3.1.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res3.1.conv2.weight
  backbone.res3.1.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res3.1.conv3.weight
  backbone.res3.2.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res3.2.conv1.weight
  backbone.res3.2.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res3.2.conv2.weight
  backbone.res3.2.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res3.2.conv3.weight
  backbone.res3.3.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res3.3.conv1.weight
  backbone.res3.3.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res3.3.conv2.weight
  backbone.res3.3.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res3.3.conv3.weight
  backbone.res4.0.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res4.0.conv1.weight
  backbone.res4.0.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res4.0.conv2.weight
  backbone.res4.0.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res4.0.conv3.weight
  backbone.res4.0.shortcut.norm.{bias, running_mean, running_var, weight}
  backbone.res4.0.shortcut.weight
  backbone.res4.1.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res4.1.conv1.weight
  backbone.res4.1.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res4.1.conv2.weight
  backbone.res4.1.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res4.1.conv3.weight
  backbone.res4.2.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res4.2.conv1.weight
  backbone.res4.2.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res4.2.conv2.weight
  backbone.res4.2.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res4.2.conv3.weight
  backbone.res4.3.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res4.3.conv1.weight
  backbone.res4.3.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res4.3.conv2.weight
  backbone.res4.3.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res4.3.conv3.weight
  backbone.res4.4.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res4.4.conv1.weight
  backbone.res4.4.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res4.4.conv2.weight
  backbone.res4.4.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res4.4.conv3.weight
  backbone.res4.5.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res4.5.conv1.weight
  backbone.res4.5.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res4.5.conv2.weight
  backbone.res4.5.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res4.5.conv3.weight
  backbone.res5.0.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res5.0.conv1.weight
  backbone.res5.0.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res5.0.conv2.weight
  backbone.res5.0.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res5.0.conv3.weight
  backbone.res5.0.shortcut.norm.{bias, running_mean, running_var, weight}
  backbone.res5.0.shortcut.weight
  backbone.res5.1.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res5.1.conv1.weight
  backbone.res5.1.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res5.1.conv2.weight
  backbone.res5.1.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res5.1.conv3.weight
  backbone.res5.2.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.res5.2.conv1.weight
  backbone.res5.2.conv2.norm.{bias, running_mean, running_var, weight}
  backbone.res5.2.conv2.weight
  backbone.res5.2.conv3.norm.{bias, running_mean, running_var, weight}
  backbone.res5.2.conv3.weight
  backbone.stem.conv1.norm.{bias, running_mean, running_var, weight}
  backbone.stem.conv1.weight
  sem_seg_head.pixel_decoder.pixel_decoder.adapter_1.norm.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.input_proj.0.0.bias
  sem_seg_head.pixel_decoder.pixel_decoder.input_proj.0.1.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.input_proj.1.0.bias
  sem_seg_head.pixel_decoder.pixel_decoder.input_proj.1.1.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.input_proj.2.0.bias
  sem_seg_head.pixel_decoder.pixel_decoder.input_proj.2.1.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.layer_1.norm.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.layer_1.weight
  sem_seg_head.pixel_decoder.pixel_decoder.mask_features.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.0.linear1.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.0.linear2.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.0.norm1.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.0.norm2.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.0.self_attn.attention_weights.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.0.self_attn.output_proj.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.0.self_attn.sampling_offsets.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.0.self_attn.value_proj.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.1.linear1.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.1.linear2.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.1.norm1.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.1.norm2.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.1.self_attn.attention_weights.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.1.self_attn.output_proj.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.1.self_attn.sampling_offsets.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.1.self_attn.value_proj.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.2.linear1.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.2.linear2.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.2.norm1.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.2.norm2.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.2.self_attn.attention_weights.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.2.self_attn.output_proj.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.2.self_attn.sampling_offsets.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.2.self_attn.value_proj.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.3.linear1.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.3.linear2.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.3.norm1.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.3.norm2.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.3.self_attn.attention_weights.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.3.self_attn.output_proj.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.3.self_attn.sampling_offsets.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.3.self_attn.value_proj.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.4.linear1.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.4.linear2.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.4.norm1.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.4.norm2.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.4.self_attn.attention_weights.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.4.self_attn.output_proj.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.4.self_attn.sampling_offsets.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.4.self_attn.value_proj.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.5.linear1.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.5.linear2.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.5.norm1.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.5.norm2.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.5.self_attn.attention_weights.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.5.self_attn.output_proj.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.5.self_attn.sampling_offsets.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.encoder.layers.5.self_attn.value_proj.{bias, weight}
  sem_seg_head.pixel_decoder.pixel_decoder.transformer.level_embed

So there seems to be some mismatches in the model mapping. Has there been a config change or are the models within the model zoo now out of date?

Thanks!

[asrlhhh]

Same thing. The result is so poor that I have to check several times to see if I did anything wrong

[asrlhhh]

if the above image can be considered "without obvious foreground object", this one also fails miserably.

[PinxueGuo]

I also face the problem. I found it's because the module names differ between the pre-trained weight and the current model.
Just modify the line34 in mask2former/modeling/meta_arch/
newk = k.replace(prefix, prefix + "pixel_decoder.")
to
newk = k.replace(prefix, prefix)

[HarryHsing]

I also face the problem. I found it's because the module names differ between the pre-trained weight and the current model. Just modify the line34 in mask2former/modeling/meta_arch/ newk = k.replace(prefix, prefix + "pixel_decoder.") to newk = k.replace(prefix, prefix)

Thanks! This comment works.