Error when running Deformable DETR

[goku-krish10]

Hello,

Thank you for this work and making the code available public to help us with our research purposes.
I have been trying to run Deformable DETR with multiple feature levels for swinb. I get the following error. But I have been able to run it successfully with resnet50. Can you please help me with that ?

File "main.py", line 301, in main
model, criterion, data_loader_train, optimizer, device, epoch, args.clip_max_norm)
File "/netscratch/govindan/transvod_lite/TransVOD_Lite/engine_single.py", line 44, in train_one_epoch
outputs = model(samples)
File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.6/site-packages/torch/nn/parallel/distributed.py", line 596, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/netscratch/govindan/transvod_lite/TransVOD_Lite/models/deformable_detr_single.py", line 151, in forward
src = self.input_projl
File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/container.py", line 117, in forward
input = module(input)
File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 423, in forward
return self._conv_forward(input, self.weight)
File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 420, in _conv_forward
self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight of size [256, 512, 1, 1], expected input[4, 256, 71, 125] to have 512 channels, but got 256 channels instead

[qianyuzqy]

Hi, @goku-krish10 ,

Sorry for the late reply. Did you use the same implementation of our code or directly modify the Deformable DETR?
Are you trying to pass multi-scale features coming from the Swin_b backbone to Deformable DETR as [Sairam13001] done in this issue: #10?

In our code, we upsample and aggregate the features of swinb to a single scale feature, since we want to use each frame of the video clip as a 'scale' in the multi-scale deformable attention. We have not tried to pass multi-scale features coming from the Swin_b backbone to Deformable DETR yet.

Thanks for your interest in our work.

[goku-krish10]

I did not make any changes. I used the same code for Deformable DETR with num of features as more than 1 and I got this error.