Performance on DEIT-base model

[Ferenas]

Hi, thanks for sharing your good work! I would like to raise a question after I run the code. Since MCT-former is only adopted in VIT-small architecture which is pretrained with Deit, I have tried the DEIT-base model on this architecture. However, I found that the CAM of DEIT-base is drastically degraded, which only achieves 22.5% mIoU with patch refine in MCT-former1. Therefore, I would like to ask that whether you have tried this method on other backbones.

[111chengxuyuan]

Hello, I have encountered the same problem as you. I also tried to replace Backbone with DeiT-base, but only achieved 19.7 results. I wonder that have you solved this problem? thank you

[Ferenas]

Actually I have not solved this problem yet after several trials, including parameters-tuning. Therefore, personally I think this method may simply be effective on diet-small, and same phenomenon could be found in some new papers related to WSSS.在 2023年3月21日，14:52，111chengxuyuan ***@***.***> 写道： Hello, I have encountered the same problem as you. I also tried to replace Backbone with DeiT-base, but only achieved 19.7 results. I wonder that have you solved this problem? thank you —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[111chengxuyuan]

Ok, Thank you for your reply. @Ferenas

[yijiangWang0316]

Ok, Thank you for your reply. @Ferenas

Hello，Could you provide the command line parameters for "python seg/train_seg.py" and "python seg/infer_seg.py" during training? (There are some path issues, and you haven't provided detailed paths, so I don't know how to match the required paths.) When I am training, I keep getting errors when reading. When "img_path" is set to "JPEGImages", it prompts that there is no xxx image in the pgt-psa-rw file in the - -seg_pgt_path: MCTformerV2 result file. When the --seg_pgt_path path is set to MCTformer_v2/attn-patchrefine, the same error occurs, saying that there is no file named attn-patchrefine/2009_004377.png. I am not very clear about this.
I have been following the author's running script to implement this, and I don't know why this problem occurs.Thank you very much for your help。

[yijiangWang0316]

Hello，Could you provide the command line parameters for "python seg/train_seg.py" and "python seg/infer_seg.py" during training? (There are some path issues, and you haven't provided detailed paths, so I don't know how to match the required paths.) When I am training, I keep getting errors when reading. When "img_path" is set to "JPEGImages", it prompts that there is no xxx image in the pgt-psa-rw file in the - -seg_pgt_path: MCTformerV2 result file. When the --seg_pgt_path path is set to MCTformer_v2/attn-patchrefine, the same error occurs, saying that there is no file named attn-patchrefine/2009_004377.png. I am not very clear about this.
I have been following the author's running script to implement this, and I don't know why this problem occurs.Thank you very much for your help。

[111chengxuyuan]

@yijiangWang0316 Hi,my train.sh is as follows:
`python seg/train_seg.py --network resnet38_seg
--num_epochs 30
--seg_pgt_path ./VOCdevkit/VOC2012/SegmentationClassAug/
--init_weights /home/Weights/MCTformer/res38_cls.pth
--save_path ./temp_results
--list_path voc12/train_aug_id.txt
--img_path ./VOCdevkit/VOC2012/JPEGImages/
--num_classes 21
--batch_size 4

python seg/infer_seg.py --weights ./temp_results/model_29.pth
--network resnet38_seg
--list_path voc12/val_id.txt
--gt_path ./VOCdevkit/VOC2012/SegmentationClass/
--img_path ./VOCdevkit/VOC2012/JPEGImages/
--save_path val_ms_crf
--save_path_c val_ms_crf_c
--scales 0.5 0.75 1.0 1.25 1.5
--use_crf True`

[yijiangWang0316]

@111chengxuyuan Hello, Thank you very much for your response and assistance. Using the path you mentioned, the problem has been resolved and can be trained normally. The number of split iterations is 105800, and the training is not fast.

[yijiangWang0316]

@111chengxuyuan Hello, why does my segmentation result on the verification set become very high after setting up your path? The MIOU can reach 76.4%! It is 5 points higher than the author's. Is there a problem with the path?
Thank you very much for your reply! Here are my verification set results：
class 0 background IU 0.938
class 1 aeroplane IU 0.876
class 2 bicycle IU 0.393
class 3 bird IU 0.903
class 4 boat IU 0.679
class 5 bottle IU 0.781
class 6 bus IU 0.924
class 7 car IU 0.843
class 8 cat IU 0.918
class 9 chair IU 0.369
class 10 cow IU 0.859
class 11 diningtable IU 0.646
class 12 dog IU 0.870
class 13 horse IU 0.836
class 14 motorbike IU 0.824
class 15 person IU 0.838
class 16 pottedplant IU 0.669
class 17 sheep IU 0.888
class 18 sofa IU 0.490
class 19 train IU 0.835
class 20 tvmonitor IU 0.658
mIoU = 0.764

@yijiangWang0316 Hi,my train.sh is as follows: `python seg/train_seg.py --network resnet38_seg --num_epochs 30 --seg_pgt_path ./VOCdevkit/VOC2012/SegmentationClassAug/ --init_weights /home/Weights/MCTformer/res38_cls.pth --save_path ./temp_results --list_path voc12/train_aug_id.txt --img_path ./VOCdevkit/VOC2012/JPEGImages/ --num_classes 21 --batch_size 4

python seg/infer_seg.py --weights ./temp_results/model_29.pth --network resnet38_seg --list_path voc12/val_id.txt --gt_path ./VOCdevkit/VOC2012/SegmentationClass/ --img_path ./VOCdevkit/VOC2012/JPEGImages/ --save_path val_ms_crf --save_path_c val_ms_crf_c --scales 0.5 0.75 1.0 1.25 1.5 --use_crf True`