problem in visualizing the ground truth and prediction keypoints by Visualization.py

[giesth39]

Hi, let me ask you just a few more questions;

i tried
python3 visualize.py --checkpoint pretrain/evaluation/ah -eid 37 --sample_idx 0 --setup 0 --pair 1,3

1. (I've changed in_dataset_adapt to pretrain since there is no 37-set0-1,3 log in in_dataset_adapt folder
   when i tried
   python3 adapt_detnet_dual.py -trs ah -tes ah --root_idx 0 --pic 1024 --resume -eid 37 --epochs 10 --start_epoch 1 --gpus 0 --checkpoint in_dataset_adapt --setup 0 --pair 1,2
   and
   python3 adapt_detnet_dual.py -trs ah -tes ah --evaluate -eid 37 --gpus 0 --pic -1 --checkpoint pretrain --setup 0 --pair 1,3.
   the former would output logs like 1-set0-1,2.log ... 10-set0-1,2.log in in_dataset_adapt/evaluation/ah folder, and the latter would output log 37-set0-1,3.log in pretrain/evaluation/ah folder.)

2. (also, i changed crop root from './data/assemblyhands_crop/images/ego_images_rectified/test/' to './data/assemblyhands_crop/images/ego_images_rectified/val/', since i changed data_split ='val'.)

but since i didn't have a display on the terminal where i was trying the command, i used jupyter notebook to run the visualize code.

i changed the crop_root from /test/ to /val/, since i did
python3 adapt_detnet_dual.py -trs ah -tes ah --evaluate -eid 37 --gpus 0 --pic -1 --checkpoint pretrain --setup 0 --pair 1,3
on the 'val' folder

but i got the result like the image 3.

i wonder what would be the problem about the visualization. plus, when i tried print(kp),
i get
[[ -41 432 0]
[ -588 700 -16]
[-3075 2343 -5]
[ 2565 -1444 9]
[ 1605 -956 17]
[ 607 544 32]
[ 332 476 70]
[ 343 365 77]
[ 350 272 76]
[ 284 545 47]
[ 228 393 84]
[ 243 293 84]
[ 239 213 80]
[ 140 494 56]
[ 131 309 76]
[ 185 157 52]
[ 273 -71 33]
[ 26 460 62]
[ 54 307 71]
[ 74 185 54]
[ 89 83 41]]
[[-159 -85 0]
[ -79 -398 23]
[ 1 -246 51]
[ 15 -201 76]
[ 13 -196 95]
[ 680 -380 21]
[ 562 -196 39]
[ 344 -128 60]
[ 247 -91 74]
[1054 -88 14]
[ 470 -39 43]
[ 229 -43 71]
[ 147 -42 92]
[ 882 241 15]
[ 224 38 52]
[ 78 -6 79]
[ 8 -35 84]
[ 542 402 18]
[ 121 90 53]
[ 35 37 65]
[ -16 4 70]]

and also i got same print(kp) result when i tried in terminal.

Could you tell me what would be the problem?

[MickeyLLG]

Hi Giesth,
Thanks for being interested in our work and pointing our this issue. I have gone through your comment and your operations are all correct.
I have addressed this bug and it is because the predictions and gts are wrist-aligned during inference, leading to invalid 3D position in the camera coordinate systems. To address this issue, I have updated the align.py, adapt_detnet_dual.py, and visualize.py.
From your point of view, you can

1. pull the updated python files above and regenerate those log files.
2. comment Lines 54-55 of align.py, then regenerate those log files so that the predictions and gts in the log files will be correctly visualized. However, this will lead to wrong results when running calc_metrics.py. This is a quicker way to see the effect but I would recommend the first solution.