about the evaluation of davis2016

[zhouweii234]

I want to evaluate davis2016 with this code https://github.com/davisvideochallenge/davis. But when I want to run eval_view.py to see the result in HDF5 file generated by eval.py, the program can't find the file db_benchmark.yml. This file db_benchmark.yml is neither in the davis2016 dataset nor generated from the program eval.py. Do you know where can I find the program db_benchmark.yml?

Or could you kindly tell me how do you evaluate davis2016? Which code or website do you use?

[hkchengrex]

I used this:
https://github.com/davisvideochallenge/davis2017-evaluation

with this modification:
davisvideochallenge/davis2017-evaluation#4

[zhouweii234]

@hkchengrex
There seems to be something wrong with this modification https://github.com/davisvideochallenge/davis2017-evaluation/issues/4 . It reports In your PNG files there is an index higher than the number of objects in the sequence!.
The modification change the way of loading gt masks (i.e. directly add a dimension to the mask as num_objects , as shown below.)

However, the way of loading result masks didn't change (in results.py def read_masks() ,as shown below), so num_objects is different from that of gt masks.

May I ask you how you solved this problem？

[hkchengrex]

Reading back my code I have modified the previous line to be masks[ii, ...] = (self._read_mask(sequence, m)>128).astype(np.uint8) as well.

[zhouweii234]

@hkchengrex
My evaluation results are all 0 because gt_masks are all become to 0 after the steps in the red box

May I ask how do you modified the get_all_masks() in davis.py?

[hkchengrex]

I'll just share my code here: https://github.com/hkchengrex/davis2016-evaluation
It is not official.

[frank-xwang]

@zhouweii234 Hi, I am facing the same issue. All GT masks became 0 after executing the codes in the red box. Have you addressed this issue?