The reproduction results look poor -- (edit: stacked foreground object due to noisy motion mask)

[Yuhuoo]

Why does the result I reproduced perform worse than the result reported in your report?
Here are the results of my reproduction.

2024-10-22.125247.mp4

[Junyi42]

Hi @Yuhuoo,

Thanks for your feedback. As mentioned in #3, our visualization is based on the ground truth motion mask of DAVIS.

To get a better visualization when the motion mask is noisy, you could simply add a --no_mask for the visualization code (i.e., python viser/visualizer_monst3r.py --data path/schoolgirls --no_mask), which will not stack all the background pointclouds.

Alternatively, you could also add a --use_gt_mask for the evaluation script here https://github.com/Junyi42/monst3r?tab=readme-ov-file#evaluation, which should be able to generate similar results as our online visualization.

Thanks.

[Junyi42]

Hi @Yuhuoo,

Here's the result of using --no_mask for the visualization, as reference.

Screen.Recording.2024-10-22.at.12.20.17.AM.mov

I have also updated the README for detailed information. Please feel free to let me know if you have any further questions.

Thanks.

[koalamind]

@Yuhuoo a couple of questions about your Screen.Recording.2024-10-22.at.12.20.17.AM.mov : 1) are the kids moving back and forth a model projection, that is, a simplified 3D prediction of points moving over time? 2) is it possible to operate a video sensistivity analysis on the 3D projections (i.e., simulating a kid initiatl movement and predicting its 3D move projections over time based on historical data generalising movements over time? Dan