How can I get the ground truth of flow vector？

[Daisy-7]

Thank you for your significant work, it helps me a lot!
I want to train the model on other datasets with lidar data. How can I get the flow vectors from the lidar data? Could you please provide some references or public code?

[myronzbrown]

Our code for that part was not publicly released because there wasn't much other public image data that would directly make use of it. The high level steps would be (1) produce accurate DSM and DTM layers from lidar and then the difference is a DHM indicating object heights above ground (in meters), (2) align an unrectified image to the lidar DSM and update the camera model, and (3) project DSM and DTM coordinates into the image and then the difference is the flow vector (in pixels).

[Daisy-7]

@myronzbrown
Thanks for your reply!

Is there any additional requirement for lidar data to directly use the code? Some cities, like New York and Maryland, have released lidar data in laz or las files. Can these data be used to generate flow vectors with the code?

Unfortunately, I realize I cannot reproduce this method based on current information as I do not have the knowledge base to do so.

The flow vector is an essential character of ground objects, which can be used to demonstrate other tasks for remote sensing. It would be meaningful if you could provide a more detailed process for obtaining the flow vector.

[myronzbrown]

Producing a DSM from LAS/LAZ is easy. Producing a bare earth DTM is complex, requiring accurate classification of ground and non-ground points. There are open source algorithms that can help with this, but if it's not your focus then it may make more sense to identify public DTM products that were produced professionally. Many federal and local governments have this data available from their surveys.

For the specifics of how to get the flow vectors, the math will depend on the camera model for your image, but you'll want to project xyz values from the DSM and DTM to image coordinates to get the end points of the flow vectors.