add a way to orient based on main direction of the surface

[hoanhle]

essentially implement #1887

toy problem visualization

orientation_method == "up" Screencast from 08-26-2024 01:31:40 PM.webm

orientation_method == "align" Screencast from 08-26-2024 12:53:25 PM.webm

[jb-ye]

@hoanhle Could you share a sample visualization of this new orientation method?

[jb-ye]

I feel major refactor of current nerfstudio parser are needed to achieve a clean separation between orientation logics from data parsing.

(1) Currently orientation and centering logics is simply go through helper function: camera_utils.auto_orient_and_center_poses. This is important, because the same helper function is used in multiple parsers (e.g. colmap parser, etc). The current PR only applies to one specific parser and it makes maintaining these logics much more difficult.

(2) Because the new align method depends on the point cloud loaded from PLY, we need to first load point cloud (if they exist) and then compute orientation transforms. This require some refactor of existing codes.

As such, I feel the current PR is not ready to merge until (1) and (2) are addressed.

[jb-ye]

Should you keep the default method be "up"?

[jb-ye]

delete this comment?

[hoanhle]

I feel major refactor of current nerfstudio parser are needed to achieve a clean separation between orientation logics from data parsing.

(1) Currently orientation and centering logics is simply go through helper function: camera_utils.auto_orient_and_center_poses. This is important, because the same helper function is used in multiple parsers (e.g. colmap parser, etc). The current PR only applies to one specific parser and it makes maintaining these logics much more difficult.

(2) Because the new align method depends on the point cloud loaded from PLY, we need to first load point cloud (if they exist) and then compute orientation transforms. This require some refactor of existing codes.

As such, I feel the current PR is not ready to merge until (1) and (2) are addressed.

@jb-ye This makes sense to me. I will proceed with the refactoring unless you know of a simpler way to achieve the same result.

[jb-ye]

@hoanhle That would be great, I'd love to help review your change once they are ready.