image2world

[amin-asdzdh]

Hi,

First, thank you for sharing your excellent work and providing the code!

I have a question regarding the image2world method in BaseCamera. The docstring mentions that the method transforms a point from the image plane to 3D world coordinates. Could you clarify if the returned value represents the coordinates of the 3D ray in the Camera Coordinate System?

Additionally, if we assume that the point in the image corresponds to a location on the ground plane in the world coordinate system (i.e., Z=0), is it possible to use this method to estimate the XY coordinates of that point in the World Coordinate System?

Thanks again for your time and assistance!

[sarlinpe]

For a 2D image point p and a pinhole camera, image2world essentially compute P = [(p - c) / f), 1], which is indeed an unnormalized 3D ray.

GeoCalib/geocalib/camera.py

Lines 353 to 359 in a7ef2cf

	def image2world(self, p2d: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]:
	"""Transform point in the image plane to 3D world coordinates."""
	p2d = self.normalize(p2d)
	p2d, valid = self.undistort(p2d)
	ones = p2d.new_ones(p2d.shape[:-1] + (1,))
	p3d = torch.cat([p2d, ones], -1)
	return p3d, valid

To obtain a point on a ground plane, you would need its depth, which GeoCalib does not provide.