frame's timestamp value loses its precision when cast from int64 to np.float32 during the conversion of Waymo dataset

[zhanggefan]

Hi,

It looks like a small bug exists here. Waymo dataset uses type int64 for timestamp, but the following code casts its value from int64 to float64 at line 239 (behavior of numpy v1.18.5, though intensity is float32), and at line 247 casts its value further from float64 to float32.

For a typical waymo frame timestamp value, e.g. 1550083467346370, will lose precision and be cast into 1550083493462016. A big value of similar range has a low precision of 134,217,728 (physically 134s) when represented by float32.

This should have little impact though. But I found that the output waymo_infos_*.pkl files are affected, in which nearly all frames has the same issue: they and their previous 'sweeps' have exactly the same timestamp.

mmdetection3d/tools/data_converter/waymo_converter.py

Lines 236 to 247 in 1b39a48

	points = np.concatenate([points_0, points_1], axis=0)
	intensity = np.concatenate([intensity_0, intensity_1], axis=0)
	elongation = np.concatenate([elongation_0, elongation_1], axis=0)
	timestamp = frame.timestamp_micros * np.ones_like(intensity)
	# concatenate x,y,z, intensity, elongation, timestamp (6-dim)
	point_cloud = np.column_stack(
	(points, intensity, elongation, timestamp))
	pc_path = f'{self.point_cloud_save_dir}/{self.prefix}' + \
	f'{str(file_idx).zfill(3)}{str(frame_idx).zfill(3)}.bin'
	point_cloud.astype(np.float32).tofile(pc_path)

[Tai-Wang]

Thanks for your suggestion. Yes, I agree with you. Because we actually do not play much with consecutive frames or use this timestamp information, we are not aware of this potential information loss. If you have some experience of it, could you please create a PR to fix it?

[zhanggefan]

Hi,

Thank you for your response!

I am now working on it and the PR is on the way. But I have a question I need to confirm with you in advance.

Could I change the output point cloud NumPy array's shape?

If I could, I would remove the timestamp column, and store the mappings from the output bin file name to its timestamp in a separate file.

• Reason: extending an output column for an identical value (waymo_converter.py L239) is obviously not efficient. It is even more problematic when generating waymo_infos_*.pkl, where for each bin file we need to read other 10 bin files to extract ONLY the timestamp.

mmdetection3d/tools/data_converter/kitti_data_utils.py

Lines 318 to 321 in 1b39a48

	points = np.fromfile(
	Path(path) / pc_info['velodyne_path'], dtype=np.float32)
	points = np.copy(points).reshape(-1, pc_info['num_features'])
	info['timestamp'] = np.int64(points[0, -1])

mmdetection3d/tools/data_converter/kitti_data_utils.py

Lines 418 to 422 in 1b39a48

	prev_points = np.fromfile(
	Path(path) / prev_info['velodyne_path'], dtype=np.float32)
	prev_points = np.copy(prev_points).reshape(
	-1, pc_info['num_features'])
	prev_info['timestamp'] = np.int64(prev_points[0, -1])

• Drawback: this would be a relatively big change because the code of WaymoDataset and config files would need to adapt as well. And tons of *.bin files would need to be regenerated again (harmful to the community cause many people spend days to convert those data).

[Tai-Wang]

Yes, it makes sense. I find it is a little inappropriate to save timestamp with point features indeed. NuScenes also records the timestamp information as one kind of meta data per frame.
As for the influence of this change over other current implementation, it would be appreciate if you could basically validate it by going through an entire pipeline (including data generation, model training and testing). I guess only very few changes are needed to make it compatible because our current models only use the first 5 features (without timestamp).