Bugs when run tools/create_data.py

[ZhangYu1ing]

Errors
we try to run the following code to generate .pkl files. However, it met the bugs when the GT database was generated:
python tools/create_data.py nuscenes --root-path ./data/nuscenes --version "v1.0-mini" --out-dir ./data/nuscenes --extra-tag nuscenes

Reproduction

1. What command or script did you run?
   python tools/create_data.py nuscenes --root-path ./data/nuscenes --version "v1.0-mini" --out-dir ./data/nuscenes --extra-tag nuscenes

2. Did you make any modifications on the code or config? Did you understand what you have modified?

• No, we did not modify anything on the code.

3. What dataset did you use?

• nuScenes v1.0-mini

Environment

1. Please run python mmdet3d/utils/collect_env.py to collect necessary environment infomation and paste it here.
   sys.platform: linux
   Python: 3.7.10 (default, Feb 26 2021, 18:47:35) [GCC 7.3.0]
   CUDA available: True
   GPU 0: GeForce RTX 3090
   CUDA_HOME: /usr/local/cuda
   NVCC: Build cuda_11.0_bu.TC445_37.28540450_0
   GCC: gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
   PyTorch: 1.7.1
   PyTorch compiling details: PyTorch built with:

• GCC 7.3
• C++ Version: 201402
• Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v1.6.0 (Git Hash 5ef631a030a6f73131c77892041042805a06064f)
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• NNPACK is enabled
• CPU capability usage: AVX2
• CUDA Runtime 11.0
• NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_37,code=compute_37
• CuDNN 8.0.5
• Magma 2.5.2
• Build settings: BLAS=MKL, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN_WRAPPER -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.8.2
OpenCV: 4.5.1
MMCV: 1.3.0
MMCV Compiler: GCC 9.3
MMCV CUDA Compiler: 11.0
MMDetection: 2.11.0
MMDetection3D: 0.12.0+78ff0ef

Error traceback

[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 81/81, 6.8 task/s, elapsed: 12s, ETA: 0s
Create GT Database of NuScenesDataset
[ ] 0/323, elapsed: 0s, ETA:Traceback (most recent call last):
File "tools/create_data.py", line 256, in
max_sweeps=args.max_sweeps)
File "tools/create_data.py", line 81, in nuscenes_data_prep
f'{out_dir}/{info_prefix}infos_train.pkl')
File "/home/xymbiotec/mmdetection3d/tools/data_converter/create_gt_database.py", line 254, in create_groundtruth_database
point_indices = box_np_ops.points_in_rbbox(points, gt_boxes_3d)
File "/home/xymbiotec/mmdetection3d/mmdet3d/core/bbox/box_np_ops.py", line 434, in points_in_rbbox
indices = points_in_convex_polygon_3d_jit(points[:, :3], surfaces)
File "/home/xymbiotec/mmdetection3d/mmdet3d/core/bbox/box_np_ops.py", line 764, in points_in_convex_polygon_3d_jit
normal_vec, d, num_surfaces)
TypeError: expected dtype object, got 'numpy.dtype[bool]'

Additional information
We implemented the same code on the PC and Colab. Both Colab and PC have the totally same installation versions and environment. However, this code could work successfully on Colab. And when we implement it on the PC, we would have this error.

Thanks in advance.

[Wuziyi616]

@Tai-Wang can you kindly check this issue related to NuScene?

[Tai-Wang]

@ZhangYu1ing It's so strange that this could work on Colab while not on your PC. Are you sure that the environments are exactly the same? Maybe you can first check the version of numba and numpy like said in the FAQ.

[ZhangYu1ing]

Hi, we have numba with version 0.48.0. If we use numpy==1.20.0, we will have the bug shown as above. And if we downgrade numpy < 1.20.0, there will be another bug shown at the beginning running the create_data.py:
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject.

Do you think this problem may relate to the version of Python? Or are there any other problems that may lead to these bugs?

Thanks.

[Tai-Wang]

After changing the version of numpy, maybe you need to recompile all related packages (like mmcv, mmdet and mmdet3d) to ensure the compatibility.