LabelImg3D is a labeling tool for image graphics. It is written in Python and uses Qt for its graphical interface. The annotations are saved as json files and used in the SMPOE network.
We human beings can do 3D reconstruction from single image. Instead of depth estimation from image, we humans beings seems to use a different way, which is to call 3D model in mind, re-align these 3D models and change hyper-parameters of 3D models in the virtual world of our brain. Therefore, why not try to find a way to realign 3D models by single image instead of directly estimating depth.
In this work, we try to find a way to re-align 3D models in 3D space guided by the single images. There are two steps needs to be done, as follows:
- Image-guided 3D Re-Alignment: object detection, 3D model matching, and 3D model pose estimation.
- Image-guided 3D Hyper-parameter of variants model estimation.
- Image annotation of the object's true pose relative to the camera.
- Annotation of the target's 2D boxes and 3D boxes in the image.
- GUI customization (predefined labels, auto-saving, etc).
- Exporting linemod dataset for 6D Pose Estimation.
- Ubuntu / macOS / Windows
- Python3
- PyQt5
- VTK
- numpy
- pandas
- scipy
- PyYAML
- opencv-python
There are two ways for installing the labeltools:
conda create -n labelimg3D python=3.8- install from the released .exe
or
- run from the code, by the following sh
conda create -n pylabelimg3D python=3.8
pip install -r requirement.txt
conda activate pylabelimg3D
python main.pyIf you want to make the installer, you can use the pyinstaller. Try the following command:
pip install pyinstaller
pyinstaller --clean -y LabelImg3D.specPlease download the demo scene and open with LabelImg3D.
Scenes
├── Scene1
│ ├── images
│ │ ├── 000000.png ......
│ ├── models
│ │ ├── Car.obj bus.obj ......
| ├── annotations
| │ ├── 000000.json ......
├── Scene2
├── ├── images
│ │ ├── ......
labelImg3D # open guior
python3 labelImg3D.py-
Build and launch using the instructions above.
-
Click 'Load Scenes' in Menu/File, and choose the scene folder as Data Preparation.
-
Click 'System Config' in Menu/Config.
-
Click "camera" at System_config Page, and set the FOV to the field of view of the camera taking the scene images.
-
Click "model" at System_config page, and you can set the initial position, max position, position accuracy, size accuracy, scaling factor of the model.
- initial position: The initial position of the model loaded into the scene.
- max position: The position of the model in the scene closest to the camera. Moving the model beyond this position resets the model to its initial position during the move.
- position accuracy: Display accuracy of model x,y,z position, located in 3DProperty module.
- size accuracy: Display accuracy of model size(w, l, h), located in 3DProperty module.
- scaling factor: Model movement speed in Z-axis.
-
Click the model name at Models module, and we can see the model loaded to its initial position.
- If a model is selected and loaded as described above, if both model categories are the same, the new model will be loaded to the selected model location.
-
You can press the Shift key and press the left mouse button to select the model and move the mouse position to move the model in X and Y axis.
-
You can press the right mouse button to select the model and move the mouse position to move the model in the Z-axis.
-
In the visual perception of the human eye, object labeling is considered to be over when the model in the scene completely overlaps with the image object.
The annotation will be saved to the Scenes//annotations.
You can refer to the below hotkeys to speed up your workflow.
| Hot key | Description |
|---|---|
| Ctrl + s | Save |
| Ctrl + o | Load scenes |
| Ctrl + c | Copy model |
| Ctrl + v | Paste model |
| Ctrl + Space | Copy Scene |
| 1 | Previous image |
| Space | Next image |
| x | Delete Model |
| Shift + Left Mouse Button Move | Move the model along X, Y axis |
| Right Mouse Button Move | Move the model along Z axis |
| w | Rotate around the positive x-axis |
| s | Rotate around the negative x-axis |
| e | Rotate around the positive y-axis |
| q | Rotate around the negative y-axis |
| a | Rotate around the positive z-axis |
| d | Rotate around the negative z-axis |
The annotation files contain 3 main sections, as follows:
Annotation.json
├── image_file 1: "images/0000.png"
├── model 2
│ ├── num 3: 5
│ ├── 0 4
│ │ ├── model_file 5: "models/Car.obj"
│ │ ├── matrix: [0, …, ] # (16x1) matrix 6
│ │ ├── R_matrix_c2o:[0, …, ] # (9x1) matrix 7
│ │ ├── T_matrix_c2o:[0, …, ] # (3x1) matrix 8
│ │ ├── 2d_bbox: [700, …, ] # (4x1) matrix 9
│ │ ├── 3d_bbox: [[721,500] …, ] # (8x2) matrix 10
│ │ ├── 3d_bbox_w: [[3.8, 2.4, 1.4],…，] #(3x1) matrix 11
│ │ ├── class: 1 # object class num 12
│ │ ├── class_name: Car # object class name 13
│ │ ├── size: [1.99, 1.55, 18.24] # (3x1) matrix 14
│ ├── 14
│ │ ├── model_file 5: "models/Car.obj"
│ │ ├── ......
├── camera 15
│ ├── matrix: [1.0, …, ] # (16x1) matrix 16
│ ├── position: [0.0, 0.0, 0.52] # (3x1) matrix 17
│ ├── focalPoint: [0.0, 0.0, 0.0] # (3x1) matrix 18
│ ├── fov: 88.0 # camera fov 18
│ ├── viewup: [0.0, 1.0, 0.0] # camera viewup 18
│ ├── distance: 0.52 # camera distance 18
This is a dataset named KITTI3D
git clone https://github.com/CongliangLi/LabelImg3D.git
cd labelImg3D| name | desc |
|---|---|
| pyqtconfig | for gui configure |
The methods provided on this page are published under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License . This means that you must attribute the work in the manner specified by the authors, you may not use this work for commercial purposes and if you alter, transform, or build upon this work, you may distribute the resulting work only under the same license. If you are interested in commercial usage you can contact us for further options.
Commercial Cooperation, Contact [email protected]