DV-SLAM

ROB 530 Final Project Group 9: DV SLAM

This repository holds all the code for our NAVARCH/EECS 568, ROB530: Mobile Robotics final project. The title of our project is DV-SLAM. Team members are: Joshua Friesen, Yu-Min Jian, Yi-Cheng Lai, Shrey Sahgal, Shanzhao Wang (listed in alphabetical order).

Main Idea and Objective

The objective of our project is to enhance the performance of existing visual odometry (VO) SLAM techniques by replacing the front-end with a deep learning method. In this repository we have created a type of SLAM called DV-SLAM that takes the loop-closure aspects of graph SLAM and implements the afformentioned deep-learning front end. Additionally we have included the code for our future-work, ORB-SLAM2 [2] with a modified front end visual odometry. For our models we use the monocular camera information from the KITTI dataset as the input data and replaced the VO algorithm with a pre-trained end-to-end DL-based model called DeepVO [4].

Running the code

In order to configure this project, please follow these steps:

1. Clone the repository onto your local system.

$ git clone https://github.com/shreysahgal/DV-SLAM.git

DeepVO

We implement DeepVO by modifying this repository DeepVO-pytorch, using a pre-trained model and optimizer provided by alexart13.

Prerequisites

• pytorch 0.4.0
• torchvision 0.2.1
• numpy
• pandas
• pillow
• matplotlib
• glob

1. Put the pre-trained model and optimizer into the models folder.
2. Change self.data_dir in params.py to your own data path.
3. Run test.py to get the visual odometry output from the model.

ORB-SLAM2

The ORB-SLAM2 repository is forked from ORB-SLAM2. We modified the Monocular example with KITTI Dataset of ORB-SLAM2 using DeepVO's as visual odometry.

Preresuisites

• C++11
• Pangolin
• OpenCV 2.0.4
• Eigen3
• DBoW2 and g2o

$ ./Examples/Monocular/mono_kitti Vocabulary/ORBvoc.txt Examples/Monocular/KITTIX.yaml PATH_TO_DATASET_FOLDER/dataset/sequences/SEQUENCE_NUMBER

DV SLAM

1. Fill in 'fill in' part in main_DV.py
2. Run main_DV.py to get the result from DV SLAM using gtsam solver

$ main_DV.py

Requirements

• Python 3.8.10
• Numpy 1.22.3
• OpenCV 4.5.5
• Gtsam 4.1.1
• Matplotlib 3.4.3
• Tqdm 4.64.0
• Scipy 1.8.0

Future Results of DeepVO+ORB_SLAM2

Testing on KITTI Dataset 07 with scaled odometry (Lost halfway)

Testing on KITTI Dataset 04 with scaled odometry (success)

Testing on KITTI Dataset 04 with unscaled odometry (Lost at the beginning)

DV-SLAM Results

The results show a substantial improvement with loop closures when compared to pure Deep VO output. These results are from running our program on KITTI rgb odometry dataset 07. To replicate them follow the "Running the code" section below.

Obtained Results	ORB-SLAM Comparison
RMSE: 7.271161 m	RMSE: 3.194608 m
MSE: 6.651275 m	MSE: 2.960466 m
Median Error: 5.993425 m	Median Error: 2.950503 m
Min Error: 2.430470 m	Min Error: 0.476634 m
Max Error: 15.652031 m	Max Error: 6.092447 m
Standard Deviation: 5.993425 m	Standard Deviation: 1.200484 m

The results above show our DV-algorithm with a side by side comparison with ORB-SLAM modified by the Horn [1] trajectory alignment algorithm. In comparison with a widely used monocular SLAM solution, ORB-SLAM2, our DV-SLAM does not perform as well across all metrics listed. However, we believe that the modification of other algorithm's front-ends, such as ORB-SLAM2, with the deepVO model has the potential to generate results surpassing either of the above methods. We leave this to future works.

To replicate the above results simply clone the repository, and run the deepVO_stats.py file and orb-slam alignment.py files. DeepVO_stats.py generates the error statistics and graph for DV-SLAM, and orb-slam alignment.py runs the results of ORB-SLAM through the Horn [1] algorithm before before generating the error statistics and graph. All necessary files are provided. If you wish to generate your own, KeyFrameTrajectory.txt can be generated by running ORB-SLAM2 [2] on the KITTI [3] odometry dataset. The gtsam7.txt file (results of DV-SLAM) can be generated as by running the DeepVO section.

References

[1] Horn, Berthold K.P. (1987). Closed-Form Solution of Absolute Orientation Using Unit Quaternions. Journal of the Optical Society of America A, vol. 4, no. 4, 629.

[2] Mur-Artal, Raul (2016). ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras. IEEE Transactions on Robotics.

[3] Geiger et al. (2013). Vision meets robotics: The KITTI dataset. International Journal of Robotics Research, vol. 32, 1231-1237.

[4] Sen Wang, Ronald Clark, Hongkai Wen, Niki Trigoni (2017). DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks. 2017 IEEE International Conference on Robotics and Automation (ICRA 2017).