README.md

7 - Computer vision, Remote Sensing and GIS Technology Stack for an Autonomous Ground Vehicle

Disclaimer

• This is just the "DOCUMENTATION" of the said project, to showcase the quality of work conducted.
• All the rights of code, models, weights, custom datasets, results and work conducted during the said project belong to e-Yantra, ERTS Lab, CSE Dept, IIT Bombay, Mumbai, India.

Table of Contents

• 7 - Computer vision, Remote Sensing and GIS Technology Stack for an Autonomous Ground Vehicle
• Disclaimer
• Table of Contents
• Objectives and Deliverables
• Abstract
• File Structure
• Domains
• TechStack
• Video
• Presentation
• Results
  • Finding High-Quality Datasets of Optical Aerial and Satellite Images
  • Model Weights
  • Classification and Segmentation of Optical Satellite and Aerial Images using various Computer Vision and Deep-Learning based algorithms
    • SAM
    • SAM: House Segmentation
    • Multi-Class Image Segmentation of Aerial Images using U-Net
    • Road Segmentation of Aerial Images using U-Net
    • Road Segmentation of Arena Image using U-Net
    • Road Segmentation of Low Quality Arena Image using U-Net
    • Potsdam: Multi-Class Image Segmentation of Aerial Images using U-Net
    • 1024-Batch-Size: Potsdam: Multi-Class Image Segmentation of Aerial Images using U-Net
  • Computer Vision: Modules
    • Data Augmentation
    • Resize: using Interpolation
    • SuperResolution
  • Path-Planning
    • Occupancy Grid for Path-Planning using Erosion
    • Path-Planning using ArUco Markers
  • QGIS
    • QGIS: Markers using Script
• Challenges Faced
• Future Work
• Contributors
• Acknowledgements and References

Objectives and Deliverables:

Exploration and Path Planning for a Autonomous Ground Robot using Optical Aerial/Satellite Imagery.

• Develop a low-cost, compact ground robot using minimal hardware.
• Finding High-Quality Optical Aerial/Satellite Imagery datasets.
• Explore various Computer Vision and Deep Learning algorithms for Semantic Segmentation of Optical Aerial/Satellite Imagery
• Exploring various algorithms for path-planning and routing the ground vehicle autonomously.
• Perform real-time tracking and routing using GIS techniques.
• Implement entire pipeline developed as a prototype on the ground robot using an arena, wherein the ground robot will be tracked, controlled and routed in real time.
• Finally, evaluate the performance of the pipeline implemented.

Abstract:

• Prototyped a path-planning pipeline for an autonomous ground vehicle utilizing computer vision methods and ArUco Markers.
• Remote sensing and GPS tracking using on-ground markers.

File Structure

👨‍💻7-Computer-vision-Remote-Sensing-and-GIS-application-for-autonmous-ground-vehicle 
 ┣ 📂assets                             // Contains all the reference images
 ┣ 📂Datasets                           // Contains links of all Datasets referenced
 ┣ 📂Model_Weights                      // Contains model weights of DL models trained using Transfer-Learning Technique
 ┃ ┣ 📂Potsdam_1024_Batch-Size_Multi_Class_Image_Segmentation_of_Aerial_Images_using_U_Net                                           
 ┃ ┣ 📂Potsdam_Multi_Class_Image_Segmentation_of_Aerial_Images_using_U_Net                                           
 ┃ ┣ 📂Road_Segmentation_of_Low_Quality_Arena_Image_using_U_Net                                                                                    
 ┃ ┣ 📂U-Net                                          
 ┃ ┗ 📄README.md
 ┣ 📂Models                             // DL Models and Computer Vision Modules 
 ┃ ┣ 📂Modules                          // Computer Vision Modules utilized in DL Models                 
 ┃ ┃ ┣ 📂data_augmentation    
 ┃ ┃ ┣ 📂resize_using_interpolation
 ┃ ┃ ┣ 📂superresolution 
 ┃ ┃ ┗ 📄README.md                  
 ┃ ┣ 📂Potsdam_1024_Batch-Size_Multi_Class_Image_Segmentation_of_Aerial_Images_using_U_Net                                           
 ┃ ┣ 📂Potsdam_Multi_Class_Image_Segmentation_of_Aerial_Images_using_U_Net                                           
 ┃ ┣ 📂Road_Segmentation_of_Arena_Image_using_U-Net                                           
 ┃ ┣ 📂Road_Segmentation_of_Low_Quality_Arena_Image_using_U_Net                                           
 ┃ ┣ 📂Road_Segmentation_using_U-Net                                           
 ┃ ┣ 📂SAM                                           
 ┃ ┣ 📂SAM_House_Segmentation                                           
 ┃ ┣ 📂U-Net                                          
 ┃ ┗ 📄README.md   
 ┣ 📂Navigation                         // Contains code for Navigation of Autonomous Ground Vehicle                                          
 ┃ ┗ 📄README.md
 ┣ 📂Path-Planning                      // Contains Path-Planning Code
 ┃ ┣ 📂Occupancy_Grid_for_Path_Planning_using_Erosion                                           
 ┃ ┣ 📂Path-Planning_using_ArUcoMarkers                                          
 ┃ ┗ 📄README.md  
 ┣ 📂QGIS                               // Contains QGIS Code
 ┃ ┣ 📂Path Planning                    // Contains Path Planning Code using Global Coordinate System                 
 ┃ ┃ ┣ 📂A* Algorithm    
 ┃ ┃ ┣ 📂ORS (openrouteservice)
 ┃ ┃ ┗ 📄README.md  
 ┃ ┣ 📂markers_using_script
 ┃ ┗ 📄README.md                                   
 ┣ 📄LICENSE      
 ┗ 📄README.md                                               

Domains

Image Processing, Computer Vision, Machine Learning, Deep Learning, Python, Path-Planning, Routing, Remote Sensing, QGIS

TechStack

Video

• For explanation Video Click Here

Presentation

• For Brief Presentation Click Here
• For Complete Presentation Click Here

Results

Finding High-Quality Datasets of Optical Aerial and Satellite Images

• For further reading Click Here

Original Image	Ground Truth

Arena Image

Model Weights

Model Weights of Deep Learning Models for Semantic Segmenation of Aerial/Satellite Imagery (using Transfer Learning Technique)

• For further reading Click Here

Classification and Segmentation of Optical Satellite and Aerial Images using various Computer Vision and Deep-Learning based algorithms

SAM

• For further reading Click Here

Original Input Image	Road Segmentation	Road Segmentation Mask

SAM: House Segmentation

• For further reading Click Here

Original Input Image	House Segmentation	House Segmentation Mask

Multi-Class Image Segmentation of Aerial Images using U-Net

• For further reading Click Here

Original Input Image	Ground Truth Mask	Predicted Mask without Smooth Blending	Predicted Mask with Smooth Blending

Road Segmentation of Aerial Images using U-Net

• For further reading Click Here

Original Input Image	Predicted Mask without Smooth Blending	Predicted Mask with Smooth Blending

Road Segmentation of Arena Image using U-Net

• For further reading Click Here

Original Input Image	Predicted Mask without Smooth Blending	Predicted Mask with Smooth Blending

Road Segmentation of Low Quality Arena Image using U-Net

• For further reading Click Here

Original Input Image	Ground Truth Mask	Predicted Mask without Smooth Blending	Predicted Mask with Smooth Blending

Potsdam: Multi-Class Image Segmentation of Aerial Images using U-Net

• For further reading Click Here

Original Input Image	Ground Truth Mask	Predicted Mask without Smooth Blending	Predicted Mask with Smooth Blending

1024-Batch-Size: Potsdam: Multi-Class Image Segmentation of Aerial Images using U-Net

• For further reading Click Here

Original Input Image	Ground Truth Mask	Predicted Mask without Smooth Blending	Predicted Mask with Smooth Blending

Computer Vision: Modules

Data Augmentation

• For further reading Click Here

Resize: using Interpolation

• For further reading Click Here

SuperResolution

• For further reading Click Here

Path-Planning

Occupancy Grid for Path-Planning using Erosion

• For further reading Click Here

Original Input Mask	Occupancy Grid

Path-Planning using ArUco Markers

• For further reading Click Here

QGIS

QGIS: Markers using Script

• For further reading Click Here

Challenges Faced

• Aerial/Satellite Images were quite large in size (some even 200-350 MB per image), making it very difficult to load and utilize the datasets for transfer learning/inputs for predictions.
• Limitations of RAM, GPU Memory in local systems - leading to longer training/prediction time, or crashes
• Limitations of RAM in Google Colab - owing to large dataset sizes - leading to crashes
  • Memory deallocation of not-required variables.
  • Utilizing smaller/compressed datasets
• Distorted geo-referenced images, hence unable map pixel & global coordinated accurately
• Path planning using A*(star) on cost map, was very time consuming and computationally expensive
• High latency in communication between controller(laptop) and robot, causing issues in live-control of on-ground vehicle

Future Work

• Path-Planning and Live-Control of Autonomous Ground Vehicle using Edges of Road - found using Segmented Mask
• Utilize Geo-referenced Image & Masks for global tracking/control of autonomous ground vehicle.
• Path-Planning and Live-Control of Autonomous Ground Vehicle using optimised Deep-Learning Techniques like “Neural A*(Star)”

A*(star)	Neural A*(star)

Contributors

• Aryan Sanjay Shah
• Agnael Karan
• Ansh Gupta

Acknowledgements and References

• e-Yantra Summer Internship Program - 2023(eYSIP)
• Special thanks to our mentors Saail Narvekar Sir, Aditya Panwar Sir, and all the mentors at e-Yantra for their constant support and guidance throughout the project