- This is just the "DOCUMENTATION" of the said project, to showcase the quality of work conducted.
- All the rights of code, models, weights, custom datasets, results and work conducted during the said project belong to e-Yantra, ERTS Lab, CSE Dept, IIT Bombay, Mumbai, India.
- 7 - Computer vision, Remote Sensing and GIS Technology Stack for an Autonomous Ground Vehicle
- Disclaimer
- Table of Contents
- Objectives and Deliverables
- Abstract
- File Structure
- Domains
- TechStack
- Video
- Presentation
- Results
- Finding High-Quality Datasets of Optical Aerial and Satellite Images
- Model Weights
- Classification and Segmentation of Optical Satellite and Aerial Images using various Computer Vision and Deep-Learning based algorithms
- SAM
- SAM: House Segmentation
- Multi-Class Image Segmentation of Aerial Images using U-Net
- Road Segmentation of Aerial Images using U-Net
- Road Segmentation of Arena Image using U-Net
- Road Segmentation of Low Quality Arena Image using U-Net
- Potsdam: Multi-Class Image Segmentation of Aerial Images using U-Net
- 1024-Batch-Size: Potsdam: Multi-Class Image Segmentation of Aerial Images using U-Net
- Computer Vision: Modules
- Path-Planning
- QGIS
- Challenges Faced
- Future Work
- Contributors
- Acknowledgements and References
Exploration and Path Planning for a Autonomous Ground Robot using Optical Aerial/Satellite Imagery.
- Develop a low-cost, compact ground robot using minimal hardware.
- Finding High-Quality Optical Aerial/Satellite Imagery datasets.
- Explore various Computer Vision and Deep Learning algorithms for Semantic Segmentation of Optical Aerial/Satellite Imagery
- Exploring various algorithms for path-planning and routing the ground vehicle autonomously.
- Perform real-time tracking and routing using GIS techniques.
- Implement entire pipeline developed as a prototype on the ground robot using an arena, wherein the ground robot will be tracked, controlled and routed in real time.
- Finally, evaluate the performance of the pipeline implemented.
- Prototyped a path-planning pipeline for an autonomous ground vehicle utilizing computer vision methods and ArUco Markers.
- Remote sensing and GPS tracking using on-ground markers.
👨💻7-Computer-vision-Remote-Sensing-and-GIS-application-for-autonmous-ground-vehicle
┣ 📂assets // Contains all the reference images
┣ 📂Datasets // Contains links of all Datasets referenced
┣ 📂Model_Weights // Contains model weights of DL models trained using Transfer-Learning Technique
┃ ┣ 📂Potsdam_1024_Batch-Size_Multi_Class_Image_Segmentation_of_Aerial_Images_using_U_Net
┃ ┣ 📂Potsdam_Multi_Class_Image_Segmentation_of_Aerial_Images_using_U_Net
┃ ┣ 📂Road_Segmentation_of_Low_Quality_Arena_Image_using_U_Net
┃ ┣ 📂U-Net
┃ ┗ 📄README.md
┣ 📂Models // DL Models and Computer Vision Modules
┃ ┣ 📂Modules // Computer Vision Modules utilized in DL Models
┃ ┃ ┣ 📂data_augmentation
┃ ┃ ┣ 📂resize_using_interpolation
┃ ┃ ┣ 📂superresolution
┃ ┃ ┗ 📄README.md
┃ ┣ 📂Potsdam_1024_Batch-Size_Multi_Class_Image_Segmentation_of_Aerial_Images_using_U_Net
┃ ┣ 📂Potsdam_Multi_Class_Image_Segmentation_of_Aerial_Images_using_U_Net
┃ ┣ 📂Road_Segmentation_of_Arena_Image_using_U-Net
┃ ┣ 📂Road_Segmentation_of_Low_Quality_Arena_Image_using_U_Net
┃ ┣ 📂Road_Segmentation_using_U-Net
┃ ┣ 📂SAM
┃ ┣ 📂SAM_House_Segmentation
┃ ┣ 📂U-Net
┃ ┗ 📄README.md
┣ 📂Navigation // Contains code for Navigation of Autonomous Ground Vehicle
┃ ┗ 📄README.md
┣ 📂Path-Planning // Contains Path-Planning Code
┃ ┣ 📂Occupancy_Grid_for_Path_Planning_using_Erosion
┃ ┣ 📂Path-Planning_using_ArUcoMarkers
┃ ┗ 📄README.md
┣ 📂QGIS // Contains QGIS Code
┃ ┣ 📂Path Planning // Contains Path Planning Code using Global Coordinate System
┃ ┃ ┣ 📂A* Algorithm
┃ ┃ ┣ 📂ORS (openrouteservice)
┃ ┃ ┗ 📄README.md
┃ ┣ 📂markers_using_script
┃ ┗ 📄README.md
┣ 📄LICENSE
┗ 📄README.md
Image Processing, Computer Vision, Machine Learning, Deep Learning, Python, Path-Planning, Routing, Remote Sensing, QGIS
- For explanation Video Click Here
- For Brief Presentation Click Here
- For Complete Presentation Click Here
- For further reading Click Here
| Original Image | Ground Truth |
|---|---|
 |
 |
| Arena Image |
|---|
 |
Model Weights of Deep Learning Models for Semantic Segmenation of Aerial/Satellite Imagery (using Transfer Learning Technique)
- For further reading Click Here
Classification and Segmentation of Optical Satellite and Aerial Images using various Computer Vision and Deep-Learning based algorithms
- For further reading Click Here
| Original Input Image | Road Segmentation | Road Segmentation Mask |
|---|---|---|
 |
 |
 |
- For further reading Click Here
| Original Input Image | House Segmentation | House Segmentation Mask |
|---|---|---|
 |
 |
 |
- For further reading Click Here
| Original Input Image | Ground Truth Mask | Predicted Mask without Smooth Blending | Predicted Mask with Smooth Blending |
|---|---|---|---|
 |
 |
 |
 |
- For further reading Click Here
| Original Input Image | Predicted Mask without Smooth Blending | Predicted Mask with Smooth Blending |
|---|---|---|
 |
 |
 |
- For further reading Click Here
| Original Input Image | Predicted Mask without Smooth Blending | Predicted Mask with Smooth Blending |
|---|---|---|
 |
 |
 |
- For further reading Click Here
| Original Input Image | Ground Truth Mask | Predicted Mask without Smooth Blending | Predicted Mask with Smooth Blending |
|---|---|---|---|
 |
 |
 |
 |
- For further reading Click Here
| Original Input Image | Ground Truth Mask | Predicted Mask without Smooth Blending | Predicted Mask with Smooth Blending |
|---|---|---|---|
 |
 |
 |
 |
- For further reading Click Here
| Original Input Image | Ground Truth Mask | Predicted Mask without Smooth Blending | Predicted Mask with Smooth Blending |
|---|---|---|---|
 |
 |
 |
 |
- For further reading Click Here
- For further reading Click Here
- For further reading Click Here
- For further reading Click Here
| Original Input Mask | Occupancy Grid |
|---|---|
 |
 |
- For further reading Click Here
- For further reading Click Here
- Aerial/Satellite Images were quite large in size (some even 200-350 MB per image), making it very difficult to load and utilize the datasets for transfer learning/inputs for predictions.
- Limitations of RAM, GPU Memory in local systems - leading to longer training/prediction time, or crashes
- Limitations of RAM in Google Colab - owing to large dataset sizes - leading to crashes
- Memory deallocation of not-required variables.
- Utilizing smaller/compressed datasets
- Distorted geo-referenced images, hence unable map pixel & global coordinated accurately
- Path planning using A*(star) on cost map, was very time consuming and computationally expensive
- High latency in communication between controller(laptop) and robot, causing issues in live-control of on-ground vehicle
- Path-Planning and Live-Control of Autonomous Ground Vehicle using Edges of Road - found using Segmented Mask
- Utilize Geo-referenced Image & Masks for global tracking/control of autonomous ground vehicle.
- Path-Planning and Live-Control of Autonomous Ground Vehicle using optimised Deep-Learning Techniques like “Neural A*(Star)”
| A*(star) | Neural A*(star) |
|---|---|
 |
 |
- e-Yantra Summer Internship Program - 2023(eYSIP)
- Special thanks to our mentors Saail Narvekar Sir, Aditya Panwar Sir, and all the mentors at e-Yantra for their constant support and guidance throughout the project