This project converts a 2D image into a 3D representation using depth generation techniques, 3D mesh reconstruction, and AI-based enhancements for missing parts of the image.
Ensure you have Python 3.6 or later installed. You can install the necessary dependencies using the requirements.txt file.
pip install -r requirements.txt- Clone the repository and navigate to the project directory:
git clone <REPOSITORY_URL>
cd 2d_to_3d- Install the dependencies:
pip install -r requirements.txt- Run the
main.pyfile to start the application:
python main.py- Access the web interface in your browser:
http://127.0.0.1:5000
- Select an image from your computer using the "Choose File" button.
- Click "Upload and Process" to start the transformation.
After the upload and processing, the interface will display the generated 3D image. You can validate this step and move on by clicking "Validate and Proceed."
The AI will generate missing parts of the image, improving the 3D model. This updated image will be shown. You can validate and proceed as before.
The final 3D model will be displayed with all enhancements applied.
- Depth Estimation: The project uses the MiDaS model to estimate depth from a 2D image.
- Depth to Point Cloud: Converts the depth map into a 3D point cloud.
- Point Cloud to Mesh: Converts the point cloud into a 3D mesh model.
- AI-based Enhancements:
- Inpainting for Missing Parts: Completes missing sections using AI-based inpainting.
- Texture Generation: GPT-4 analyzes the image and improves textures based on AI suggestions.
- Annotation & Segmentation: Assists in segmenting objects within the image for better 3D reconstruction.
- Parameter Optimization: GPT-4 optimizes reconstruction parameters based on analysis.
- 3D Model Synthesis: GPT-4 helps fuse data to create more precise 3D models.