Replace pycuda with cuda-python calls (#105)

This change-set deprecates the pycuda usage, which complicated the Wisp installation due to GL extension compilation. Requires `pip install cuda-python` Changes: * pycuda replace with direct cuda api (`cuda-python` bindings) * `cuda_guard.py` have been removed (no longer needed as context bugs are bypassed) * GLX error messages when closing windows are now gone. * Support for torch 1.9.1 is fixed. * `pycuda` can be used as a fallback by setting the env variable `ENABLE_PYCUDA=1` * Fix support for WSL through bypassing cuda-opengl interop. New Wisp installation: ``` pip install -r requirements.txt pip install -r requirements_app.txt python setup.py develop ``` Signed-off-by: operel <[email protected]>
NVIDIAGameWorks · Jan 17, 2023 · 351a04a · 351a04a
1 parent 16a2ae5
commit 351a04a
Show file tree

Hide file tree

Showing 12 changed files with 251 additions and 160 deletions.
diff --git a/INSTALL.md b/INSTALL.md
@@ -4,64 +4,82 @@ NVIDIA Kaolin Wisp can be installed either manually or using Docker.
 
 ## Manual Installation
 
-### 1. Create an anaconda environment
+### Prerequisite
 
-The easiest way to get started is to create a virtual Python 3.8 Anaconda environment:
-```
-conda create -n wisp python=3.8
-conda activate wisp
-pip install --upgrade pip
-```
-
-### 2. Install OpenEXR
-
-On Ubuntu:
+Install OpenEXR on Ubuntu:
 
 ```
 sudo apt-get update
 sudo apt-get install libopenexr-dev 
 ```
 
-On Windows:
+Install OpenEXR on Windows:
 
 ```
 pip install pipwin
 pipwin install openexr
 ```
 
-### 3. Install PyTorch
+### Quick Start
+Full installation with interactive visualizer, for torch 1.12.1, cuda 11.3 and kaolin 0.12.0:
+```
+conda create -n wisp python=3.9
+git clone [email protected]:NVIDIAGameWorks/kaolin-wisp.git
+cd kaolin-wisp
+pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113
+pip install kaolin==0.12.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-1.12.1_cu113.html
+pip install -r requirements.txt
+pip install -r requirements_app.txt
+python setup.py develop
+```
+Test the NeRF app with lego data (download separately):
+``` 
+python app/nerf/main_nerf.py --dataset-path data/lego --config app/nerf/configs/nerf_hash.yaml
+```
 
-You should first install PyTorch by following the [official instructions](https://pytorch.org/). The code has been tested with `1.9.1` to `1.12.0` on Ubuntu 20.04. 
+### Installation Steps in Detail
 
-### 4. Install Kaolin
+#### 1. Create an anaconda environment
 
-You should also install Kaolin, following the [instructions here](https://kaolin.readthedocs.io/en/latest/notes/installation.html). **WARNING:** The minimum required version of Kaolin is `0.12.0`. If you have any issues specifically with Camera classes not existing, make sure you have an up-to-date version of Kaolin. 
+The easiest way to get started is to create a virtual Python 3.8 Anaconda environment:
+```
+conda create -n wisp python=3.8
+conda activate wisp
+pip install --upgrade pip
+```
 
-### 5. Install the rest of the dependencies
+#### 2. Install PyTorch
+
+You should first install PyTorch by following the [official instructions](https://pytorch.org/). The code has been tested with `1.9.1` to `1.12.0` on Ubuntu 20.04. 
 
-Install the rest of the dependencies from [requirements](requirements.txt):
+#### 3. Install Kaolin
+
+kaolin can be installed with pip (use the correct torch + cuda version):
 ```
-pip install -r requirements.txt
+pip install kaolin==0.12.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-${TORCH_VER}_cu${CUDA_VER}.html
 ```
 
-### 6. Installing the interactive renderer (optional)
+For example, for torch 1.12.1 + cuda 11.3:
+```
+pip install kaolin==0.12.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-1.12.1_cu113.html
+```
 
-If you wish to use the interactive renderer and training visualizer, you will need additional dependencies. 
-Note that you need to have OpenGL available on your system.
+See the [Kaolin Installation Doc](https://kaolin.readthedocs.io/en/latest/notes/installation.html) for additional methods.
+
+_The minimum required version of Kaolin is `0.12.0`._  
+
+#### 4. Installing Wisp
+
+Install the rest of the dependencies from [requirements](requirements.txt).
 
-To install (**make sure you have the CUDA_HOME environment variable set!**):
+If you wish to use the interactive renderer and training visualizer, you will need [additional dependencies](requirements_app.txt).
+Note that you need to have OpenGL available on your system.
 
 ```
-git clone --recursive https://github.com/inducer/pycuda
-cd pycuda
-python configure.py --cuda-root=$CUDA_HOME --cuda-enable-gl
-python setup.py develop
-cd ..
+pip install -r requirements.txt
 pip install -r requirements_app.txt
 ```
 
-### 7. Installing Wisp
-
 To install wisp, simply execute:
 ```
 python setup.py develop

diff --git a/README.md b/README.md
@@ -16,6 +16,7 @@ For an overview on neural fields, we recommend you check out the EG STAR report:
 
 ## Latest Updates
 
+* _17/01/23_ `pycuda` replaced with `cuda-python`. Wisp can be installed from pip now  (If you pull, run **pip install -r requirements_app.txt**)
 * _05/01/23_ Mains are now introduced as standalone apps, for easier support of new pipelines (**breaking change**) 
 * _21/12/22_ Most modules have been cleaned, reorganized and documented. 
 

diff --git a/app/nerf/main_nerf.py b/app/nerf/main_nerf.py
@@ -7,9 +7,6 @@
 # license agreement from NVIDIA CORPORATION & AFFILIATES is strictly prohibited.
 
 
-from wisp.cuda_guard import setup_cuda_context
-setup_cuda_context()  # Must be called before any torch operations take place
-
 import os
 import argparse
 import logging

diff --git a/app/nglod/main_nglod.py b/app/nglod/main_nglod.py
@@ -7,9 +7,6 @@
 # license agreement from NVIDIA CORPORATION & AFFILIATES is strictly prohibited.
 
 
-from wisp.cuda_guard import setup_cuda_context
-setup_cuda_context()  # Must be called before any torch operations take place
-
 import os
 import argparse
 import logging

diff --git a/examples/latent_nerf/main_demo.py b/examples/latent_nerf/main_demo.py
@@ -6,8 +6,6 @@
 # distribution of this software and related documentation without an express
 # license agreement from NVIDIA CORPORATION & AFFILIATES is strictly prohibited.
 
-from wisp.cuda_guard import setup_cuda_context
-setup_cuda_context()  # Must be called before any torch operations take place
 
 import os
 import argparse

diff --git a/examples/spc_browser/main_spc_browser.py b/examples/spc_browser/main_spc_browser.py
@@ -14,8 +14,6 @@
 Usage: python examples/spc_browser/main_spc_browser.py --dataset-dir <PATH_TO_SPC_FOLDER>
 """
 
-from wisp.cuda_guard import setup_cuda_context
-setup_cuda_context()  # Must be called before any torch operations take place
 
 import argparse
 import torch

diff --git a/requirements_app.txt b/requirements_app.txt
@@ -2,3 +2,4 @@ glfw
 imgui
 git+https://github.com/glumpy/glumpy.git@46a7635c08d3a200478397edbe0371a6c59cd9d7#egg=glumpy
 pyopengl
+cuda-python
diff --git a/wisp/core/transforms.py b/wisp/core/transforms.py
@@ -8,9 +8,14 @@
 
 from __future__ import annotations
 from typing import Union
+
+import numpy as np
 import torch
 
 
+PI = torch.pi if hasattr(torch, 'pi') else np.pi  # Fallback for older torch versions
+
+
 class ObjectTransform:
     """ObjectTransform represents the local transformations that convert the object from local space coordinates
     to world coordinates.
@@ -63,7 +68,7 @@ def scale(self, scale: Union[float, torch.Tensor]):
     def permute(self, permutation):
         """
         Args:
-            permutation (list[int]): The permutation of the axis. 
+            permutation (list[int]): The permutation of the axis.
                                      For example, [1, 0, 2] will swap the x and y axis.
         """
         permutation = torch.tensor(permutation, device=self.device, dtype=torch.long)
@@ -196,7 +201,7 @@ def model_matrix(self):
         # Transformations applied in order of
         # Translation @ Rotation @ Scale
         scale_mat = self._scale_mat(self._scale)
-        rotation_rads = self._rotation.div(180.0).mul(torch.pi)
+        rotation_rads = self._rotation.div(180.0).mul(PI)
         rotation_mat = self._rotation_mat(*rotation_rads)
         translation_mat = self._translation_mat(self._translation)
         model_mat = translation_mat @ rotation_mat @ scale_mat @ self._permutation
@@ -213,7 +218,7 @@ def inv_model_matrix(self):
         # Transformations applied in order of
         # Scale^(-1) @ Rotation^(-1) @ Translation^(-1)
         scale_mat = self._inv_scale_mat(self._scale)
-        rotation_rads = self._rotation.div(180.0).mul(torch.pi)
+        rotation_rads = self._rotation.div(180.0).mul(PI)
         rotation_mat = self._inv_rotation_mat(*rotation_rads)
         translation_mat = self._inv_translation_mat(self._translation)
         inv_mat = self._permutation @ scale_mat @ rotation_mat @ translation_mat

diff --git a/wisp/cuda_guard.py b/wisp/cuda_guard.py