Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Point cloud compression #270

Closed
wants to merge 27 commits into from

Conversation

YodaEmbedding
Copy link
Contributor

@YodaEmbedding YodaEmbedding commented Feb 2, 2024

This PR adds learned point cloud compression (PCC) to CompressAI. Namely:

  • Models: sfu-pcc-rec-pointnet, sfu-pcc-rec-pointnet2-ssg, hrtzxf2022-pcc-rec (He2022)
  • Datasets: ModelNet, ShapeNetPart, SemanticKITTI, S3DIS
  • Transforms: RandomSample, RandomRotateFull [3D], torch-geometric transforms, etc
  • Scripts: examples/train_pointcloud.py

Example usage:

pip install -e '.[pointcloud]'

# NOTE: Automatically downloads and preprocesses the dataset if it is missing.
python examples/train_pointcloud.py --cuda --dataset="datasets/modelnet40" --model="sfu2023-pcc-rec-pointnet"

Other important changes:

  • Drop Python 3.6 support since torch-geometric>=2.3.0 requires Python 3.7+. Furthermore, Python 3.6 (released 2016) went EOL in 2021, and Ubuntu 18.04 went EOL in 2023.

Other notes:

  • pyntcloud breaks with ModelNet40. I created a PR to fix this: perf: read_ply replace pandas.read_csv engine=python with c; improve read_off header-parsing robustness daavoo/pyntcloud#352. While we wait for that to be accepted/released, CompressAI installs the PR version directly.
  • pointops's setup.py contains torch.utils.cpp_extension.CUDAExtension. The installer pip (or whatever) creates an isolated build environment, and installs packages specified in pyproject.toml's build-system.requires. (e.g., requires = ["setuptools", "wheel", "torch"].) However, this version of torch may not match the configured torch version outside the isolated environment. The outer torch version should already be correctly chosen for the CUDA version. But since the isolated torch version may not be CUDA-compatible, this can lead to the error "The detected CUDA version (*) mismatches the version that was used to compile PyTorch (*). Please make sure to use the same CUDA versions.". Ideally, the torch version available within the isolated build environment should be the same as the outside environment. pip has a --no-build-isolation flag, though that's not always available (e.g., poetry), and seems like a bit of a hack. I'll need to see if there's a workaround... [Minor rant.]

Some code is adapted with attribution from the following repositories:


Possible future work:

@fracape fracape self-requested a review February 2, 2024 22:52
YodaEmbedding and others added 21 commits February 2, 2024 14:55
[Currently disabled by default.]

This method completes in <1 second and reduces aux_loss to <0.01.

This makes the aux_loss optimization during training unnecessary.

Another alternative would be to run the following post-training:
```python
while aux_loss > 0.1:
    aux_loss = model.aux_loss()
    aux_loss.backward()
    aux_optimizer.step()
    aux_optimizer.zero_grad()
```
...but since we do not manage aux_loss learning rates,
the bisection search method might converge better.
Improves robustness of header parsing.

In particular, ModelNet40 has faulty headers:
```bash
$ head -n 1 ModelNet40/chair/train/chair_0856.off
OFF6586 5534 0
```

For reference, the correct format is:
```
OFF
6586 5534 0
```
```bash
python examples/train_pointcloud.py --cuda --dataset="datasets/modelnet40"
```
Drop Python 3.6 support since `torch-geometric>=2.3.0` requires Python 3.7+.

Python 3.6 (released 2016) went EOL in 2021, and
Ubuntu 18.04 LTS went EOL in 2023.
@fracape
Copy link
Collaborator

fracape commented Feb 28, 2024

merged via direct push after revert of torch.extension and rebase

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants