[ECCV-2022] The official repo for the paper "UNIF: United Neural Implicit Functions for Clothed Human Reconstruction and Animation".
project / arxiv / video / poster
sponsored by imgcreator.zmo.ai
Checkpoints and logs can be downloaded here.
# Create a new virtual environment with conda
conda create --name UNIF python=3.9
# Install PyTorch along with CUDA runtime
conda install cudatoolkit=11.3 pytorch=1.12.0 -c pytorch
You can either install from prebuilt from binaries
# install with conda
conda install pytorch3d -c pytorch3d
or install from source
# runtime dependencies
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
# build time dependency
conda install -c bottler nvidiacub
# building from source
pip install "git+https://github.com/facebookresearch/pytorch3d.git"
You can refer to the official doc in case that problems occur.
pip install -r requirements.txt
# We use `mise` from *Occupancy Networks* to speed up mesh generation with adaptive point inferencing. This package can be automatically configured by running
pip install -e .
# To use the offscreen render of `PyRender`, set the environmental variable:
export PYOPENGL_PLATFORM=egl
Download SMPL-1.0.0 from the homepage, extract it, and put basicModel_f_lbs_10_207_0_v1.0.0.pkl and basicmodel_m_lbs_10_207_0_v1.0.0.pkl under ./data/smpl/models/.
The CAPE dataset can be downloaded from the homepage.
- Follow the Option 2: Download by subject section in the Download page. Download per-subject mesh data for
00032,00096,00159, and03223, as only these four subjects have raw scans released. - Follow the Raw Scans section in the Download page, and download per-subject scan data into
./data/cape_release/raw_scans - The dataset does not provide the shape parameters (beta) for each subject. You can download the beta parameters fitted by us from this OneDrive link. You can also refer to this document to fit SMPL parameters within our framework.
- Download the dataset from the Neural-GIF repo. Arrange the files under
./data/ClothSeq/. - The scale of raws scans of the
ShrugsPantssequence is 1000 times larger than the other two. Therefore, we scale it down with the following command:
mv data/ClothSeq/ShrugsPants/scans data/ClothSeq/ShrugsPants/scans_old
mkdir data/ClothSeq/ShrugsPants/scans
./tools/clothseq_clean.py data/ClothSeq/ShrugsPants/scans_old data/ClothSeq/ShrugsPants/scans
- Since the
.objfiles are slow to load, we transform them into.plyfiles by
python tools/obj2ply.py data/ClothSeq/JacketPants/scans data/ClothSeq/JacketPants/scans-ply
Train and validation (unseen poses)
python main.py --cfg config/cape-scan-subject-cloth_unif.py \
EXP.tag 00032_SS_SCAN_UNIF-20_APS-alpha2beta0_deltaSoftMin200 \
DATASET.kwargs.subject_name 00032 \
DATASET.kwargs.cloth_type shortshort \To test on extrapolated poses for metrics and partial visual results, add the arguments
...
EXP.test_only True \
EXP.checkpoint <ckpt-path> \To test on interpolated poses, further add the argument
...
DATASET.kwargs.test_interpolation True \If you only need the visual results (without metrics and losses), then you can save computation by adding the argument
...
EXP.TEST.external_query True \To save all results in each batch, add the argument
...
EXP.TEST.save_all_results True \python main.py --cfg config/clothseq_frames_unif.py \
EXP.tag JacketShorts_SCAN_UNIF-20_APS-alpha2beta0_deltaSoftMin200 \
DATASET.kwargs.clip_name JacketShorts \- ImportError: ('Unable to load EGL library', 'EGL: cannot open shared object file: No such file or directory', 'EGL', None)
sudo apt install libosmesa6-dev freeglut3-dev@inproceedings{qian2022unif,
title={UNIF: United Neural Implicit Functions for Clothed Human Reconstruction and Animation},
author={Qian, Shenhan and Xu, Jiale and Liu, Ziwei and Ma, Liqian and Gao, Shenghua},
booktitle={European Conference on Computer Vision},
pages={121--137},
year={2022},
organization={Springer}
}