🚀 Track with landmark 🚀
🚅 Track with synthesis 🚅
🧐 Landmarks (2D/3D) and bounding box 🧐
Lightning Track is a monocular face tracker built on FLAME. It provides optimized FLAME parameters and camera parameters, along with the bounding box and landmarks used during optimization.
Our tracker operates at a remarkable speed 🚀, processing 250 frames in approximately 60 seconds under landmark mode and 250 frames in around 200 seconds under synthesis mode.
Install step by step
conda create -n track python=3.9
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.8 -c pytorch -c nvidia
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install pytorch3d -c pytorch3d
pip3 install mediapipe tqdm rich lmdb einops colored ninja av opencv-python scikit-image onnxruntime-gpu onnx transformers pykalman
Install with environment.yml (recommend)
conda env create -f environment.yml
Run with Dockerfile
If your environment has unknown or unsolvable issues, use the Dockerfile as a final solution.
Check the build_resources.sh.
Track on video:
python track_video.py -v demos/demo.mp4 --synthesis
or track all videos in a directory:
python track_video.py -v demos/ --no_vis
If you find our work useful in your research, please consider citing:
@inproceedings{
chu2024gpavatar,
title={{GPA}vatar: Generalizable and Precise Head Avatar from Image(s)},
author={Xuangeng Chu and Yu Li and Ailing Zeng and Tianyu Yang and Lijian Lin and Yunfei Liu and Tatsuya Harada},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=hgehGq2bDv}
}